2024-04-25T13:55:48.457 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T13:55:48.457 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --owner scheduled_pdonnell@teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/direct_io}} --archive /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673194 --name pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi -- /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673194/orig.config.yaml 2024-04-25T13:55:48.512 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T13:55:48.626 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673194 branch: wip-pdonnell-testing-20240425.015853-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/direct_io}} email: pdonnell@redhat.com first_in_suite: false job_id: '7673194' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240425.015853-debug ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt dry_run: true install: ceph: flavor: default sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240425.015853-debug sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 owner: scheduled_pdonnell@teuthology priority: 85 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1880 sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 sleep_before_teardown: 0 subset: 178/512 suite: fs suite_branch: wip-pdonnell-testing-20240425.015853-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi096.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD6S7z6EanfkxkSTpnMn5Q/496IEfbEL104/VUM67VMDt552ezR+t0RtEadg0Ahce1FWAuy0twRngBv71F1TKQk= smithi097.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPw+3Jorn818C8ZBhPS//v78ycGPcx4a19WLIEn5ubuQikag3xeEv4bGqM5vn9Bne1z9lkEBXRGc9CXFP3h3obo= smithi151.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBs5Q9hPMAWJwYg9S2voVyQYL6BP8H6dciWUY6blehDRI0YhjkAgl2qOx8dv8LrhYNDx2ILo5yVH3VnDf/edXVY= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - direct_io teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_10:00:23 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T13:55:48.626 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa; will attempt to use it 2024-04-25T13:55:48.627 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks 2024-04-25T13:55:48.627 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T13:55:48.629 INFO:teuthology.task.internal:Checking packages... 2024-04-25T13:55:48.650 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '994e2f20102a20fa9a27966c14b625cc01884df8' 2024-04-25T13:55:48.650 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T13:55:48.650 INFO:teuthology.packaging:ref: None 2024-04-25T13:55:48.650 INFO:teuthology.packaging:tag: None 2024-04-25T13:55:48.650 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240425.015853-debug 2024-04-25T13:55:48.650 INFO:teuthology.packaging:sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T13:55:48.650 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240425.015853-debug 2024-04-25T13:55:48.873 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3291.g994e2f20 2024-04-25T13:55:48.874 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T13:55:48.881 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T13:55:48.881 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T13:55:48.930 INFO:teuthology.task.internal:Saving configuration 2024-04-25T13:55:48.952 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T13:55:48.963 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T13:55:48.989 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi096.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673194', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 13:50:30.738040', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD6S7z6EanfkxkSTpnMn5Q/496IEfbEL104/VUM67VMDt552ezR+t0RtEadg0Ahce1FWAuy0twRngBv71F1TKQk='} 2024-04-25T13:55:49.010 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi097.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673194', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 13:50:30.739609', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPw+3Jorn818C8ZBhPS//v78ycGPcx4a19WLIEn5ubuQikag3xeEv4bGqM5vn9Bne1z9lkEBXRGc9CXFP3h3obo='} 2024-04-25T13:55:49.031 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi151.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673194', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 13:50:30.741129', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBs5Q9hPMAWJwYg9S2voVyQYL6BP8H6dciWUY6blehDRI0YhjkAgl2qOx8dv8LrhYNDx2ILo5yVH3VnDf/edXVY='} 2024-04-25T13:55:49.031 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T13:55:49.038 INFO:teuthology.task.internal:roles: ubuntu@smithi096.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-25T13:55:49.038 INFO:teuthology.task.internal:roles: ubuntu@smithi097.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-25T13:55:49.038 INFO:teuthology.task.internal:roles: ubuntu@smithi151.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-25T13:55:49.038 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T13:55:49.170 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa8d1c680d0>, signals=[15]) 2024-04-25T13:55:49.171 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T13:55:49.177 INFO:teuthology.task.internal:Opening connections... 2024-04-25T13:55:49.177 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi096.front.sepia.ceph.com 2024-04-25T13:55:49.179 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:55:49.261 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi097.front.sepia.ceph.com 2024-04-25T13:55:49.262 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:55:49.338 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi151.front.sepia.ceph.com 2024-04-25T13:55:49.339 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:55:49.451 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T13:55:49.458 DEBUG:teuthology.orchestra.run.smithi096:> uname -m 2024-04-25T13:55:49.476 INFO:teuthology.orchestra.run.smithi096.stdout:x86_64 2024-04-25T13:55:49.477 DEBUG:teuthology.orchestra.run.smithi096:> cat /etc/os-release 2024-04-25T13:55:49.532 INFO:teuthology.orchestra.run.smithi096.stdout:NAME="CentOS Stream" 2024-04-25T13:55:49.533 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION="9" 2024-04-25T13:55:49.533 INFO:teuthology.orchestra.run.smithi096.stdout:ID="centos" 2024-04-25T13:55:49.533 INFO:teuthology.orchestra.run.smithi096.stdout:ID_LIKE="rhel fedora" 2024-04-25T13:55:49.533 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION_ID="9" 2024-04-25T13:55:49.533 INFO:teuthology.orchestra.run.smithi096.stdout:PLATFORM_ID="platform:el9" 2024-04-25T13:55:49.533 INFO:teuthology.orchestra.run.smithi096.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T13:55:49.533 INFO:teuthology.orchestra.run.smithi096.stdout:ANSI_COLOR="0;31" 2024-04-25T13:55:49.533 INFO:teuthology.orchestra.run.smithi096.stdout:LOGO="fedora-logo-icon" 2024-04-25T13:55:49.533 INFO:teuthology.orchestra.run.smithi096.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T13:55:49.534 INFO:teuthology.orchestra.run.smithi096.stdout:HOME_URL="https://centos.org/" 2024-04-25T13:55:49.534 INFO:teuthology.orchestra.run.smithi096.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T13:55:49.534 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T13:55:49.534 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T13:55:49.534 INFO:teuthology.lock.ops:Updating smithi096.front.sepia.ceph.com on lock server 2024-04-25T13:55:49.563 DEBUG:teuthology.orchestra.run.smithi097:> uname -m 2024-04-25T13:55:49.580 INFO:teuthology.orchestra.run.smithi097.stdout:x86_64 2024-04-25T13:55:49.581 DEBUG:teuthology.orchestra.run.smithi097:> cat /etc/os-release 2024-04-25T13:55:49.638 INFO:teuthology.orchestra.run.smithi097.stdout:NAME="CentOS Stream" 2024-04-25T13:55:49.639 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION="9" 2024-04-25T13:55:49.639 INFO:teuthology.orchestra.run.smithi097.stdout:ID="centos" 2024-04-25T13:55:49.639 INFO:teuthology.orchestra.run.smithi097.stdout:ID_LIKE="rhel fedora" 2024-04-25T13:55:49.639 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION_ID="9" 2024-04-25T13:55:49.639 INFO:teuthology.orchestra.run.smithi097.stdout:PLATFORM_ID="platform:el9" 2024-04-25T13:55:49.639 INFO:teuthology.orchestra.run.smithi097.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T13:55:49.639 INFO:teuthology.orchestra.run.smithi097.stdout:ANSI_COLOR="0;31" 2024-04-25T13:55:49.639 INFO:teuthology.orchestra.run.smithi097.stdout:LOGO="fedora-logo-icon" 2024-04-25T13:55:49.639 INFO:teuthology.orchestra.run.smithi097.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T13:55:49.639 INFO:teuthology.orchestra.run.smithi097.stdout:HOME_URL="https://centos.org/" 2024-04-25T13:55:49.639 INFO:teuthology.orchestra.run.smithi097.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T13:55:49.640 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T13:55:49.640 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T13:55:49.640 INFO:teuthology.lock.ops:Updating smithi097.front.sepia.ceph.com on lock server 2024-04-25T13:55:49.662 DEBUG:teuthology.orchestra.run.smithi151:> uname -m 2024-04-25T13:55:49.680 INFO:teuthology.orchestra.run.smithi151.stdout:x86_64 2024-04-25T13:55:49.680 DEBUG:teuthology.orchestra.run.smithi151:> cat /etc/os-release 2024-04-25T13:55:49.737 INFO:teuthology.orchestra.run.smithi151.stdout:NAME="CentOS Stream" 2024-04-25T13:55:49.737 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION="9" 2024-04-25T13:55:49.737 INFO:teuthology.orchestra.run.smithi151.stdout:ID="centos" 2024-04-25T13:55:49.737 INFO:teuthology.orchestra.run.smithi151.stdout:ID_LIKE="rhel fedora" 2024-04-25T13:55:49.738 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION_ID="9" 2024-04-25T13:55:49.738 INFO:teuthology.orchestra.run.smithi151.stdout:PLATFORM_ID="platform:el9" 2024-04-25T13:55:49.738 INFO:teuthology.orchestra.run.smithi151.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T13:55:49.738 INFO:teuthology.orchestra.run.smithi151.stdout:ANSI_COLOR="0;31" 2024-04-25T13:55:49.738 INFO:teuthology.orchestra.run.smithi151.stdout:LOGO="fedora-logo-icon" 2024-04-25T13:55:49.738 INFO:teuthology.orchestra.run.smithi151.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T13:55:49.738 INFO:teuthology.orchestra.run.smithi151.stdout:HOME_URL="https://centos.org/" 2024-04-25T13:55:49.738 INFO:teuthology.orchestra.run.smithi151.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T13:55:49.738 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T13:55:49.738 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T13:55:49.738 INFO:teuthology.lock.ops:Updating smithi151.front.sepia.ceph.com on lock server 2024-04-25T13:55:49.771 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T13:55:49.783 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T13:55:49.805 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T13:55:49.805 DEBUG:teuthology.orchestra.run.smithi096:> test '!' -e /home/ubuntu/cephtest 2024-04-25T13:55:49.808 DEBUG:teuthology.orchestra.run.smithi097:> test '!' -e /home/ubuntu/cephtest 2024-04-25T13:55:49.811 DEBUG:teuthology.orchestra.run.smithi151:> test '!' -e /home/ubuntu/cephtest 2024-04-25T13:55:49.827 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T13:55:49.833 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T13:55:49.833 DEBUG:teuthology.orchestra.run.smithi096:> test -z $(ls -A /var/lib/ceph) 2024-04-25T13:55:49.865 DEBUG:teuthology.orchestra.run.smithi097:> test -z $(ls -A /var/lib/ceph) 2024-04-25T13:55:49.868 DEBUG:teuthology.orchestra.run.smithi151:> test -z $(ls -A /var/lib/ceph) 2024-04-25T13:55:49.972 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T13:55:50.045 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T13:55:50.059 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-04-25T13:55:50.059 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-04-25T13:55:50.059 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-04-25T13:55:50.060 DEBUG:teuthology.orchestra.run.smithi096:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T13:55:50.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:55:50.076 DEBUG:teuthology.orchestra.run.smithi096:> uname -r 2024-04-25T13:55:50.133 INFO:teuthology.orchestra.run.smithi096.stdout:5.14.0-437.el9.x86_64 2024-04-25T13:55:50.134 INFO:teuthology.task.kernel:Running kernel on smithi096: 5.14.0-437.el9.x86_64 2024-04-25T13:55:50.134 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum install -y kernel 2024-04-25T13:55:52.503 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-04-25T13:55:54.296 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-04-25T13:55:58.430 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-04-25T13:56:01.069 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - Extras packages 11 kB/s | 16 kB 00:01 2024-04-25T13:56:02.354 INFO:teuthology.orchestra.run.smithi096.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-04-25T13:56:07.287 INFO:teuthology.orchestra.run.smithi096.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-25T13:56:09.126 INFO:teuthology.orchestra.run.smithi096.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T13:56:09.126 INFO:teuthology.orchestra.run.smithi096.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T13:56:09.227 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-04-25T13:56:09.228 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2024-04-25T13:56:09.228 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-04-25T13:56:09.358 DEBUG:teuthology.orchestra.run.smithi096:> echo no | sudo yum reinstall kernel || true 2024-04-25T13:56:09.856 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:00:02 ago on Thu 25 Apr 2024 01:56:07 PM UTC. 2024-04-25T13:56:09.987 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-04-25T13:56:09.999 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-25T13:56:09.999 INFO:teuthology.orchestra.run.smithi096.stdout: Package Architecture Version Repository Size 2024-04-25T13:56:09.999 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-25T13:56:09.999 INFO:teuthology.orchestra.run.smithi096.stdout:Reinstalling: 2024-04-25T13:56:09.999 INFO:teuthology.orchestra.run.smithi096.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T13:56:09.999 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T13:56:09.999 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-04-25T13:56:09.999 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-25T13:56:09.999 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T13:56:09.999 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 255 k 2024-04-25T13:56:09.999 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 0 2024-04-25T13:56:10.000 INFO:teuthology.orchestra.run.smithi096.stderr:Operation aborted. 2024-04-25T13:56:10.061 INFO:teuthology.orchestra.run.smithi096.stdout:Is this ok [y/N]: 2024-04-25T13:56:10.061 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum reinstall -y kernel || true 2024-04-25T13:56:10.555 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 01:56:07 PM UTC. 2024-04-25T13:56:10.672 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-04-25T13:56:10.672 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-25T13:56:10.673 INFO:teuthology.orchestra.run.smithi096.stdout: Package Architecture Version Repository Size 2024-04-25T13:56:10.673 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-25T13:56:10.673 INFO:teuthology.orchestra.run.smithi096.stdout:Reinstalling: 2024-04-25T13:56:10.673 INFO:teuthology.orchestra.run.smithi096.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T13:56:10.673 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T13:56:10.673 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-04-25T13:56:10.673 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-25T13:56:10.673 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T13:56:10.673 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 255 k 2024-04-25T13:56:10.673 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 0 2024-04-25T13:56:10.673 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-04-25T13:56:11.196 INFO:teuthology.orchestra.run.smithi096.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.0 MB/s | 255 kB 00:00 2024-04-25T13:56:11.197 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-04-25T13:56:11.197 INFO:teuthology.orchestra.run.smithi096.stdout:Total 486 kB/s | 255 kB 00:00 2024-04-25T13:56:11.219 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-04-25T13:56:11.389 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-04-25T13:56:11.390 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-04-25T13:56:11.394 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-04-25T13:56:11.395 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-04-25T13:56:11.545 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-04-25T13:56:11.614 INFO:teuthology.orchestra.run.smithi096.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T13:56:11.614 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T13:56:12.251 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T13:56:12.251 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T13:56:12.942 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T13:56:12.942 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T13:56:12.942 INFO:teuthology.orchestra.run.smithi096.stdout:Reinstalled: 2024-04-25T13:56:12.942 INFO:teuthology.orchestra.run.smithi096.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T13:56:12.943 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T13:56:12.943 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-04-25T13:56:13.099 DEBUG:teuthology.orchestra.run.smithi096:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T13:56:13.145 INFO:teuthology.orchestra.run.smithi096.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T13:56:13.145 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T13:56:13.146 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T13:56:13.146 DEBUG:teuthology.parallel:result is None 2024-04-25T13:56:13.146 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T13:56:13.154 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T13:56:13.154 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T13:56:13.156 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T13:56:13.159 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T13:56:13.177 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T13:56:13.201 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T13:56:13.252 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T13:56:13.252 DEBUG:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T13:56:13.254 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T13:56:13.257 DEBUG:teuthology.orchestra.run.smithi151:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T13:56:13.354 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T13:56:13.361 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T13:56:13.361 DEBUG:teuthology.orchestra.run.smithi097:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T13:56:13.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:56:13.378 DEBUG:teuthology.orchestra.run.smithi151:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T13:56:13.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:56:13.394 DEBUG:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T13:56:13.397 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T13:56:13.420 DEBUG:teuthology.orchestra.run.smithi151:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T13:56:13.442 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T13:56:13.460 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T13:56:13.464 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T13:56:13.480 INFO:teuthology.orchestra.run.smithi151.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T13:56:13.486 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T13:56:13.514 INFO:teuthology.orchestra.run.smithi151.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T13:56:13.516 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T13:56:13.523 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T13:56:13.523 DEBUG:teuthology.orchestra.run.smithi096:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T13:56:13.525 DEBUG:teuthology.orchestra.run.smithi097:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T13:56:13.528 DEBUG:teuthology.orchestra.run.smithi151:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T13:56:13.554 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T13:56:13.562 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T13:56:13.563 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T13:56:13.589 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T13:56:13.594 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T13:56:13.610 DEBUG:teuthology.orchestra.run.smithi096:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T13:56:13.683 DEBUG:teuthology.orchestra.run.smithi096:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T13:56:13.737 DEBUG:teuthology.orchestra.run.smithi096:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T13:56:13.818 DEBUG:teuthology.orchestra.run.smithi096:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T13:56:13.856 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T13:56:13.856 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T13:56:13.919 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T13:56:13.963 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T13:56:14.012 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T13:56:14.095 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T13:56:14.135 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T13:56:14.135 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T13:56:14.199 DEBUG:teuthology.orchestra.run.smithi151:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T13:56:14.246 DEBUG:teuthology.orchestra.run.smithi151:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T13:56:14.314 DEBUG:teuthology.orchestra.run.smithi151:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T13:56:14.358 DEBUG:teuthology.orchestra.run.smithi151:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T13:56:14.400 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-25T13:56:14.400 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T13:56:14.466 DEBUG:teuthology.orchestra.run.smithi096:> sudo service rsyslog restart 2024-04-25T13:56:14.468 DEBUG:teuthology.orchestra.run.smithi097:> sudo service rsyslog restart 2024-04-25T13:56:14.470 DEBUG:teuthology.orchestra.run.smithi151:> sudo service rsyslog restart 2024-04-25T13:56:14.498 INFO:teuthology.orchestra.run.smithi096.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T13:56:14.505 INFO:teuthology.orchestra.run.smithi097.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T13:56:14.544 INFO:teuthology.orchestra.run.smithi151.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T13:56:14.958 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T13:56:14.967 INFO:teuthology.task.internal:Starting timer... 2024-04-25T13:56:14.967 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T13:56:15.014 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T13:56:15.062 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-25T13:56:15.063 DEBUG:teuthology.orchestra.run.smithi096:> sudo service auditd rotate 2024-04-25T13:56:15.113 INFO:teuthology.orchestra.run.smithi096.stdout:Rotating logs: 2024-04-25T13:56:15.114 DEBUG:teuthology.orchestra.run.smithi097:> sudo service auditd rotate 2024-04-25T13:56:15.179 INFO:teuthology.orchestra.run.smithi097.stdout:Rotating logs: 2024-04-25T13:56:15.181 DEBUG:teuthology.orchestra.run.smithi151:> sudo service auditd rotate 2024-04-25T13:56:15.242 INFO:teuthology.orchestra.run.smithi151.stdout:Rotating logs: 2024-04-25T13:56:15.244 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T13:56:15.244 DEBUG:teuthology.orchestra.run.smithi096:> /usr/sbin/getenforce 2024-04-25T13:56:15.280 INFO:teuthology.orchestra.run.smithi096.stdout:Permissive 2024-04-25T13:56:15.280 DEBUG:teuthology.orchestra.run.smithi097:> /usr/sbin/getenforce 2024-04-25T13:56:15.304 INFO:teuthology.orchestra.run.smithi097.stdout:Permissive 2024-04-25T13:56:15.304 DEBUG:teuthology.orchestra.run.smithi151:> /usr/sbin/getenforce 2024-04-25T13:56:15.323 INFO:teuthology.orchestra.run.smithi151.stdout:Permissive 2024-04-25T13:56:15.324 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi096.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi097.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi151.front.sepia.ceph.com': 'permissive'} 2024-04-25T13:56:15.324 DEBUG:teuthology.orchestra.run.smithi096:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T13:56:15.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:56:15.348 DEBUG:teuthology.orchestra.run.smithi097:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T13:56:15.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:56:15.375 DEBUG:teuthology.orchestra.run.smithi151:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T13:56:15.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:56:15.400 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T13:56:15.400 DEBUG:teuthology.orchestra.run.smithi096:> sudo /usr/sbin/setenforce permissive 2024-04-25T13:56:15.432 DEBUG:teuthology.orchestra.run.smithi097:> sudo /usr/sbin/setenforce permissive 2024-04-25T13:56:15.459 DEBUG:teuthology.orchestra.run.smithi151:> sudo /usr/sbin/setenforce permissive 2024-04-25T13:56:15.483 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T13:56:15.493 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T13:56:15.509 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T13:56:15.510 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi096.front.sepia.ceph.com,smithi097.front.sepia.ceph.com,smithi151.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T14:00:02.303 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi096.front.sepia.ceph.com'), Remote(name='ubuntu@smithi097.front.sepia.ceph.com'), Remote(name='ubuntu@smithi151.front.sepia.ceph.com')] 2024-04-25T14:00:02.304 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-04-25T14:00:02.305 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T14:00:02.379 DEBUG:teuthology.orchestra.run.smithi096:> true 2024-04-25T14:00:02.457 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-04-25T14:00:02.458 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-04-25T14:00:02.458 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T14:00:02.532 DEBUG:teuthology.orchestra.run.smithi097:> true 2024-04-25T14:00:02.622 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-04-25T14:00:02.623 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-04-25T14:00:02.623 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T14:00:02.698 DEBUG:teuthology.orchestra.run.smithi151:> true 2024-04-25T14:00:02.790 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-04-25T14:00:02.791 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T14:00:02.801 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T14:00:02.801 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T14:00:02.801 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T14:00:02.804 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T14:00:02.804 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T14:00:02.806 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T14:00:02.806 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T14:00:02.831 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T14:00:02.834 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T14:00:02.846 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T14:00:02.848 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T14:00:02.862 INFO:teuthology.orchestra.run.smithi151.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T14:00:02.869 INFO:teuthology.orchestra.run.smithi096.stderr:sudo: ntpd: command not found 2024-04-25T14:00:02.872 INFO:teuthology.orchestra.run.smithi097.stderr:sudo: ntpd: command not found 2024-04-25T14:00:02.875 INFO:teuthology.orchestra.run.smithi151.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T14:00:02.880 INFO:teuthology.orchestra.run.smithi096.stdout:506 Cannot talk to daemon 2024-04-25T14:00:02.883 INFO:teuthology.orchestra.run.smithi097.stdout:506 Cannot talk to daemon 2024-04-25T14:00:02.894 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T14:00:02.896 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T14:00:02.898 INFO:teuthology.orchestra.run.smithi151.stderr:sudo: ntpd: command not found 2024-04-25T14:00:02.907 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T14:00:02.909 INFO:teuthology.orchestra.run.smithi151.stdout:506 Cannot talk to daemon 2024-04-25T14:00:02.910 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T14:00:02.922 INFO:teuthology.orchestra.run.smithi151.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T14:00:02.935 INFO:teuthology.orchestra.run.smithi151.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T14:00:02.960 INFO:teuthology.orchestra.run.smithi096.stderr:bash: line 1: ntpq: command not found 2024-04-25T14:00:02.964 INFO:teuthology.orchestra.run.smithi096.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T14:00:02.964 INFO:teuthology.orchestra.run.smithi096.stdout:=============================================================================== 2024-04-25T14:00:02.964 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:00:02.964 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:00:02.964 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:00:02.964 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:00:02.973 INFO:teuthology.orchestra.run.smithi097.stderr:bash: line 1: ntpq: command not found 2024-04-25T14:00:02.977 INFO:teuthology.orchestra.run.smithi097.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T14:00:02.977 INFO:teuthology.orchestra.run.smithi097.stdout:=============================================================================== 2024-04-25T14:00:02.977 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:00:02.978 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:00:02.978 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:00:02.978 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:00:02.992 INFO:teuthology.orchestra.run.smithi151.stderr:bash: line 1: ntpq: command not found 2024-04-25T14:00:02.996 INFO:teuthology.orchestra.run.smithi151.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T14:00:02.996 INFO:teuthology.orchestra.run.smithi151.stdout:=============================================================================== 2024-04-25T14:00:02.996 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:00:02.996 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:00:02.996 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:00:02.997 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:00:02.997 INFO:teuthology.run_tasks:Running task install... 2024-04-25T14:00:03.006 DEBUG:teuthology.task.install:project ceph 2024-04-25T14:00:03.006 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}} 2024-04-25T14:00:03.006 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'} 2024-04-25T14:00:03.006 INFO:teuthology.task.install:Using flavor: default 2024-04-25T14:00:03.011 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T14:00:03.011 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T14:00:03.012 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'tag': None, 'wait_for_package': False} 2024-04-25T14:00:03.012 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T14:00:03.014 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'tag': None, 'wait_for_package': False} 2024-04-25T14:00:03.014 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T14:00:03.015 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'tag': None, 'wait_for_package': False} 2024-04-25T14:00:03.016 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T14:00:03.161 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/ 2024-04-25T14:00:03.162 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3291.g994e2f20 2024-04-25T14:00:03.266 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/ 2024-04-25T14:00:03.267 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3291.g994e2f20 2024-04-25T14:00:03.287 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/ 2024-04-25T14:00:03.287 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3291.g994e2f20 2024-04-25T14:00:03.363 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T14:00:03.363 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T14:00:03.363 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T14:00:03.389 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T14:00:03.389 DEBUG:teuthology.orchestra.run.smithi096:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/994e2f20102a20fa9a27966c14b625cc01884df8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T14:00:03.422 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T14:00:03.422 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-25T14:00:03.422 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T14:00:03.447 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T14:00:03.447 DEBUG:teuthology.orchestra.run.smithi151:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/994e2f20102a20fa9a27966c14b625cc01884df8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T14:00:03.453 DEBUG:teuthology.orchestra.run.smithi096:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T14:00:03.504 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T14:00:03.504 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T14:00:03.504 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T14:00:03.511 DEBUG:teuthology.orchestra.run.smithi151:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T14:00:03.528 DEBUG:teuthology.orchestra.run.smithi096:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T14:00:03.529 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T14:00:03.529 DEBUG:teuthology.orchestra.run.smithi097:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/994e2f20102a20fa9a27966c14b625cc01884df8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T14:00:03.583 DEBUG:teuthology.orchestra.run.smithi151:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T14:00:03.591 INFO:teuthology.orchestra.run.smithi096.stdout:check_obsoletes = 1 2024-04-25T14:00:03.593 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum clean all 2024-04-25T14:00:03.595 DEBUG:teuthology.orchestra.run.smithi097:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T14:00:03.647 INFO:teuthology.orchestra.run.smithi151.stdout:check_obsoletes = 1 2024-04-25T14:00:03.648 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum clean all 2024-04-25T14:00:03.669 DEBUG:teuthology.orchestra.run.smithi097:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T14:00:03.733 INFO:teuthology.orchestra.run.smithi097.stdout:check_obsoletes = 1 2024-04-25T14:00:03.734 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum clean all 2024-04-25T14:00:03.816 INFO:teuthology.orchestra.run.smithi096.stdout:48 files removed 2024-04-25T14:00:03.842 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T14:00:03.873 INFO:teuthology.orchestra.run.smithi151.stdout:48 files removed 2024-04-25T14:00:03.900 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T14:00:03.959 INFO:teuthology.orchestra.run.smithi097.stdout:48 files removed 2024-04-25T14:00:03.986 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T14:00:04.422 INFO:teuthology.orchestra.run.smithi096.stdout:ceph packages for x86_64 259 kB/s | 84 kB 00:00 2024-04-25T14:00:04.568 INFO:teuthology.orchestra.run.smithi097.stdout:ceph packages for x86_64 260 kB/s | 84 kB 00:00 2024-04-25T14:00:04.666 INFO:teuthology.orchestra.run.smithi151.stdout:ceph packages for x86_64 166 kB/s | 84 kB 00:00 2024-04-25T14:00:04.817 INFO:teuthology.orchestra.run.smithi096.stdout:ceph noarch packages 33 kB/s | 12 kB 00:00 2024-04-25T14:00:04.886 INFO:teuthology.orchestra.run.smithi097.stdout:ceph noarch packages 43 kB/s | 12 kB 00:00 2024-04-25T14:00:05.033 INFO:teuthology.orchestra.run.smithi151.stdout:ceph noarch packages 36 kB/s | 12 kB 00:00 2024-04-25T14:00:05.176 INFO:teuthology.orchestra.run.smithi096.stdout:ceph source packages 5.8 kB/s | 1.9 kB 00:00 2024-04-25T14:00:05.219 INFO:teuthology.orchestra.run.smithi097.stdout:ceph source packages 6.3 kB/s | 1.9 kB 00:00 2024-04-25T14:00:05.525 INFO:teuthology.orchestra.run.smithi151.stdout:ceph source packages 4.2 kB/s | 1.9 kB 00:00 2024-04-25T14:00:05.953 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-25T14:00:06.347 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - BaseOS 7.4 MB/s | 8.0 MB 00:01 2024-04-25T14:00:06.762 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - BaseOS 6.7 MB/s | 8.0 MB 00:01 2024-04-25T14:00:07.913 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-04-25T14:00:08.348 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-04-25T14:00:08.854 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - AppStream 19 MB/s | 19 MB 00:01 2024-04-25T14:00:11.980 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-04-25T14:00:12.382 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-04-25T14:00:13.080 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - CRB 9.9 MB/s | 6.2 MB 00:00 2024-04-25T14:00:13.729 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - Extras packages 26 kB/s | 16 kB 00:00 2024-04-25T14:00:14.006 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-04-25T14:00:14.703 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 16 kB 00:00 2024-04-25T14:00:14.865 INFO:teuthology.orchestra.run.smithi096.stdout:Extra Packages for Enterprise Linux 21 MB/s | 21 MB 00:01 2024-04-25T14:00:16.590 INFO:teuthology.orchestra.run.smithi097.stdout:Extra Packages for Enterprise Linux 8.5 MB/s | 21 MB 00:02 2024-04-25T14:00:17.355 INFO:teuthology.orchestra.run.smithi151.stdout:Extra Packages for Enterprise Linux 8.3 MB/s | 21 MB 00:02 2024-04-25T14:00:19.928 INFO:teuthology.orchestra.run.smithi096.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-25T14:00:21.690 INFO:teuthology.orchestra.run.smithi097.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-25T14:00:22.046 INFO:teuthology.orchestra.run.smithi096.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T14:00:22.047 INFO:teuthology.orchestra.run.smithi096.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T14:00:22.058 INFO:teuthology.orchestra.run.smithi096.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T14:00:22.059 INFO:teuthology.orchestra.run.smithi096.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T14:00:22.060 INFO:teuthology.orchestra.run.smithi096.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T14:00:22.060 INFO:teuthology.orchestra.run.smithi096.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T14:00:22.062 INFO:teuthology.orchestra.run.smithi096.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T14:00:22.064 INFO:teuthology.orchestra.run.smithi096.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T14:00:22.065 INFO:teuthology.orchestra.run.smithi096.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T14:00:22.127 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-04-25T14:00:22.136 INFO:teuthology.orchestra.run.smithi096.stdout:======================================================================================= 2024-04-25T14:00:22.136 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-04-25T14:00:22.136 INFO:teuthology.orchestra.run.smithi096.stdout:======================================================================================= 2024-04-25T14:00:22.136 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-04-25T14:00:22.136 INFO:teuthology.orchestra.run.smithi096.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T14:00:22.136 INFO:teuthology.orchestra.run.smithi096.stdout: ceph x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 6.6 k 2024-04-25T14:00:22.136 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-base x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.4 M 2024-04-25T14:00:22.137 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 863 k 2024-04-25T14:00:22.137 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 147 k 2024-04-25T14:00:22.137 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 1.5 M 2024-04-25T14:00:22.137 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 142 k 2024-04-25T14:00:22.137 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 3.5 M 2024-04-25T14:00:22.137 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 7.4 M 2024-04-25T14:00:22.137 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-rook noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 49 k 2024-04-25T14:00:22.137 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-radosgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 10 M 2024-04-25T14:00:22.137 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-test x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 43 M 2024-04-25T14:00:22.137 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-volume noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 273 k 2024-04-25T14:00:22.137 INFO:teuthology.orchestra.run.smithi096.stdout: cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 762 k 2024-04-25T14:00:22.137 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 224 k 2024-04-25T14:00:22.137 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-shell x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 27 k 2024-04-25T14:00:22.138 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-top noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 18 k 2024-04-25T14:00:22.138 INFO:teuthology.orchestra.run.smithi096.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T14:00:22.138 INFO:teuthology.orchestra.run.smithi096.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T14:00:22.138 INFO:teuthology.orchestra.run.smithi096.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T14:00:22.138 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 32 k 2024-04-25T14:00:22.138 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 756 k 2024-04-25T14:00:22.138 INFO:teuthology.orchestra.run.smithi096.stdout: librados-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 127 k 2024-04-25T14:00:22.138 INFO:teuthology.orchestra.run.smithi096.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T14:00:22.138 INFO:teuthology.orchestra.run.smithi096.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T14:00:22.138 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T14:00:22.138 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T14:00:22.138 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T14:00:22.139 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T14:00:22.139 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cephfs x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 161 k 2024-04-25T14:00:22.139 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rados x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 321 k 2024-04-25T14:00:22.139 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 298 k 2024-04-25T14:00:22.139 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 100 k 2024-04-25T14:00:22.139 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 86 k 2024-04-25T14:00:22.139 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.0 M 2024-04-25T14:00:22.139 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-nbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 169 k 2024-04-25T14:00:22.139 INFO:teuthology.orchestra.run.smithi096.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T14:00:22.139 INFO:teuthology.orchestra.run.smithi096.stdout:Upgrading: 2024-04-25T14:00:22.139 INFO:teuthology.orchestra.run.smithi096.stdout: librados2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.3 M 2024-04-25T14:00:22.139 INFO:teuthology.orchestra.run.smithi096.stdout: librbd1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.1 M 2024-04-25T14:00:22.139 INFO:teuthology.orchestra.run.smithi096.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T14:00:22.140 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T14:00:22.140 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-04-25T14:00:22.140 INFO:teuthology.orchestra.run.smithi096.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T14:00:22.140 INFO:teuthology.orchestra.run.smithi096.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T14:00:22.140 INFO:teuthology.orchestra.run.smithi096.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T14:00:22.140 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 20 M 2024-04-25T14:00:22.140 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 34 k 2024-04-25T14:00:22.140 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mds x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 2.3 M 2024-04-25T14:00:22.140 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 249 k 2024-04-25T14:00:22.140 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mon x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 4.9 M 2024-04-25T14:00:22.140 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-osd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 18 M 2024-04-25T14:00:22.140 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 16 k 2024-04-25T14:00:22.141 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-selinux x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 25 k 2024-04-25T14:00:22.141 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T14:00:22.141 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T14:00:22.141 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T14:00:22.141 INFO:teuthology.orchestra.run.smithi096.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T14:00:22.141 INFO:teuthology.orchestra.run.smithi096.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T14:00:22.141 INFO:teuthology.orchestra.run.smithi096.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T14:00:22.141 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T14:00:22.141 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T14:00:22.141 INFO:teuthology.orchestra.run.smithi096.stdout: libcephsqlite x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 158 k 2024-04-25T14:00:22.141 INFO:teuthology.orchestra.run.smithi096.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T14:00:22.141 INFO:teuthology.orchestra.run.smithi096.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T14:00:22.141 INFO:teuthology.orchestra.run.smithi096.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T14:00:22.141 INFO:teuthology.orchestra.run.smithi096.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T14:00:22.142 INFO:teuthology.orchestra.run.smithi096.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T14:00:22.142 INFO:teuthology.orchestra.run.smithi096.stdout: libradosstriper1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 497 k 2024-04-25T14:00:22.142 INFO:teuthology.orchestra.run.smithi096.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T14:00:22.142 INFO:teuthology.orchestra.run.smithi096.stdout: librgw2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.2 M 2024-04-25T14:00:22.142 INFO:teuthology.orchestra.run.smithi096.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T14:00:22.142 INFO:teuthology.orchestra.run.smithi096.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T14:00:22.142 INFO:teuthology.orchestra.run.smithi096.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T14:00:22.142 INFO:teuthology.orchestra.run.smithi096.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T14:00:22.142 INFO:teuthology.orchestra.run.smithi096.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T14:00:22.142 INFO:teuthology.orchestra.run.smithi096.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T14:00:22.142 INFO:teuthology.orchestra.run.smithi096.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T14:00:22.142 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T14:00:22.142 INFO:teuthology.orchestra.run.smithi096.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T14:00:22.142 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T14:00:22.143 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T14:00:22.143 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T14:00:22.143 INFO:teuthology.orchestra.run.smithi096.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T14:00:22.143 INFO:teuthology.orchestra.run.smithi096.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T14:00:22.143 INFO:teuthology.orchestra.run.smithi096.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T14:00:22.143 INFO:teuthology.orchestra.run.smithi096.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T14:00:22.143 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T14:00:22.143 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-argparse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 45 k 2024-04-25T14:00:22.143 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 135 k 2024-04-25T14:00:22.143 INFO:teuthology.orchestra.run.smithi096.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T14:00:22.143 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T14:00:22.143 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T14:00:22.143 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T14:00:22.144 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T14:00:22.144 INFO:teuthology.orchestra.run.smithi096.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T14:00:22.144 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T14:00:22.144 INFO:teuthology.orchestra.run.smithi096.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T14:00:22.144 INFO:teuthology.orchestra.run.smithi096.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T14:00:22.144 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T14:00:22.144 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T14:00:22.144 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T14:00:22.144 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T14:00:22.144 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T14:00:22.144 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T14:00:22.144 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T14:00:22.144 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T14:00:22.145 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T14:00:22.145 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T14:00:22.145 INFO:teuthology.orchestra.run.smithi096.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T14:00:22.145 INFO:teuthology.orchestra.run.smithi096.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T14:00:22.145 INFO:teuthology.orchestra.run.smithi096.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T14:00:22.145 INFO:teuthology.orchestra.run.smithi096.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T14:00:22.145 INFO:teuthology.orchestra.run.smithi096.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T14:00:22.145 INFO:teuthology.orchestra.run.smithi096.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T14:00:22.145 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T14:00:22.145 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T14:00:22.145 INFO:teuthology.orchestra.run.smithi096.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T14:00:22.145 INFO:teuthology.orchestra.run.smithi096.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T14:00:22.145 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T14:00:22.145 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T14:00:22.146 INFO:teuthology.orchestra.run.smithi096.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T14:00:22.146 INFO:teuthology.orchestra.run.smithi096.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T14:00:22.146 INFO:teuthology.orchestra.run.smithi096.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T14:00:22.146 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T14:00:22.146 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T14:00:22.146 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T14:00:22.146 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T14:00:22.146 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T14:00:22.146 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T14:00:22.146 INFO:teuthology.orchestra.run.smithi096.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T14:00:22.146 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T14:00:22.146 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T14:00:22.146 INFO:teuthology.orchestra.run.smithi096.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T14:00:22.146 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T14:00:22.147 INFO:teuthology.orchestra.run.smithi096.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T14:00:22.147 INFO:teuthology.orchestra.run.smithi096.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T14:00:22.147 INFO:teuthology.orchestra.run.smithi096.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T14:00:22.147 INFO:teuthology.orchestra.run.smithi096.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T14:00:22.147 INFO:teuthology.orchestra.run.smithi096.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T14:00:22.147 INFO:teuthology.orchestra.run.smithi096.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T14:00:22.147 INFO:teuthology.orchestra.run.smithi096.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T14:00:22.147 INFO:teuthology.orchestra.run.smithi096.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T14:00:22.147 INFO:teuthology.orchestra.run.smithi096.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T14:00:22.147 INFO:teuthology.orchestra.run.smithi096.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T14:00:22.147 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T14:00:22.147 INFO:teuthology.orchestra.run.smithi096.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T14:00:22.147 INFO:teuthology.orchestra.run.smithi096.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T14:00:22.148 INFO:teuthology.orchestra.run.smithi096.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T14:00:22.148 INFO:teuthology.orchestra.run.smithi096.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T14:00:22.148 INFO:teuthology.orchestra.run.smithi096.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T14:00:22.148 INFO:teuthology.orchestra.run.smithi096.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T14:00:22.148 INFO:teuthology.orchestra.run.smithi096.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T14:00:22.148 INFO:teuthology.orchestra.run.smithi096.stdout:Installing weak dependencies: 2024-04-25T14:00:22.148 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T14:00:22.148 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:00:22.148 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-04-25T14:00:22.148 INFO:teuthology.orchestra.run.smithi096.stdout:======================================================================================= 2024-04-25T14:00:22.148 INFO:teuthology.orchestra.run.smithi096.stdout:Install 148 Packages 2024-04-25T14:00:22.148 INFO:teuthology.orchestra.run.smithi096.stdout:Upgrade 4 Packages 2024-04-25T14:00:22.148 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:00:22.149 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 214 M 2024-04-25T14:00:22.149 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-04-25T14:00:22.588 INFO:teuthology.orchestra.run.smithi151.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-04-25T14:00:22.996 INFO:teuthology.orchestra.run.smithi096.stdout:(1/152): ceph-19.0.0-3291.g994e2f20.el9.x86_64. 28 kB/s | 6.6 kB 00:00 2024-04-25T14:00:23.388 INFO:teuthology.orchestra.run.smithi096.stdout:(2/152): ceph-fuse-19.0.0-3291.g994e2f20.el9.x8 2.2 MB/s | 863 kB 00:00 2024-04-25T14:00:23.497 INFO:teuthology.orchestra.run.smithi096.stdout:(3/152): ceph-immutable-object-cache-19.0.0-329 1.3 MB/s | 147 kB 00:00 2024-04-25T14:00:23.664 INFO:teuthology.orchestra.run.smithi096.stdout:(4/152): ceph-base-19.0.0-3291.g994e2f20.el9.x8 6.0 MB/s | 5.4 MB 00:00 2024-04-25T14:00:23.833 INFO:teuthology.orchestra.run.smithi097.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T14:00:23.833 INFO:teuthology.orchestra.run.smithi097.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T14:00:23.844 INFO:teuthology.orchestra.run.smithi097.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T14:00:23.845 INFO:teuthology.orchestra.run.smithi097.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T14:00:23.846 INFO:teuthology.orchestra.run.smithi097.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T14:00:23.847 INFO:teuthology.orchestra.run.smithi097.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T14:00:23.848 INFO:teuthology.orchestra.run.smithi097.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T14:00:23.851 INFO:teuthology.orchestra.run.smithi097.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T14:00:23.851 INFO:teuthology.orchestra.run.smithi097.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T14:00:23.864 INFO:teuthology.orchestra.run.smithi096.stdout:(5/152): ceph-mgr-19.0.0-3291.g994e2f20.el9.x86 7.5 MB/s | 1.5 MB 00:00 2024-04-25T14:00:23.913 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-04-25T14:00:23.941 INFO:teuthology.orchestra.run.smithi097.stdout:======================================================================================= 2024-04-25T14:00:23.941 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2024-04-25T14:00:23.941 INFO:teuthology.orchestra.run.smithi097.stdout:======================================================================================= 2024-04-25T14:00:23.941 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-04-25T14:00:23.941 INFO:teuthology.orchestra.run.smithi097.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T14:00:23.941 INFO:teuthology.orchestra.run.smithi097.stdout: ceph x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 6.6 k 2024-04-25T14:00:23.942 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.4 M 2024-04-25T14:00:23.942 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 863 k 2024-04-25T14:00:23.942 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 147 k 2024-04-25T14:00:23.942 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 1.5 M 2024-04-25T14:00:23.942 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 142 k 2024-04-25T14:00:23.942 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 3.5 M 2024-04-25T14:00:23.942 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 7.4 M 2024-04-25T14:00:23.942 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-rook noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 49 k 2024-04-25T14:00:23.942 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-radosgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 10 M 2024-04-25T14:00:23.942 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-test x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 43 M 2024-04-25T14:00:23.942 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-volume noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 273 k 2024-04-25T14:00:23.942 INFO:teuthology.orchestra.run.smithi097.stdout: cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 762 k 2024-04-25T14:00:23.942 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 224 k 2024-04-25T14:00:23.943 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-shell x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 27 k 2024-04-25T14:00:23.943 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-top noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 18 k 2024-04-25T14:00:23.943 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T14:00:23.943 INFO:teuthology.orchestra.run.smithi097.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T14:00:23.943 INFO:teuthology.orchestra.run.smithi097.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T14:00:23.943 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 32 k 2024-04-25T14:00:23.943 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 756 k 2024-04-25T14:00:23.943 INFO:teuthology.orchestra.run.smithi097.stdout: librados-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 127 k 2024-04-25T14:00:23.943 INFO:teuthology.orchestra.run.smithi097.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T14:00:23.943 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T14:00:23.943 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T14:00:23.943 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T14:00:23.943 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T14:00:23.944 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T14:00:23.944 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cephfs x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 161 k 2024-04-25T14:00:23.944 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rados x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 321 k 2024-04-25T14:00:23.944 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 298 k 2024-04-25T14:00:23.944 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 100 k 2024-04-25T14:00:23.944 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 86 k 2024-04-25T14:00:23.944 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.0 M 2024-04-25T14:00:23.944 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-nbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 169 k 2024-04-25T14:00:23.944 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T14:00:23.944 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrading: 2024-04-25T14:00:23.944 INFO:teuthology.orchestra.run.smithi097.stdout: librados2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.3 M 2024-04-25T14:00:23.944 INFO:teuthology.orchestra.run.smithi097.stdout: librbd1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.1 M 2024-04-25T14:00:23.944 INFO:teuthology.orchestra.run.smithi097.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T14:00:23.945 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T14:00:23.945 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2024-04-25T14:00:23.945 INFO:teuthology.orchestra.run.smithi097.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T14:00:23.945 INFO:teuthology.orchestra.run.smithi097.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T14:00:23.945 INFO:teuthology.orchestra.run.smithi097.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T14:00:23.945 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 20 M 2024-04-25T14:00:23.945 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 34 k 2024-04-25T14:00:23.945 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mds x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 2.3 M 2024-04-25T14:00:23.945 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 249 k 2024-04-25T14:00:23.945 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mon x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 4.9 M 2024-04-25T14:00:23.945 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-osd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 18 M 2024-04-25T14:00:23.945 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 16 k 2024-04-25T14:00:23.945 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-selinux x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 25 k 2024-04-25T14:00:23.945 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T14:00:23.946 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T14:00:23.946 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T14:00:23.946 INFO:teuthology.orchestra.run.smithi097.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T14:00:23.946 INFO:teuthology.orchestra.run.smithi097.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T14:00:23.946 INFO:teuthology.orchestra.run.smithi097.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T14:00:23.946 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T14:00:23.946 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T14:00:23.946 INFO:teuthology.orchestra.run.smithi097.stdout: libcephsqlite x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 158 k 2024-04-25T14:00:23.946 INFO:teuthology.orchestra.run.smithi097.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T14:00:23.946 INFO:teuthology.orchestra.run.smithi097.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T14:00:23.946 INFO:teuthology.orchestra.run.smithi097.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T14:00:23.946 INFO:teuthology.orchestra.run.smithi097.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T14:00:23.946 INFO:teuthology.orchestra.run.smithi097.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T14:00:23.946 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 497 k 2024-04-25T14:00:23.947 INFO:teuthology.orchestra.run.smithi097.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T14:00:23.947 INFO:teuthology.orchestra.run.smithi097.stdout: librgw2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.2 M 2024-04-25T14:00:23.947 INFO:teuthology.orchestra.run.smithi097.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T14:00:23.947 INFO:teuthology.orchestra.run.smithi097.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T14:00:23.947 INFO:teuthology.orchestra.run.smithi097.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T14:00:23.947 INFO:teuthology.orchestra.run.smithi097.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T14:00:23.947 INFO:teuthology.orchestra.run.smithi097.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T14:00:23.947 INFO:teuthology.orchestra.run.smithi097.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T14:00:23.947 INFO:teuthology.orchestra.run.smithi097.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T14:00:23.947 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T14:00:23.947 INFO:teuthology.orchestra.run.smithi097.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T14:00:23.947 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T14:00:23.947 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T14:00:23.947 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T14:00:23.948 INFO:teuthology.orchestra.run.smithi097.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T14:00:23.948 INFO:teuthology.orchestra.run.smithi097.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T14:00:23.948 INFO:teuthology.orchestra.run.smithi097.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T14:00:23.948 INFO:teuthology.orchestra.run.smithi097.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T14:00:23.948 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T14:00:23.948 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-argparse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 45 k 2024-04-25T14:00:23.948 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 135 k 2024-04-25T14:00:23.948 INFO:teuthology.orchestra.run.smithi097.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T14:00:23.948 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T14:00:23.948 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T14:00:23.948 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T14:00:23.948 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T14:00:23.948 INFO:teuthology.orchestra.run.smithi097.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T14:00:23.948 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T14:00:23.949 INFO:teuthology.orchestra.run.smithi097.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T14:00:23.949 INFO:teuthology.orchestra.run.smithi097.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T14:00:23.949 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T14:00:23.949 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T14:00:23.949 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T14:00:23.949 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T14:00:23.949 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T14:00:23.949 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T14:00:23.949 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T14:00:23.949 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T14:00:23.949 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T14:00:23.949 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T14:00:23.949 INFO:teuthology.orchestra.run.smithi097.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T14:00:23.949 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T14:00:23.950 INFO:teuthology.orchestra.run.smithi097.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T14:00:23.950 INFO:teuthology.orchestra.run.smithi097.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T14:00:23.950 INFO:teuthology.orchestra.run.smithi097.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T14:00:23.950 INFO:teuthology.orchestra.run.smithi097.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T14:00:23.950 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T14:00:23.950 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T14:00:23.950 INFO:teuthology.orchestra.run.smithi097.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T14:00:23.950 INFO:teuthology.orchestra.run.smithi097.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T14:00:23.950 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T14:00:23.950 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T14:00:23.950 INFO:teuthology.orchestra.run.smithi097.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T14:00:23.950 INFO:teuthology.orchestra.run.smithi097.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T14:00:23.950 INFO:teuthology.orchestra.run.smithi097.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T14:00:23.951 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T14:00:23.951 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T14:00:23.951 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T14:00:23.951 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T14:00:23.951 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T14:00:23.951 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T14:00:23.951 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T14:00:23.951 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T14:00:23.951 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T14:00:23.951 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T14:00:23.951 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T14:00:23.951 INFO:teuthology.orchestra.run.smithi097.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T14:00:23.951 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T14:00:23.951 INFO:teuthology.orchestra.run.smithi097.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T14:00:23.952 INFO:teuthology.orchestra.run.smithi097.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T14:00:23.952 INFO:teuthology.orchestra.run.smithi097.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T14:00:23.952 INFO:teuthology.orchestra.run.smithi097.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T14:00:23.952 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T14:00:23.952 INFO:teuthology.orchestra.run.smithi097.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T14:00:23.952 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T14:00:23.952 INFO:teuthology.orchestra.run.smithi097.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T14:00:23.952 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T14:00:23.952 INFO:teuthology.orchestra.run.smithi097.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T14:00:23.952 INFO:teuthology.orchestra.run.smithi097.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T14:00:23.952 INFO:teuthology.orchestra.run.smithi097.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T14:00:23.952 INFO:teuthology.orchestra.run.smithi097.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T14:00:23.952 INFO:teuthology.orchestra.run.smithi097.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T14:00:23.952 INFO:teuthology.orchestra.run.smithi097.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T14:00:23.952 INFO:teuthology.orchestra.run.smithi097.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T14:00:23.953 INFO:teuthology.orchestra.run.smithi097.stdout:Installing weak dependencies: 2024-04-25T14:00:23.953 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T14:00:23.953 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T14:00:23.953 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-04-25T14:00:23.953 INFO:teuthology.orchestra.run.smithi097.stdout:======================================================================================= 2024-04-25T14:00:23.953 INFO:teuthology.orchestra.run.smithi097.stdout:Install 148 Packages 2024-04-25T14:00:23.953 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrade 4 Packages 2024-04-25T14:00:23.953 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T14:00:23.953 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 214 M 2024-04-25T14:00:23.954 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-04-25T14:00:24.256 INFO:teuthology.orchestra.run.smithi096.stdout:(6/152): ceph-mds-19.0.0-3291.g994e2f20.el9.x86 3.0 MB/s | 2.3 MB 00:00 2024-04-25T14:00:24.398 INFO:teuthology.orchestra.run.smithi096.stdout:(7/152): ceph-mon-19.0.0-3291.g994e2f20.el9.x86 9.1 MB/s | 4.9 MB 00:00 2024-04-25T14:00:24.665 INFO:teuthology.orchestra.run.smithi096.stdout:(8/152): ceph-common-19.0.0-3291.g994e2f20.el9. 10 MB/s | 20 MB 00:01 2024-04-25T14:00:24.715 INFO:teuthology.orchestra.run.smithi096.stdout:(9/152): ceph-selinux-19.0.0-3291.g994e2f20.el9 507 kB/s | 25 kB 00:00 2024-04-25T14:00:24.739 INFO:teuthology.orchestra.run.smithi151.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T14:00:24.740 INFO:teuthology.orchestra.run.smithi151.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T14:00:24.751 INFO:teuthology.orchestra.run.smithi151.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T14:00:24.752 INFO:teuthology.orchestra.run.smithi151.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T14:00:24.753 INFO:teuthology.orchestra.run.smithi151.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T14:00:24.754 INFO:teuthology.orchestra.run.smithi151.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T14:00:24.755 INFO:teuthology.orchestra.run.smithi151.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T14:00:24.758 INFO:teuthology.orchestra.run.smithi151.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T14:00:24.759 INFO:teuthology.orchestra.run.smithi151.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T14:00:24.819 INFO:teuthology.orchestra.run.smithi151.stdout:Dependencies resolved. 2024-04-25T14:00:24.839 INFO:teuthology.orchestra.run.smithi151.stdout:======================================================================================= 2024-04-25T14:00:24.839 INFO:teuthology.orchestra.run.smithi151.stdout: Package Arch Version Repository Size 2024-04-25T14:00:24.839 INFO:teuthology.orchestra.run.smithi151.stdout:======================================================================================= 2024-04-25T14:00:24.839 INFO:teuthology.orchestra.run.smithi151.stdout:Installing: 2024-04-25T14:00:24.839 INFO:teuthology.orchestra.run.smithi151.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T14:00:24.839 INFO:teuthology.orchestra.run.smithi151.stdout: ceph x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 6.6 k 2024-04-25T14:00:24.839 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-base x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.4 M 2024-04-25T14:00:24.839 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 863 k 2024-04-25T14:00:24.839 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 147 k 2024-04-25T14:00:24.839 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 1.5 M 2024-04-25T14:00:24.839 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 142 k 2024-04-25T14:00:24.840 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 3.5 M 2024-04-25T14:00:24.840 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 7.4 M 2024-04-25T14:00:24.840 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-rook noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 49 k 2024-04-25T14:00:24.840 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-radosgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 10 M 2024-04-25T14:00:24.840 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-test x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 43 M 2024-04-25T14:00:24.840 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-volume noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 273 k 2024-04-25T14:00:24.840 INFO:teuthology.orchestra.run.smithi151.stdout: cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 762 k 2024-04-25T14:00:24.840 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 224 k 2024-04-25T14:00:24.840 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-shell x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 27 k 2024-04-25T14:00:24.840 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-top noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 18 k 2024-04-25T14:00:24.840 INFO:teuthology.orchestra.run.smithi151.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T14:00:24.840 INFO:teuthology.orchestra.run.smithi151.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T14:00:24.840 INFO:teuthology.orchestra.run.smithi151.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T14:00:24.840 INFO:teuthology.orchestra.run.smithi151.stdout: libcephfs-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 32 k 2024-04-25T14:00:24.841 INFO:teuthology.orchestra.run.smithi151.stdout: libcephfs2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 756 k 2024-04-25T14:00:24.841 INFO:teuthology.orchestra.run.smithi151.stdout: librados-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 127 k 2024-04-25T14:00:24.841 INFO:teuthology.orchestra.run.smithi151.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T14:00:24.841 INFO:teuthology.orchestra.run.smithi151.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T14:00:24.841 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T14:00:24.841 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T14:00:24.841 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T14:00:24.841 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T14:00:24.841 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cephfs x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 161 k 2024-04-25T14:00:24.841 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rados x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 321 k 2024-04-25T14:00:24.841 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 298 k 2024-04-25T14:00:24.841 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 100 k 2024-04-25T14:00:24.841 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 86 k 2024-04-25T14:00:24.841 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.0 M 2024-04-25T14:00:24.842 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-nbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 169 k 2024-04-25T14:00:24.842 INFO:teuthology.orchestra.run.smithi151.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T14:00:24.842 INFO:teuthology.orchestra.run.smithi151.stdout:Upgrading: 2024-04-25T14:00:24.842 INFO:teuthology.orchestra.run.smithi151.stdout: librados2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.3 M 2024-04-25T14:00:24.842 INFO:teuthology.orchestra.run.smithi151.stdout: librbd1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.1 M 2024-04-25T14:00:24.842 INFO:teuthology.orchestra.run.smithi151.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T14:00:24.842 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T14:00:24.842 INFO:teuthology.orchestra.run.smithi151.stdout:Installing dependencies: 2024-04-25T14:00:24.842 INFO:teuthology.orchestra.run.smithi151.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T14:00:24.842 INFO:teuthology.orchestra.run.smithi151.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T14:00:24.842 INFO:teuthology.orchestra.run.smithi151.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T14:00:24.842 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 20 M 2024-04-25T14:00:24.842 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 34 k 2024-04-25T14:00:24.843 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mds x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 2.3 M 2024-04-25T14:00:24.843 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 249 k 2024-04-25T14:00:24.843 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mon x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 4.9 M 2024-04-25T14:00:24.843 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-osd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 18 M 2024-04-25T14:00:24.843 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 16 k 2024-04-25T14:00:24.843 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-selinux x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 25 k 2024-04-25T14:00:24.843 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T14:00:24.843 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T14:00:24.843 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T14:00:24.843 INFO:teuthology.orchestra.run.smithi151.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T14:00:24.843 INFO:teuthology.orchestra.run.smithi151.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T14:00:24.844 INFO:teuthology.orchestra.run.smithi151.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T14:00:24.844 INFO:teuthology.orchestra.run.smithi151.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T14:00:24.844 INFO:teuthology.orchestra.run.smithi151.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T14:00:24.844 INFO:teuthology.orchestra.run.smithi151.stdout: libcephsqlite x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 158 k 2024-04-25T14:00:24.844 INFO:teuthology.orchestra.run.smithi151.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T14:00:24.844 INFO:teuthology.orchestra.run.smithi151.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T14:00:24.844 INFO:teuthology.orchestra.run.smithi151.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T14:00:24.844 INFO:teuthology.orchestra.run.smithi151.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T14:00:24.844 INFO:teuthology.orchestra.run.smithi151.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T14:00:24.844 INFO:teuthology.orchestra.run.smithi151.stdout: libradosstriper1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 497 k 2024-04-25T14:00:24.844 INFO:teuthology.orchestra.run.smithi151.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T14:00:24.845 INFO:teuthology.orchestra.run.smithi151.stdout: librgw2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.2 M 2024-04-25T14:00:24.845 INFO:teuthology.orchestra.run.smithi151.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T14:00:24.845 INFO:teuthology.orchestra.run.smithi151.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T14:00:24.845 INFO:teuthology.orchestra.run.smithi151.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T14:00:24.845 INFO:teuthology.orchestra.run.smithi151.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T14:00:24.845 INFO:teuthology.orchestra.run.smithi151.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T14:00:24.845 INFO:teuthology.orchestra.run.smithi151.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T14:00:24.845 INFO:teuthology.orchestra.run.smithi151.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T14:00:24.845 INFO:teuthology.orchestra.run.smithi151.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T14:00:24.845 INFO:teuthology.orchestra.run.smithi151.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T14:00:24.845 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T14:00:24.845 INFO:teuthology.orchestra.run.smithi151.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T14:00:24.845 INFO:teuthology.orchestra.run.smithi151.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T14:00:24.846 INFO:teuthology.orchestra.run.smithi151.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T14:00:24.846 INFO:teuthology.orchestra.run.smithi151.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T14:00:24.846 INFO:teuthology.orchestra.run.smithi151.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T14:00:24.846 INFO:teuthology.orchestra.run.smithi151.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T14:00:24.846 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T14:00:24.847 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-argparse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 45 k 2024-04-25T14:00:24.847 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 135 k 2024-04-25T14:00:24.847 INFO:teuthology.orchestra.run.smithi151.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T14:00:24.847 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T14:00:24.848 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T14:00:24.848 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T14:00:24.848 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T14:00:24.848 INFO:teuthology.orchestra.run.smithi151.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T14:00:24.848 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T14:00:24.848 INFO:teuthology.orchestra.run.smithi151.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T14:00:24.848 INFO:teuthology.orchestra.run.smithi151.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T14:00:24.848 INFO:teuthology.orchestra.run.smithi151.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T14:00:24.848 INFO:teuthology.orchestra.run.smithi151.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T14:00:24.848 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T14:00:24.848 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T14:00:24.848 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T14:00:24.848 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T14:00:24.848 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T14:00:24.849 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T14:00:24.849 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T14:00:24.849 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T14:00:24.849 INFO:teuthology.orchestra.run.smithi151.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T14:00:24.849 INFO:teuthology.orchestra.run.smithi151.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T14:00:24.849 INFO:teuthology.orchestra.run.smithi151.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T14:00:24.849 INFO:teuthology.orchestra.run.smithi151.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T14:00:24.849 INFO:teuthology.orchestra.run.smithi151.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T14:00:24.849 INFO:teuthology.orchestra.run.smithi151.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T14:00:24.849 INFO:teuthology.orchestra.run.smithi151.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T14:00:24.849 INFO:teuthology.orchestra.run.smithi151.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T14:00:24.849 INFO:teuthology.orchestra.run.smithi151.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T14:00:24.849 INFO:teuthology.orchestra.run.smithi151.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T14:00:24.849 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T14:00:24.850 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T14:00:24.850 INFO:teuthology.orchestra.run.smithi151.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T14:00:24.850 INFO:teuthology.orchestra.run.smithi151.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T14:00:24.850 INFO:teuthology.orchestra.run.smithi151.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T14:00:24.850 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T14:00:24.850 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T14:00:24.850 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T14:00:24.850 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T14:00:24.850 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T14:00:24.850 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T14:00:24.850 INFO:teuthology.orchestra.run.smithi151.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T14:00:24.850 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T14:00:24.850 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T14:00:24.850 INFO:teuthology.orchestra.run.smithi151.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T14:00:24.851 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T14:00:24.851 INFO:teuthology.orchestra.run.smithi151.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T14:00:24.851 INFO:teuthology.orchestra.run.smithi151.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T14:00:24.851 INFO:teuthology.orchestra.run.smithi151.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T14:00:24.851 INFO:teuthology.orchestra.run.smithi151.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T14:00:24.851 INFO:teuthology.orchestra.run.smithi151.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T14:00:24.851 INFO:teuthology.orchestra.run.smithi151.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T14:00:24.851 INFO:teuthology.orchestra.run.smithi151.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T14:00:24.851 INFO:teuthology.orchestra.run.smithi151.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T14:00:24.851 INFO:teuthology.orchestra.run.smithi151.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T14:00:24.851 INFO:teuthology.orchestra.run.smithi151.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T14:00:24.851 INFO:teuthology.orchestra.run.smithi151.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T14:00:24.851 INFO:teuthology.orchestra.run.smithi151.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T14:00:24.852 INFO:teuthology.orchestra.run.smithi151.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T14:00:24.852 INFO:teuthology.orchestra.run.smithi151.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T14:00:24.852 INFO:teuthology.orchestra.run.smithi151.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T14:00:24.852 INFO:teuthology.orchestra.run.smithi151.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T14:00:24.852 INFO:teuthology.orchestra.run.smithi151.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T14:00:24.852 INFO:teuthology.orchestra.run.smithi151.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T14:00:24.852 INFO:teuthology.orchestra.run.smithi151.stdout:Installing weak dependencies: 2024-04-25T14:00:24.852 INFO:teuthology.orchestra.run.smithi151.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T14:00:24.852 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-25T14:00:24.852 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction Summary 2024-04-25T14:00:24.852 INFO:teuthology.orchestra.run.smithi151.stdout:======================================================================================= 2024-04-25T14:00:24.852 INFO:teuthology.orchestra.run.smithi151.stdout:Install 148 Packages 2024-04-25T14:00:24.852 INFO:teuthology.orchestra.run.smithi151.stdout:Upgrade 4 Packages 2024-04-25T14:00:24.853 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-25T14:00:24.853 INFO:teuthology.orchestra.run.smithi151.stdout:Total download size: 214 M 2024-04-25T14:00:24.853 INFO:teuthology.orchestra.run.smithi151.stdout:Downloading Packages: 2024-04-25T14:00:25.198 INFO:teuthology.orchestra.run.smithi096.stdout:(10/152): ceph-radosgw-19.0.0-3291.g994e2f20.el 13 MB/s | 10 MB 00:00 2024-04-25T14:00:25.273 INFO:teuthology.orchestra.run.smithi096.stdout:(11/152): cephfs-mirror-19.0.0-3291.g994e2f20.e 2.9 MB/s | 224 kB 00:00 2024-04-25T14:00:25.341 INFO:teuthology.orchestra.run.smithi096.stdout:(12/152): cephfs-shell-19.0.0-3291.g994e2f20.el 405 kB/s | 27 kB 00:00 2024-04-25T14:00:25.424 INFO:teuthology.orchestra.run.smithi096.stdout:(13/152): libcephfs-devel-19.0.0-3291.g994e2f20 381 kB/s | 32 kB 00:00 2024-04-25T14:00:25.425 INFO:teuthology.orchestra.run.smithi097.stdout:(1/152): ceph-19.0.0-3291.g994e2f20.el9.x86_64. 11 kB/s | 6.6 kB 00:00 2024-04-25T14:00:25.558 INFO:teuthology.orchestra.run.smithi096.stdout:(14/152): libcephfs2-19.0.0-3291.g994e2f20.el9. 5.5 MB/s | 756 kB 00:00 2024-04-25T14:00:25.633 INFO:teuthology.orchestra.run.smithi096.stdout:(15/152): libcephsqlite-19.0.0-3291.g994e2f20.e 2.1 MB/s | 158 kB 00:00 2024-04-25T14:00:25.708 INFO:teuthology.orchestra.run.smithi096.stdout:(16/152): librados-devel-19.0.0-3291.g994e2f20. 1.7 MB/s | 127 kB 00:00 2024-04-25T14:00:25.858 INFO:teuthology.orchestra.run.smithi096.stdout:(17/152): libradosstriper1-19.0.0-3291.g994e2f2 3.2 MB/s | 497 kB 00:00 2024-04-25T14:00:26.172 INFO:teuthology.orchestra.run.smithi151.stdout:(1/152): ceph-19.0.0-3291.g994e2f20.el9.x86_64. 9.8 kB/s | 6.6 kB 00:00 2024-04-25T14:00:26.767 INFO:teuthology.orchestra.run.smithi096.stdout:(18/152): librgw2-19.0.0-3291.g994e2f20.el9.x86 5.7 MB/s | 5.2 MB 00:00 2024-04-25T14:00:26.808 INFO:teuthology.orchestra.run.smithi097.stdout:(2/152): ceph-fuse-19.0.0-3291.g994e2f20.el9.x8 624 kB/s | 863 kB 00:01 2024-04-25T14:00:26.859 INFO:teuthology.orchestra.run.smithi096.stdout:(19/152): python3-ceph-argparse-19.0.0-3291.g99 494 kB/s | 45 kB 00:00 2024-04-25T14:00:26.883 INFO:teuthology.orchestra.run.smithi097.stdout:(3/152): ceph-base-19.0.0-3291.g994e2f20.el9.x8 2.6 MB/s | 5.4 MB 00:02 2024-04-25T14:00:26.926 INFO:teuthology.orchestra.run.smithi096.stdout:(20/152): python3-ceph-common-19.0.0-3291.g994e 2.0 MB/s | 135 kB 00:00 2024-04-25T14:00:27.017 INFO:teuthology.orchestra.run.smithi097.stdout:(4/152): ceph-immutable-object-cache-19.0.0-329 705 kB/s | 147 kB 00:00 2024-04-25T14:00:27.017 INFO:teuthology.orchestra.run.smithi096.stdout:(21/152): python3-cephfs-19.0.0-3291.g994e2f20. 1.7 MB/s | 161 kB 00:00 2024-04-25T14:00:27.135 INFO:teuthology.orchestra.run.smithi096.stdout:(22/152): python3-rados-19.0.0-3291.g994e2f20.e 2.7 MB/s | 321 kB 00:00 2024-04-25T14:00:27.243 INFO:teuthology.orchestra.run.smithi096.stdout:(23/152): python3-rbd-19.0.0-3291.g994e2f20.el9 2.7 MB/s | 298 kB 00:00 2024-04-25T14:00:27.335 INFO:teuthology.orchestra.run.smithi096.stdout:(24/152): python3-rgw-19.0.0-3291.g994e2f20.el9 1.1 MB/s | 100 kB 00:00 2024-04-25T14:00:27.341 INFO:teuthology.orchestra.run.smithi097.stdout:(5/152): ceph-mds-19.0.0-3291.g994e2f20.el9.x86 4.9 MB/s | 2.3 MB 00:00 2024-04-25T14:00:27.398 INFO:teuthology.orchestra.run.smithi151.stdout:(2/152): ceph-fuse-19.0.0-3291.g994e2f20.el9.x8 704 kB/s | 863 kB 00:01 2024-04-25T14:00:27.411 INFO:teuthology.orchestra.run.smithi096.stdout:(25/152): rbd-fuse-19.0.0-3291.g994e2f20.el9.x8 1.1 MB/s | 86 kB 00:00 2024-04-25T14:00:27.623 INFO:teuthology.orchestra.run.smithi151.stdout:(3/152): ceph-immutable-object-cache-19.0.0-329 654 kB/s | 147 kB 00:00 2024-04-25T14:00:28.068 INFO:teuthology.orchestra.run.smithi097.stdout:(6/152): ceph-mon-19.0.0-3291.g994e2f20.el9.x86 6.7 MB/s | 4.9 MB 00:00 2024-04-25T14:00:28.156 INFO:teuthology.orchestra.run.smithi151.stdout:(4/152): ceph-base-19.0.0-3291.g994e2f20.el9.x8 2.0 MB/s | 5.4 MB 00:02 2024-04-25T14:00:28.286 INFO:teuthology.orchestra.run.smithi096.stdout:(26/152): rbd-mirror-19.0.0-3291.g994e2f20.el9. 3.5 MB/s | 3.0 MB 00:00 2024-04-25T14:00:28.656 INFO:teuthology.orchestra.run.smithi151.stdout:(5/152): ceph-mgr-19.0.0-3291.g994e2f20.el9.x86 3.0 MB/s | 1.5 MB 00:00 2024-04-25T14:00:28.720 INFO:teuthology.orchestra.run.smithi096.stdout:(27/152): ceph-test-19.0.0-3291.g994e2f20.el9.x 11 MB/s | 43 MB 00:04 2024-04-25T14:00:28.745 INFO:teuthology.orchestra.run.smithi096.stdout:(28/152): rbd-nbd-19.0.0-3291.g994e2f20.el9.x86 370 kB/s | 169 kB 00:00 2024-04-25T14:00:28.812 INFO:teuthology.orchestra.run.smithi096.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3291.g 373 kB/s | 34 kB 00:00 2024-04-25T14:00:28.845 INFO:teuthology.orchestra.run.smithi096.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3291.g994e2f2 1.4 MB/s | 142 kB 00:00 2024-04-25T14:00:29.273 INFO:teuthology.orchestra.run.smithi151.stdout:(6/152): ceph-mds-19.0.0-3291.g994e2f20.el9.x86 1.4 MB/s | 2.3 MB 00:01 2024-04-25T14:00:29.426 INFO:teuthology.orchestra.run.smithi097.stdout:(7/152): ceph-mgr-19.0.0-3291.g994e2f20.el9.x86 639 kB/s | 1.5 MB 00:02 2024-04-25T14:00:29.579 INFO:teuthology.orchestra.run.smithi096.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3291.g994e2 4.6 MB/s | 3.5 MB 00:00 2024-04-25T14:00:29.626 INFO:teuthology.orchestra.run.smithi097.stdout:(8/152): ceph-osd-19.0.0-3291.g994e2f20.el9.x86 11 MB/s | 18 MB 00:01 2024-04-25T14:00:29.629 INFO:teuthology.orchestra.run.smithi096.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3291.g99 4.9 MB/s | 249 kB 00:00 2024-04-25T14:00:29.668 INFO:teuthology.orchestra.run.smithi097.stdout:(9/152): ceph-selinux-19.0.0-3291.g994e2f20.el9 610 kB/s | 25 kB 00:00 2024-04-25T14:00:29.679 INFO:teuthology.orchestra.run.smithi096.stdout:(33/152): ceph-mgr-rook-19.0.0-3291.g994e2f20.e 989 kB/s | 49 kB 00:00 2024-04-25T14:00:29.746 INFO:teuthology.orchestra.run.smithi096.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3291.g9 247 kB/s | 16 kB 00:00 2024-04-25T14:00:29.838 INFO:teuthology.orchestra.run.smithi096.stdout:(35/152): ceph-volume-19.0.0-3291.g994e2f20.el9 2.9 MB/s | 273 kB 00:00 2024-04-25T14:00:30.080 INFO:teuthology.orchestra.run.smithi096.stdout:(36/152): cephadm-19.0.0-3291.g994e2f20.el9.noa 3.1 MB/s | 762 kB 00:00 2024-04-25T14:00:30.124 INFO:teuthology.orchestra.run.smithi151.stdout:(7/152): ceph-mon-19.0.0-3291.g994e2f20.el9.x86 3.3 MB/s | 4.9 MB 00:01 2024-04-25T14:00:30.155 INFO:teuthology.orchestra.run.smithi096.stdout:(37/152): cephfs-top-19.0.0-3291.g994e2f20.el9. 243 kB/s | 18 kB 00:00 2024-04-25T14:00:30.431 INFO:teuthology.orchestra.run.smithi096.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 292 kB/s | 80 kB 00:00 2024-04-25T14:00:30.664 INFO:teuthology.orchestra.run.smithi096.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 3.4 MB/s | 803 kB 00:00 2024-04-25T14:00:30.739 INFO:teuthology.orchestra.run.smithi096.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 195 kB 00:00 2024-04-25T14:00:30.823 INFO:teuthology.orchestra.run.smithi096.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-04-25T14:00:31.007 INFO:teuthology.orchestra.run.smithi096.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 6.7 MB/s | 1.2 MB 00:00 2024-04-25T14:00:31.065 INFO:teuthology.orchestra.run.smithi096.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-25T14:00:31.166 INFO:teuthology.orchestra.run.smithi096.stdout:(44/152): ceph-mgr-diskprediction-local-19.0.0- 3.2 MB/s | 7.4 MB 00:02 2024-04-25T14:00:31.191 INFO:teuthology.orchestra.run.smithi096.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-04-25T14:00:31.208 INFO:teuthology.orchestra.run.smithi096.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 3.0 MB/s | 125 kB 00:00 2024-04-25T14:00:31.250 INFO:teuthology.orchestra.run.smithi096.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-04-25T14:00:31.283 INFO:teuthology.orchestra.run.smithi096.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 8.9 MB/s | 689 kB 00:00 2024-04-25T14:00:31.333 INFO:teuthology.orchestra.run.smithi096.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 11 MB/s | 943 kB 00:00 2024-04-25T14:00:31.359 INFO:teuthology.orchestra.run.smithi096.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.4 MB/s | 107 kB 00:00 2024-04-25T14:00:31.384 INFO:teuthology.orchestra.run.smithi096.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 430 kB/s | 21 kB 00:00 2024-04-25T14:00:31.409 INFO:teuthology.orchestra.run.smithi096.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-25T14:00:31.435 INFO:teuthology.orchestra.run.smithi096.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 659 kB/s | 33 kB 00:00 2024-04-25T14:00:31.552 INFO:teuthology.orchestra.run.smithi096.stdout:(54/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 21 MB/s | 3.0 MB 00:00 2024-04-25T14:00:31.577 INFO:teuthology.orchestra.run.smithi096.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 128 kB/s | 18 kB 00:00 2024-04-25T14:00:31.619 INFO:teuthology.orchestra.run.smithi096.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 269 kB/s | 11 kB 00:00 2024-04-25T14:00:31.644 INFO:teuthology.orchestra.run.smithi096.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 160 kB 00:00 2024-04-25T14:00:31.670 INFO:teuthology.orchestra.run.smithi096.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 906 kB/s | 45 kB 00:00 2024-04-25T14:00:31.703 INFO:teuthology.orchestra.run.smithi096.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-04-25T14:00:31.728 INFO:teuthology.orchestra.run.smithi096.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 9.8 MB/s | 582 kB 00:00 2024-04-25T14:00:31.745 INFO:teuthology.orchestra.run.smithi096.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 622 kB/s | 25 kB 00:00 2024-04-25T14:00:31.762 INFO:teuthology.orchestra.run.smithi096.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-04-25T14:00:31.796 INFO:teuthology.orchestra.run.smithi096.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-25T14:00:31.821 INFO:teuthology.orchestra.run.smithi096.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-04-25T14:00:31.847 INFO:teuthology.orchestra.run.smithi096.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 708 kB/s | 35 kB 00:00 2024-04-25T14:00:31.860 INFO:teuthology.orchestra.run.smithi097.stdout:(10/152): ceph-common-19.0.0-3291.g994e2f20.el9 2.8 MB/s | 20 MB 00:07 2024-04-25T14:00:31.987 INFO:teuthology.orchestra.run.smithi097.stdout:(11/152): cephfs-mirror-19.0.0-3291.g994e2f20.e 1.7 MB/s | 224 kB 00:00 2024-04-25T14:00:32.047 INFO:teuthology.orchestra.run.smithi096.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 21 MB/s | 4.7 MB 00:00 2024-04-25T14:00:32.062 INFO:teuthology.orchestra.run.smithi097.stdout:(12/152): cephfs-shell-19.0.0-3291.g994e2f20.el 361 kB/s | 27 kB 00:00 2024-04-25T14:00:32.130 INFO:teuthology.orchestra.run.smithi096.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-04-25T14:00:32.146 INFO:teuthology.orchestra.run.smithi097.stdout:(13/152): libcephfs-devel-19.0.0-3291.g994e2f20 381 kB/s | 32 kB 00:00 2024-04-25T14:00:32.172 INFO:teuthology.orchestra.run.smithi096.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-04-25T14:00:32.206 INFO:teuthology.orchestra.run.smithi096.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-04-25T14:00:32.231 INFO:teuthology.orchestra.run.smithi096.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.3 MB/s | 133 kB 00:00 2024-04-25T14:00:32.431 INFO:teuthology.orchestra.run.smithi096.stdout:(71/152): postgresql-server-13.14-1.el9.x86_64. 26 MB/s | 5.8 MB 00:00 2024-04-25T14:00:32.465 INFO:teuthology.orchestra.run.smithi096.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 4.3 MB/s | 1.0 MB 00:00 2024-04-25T14:00:32.490 INFO:teuthology.orchestra.run.smithi096.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-25T14:00:32.663 INFO:teuthology.orchestra.run.smithi097.stdout:(14/152): ceph-test-19.0.0-3291.g994e2f20.el9.x 14 MB/s | 43 MB 00:02 2024-04-25T14:00:32.691 INFO:teuthology.orchestra.run.smithi096.stdout:(74/152): python3-babel-2.9.1-2.el9.noarch.rpm 26 MB/s | 6.0 MB 00:00 2024-04-25T14:00:32.696 INFO:teuthology.orchestra.run.smithi097.stdout:(15/152): libcephfs2-19.0.0-3291.g994e2f20.el9. 1.3 MB/s | 756 kB 00:00 2024-04-25T14:00:32.716 INFO:teuthology.orchestra.run.smithi096.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.1 MB/s | 244 kB 00:00 2024-04-25T14:00:32.721 INFO:teuthology.orchestra.run.smithi097.stdout:(16/152): libcephsqlite-19.0.0-3291.g994e2f20.e 2.7 MB/s | 158 kB 00:00 2024-04-25T14:00:32.741 INFO:teuthology.orchestra.run.smithi096.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-04-25T14:00:32.764 INFO:teuthology.orchestra.run.smithi097.stdout:(17/152): librados-devel-19.0.0-3291.g994e2f20. 1.9 MB/s | 127 kB 00:00 2024-04-25T14:00:32.766 INFO:teuthology.orchestra.run.smithi096.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 513 kB/s | 26 kB 00:00 2024-04-25T14:00:32.773 INFO:teuthology.orchestra.run.smithi151.stdout:(8/152): ceph-radosgw-19.0.0-3291.g994e2f20.el9 3.8 MB/s | 10 MB 00:02 2024-04-25T14:00:32.791 INFO:teuthology.orchestra.run.smithi096.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-04-25T14:00:32.797 INFO:teuthology.orchestra.run.smithi097.stdout:(18/152): libradosstriper1-19.0.0-3291.g994e2f2 6.5 MB/s | 497 kB 00:00 2024-04-25T14:00:32.815 INFO:teuthology.orchestra.run.smithi151.stdout:(9/152): ceph-selinux-19.0.0-3291.g994e2f20.el9 609 kB/s | 25 kB 00:00 2024-04-25T14:00:32.817 INFO:teuthology.orchestra.run.smithi096.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-04-25T14:00:32.839 INFO:teuthology.orchestra.run.smithi097.stdout:(19/152): python3-ceph-argparse-19.0.0-3291.g99 1.1 MB/s | 45 kB 00:00 2024-04-25T14:00:32.842 INFO:teuthology.orchestra.run.smithi096.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 695 kB/s | 35 kB 00:00 2024-04-25T14:00:32.890 INFO:teuthology.orchestra.run.smithi097.stdout:(20/152): python3-ceph-common-19.0.0-3291.g994e 2.6 MB/s | 135 kB 00:00 2024-04-25T14:00:32.940 INFO:teuthology.orchestra.run.smithi097.stdout:(21/152): python3-cephfs-19.0.0-3291.g994e2f20. 3.2 MB/s | 161 kB 00:00 2024-04-25T14:00:32.998 INFO:teuthology.orchestra.run.smithi097.stdout:(22/152): python3-rados-19.0.0-3291.g994e2f20.e 5.4 MB/s | 321 kB 00:00 2024-04-25T14:00:33.018 INFO:teuthology.orchestra.run.smithi096.stdout:(81/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 25 MB/s | 5.1 MB 00:00 2024-04-25T14:00:33.043 INFO:teuthology.orchestra.run.smithi096.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.2 MB/s | 250 kB 00:00 2024-04-25T14:00:33.057 INFO:teuthology.orchestra.run.smithi097.stdout:(23/152): python3-rbd-19.0.0-3291.g994e2f20.el9 5.0 MB/s | 298 kB 00:00 2024-04-25T14:00:33.068 INFO:teuthology.orchestra.run.smithi096.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 4.4 MB/s | 222 kB 00:00 2024-04-25T14:00:33.107 INFO:teuthology.orchestra.run.smithi097.stdout:(24/152): python3-rgw-19.0.0-3291.g994e2f20.el9 2.0 MB/s | 100 kB 00:00 2024-04-25T14:00:33.122 INFO:teuthology.orchestra.run.smithi096.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 976 kB/s | 77 kB 00:00 2024-04-25T14:00:33.152 INFO:teuthology.orchestra.run.smithi096.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 504 kB/s | 42 kB 00:00 2024-04-25T14:00:33.157 INFO:teuthology.orchestra.run.smithi097.stdout:(25/152): rbd-fuse-19.0.0-3291.g994e2f20.el9.x8 1.7 MB/s | 86 kB 00:00 2024-04-25T14:00:33.177 INFO:teuthology.orchestra.run.smithi096.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 4.8 MB/s | 269 kB 00:00 2024-04-25T14:00:33.202 INFO:teuthology.orchestra.run.smithi096.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-25T14:00:33.228 INFO:teuthology.orchestra.run.smithi096.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.5 MB/s | 279 kB 00:00 2024-04-25T14:00:33.253 INFO:teuthology.orchestra.run.smithi096.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-25T14:00:33.278 INFO:teuthology.orchestra.run.smithi096.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-25T14:00:33.399 INFO:teuthology.orchestra.run.smithi097.stdout:(26/152): rbd-mirror-19.0.0-3291.g994e2f20.el9. 13 MB/s | 3.0 MB 00:00 2024-04-25T14:00:33.450 INFO:teuthology.orchestra.run.smithi097.stdout:(27/152): rbd-nbd-19.0.0-3291.g994e2f20.el9.x86 3.3 MB/s | 169 kB 00:00 2024-04-25T14:00:33.491 INFO:teuthology.orchestra.run.smithi097.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3291.g 826 kB/s | 34 kB 00:00 2024-04-25T14:00:33.542 INFO:teuthology.orchestra.run.smithi097.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3291.g994e2f2 2.8 MB/s | 142 kB 00:00 2024-04-25T14:00:33.604 INFO:teuthology.orchestra.run.smithi096.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 128 kB/s | 42 kB 00:00 2024-04-25T14:00:33.625 INFO:teuthology.orchestra.run.smithi097.stdout:(30/152): librgw2-19.0.0-3291.g994e2f20.el9.x86 6.0 MB/s | 5.2 MB 00:00 2024-04-25T14:00:33.784 INFO:teuthology.orchestra.run.smithi097.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3291.g994e2 15 MB/s | 3.5 MB 00:00 2024-04-25T14:00:33.804 INFO:teuthology.orchestra.run.smithi096.stdout:(92/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 30 MB/s | 16 MB 00:00 2024-04-25T14:00:33.821 INFO:teuthology.orchestra.run.smithi096.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 205 kB/s | 44 kB 00:00 2024-04-25T14:00:33.835 INFO:teuthology.orchestra.run.smithi097.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3291.g99 4.9 MB/s | 249 kB 00:00 2024-04-25T14:00:33.846 INFO:teuthology.orchestra.run.smithi096.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 5.2 MB/s | 221 kB 00:00 2024-04-25T14:00:33.885 INFO:teuthology.orchestra.run.smithi097.stdout:(33/152): ceph-mgr-rook-19.0.0-3291.g994e2f20.e 990 kB/s | 49 kB 00:00 2024-04-25T14:00:33.935 INFO:teuthology.orchestra.run.smithi097.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3291.g9 330 kB/s | 16 kB 00:00 2024-04-25T14:00:33.994 INFO:teuthology.orchestra.run.smithi097.stdout:(35/152): ceph-volume-19.0.0-3291.g994e2f20.el9 4.6 MB/s | 273 kB 00:00 2024-04-25T14:00:34.021 INFO:teuthology.orchestra.run.smithi096.stdout:(95/152): ceph-osd-19.0.0-3291.g994e2f20.el9.x8 1.8 MB/s | 18 MB 00:09 2024-04-25T14:00:34.046 INFO:teuthology.orchestra.run.smithi096.stdout:(96/152): qatlib-service-24.02.0-1.el9.x86_64.r 159 kB/s | 36 kB 00:00 2024-04-25T14:00:34.069 INFO:teuthology.orchestra.run.smithi097.stdout:(36/152): cephadm-19.0.0-3291.g994e2f20.el9.noa 10 MB/s | 762 kB 00:00 2024-04-25T14:00:34.071 INFO:teuthology.orchestra.run.smithi096.stdout:(97/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 210 kB/s | 47 kB 00:00 2024-04-25T14:00:34.097 INFO:teuthology.orchestra.run.smithi096.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 6.0 MB/s | 305 kB 00:00 2024-04-25T14:00:34.103 INFO:teuthology.orchestra.run.smithi097.stdout:(37/152): cephfs-top-19.0.0-3291.g994e2f20.el9. 550 kB/s | 18 kB 00:00 2024-04-25T14:00:34.122 INFO:teuthology.orchestra.run.smithi096.stdout:(99/152): uuid-1.6.2-55.el9.x86_64.rpm 769 kB/s | 58 kB 00:00 2024-04-25T14:00:34.147 INFO:teuthology.orchestra.run.smithi096.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.2 MB/s | 94 kB 00:00 2024-04-25T14:00:34.173 INFO:teuthology.orchestra.run.smithi096.stdout:(101/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 847 kB/s | 64 kB 00:00 2024-04-25T14:00:34.198 INFO:teuthology.orchestra.run.smithi096.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 898 kB/s | 45 kB 00:00 2024-04-25T14:00:34.223 INFO:teuthology.orchestra.run.smithi096.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 297 kB/s | 22 kB 00:00 2024-04-25T14:00:34.257 INFO:teuthology.orchestra.run.smithi096.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 10 MB/s | 863 kB 00:00 2024-04-25T14:00:34.278 INFO:teuthology.orchestra.run.smithi097.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 459 kB/s | 80 kB 00:00 2024-04-25T14:00:34.454 INFO:teuthology.orchestra.run.smithi097.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 4.5 MB/s | 803 kB 00:00 2024-04-25T14:00:34.465 INFO:teuthology.orchestra.run.smithi096.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 93 kB/s | 19 kB 00:00 2024-04-25T14:00:34.495 INFO:teuthology.orchestra.run.smithi097.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 4.6 MB/s | 195 kB 00:00 2024-04-25T14:00:34.554 INFO:teuthology.orchestra.run.smithi097.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-04-25T14:00:34.705 INFO:teuthology.orchestra.run.smithi097.stdout:(42/152): ceph-mgr-diskprediction-local-19.0.0- 6.9 MB/s | 7.4 MB 00:01 2024-04-25T14:00:34.732 INFO:teuthology.orchestra.run.smithi096.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 609 kB/s | 309 kB 00:00 2024-04-25T14:00:34.746 INFO:teuthology.orchestra.run.smithi097.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 6.4 MB/s | 1.2 MB 00:00 2024-04-25T14:00:34.774 INFO:teuthology.orchestra.run.smithi097.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-04-25T14:00:34.775 INFO:teuthology.orchestra.run.smithi096.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 998 kB/s | 548 kB 00:00 2024-04-25T14:00:34.797 INFO:teuthology.orchestra.run.smithi097.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-25T14:00:34.822 INFO:teuthology.orchestra.run.smithi097.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-25T14:00:34.824 INFO:teuthology.orchestra.run.smithi096.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 282 kB/s | 26 kB 00:00 2024-04-25T14:00:34.847 INFO:teuthology.orchestra.run.smithi097.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-25T14:00:34.858 INFO:teuthology.orchestra.run.smithi096.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 581 kB/s | 49 kB 00:00 2024-04-25T14:00:34.889 INFO:teuthology.orchestra.run.smithi097.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 10 MB/s | 689 kB 00:00 2024-04-25T14:00:34.933 INFO:teuthology.orchestra.run.smithi096.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 620 kB/s | 67 kB 00:00 2024-04-25T14:00:34.965 INFO:teuthology.orchestra.run.smithi097.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 7.9 MB/s | 943 kB 00:00 2024-04-25T14:00:34.966 INFO:teuthology.orchestra.run.smithi096.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-04-25T14:00:34.990 INFO:teuthology.orchestra.run.smithi097.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.0 MB/s | 107 kB 00:00 2024-04-25T14:00:35.023 INFO:teuthology.orchestra.run.smithi097.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 369 kB/s | 21 kB 00:00 2024-04-25T14:00:35.049 INFO:teuthology.orchestra.run.smithi097.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-04-25T14:00:35.091 INFO:teuthology.orchestra.run.smithi097.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 490 kB/s | 33 kB 00:00 2024-04-25T14:00:35.300 INFO:teuthology.orchestra.run.smithi096.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 1.6 MB/s | 548 kB 00:00 2024-04-25T14:00:35.333 INFO:teuthology.orchestra.run.smithi097.stdout:(54/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2024-04-25T14:00:35.358 INFO:teuthology.orchestra.run.smithi097.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 68 kB/s | 18 kB 00:00 2024-04-25T14:00:35.384 INFO:teuthology.orchestra.run.smithi097.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-04-25T14:00:35.400 INFO:teuthology.orchestra.run.smithi096.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 434 kB/s | 43 kB 00:00 2024-04-25T14:00:35.425 INFO:teuthology.orchestra.run.smithi097.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 160 kB 00:00 2024-04-25T14:00:35.450 INFO:teuthology.orchestra.run.smithi097.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 678 kB/s | 45 kB 00:00 2024-04-25T14:00:35.501 INFO:teuthology.orchestra.run.smithi096.stdout:(114/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.4 MB/s | 838 kB 00:00 2024-04-25T14:00:35.517 INFO:teuthology.orchestra.run.smithi097.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 7.1 MB/s | 662 kB 00:00 2024-04-25T14:00:35.526 INFO:teuthology.orchestra.run.smithi096.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 257 kB/s | 32 kB 00:00 2024-04-25T14:00:35.559 INFO:teuthology.orchestra.run.smithi097.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 5.2 MB/s | 582 kB 00:00 2024-04-25T14:00:35.584 INFO:teuthology.orchestra.run.smithi097.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 385 kB/s | 25 kB 00:00 2024-04-25T14:00:35.601 INFO:teuthology.orchestra.run.smithi096.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 150 kB/s | 15 kB 00:00 2024-04-25T14:00:35.626 INFO:teuthology.orchestra.run.smithi097.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 711 kB/s | 47 kB 00:00 2024-04-25T14:00:35.635 INFO:teuthology.orchestra.run.smithi096.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-04-25T14:00:35.669 INFO:teuthology.orchestra.run.smithi097.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-04-25T14:00:35.694 INFO:teuthology.orchestra.run.smithi097.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 188 kB 00:00 2024-04-25T14:00:35.736 INFO:teuthology.orchestra.run.smithi097.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 530 kB/s | 35 kB 00:00 2024-04-25T14:00:35.860 INFO:teuthology.orchestra.run.smithi096.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 1.4 MB/s | 358 kB 00:00 2024-04-25T14:00:35.919 INFO:teuthology.orchestra.run.smithi096.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.0 MB/s | 303 kB 00:00 2024-04-25T14:00:35.961 INFO:teuthology.orchestra.run.smithi096.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 352 kB/s | 35 kB 00:00 2024-04-25T14:00:36.044 INFO:teuthology.orchestra.run.smithi096.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 1.6 MB/s | 206 kB 00:00 2024-04-25T14:00:36.119 INFO:teuthology.orchestra.run.smithi097.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 11 MB/s | 4.7 MB 00:00 2024-04-25T14:00:36.162 INFO:teuthology.orchestra.run.smithi096.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 1.2 MB/s | 144 kB 00:00 2024-04-25T14:00:36.303 INFO:teuthology.orchestra.run.smithi096.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 75 kB/s | 11 kB 00:00 2024-04-25T14:00:36.395 INFO:teuthology.orchestra.run.smithi096.stdout:(124/152): python3-jaraco-classes-3.2.1-5.el9.n 193 kB/s | 18 kB 00:00 2024-04-25T14:00:36.411 INFO:teuthology.orchestra.run.smithi097.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 6.6 MB/s | 4.4 MB 00:00 2024-04-25T14:00:36.479 INFO:teuthology.orchestra.run.smithi096.stdout:(125/152): python3-jaraco-collections-3.0.0-8.e 278 kB/s | 23 kB 00:00 2024-04-25T14:00:36.494 INFO:teuthology.orchestra.run.smithi097.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 4.2 MB/s | 1.6 MB 00:00 2024-04-25T14:00:36.537 INFO:teuthology.orchestra.run.smithi097.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 7.2 MB/s | 911 kB 00:00 2024-04-25T14:00:36.562 INFO:teuthology.orchestra.run.smithi097.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-04-25T14:00:36.562 INFO:teuthology.orchestra.run.smithi096.stdout:(126/152): python3-jaraco-functools-3.5.0-2.el9 233 kB/s | 19 kB 00:00 2024-04-25T14:00:36.646 INFO:teuthology.orchestra.run.smithi096.stdout:(127/152): python3-jaraco-text-3.2.0-6.el9.noar 236 kB/s | 20 kB 00:00 2024-04-25T14:00:36.673 INFO:teuthology.orchestra.run.smithi151.stdout:(10/152): ceph-osd-19.0.0-3291.g994e2f20.el9.x8 2.4 MB/s | 18 MB 00:07 2024-04-25T14:00:36.696 INFO:teuthology.orchestra.run.smithi096.stdout:(128/152): python3-grpcio-1.46.7-10.el9.x86_64. 2.8 MB/s | 2.0 MB 00:00 2024-04-25T14:00:36.779 INFO:teuthology.orchestra.run.smithi097.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 4.7 MB/s | 1.0 MB 00:00 2024-04-25T14:00:36.788 INFO:teuthology.orchestra.run.smithi096.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 505 kB/s | 46 kB 00:00 2024-04-25T14:00:36.872 INFO:teuthology.orchestra.run.smithi096.stdout:(130/152): python3-more-itertools-8.12.0-2.el9. 947 kB/s | 79 kB 00:00 2024-04-25T14:00:37.013 INFO:teuthology.orchestra.run.smithi097.stdout:(72/152): postgresql-server-13.14-1.el9.x86_64. 12 MB/s | 5.8 MB 00:00 2024-04-25T14:00:37.018 INFO:teuthology.orchestra.run.smithi096.stdout:(131/152): python3-natsort-7.1.1-5.el9.noarch.r 407 kB/s | 58 kB 00:00 2024-04-25T14:00:37.038 INFO:teuthology.orchestra.run.smithi097.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 340 kB/s | 87 kB 00:00 2024-04-25T14:00:37.047 INFO:teuthology.orchestra.run.smithi096.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 2.5 MB/s | 1.0 MB 00:00 2024-04-25T14:00:37.132 INFO:teuthology.orchestra.run.smithi151.stdout:(11/152): ceph-test-19.0.0-3291.g994e2f20.el9.x 10 MB/s | 43 MB 00:04 2024-04-25T14:00:37.174 INFO:teuthology.orchestra.run.smithi151.stdout:(12/152): cephfs-mirror-19.0.0-3291.g994e2f20.e 449 kB/s | 224 kB 00:00 2024-04-25T14:00:37.199 INFO:teuthology.orchestra.run.smithi151.stdout:(13/152): cephfs-shell-19.0.0-3291.g994e2f20.el 407 kB/s | 27 kB 00:00 2024-04-25T14:00:37.214 INFO:teuthology.orchestra.run.smithi096.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 1.3 MB/s | 272 kB 00:00 2024-04-25T14:00:37.231 INFO:teuthology.orchestra.run.smithi096.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 89 kB/s | 16 kB 00:00 2024-04-25T14:00:37.258 INFO:teuthology.orchestra.run.smithi151.stdout:(14/152): libcephfs-devel-19.0.0-3291.g994e2f20 378 kB/s | 32 kB 00:00 2024-04-25T14:00:37.298 INFO:teuthology.orchestra.run.smithi096.stdout:(135/152): libarrow-9.0.0-10.el9.x86_64.rpm 1.6 MB/s | 4.4 MB 00:02 2024-04-25T14:00:37.323 INFO:teuthology.orchestra.run.smithi096.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 827 kB/s | 90 kB 00:00 2024-04-25T14:00:37.325 INFO:teuthology.orchestra.run.smithi151.stdout:(15/152): libcephfs2-19.0.0-3291.g994e2f20.el9. 5.9 MB/s | 756 kB 00:00 2024-04-25T14:00:37.338 INFO:teuthology.orchestra.run.smithi097.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 814 kB/s | 244 kB 00:00 2024-04-25T14:00:37.340 INFO:teuthology.orchestra.run.smithi096.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 259 kB/s | 28 kB 00:00 2024-04-25T14:00:37.374 INFO:teuthology.orchestra.run.smithi096.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-04-25T14:00:37.375 INFO:teuthology.orchestra.run.smithi151.stdout:(16/152): librados-devel-19.0.0-3291.g994e2f20. 2.5 MB/s | 127 kB 00:00 2024-04-25T14:00:37.408 INFO:teuthology.orchestra.run.smithi096.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-04-25T14:00:37.409 INFO:teuthology.orchestra.run.smithi151.stdout:(17/152): libcephsqlite-19.0.0-3291.g994e2f20.e 1.0 MB/s | 158 kB 00:00 2024-04-25T14:00:37.433 INFO:teuthology.orchestra.run.smithi096.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 640 kB/s | 59 kB 00:00 2024-04-25T14:00:37.450 INFO:teuthology.orchestra.run.smithi096.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 473 kB/s | 36 kB 00:00 2024-04-25T14:00:37.459 INFO:teuthology.orchestra.run.smithi151.stdout:(18/152): libradosstriper1-19.0.0-3291.g994e2f2 5.8 MB/s | 497 kB 00:00 2024-04-25T14:00:37.463 INFO:teuthology.orchestra.run.smithi097.stdout:(75/152): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 6.0 MB 00:00 2024-04-25T14:00:37.489 INFO:teuthology.orchestra.run.smithi097.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.6 MB/s | 248 kB 00:00 2024-04-25T14:00:37.492 INFO:teuthology.orchestra.run.smithi096.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 608 kB/s | 51 kB 00:00 2024-04-25T14:00:37.509 INFO:teuthology.orchestra.run.smithi151.stdout:(19/152): python3-ceph-argparse-19.0.0-3291.g99 909 kB/s | 45 kB 00:00 2024-04-25T14:00:37.522 INFO:teuthology.orchestra.run.smithi097.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 441 kB/s | 26 kB 00:00 2024-04-25T14:00:37.525 INFO:teuthology.orchestra.run.smithi096.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-04-25T14:00:37.548 INFO:teuthology.orchestra.run.smithi097.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 328 kB/s | 19 kB 00:00 2024-04-25T14:00:37.550 INFO:teuthology.orchestra.run.smithi096.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 893 kB/s | 90 kB 00:00 2024-04-25T14:00:37.559 INFO:teuthology.orchestra.run.smithi151.stdout:(20/152): python3-ceph-common-19.0.0-3291.g994e 2.6 MB/s | 135 kB 00:00 2024-04-25T14:00:37.573 INFO:teuthology.orchestra.run.smithi097.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-04-25T14:00:37.598 INFO:teuthology.orchestra.run.smithi097.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 695 kB/s | 35 kB 00:00 2024-04-25T14:00:37.609 INFO:teuthology.orchestra.run.smithi096.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 239 kB/s | 20 kB 00:00 2024-04-25T14:00:37.610 INFO:teuthology.orchestra.run.smithi151.stdout:(21/152): python3-cephfs-19.0.0-3291.g994e2f20. 3.2 MB/s | 161 kB 00:00 2024-04-25T14:00:37.643 INFO:teuthology.orchestra.run.smithi096.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 2.8 MB/s | 427 kB 00:00 2024-04-25T14:00:37.660 INFO:teuthology.orchestra.run.smithi151.stdout:(22/152): python3-rados-19.0.0-3291.g994e2f20.e 6.3 MB/s | 321 kB 00:00 2024-04-25T14:00:37.668 INFO:teuthology.orchestra.run.smithi096.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.6 MB/s | 192 kB 00:00 2024-04-25T14:00:37.710 INFO:teuthology.orchestra.run.smithi151.stdout:(23/152): python3-rbd-19.0.0-3291.g994e2f20.el9 5.8 MB/s | 298 kB 00:00 2024-04-25T14:00:37.752 INFO:teuthology.orchestra.run.smithi151.stdout:(24/152): python3-rgw-19.0.0-3291.g994e2f20.el9 2.3 MB/s | 100 kB 00:00 2024-04-25T14:00:37.802 INFO:teuthology.orchestra.run.smithi151.stdout:(25/152): rbd-fuse-19.0.0-3291.g994e2f20.el9.x8 1.7 MB/s | 86 kB 00:00 2024-04-25T14:00:37.907 INFO:teuthology.orchestra.run.smithi097.stdout:(81/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 15 MB/s | 5.1 MB 00:00 2024-04-25T14:00:37.932 INFO:teuthology.orchestra.run.smithi097.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 752 kB/s | 250 kB 00:00 2024-04-25T14:00:37.974 INFO:teuthology.orchestra.run.smithi097.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.3 MB/s | 222 kB 00:00 2024-04-25T14:00:37.999 INFO:teuthology.orchestra.run.smithi097.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-04-25T14:00:38.033 INFO:teuthology.orchestra.run.smithi097.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 726 kB/s | 42 kB 00:00 2024-04-25T14:00:38.067 INFO:teuthology.orchestra.run.smithi097.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 3.9 MB/s | 269 kB 00:00 2024-04-25T14:00:38.100 INFO:teuthology.orchestra.run.smithi097.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-04-25T14:00:38.111 INFO:teuthology.orchestra.run.smithi151.stdout:(26/152): rbd-mirror-19.0.0-3291.g994e2f20.el9. 9.9 MB/s | 3.0 MB 00:00 2024-04-25T14:00:38.127 INFO:teuthology.orchestra.run.smithi096.stdout:(148/152): librbd1-19.0.0-3291.g994e2f20.el9.x8 6.7 MB/s | 3.1 MB 00:00 2024-04-25T14:00:38.142 INFO:teuthology.orchestra.run.smithi097.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-04-25T14:00:38.160 INFO:teuthology.orchestra.run.smithi096.stdout:(149/152): thrift-0.15.0-2.el9.x86_64.rpm 2.9 MB/s | 1.6 MB 00:00 2024-04-25T14:00:38.161 INFO:teuthology.orchestra.run.smithi151.stdout:(27/152): rbd-nbd-19.0.0-3291.g994e2f20.el9.x86 3.3 MB/s | 169 kB 00:00 2024-04-25T14:00:38.167 INFO:teuthology.orchestra.run.smithi097.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 761 kB/s | 51 kB 00:00 2024-04-25T14:00:38.203 INFO:teuthology.orchestra.run.smithi151.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3291.g 825 kB/s | 34 kB 00:00 2024-04-25T14:00:38.209 INFO:teuthology.orchestra.run.smithi097.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 806 kB/s | 54 kB 00:00 2024-04-25T14:00:38.236 INFO:teuthology.orchestra.run.smithi096.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 12 MB/s | 1.3 MB 00:00 2024-04-25T14:00:38.254 INFO:teuthology.orchestra.run.smithi151.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3291.g994e2f2 2.8 MB/s | 142 kB 00:00 2024-04-25T14:00:38.369 INFO:teuthology.orchestra.run.smithi096.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 11 MB/s | 2.4 MB 00:00 2024-04-25T14:00:38.476 INFO:teuthology.orchestra.run.smithi097.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 156 kB/s | 42 kB 00:00 2024-04-25T14:00:38.546 INFO:teuthology.orchestra.run.smithi151.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3291.g994e2 12 MB/s | 3.5 MB 00:00 2024-04-25T14:00:38.801 INFO:teuthology.orchestra.run.smithi097.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 136 kB/s | 44 kB 00:00 2024-04-25T14:00:38.830 INFO:teuthology.orchestra.run.smithi151.stdout:(31/152): ceph-common-19.0.0-3291.g994e2f20.el9 1.5 MB/s | 20 MB 00:13 2024-04-25T14:00:39.022 INFO:teuthology.orchestra.run.smithi151.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3291.g99 1.3 MB/s | 249 kB 00:00 2024-04-25T14:00:39.085 INFO:teuthology.orchestra.run.smithi097.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 782 kB/s | 221 kB 00:00 2024-04-25T14:00:39.097 INFO:teuthology.orchestra.run.smithi151.stdout:(33/152): ceph-mgr-rook-19.0.0-3291.g994e2f20.e 661 kB/s | 49 kB 00:00 2024-04-25T14:00:39.231 INFO:teuthology.orchestra.run.smithi151.stdout:(34/152): ceph-mgr-diskprediction-local-19.0.0- 11 MB/s | 7.4 MB 00:00 2024-04-25T14:00:39.244 INFO:teuthology.orchestra.run.smithi097.stdout:(94/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 15 MB/s | 16 MB 00:01 2024-04-25T14:00:39.247 INFO:teuthology.orchestra.run.smithi151.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3291.g9 109 kB/s | 16 kB 00:00 2024-04-25T14:00:39.269 INFO:teuthology.orchestra.run.smithi097.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 197 kB/s | 36 kB 00:00 2024-04-25T14:00:39.281 INFO:teuthology.orchestra.run.smithi151.stdout:(36/152): ceph-volume-19.0.0-3291.g994e2f20.el9 5.3 MB/s | 273 kB 00:00 2024-04-25T14:00:39.294 INFO:teuthology.orchestra.run.smithi097.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 954 kB/s | 47 kB 00:00 2024-04-25T14:00:39.328 INFO:teuthology.orchestra.run.smithi097.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.1 MB/s | 305 kB 00:00 2024-04-25T14:00:39.348 INFO:teuthology.orchestra.run.smithi151.stdout:(37/152): librgw2-19.0.0-3291.g994e2f20.el9.x86 2.7 MB/s | 5.2 MB 00:01 2024-04-25T14:00:39.353 INFO:teuthology.orchestra.run.smithi097.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 994 kB/s | 58 kB 00:00 2024-04-25T14:00:39.365 INFO:teuthology.orchestra.run.smithi151.stdout:(38/152): cephfs-top-19.0.0-3291.g994e2f20.el9. 218 kB/s | 18 kB 00:00 2024-04-25T14:00:39.387 INFO:teuthology.orchestra.run.smithi097.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-25T14:00:39.412 INFO:teuthology.orchestra.run.smithi097.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-25T14:00:39.437 INFO:teuthology.orchestra.run.smithi097.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 898 kB/s | 45 kB 00:00 2024-04-25T14:00:39.463 INFO:teuthology.orchestra.run.smithi097.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 446 kB/s | 22 kB 00:00 2024-04-25T14:00:39.530 INFO:teuthology.orchestra.run.smithi097.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 9.2 MB/s | 863 kB 00:00 2024-04-25T14:00:39.549 INFO:teuthology.orchestra.run.smithi151.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 401 kB/s | 80 kB 00:00 2024-04-25T14:00:39.632 INFO:teuthology.orchestra.run.smithi151.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-04-25T14:00:39.658 INFO:teuthology.orchestra.run.smithi151.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.7 MB/s | 803 kB 00:00 2024-04-25T14:00:39.683 INFO:teuthology.orchestra.run.smithi151.stdout:(42/152): cephadm-19.0.0-3291.g994e2f20.el9.noa 1.7 MB/s | 762 kB 00:00 2024-04-25T14:00:39.708 INFO:teuthology.orchestra.run.smithi151.stdout:(43/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.3 MB/s | 253 kB 00:00 2024-04-25T14:00:39.767 INFO:teuthology.orchestra.run.smithi151.stdout:(44/152): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.2 MB 00:00 2024-04-25T14:00:39.792 INFO:teuthology.orchestra.run.smithi151.stdout:(45/152): python3-ply-3.11-14.el9.noarch.rpm 984 kB/s | 106 kB 00:00 2024-04-25T14:00:39.817 INFO:teuthology.orchestra.run.smithi151.stdout:(46/152): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-04-25T14:00:39.843 INFO:teuthology.orchestra.run.smithi151.stdout:(47/152): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-04-25T14:00:39.861 INFO:teuthology.orchestra.run.smithi096.stdout:(152/152): librados2-19.0.0-3291.g994e2f20.el9. 1.5 MB/s | 3.3 MB 00:02 2024-04-25T14:00:39.865 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-04-25T14:00:39.865 INFO:teuthology.orchestra.run.smithi096.stdout:Total 12 MB/s | 214 MB 00:17 2024-04-25T14:00:39.868 INFO:teuthology.orchestra.run.smithi151.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-25T14:00:39.893 INFO:teuthology.orchestra.run.smithi151.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 9.0 MB/s | 689 kB 00:00 2024-04-25T14:00:39.927 INFO:teuthology.orchestra.run.smithi151.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 11 MB/s | 943 kB 00:00 2024-04-25T14:00:39.952 INFO:teuthology.orchestra.run.smithi151.stdout:(51/152): boost-program-options-1.75.0-8.el9.x8 1.3 MB/s | 107 kB 00:00 2024-04-25T14:00:39.977 INFO:teuthology.orchestra.run.smithi151.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 261 kB/s | 21 kB 00:00 2024-04-25T14:00:40.002 INFO:teuthology.orchestra.run.smithi151.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-25T14:00:40.028 INFO:teuthology.orchestra.run.smithi151.stdout:(54/152): flexiblas-3.0.4-8.el9.x86_64.rpm 435 kB/s | 33 kB 00:00 2024-04-25T14:00:40.097 INFO:teuthology.orchestra.run.smithi097.stdout:(104/152): ceph-radosgw-19.0.0-3291.g994e2f20.e 971 kB/s | 10 MB 00:10 2024-04-25T14:00:40.178 INFO:teuthology.orchestra.run.smithi151.stdout:(55/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 15 MB/s | 3.0 MB 00:00 2024-04-25T14:00:40.180 INFO:teuthology.orchestra.run.smithi097.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 764 kB/s | 548 kB 00:00 2024-04-25T14:00:40.195 INFO:teuthology.orchestra.run.smithi151.stdout:(56/152): flexiblas-openblas-openmp-3.0.4-8.el9 110 kB/s | 18 kB 00:00 2024-04-25T14:00:40.211 INFO:teuthology.orchestra.run.smithi151.stdout:(57/152): libaio-devel-0.3.111-13.el9.x86_64.rp 61 kB/s | 11 kB 00:00 2024-04-25T14:00:40.230 INFO:teuthology.orchestra.run.smithi097.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 441 kB/s | 309 kB 00:00 2024-04-25T14:00:40.254 INFO:teuthology.orchestra.run.smithi151.stdout:(58/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.8 MB/s | 160 kB 00:00 2024-04-25T14:00:40.279 INFO:teuthology.orchestra.run.smithi151.stdout:(59/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 679 kB/s | 45 kB 00:00 2024-04-25T14:00:40.312 INFO:teuthology.orchestra.run.smithi151.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 6.5 MB/s | 662 kB 00:00 2024-04-25T14:00:40.338 INFO:teuthology.orchestra.run.smithi151.stdout:(61/152): libtool-2.4.6-45.el9.x86_64.rpm 6.9 MB/s | 582 kB 00:00 2024-04-25T14:00:40.339 INFO:teuthology.orchestra.run.smithi097.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 239 kB/s | 26 kB 00:00 2024-04-25T14:00:40.354 INFO:teuthology.orchestra.run.smithi151.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-04-25T14:00:40.371 INFO:teuthology.orchestra.run.smithi151.stdout:(63/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 440 kB/s | 25 kB 00:00 2024-04-25T14:00:40.397 INFO:teuthology.orchestra.run.smithi151.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-04-25T14:00:40.422 INFO:teuthology.orchestra.run.smithi151.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 188 kB 00:00 2024-04-25T14:00:40.439 INFO:teuthology.orchestra.run.smithi151.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 528 kB/s | 35 kB 00:00 2024-04-25T14:00:40.448 INFO:teuthology.orchestra.run.smithi097.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 449 kB/s | 49 kB 00:00 2024-04-25T14:00:40.473 INFO:teuthology.orchestra.run.smithi097.stdout:(109/152): grpc-data-1.46.7-10.el9.noarch.rpm 52 kB/s | 19 kB 00:00 2024-04-25T14:00:40.557 INFO:teuthology.orchestra.run.smithi097.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 621 kB/s | 67 kB 00:00 2024-04-25T14:00:40.589 INFO:teuthology.orchestra.run.smithi151.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 25 MB/s | 4.7 MB 00:00 2024-04-25T14:00:40.615 INFO:teuthology.orchestra.run.smithi097.stdout:(111/152): libarrow-9.0.0-10.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-04-25T14:00:40.648 INFO:teuthology.orchestra.run.smithi151.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 7.7 MB/s | 1.6 MB 00:00 2024-04-25T14:00:40.715 INFO:teuthology.orchestra.run.smithi151.stdout:(69/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-04-25T14:00:40.724 INFO:teuthology.orchestra.run.smithi097.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 4.9 MB/s | 548 kB 00:00 2024-04-25T14:00:40.748 INFO:teuthology.orchestra.run.smithi151.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 5.7 MB/s | 911 kB 00:00 2024-04-25T14:00:40.773 INFO:teuthology.orchestra.run.smithi151.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 1.1 MB/s | 133 kB 00:00 2024-04-25T14:00:40.774 INFO:teuthology.orchestra.run.smithi097.stdout:(113/152): luarocks-3.9.2-1.el9.noarch.rpm 503 kB/s | 151 kB 00:00 2024-04-25T14:00:40.816 INFO:teuthology.orchestra.run.smithi097.stdout:(114/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.2 MB/s | 838 kB 00:00 2024-04-25T14:00:40.833 INFO:teuthology.orchestra.run.smithi097.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 400 kB/s | 43 kB 00:00 2024-04-25T14:00:40.875 INFO:teuthology.orchestra.run.smithi097.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 320 kB/s | 32 kB 00:00 2024-04-25T14:00:40.908 INFO:teuthology.orchestra.run.smithi097.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 163 kB/s | 15 kB 00:00 2024-04-25T14:00:40.924 INFO:teuthology.orchestra.run.smithi151.stdout:(72/152): postgresql-server-13.14-1.el9.x86_64. 28 MB/s | 5.8 MB 00:00 2024-04-25T14:00:40.933 INFO:teuthology.orchestra.run.smithi097.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-04-25T14:00:40.957 INFO:teuthology.orchestra.run.smithi151.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 5.6 MB/s | 1.0 MB 00:00 2024-04-25T14:00:40.982 INFO:teuthology.orchestra.run.smithi151.stdout:(74/152): python3-attrs-20.3.0-7.el9.noarch.rpm 424 kB/s | 87 kB 00:00 2024-04-25T14:00:40.983 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-04-25T14:00:41.026 INFO:teuthology.orchestra.run.smithi097.stdout:(119/152): python3-colorama-0.4.4-12.el9.noarch 385 kB/s | 35 kB 00:00 2024-04-25T14:00:41.051 INFO:teuthology.orchestra.run.smithi097.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.1 MB/s | 303 kB 00:00 2024-04-25T14:00:41.101 INFO:teuthology.orchestra.run.smithi097.stdout:(121/152): python3-cherrypy-18.6.1-2.el9.noarch 1.5 MB/s | 358 kB 00:00 2024-04-25T14:00:41.122 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-04-25T14:00:41.123 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-04-25T14:00:41.143 INFO:teuthology.orchestra.run.smithi097.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 1.7 MB/s | 206 kB 00:00 2024-04-25T14:00:41.183 INFO:teuthology.orchestra.run.smithi151.stdout:(75/152): python3-babel-2.9.1-2.el9.noarch.rpm 23 MB/s | 6.0 MB 00:00 2024-04-25T14:00:41.208 INFO:teuthology.orchestra.run.smithi151.stdout:(76/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.1 MB/s | 244 kB 00:00 2024-04-25T14:00:41.218 INFO:teuthology.orchestra.run.smithi097.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.2 MB/s | 144 kB 00:00 2024-04-25T14:00:41.233 INFO:teuthology.orchestra.run.smithi151.stdout:(77/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.0 MB/s | 248 kB 00:00 2024-04-25T14:00:41.243 INFO:teuthology.orchestra.run.smithi097.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 106 kB/s | 11 kB 00:00 2024-04-25T14:00:41.258 INFO:teuthology.orchestra.run.smithi151.stdout:(78/152): python3-jsonpatch-1.21-16.el9.noarch. 344 kB/s | 26 kB 00:00 2024-04-25T14:00:41.284 INFO:teuthology.orchestra.run.smithi151.stdout:(79/152): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-04-25T14:00:41.294 INFO:teuthology.orchestra.run.smithi097.stdout:(125/152): python3-grpcio-1.46.7-10.el9.x86_64. 8.4 MB/s | 2.0 MB 00:00 2024-04-25T14:00:41.309 INFO:teuthology.orchestra.run.smithi151.stdout:(80/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-04-25T14:00:41.319 INFO:teuthology.orchestra.run.smithi097.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-04-25T14:00:41.334 INFO:teuthology.orchestra.run.smithi151.stdout:(81/152): python3-markupsafe-1.1.1-12.el9.x86_6 464 kB/s | 35 kB 00:00 2024-04-25T14:00:41.344 INFO:teuthology.orchestra.run.smithi097.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 231 kB/s | 23 kB 00:00 2024-04-25T14:00:41.394 INFO:teuthology.orchestra.run.smithi097.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 194 kB/s | 19 kB 00:00 2024-04-25T14:00:41.401 INFO:teuthology.orchestra.run.smithi151.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-04-25T14:00:41.428 INFO:teuthology.orchestra.run.smithi097.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 180 kB/s | 20 kB 00:00 2024-04-25T14:00:41.479 INFO:teuthology.orchestra.run.smithi097.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 7.6 MB/s | 1.0 MB 00:00 2024-04-25T14:00:41.504 INFO:teuthology.orchestra.run.smithi097.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 425 kB/s | 46 kB 00:00 2024-04-25T14:00:41.510 INFO:teuthology.orchestra.run.smithi151.stdout:(83/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 25 MB/s | 5.1 MB 00:00 2024-04-25T14:00:41.535 INFO:teuthology.orchestra.run.smithi151.stdout:(84/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.1 MB/s | 222 kB 00:00 2024-04-25T14:00:41.537 INFO:teuthology.orchestra.run.smithi097.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 725 kB/s | 79 kB 00:00 2024-04-25T14:00:41.560 INFO:teuthology.orchestra.run.smithi151.stdout:(85/152): python3-packaging-20.9-5.el9.noarch.r 493 kB/s | 77 kB 00:00 2024-04-25T14:00:41.579 INFO:teuthology.orchestra.run.smithi097.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 575 kB/s | 58 kB 00:00 2024-04-25T14:00:41.585 INFO:teuthology.orchestra.run.smithi151.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 565 kB/s | 42 kB 00:00 2024-04-25T14:00:41.611 INFO:teuthology.orchestra.run.smithi151.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-04-25T14:00:41.613 INFO:teuthology.orchestra.run.smithi097.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-04-25T14:00:41.636 INFO:teuthology.orchestra.run.smithi151.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-04-25T14:00:41.646 INFO:teuthology.orchestra.run.smithi097.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 151 kB/s | 16 kB 00:00 2024-04-25T14:00:41.661 INFO:teuthology.orchestra.run.smithi151.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-04-25T14:00:41.680 INFO:teuthology.orchestra.run.smithi097.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 894 kB/s | 90 kB 00:00 2024-04-25T14:00:41.686 INFO:teuthology.orchestra.run.smithi151.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-25T14:00:41.714 INFO:teuthology.orchestra.run.smithi151.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 719 kB/s | 54 kB 00:00 2024-04-25T14:00:41.714 INFO:teuthology.orchestra.run.smithi097.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 281 kB/s | 28 kB 00:00 2024-04-25T14:00:41.747 INFO:teuthology.orchestra.run.smithi097.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 306 kB/s | 31 kB 00:00 2024-04-25T14:00:41.781 INFO:teuthology.orchestra.run.smithi097.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-04-25T14:00:41.814 INFO:teuthology.orchestra.run.smithi097.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 588 kB/s | 59 kB 00:00 2024-04-25T14:00:41.848 INFO:teuthology.orchestra.run.smithi097.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 356 kB/s | 36 kB 00:00 2024-04-25T14:00:41.882 INFO:teuthology.orchestra.run.smithi097.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 507 kB/s | 51 kB 00:00 2024-04-25T14:00:41.918 INFO:teuthology.orchestra.run.smithi097.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-25T14:00:41.957 INFO:teuthology.orchestra.run.smithi097.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 825 kB/s | 90 kB 00:00 2024-04-25T14:00:41.991 INFO:teuthology.orchestra.run.smithi097.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-04-25T14:00:42.016 INFO:teuthology.orchestra.run.smithi097.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 199 kB/s | 20 kB 00:00 2024-04-25T14:00:42.020 INFO:teuthology.orchestra.run.smithi151.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 136 kB/s | 42 kB 00:00 2024-04-25T14:00:42.045 INFO:teuthology.orchestra.run.smithi151.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 134 kB/s | 44 kB 00:00 2024-04-25T14:00:42.100 INFO:teuthology.orchestra.run.smithi097.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.3 MB/s | 192 kB 00:00 2024-04-25T14:00:42.141 INFO:teuthology.orchestra.run.smithi097.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-25T14:00:42.204 INFO:teuthology.orchestra.run.smithi151.stdout:(94/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 30 MB/s | 16 MB 00:00 2024-04-25T14:00:42.229 INFO:teuthology.orchestra.run.smithi151.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 1.0 MB/s | 221 kB 00:00 2024-04-25T14:00:42.255 INFO:teuthology.orchestra.run.smithi151.stdout:(96/152): qatlib-service-24.02.0-1.el9.x86_64.r 173 kB/s | 36 kB 00:00 2024-04-25T14:00:42.267 INFO:teuthology.orchestra.run.smithi097.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 11 MB/s | 1.3 MB 00:00 2024-04-25T14:00:42.280 INFO:teuthology.orchestra.run.smithi151.stdout:(97/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 949 kB/s | 47 kB 00:00 2024-04-25T14:00:42.305 INFO:teuthology.orchestra.run.smithi151.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.0 MB/s | 305 kB 00:00 2024-04-25T14:00:42.330 INFO:teuthology.orchestra.run.smithi151.stdout:(99/152): uuid-1.6.2-55.el9.x86_64.rpm 774 kB/s | 58 kB 00:00 2024-04-25T14:00:42.355 INFO:teuthology.orchestra.run.smithi151.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-25T14:00:42.381 INFO:teuthology.orchestra.run.smithi151.stdout:(101/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.2 MB/s | 94 kB 00:00 2024-04-25T14:00:42.406 INFO:teuthology.orchestra.run.smithi151.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 595 kB/s | 45 kB 00:00 2024-04-25T14:00:42.431 INFO:teuthology.orchestra.run.smithi151.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 296 kB/s | 22 kB 00:00 2024-04-25T14:00:42.434 INFO:teuthology.orchestra.run.smithi097.stdout:(150/152): librbd1-19.0.0-3291.g994e2f20.el9.x8 9.3 MB/s | 3.1 MB 00:00 2024-04-25T14:00:42.465 INFO:teuthology.orchestra.run.smithi151.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 14 MB/s | 863 kB 00:00 2024-04-25T14:00:42.484 INFO:teuthology.orchestra.run.smithi097.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 11 MB/s | 2.4 MB 00:00 2024-04-25T14:00:42.582 INFO:teuthology.orchestra.run.smithi151.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 167 kB/s | 19 kB 00:00 2024-04-25T14:00:42.707 INFO:teuthology.orchestra.run.smithi151.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.1 MB/s | 309 kB 00:00 2024-04-25T14:00:42.734 INFO:teuthology.orchestra.run.smithi151.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.6 MB/s | 548 kB 00:00 2024-04-25T14:00:42.758 INFO:teuthology.orchestra.run.smithi151.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 514 kB/s | 26 kB 00:00 2024-04-25T14:00:42.783 INFO:teuthology.orchestra.run.smithi151.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 963 kB/s | 49 kB 00:00 2024-04-25T14:00:42.825 INFO:teuthology.orchestra.run.smithi151.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-04-25T14:00:42.839 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-04-25T14:00:42.839 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-04-25T14:00:42.842 INFO:teuthology.orchestra.run.smithi151.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-04-25T14:00:42.943 INFO:teuthology.orchestra.run.smithi151.stdout:(112/152): libarrow-9.0.0-10.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-04-25T14:00:43.006 INFO:teuthology.orchestra.run.smithi151.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 3.3 MB/s | 548 kB 00:00 2024-04-25T14:00:43.060 INFO:teuthology.orchestra.run.smithi151.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 371 kB/s | 43 kB 00:00 2024-04-25T14:00:43.085 INFO:teuthology.orchestra.run.smithi151.stdout:(115/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.2 MB/s | 838 kB 00:00 2024-04-25T14:00:43.110 INFO:teuthology.orchestra.run.smithi151.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 311 kB/s | 32 kB 00:00 2024-04-25T14:00:43.136 INFO:teuthology.orchestra.run.smithi151.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 198 kB/s | 15 kB 00:00 2024-04-25T14:00:43.161 INFO:teuthology.orchestra.run.smithi151.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-04-25T14:00:43.186 INFO:teuthology.orchestra.run.smithi151.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 4.6 MB/s | 358 kB 00:00 2024-04-25T14:00:43.219 INFO:teuthology.orchestra.run.smithi151.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-04-25T14:00:43.245 INFO:teuthology.orchestra.run.smithi151.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 421 kB/s | 35 kB 00:00 2024-04-25T14:00:43.270 INFO:teuthology.orchestra.run.smithi151.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.4 MB/s | 206 kB 00:00 2024-04-25T14:00:43.312 INFO:teuthology.orchestra.run.smithi151.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-04-25T14:00:43.337 INFO:teuthology.orchestra.run.smithi151.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 160 kB/s | 11 kB 00:00 2024-04-25T14:00:43.379 INFO:teuthology.orchestra.run.smithi151.stdout:(125/152): python3-grpcio-1.46.7-10.el9.x86_64. 13 MB/s | 2.0 MB 00:00 2024-04-25T14:00:43.405 INFO:teuthology.orchestra.run.smithi151.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2024-04-25T14:00:43.429 INFO:teuthology.orchestra.run.smithi151.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 251 kB/s | 23 kB 00:00 2024-04-25T14:00:43.455 INFO:teuthology.orchestra.run.smithi151.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 257 kB/s | 19 kB 00:00 2024-04-25T14:00:43.480 INFO:teuthology.orchestra.run.smithi151.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 394 kB/s | 20 kB 00:00 2024-04-25T14:00:43.505 INFO:teuthology.orchestra.run.smithi151.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 924 kB/s | 46 kB 00:00 2024-04-25T14:00:43.539 INFO:teuthology.orchestra.run.smithi151.stdout:(131/152): python3-kubernetes-26.1.0-2.el9.noar 9.4 MB/s | 1.0 MB 00:00 2024-04-25T14:00:43.564 INFO:teuthology.orchestra.run.smithi151.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 989 kB/s | 58 kB 00:00 2024-04-25T14:00:43.589 INFO:teuthology.orchestra.run.smithi151.stdout:(133/152): python3-more-itertools-8.12.0-2.el9. 722 kB/s | 79 kB 00:00 2024-04-25T14:00:43.615 INFO:teuthology.orchestra.run.smithi151.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-04-25T14:00:43.634 INFO:teuthology.orchestra.run.smithi097.stdout:(152/152): librados2-19.0.0-3291.g994e2f20.el9. 2.1 MB/s | 3.3 MB 00:01 2024-04-25T14:00:43.640 INFO:teuthology.orchestra.run.smithi151.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 217 kB/s | 16 kB 00:00 2024-04-25T14:00:43.641 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-04-25T14:00:43.641 INFO:teuthology.orchestra.run.smithi097.stdout:Total 11 MB/s | 214 MB 00:19 2024-04-25T14:00:43.665 INFO:teuthology.orchestra.run.smithi151.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-25T14:00:43.690 INFO:teuthology.orchestra.run.smithi151.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 374 kB/s | 28 kB 00:00 2024-04-25T14:00:43.725 INFO:teuthology.orchestra.run.smithi151.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 365 kB/s | 31 kB 00:00 2024-04-25T14:00:43.750 INFO:teuthology.orchestra.run.smithi151.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-04-25T14:00:43.775 INFO:teuthology.orchestra.run.smithi151.stdout:(140/152): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-04-25T14:00:43.800 INFO:teuthology.orchestra.run.smithi151.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 475 kB/s | 36 kB 00:00 2024-04-25T14:00:43.826 INFO:teuthology.orchestra.run.smithi151.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 676 kB/s | 51 kB 00:00 2024-04-25T14:00:43.851 INFO:teuthology.orchestra.run.smithi151.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-25T14:00:43.876 INFO:teuthology.orchestra.run.smithi151.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-25T14:00:43.910 INFO:teuthology.orchestra.run.smithi151.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-04-25T14:00:43.935 INFO:teuthology.orchestra.run.smithi151.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 238 kB/s | 20 kB 00:00 2024-04-25T14:00:43.962 INFO:teuthology.orchestra.run.smithi151.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-04-25T14:00:44.035 INFO:teuthology.orchestra.run.smithi151.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-04-25T14:00:44.127 INFO:teuthology.orchestra.run.smithi151.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 15 MB/s | 1.3 MB 00:00 2024-04-25T14:00:44.253 INFO:teuthology.orchestra.run.smithi151.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 19 MB/s | 2.4 MB 00:00 2024-04-25T14:00:44.336 INFO:teuthology.orchestra.run.smithi151.stdout:(151/152): librbd1-19.0.0-3291.g994e2f20.el9.x8 8.2 MB/s | 3.1 MB 00:00 2024-04-25T14:00:44.750 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-04-25T14:00:44.895 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-04-25T14:00:44.895 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-04-25T14:00:45.061 INFO:teuthology.orchestra.run.smithi151.stdout:(152/152): librados2-19.0.0-3291.g994e2f20.el9. 3.0 MB/s | 3.3 MB 00:01 2024-04-25T14:00:45.067 INFO:teuthology.orchestra.run.smithi151.stdout:-------------------------------------------------------------------------------- 2024-04-25T14:00:45.067 INFO:teuthology.orchestra.run.smithi151.stdout:Total 11 MB/s | 214 MB 00:20 2024-04-25T14:00:46.203 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction check 2024-04-25T14:00:46.347 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction check succeeded. 2024-04-25T14:00:46.348 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction test 2024-04-25T14:00:46.677 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-04-25T14:00:46.678 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-04-25T14:00:48.072 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction test succeeded. 2024-04-25T14:00:48.073 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction 2024-04-25T14:00:48.750 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-04-25T14:00:48.872 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T14:00:49.078 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T14:00:49.132 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T14:00:49.181 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T14:00:49.207 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T14:00:49.275 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T14:00:49.616 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T14:00:49.741 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T14:00:49.865 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T14:00:49.900 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T14:00:49.974 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T14:00:50.013 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T14:00:50.158 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T14:00:50.218 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T14:00:50.246 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T14:00:50.350 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T14:00:50.372 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T14:00:50.434 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T14:00:50.456 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T14:00:50.478 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T14:00:50.649 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T14:00:50.681 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T14:00:50.793 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T14:00:50.887 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T14:00:50.922 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T14:00:50.957 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T14:00:51.031 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T14:00:51.064 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T14:00:51.108 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T14:00:51.140 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T14:00:51.176 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T14:00:51.246 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T14:00:51.299 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T14:00:51.329 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T14:00:51.373 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T14:00:51.450 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T14:00:51.492 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 31/156 2024-04-25T14:00:51.527 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 32/156 2024-04-25T14:00:51.555 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 33/156 2024-04-25T14:00:51.626 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T14:00:51.670 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T14:00:51.744 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T14:00:51.879 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T14:00:51.931 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T14:00:52.462 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T14:00:52.551 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T14:00:52.580 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T14:00:52.581 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-04-25T14:00:52.623 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T14:00:52.661 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 43/156 2024-04-25T14:00:52.671 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T14:00:52.702 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T14:00:52.810 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T14:00:52.867 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T14:00:52.922 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T14:00:52.978 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T14:00:52.990 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T14:00:53.031 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T14:00:53.051 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T14:00:53.056 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T14:00:53.081 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T14:00:53.100 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T14:00:53.117 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T14:00:53.165 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T14:00:53.200 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T14:00:53.243 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T14:00:53.293 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T14:00:53.430 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T14:00:53.434 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T14:00:53.482 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T14:00:53.513 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T14:00:53.532 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T14:00:53.554 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T14:00:53.641 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T14:00:53.648 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T14:00:53.679 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T14:00:53.719 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T14:00:53.741 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T14:00:53.753 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T14:00:53.778 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T14:00:53.823 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T14:00:53.852 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T14:00:53.853 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T14:00:53.978 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T14:00:53.999 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T14:00:54.010 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T14:00:54.051 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T14:00:54.079 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T14:00:54.103 INFO:teuthology.orchestra.run.smithi151.stdout: Preparing : 1/1 2024-04-25T14:00:54.147 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T14:00:54.181 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T14:00:54.181 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T14:00:54.183 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T14:00:54.205 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T14:00:54.220 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T14:00:54.220 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T14:00:54.260 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T14:00:54.267 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T14:00:54.273 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T14:00:54.289 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T14:00:54.290 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T14:00:54.308 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T14:00:54.312 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T14:00:54.324 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T14:00:54.333 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T14:00:54.357 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T14:00:54.376 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T14:00:54.398 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T14:00:54.433 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T14:00:54.505 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T14:00:54.511 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T14:00:54.538 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T14:00:54.564 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T14:00:54.594 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T14:00:54.641 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T14:00:54.644 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T14:00:54.711 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T14:00:54.739 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T14:00:54.751 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T14:00:54.779 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T14:00:54.788 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T14:00:54.814 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T14:00:54.817 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T14:00:54.878 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T14:00:54.913 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T14:00:54.942 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T14:00:54.957 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T14:00:54.968 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T14:00:54.990 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T14:00:55.018 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T14:00:55.059 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T14:00:55.087 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T14:00:55.089 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T14:00:55.148 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T14:00:55.179 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T14:00:55.223 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T14:00:55.240 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T14:00:55.286 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T14:00:55.299 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T14:00:55.314 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T14:00:55.333 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 31/156 2024-04-25T14:00:55.377 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 32/156 2024-04-25T14:00:55.405 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 33/156 2024-04-25T14:00:55.411 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T14:00:55.440 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T14:00:55.475 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T14:00:55.486 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T14:00:55.498 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T14:00:55.508 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T14:00:55.527 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T14:00:55.528 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T14:00:55.543 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T14:00:55.602 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T14:00:55.603 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T14:00:55.691 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T14:00:55.702 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T14:00:55.724 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T14:00:55.737 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T14:00:55.751 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T14:00:55.789 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T14:00:55.833 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T14:00:55.838 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T14:00:55.879 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T14:00:55.926 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T14:00:55.965 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T14:00:56.017 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T14:00:56.055 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T14:00:56.085 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T14:00:56.091 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T14:00:56.132 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T14:00:56.177 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T14:00:56.179 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T14:00:56.209 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T14:00:56.245 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T14:00:56.305 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T14:00:56.315 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T14:00:56.368 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T14:00:56.383 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T14:00:56.398 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T14:00:56.414 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T14:00:56.442 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T14:00:56.456 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T14:00:56.494 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 43/156 2024-04-25T14:00:56.519 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T14:00:56.535 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T14:00:56.561 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 31/156 2024-04-25T14:00:56.604 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 32/156 2024-04-25T14:00:56.632 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 33/156 2024-04-25T14:00:56.688 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T14:00:56.702 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T14:00:56.739 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T14:00:56.747 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T14:00:56.794 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T14:00:56.819 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T14:00:56.822 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T14:00:56.866 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T14:00:56.897 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T14:00:56.933 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T14:00:56.940 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T14:00:56.966 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T14:00:56.980 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T14:00:57.016 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T14:00:57.025 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T14:00:57.059 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T14:00:57.102 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T14:00:57.238 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T14:00:57.307 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T14:00:57.337 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T14:00:57.378 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T14:00:57.472 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T14:00:57.505 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T14:00:57.544 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T14:00:57.545 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T14:00:57.569 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T14:00:57.594 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T14:00:57.629 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T14:00:57.657 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T14:00:57.668 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T14:00:57.700 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T14:00:57.738 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 43/156 2024-04-25T14:00:57.779 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T14:00:57.795 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T14:00:57.835 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T14:00:57.863 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T14:00:57.897 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T14:00:57.930 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T14:00:57.936 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T14:00:57.967 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T14:00:57.982 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T14:00:57.997 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T14:00:58.032 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T14:00:58.037 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T14:00:58.065 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T14:00:58.106 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T14:00:58.110 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T14:00:58.141 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T14:00:58.141 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T14:00:58.172 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T14:00:58.178 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T14:00:58.225 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T14:00:58.228 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T14:00:58.255 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T14:00:58.261 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T14:00:58.280 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T14:00:58.287 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T14:00:58.304 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T14:00:58.311 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T14:00:58.316 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T14:00:58.347 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T14:00:58.358 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T14:00:58.457 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T14:00:58.482 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T14:00:58.504 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T14:00:58.534 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T14:00:58.565 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T14:00:58.606 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T14:00:58.636 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T14:00:58.702 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T14:00:58.740 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T14:00:58.780 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T14:00:58.813 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T14:00:58.838 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T14:00:58.842 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T14:00:58.913 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T14:00:58.969 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T14:00:59.000 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 100/156 2024-04-25T14:00:59.036 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T14:00:59.048 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T14:00:59.079 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T14:00:59.109 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T14:00:59.141 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T14:00:59.189 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T14:00:59.216 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T14:00:59.220 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T14:00:59.250 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T14:00:59.259 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T14:00:59.284 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T14:00:59.293 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T14:00:59.318 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T14:00:59.392 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T14:00:59.443 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T14:00:59.443 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T14:00:59.486 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T14:00:59.526 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T14:00:59.529 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T14:00:59.573 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T14:00:59.573 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T14:00:59.579 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T14:00:59.624 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T14:00:59.644 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T14:00:59.655 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T14:00:59.666 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 104/156 2024-04-25T14:00:59.702 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T14:00:59.755 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T14:00:59.784 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T14:00:59.801 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T14:00:59.857 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T14:00:59.862 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T14:01:00.499 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T14:01:00.580 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T14:01:00.606 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T14:01:00.629 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T14:01:00.662 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T14:01:00.762 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T14:01:00.821 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T14:01:00.907 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T14:01:00.940 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T14:01:01.120 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T14:01:01.146 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T14:01:01.225 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T14:01:01.825 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T14:01:01.869 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T14:01:01.913 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T14:01:01.926 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T14:01:01.952 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T14:01:01.962 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T14:01:01.983 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T14:01:01.987 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T14:01:02.306 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T14:01:02.354 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T14:01:02.445 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T14:01:02.492 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 100/156 2024-04-25T14:01:02.527 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T14:01:02.528 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T14:01:02.566 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T14:01:02.662 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T14:01:02.662 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T14:01:02.663 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T14:01:02.663 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:01:02.709 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T14:01:03.024 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T14:01:03.065 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T14:01:03.120 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T14:01:03.142 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 104/156 2024-04-25T14:01:03.211 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T14:01:03.300 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T14:01:03.328 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T14:01:03.361 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T14:01:03.683 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T14:01:03.724 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T14:01:03.856 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T14:01:03.887 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 100/156 2024-04-25T14:01:03.924 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T14:01:04.413 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T14:01:04.444 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T14:01:04.515 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T14:01:04.537 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 104/156 2024-04-25T14:01:05.384 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T14:01:05.438 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T14:01:06.003 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T14:01:06.041 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T14:01:06.129 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T14:01:06.129 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T14:01:06.129 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T14:01:06.129 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T14:01:06.201 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T14:01:06.787 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T14:01:06.817 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T14:01:07.380 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T14:01:07.412 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T14:01:07.503 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T14:01:07.503 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T14:01:07.503 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T14:01:07.503 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-25T14:01:07.538 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T14:01:14.134 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T14:01:14.134 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /sys 2024-04-25T14:01:14.134 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /proc 2024-04-25T14:01:14.134 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /mnt 2024-04-25T14:01:14.134 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /var/tmp 2024-04-25T14:01:14.134 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /home 2024-04-25T14:01:14.134 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /root 2024-04-25T14:01:14.134 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /tmp 2024-04-25T14:01:14.134 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:01:14.320 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T14:01:14.337 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T14:01:14.337 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:01:14.337 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T14:01:14.337 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:01:15.906 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T14:01:15.931 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T14:01:15.932 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:01:15.932 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T14:01:15.932 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T14:01:15.932 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T14:01:15.932 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:01:16.156 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T14:01:16.179 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T14:01:16.180 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:01:16.180 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T14:01:16.180 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T14:01:16.180 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T14:01:16.180 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:01:16.640 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T14:01:16.663 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T14:01:16.663 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:01:16.663 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T14:01:16.664 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T14:01:16.664 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T14:01:16.664 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:01:16.770 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T14:01:16.819 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T14:01:16.874 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T14:01:17.057 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T14:01:17.107 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T14:01:17.177 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T14:01:17.265 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T14:01:17.403 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T14:01:17.441 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T14:01:17.490 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T14:01:17.517 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T14:01:17.579 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T14:01:17.579 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /sys 2024-04-25T14:01:17.579 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /proc 2024-04-25T14:01:17.579 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /mnt 2024-04-25T14:01:17.579 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /var/tmp 2024-04-25T14:01:17.580 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /home 2024-04-25T14:01:17.580 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /root 2024-04-25T14:01:17.580 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /tmp 2024-04-25T14:01:17.580 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T14:01:17.754 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T14:01:17.770 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T14:01:17.770 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:01:17.770 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T14:01:17.770 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T14:01:17.895 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T14:01:17.924 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T14:01:17.948 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 124/156 2024-04-25T14:01:18.009 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 125/156 2024-04-25T14:01:18.039 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T14:01:19.074 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T14:01:19.113 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T14:01:19.114 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /sys 2024-04-25T14:01:19.114 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /proc 2024-04-25T14:01:19.114 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /mnt 2024-04-25T14:01:19.114 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /var/tmp 2024-04-25T14:01:19.114 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /home 2024-04-25T14:01:19.114 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /root 2024-04-25T14:01:19.114 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /tmp 2024-04-25T14:01:19.114 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-25T14:01:19.132 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T14:01:19.310 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T14:01:19.326 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T14:01:19.326 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:01:19.326 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T14:01:19.326 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-25T14:01:19.366 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T14:01:19.391 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T14:01:19.392 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:01:19.392 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T14:01:19.392 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T14:01:19.392 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T14:01:19.392 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T14:01:19.616 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T14:01:19.638 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T14:01:19.639 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:01:19.639 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T14:01:19.639 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T14:01:19.639 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T14:01:19.639 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T14:01:20.092 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T14:01:20.101 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T14:01:20.114 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T14:01:20.114 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:01:20.114 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T14:01:20.114 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T14:01:20.115 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T14:01:20.115 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T14:01:20.124 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T14:01:20.213 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T14:01:20.225 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T14:01:20.261 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T14:01:20.316 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T14:01:20.383 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 129/156 2024-04-25T14:01:20.417 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T14:01:20.441 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T14:01:20.441 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:01:20.441 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T14:01:20.441 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T14:01:20.441 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T14:01:20.441 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:01:20.485 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T14:01:20.496 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T14:01:20.499 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T14:01:20.550 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T14:01:20.594 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 132/156 2024-04-25T14:01:20.627 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T14:01:20.649 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 133/156 2024-04-25T14:01:20.715 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T14:01:20.845 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T14:01:20.884 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T14:01:20.896 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T14:01:20.916 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T14:01:20.922 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T14:01:20.922 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:01:20.922 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T14:01:20.922 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T14:01:20.923 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T14:01:20.923 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-25T14:01:20.943 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T14:01:21.146 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T14:01:21.170 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T14:01:21.170 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:01:21.170 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T14:01:21.170 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T14:01:21.170 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T14:01:21.170 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-25T14:01:21.379 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T14:01:21.409 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T14:01:21.441 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 124/156 2024-04-25T14:01:21.502 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 125/156 2024-04-25T14:01:21.548 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T14:01:21.591 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T14:01:21.620 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T14:01:21.622 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T14:01:21.643 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T14:01:21.644 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:01:21.644 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T14:01:21.644 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T14:01:21.644 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T14:01:21.644 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:01:21.645 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T14:01:21.645 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:01:21.646 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T14:01:21.646 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T14:01:21.646 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T14:01:21.646 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-25T14:01:21.696 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T14:01:21.718 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T14:01:21.718 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:01:21.718 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T14:01:21.718 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:01:21.752 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T14:01:21.763 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T14:01:21.786 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T14:01:21.786 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:01:21.786 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T14:01:21.786 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T14:01:21.786 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T14:01:21.786 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:01:21.801 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T14:01:21.856 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T14:01:22.040 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T14:01:22.089 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T14:01:22.106 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T14:01:22.129 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T14:01:22.129 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:01:22.129 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T14:01:22.129 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T14:01:22.129 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T14:01:22.129 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:01:22.158 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T14:01:22.247 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T14:01:22.385 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T14:01:22.423 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T14:01:22.455 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T14:01:22.491 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T14:01:22.575 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T14:01:22.617 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T14:01:22.877 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T14:01:22.907 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T14:01:22.939 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 124/156 2024-04-25T14:01:23.000 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 125/156 2024-04-25T14:01:23.038 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T14:01:23.578 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T14:01:23.617 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T14:01:23.718 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T14:01:23.883 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 129/156 2024-04-25T14:01:23.911 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T14:01:23.934 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T14:01:23.934 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:01:23.934 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T14:01:23.934 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T14:01:23.934 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T14:01:23.934 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T14:01:23.979 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T14:01:23.991 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T14:01:24.084 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T14:01:24.087 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 132/156 2024-04-25T14:01:24.123 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T14:01:24.145 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 133/156 2024-04-25T14:01:25.084 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T14:01:25.085 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T14:01:25.114 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T14:01:25.115 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T14:01:25.137 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T14:01:25.137 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:01:25.138 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T14:01:25.138 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T14:01:25.138 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T14:01:25.138 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T14:01:25.182 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T14:01:25.204 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T14:01:25.204 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:01:25.204 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T14:01:25.204 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T14:01:25.217 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T14:01:25.249 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T14:01:25.271 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T14:01:25.271 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:01:25.271 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T14:01:25.271 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T14:01:25.271 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T14:01:25.272 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T14:01:25.389 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 129/156 2024-04-25T14:01:25.417 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T14:01:25.441 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T14:01:25.441 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:01:25.441 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T14:01:25.441 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T14:01:25.441 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T14:01:25.441 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-25T14:01:25.485 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T14:01:25.497 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T14:01:25.583 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T14:01:25.594 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 132/156 2024-04-25T14:01:25.605 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T14:01:25.605 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:01:25.605 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T14:01:25.606 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T14:01:25.606 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T14:01:25.606 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T14:01:25.650 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 133/156 2024-04-25T14:01:26.401 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 139/156 2024-04-25T14:01:26.442 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 140/156 2024-04-25T14:01:26.483 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 141/156 2024-04-25T14:01:26.518 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T14:01:26.542 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T14:01:26.589 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T14:01:26.612 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T14:01:26.635 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T14:01:26.635 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:01:26.635 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T14:01:26.636 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T14:01:26.636 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T14:01:26.636 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-25T14:01:26.679 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T14:01:26.702 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T14:01:26.702 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:01:26.702 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T14:01:26.702 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-25T14:01:26.722 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 144/156 2024-04-25T14:01:26.747 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T14:01:26.763 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T14:01:26.769 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T14:01:26.770 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:01:26.770 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T14:01:26.770 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T14:01:26.770 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T14:01:26.770 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-25T14:01:26.793 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 146/156 2024-04-25T14:01:27.080 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T14:01:27.103 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T14:01:27.103 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:01:27.103 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T14:01:27.104 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T14:01:27.104 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T14:01:27.104 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-25T14:01:27.372 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T14:01:27.415 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T14:01:27.435 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T14:01:28.247 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 148/156 2024-04-25T14:01:28.337 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T14:01:28.368 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T14:01:28.457 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T14:01:28.491 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T14:01:28.491 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T14:01:28.566 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T14:01:28.591 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T14:01:28.591 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T14:01:28.667 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T14:01:28.730 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T14:01:29.846 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 139/156 2024-04-25T14:01:29.886 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 140/156 2024-04-25T14:01:29.926 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 141/156 2024-04-25T14:01:29.967 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T14:01:30.003 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T14:01:30.184 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 144/156 2024-04-25T14:01:30.224 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T14:01:30.254 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 146/156 2024-04-25T14:01:30.872 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T14:01:30.910 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T14:01:30.930 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T14:01:31.378 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 139/156 2024-04-25T14:01:31.422 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 140/156 2024-04-25T14:01:31.460 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 141/156 2024-04-25T14:01:31.492 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T14:01:31.519 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T14:01:31.700 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 144/156 2024-04-25T14:01:31.748 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T14:01:31.760 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 148/156 2024-04-25T14:01:31.778 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 146/156 2024-04-25T14:01:31.856 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T14:01:31.897 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T14:01:31.985 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T14:01:32.027 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T14:01:32.027 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T14:01:32.119 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T14:01:32.152 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T14:01:32.152 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T14:01:32.241 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T14:01:32.309 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T14:01:32.363 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T14:01:32.400 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T14:01:32.421 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T14:01:33.264 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 148/156 2024-04-25T14:01:33.355 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T14:01:33.396 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T14:01:33.477 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T14:01:33.517 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T14:01:33.517 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T14:01:33.601 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T14:01:33.634 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T14:01:33.634 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T14:01:33.726 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T14:01:33.791 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T14:01:34.639 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T14:01:34.640 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 1/156 2024-04-25T14:01:34.640 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2/156 2024-04-25T14:01:34.640 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 3/156 2024-04-25T14:01:34.640 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 4/156 2024-04-25T14:01:34.640 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 5/156 2024-04-25T14:01:34.640 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 6/156 2024-04-25T14:01:34.640 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 7/156 2024-04-25T14:01:34.640 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T14:01:34.640 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T14:01:34.640 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T14:01:34.640 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 11/156 2024-04-25T14:01:34.641 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 12/156 2024-04-25T14:01:34.641 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T14:01:34.641 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 14/156 2024-04-25T14:01:34.641 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 15/156 2024-04-25T14:01:34.641 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 16/156 2024-04-25T14:01:34.641 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 17/156 2024-04-25T14:01:34.641 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 18/156 2024-04-25T14:01:34.641 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 19/156 2024-04-25T14:01:34.641 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 20/156 2024-04-25T14:01:34.641 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 21/156 2024-04-25T14:01:34.641 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 22/156 2024-04-25T14:01:34.641 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 23/156 2024-04-25T14:01:34.641 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 24/156 2024-04-25T14:01:34.642 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 25/156 2024-04-25T14:01:34.642 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 26/156 2024-04-25T14:01:34.642 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 27/156 2024-04-25T14:01:34.654 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 28/156 2024-04-25T14:01:34.654 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 29/156 2024-04-25T14:01:34.654 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 30/156 2024-04-25T14:01:34.654 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 31/156 2024-04-25T14:01:34.654 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 32/156 2024-04-25T14:01:34.654 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 33/156 2024-04-25T14:01:34.654 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 34/156 2024-04-25T14:01:34.654 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 35/156 2024-04-25T14:01:34.654 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 36/156 2024-04-25T14:01:34.654 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 37/156 2024-04-25T14:01:34.654 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 38/156 2024-04-25T14:01:34.654 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 39/156 2024-04-25T14:01:34.655 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T14:01:34.655 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T14:01:34.655 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T14:01:34.655 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T14:01:34.655 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T14:01:34.655 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T14:01:34.655 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T14:01:34.655 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T14:01:34.655 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T14:01:34.655 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T14:01:34.655 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T14:01:34.655 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T14:01:34.655 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T14:01:34.656 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T14:01:34.656 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T14:01:34.656 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T14:01:34.656 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T14:01:34.656 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T14:01:34.656 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T14:01:34.656 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T14:01:34.656 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T14:01:34.656 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T14:01:34.656 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T14:01:34.656 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T14:01:34.656 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T14:01:34.656 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T14:01:34.656 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T14:01:34.657 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T14:01:34.657 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T14:01:34.657 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T14:01:34.657 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T14:01:34.657 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T14:01:34.657 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T14:01:34.657 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T14:01:34.657 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T14:01:34.657 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T14:01:34.657 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T14:01:34.658 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T14:01:34.658 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T14:01:34.658 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T14:01:34.658 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T14:01:34.658 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T14:01:34.658 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T14:01:34.658 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T14:01:34.658 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T14:01:34.658 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T14:01:34.658 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T14:01:34.659 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T14:01:34.659 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T14:01:34.659 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T14:01:34.659 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T14:01:34.659 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T14:01:34.659 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T14:01:34.659 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T14:01:34.659 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T14:01:34.659 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T14:01:34.659 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T14:01:34.659 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T14:01:34.659 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T14:01:34.659 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T14:01:34.660 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T14:01:34.660 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T14:01:34.660 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T14:01:34.660 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T14:01:34.660 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T14:01:34.660 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T14:01:34.660 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T14:01:34.660 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T14:01:34.660 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T14:01:34.661 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T14:01:34.661 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T14:01:34.661 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T14:01:34.661 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T14:01:34.661 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T14:01:34.661 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T14:01:34.661 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T14:01:34.661 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T14:01:34.661 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T14:01:34.661 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T14:01:34.661 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T14:01:34.661 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T14:01:34.661 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T14:01:34.661 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T14:01:34.662 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T14:01:34.662 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T14:01:34.662 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T14:01:34.662 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T14:01:34.662 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T14:01:34.662 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T14:01:34.662 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T14:01:34.662 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T14:01:34.662 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T14:01:34.663 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T14:01:34.663 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T14:01:34.663 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T14:01:34.663 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T14:01:34.663 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T14:01:34.663 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T14:01:34.663 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T14:01:34.663 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T14:01:34.663 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T14:01:34.663 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T14:01:34.663 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T14:01:34.663 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T14:01:34.663 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T14:01:34.663 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T14:01:34.664 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T14:01:34.664 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T14:01:34.664 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T14:01:34.664 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 149/156 2024-04-25T14:01:34.664 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T14:01:34.664 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 151/156 2024-04-25T14:01:34.664 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T14:01:34.664 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T14:01:34.664 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T14:01:34.664 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T14:01:36.776 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T14:01:36.777 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:01:36.777 INFO:teuthology.orchestra.run.smithi096.stdout:Upgraded: 2024-04-25T14:01:36.777 INFO:teuthology.orchestra.run.smithi096.stdout: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:36.777 INFO:teuthology.orchestra.run.smithi096.stdout: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:36.777 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T14:01:36.777 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T14:01:36.777 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-04-25T14:01:36.777 INFO:teuthology.orchestra.run.smithi096.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T14:01:36.777 INFO:teuthology.orchestra.run.smithi096.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T14:01:36.778 INFO:teuthology.orchestra.run.smithi096.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T14:01:36.778 INFO:teuthology.orchestra.run.smithi096.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T14:01:36.778 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:36.778 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:36.778 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:36.778 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:36.778 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T14:01:36.778 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-immutable-object-cache-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:36.778 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:36.778 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:36.778 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T14:01:36.779 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T14:01:36.779 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T14:01:36.779 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T14:01:36.779 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T14:01:36.779 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:36.779 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:36.779 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T14:01:36.779 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:36.779 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:36.779 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:36.779 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T14:01:36.779 INFO:teuthology.orchestra.run.smithi096.stdout: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T14:01:36.780 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:36.780 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:36.780 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T14:01:36.780 INFO:teuthology.orchestra.run.smithi096.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T14:01:36.780 INFO:teuthology.orchestra.run.smithi096.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T14:01:36.780 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T14:01:36.780 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T14:01:36.780 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T14:01:36.780 INFO:teuthology.orchestra.run.smithi096.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T14:01:36.780 INFO:teuthology.orchestra.run.smithi096.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T14:01:36.780 INFO:teuthology.orchestra.run.smithi096.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T14:01:36.780 INFO:teuthology.orchestra.run.smithi096.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T14:01:36.780 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T14:01:36.780 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T14:01:36.781 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:36.781 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:36.781 INFO:teuthology.orchestra.run.smithi096.stdout: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:36.781 INFO:teuthology.orchestra.run.smithi096.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T14:01:36.781 INFO:teuthology.orchestra.run.smithi096.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T14:01:36.781 INFO:teuthology.orchestra.run.smithi096.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T14:01:36.781 INFO:teuthology.orchestra.run.smithi096.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T14:01:36.781 INFO:teuthology.orchestra.run.smithi096.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T14:01:36.781 INFO:teuthology.orchestra.run.smithi096.stdout: librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:36.781 INFO:teuthology.orchestra.run.smithi096.stdout: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:36.781 INFO:teuthology.orchestra.run.smithi096.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T14:01:36.782 INFO:teuthology.orchestra.run.smithi096.stdout: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:36.782 INFO:teuthology.orchestra.run.smithi096.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T14:01:36.782 INFO:teuthology.orchestra.run.smithi096.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T14:01:36.782 INFO:teuthology.orchestra.run.smithi096.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T14:01:36.782 INFO:teuthology.orchestra.run.smithi096.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T14:01:36.782 INFO:teuthology.orchestra.run.smithi096.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T14:01:36.782 INFO:teuthology.orchestra.run.smithi096.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T14:01:36.782 INFO:teuthology.orchestra.run.smithi096.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T14:01:36.782 INFO:teuthology.orchestra.run.smithi096.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T14:01:36.782 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T14:01:36.782 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T14:01:36.782 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T14:01:36.782 INFO:teuthology.orchestra.run.smithi096.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T14:01:36.783 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T14:01:36.783 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T14:01:36.783 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T14:01:36.783 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T14:01:36.783 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T14:01:36.783 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T14:01:36.783 INFO:teuthology.orchestra.run.smithi096.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T14:01:36.783 INFO:teuthology.orchestra.run.smithi096.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T14:01:36.783 INFO:teuthology.orchestra.run.smithi096.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T14:01:36.783 INFO:teuthology.orchestra.run.smithi096.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T14:01:36.783 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T14:01:36.783 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:36.783 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:36.783 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:36.784 INFO:teuthology.orchestra.run.smithi096.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T14:01:36.784 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T14:01:36.784 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T14:01:36.784 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T14:01:36.784 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T14:01:36.784 INFO:teuthology.orchestra.run.smithi096.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T14:01:36.784 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T14:01:36.784 INFO:teuthology.orchestra.run.smithi096.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T14:01:36.784 INFO:teuthology.orchestra.run.smithi096.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T14:01:36.784 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T14:01:36.784 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T14:01:36.784 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T14:01:36.784 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T14:01:36.784 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T14:01:36.785 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T14:01:36.785 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T14:01:36.785 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T14:01:36.785 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T14:01:36.785 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T14:01:36.785 INFO:teuthology.orchestra.run.smithi096.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T14:01:36.785 INFO:teuthology.orchestra.run.smithi096.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T14:01:36.785 INFO:teuthology.orchestra.run.smithi096.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T14:01:36.785 INFO:teuthology.orchestra.run.smithi096.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T14:01:36.785 INFO:teuthology.orchestra.run.smithi096.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T14:01:36.785 INFO:teuthology.orchestra.run.smithi096.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T14:01:36.785 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T14:01:36.785 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T14:01:36.785 INFO:teuthology.orchestra.run.smithi096.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T14:01:36.786 INFO:teuthology.orchestra.run.smithi096.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T14:01:36.786 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T14:01:36.786 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T14:01:36.786 INFO:teuthology.orchestra.run.smithi096.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T14:01:36.786 INFO:teuthology.orchestra.run.smithi096.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T14:01:36.786 INFO:teuthology.orchestra.run.smithi096.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T14:01:36.786 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T14:01:36.786 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T14:01:36.786 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T14:01:36.786 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T14:01:36.786 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T14:01:36.786 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T14:01:36.786 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:36.786 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:36.787 INFO:teuthology.orchestra.run.smithi096.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T14:01:36.787 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T14:01:36.787 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T14:01:36.787 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:36.787 INFO:teuthology.orchestra.run.smithi096.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T14:01:36.787 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T14:01:36.787 INFO:teuthology.orchestra.run.smithi096.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T14:01:36.787 INFO:teuthology.orchestra.run.smithi096.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T14:01:36.787 INFO:teuthology.orchestra.run.smithi096.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T14:01:36.787 INFO:teuthology.orchestra.run.smithi096.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T14:01:36.787 INFO:teuthology.orchestra.run.smithi096.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T14:01:36.787 INFO:teuthology.orchestra.run.smithi096.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T14:01:36.787 INFO:teuthology.orchestra.run.smithi096.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T14:01:36.787 INFO:teuthology.orchestra.run.smithi096.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T14:01:36.788 INFO:teuthology.orchestra.run.smithi096.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T14:01:36.788 INFO:teuthology.orchestra.run.smithi096.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T14:01:36.788 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T14:01:36.788 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T14:01:36.788 INFO:teuthology.orchestra.run.smithi096.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T14:01:36.788 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:36.788 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:36.788 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:36.788 INFO:teuthology.orchestra.run.smithi096.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T14:01:36.788 INFO:teuthology.orchestra.run.smithi096.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T14:01:36.788 INFO:teuthology.orchestra.run.smithi096.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T14:01:36.788 INFO:teuthology.orchestra.run.smithi096.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T14:01:36.788 INFO:teuthology.orchestra.run.smithi096.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T14:01:36.788 INFO:teuthology.orchestra.run.smithi096.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T14:01:36.789 INFO:teuthology.orchestra.run.smithi096.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T14:01:36.789 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:01:36.789 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-04-25T14:01:37.166 DEBUG:teuthology.parallel:result is None 2024-04-25T14:01:38.322 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T14:01:38.323 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 1/156 2024-04-25T14:01:38.323 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2/156 2024-04-25T14:01:38.323 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 3/156 2024-04-25T14:01:38.323 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 4/156 2024-04-25T14:01:38.323 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 5/156 2024-04-25T14:01:38.323 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 6/156 2024-04-25T14:01:38.323 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 7/156 2024-04-25T14:01:38.323 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T14:01:38.323 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T14:01:38.324 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T14:01:38.324 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 11/156 2024-04-25T14:01:38.324 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 12/156 2024-04-25T14:01:38.324 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T14:01:38.324 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 14/156 2024-04-25T14:01:38.324 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 15/156 2024-04-25T14:01:38.324 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 16/156 2024-04-25T14:01:38.324 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 17/156 2024-04-25T14:01:38.324 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 18/156 2024-04-25T14:01:38.324 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 19/156 2024-04-25T14:01:38.325 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 20/156 2024-04-25T14:01:38.325 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 21/156 2024-04-25T14:01:38.325 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 22/156 2024-04-25T14:01:38.325 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 23/156 2024-04-25T14:01:38.325 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 24/156 2024-04-25T14:01:38.325 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 25/156 2024-04-25T14:01:38.326 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 26/156 2024-04-25T14:01:38.326 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 27/156 2024-04-25T14:01:38.339 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 28/156 2024-04-25T14:01:38.339 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 29/156 2024-04-25T14:01:38.339 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 30/156 2024-04-25T14:01:38.340 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 31/156 2024-04-25T14:01:38.340 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 32/156 2024-04-25T14:01:38.340 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 33/156 2024-04-25T14:01:38.340 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 34/156 2024-04-25T14:01:38.340 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 35/156 2024-04-25T14:01:38.340 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 36/156 2024-04-25T14:01:38.340 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 37/156 2024-04-25T14:01:38.340 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 38/156 2024-04-25T14:01:38.340 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 39/156 2024-04-25T14:01:38.340 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T14:01:38.340 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T14:01:38.340 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T14:01:38.341 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T14:01:38.341 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T14:01:38.341 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T14:01:38.341 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T14:01:38.341 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T14:01:38.341 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T14:01:38.341 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T14:01:38.341 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T14:01:38.342 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T14:01:38.342 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T14:01:38.342 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T14:01:38.342 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T14:01:38.342 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T14:01:38.342 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T14:01:38.342 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T14:01:38.342 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T14:01:38.342 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T14:01:38.343 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T14:01:38.343 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T14:01:38.343 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T14:01:38.343 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T14:01:38.343 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T14:01:38.343 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T14:01:38.343 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T14:01:38.343 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T14:01:38.343 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T14:01:38.343 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T14:01:38.343 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T14:01:38.343 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T14:01:38.344 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T14:01:38.344 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T14:01:38.344 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T14:01:38.344 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T14:01:38.344 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T14:01:38.344 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T14:01:38.344 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T14:01:38.344 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T14:01:38.344 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T14:01:38.344 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T14:01:38.344 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T14:01:38.344 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T14:01:38.345 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T14:01:38.345 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T14:01:38.345 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T14:01:38.345 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T14:01:38.345 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T14:01:38.345 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T14:01:38.345 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T14:01:38.345 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T14:01:38.345 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T14:01:38.345 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T14:01:38.345 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T14:01:38.345 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T14:01:38.346 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T14:01:38.346 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T14:01:38.346 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T14:01:38.346 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T14:01:38.346 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T14:01:38.346 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T14:01:38.346 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T14:01:38.346 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T14:01:38.346 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T14:01:38.346 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T14:01:38.347 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T14:01:38.347 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T14:01:38.347 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T14:01:38.347 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T14:01:38.347 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T14:01:38.347 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T14:01:38.347 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T14:01:38.347 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T14:01:38.347 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T14:01:38.347 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T14:01:38.348 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T14:01:38.348 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T14:01:38.348 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T14:01:38.348 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T14:01:38.348 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T14:01:38.348 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T14:01:38.348 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T14:01:38.348 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T14:01:38.348 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T14:01:38.348 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T14:01:38.348 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T14:01:38.348 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T14:01:38.349 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T14:01:38.349 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T14:01:38.349 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T14:01:38.349 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T14:01:38.349 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T14:01:38.349 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T14:01:38.349 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T14:01:38.349 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T14:01:38.350 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T14:01:38.350 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T14:01:38.350 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T14:01:38.350 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T14:01:38.350 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T14:01:38.350 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T14:01:38.350 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T14:01:38.350 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T14:01:38.350 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T14:01:38.350 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T14:01:38.350 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T14:01:38.351 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T14:01:38.351 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T14:01:38.351 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 149/156 2024-04-25T14:01:38.351 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T14:01:38.351 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 151/156 2024-04-25T14:01:38.351 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T14:01:38.351 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T14:01:38.351 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T14:01:38.351 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T14:01:40.204 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T14:01:40.204 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 1/156 2024-04-25T14:01:40.204 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2/156 2024-04-25T14:01:40.204 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 3/156 2024-04-25T14:01:40.205 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 4/156 2024-04-25T14:01:40.205 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 5/156 2024-04-25T14:01:40.205 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 6/156 2024-04-25T14:01:40.205 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 7/156 2024-04-25T14:01:40.205 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T14:01:40.205 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T14:01:40.205 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T14:01:40.205 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 11/156 2024-04-25T14:01:40.205 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 12/156 2024-04-25T14:01:40.205 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T14:01:40.205 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 14/156 2024-04-25T14:01:40.206 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 15/156 2024-04-25T14:01:40.206 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 16/156 2024-04-25T14:01:40.206 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 17/156 2024-04-25T14:01:40.206 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 18/156 2024-04-25T14:01:40.206 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 19/156 2024-04-25T14:01:40.206 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 20/156 2024-04-25T14:01:40.206 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 21/156 2024-04-25T14:01:40.206 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 22/156 2024-04-25T14:01:40.206 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 23/156 2024-04-25T14:01:40.206 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 24/156 2024-04-25T14:01:40.206 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 25/156 2024-04-25T14:01:40.207 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 26/156 2024-04-25T14:01:40.207 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 27/156 2024-04-25T14:01:40.207 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 28/156 2024-04-25T14:01:40.207 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 29/156 2024-04-25T14:01:40.207 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 30/156 2024-04-25T14:01:40.207 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 31/156 2024-04-25T14:01:40.207 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 32/156 2024-04-25T14:01:40.207 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 33/156 2024-04-25T14:01:40.207 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 34/156 2024-04-25T14:01:40.207 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 35/156 2024-04-25T14:01:40.208 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 36/156 2024-04-25T14:01:40.208 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 37/156 2024-04-25T14:01:40.208 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 38/156 2024-04-25T14:01:40.208 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 39/156 2024-04-25T14:01:40.208 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T14:01:40.208 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T14:01:40.208 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T14:01:40.208 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T14:01:40.209 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T14:01:40.209 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T14:01:40.209 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T14:01:40.209 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T14:01:40.209 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T14:01:40.209 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T14:01:40.209 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T14:01:40.209 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T14:01:40.210 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T14:01:40.210 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T14:01:40.210 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T14:01:40.210 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T14:01:40.210 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T14:01:40.210 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T14:01:40.210 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T14:01:40.211 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T14:01:40.211 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T14:01:40.211 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T14:01:40.211 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T14:01:40.211 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T14:01:40.211 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T14:01:40.211 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T14:01:40.211 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T14:01:40.211 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T14:01:40.211 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T14:01:40.211 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T14:01:40.214 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T14:01:40.214 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T14:01:40.214 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T14:01:40.215 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T14:01:40.215 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T14:01:40.215 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T14:01:40.215 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T14:01:40.215 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T14:01:40.215 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T14:01:40.215 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T14:01:40.215 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T14:01:40.216 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T14:01:40.216 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T14:01:40.216 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T14:01:40.216 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T14:01:40.216 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T14:01:40.216 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T14:01:40.216 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T14:01:40.216 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T14:01:40.217 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T14:01:40.217 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T14:01:40.217 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T14:01:40.217 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T14:01:40.217 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T14:01:40.217 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T14:01:40.217 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T14:01:40.217 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T14:01:40.218 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T14:01:40.218 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T14:01:40.218 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T14:01:40.218 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T14:01:40.218 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T14:01:40.218 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T14:01:40.218 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T14:01:40.218 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T14:01:40.219 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T14:01:40.219 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T14:01:40.219 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T14:01:40.219 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T14:01:40.219 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T14:01:40.219 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T14:01:40.219 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T14:01:40.219 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T14:01:40.220 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T14:01:40.220 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T14:01:40.220 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T14:01:40.220 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T14:01:40.220 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T14:01:40.220 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T14:01:40.220 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T14:01:40.220 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T14:01:40.220 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T14:01:40.221 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T14:01:40.221 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T14:01:40.221 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T14:01:40.221 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T14:01:40.221 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T14:01:40.221 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T14:01:40.221 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T14:01:40.221 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T14:01:40.221 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T14:01:40.222 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T14:01:40.222 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T14:01:40.222 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T14:01:40.222 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T14:01:40.222 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T14:01:40.222 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T14:01:40.222 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T14:01:40.222 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T14:01:40.222 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T14:01:40.223 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T14:01:40.223 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T14:01:40.223 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T14:01:40.223 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T14:01:40.223 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T14:01:40.223 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T14:01:40.223 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T14:01:40.223 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T14:01:40.223 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T14:01:40.224 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 149/156 2024-04-25T14:01:40.224 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T14:01:40.224 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 151/156 2024-04-25T14:01:40.224 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T14:01:40.224 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T14:01:40.224 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T14:01:40.224 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T14:01:40.341 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T14:01:40.341 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T14:01:40.341 INFO:teuthology.orchestra.run.smithi097.stdout:Upgraded: 2024-04-25T14:01:40.341 INFO:teuthology.orchestra.run.smithi097.stdout: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:40.341 INFO:teuthology.orchestra.run.smithi097.stdout: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:40.341 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T14:01:40.341 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T14:01:40.341 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-04-25T14:01:40.341 INFO:teuthology.orchestra.run.smithi097.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T14:01:40.342 INFO:teuthology.orchestra.run.smithi097.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T14:01:40.342 INFO:teuthology.orchestra.run.smithi097.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T14:01:40.342 INFO:teuthology.orchestra.run.smithi097.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T14:01:40.342 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:40.342 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:40.342 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:40.342 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:40.342 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T14:01:40.342 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-immutable-object-cache-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:40.342 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:40.342 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:40.342 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T14:01:40.342 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T14:01:40.343 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T14:01:40.343 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T14:01:40.343 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T14:01:40.343 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:40.343 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:40.343 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T14:01:40.343 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:40.343 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:40.343 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:40.343 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T14:01:40.343 INFO:teuthology.orchestra.run.smithi097.stdout: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T14:01:40.343 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:40.343 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:40.344 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T14:01:40.344 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T14:01:40.344 INFO:teuthology.orchestra.run.smithi097.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T14:01:40.344 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T14:01:40.344 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T14:01:40.344 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T14:01:40.344 INFO:teuthology.orchestra.run.smithi097.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T14:01:40.344 INFO:teuthology.orchestra.run.smithi097.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T14:01:40.345 INFO:teuthology.orchestra.run.smithi097.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T14:01:40.345 INFO:teuthology.orchestra.run.smithi097.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T14:01:40.345 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T14:01:40.345 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T14:01:40.345 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:40.345 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:40.345 INFO:teuthology.orchestra.run.smithi097.stdout: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:40.345 INFO:teuthology.orchestra.run.smithi097.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T14:01:40.345 INFO:teuthology.orchestra.run.smithi097.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T14:01:40.345 INFO:teuthology.orchestra.run.smithi097.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T14:01:40.345 INFO:teuthology.orchestra.run.smithi097.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T14:01:40.346 INFO:teuthology.orchestra.run.smithi097.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T14:01:40.346 INFO:teuthology.orchestra.run.smithi097.stdout: librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:40.346 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:40.346 INFO:teuthology.orchestra.run.smithi097.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T14:01:40.346 INFO:teuthology.orchestra.run.smithi097.stdout: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:40.346 INFO:teuthology.orchestra.run.smithi097.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T14:01:40.346 INFO:teuthology.orchestra.run.smithi097.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T14:01:40.346 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T14:01:40.346 INFO:teuthology.orchestra.run.smithi097.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T14:01:40.346 INFO:teuthology.orchestra.run.smithi097.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T14:01:40.346 INFO:teuthology.orchestra.run.smithi097.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T14:01:40.346 INFO:teuthology.orchestra.run.smithi097.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T14:01:40.347 INFO:teuthology.orchestra.run.smithi097.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T14:01:40.347 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T14:01:40.347 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T14:01:40.347 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T14:01:40.347 INFO:teuthology.orchestra.run.smithi097.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T14:01:40.347 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T14:01:40.347 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T14:01:40.347 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T14:01:40.347 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T14:01:40.347 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T14:01:40.347 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T14:01:40.347 INFO:teuthology.orchestra.run.smithi097.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T14:01:40.347 INFO:teuthology.orchestra.run.smithi097.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T14:01:40.347 INFO:teuthology.orchestra.run.smithi097.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T14:01:40.348 INFO:teuthology.orchestra.run.smithi097.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T14:01:40.348 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T14:01:40.348 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:40.348 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:40.348 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:40.348 INFO:teuthology.orchestra.run.smithi097.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T14:01:40.348 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T14:01:40.348 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T14:01:40.348 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T14:01:40.348 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T14:01:40.348 INFO:teuthology.orchestra.run.smithi097.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T14:01:40.348 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T14:01:40.348 INFO:teuthology.orchestra.run.smithi097.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T14:01:40.348 INFO:teuthology.orchestra.run.smithi097.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T14:01:40.349 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T14:01:40.349 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T14:01:40.349 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T14:01:40.349 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T14:01:40.349 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T14:01:40.349 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T14:01:40.349 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T14:01:40.349 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T14:01:40.349 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T14:01:40.349 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T14:01:40.349 INFO:teuthology.orchestra.run.smithi097.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T14:01:40.349 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T14:01:40.349 INFO:teuthology.orchestra.run.smithi097.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T14:01:40.349 INFO:teuthology.orchestra.run.smithi097.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T14:01:40.350 INFO:teuthology.orchestra.run.smithi097.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T14:01:40.350 INFO:teuthology.orchestra.run.smithi097.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T14:01:40.350 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T14:01:40.350 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T14:01:40.350 INFO:teuthology.orchestra.run.smithi097.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T14:01:40.350 INFO:teuthology.orchestra.run.smithi097.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T14:01:40.350 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T14:01:40.350 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T14:01:40.350 INFO:teuthology.orchestra.run.smithi097.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T14:01:40.350 INFO:teuthology.orchestra.run.smithi097.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T14:01:40.350 INFO:teuthology.orchestra.run.smithi097.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T14:01:40.350 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T14:01:40.351 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T14:01:40.351 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T14:01:40.351 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T14:01:40.351 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T14:01:40.351 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T14:01:40.351 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:40.351 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:40.351 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T14:01:40.351 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T14:01:40.352 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T14:01:40.352 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:40.352 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T14:01:40.352 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T14:01:40.352 INFO:teuthology.orchestra.run.smithi097.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T14:01:40.352 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T14:01:40.352 INFO:teuthology.orchestra.run.smithi097.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T14:01:40.352 INFO:teuthology.orchestra.run.smithi097.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T14:01:40.352 INFO:teuthology.orchestra.run.smithi097.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T14:01:40.352 INFO:teuthology.orchestra.run.smithi097.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T14:01:40.352 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T14:01:40.352 INFO:teuthology.orchestra.run.smithi097.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T14:01:40.352 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T14:01:40.352 INFO:teuthology.orchestra.run.smithi097.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T14:01:40.353 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T14:01:40.353 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T14:01:40.353 INFO:teuthology.orchestra.run.smithi097.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T14:01:40.353 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:40.353 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:40.353 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:40.353 INFO:teuthology.orchestra.run.smithi097.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T14:01:40.353 INFO:teuthology.orchestra.run.smithi097.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T14:01:40.353 INFO:teuthology.orchestra.run.smithi097.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T14:01:40.353 INFO:teuthology.orchestra.run.smithi097.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T14:01:40.354 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T14:01:40.354 INFO:teuthology.orchestra.run.smithi097.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T14:01:40.354 INFO:teuthology.orchestra.run.smithi097.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T14:01:40.354 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T14:01:40.354 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-04-25T14:01:40.751 DEBUG:teuthology.parallel:result is None 2024-04-25T14:01:42.263 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T14:01:42.263 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-25T14:01:42.263 INFO:teuthology.orchestra.run.smithi151.stdout:Upgraded: 2024-04-25T14:01:42.263 INFO:teuthology.orchestra.run.smithi151.stdout: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:42.263 INFO:teuthology.orchestra.run.smithi151.stdout: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:42.263 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T14:01:42.264 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T14:01:42.264 INFO:teuthology.orchestra.run.smithi151.stdout:Installed: 2024-04-25T14:01:42.264 INFO:teuthology.orchestra.run.smithi151.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T14:01:42.264 INFO:teuthology.orchestra.run.smithi151.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T14:01:42.264 INFO:teuthology.orchestra.run.smithi151.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T14:01:42.264 INFO:teuthology.orchestra.run.smithi151.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T14:01:42.264 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:42.264 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:42.264 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:42.264 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:42.264 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T14:01:42.264 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-immutable-object-cache-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:42.264 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:42.265 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:42.265 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T14:01:42.265 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T14:01:42.265 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T14:01:42.265 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T14:01:42.265 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T14:01:42.265 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:42.265 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:42.265 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T14:01:42.265 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:42.265 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:42.265 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:42.265 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T14:01:42.266 INFO:teuthology.orchestra.run.smithi151.stdout: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T14:01:42.266 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:42.266 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:42.266 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T14:01:42.266 INFO:teuthology.orchestra.run.smithi151.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T14:01:42.266 INFO:teuthology.orchestra.run.smithi151.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T14:01:42.266 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T14:01:42.266 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T14:01:42.266 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T14:01:42.266 INFO:teuthology.orchestra.run.smithi151.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T14:01:42.266 INFO:teuthology.orchestra.run.smithi151.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T14:01:42.266 INFO:teuthology.orchestra.run.smithi151.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T14:01:42.266 INFO:teuthology.orchestra.run.smithi151.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T14:01:42.267 INFO:teuthology.orchestra.run.smithi151.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T14:01:42.267 INFO:teuthology.orchestra.run.smithi151.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T14:01:42.267 INFO:teuthology.orchestra.run.smithi151.stdout: libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:42.267 INFO:teuthology.orchestra.run.smithi151.stdout: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:42.267 INFO:teuthology.orchestra.run.smithi151.stdout: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:42.267 INFO:teuthology.orchestra.run.smithi151.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T14:01:42.267 INFO:teuthology.orchestra.run.smithi151.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T14:01:42.267 INFO:teuthology.orchestra.run.smithi151.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T14:01:42.267 INFO:teuthology.orchestra.run.smithi151.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T14:01:42.267 INFO:teuthology.orchestra.run.smithi151.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T14:01:42.267 INFO:teuthology.orchestra.run.smithi151.stdout: librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:42.267 INFO:teuthology.orchestra.run.smithi151.stdout: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:42.267 INFO:teuthology.orchestra.run.smithi151.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T14:01:42.268 INFO:teuthology.orchestra.run.smithi151.stdout: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:42.268 INFO:teuthology.orchestra.run.smithi151.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T14:01:42.268 INFO:teuthology.orchestra.run.smithi151.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T14:01:42.268 INFO:teuthology.orchestra.run.smithi151.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T14:01:42.268 INFO:teuthology.orchestra.run.smithi151.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T14:01:42.268 INFO:teuthology.orchestra.run.smithi151.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T14:01:42.268 INFO:teuthology.orchestra.run.smithi151.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T14:01:42.268 INFO:teuthology.orchestra.run.smithi151.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T14:01:42.268 INFO:teuthology.orchestra.run.smithi151.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T14:01:42.268 INFO:teuthology.orchestra.run.smithi151.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T14:01:42.268 INFO:teuthology.orchestra.run.smithi151.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T14:01:42.268 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T14:01:42.268 INFO:teuthology.orchestra.run.smithi151.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T14:01:42.268 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T14:01:42.269 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T14:01:42.269 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T14:01:42.269 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T14:01:42.269 INFO:teuthology.orchestra.run.smithi151.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T14:01:42.269 INFO:teuthology.orchestra.run.smithi151.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T14:01:42.269 INFO:teuthology.orchestra.run.smithi151.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T14:01:42.269 INFO:teuthology.orchestra.run.smithi151.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T14:01:42.269 INFO:teuthology.orchestra.run.smithi151.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T14:01:42.269 INFO:teuthology.orchestra.run.smithi151.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T14:01:42.269 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T14:01:42.269 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:42.269 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:42.269 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:42.269 INFO:teuthology.orchestra.run.smithi151.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T14:01:42.270 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T14:01:42.270 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T14:01:42.270 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T14:01:42.270 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T14:01:42.270 INFO:teuthology.orchestra.run.smithi151.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T14:01:42.270 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T14:01:42.270 INFO:teuthology.orchestra.run.smithi151.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T14:01:42.270 INFO:teuthology.orchestra.run.smithi151.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T14:01:42.270 INFO:teuthology.orchestra.run.smithi151.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T14:01:42.270 INFO:teuthology.orchestra.run.smithi151.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T14:01:42.270 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T14:01:42.270 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T14:01:42.270 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T14:01:42.271 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T14:01:42.271 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T14:01:42.271 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T14:01:42.271 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T14:01:42.271 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T14:01:42.271 INFO:teuthology.orchestra.run.smithi151.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T14:01:42.271 INFO:teuthology.orchestra.run.smithi151.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T14:01:42.271 INFO:teuthology.orchestra.run.smithi151.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T14:01:42.271 INFO:teuthology.orchestra.run.smithi151.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T14:01:42.271 INFO:teuthology.orchestra.run.smithi151.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T14:01:42.271 INFO:teuthology.orchestra.run.smithi151.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T14:01:42.271 INFO:teuthology.orchestra.run.smithi151.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T14:01:42.271 INFO:teuthology.orchestra.run.smithi151.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T14:01:42.272 INFO:teuthology.orchestra.run.smithi151.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T14:01:42.272 INFO:teuthology.orchestra.run.smithi151.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T14:01:42.272 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T14:01:42.272 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T14:01:42.272 INFO:teuthology.orchestra.run.smithi151.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T14:01:42.272 INFO:teuthology.orchestra.run.smithi151.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T14:01:42.272 INFO:teuthology.orchestra.run.smithi151.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T14:01:42.272 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T14:01:42.272 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T14:01:42.272 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T14:01:42.272 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T14:01:42.272 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T14:01:42.272 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T14:01:42.272 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:42.272 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:42.273 INFO:teuthology.orchestra.run.smithi151.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T14:01:42.273 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T14:01:42.273 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T14:01:42.273 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:42.273 INFO:teuthology.orchestra.run.smithi151.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T14:01:42.273 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T14:01:42.273 INFO:teuthology.orchestra.run.smithi151.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T14:01:42.273 INFO:teuthology.orchestra.run.smithi151.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T14:01:42.273 INFO:teuthology.orchestra.run.smithi151.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T14:01:42.273 INFO:teuthology.orchestra.run.smithi151.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T14:01:42.273 INFO:teuthology.orchestra.run.smithi151.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T14:01:42.273 INFO:teuthology.orchestra.run.smithi151.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T14:01:42.273 INFO:teuthology.orchestra.run.smithi151.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T14:01:42.273 INFO:teuthology.orchestra.run.smithi151.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T14:01:42.274 INFO:teuthology.orchestra.run.smithi151.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T14:01:42.274 INFO:teuthology.orchestra.run.smithi151.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T14:01:42.274 INFO:teuthology.orchestra.run.smithi151.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T14:01:42.274 INFO:teuthology.orchestra.run.smithi151.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T14:01:42.274 INFO:teuthology.orchestra.run.smithi151.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T14:01:42.274 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:42.274 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:42.274 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T14:01:42.274 INFO:teuthology.orchestra.run.smithi151.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T14:01:42.274 INFO:teuthology.orchestra.run.smithi151.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T14:01:42.274 INFO:teuthology.orchestra.run.smithi151.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T14:01:42.274 INFO:teuthology.orchestra.run.smithi151.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T14:01:42.274 INFO:teuthology.orchestra.run.smithi151.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T14:01:42.275 INFO:teuthology.orchestra.run.smithi151.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T14:01:42.275 INFO:teuthology.orchestra.run.smithi151.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T14:01:42.275 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-25T14:01:42.275 INFO:teuthology.orchestra.run.smithi151.stdout:Complete! 2024-04-25T14:01:42.674 DEBUG:teuthology.parallel:result is None 2024-04-25T14:01:42.675 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T14:01:42.815 DEBUG:teuthology.orchestra.run.smithi096:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T14:01:42.841 INFO:teuthology.orchestra.run.smithi096.stdout:19.0.0-3291.g994e2f20.el9 2024-04-25T14:01:42.842 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291.g994e2f20.el9 2024-04-25T14:01:42.842 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291.g994e2f20 is installed. 2024-04-25T14:01:42.844 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T14:01:43.073 DEBUG:teuthology.orchestra.run.smithi097:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T14:01:43.095 INFO:teuthology.orchestra.run.smithi097.stdout:19.0.0-3291.g994e2f20.el9 2024-04-25T14:01:43.095 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291.g994e2f20.el9 2024-04-25T14:01:43.096 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291.g994e2f20 is installed. 2024-04-25T14:01:43.098 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T14:01:43.238 DEBUG:teuthology.orchestra.run.smithi151:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T14:01:43.261 INFO:teuthology.orchestra.run.smithi151.stdout:19.0.0-3291.g994e2f20.el9 2024-04-25T14:01:43.261 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291.g994e2f20.el9 2024-04-25T14:01:43.261 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291.g994e2f20 is installed. 2024-04-25T14:01:43.263 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T14:01:43.263 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T14:01:43.263 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T14:01:43.294 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T14:01:43.295 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T14:01:43.322 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-25T14:01:43.323 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T14:01:43.349 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T14:01:43.349 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T14:01:43.349 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T14:01:43.377 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T14:01:43.445 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T14:01:43.445 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T14:01:43.470 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T14:01:43.534 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-25T14:01:43.534 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T14:01:43.558 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T14:01:43.622 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T14:01:43.623 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T14:01:43.623 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T14:01:43.651 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T14:01:43.719 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T14:01:43.719 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T14:01:43.744 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T14:01:43.808 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-25T14:01:43.808 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T14:01:43.832 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T14:01:43.895 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T14:01:43.895 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T14:01:43.895 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T14:01:43.926 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T14:01:43.994 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T14:01:43.994 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T14:01:44.017 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T14:01:44.081 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-25T14:01:44.081 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T14:01:44.105 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T14:01:44.169 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-25T14:01:44.279 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-25T14:01:44.279 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T14:01:44.279 INFO:tasks.cephadm:Cluster fsid is 59334182-030c-11ef-bc93-c7b262605968 2024-04-25T14:01:44.279 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-25T14:01:44.280 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.96', 'mon.b': '172.21.15.97', 'mon.c': '172.21.15.151'} 2024-04-25T14:01:44.280 INFO:tasks.cephadm:First mon is mon.a on smithi096 2024-04-25T14:01:44.280 INFO:tasks.cephadm:First mgr is x 2024-04-25T14:01:44.280 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-25T14:01:44.280 DEBUG:teuthology.orchestra.run.smithi096:> sudo hostname $(hostname -s) 2024-04-25T14:01:44.309 DEBUG:teuthology.orchestra.run.smithi097:> sudo hostname $(hostname -s) 2024-04-25T14:01:44.332 DEBUG:teuthology.orchestra.run.smithi151:> sudo hostname $(hostname -s) 2024-04-25T14:01:44.356 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-25T14:01:44.356 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T14:01:44.492 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78872/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3291-g994e2f20', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3291.g994e2f20'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-25 04:02:18.990013', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240425.015853-debug', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-04-25T14:01:44.711 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-pdonnell-testing-20240425.015853-debug, sha1 994e2f20102a20fa9a27966c14b625cc01884df8 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T14:01:44.713 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/x86_64/flavors/default/cephadm 2024-04-25T14:01:44.713 DEBUG:teuthology.orchestra.run.smithi096:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T14:01:45.142 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-r--r--. 1 ubuntu ubuntu 781647 Apr 25 14:01 /home/ubuntu/cephtest/cephadm 2024-04-25T14:01:45.143 DEBUG:teuthology.orchestra.run.smithi097:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T14:01:45.549 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r--. 1 ubuntu ubuntu 781647 Apr 25 14:01 /home/ubuntu/cephtest/cephadm 2024-04-25T14:01:45.549 DEBUG:teuthology.orchestra.run.smithi151:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T14:01:46.313 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r--. 1 ubuntu ubuntu 781647 Apr 25 14:01 /home/ubuntu/cephtest/cephadm 2024-04-25T14:01:46.313 DEBUG:teuthology.orchestra.run.smithi096:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T14:01:46.332 DEBUG:teuthology.orchestra.run.smithi097:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T14:01:46.351 DEBUG:teuthology.orchestra.run.smithi151:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T14:01:46.387 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 on all hosts... 2024-04-25T14:01:46.387 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 pull 2024-04-25T14:01:46.389 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 pull 2024-04-25T14:01:46.393 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 pull 2024-04-25T14:01:47.989 INFO:teuthology.orchestra.run.smithi151.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8... 2024-04-25T14:01:48.028 INFO:teuthology.orchestra.run.smithi097.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8... 2024-04-25T14:01:48.052 INFO:teuthology.orchestra.run.smithi096.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8... 2024-04-25T14:02:50.368 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-25T14:02:50.368 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T14:02:50.368 INFO:teuthology.orchestra.run.smithi151.stdout: "image_id": "d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4", 2024-04-25T14:02:50.368 INFO:teuthology.orchestra.run.smithi151.stdout: "repo_digests": [ 2024-04-25T14:02:50.368 INFO:teuthology.orchestra.run.smithi151.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385" 2024-04-25T14:02:50.368 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-25T14:02:50.368 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-25T14:03:04.421 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T14:03:04.421 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T14:03:04.422 INFO:teuthology.orchestra.run.smithi096.stdout: "image_id": "d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4", 2024-04-25T14:03:04.422 INFO:teuthology.orchestra.run.smithi096.stdout: "repo_digests": [ 2024-04-25T14:03:04.422 INFO:teuthology.orchestra.run.smithi096.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385" 2024-04-25T14:03:04.422 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-25T14:03:04.422 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T14:03:06.741 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T14:03:06.742 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T14:03:06.742 INFO:teuthology.orchestra.run.smithi097.stdout: "image_id": "d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4", 2024-04-25T14:03:06.742 INFO:teuthology.orchestra.run.smithi097.stdout: "repo_digests": [ 2024-04-25T14:03:06.742 INFO:teuthology.orchestra.run.smithi097.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385" 2024-04-25T14:03:06.742 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-04-25T14:03:06.742 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T14:03:06.765 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /etc/ceph 2024-04-25T14:03:06.796 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /etc/ceph 2024-04-25T14:03:06.823 DEBUG:teuthology.orchestra.run.smithi151:> sudo mkdir -p /etc/ceph 2024-04-25T14:03:06.856 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 777 /etc/ceph 2024-04-25T14:03:06.883 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 777 /etc/ceph 2024-04-25T14:03:06.907 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 777 /etc/ceph 2024-04-25T14:03:06.935 INFO:tasks.cephadm:Writing seed config... 2024-04-25T14:03:06.936 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-25T14:03:06.936 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-25T14:03:06.936 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-25T14:03:06.936 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-25T14:03:06.936 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-25T14:03:06.936 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-25T14:03:06.936 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-25T14:03:06.936 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-25T14:03:06.936 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-25T14:03:06.936 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-25T14:03:06.937 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-25T14:03:06.937 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-25T14:03:06.937 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-25T14:03:06.937 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-25T14:03:06.937 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-25T14:03:06.937 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-25T14:03:06.937 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-25T14:03:06.937 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-04-25T14:03:06.937 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-25T14:03:06.937 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-25T14:03:06.937 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-25T14:03:06.937 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-25T14:03:06.937 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-25T14:03:06.937 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-25T14:03:06.937 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-25T14:03:06.937 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-25T14:03:06.938 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-25T14:03:06.938 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-25T14:03:06.938 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-25T14:03:06.938 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-25T14:03:06.938 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-25T14:03:06.938 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-04-25T14:03:06.938 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-25T14:03:06.938 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-25T14:03:06.938 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-25T14:03:06.938 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-25T14:03:06.938 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-25T14:03:06.938 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-25T14:03:06.938 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-25T14:03:06.938 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-25T14:03:06.939 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-25T14:03:06.939 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-25T14:03:06.939 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-25T14:03:06.939 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-25T14:03:06.939 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-25T14:03:06.939 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-25T14:03:06.940 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T14:03:06.940 DEBUG:teuthology.orchestra.run.smithi096:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-25T14:03:06.957 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 59334182-030c-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-25T14:03:06.957 DEBUG:teuthology.orchestra.run.smithi096:mon.a> sudo journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@mon.a.service 2024-04-25T14:03:07.000 DEBUG:teuthology.orchestra.run.smithi096:mgr.x> sudo journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@mgr.x.service 2024-04-25T14:03:07.042 INFO:tasks.cephadm:Bootstrapping... 2024-04-25T14:03:07.042 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 -v bootstrap --fsid 59334182-030c-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.96 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-25T14:03:07.238 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-04-25T14:03:07.238 INFO:teuthology.orchestra.run.smithi096.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8', '-v', 'bootstrap', '--fsid', '59334182-030c-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.96', '--skip-admin-label'] 2024-04-25T14:03:07.239 INFO:teuthology.orchestra.run.smithi096.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-25T14:03:07.239 INFO:teuthology.orchestra.run.smithi096.stdout:Verifying podman|docker is present... 2024-04-25T14:03:07.265 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T14:03:07.265 INFO:teuthology.orchestra.run.smithi096.stdout:Verifying lvm2 is present... 2024-04-25T14:03:07.266 INFO:teuthology.orchestra.run.smithi096.stdout:Verifying time synchronization is in place... 2024-04-25T14:03:07.273 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T14:03:07.273 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T14:03:07.279 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T14:03:07.279 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout inactive 2024-04-25T14:03:07.285 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout enabled 2024-04-25T14:03:07.291 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout active 2024-04-25T14:03:07.291 INFO:teuthology.orchestra.run.smithi096.stdout:Unit chronyd.service is enabled and running 2024-04-25T14:03:07.291 INFO:teuthology.orchestra.run.smithi096.stdout:Repeating the final host check... 2024-04-25T14:03:07.316 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T14:03:07.316 INFO:teuthology.orchestra.run.smithi096.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-25T14:03:07.316 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl is present 2024-04-25T14:03:07.316 INFO:teuthology.orchestra.run.smithi096.stdout:lvcreate is present 2024-04-25T14:03:07.322 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T14:03:07.322 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T14:03:07.328 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T14:03:07.328 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout inactive 2024-04-25T14:03:07.334 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout enabled 2024-04-25T14:03:07.340 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout active 2024-04-25T14:03:07.340 INFO:teuthology.orchestra.run.smithi096.stdout:Unit chronyd.service is enabled and running 2024-04-25T14:03:07.341 INFO:teuthology.orchestra.run.smithi096.stdout:Host looks OK 2024-04-25T14:03:07.341 INFO:teuthology.orchestra.run.smithi096.stdout:Cluster fsid: 59334182-030c-11ef-bc93-c7b262605968 2024-04-25T14:03:07.341 INFO:teuthology.orchestra.run.smithi096.stdout:Acquiring lock 140075637120160 on /run/cephadm/59334182-030c-11ef-bc93-c7b262605968.lock 2024-04-25T14:03:07.341 INFO:teuthology.orchestra.run.smithi096.stdout:Lock 140075637120160 acquired on /run/cephadm/59334182-030c-11ef-bc93-c7b262605968.lock 2024-04-25T14:03:07.341 INFO:teuthology.orchestra.run.smithi096.stdout:Verifying IP 172.21.15.96 port 3300 ... 2024-04-25T14:03:07.341 INFO:teuthology.orchestra.run.smithi096.stdout:Verifying IP 172.21.15.96 port 6789 ... 2024-04-25T14:03:07.342 INFO:teuthology.orchestra.run.smithi096.stdout:Base mon IP(s) is [172.21.15.96:3300, 172.21.15.96:6789], mon addrv is [v2:172.21.15.96:3300,v1:172.21.15.96:6789] 2024-04-25T14:03:07.344 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.96 metric 100 2024-04-25T14:03:07.345 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.96 metric 100 2024-04-25T14:03:07.347 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-25T14:03:07.347 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-04-25T14:03:07.347 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-04-25T14:03:07.349 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-25T14:03:07.349 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-25T14:03:07.350 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T14:03:07.350 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-25T14:03:07.350 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout inet6 fe80::90f7:1abc:dbc0:da5a/64 scope link noprefixroute 2024-04-25T14:03:07.350 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T14:03:07.350 INFO:teuthology.orchestra.run.smithi096.stdout:Mon IP `172.21.15.96` is in CIDR network `172.21.0.0/20` 2024-04-25T14:03:07.350 INFO:teuthology.orchestra.run.smithi096.stdout:Mon IP `172.21.15.96` is in CIDR network `172.21.0.0/20` 2024-04-25T14:03:07.351 INFO:teuthology.orchestra.run.smithi096.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-25T14:03:07.351 INFO:teuthology.orchestra.run.smithi096.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-25T14:03:07.352 INFO:teuthology.orchestra.run.smithi096.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8... 2024-04-25T14:03:07.873 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/podman: stdout d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 2024-04-25T14:03:07.873 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8... 2024-04-25T14:03:07.873 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/podman: stderr Getting image source signatures 2024-04-25T14:03:07.873 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/podman: stderr Copying blob sha256:665339d96aad80d083ba9988a62422bcef0a42210fb2b28340e33ab98589f7b8 2024-04-25T14:03:07.873 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/podman: stderr Copying blob sha256:3d665f4c032644be9cae748af04bee69f2bfe96cdc3616dabe0ab17899633ab5 2024-04-25T14:03:07.873 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/podman: stderr Copying config sha256:d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 2024-04-25T14:03:07.873 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-25T14:03:08.786 INFO:teuthology.orchestra.run.smithi096.stdout:ceph: stdout ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev) 2024-04-25T14:03:08.786 INFO:teuthology.orchestra.run.smithi096.stdout:Ceph version: ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev) 2024-04-25T14:03:08.786 INFO:teuthology.orchestra.run.smithi096.stdout:Extracting ceph user uid/gid from container image... 2024-04-25T14:03:09.790 INFO:teuthology.orchestra.run.smithi096.stdout:stat: stdout 167 167 2024-04-25T14:03:09.790 INFO:teuthology.orchestra.run.smithi096.stdout:Creating initial keys... 2024-04-25T14:03:10.646 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph-authtool: stdout AQCeYipmHMhaBRAAs90y73dS6FgnmRQgfYTA8w== 2024-04-25T14:03:11.503 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph-authtool: stdout AQCeYipmcVlcOhAAqDrPCo+PnPq9mM7Yqp/E1Q== 2024-04-25T14:03:12.441 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph-authtool: stdout AQCfYipmu8FFMhAAxCtvK9L1texstMNDA/iEig== 2024-04-25T14:03:12.441 INFO:teuthology.orchestra.run.smithi096.stdout:Creating initial monmap... 2024-04-25T14:03:19.756 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T14:03:19.756 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-25T14:03:19.758 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 59334182-030c-11ef-bc93-c7b262605968 2024-04-25T14:03:19.758 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T14:03:19.758 INFO:teuthology.orchestra.run.smithi096.stdout:monmaptool for a [v2:172.21.15.96:3300,v1:172.21.15.96:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T14:03:19.758 INFO:teuthology.orchestra.run.smithi096.stdout:setting min_mon_release = quincy 2024-04-25T14:03:19.758 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/monmaptool: set fsid to 59334182-030c-11ef-bc93-c7b262605968 2024-04-25T14:03:19.758 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T14:03:19.758 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:03:19.759 INFO:teuthology.orchestra.run.smithi096.stdout:Creating mon... 2024-04-25T14:03:24.188 INFO:teuthology.orchestra.run.smithi096.stdout:create mon.a on 2024-04-25T14:03:24.398 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-25T14:03:24.534 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-25T14:03:24.700 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-59334182-030c-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-59334182-030c-11ef-bc93-c7b262605968.target. 2024-04-25T14:03:24.701 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-59334182-030c-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-59334182-030c-11ef-bc93-c7b262605968.target. 2024-04-25T14:03:24.937 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-59334182-030c-11ef-bc93-c7b262605968@mon.a 2024-04-25T14:03:24.938 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Failed to reset failed state of unit ceph-59334182-030c-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-59334182-030c-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-25T14:03:25.100 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-59334182-030c-11ef-bc93-c7b262605968.target.wants/ceph-59334182-030c-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-59334182-030c-11ef-bc93-c7b262605968@.service. 2024-04-25T14:03:25.463 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:25 smithi096 podman[28818]: 2024-04-25 14:03:25.445703512 +0000 UTC m=+0.096819630 container create ed8f63849bb17b30f3d814f6d1d698c6b7f38d8453ba40741a094f3628c2cf8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424) 2024-04-25T14:03:25.463 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:25 smithi096 podman[28818]: 2024-04-25 14:03:25.365948557 +0000 UTC m=+0.017064689 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T14:03:25.710 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T14:03:25.710 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout disabled 2024-04-25T14:03:25.717 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T14:03:25.717 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout inactive 2024-04-25T14:03:25.717 INFO:teuthology.orchestra.run.smithi096.stdout:firewalld.service is not enabled 2024-04-25T14:03:25.717 INFO:teuthology.orchestra.run.smithi096.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T14:03:25.717 INFO:teuthology.orchestra.run.smithi096.stdout:Waiting for mon to start... 2024-04-25T14:03:25.718 INFO:teuthology.orchestra.run.smithi096.stdout:Waiting for mon... 2024-04-25T14:03:25.840 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:25 smithi096 podman[28818]: 2024-04-25 14:03:25.623899135 +0000 UTC m=+0.275015255 container init ed8f63849bb17b30f3d814f6d1d698c6b7f38d8453ba40741a094f3628c2cf8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T14:03:25.840 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:25 smithi096 podman[28818]: 2024-04-25 14:03:25.627148639 +0000 UTC m=+0.278264758 container start ed8f63849bb17b30f3d814f6d1d698c6b7f38d8453ba40741a094f3628c2cf8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mon-a, ceph=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-25T14:03:25.840 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:25 smithi096 ceph-mon[28836]: mkfs 59334182-030c-11ef-bc93-c7b262605968 2024-04-25T14:03:25.840 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:25 smithi096 ceph-mon[28836]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T14:03:25.840 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:25 smithi096 bash[28818]: ed8f63849bb17b30f3d814f6d1d698c6b7f38d8453ba40741a094f3628c2cf8f 2024-04-25T14:03:25.840 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:25 smithi096 systemd[1]: Started Ceph mon.a for 59334182-030c-11ef-bc93-c7b262605968. 2024-04-25T14:03:26.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:26 smithi096 ceph-mon[28836]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T14:03:26.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:26 smithi096 ceph-mon[28836]: monmap epoch 1 2024-04-25T14:03:26.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:26 smithi096 ceph-mon[28836]: fsid 59334182-030c-11ef-bc93-c7b262605968 2024-04-25T14:03:26.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:26 smithi096 ceph-mon[28836]: last_changed 2024-04-25T14:03:12.788276+0000 2024-04-25T14:03:26.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:26 smithi096 ceph-mon[28836]: created 2024-04-25T14:03:12.788276+0000 2024-04-25T14:03:26.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:26 smithi096 ceph-mon[28836]: min_mon_release 19 (squid) 2024-04-25T14:03:26.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:26 smithi096 ceph-mon[28836]: election_strategy: 1 2024-04-25T14:03:26.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:26 smithi096 ceph-mon[28836]: 0: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.a 2024-04-25T14:03:26.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:26 smithi096 ceph-mon[28836]: fsmap 2024-04-25T14:03:26.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:26 smithi096 ceph-mon[28836]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T14:03:26.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:26 smithi096 ceph-mon[28836]: mgrmap e1: no daemons active 2024-04-25T14:03:26.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:26 smithi096 ceph-mon[28836]: from='client.? 172.21.15.96:0/1443737626' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-25T14:03:26.886 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout cluster: 2024-04-25T14:03:26.892 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout id: 59334182-030c-11ef-bc93-c7b262605968 2024-04-25T14:03:26.892 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-25T14:03:26.892 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-04-25T14:03:26.892 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout services: 2024-04-25T14:03:26.892 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.548822s) 2024-04-25T14:03:26.892 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-25T14:03:26.892 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-25T14:03:26.893 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-04-25T14:03:26.893 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout data: 2024-04-25T14:03:26.893 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-25T14:03:26.893 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-25T14:03:26.893 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-25T14:03:26.893 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout pgs: 2024-04-25T14:03:26.893 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-04-25T14:03:26.893 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:26.152+0000 7f70949d5700 1 Processor -- start 2024-04-25T14:03:26.893 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:26.152+0000 7f70949d5700 1 -- start start 2024-04-25T14:03:26.893 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:26.152+0000 7f70949d5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f70901088a0 0x7f7090108cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:26.893 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:26.152+0000 7f70949d5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f70901092a0 con 0x7f70901088a0 2024-04-25T14:03:26.893 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:26.152+0000 7f708e59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f70901088a0 0x7f7090108cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:26.893 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:26.152+0000 7f708e59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f70901088a0 0x7f7090108cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43800/0 (socket says 172.21.15.96:43800) 2024-04-25T14:03:26.894 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:26.152+0000 7f708e59c700 1 -- 172.21.15.96:0/350217566 learned_addr learned my addr 172.21.15.96:0/350217566 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:03:26.894 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:26.153+0000 7f708e59c700 1 -- 172.21.15.96:0/350217566 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7090109a70 con 0x7f70901088a0 2024-04-25T14:03:26.894 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:26.153+0000 7f708e59c700 1 --2- 172.21.15.96:0/350217566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f70901088a0 0x7f7090108cb0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f70840098a0 tx=0x7f7084009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfeca3df3245e13a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:26.894 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:26.153+0000 7f708d59a700 1 -- 172.21.15.96:0/350217566 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7084004160 con 0x7f70901088a0 2024-04-25T14:03:26.894 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:26.153+0000 7f708d59a700 1 -- 172.21.15.96:0/350217566 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7084004300 con 0x7f70901088a0 2024-04-25T14:03:26.894 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:26.153+0000 7f708d59a700 1 -- 172.21.15.96:0/350217566 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70840036a0 con 0x7f70901088a0 2024-04-25T14:03:26.898 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:26.154+0000 7f70949d5700 1 -- 172.21.15.96:0/350217566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f70901088a0 msgr2=0x7f7090108cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:26.898 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:26.154+0000 7f70949d5700 1 --2- 172.21.15.96:0/350217566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f70901088a0 0x7f7090108cb0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f70840098a0 tx=0x7f7084009bb0 comp rx=0 tx=0).stop 2024-04-25T14:03:26.898 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:26.154+0000 7f70949d5700 1 -- 172.21.15.96:0/350217566 shutdown_connections 2024-04-25T14:03:26.898 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:26.154+0000 7f70949d5700 1 --2- 172.21.15.96:0/350217566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f70901088a0 0x7f7090108cb0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:26.899 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:26.154+0000 7f70949d5700 1 -- 172.21.15.96:0/350217566 >> 172.21.15.96:0/350217566 conn(0x7f7090103f60 msgr2=0x7f70901063b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:26.899 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:26.154+0000 7f70949d5700 1 -- 172.21.15.96:0/350217566 shutdown_connections 2024-04-25T14:03:26.899 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:26.154+0000 7f70949d5700 1 -- 172.21.15.96:0/350217566 wait complete. 2024-04-25T14:03:26.899 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:26.155+0000 7f70949d5700 1 Processor -- start 2024-04-25T14:03:26.899 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:26.155+0000 7f70949d5700 1 -- start start 2024-04-25T14:03:26.899 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:26.155+0000 7f70949d5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f70901088a0 0x7f709007d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:26.899 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:26.155+0000 7f70949d5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7090109fa0 con 0x7f70901088a0 2024-04-25T14:03:26.899 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:26.155+0000 7f708e59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f70901088a0 0x7f709007d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:26.899 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:26.155+0000 7f708e59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f70901088a0 0x7f709007d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43808/0 (socket says 172.21.15.96:43808) 2024-04-25T14:03:26.899 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:26.155+0000 7f708e59c700 1 -- 172.21.15.96:0/1443737626 learned_addr learned my addr 172.21.15.96:0/1443737626 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:03:26.900 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:26.155+0000 7f708e59c700 1 -- 172.21.15.96:0/1443737626 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f709007d6a0 con 0x7f70901088a0 2024-04-25T14:03:26.900 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:26.155+0000 7f708e59c700 1 --2- 172.21.15.96:0/1443737626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f70901088a0 0x7f709007d160 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7084000c00 tx=0x7f7084004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:26.900 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:26.156+0000 7f707f7fe700 1 -- 172.21.15.96:0/1443737626 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7084003db0 con 0x7f70901088a0 2024-04-25T14:03:26.900 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:26.156+0000 7f707f7fe700 1 -- 172.21.15.96:0/1443737626 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7084011630 con 0x7f70901088a0 2024-04-25T14:03:26.900 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:26.156+0000 7f707f7fe700 1 -- 172.21.15.96:0/1443737626 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7084011950 con 0x7f70901088a0 2024-04-25T14:03:26.900 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:26.156+0000 7f70949d5700 1 -- 172.21.15.96:0/1443737626 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70901a01d0 con 0x7f70901088a0 2024-04-25T14:03:26.900 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:26.156+0000 7f70949d5700 1 -- 172.21.15.96:0/1443737626 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70901a0650 con 0x7f70901088a0 2024-04-25T14:03:26.900 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:26.156+0000 7f707f7fe700 1 -- 172.21.15.96:0/1443737626 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7084011af0 con 0x7f70901088a0 2024-04-25T14:03:26.900 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:26.156+0000 7f707f7fe700 1 -- 172.21.15.96:0/1443737626 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f708401c450 con 0x7f70901088a0 2024-04-25T14:03:26.900 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:26.157+0000 7f70949d5700 1 -- 172.21.15.96:0/1443737626 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7090064cf0 con 0x7f70901088a0 2024-04-25T14:03:26.900 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:26.159+0000 7f707f7fe700 1 -- 172.21.15.96:0/1443737626 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f7084004260 con 0x7f70901088a0 2024-04-25T14:03:26.900 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:26.228+0000 7f70949d5700 1 -- 172.21.15.96:0/1443737626 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f70900642a0 con 0x7f70901088a0 2024-04-25T14:03:26.900 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:26.228+0000 7f707f7fe700 1 -- 172.21.15.96:0/1443737626 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f708401b450 con 0x7f70901088a0 2024-04-25T14:03:26.901 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:26.229+0000 7f70949d5700 1 -- 172.21.15.96:0/1443737626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f70901088a0 msgr2=0x7f709007d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:26.901 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:26.229+0000 7f70949d5700 1 --2- 172.21.15.96:0/1443737626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f70901088a0 0x7f709007d160 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7084000c00 tx=0x7f7084004750 comp rx=0 tx=0).stop 2024-04-25T14:03:26.901 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:26.229+0000 7f70949d5700 1 -- 172.21.15.96:0/1443737626 shutdown_connections 2024-04-25T14:03:26.901 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:26.229+0000 7f70949d5700 1 --2- 172.21.15.96:0/1443737626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f70901088a0 0x7f709007d160 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:26.901 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:26.229+0000 7f70949d5700 1 -- 172.21.15.96:0/1443737626 >> 172.21.15.96:0/1443737626 conn(0x7f7090103f60 msgr2=0x7f7090192a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:26.901 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:26.229+0000 7f70949d5700 1 -- 172.21.15.96:0/1443737626 shutdown_connections 2024-04-25T14:03:26.901 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:26.229+0000 7f70949d5700 1 -- 172.21.15.96:0/1443737626 wait complete. 2024-04-25T14:03:26.901 INFO:teuthology.orchestra.run.smithi096.stdout:mon is available 2024-04-25T14:03:26.901 INFO:teuthology.orchestra.run.smithi096.stdout:Assimilating anything we can from ceph.conf... 2024-04-25T14:03:27.964 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:27 smithi096 ceph-mon[28836]: from='client.? 172.21.15.96:0/3303128185' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T14:03:27.964 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:27 smithi096 ceph-mon[28836]: from='client.? 172.21.15.96:0/3303128185' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-25T14:03:27.972 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-04-25T14:03:27.972 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T14:03:27.973 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout fsid = 59334182-030c-11ef-bc93-c7b262605968 2024-04-25T14:03:27.973 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T14:03:27.973 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.96:3300,v1:172.21.15.96:6789] 2024-04-25T14:03:27.973 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T14:03:27.973 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T14:03:27.973 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T14:03:27.973 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T14:03:27.973 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-04-25T14:03:27.973 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T14:03:27.973 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T14:03:27.973 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-04-25T14:03:27.973 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T14:03:27.973 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T14:03:27.973 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T14:03:27.974 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T14:03:27.974 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T14:03:27.974 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T14:03:27.974 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:27.352+0000 7efd772c1700 1 Processor -- start 2024-04-25T14:03:27.974 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:27.353+0000 7efd772c1700 1 -- start start 2024-04-25T14:03:27.974 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:27.353+0000 7efd772c1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd70108880 0x7efd70108c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:27.974 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:27.353+0000 7efd772c1700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efd70109280 con 0x7efd70108880 2024-04-25T14:03:27.974 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:27.353+0000 7efd7505d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd70108880 0x7efd70108c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:27.974 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:27.353+0000 7efd7505d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd70108880 0x7efd70108c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43812/0 (socket says 172.21.15.96:43812) 2024-04-25T14:03:27.974 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:27.353+0000 7efd7505d700 1 -- 172.21.15.96:0/607748705 learned_addr learned my addr 172.21.15.96:0/607748705 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:03:27.974 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:27.353+0000 7efd7505d700 1 -- 172.21.15.96:0/607748705 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd70109a50 con 0x7efd70108880 2024-04-25T14:03:27.974 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:27.353+0000 7efd7505d700 1 --2- 172.21.15.96:0/607748705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd70108880 0x7efd70108c90 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7efd6c0098a0 tx=0x7efd6c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b8f6cbb9c1becb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:27.974 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:27.354+0000 7efd63fff700 1 -- 172.21.15.96:0/607748705 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd6c004160 con 0x7efd70108880 2024-04-25T14:03:27.975 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:27.354+0000 7efd63fff700 1 -- 172.21.15.96:0/607748705 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd6c004300 con 0x7efd70108880 2024-04-25T14:03:27.975 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:27.354+0000 7efd63fff700 1 -- 172.21.15.96:0/607748705 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd6c0036a0 con 0x7efd70108880 2024-04-25T14:03:27.975 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:27.354+0000 7efd772c1700 1 -- 172.21.15.96:0/607748705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd70108880 msgr2=0x7efd70108c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:27.975 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:27.354+0000 7efd772c1700 1 --2- 172.21.15.96:0/607748705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd70108880 0x7efd70108c90 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7efd6c0098a0 tx=0x7efd6c009bb0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.975 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:27.354+0000 7efd772c1700 1 -- 172.21.15.96:0/607748705 shutdown_connections 2024-04-25T14:03:27.975 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:27.354+0000 7efd772c1700 1 --2- 172.21.15.96:0/607748705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd70108880 0x7efd70108c90 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.975 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:27.354+0000 7efd772c1700 1 -- 172.21.15.96:0/607748705 >> 172.21.15.96:0/607748705 conn(0x7efd70103f40 msgr2=0x7efd70106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:27.975 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:27.354+0000 7efd772c1700 1 -- 172.21.15.96:0/607748705 shutdown_connections 2024-04-25T14:03:27.975 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:27.355+0000 7efd772c1700 1 -- 172.21.15.96:0/607748705 wait complete. 2024-04-25T14:03:27.975 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:27.355+0000 7efd772c1700 1 Processor -- start 2024-04-25T14:03:27.975 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:27.355+0000 7efd772c1700 1 -- start start 2024-04-25T14:03:27.975 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:27.355+0000 7efd772c1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd70108880 0x7efd7019d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:27.975 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:27.355+0000 7efd772c1700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efd70109f80 con 0x7efd70108880 2024-04-25T14:03:27.975 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:27.355+0000 7efd7505d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd70108880 0x7efd7019d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:27.975 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:27.356+0000 7efd7505d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd70108880 0x7efd7019d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43826/0 (socket says 172.21.15.96:43826) 2024-04-25T14:03:27.976 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:27.356+0000 7efd7505d700 1 -- 172.21.15.96:0/3303128185 learned_addr learned my addr 172.21.15.96:0/3303128185 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:03:27.976 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:27.356+0000 7efd7505d700 1 -- 172.21.15.96:0/3303128185 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd7019d970 con 0x7efd70108880 2024-04-25T14:03:27.976 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:27.356+0000 7efd7505d700 1 --2- 172.21.15.96:0/3303128185 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd70108880 0x7efd7019d430 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7efd6c000c00 tx=0x7efd6c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:27.976 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:27.356+0000 7efd627fc700 1 -- 172.21.15.96:0/3303128185 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd6c003db0 con 0x7efd70108880 2024-04-25T14:03:27.976 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:27.356+0000 7efd627fc700 1 -- 172.21.15.96:0/3303128185 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd6c011630 con 0x7efd70108880 2024-04-25T14:03:27.976 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:27.356+0000 7efd772c1700 1 -- 172.21.15.96:0/3303128185 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd7019dc00 con 0x7efd70108880 2024-04-25T14:03:27.976 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:27.356+0000 7efd627fc700 1 -- 172.21.15.96:0/3303128185 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd6c011950 con 0x7efd70108880 2024-04-25T14:03:27.976 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:27.356+0000 7efd772c1700 1 -- 172.21.15.96:0/3303128185 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd701a0810 con 0x7efd70108880 2024-04-25T14:03:27.976 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:27.356+0000 7efd627fc700 1 -- 172.21.15.96:0/3303128185 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7efd6c011af0 con 0x7efd70108880 2024-04-25T14:03:27.976 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:27.356+0000 7efd627fc700 1 -- 172.21.15.96:0/3303128185 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7efd6c01c450 con 0x7efd70108880 2024-04-25T14:03:27.976 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:27.357+0000 7efd772c1700 1 -- 172.21.15.96:0/3303128185 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd70064cf0 con 0x7efd70108880 2024-04-25T14:03:27.976 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:27.359+0000 7efd627fc700 1 -- 172.21.15.96:0/3303128185 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7efd6c004260 con 0x7efd70108880 2024-04-25T14:03:27.976 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:27.426+0000 7efd772c1700 1 -- 172.21.15.96:0/3303128185 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7efd7002e810 con 0x7efd70108880 2024-04-25T14:03:27.976 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:27.430+0000 7efd627fc700 1 -- 172.21.15.96:0/3303128185 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+465 (secure 0 0 0) 0x7efd6c01b450 con 0x7efd70108880 2024-04-25T14:03:27.976 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:27.431+0000 7efd772c1700 1 -- 172.21.15.96:0/3303128185 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd70108880 msgr2=0x7efd7019d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:27.977 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:27.431+0000 7efd627fc700 1 -- 172.21.15.96:0/3303128185 <== mon.0 v2:172.21.15.96:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7efd6c01b630 con 0x7efd70108880 2024-04-25T14:03:27.977 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:27.431+0000 7efd772c1700 1 --2- 172.21.15.96:0/3303128185 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd70108880 0x7efd7019d430 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7efd6c000c00 tx=0x7efd6c004750 comp rx=0 tx=0).stop 2024-04-25T14:03:27.977 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:27.431+0000 7efd772c1700 1 -- 172.21.15.96:0/3303128185 shutdown_connections 2024-04-25T14:03:27.977 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:27.431+0000 7efd772c1700 1 --2- 172.21.15.96:0/3303128185 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd70108880 0x7efd7019d430 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.977 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:27.431+0000 7efd772c1700 1 -- 172.21.15.96:0/3303128185 >> 172.21.15.96:0/3303128185 conn(0x7efd70103f40 msgr2=0x7efd70193ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:27.977 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:27.431+0000 7efd772c1700 1 -- 172.21.15.96:0/3303128185 shutdown_connections 2024-04-25T14:03:27.977 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:27.431+0000 7efd772c1700 1 -- 172.21.15.96:0/3303128185 wait complete. 2024-04-25T14:03:27.977 INFO:teuthology.orchestra.run.smithi096.stdout:Generating new minimal ceph.conf... 2024-04-25T14:03:29.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:28 smithi096 ceph-mon[28836]: from='client.? 172.21.15.96:0/1724593781' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:03:30.337 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:28.397+0000 7fb083ad2700 1 Processor -- start 2024-04-25T14:03:30.338 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:28.397+0000 7fb083ad2700 1 -- start start 2024-04-25T14:03:30.338 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:28.397+0000 7fb083ad2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb07c1082f0 0x7fb07c108700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:30.338 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:28.397+0000 7fb083ad2700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb07c07d610 con 0x7fb07c1082f0 2024-04-25T14:03:30.338 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:28.398+0000 7fb08186e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb07c1082f0 0x7fb07c108700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:30.338 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:28.398+0000 7fb08186e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb07c1082f0 0x7fb07c108700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43838/0 (socket says 172.21.15.96:43838) 2024-04-25T14:03:30.338 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:28.398+0000 7fb08186e700 1 -- 172.21.15.96:0/892468521 learned_addr learned my addr 172.21.15.96:0/892468521 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:03:30.338 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:28.398+0000 7fb08186e700 1 -- 172.21.15.96:0/892468521 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb07c108c40 con 0x7fb07c1082f0 2024-04-25T14:03:30.338 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:28.398+0000 7fb08186e700 1 --2- 172.21.15.96:0/892468521 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb07c1082f0 0x7fb07c108700 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb0780098a0 tx=0x7fb078009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffdfa85ae9d113e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:30.338 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:28.398+0000 7fb08086c700 1 -- 172.21.15.96:0/892468521 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb078004160 con 0x7fb07c1082f0 2024-04-25T14:03:30.338 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:28.398+0000 7fb08086c700 1 -- 172.21.15.96:0/892468521 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb078004300 con 0x7fb07c1082f0 2024-04-25T14:03:30.338 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:28.398+0000 7fb08086c700 1 -- 172.21.15.96:0/892468521 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb078011af0 con 0x7fb07c1082f0 2024-04-25T14:03:30.339 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:28.399+0000 7fb083ad2700 1 -- 172.21.15.96:0/892468521 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb07c1082f0 msgr2=0x7fb07c108700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:30.339 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:28.399+0000 7fb083ad2700 1 --2- 172.21.15.96:0/892468521 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb07c1082f0 0x7fb07c108700 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb0780098a0 tx=0x7fb078009bb0 comp rx=0 tx=0).stop 2024-04-25T14:03:30.339 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:28.399+0000 7fb083ad2700 1 -- 172.21.15.96:0/892468521 shutdown_connections 2024-04-25T14:03:30.339 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:28.399+0000 7fb083ad2700 1 --2- 172.21.15.96:0/892468521 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb07c1082f0 0x7fb07c108700 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:30.339 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:28.399+0000 7fb083ad2700 1 -- 172.21.15.96:0/892468521 >> 172.21.15.96:0/892468521 conn(0x7fb07c103f60 msgr2=0x7fb07c1063b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:30.339 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:28.399+0000 7fb083ad2700 1 -- 172.21.15.96:0/892468521 shutdown_connections 2024-04-25T14:03:30.339 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:28.399+0000 7fb083ad2700 1 -- 172.21.15.96:0/892468521 wait complete. 2024-04-25T14:03:30.339 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:28.400+0000 7fb083ad2700 1 Processor -- start 2024-04-25T14:03:30.339 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:28.400+0000 7fb083ad2700 1 -- start start 2024-04-25T14:03:30.339 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:28.400+0000 7fb083ad2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb07c1082f0 0x7fb07c19ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:30.339 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:28.400+0000 7fb083ad2700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb07c109cc0 con 0x7fb07c1082f0 2024-04-25T14:03:30.339 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:28.400+0000 7fb08186e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb07c1082f0 0x7fb07c19ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:30.339 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:28.400+0000 7fb08186e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb07c1082f0 0x7fb07c19ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43848/0 (socket says 172.21.15.96:43848) 2024-04-25T14:03:30.339 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:28.400+0000 7fb08186e700 1 -- 172.21.15.96:0/1724593781 learned_addr learned my addr 172.21.15.96:0/1724593781 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:03:30.340 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:28.400+0000 7fb08186e700 1 -- 172.21.15.96:0/1724593781 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb07c1a0520 con 0x7fb07c1082f0 2024-04-25T14:03:30.340 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:28.400+0000 7fb08186e700 1 --2- 172.21.15.96:0/1724593781 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb07c1082f0 0x7fb07c19ffe0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb0780117b0 tx=0x7fb078011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:30.340 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:28.400+0000 7fb072ffd700 1 -- 172.21.15.96:0/1724593781 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb078003af0 con 0x7fb07c1082f0 2024-04-25T14:03:30.340 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:28.401+0000 7fb072ffd700 1 -- 172.21.15.96:0/1724593781 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb078020070 con 0x7fb07c1082f0 2024-04-25T14:03:30.340 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:28.401+0000 7fb072ffd700 1 -- 172.21.15.96:0/1724593781 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb078019630 con 0x7fb07c1082f0 2024-04-25T14:03:30.340 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:28.401+0000 7fb083ad2700 1 -- 172.21.15.96:0/1724593781 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb07c1a07b0 con 0x7fb07c1082f0 2024-04-25T14:03:30.340 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:28.401+0000 7fb083ad2700 1 -- 172.21.15.96:0/1724593781 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb07c1a11f0 con 0x7fb07c1082f0 2024-04-25T14:03:30.340 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:28.401+0000 7fb072ffd700 1 -- 172.21.15.96:0/1724593781 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb078019c30 con 0x7fb07c1082f0 2024-04-25T14:03:30.340 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:28.401+0000 7fb072ffd700 1 -- 172.21.15.96:0/1724593781 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb07800b720 con 0x7fb07c1082f0 2024-04-25T14:03:30.340 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:28.402+0000 7fb083ad2700 1 -- 172.21.15.96:0/1724593781 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb07c064cf0 con 0x7fb07c1082f0 2024-04-25T14:03:30.340 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:28.404+0000 7fb072ffd700 1 -- 172.21.15.96:0/1724593781 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fb07801a9c0 con 0x7fb07c1082f0 2024-04-25T14:03:30.340 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:28.471+0000 7fb083ad2700 1 -- 172.21.15.96:0/1724593781 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fb07c1a1e90 con 0x7fb07c1082f0 2024-04-25T14:03:30.340 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:28.471+0000 7fb072ffd700 1 -- 172.21.15.96:0/1724593781 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fb07801a430 con 0x7fb07c1082f0 2024-04-25T14:03:30.341 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:28.472+0000 7fb083ad2700 1 -- 172.21.15.96:0/1724593781 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb07c1082f0 msgr2=0x7fb07c19ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:30.341 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:28.472+0000 7fb083ad2700 1 --2- 172.21.15.96:0/1724593781 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb07c1082f0 0x7fb07c19ffe0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb0780117b0 tx=0x7fb078011b70 comp rx=0 tx=0).stop 2024-04-25T14:03:30.341 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:28.472+0000 7fb083ad2700 1 -- 172.21.15.96:0/1724593781 shutdown_connections 2024-04-25T14:03:30.341 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:28.472+0000 7fb083ad2700 1 --2- 172.21.15.96:0/1724593781 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb07c1082f0 0x7fb07c19ffe0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:30.341 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:28.472+0000 7fb083ad2700 1 -- 172.21.15.96:0/1724593781 >> 172.21.15.96:0/1724593781 conn(0x7fb07c103f60 msgr2=0x7fb07c193e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:30.341 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:28.472+0000 7fb083ad2700 1 -- 172.21.15.96:0/1724593781 shutdown_connections 2024-04-25T14:03:30.341 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:28.472+0000 7fb083ad2700 1 -- 172.21.15.96:0/1724593781 wait complete. 2024-04-25T14:03:30.341 INFO:teuthology.orchestra.run.smithi096.stdout:Restarting the monitor... 2024-04-25T14:03:30.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:30 smithi096 systemd[1]: Stopping Ceph mon.a for 59334182-030c-11ef-bc93-c7b262605968... 2024-04-25T14:03:31.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:30 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mon-a[28832]: 2024-04-25T14:03:30.616+0000 7feca9e15700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:03:31.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:30 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mon-a[28832]: 2024-04-25T14:03:30.616+0000 7feca9e15700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-25T14:03:31.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:30 smithi096 podman[29080]: 2024-04-25 14:03:30.740273295 +0000 UTC m=+0.234339618 container died ed8f63849bb17b30f3d814f6d1d698c6b7f38d8453ba40741a094f3628c2cf8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-25T14:03:31.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:31 smithi096 podman[29080]: 2024-04-25 14:03:31.567805305 +0000 UTC m=+1.061871670 container cleanup ed8f63849bb17b30f3d814f6d1d698c6b7f38d8453ba40741a094f3628c2cf8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-25T14:03:31.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:31 smithi096 bash[29080]: ceph-59334182-030c-11ef-bc93-c7b262605968-mon-a 2024-04-25T14:03:32.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:31 smithi096 podman[29094]: 2024-04-25 14:03:31.752013894 +0000 UTC m=+1.014915995 container remove ed8f63849bb17b30f3d814f6d1d698c6b7f38d8453ba40741a094f3628c2cf8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-25T14:03:32.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:31 smithi096 systemd[1]: ceph-59334182-030c-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-25T14:03:32.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:31 smithi096 systemd[1]: Stopped Ceph mon.a for 59334182-030c-11ef-bc93-c7b262605968. 2024-04-25T14:03:32.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:31 smithi096 systemd[1]: Starting Ceph mon.a for 59334182-030c-11ef-bc93-c7b262605968... 2024-04-25T14:03:32.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 podman[29188]: 2024-04-25 14:03:32.186097403 +0000 UTC m=+0.096207379 container create 18d82172990ba4676e789e5d85a7d94bce2856d86216b79876a2f7939b23c04b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-25T14:03:32.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 podman[29188]: 2024-04-25 14:03:32.108606436 +0000 UTC m=+0.018716425 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T14:03:32.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 podman[29188]: 2024-04-25 14:03:32.363532948 +0000 UTC m=+0.273642924 container init 18d82172990ba4676e789e5d85a7d94bce2856d86216b79876a2f7939b23c04b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=) 2024-04-25T14:03:32.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 podman[29188]: 2024-04-25 14:03:32.366452629 +0000 UTC m=+0.276562605 container start 18d82172990ba4676e789e5d85a7d94bce2856d86216b79876a2f7939b23c04b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-25T14:03:32.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T14:03:32.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev), process ceph-mon, pid 2 2024-04-25T14:03:32.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: pidfile_write: ignore empty --pid-file 2024-04-25T14:03:32.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: load: jerasure load: lrc 2024-04-25T14:03:32.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: RocksDB version: 7.9.2 2024-04-25T14:03:32.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Git sha 0 2024-04-25T14:03:32.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Compile date 2024-04-25 02:57:34 2024-04-25T14:03:32.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: DB SUMMARY 2024-04-25T14:03:32.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: DB Session ID: IWH7IDULXP127IN9Y32T 2024-04-25T14:03:32.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: CURRENT file: CURRENT 2024-04-25T14:03:32.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T14:03:32.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-25T14:03:32.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-25T14:03:32.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104646 ; 2024-04-25T14:03:32.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.error_if_exists: 0 2024-04-25T14:03:32.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.create_if_missing: 0 2024-04-25T14:03:32.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.paranoid_checks: 1 2024-04-25T14:03:32.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T14:03:32.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T14:03:32.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T14:03:32.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.env: 0x55c81cab7500 2024-04-25T14:03:32.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T14:03:32.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.info_log: 0x55c81d723fc0 2024-04-25T14:03:32.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T14:03:32.441 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.statistics: (nil) 2024-04-25T14:03:32.441 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.use_fsync: 0 2024-04-25T14:03:32.441 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.max_log_file_size: 0 2024-04-25T14:03:32.441 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T14:03:32.441 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T14:03:32.441 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T14:03:32.441 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T14:03:32.441 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.allow_fallocate: 1 2024-04-25T14:03:32.441 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T14:03:32.441 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T14:03:32.441 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.use_direct_reads: 0 2024-04-25T14:03:32.441 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T14:03:32.441 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T14:03:32.441 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.db_log_dir: 2024-04-25T14:03:32.442 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.wal_dir: 2024-04-25T14:03:32.442 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T14:03:32.442 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T14:03:32.442 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T14:03:32.442 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T14:03:32.442 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T14:03:32.442 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T14:03:32.442 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T14:03:32.442 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T14:03:32.442 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.write_buffer_manager: 0x55c81d6f48c0 2024-04-25T14:03:32.442 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T14:03:32.442 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T14:03:32.442 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T14:03:32.443 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T14:03:32.443 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T14:03:32.443 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T14:03:32.443 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T14:03:32.443 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T14:03:32.443 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.unordered_write: 0 2024-04-25T14:03:32.443 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T14:03:32.443 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T14:03:32.443 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T14:03:32.443 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T14:03:32.443 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.row_cache: None 2024-04-25T14:03:32.443 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.wal_filter: None 2024-04-25T14:03:32.443 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T14:03:32.443 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T14:03:32.444 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.two_write_queues: 0 2024-04-25T14:03:32.444 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T14:03:32.444 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.wal_compression: 0 2024-04-25T14:03:32.444 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.atomic_flush: 0 2024-04-25T14:03:32.444 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T14:03:32.444 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T14:03:32.444 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T14:03:32.444 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.log_readahead_size: 0 2024-04-25T14:03:32.444 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T14:03:32.444 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T14:03:32.444 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T14:03:32.444 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T14:03:32.444 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T14:03:32.444 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T14:03:32.445 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T14:03:32.445 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.max_background_jobs: 2 2024-04-25T14:03:32.445 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.max_background_compactions: -1 2024-04-25T14:03:32.445 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.max_subcompactions: 1 2024-04-25T14:03:32.445 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T14:03:32.445 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T14:03:32.445 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T14:03:32.445 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T14:03:32.445 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T14:03:32.445 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T14:03:32.445 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T14:03:32.445 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T14:03:32.445 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.max_open_files: -1 2024-04-25T14:03:32.445 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T14:03:32.445 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T14:03:32.446 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T14:03:32.446 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T14:03:32.446 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.max_background_flushes: -1 2024-04-25T14:03:32.446 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Compression algorithms supported: 2024-04-25T14:03:32.446 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T14:03:32.446 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: kZSTD supported: 0 2024-04-25T14:03:32.446 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: kXpressCompression supported: 0 2024-04-25T14:03:32.446 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T14:03:32.446 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: kZlibCompression supported: 1 2024-04-25T14:03:32.446 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: kSnappyCompression supported: 1 2024-04-25T14:03:32.446 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: kLZ4Compression supported: 1 2024-04-25T14:03:32.446 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: kBZip2Compression supported: 0 2024-04-25T14:03:32.446 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T14:03:32.446 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T14:03:32.447 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-25T14:03:32.447 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T14:03:32.447 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T14:03:32.447 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.merge_operator: 2024-04-25T14:03:32.447 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.compaction_filter: None 2024-04-25T14:03:32.447 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T14:03:32.447 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T14:03:32.447 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T14:03:32.447 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T14:03:32.447 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c81d82a1a0) 2024-04-25T14:03:32.447 INFO:journalctl@ceph.mon.a.smithi096.stdout: cache_index_and_filter_blocks: 1 2024-04-25T14:03:32.447 INFO:journalctl@ceph.mon.a.smithi096.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T14:03:32.447 INFO:journalctl@ceph.mon.a.smithi096.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T14:03:32.447 INFO:journalctl@ceph.mon.a.smithi096.stdout: pin_top_level_index_and_filter: 1 2024-04-25T14:03:32.447 INFO:journalctl@ceph.mon.a.smithi096.stdout: index_type: 0 2024-04-25T14:03:32.448 INFO:journalctl@ceph.mon.a.smithi096.stdout: data_block_index_type: 0 2024-04-25T14:03:32.448 INFO:journalctl@ceph.mon.a.smithi096.stdout: index_shortening: 1 2024-04-25T14:03:32.448 INFO:journalctl@ceph.mon.a.smithi096.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T14:03:32.448 INFO:journalctl@ceph.mon.a.smithi096.stdout: checksum: 4 2024-04-25T14:03:32.448 INFO:journalctl@ceph.mon.a.smithi096.stdout: no_block_cache: 0 2024-04-25T14:03:32.448 INFO:journalctl@ceph.mon.a.smithi096.stdout: block_cache: 0x55c81d7271f0 2024-04-25T14:03:32.448 INFO:journalctl@ceph.mon.a.smithi096.stdout: block_cache_name: BinnedLRUCache 2024-04-25T14:03:32.448 INFO:journalctl@ceph.mon.a.smithi096.stdout: block_cache_options: 2024-04-25T14:03:32.448 INFO:journalctl@ceph.mon.a.smithi096.stdout: capacity : 536870912 2024-04-25T14:03:32.448 INFO:journalctl@ceph.mon.a.smithi096.stdout: num_shard_bits : 4 2024-04-25T14:03:32.448 INFO:journalctl@ceph.mon.a.smithi096.stdout: strict_capacity_limit : 0 2024-04-25T14:03:32.448 INFO:journalctl@ceph.mon.a.smithi096.stdout: high_pri_pool_ratio: 0.000 2024-04-25T14:03:32.448 INFO:journalctl@ceph.mon.a.smithi096.stdout: block_cache_compressed: (nil) 2024-04-25T14:03:32.448 INFO:journalctl@ceph.mon.a.smithi096.stdout: persistent_cache: (nil) 2024-04-25T14:03:32.449 INFO:journalctl@ceph.mon.a.smithi096.stdout: block_size: 4096 2024-04-25T14:03:32.449 INFO:journalctl@ceph.mon.a.smithi096.stdout: block_size_deviation: 10 2024-04-25T14:03:32.449 INFO:journalctl@ceph.mon.a.smithi096.stdout: block_restart_interval: 16 2024-04-25T14:03:32.449 INFO:journalctl@ceph.mon.a.smithi096.stdout: index_block_restart_interval: 1 2024-04-25T14:03:32.449 INFO:journalctl@ceph.mon.a.smithi096.stdout: metadata_block_size: 4096 2024-04-25T14:03:32.449 INFO:journalctl@ceph.mon.a.smithi096.stdout: partition_filters: 0 2024-04-25T14:03:32.449 INFO:journalctl@ceph.mon.a.smithi096.stdout: use_delta_encoding: 1 2024-04-25T14:03:32.449 INFO:journalctl@ceph.mon.a.smithi096.stdout: filter_policy: bloomfilter 2024-04-25T14:03:32.449 INFO:journalctl@ceph.mon.a.smithi096.stdout: whole_key_filtering: 1 2024-04-25T14:03:32.449 INFO:journalctl@ceph.mon.a.smithi096.stdout: verify_compression: 0 2024-04-25T14:03:32.449 INFO:journalctl@ceph.mon.a.smithi096.stdout: read_amp_bytes_per_bit: 0 2024-04-25T14:03:32.449 INFO:journalctl@ceph.mon.a.smithi096.stdout: format_version: 5 2024-04-25T14:03:32.449 INFO:journalctl@ceph.mon.a.smithi096.stdout: enable_index_compression: 1 2024-04-25T14:03:32.449 INFO:journalctl@ceph.mon.a.smithi096.stdout: block_align: 0 2024-04-25T14:03:32.450 INFO:journalctl@ceph.mon.a.smithi096.stdout: max_auto_readahead_size: 262144 2024-04-25T14:03:32.450 INFO:journalctl@ceph.mon.a.smithi096.stdout: prepopulate_block_cache: 0 2024-04-25T14:03:32.450 INFO:journalctl@ceph.mon.a.smithi096.stdout: initial_auto_readahead_size: 8192 2024-04-25T14:03:32.450 INFO:journalctl@ceph.mon.a.smithi096.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T14:03:32.450 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T14:03:32.450 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T14:03:32.450 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.compression: NoCompression 2024-04-25T14:03:32.450 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T14:03:32.450 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T14:03:32.450 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T14:03:32.450 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.num_levels: 7 2024-04-25T14:03:32.450 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T14:03:32.450 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T14:03:32.450 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T14:03:32.451 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T14:03:32.451 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T14:03:32.451 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T14:03:32.451 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T14:03:32.451 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T14:03:32.451 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T14:03:32.451 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T14:03:32.451 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T14:03:32.451 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T14:03:32.451 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T14:03:32.451 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T14:03:32.451 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T14:03:32.451 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T14:03:32.451 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T14:03:32.451 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T14:03:32.452 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T14:03:32.452 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T14:03:32.452 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T14:03:32.453 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T14:03:32.453 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T14:03:32.453 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T14:03:32.453 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T14:03:32.453 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T14:03:32.453 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T14:03:32.453 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T14:03:32.453 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T14:03:32.453 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T14:03:32.453 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T14:03:32.453 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T14:03:32.453 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T14:03:32.454 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T14:03:32.454 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T14:03:32.454 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T14:03:32.454 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T14:03:32.454 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T14:03:32.454 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T14:03:32.454 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T14:03:32.454 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T14:03:32.454 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T14:03:32.454 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T14:03:32.454 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T14:03:32.454 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T14:03:32.454 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T14:03:32.454 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T14:03:32.455 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T14:03:32.455 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T14:03:32.455 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T14:03:32.455 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T14:03:32.455 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T14:03:32.455 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T14:03:32.455 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T14:03:32.455 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.inplace_update_support: 0 2024-04-25T14:03:32.455 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T14:03:32.455 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T14:03:32.455 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T14:03:32.455 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T14:03:32.455 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.bloom_locality: 0 2024-04-25T14:03:32.455 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.max_successive_merges: 0 2024-04-25T14:03:32.456 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T14:03:32.456 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T14:03:32.456 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T14:03:32.456 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T14:03:32.456 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.ttl: 2592000 2024-04-25T14:03:32.456 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T14:03:32.456 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T14:03:32.456 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T14:03:32.456 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.enable_blob_files: false 2024-04-25T14:03:32.456 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.min_blob_size: 0 2024-04-25T14:03:32.456 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T14:03:32.456 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T14:03:32.456 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T14:03:32.456 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T14:03:32.456 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T14:03:32.457 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T14:03:32.457 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T14:03:32.457 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T14:03:32.457 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-25T14:03:32.457 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-25T14:03:32.457 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 4d36bdac-e971-4929-a7b3-8a3b0c5abc1f 2024-04-25T14:03:32.457 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714053812405288, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-25T14:03:32.457 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-25T14:03:32.457 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714053812406372, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99871, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 324, "table_properties": {"data_size": 97753, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14050, "raw_average_key_size": 49, "raw_value_size": 90091, "raw_average_value_size": 318, "num_data_blocks": 12, "num_entries": 283, "num_filter_entries": 283, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714053812, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "4d36bdac-e971-4929-a7b3-8a3b0c5abc1f", "db_session_id": "IWH7IDULXP127IN9Y32T", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-25T14:03:32.457 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714053812406432, "job": 1, "event": "recovery_finished"} 2024-04-25T14:03:32.457 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-25T14:03:32.457 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T14:03:32.457 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55c81d70bc00 2024-04-25T14:03:32.458 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: DB pointer 0x55c8224fe000 2024-04-25T14:03:32.458 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T14:03:32.458 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T14:03:32.458 INFO:journalctl@ceph.mon.a.smithi096.stdout: ** DB Stats ** 2024-04-25T14:03:32.458 INFO:journalctl@ceph.mon.a.smithi096.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T14:03:32.458 INFO:journalctl@ceph.mon.a.smithi096.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T14:03:32.458 INFO:journalctl@ceph.mon.a.smithi096.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T14:03:32.458 INFO:journalctl@ceph.mon.a.smithi096.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T14:03:32.458 INFO:journalctl@ceph.mon.a.smithi096.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T14:03:32.458 INFO:journalctl@ceph.mon.a.smithi096.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T14:03:32.458 INFO:journalctl@ceph.mon.a.smithi096.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T14:03:32.458 INFO:journalctl@ceph.mon.a.smithi096.stdout: 2024-04-25T14:03:32.458 INFO:journalctl@ceph.mon.a.smithi096.stdout: ** Compaction Stats [default] ** 2024-04-25T14:03:32.458 INFO:journalctl@ceph.mon.a.smithi096.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T14:03:32.459 INFO:journalctl@ceph.mon.a.smithi096.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T14:03:32.459 INFO:journalctl@ceph.mon.a.smithi096.stdout: L0 2/0 99.39 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 154.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T14:03:32.459 INFO:journalctl@ceph.mon.a.smithi096.stdout: Sum 2/0 99.39 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 154.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T14:03:32.459 INFO:journalctl@ceph.mon.a.smithi096.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 154.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T14:03:32.459 INFO:journalctl@ceph.mon.a.smithi096.stdout: 2024-04-25T14:03:32.459 INFO:journalctl@ceph.mon.a.smithi096.stdout: ** Compaction Stats [default] ** 2024-04-25T14:03:32.459 INFO:journalctl@ceph.mon.a.smithi096.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T14:03:32.459 INFO:journalctl@ceph.mon.a.smithi096.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T14:03:32.459 INFO:journalctl@ceph.mon.a.smithi096.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 154.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T14:03:32.459 INFO:journalctl@ceph.mon.a.smithi096.stdout: 2024-04-25T14:03:32.459 INFO:journalctl@ceph.mon.a.smithi096.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T14:03:32.459 INFO:journalctl@ceph.mon.a.smithi096.stdout: 2024-04-25T14:03:32.459 INFO:journalctl@ceph.mon.a.smithi096.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T14:03:32.460 INFO:journalctl@ceph.mon.a.smithi096.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T14:03:32.460 INFO:journalctl@ceph.mon.a.smithi096.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T14:03:32.460 INFO:journalctl@ceph.mon.a.smithi096.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T14:03:32.460 INFO:journalctl@ceph.mon.a.smithi096.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T14:03:32.460 INFO:journalctl@ceph.mon.a.smithi096.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T14:03:32.460 INFO:journalctl@ceph.mon.a.smithi096.stdout: Cumulative compaction: 0.00 GB write, 30.13 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T14:03:32.460 INFO:journalctl@ceph.mon.a.smithi096.stdout: Interval compaction: 0.00 GB write, 30.13 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T14:03:32.460 INFO:journalctl@ceph.mon.a.smithi096.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T14:03:32.460 INFO:journalctl@ceph.mon.a.smithi096.stdout: Block cache BinnedLRUCache@0x55c81d7271f0#2 capacity: 512.00 MB usage: 1.53 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-25T14:03:32.460 INFO:journalctl@ceph.mon.a.smithi096.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.03 KB,0.000196695%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-25T14:03:32.460 INFO:journalctl@ceph.mon.a.smithi096.stdout: 2024-04-25T14:03:32.460 INFO:journalctl@ceph.mon.a.smithi096.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T14:03:32.460 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: starting mon.a rank 0 at public addrs [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] at bind addrs [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 59334182-030c-11ef-bc93-c7b262605968 2024-04-25T14:03:32.460 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: mon.a@-1(???) e1 preinit fsid 59334182-030c-11ef-bc93-c7b262605968 2024-04-25T14:03:32.460 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: mon.a@-1(???).mds e1 new map 2024-04-25T14:03:32.460 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: mon.a@-1(???).mds e1 print_map 2024-04-25T14:03:32.461 INFO:journalctl@ceph.mon.a.smithi096.stdout: e1 2024-04-25T14:03:32.461 INFO:journalctl@ceph.mon.a.smithi096.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T14:03:32.461 INFO:journalctl@ceph.mon.a.smithi096.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T14:03:32.461 INFO:journalctl@ceph.mon.a.smithi096.stdout: legacy client fscid: -1 2024-04-25T14:03:32.461 INFO:journalctl@ceph.mon.a.smithi096.stdout: 2024-04-25T14:03:32.461 INFO:journalctl@ceph.mon.a.smithi096.stdout: No filesystems configured 2024-04-25T14:03:32.461 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T14:03:32.461 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T14:03:32.461 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T14:03:32.461 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T14:03:32.461 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-25T14:03:32.461 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 bash[29188]: 18d82172990ba4676e789e5d85a7d94bce2856d86216b79876a2f7939b23c04b 2024-04-25T14:03:32.461 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 systemd[1]: Started Ceph mon.a for 59334182-030c-11ef-bc93-c7b262605968. 2024-04-25T14:03:32.462 INFO:teuthology.orchestra.run.smithi096.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-25T14:03:32.748 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T14:03:32.748 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: monmap epoch 1 2024-04-25T14:03:32.748 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: fsid 59334182-030c-11ef-bc93-c7b262605968 2024-04-25T14:03:32.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: last_changed 2024-04-25T14:03:12.788276+0000 2024-04-25T14:03:32.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: created 2024-04-25T14:03:12.788276+0000 2024-04-25T14:03:32.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: min_mon_release 19 (squid) 2024-04-25T14:03:32.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: election_strategy: 1 2024-04-25T14:03:32.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: 0: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.a 2024-04-25T14:03:32.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: fsmap 2024-04-25T14:03:32.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T14:03:32.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:32 smithi096 ceph-mon[29207]: mgrmap e1: no daemons active 2024-04-25T14:03:33.513 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:32.864+0000 7f9126af0700 1 Processor -- start 2024-04-25T14:03:33.518 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:32.864+0000 7f9126af0700 1 -- start start 2024-04-25T14:03:33.518 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:32.864+0000 7f9126af0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9120109ea0 0x7f912010a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:33.518 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:32.864+0000 7f9126af0700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f912010a880 con 0x7f9120109ea0 2024-04-25T14:03:33.518 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:32.864+0000 7f912488c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9120109ea0 0x7f912010a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:33.518 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:32.864+0000 7f912488c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9120109ea0 0x7f912010a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44192/0 (socket says 172.21.15.96:44192) 2024-04-25T14:03:33.518 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:32.864+0000 7f912488c700 1 -- 172.21.15.96:0/3427098112 learned_addr learned my addr 172.21.15.96:0/3427098112 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:03:33.518 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:32.865+0000 7f912488c700 1 -- 172.21.15.96:0/3427098112 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9120108b20 con 0x7f9120109ea0 2024-04-25T14:03:33.518 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:32.865+0000 7f912488c700 1 --2- 172.21.15.96:0/3427098112 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9120109ea0 0x7f912010a2b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f91140098a0 tx=0x7f9114009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f872be448996a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:33.518 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:32.865+0000 7f911f7fe700 1 -- 172.21.15.96:0/3427098112 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9114004160 con 0x7f9120109ea0 2024-04-25T14:03:33.518 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:32.865+0000 7f911f7fe700 1 -- 172.21.15.96:0/3427098112 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9114004300 con 0x7f9120109ea0 2024-04-25T14:03:33.519 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:32.865+0000 7f911f7fe700 1 -- 172.21.15.96:0/3427098112 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9114011af0 con 0x7f9120109ea0 2024-04-25T14:03:33.519 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:32.866+0000 7f9126af0700 1 -- 172.21.15.96:0/3427098112 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9120109ea0 msgr2=0x7f912010a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:33.519 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:32.866+0000 7f9126af0700 1 --2- 172.21.15.96:0/3427098112 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9120109ea0 0x7f912010a2b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f91140098a0 tx=0x7f9114009bb0 comp rx=0 tx=0).stop 2024-04-25T14:03:33.519 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:32.866+0000 7f9126af0700 1 -- 172.21.15.96:0/3427098112 shutdown_connections 2024-04-25T14:03:33.519 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:32.866+0000 7f9126af0700 1 --2- 172.21.15.96:0/3427098112 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9120109ea0 0x7f912010a2b0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:33.519 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:32.866+0000 7f9126af0700 1 -- 172.21.15.96:0/3427098112 >> 172.21.15.96:0/3427098112 conn(0x7f9120103720 msgr2=0x7f9120105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:33.519 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:32.866+0000 7f9126af0700 1 -- 172.21.15.96:0/3427098112 shutdown_connections 2024-04-25T14:03:33.519 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:32.866+0000 7f9126af0700 1 -- 172.21.15.96:0/3427098112 wait complete. 2024-04-25T14:03:33.519 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:32.867+0000 7f9126af0700 1 Processor -- start 2024-04-25T14:03:33.519 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:32.867+0000 7f9126af0700 1 -- start start 2024-04-25T14:03:33.519 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:32.867+0000 7f9126af0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9120108310 0x7f91201a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:33.519 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:32.867+0000 7f9126af0700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f912010cee0 con 0x7f9120108310 2024-04-25T14:03:33.519 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:32.867+0000 7f912488c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9120108310 0x7f91201a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:33.520 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:32.867+0000 7f912488c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9120108310 0x7f91201a0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44200/0 (socket says 172.21.15.96:44200) 2024-04-25T14:03:33.520 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:32.867+0000 7f912488c700 1 -- 172.21.15.96:0/1289648600 learned_addr learned my addr 172.21.15.96:0/1289648600 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:03:33.520 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:32.867+0000 7f912488c700 1 -- 172.21.15.96:0/1289648600 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9120108780 con 0x7f9120108310 2024-04-25T14:03:33.520 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:32.867+0000 7f912488c700 1 --2- 172.21.15.96:0/1289648600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9120108310 0x7f91201a0c30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9114000c00 tx=0x7f9114004260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:33.520 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:32.868+0000 7f911dffb700 1 -- 172.21.15.96:0/1289648600 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91140037d0 con 0x7f9120108310 2024-04-25T14:03:33.520 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:32.868+0000 7f911dffb700 1 -- 172.21.15.96:0/1289648600 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9114003970 con 0x7f9120108310 2024-04-25T14:03:33.520 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:32.868+0000 7f9126af0700 1 -- 172.21.15.96:0/1289648600 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91201a1170 con 0x7f9120108310 2024-04-25T14:03:33.520 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:32.868+0000 7f911dffb700 1 -- 172.21.15.96:0/1289648600 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f911401b440 con 0x7f9120108310 2024-04-25T14:03:33.520 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:32.868+0000 7f9126af0700 1 -- 172.21.15.96:0/1289648600 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91201a1510 con 0x7f9120108310 2024-04-25T14:03:33.520 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:32.868+0000 7f911dffb700 1 -- 172.21.15.96:0/1289648600 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f911401ba80 con 0x7f9120108310 2024-04-25T14:03:33.520 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:32.868+0000 7f911dffb700 1 -- 172.21.15.96:0/1289648600 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f911402e730 con 0x7f9120108310 2024-04-25T14:03:33.520 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:32.869+0000 7f9126af0700 1 -- 172.21.15.96:0/1289648600 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9120064cf0 con 0x7f9120108310 2024-04-25T14:03:33.520 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:32.871+0000 7f911dffb700 1 -- 172.21.15.96:0/1289648600 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f9114003b20 con 0x7f9120108310 2024-04-25T14:03:33.520 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:32.940+0000 7f9126af0700 1 -- 172.21.15.96:0/1289648600 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f91201a1910 con 0x7f9120108310 2024-04-25T14:03:33.521 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:32.942+0000 7f911dffb700 1 -- 172.21.15.96:0/1289648600 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f9114033030 con 0x7f9120108310 2024-04-25T14:03:33.521 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:32.943+0000 7f9126af0700 1 -- 172.21.15.96:0/1289648600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9120108310 msgr2=0x7f91201a0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:33.521 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:32.943+0000 7f9126af0700 1 --2- 172.21.15.96:0/1289648600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9120108310 0x7f91201a0c30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9114000c00 tx=0x7f9114004260 comp rx=0 tx=0).stop 2024-04-25T14:03:33.521 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:32.943+0000 7f9126af0700 1 -- 172.21.15.96:0/1289648600 shutdown_connections 2024-04-25T14:03:33.521 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:32.943+0000 7f9126af0700 1 --2- 172.21.15.96:0/1289648600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9120108310 0x7f91201a0c30 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:33.521 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:32.943+0000 7f9126af0700 1 -- 172.21.15.96:0/1289648600 >> 172.21.15.96:0/1289648600 conn(0x7f9120103720 msgr2=0x7f9120105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:33.521 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:32.943+0000 7f9126af0700 1 -- 172.21.15.96:0/1289648600 shutdown_connections 2024-04-25T14:03:33.521 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:32.943+0000 7f9126af0700 1 -- 172.21.15.96:0/1289648600 wait complete. 2024-04-25T14:03:33.521 INFO:teuthology.orchestra.run.smithi096.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-25T14:03:33.543 INFO:teuthology.orchestra.run.smithi096.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-25T14:03:33.543 INFO:teuthology.orchestra.run.smithi096.stdout:Creating mgr... 2024-04-25T14:03:33.544 INFO:teuthology.orchestra.run.smithi096.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-25T14:03:33.544 INFO:teuthology.orchestra.run.smithi096.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-25T14:03:33.786 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-59334182-030c-11ef-bc93-c7b262605968@mgr.x 2024-04-25T14:03:33.786 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Failed to reset failed state of unit ceph-59334182-030c-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-59334182-030c-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-25T14:03:33.926 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-59334182-030c-11ef-bc93-c7b262605968.target.wants/ceph-59334182-030c-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-59334182-030c-11ef-bc93-c7b262605968@.service. 2024-04-25T14:03:34.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:33 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/1289648600' entity='client.admin' 2024-04-25T14:03:34.109 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:33 smithi096 systemd[1]: Starting Ceph mgr.x for 59334182-030c-11ef-bc93-c7b262605968... 2024-04-25T14:03:34.440 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:34 smithi096 podman[29425]: 2024-04-25 14:03:34.063318283 +0000 UTC m=+0.017662208 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T14:03:34.441 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:34 smithi096 podman[29425]: 2024-04-25 14:03:34.164278017 +0000 UTC m=+0.118621930 container create de6d45d4d2364df8115ad27d496348110559a43126c24a05ac72a74f93e2793a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T14:03:34.441 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:34 smithi096 podman[29425]: 2024-04-25 14:03:34.366930146 +0000 UTC m=+0.321274072 container init de6d45d4d2364df8115ad27d496348110559a43126c24a05ac72a74f93e2793a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , ceph=True) 2024-04-25T14:03:34.441 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:34 smithi096 podman[29425]: 2024-04-25 14:03:34.369694673 +0000 UTC m=+0.324038597 container start de6d45d4d2364df8115ad27d496348110559a43126c24a05ac72a74f93e2793a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.license=GPLv2) 2024-04-25T14:03:34.441 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:34 smithi096 bash[29425]: de6d45d4d2364df8115ad27d496348110559a43126c24a05ac72a74f93e2793a 2024-04-25T14:03:34.454 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T14:03:34.455 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout disabled 2024-04-25T14:03:34.462 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T14:03:34.462 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout inactive 2024-04-25T14:03:34.462 INFO:teuthology.orchestra.run.smithi096.stdout:firewalld.service is not enabled 2024-04-25T14:03:34.462 INFO:teuthology.orchestra.run.smithi096.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T14:03:34.470 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T14:03:34.471 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout disabled 2024-04-25T14:03:34.478 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T14:03:34.478 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout inactive 2024-04-25T14:03:34.478 INFO:teuthology.orchestra.run.smithi096.stdout:firewalld.service is not enabled 2024-04-25T14:03:34.478 INFO:teuthology.orchestra.run.smithi096.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-25T14:03:34.478 INFO:teuthology.orchestra.run.smithi096.stdout:Waiting for mgr to start... 2024-04-25T14:03:34.478 INFO:teuthology.orchestra.run.smithi096.stdout:Waiting for mgr... 2024-04-25T14:03:34.702 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:34 smithi096 systemd[1]: Started Ceph mgr.x for 59334182-030c-11ef-bc93-c7b262605968. 2024-04-25T14:03:34.703 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:34 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:34.536+0000 7fcea357f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:03:34.961 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:34 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:34.733+0000 7fcea357f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:03:34.961 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:34 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:34.846+0000 7fcea357f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:03:35.267 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:35 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/3238684231' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T14:03:35.267 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:35 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:35.180+0000 7fcea357f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:03:35.267 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:35 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:35.267+0000 7fcea357f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:03:35.690 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-04-25T14:03:35.690 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout { 2024-04-25T14:03:35.690 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "fsid": "59334182-030c-11ef-bc93-c7b262605968", 2024-04-25T14:03:35.690 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T14:03:35.690 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T14:03:35.690 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T14:03:35.690 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T14:03:35.690 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:03:35.691 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T14:03:35.691 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T14:03:35.691 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 0 2024-04-25T14:03:35.691 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:03:35.691 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T14:03:35.691 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T14:03:35.691 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:03:35.691 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-25T14:03:35.691 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T14:03:35.691 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:03:35.691 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T14:03:35.692 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T14:03:35.692 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:03:35.692 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T14:03:35.692 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:03:35.692 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T14:03:35.692 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T14:03:35.692 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T14:03:35.692 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T14:03:35.692 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T14:03:35.692 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T14:03:35.692 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:03:35.692 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T14:03:35.693 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T14:03:35.693 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T14:03:35.693 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T14:03:35.693 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T14:03:35.693 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T14:03:35.693 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T14:03:35.693 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T14:03:35.693 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T14:03:35.693 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:03:35.693 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T14:03:35.693 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:03:35.694 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T14:03:35.694 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T14:03:35.694 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:03:35.694 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T14:03:35.694 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T14:03:35.694 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T14:03:35.694 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T14:03:35.694 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T14:03:35.694 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T14:03:35.694 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T14:03:35.694 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:03:35.695 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T14:03:35.695 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:03:35.695 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T14:03:35.695 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:03:35.695 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T14:03:25.681351+0000", 2024-04-25T14:03:35.695 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T14:03:35.695 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:03:35.695 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T14:03:35.695 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout } 2024-04-25T14:03:35.695 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:34.997+0000 7f2ba4f87700 1 Processor -- start 2024-04-25T14:03:35.695 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:34.997+0000 7f2ba4f87700 1 -- start start 2024-04-25T14:03:35.695 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:34.997+0000 7f2ba4f87700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ba0108850 0x7f2ba0108c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:35.695 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:34.997+0000 7f2ba4f87700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba0109250 con 0x7f2ba0108850 2024-04-25T14:03:35.696 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:34.998+0000 7f2b9e59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ba0108850 0x7f2ba0108c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:35.696 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:34.998+0000 7f2b9e59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ba0108850 0x7f2ba0108c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44244/0 (socket says 172.21.15.96:44244) 2024-04-25T14:03:35.696 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:34.998+0000 7f2b9e59c700 1 -- 172.21.15.96:0/855228374 learned_addr learned my addr 172.21.15.96:0/855228374 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:03:35.696 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:34.998+0000 7f2b9e59c700 1 -- 172.21.15.96:0/855228374 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ba0109a20 con 0x7f2ba0108850 2024-04-25T14:03:35.696 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:34.998+0000 7f2b9e59c700 1 --2- 172.21.15.96:0/855228374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ba0108850 0x7f2ba0108c60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f2b900098a0 tx=0x7f2b90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e8b7f92369d4605 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:35.696 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:34.999+0000 7f2b9d59a700 1 -- 172.21.15.96:0/855228374 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b90004160 con 0x7f2ba0108850 2024-04-25T14:03:35.696 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:34.999+0000 7f2b9d59a700 1 -- 172.21.15.96:0/855228374 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2b90004300 con 0x7f2ba0108850 2024-04-25T14:03:35.696 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:34.999+0000 7f2b9d59a700 1 -- 172.21.15.96:0/855228374 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b90011af0 con 0x7f2ba0108850 2024-04-25T14:03:35.696 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:35.000+0000 7f2ba4f87700 1 -- 172.21.15.96:0/855228374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ba0108850 msgr2=0x7f2ba0108c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:35.696 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:35.000+0000 7f2ba4f87700 1 --2- 172.21.15.96:0/855228374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ba0108850 0x7f2ba0108c60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f2b900098a0 tx=0x7f2b90009bb0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.697 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:35.000+0000 7f2ba4f87700 1 -- 172.21.15.96:0/855228374 shutdown_connections 2024-04-25T14:03:35.697 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:35.000+0000 7f2ba4f87700 1 --2- 172.21.15.96:0/855228374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ba0108850 0x7f2ba0108c60 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.697 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:35.000+0000 7f2ba4f87700 1 -- 172.21.15.96:0/855228374 >> 172.21.15.96:0/855228374 conn(0x7f2ba0103ef0 msgr2=0x7f2ba0106340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:35.697 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:35.000+0000 7f2ba4f87700 1 -- 172.21.15.96:0/855228374 shutdown_connections 2024-04-25T14:03:35.697 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:35.000+0000 7f2ba4f87700 1 -- 172.21.15.96:0/855228374 wait complete. 2024-04-25T14:03:35.697 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:35.001+0000 7f2ba4f87700 1 Processor -- start 2024-04-25T14:03:35.697 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:35.001+0000 7f2ba4f87700 1 -- start start 2024-04-25T14:03:35.697 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:35.001+0000 7f2ba4f87700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ba0108850 0x7f2ba007c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:35.697 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:35.001+0000 7f2ba4f87700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba010cdd0 con 0x7f2ba0108850 2024-04-25T14:03:35.697 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:35.002+0000 7f2b9e59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ba0108850 0x7f2ba007c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:35.697 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:35.002+0000 7f2b9e59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ba0108850 0x7f2ba007c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44260/0 (socket says 172.21.15.96:44260) 2024-04-25T14:03:35.697 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:35.002+0000 7f2b9e59c700 1 -- 172.21.15.96:0/3238684231 learned_addr learned my addr 172.21.15.96:0/3238684231 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:03:35.697 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:35.002+0000 7f2b9e59c700 1 -- 172.21.15.96:0/3238684231 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ba007ce30 con 0x7f2ba0108850 2024-04-25T14:03:35.698 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:35.002+0000 7f2b9e59c700 1 --2- 172.21.15.96:0/3238684231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ba0108850 0x7f2ba007c8f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2b90009fb0 tx=0x7f2b90011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:35.698 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:35.002+0000 7f2b8f7fe700 1 -- 172.21.15.96:0/3238684231 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b900036a0 con 0x7f2ba0108850 2024-04-25T14:03:35.698 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:35.002+0000 7f2b8f7fe700 1 -- 172.21.15.96:0/3238684231 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2b90003cd0 con 0x7f2ba0108850 2024-04-25T14:03:35.698 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:35.003+0000 7f2b8f7fe700 1 -- 172.21.15.96:0/3238684231 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b9001a430 con 0x7f2ba0108850 2024-04-25T14:03:35.698 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:35.003+0000 7f2ba4f87700 1 -- 172.21.15.96:0/3238684231 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ba01a0ac0 con 0x7f2ba0108850 2024-04-25T14:03:35.698 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:35.003+0000 7f2ba4f87700 1 -- 172.21.15.96:0/3238684231 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ba01a0f40 con 0x7f2ba0108850 2024-04-25T14:03:35.698 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:35.003+0000 7f2b8f7fe700 1 -- 172.21.15.96:0/3238684231 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2b9001aa30 con 0x7f2ba0108850 2024-04-25T14:03:35.698 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:35.003+0000 7f2b8f7fe700 1 -- 172.21.15.96:0/3238684231 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2b900193c0 con 0x7f2ba0108850 2024-04-25T14:03:35.698 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:35.004+0000 7f2ba4f87700 1 -- 172.21.15.96:0/3238684231 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ba0064cf0 con 0x7f2ba0108850 2024-04-25T14:03:35.698 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:35.006+0000 7f2b8f7fe700 1 -- 172.21.15.96:0/3238684231 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f2b90024440 con 0x7f2ba0108850 2024-04-25T14:03:35.698 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:35.078+0000 7f2ba4f87700 1 -- 172.21.15.96:0/3238684231 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f2ba01a1340 con 0x7f2ba0108850 2024-04-25T14:03:35.698 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:35.079+0000 7f2b8f7fe700 1 -- 172.21.15.96:0/3238684231 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f2b90024760 con 0x7f2ba0108850 2024-04-25T14:03:35.699 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:35.080+0000 7f2ba4f87700 1 -- 172.21.15.96:0/3238684231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ba0108850 msgr2=0x7f2ba007c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:35.699 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:35.080+0000 7f2ba4f87700 1 --2- 172.21.15.96:0/3238684231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ba0108850 0x7f2ba007c8f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2b90009fb0 tx=0x7f2b90011890 comp rx=0 tx=0).stop 2024-04-25T14:03:35.699 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:35.080+0000 7f2ba4f87700 1 -- 172.21.15.96:0/3238684231 shutdown_connections 2024-04-25T14:03:35.699 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:35.080+0000 7f2ba4f87700 1 --2- 172.21.15.96:0/3238684231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ba0108850 0x7f2ba007c8f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.699 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:35.080+0000 7f2ba4f87700 1 -- 172.21.15.96:0/3238684231 >> 172.21.15.96:0/3238684231 conn(0x7f2ba0103ef0 msgr2=0x7f2ba01046d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:35.699 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:35.081+0000 7f2ba4f87700 1 -- 172.21.15.96:0/3238684231 shutdown_connections 2024-04-25T14:03:35.699 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:35.081+0000 7f2ba4f87700 1 -- 172.21.15.96:0/3238684231 wait complete. 2024-04-25T14:03:35.699 INFO:teuthology.orchestra.run.smithi096.stdout:mgr not available, waiting (1/15)... 2024-04-25T14:03:36.807 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:36 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:36.469+0000 7fcea357f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:03:36.807 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:36 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:36.638+0000 7fcea357f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:03:36.807 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:36 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:36.807+0000 7fcea357f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:03:37.359 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:37 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:37.092+0000 7fcea357f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:03:37.359 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:37 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:37.183+0000 7fcea357f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:03:37.811 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:37 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:37.382+0000 7fcea357f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:03:37.811 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:37 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:37.571+0000 7fcea357f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:03:38.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:38 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/4267810787' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T14:03:38.359 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:38 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:38.186+0000 7fcea357f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T14:03:38.801 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-04-25T14:03:38.801 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout { 2024-04-25T14:03:38.801 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "fsid": "59334182-030c-11ef-bc93-c7b262605968", 2024-04-25T14:03:38.802 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T14:03:38.802 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T14:03:38.802 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T14:03:38.802 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T14:03:38.802 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:03:38.802 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T14:03:38.802 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T14:03:38.802 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 0 2024-04-25T14:03:38.802 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:03:38.802 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T14:03:38.802 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T14:03:38.802 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:03:38.802 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-04-25T14:03:38.802 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T14:03:38.802 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:03:38.803 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T14:03:38.803 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T14:03:38.803 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:03:38.803 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T14:03:38.803 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:03:38.803 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T14:03:38.803 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T14:03:38.803 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T14:03:38.803 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T14:03:38.803 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T14:03:38.803 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T14:03:38.803 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:03:38.803 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T14:03:38.803 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T14:03:38.804 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T14:03:38.804 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T14:03:38.811 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T14:03:38.811 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T14:03:38.811 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T14:03:38.811 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T14:03:38.812 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T14:03:38.812 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:03:38.812 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T14:03:38.812 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:03:38.812 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T14:03:38.812 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T14:03:38.812 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:03:38.812 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T14:03:38.812 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T14:03:38.812 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T14:03:38.812 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T14:03:38.812 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T14:03:38.812 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T14:03:38.812 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T14:03:38.812 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:03:38.813 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T14:03:38.813 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:03:38.813 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T14:03:38.813 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:03:38.813 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T14:03:25.681351+0000", 2024-04-25T14:03:38.813 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T14:03:38.813 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:03:38.813 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T14:03:38.813 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout } 2024-04-25T14:03:38.813 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:38.138+0000 7f08eaa9d700 1 Processor -- start 2024-04-25T14:03:38.813 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:38.138+0000 7f08eaa9d700 1 -- start start 2024-04-25T14:03:38.813 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:38.138+0000 7f08eaa9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08e41082f0 0x7f08e4108700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:38.814 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:38.138+0000 7f08eaa9d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f08e407cd60 con 0x7f08e41082f0 2024-04-25T14:03:38.814 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:38.138+0000 7f08e8839700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08e41082f0 0x7f08e4108700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:38.814 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:38.138+0000 7f08e8839700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08e41082f0 0x7f08e4108700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44262/0 (socket says 172.21.15.96:44262) 2024-04-25T14:03:38.814 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:38.138+0000 7f08e8839700 1 -- 172.21.15.96:0/3727561066 learned_addr learned my addr 172.21.15.96:0/3727561066 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:03:38.814 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:38.139+0000 7f08e8839700 1 -- 172.21.15.96:0/3727561066 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08e4108c40 con 0x7f08e41082f0 2024-04-25T14:03:38.814 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:38.139+0000 7f08e8839700 1 --2- 172.21.15.96:0/3727561066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08e41082f0 0x7f08e4108700 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f08d80098a0 tx=0x7f08d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cd3661d825e02d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:38.814 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:38.139+0000 7f08e37fe700 1 -- 172.21.15.96:0/3727561066 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08d8004160 con 0x7f08e41082f0 2024-04-25T14:03:38.814 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:38.139+0000 7f08e37fe700 1 -- 172.21.15.96:0/3727561066 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f08d8004300 con 0x7f08e41082f0 2024-04-25T14:03:38.814 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:38.139+0000 7f08e37fe700 1 -- 172.21.15.96:0/3727561066 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08d8011af0 con 0x7f08e41082f0 2024-04-25T14:03:38.814 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:38.140+0000 7f08eaa9d700 1 -- 172.21.15.96:0/3727561066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08e41082f0 msgr2=0x7f08e4108700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:38.815 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:38.140+0000 7f08eaa9d700 1 --2- 172.21.15.96:0/3727561066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08e41082f0 0x7f08e4108700 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f08d80098a0 tx=0x7f08d8009bb0 comp rx=0 tx=0).stop 2024-04-25T14:03:38.815 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:38.140+0000 7f08eaa9d700 1 -- 172.21.15.96:0/3727561066 shutdown_connections 2024-04-25T14:03:38.815 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:38.140+0000 7f08eaa9d700 1 --2- 172.21.15.96:0/3727561066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08e41082f0 0x7f08e4108700 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:38.815 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:38.140+0000 7f08eaa9d700 1 -- 172.21.15.96:0/3727561066 >> 172.21.15.96:0/3727561066 conn(0x7f08e4103f20 msgr2=0x7f08e4106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:38.815 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:38.140+0000 7f08eaa9d700 1 -- 172.21.15.96:0/3727561066 shutdown_connections 2024-04-25T14:03:38.815 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:38.140+0000 7f08eaa9d700 1 -- 172.21.15.96:0/3727561066 wait complete. 2024-04-25T14:03:38.815 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:38.141+0000 7f08eaa9d700 1 Processor -- start 2024-04-25T14:03:38.815 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:38.141+0000 7f08eaa9d700 1 -- start start 2024-04-25T14:03:38.815 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:38.141+0000 7f08eaa9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08e41082f0 0x7f08e419fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:38.815 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:38.141+0000 7f08eaa9d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f08e4109b40 con 0x7f08e41082f0 2024-04-25T14:03:38.815 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:38.141+0000 7f08e8839700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08e41082f0 0x7f08e419fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:38.815 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:38.141+0000 7f08e8839700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08e41082f0 0x7f08e419fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44264/0 (socket says 172.21.15.96:44264) 2024-04-25T14:03:38.815 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:38.141+0000 7f08e8839700 1 -- 172.21.15.96:0/4267810787 learned_addr learned my addr 172.21.15.96:0/4267810787 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:03:38.815 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:38.142+0000 7f08e8839700 1 -- 172.21.15.96:0/4267810787 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08e41a0430 con 0x7f08e41082f0 2024-04-25T14:03:38.816 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:38.142+0000 7f08e8839700 1 --2- 172.21.15.96:0/4267810787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08e41082f0 0x7f08e419fef0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f08d80117b0 tx=0x7f08d8011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:38.816 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:38.142+0000 7f08e1ffb700 1 -- 172.21.15.96:0/4267810787 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08d8003af0 con 0x7f08e41082f0 2024-04-25T14:03:38.816 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:38.142+0000 7f08eaa9d700 1 -- 172.21.15.96:0/4267810787 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08e41a06c0 con 0x7f08e41082f0 2024-04-25T14:03:38.816 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:38.142+0000 7f08e1ffb700 1 -- 172.21.15.96:0/4267810787 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f08d8020070 con 0x7f08e41082f0 2024-04-25T14:03:38.816 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:38.142+0000 7f08e1ffb700 1 -- 172.21.15.96:0/4267810787 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08d8019630 con 0x7f08e41082f0 2024-04-25T14:03:38.816 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:38.142+0000 7f08eaa9d700 1 -- 172.21.15.96:0/4267810787 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08e41a32d0 con 0x7f08e41082f0 2024-04-25T14:03:38.816 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:38.143+0000 7f08e1ffb700 1 -- 172.21.15.96:0/4267810787 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f08d8019c80 con 0x7f08e41082f0 2024-04-25T14:03:38.816 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:38.143+0000 7f08e1ffb700 1 -- 172.21.15.96:0/4267810787 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f08d800b720 con 0x7f08e41082f0 2024-04-25T14:03:38.816 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:38.143+0000 7f08eaa9d700 1 -- 172.21.15.96:0/4267810787 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08e4064cf0 con 0x7f08e41082f0 2024-04-25T14:03:38.816 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:38.146+0000 7f08e1ffb700 1 -- 172.21.15.96:0/4267810787 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f08d80042a0 con 0x7f08e41082f0 2024-04-25T14:03:38.816 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:38.216+0000 7f08eaa9d700 1 -- 172.21.15.96:0/4267810787 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f08e402e810 con 0x7f08e41082f0 2024-04-25T14:03:38.816 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:38.217+0000 7f08e1ffb700 1 -- 172.21.15.96:0/4267810787 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f08d801a9c0 con 0x7f08e41082f0 2024-04-25T14:03:38.816 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:38.218+0000 7f08eaa9d700 1 -- 172.21.15.96:0/4267810787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08e41082f0 msgr2=0x7f08e419fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:38.817 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:38.218+0000 7f08eaa9d700 1 --2- 172.21.15.96:0/4267810787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08e41082f0 0x7f08e419fef0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f08d80117b0 tx=0x7f08d8011b70 comp rx=0 tx=0).stop 2024-04-25T14:03:38.817 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:38.218+0000 7f08eaa9d700 1 -- 172.21.15.96:0/4267810787 shutdown_connections 2024-04-25T14:03:38.817 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:38.218+0000 7f08eaa9d700 1 --2- 172.21.15.96:0/4267810787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08e41082f0 0x7f08e419fef0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:38.817 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:38.218+0000 7f08eaa9d700 1 -- 172.21.15.96:0/4267810787 >> 172.21.15.96:0/4267810787 conn(0x7f08e4103f20 msgr2=0x7f08e4193f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:38.817 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:38.218+0000 7f08eaa9d700 1 -- 172.21.15.96:0/4267810787 shutdown_connections 2024-04-25T14:03:38.817 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:38.218+0000 7f08eaa9d700 1 -- 172.21.15.96:0/4267810787 wait complete. 2024-04-25T14:03:38.817 INFO:teuthology.orchestra.run.smithi096.stdout:mgr not available, waiting (2/15)... 2024-04-25T14:03:38.925 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:38 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:38.628+0000 7fcea357f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:03:38.925 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:38 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:38.724+0000 7fcea357f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:03:38.925 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:38 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:38.925+0000 7fcea357f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T14:03:39.769 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:39 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:39.440+0000 7fcea357f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:03:39.770 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:39 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:39.531+0000 7fcea357f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:03:40.106 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:39 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:39.770+0000 7fcea357f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:03:40.106 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:39 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:39.900+0000 7fcea357f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:03:40.106 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:39 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:39.992+0000 7fcea357f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:03:40.107 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:40 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:40.106+0000 7fcea357f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:03:40.859 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:40 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:40.406+0000 7fcea357f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:03:41.223 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:41 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:41.008+0000 7fcea357f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:03:41.609 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:41 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:41.228+0000 7fcea357f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:03:41.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:41 smithi096 ceph-mon[29207]: Activating manager daemon x 2024-04-25T14:03:41.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:41 smithi096 ceph-mon[29207]: mgrmap e2: x(active, starting, since 0.00344088s) 2024-04-25T14:03:41.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:41 smithi096 ceph-mon[29207]: from='mgr.14100 172.21.15.96:0/3978628661' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T14:03:41.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:41 smithi096 ceph-mon[29207]: from='mgr.14100 172.21.15.96:0/3978628661' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T14:03:41.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:41 smithi096 ceph-mon[29207]: from='mgr.14100 172.21.15.96:0/3978628661' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T14:03:41.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:41 smithi096 ceph-mon[29207]: from='mgr.14100 172.21.15.96:0/3978628661' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:03:41.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:41 smithi096 ceph-mon[29207]: from='mgr.14100 172.21.15.96:0/3978628661' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T14:03:41.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:41 smithi096 ceph-mon[29207]: Manager daemon x is now available 2024-04-25T14:03:41.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:41 smithi096 ceph-mon[29207]: from='mgr.14100 172.21.15.96:0/3978628661' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T14:03:41.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:41 smithi096 ceph-mon[29207]: from='mgr.14100 172.21.15.96:0/3978628661' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T14:03:41.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:41 smithi096 ceph-mon[29207]: from='mgr.14100 172.21.15.96:0/3978628661' entity='mgr.x' 2024-04-25T14:03:41.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:41 smithi096 ceph-mon[29207]: from='mgr.14100 172.21.15.96:0/3978628661' entity='mgr.x' 2024-04-25T14:03:41.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:41 smithi096 ceph-mon[29207]: from='mgr.14100 172.21.15.96:0/3978628661' entity='mgr.x' 2024-04-25T14:03:41.922 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-04-25T14:03:41.923 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout { 2024-04-25T14:03:41.923 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "fsid": "59334182-030c-11ef-bc93-c7b262605968", 2024-04-25T14:03:41.923 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T14:03:41.923 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T14:03:41.923 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T14:03:41.923 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T14:03:41.923 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:03:41.923 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T14:03:41.923 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T14:03:41.923 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 0 2024-04-25T14:03:41.923 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:03:41.923 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T14:03:41.924 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T14:03:41.924 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:03:41.924 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-04-25T14:03:41.924 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T14:03:41.924 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:03:41.924 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T14:03:41.924 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T14:03:41.924 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:03:41.924 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T14:03:41.924 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:03:41.924 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T14:03:41.924 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T14:03:41.924 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T14:03:41.924 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T14:03:41.924 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T14:03:41.925 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T14:03:41.925 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:03:41.925 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T14:03:41.925 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T14:03:41.925 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T14:03:41.925 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T14:03:41.925 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T14:03:41.925 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T14:03:41.925 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T14:03:41.925 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T14:03:41.925 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T14:03:41.925 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:03:41.925 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T14:03:41.925 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:03:41.925 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T14:03:41.926 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T14:03:41.926 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:03:41.926 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T14:03:41.926 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T14:03:41.926 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T14:03:41.926 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T14:03:41.926 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T14:03:41.926 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T14:03:41.926 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T14:03:41.926 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:03:41.926 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T14:03:41.926 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:03:41.926 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T14:03:41.926 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:03:41.927 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T14:03:25.681351+0000", 2024-04-25T14:03:41.927 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T14:03:41.927 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:03:41.927 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T14:03:41.927 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout } 2024-04-25T14:03:41.927 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:41.251+0000 7fc91e273700 1 Processor -- start 2024-04-25T14:03:41.927 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:41.251+0000 7fc91e273700 1 -- start start 2024-04-25T14:03:41.927 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:41.252+0000 7fc91e273700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc918108850 0x7fc918108c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.927 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:41.252+0000 7fc91e273700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc918109250 con 0x7fc918108850 2024-04-25T14:03:41.927 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:41.252+0000 7fc917fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc918108850 0x7fc918108c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.927 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:41.252+0000 7fc917fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc918108850 0x7fc918108c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44324/0 (socket says 172.21.15.96:44324) 2024-04-25T14:03:41.927 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:41.252+0000 7fc917fff700 1 -- 172.21.15.96:0/2265570608 learned_addr learned my addr 172.21.15.96:0/2265570608 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:03:41.927 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:41.252+0000 7fc917fff700 1 -- 172.21.15.96:0/2265570608 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc918109a20 con 0x7fc918108850 2024-04-25T14:03:41.927 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:41.252+0000 7fc917fff700 1 --2- 172.21.15.96:0/2265570608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc918108850 0x7fc918108c60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc90800bb40 tx=0x7fc90800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe8cb6439615610 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:41.927 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:41.253+0000 7fc916ffd700 1 -- 172.21.15.96:0/2265570608 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc90800dbe0 con 0x7fc918108850 2024-04-25T14:03:41.928 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:41.253+0000 7fc916ffd700 1 -- 172.21.15.96:0/2265570608 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc9080044c0 con 0x7fc918108850 2024-04-25T14:03:41.928 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:41.253+0000 7fc916ffd700 1 -- 172.21.15.96:0/2265570608 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc908010470 con 0x7fc918108850 2024-04-25T14:03:41.928 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:41.254+0000 7fc91e273700 1 -- 172.21.15.96:0/2265570608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc918108850 msgr2=0x7fc918108c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:41.928 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:41.254+0000 7fc91e273700 1 --2- 172.21.15.96:0/2265570608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc918108850 0x7fc918108c60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc90800bb40 tx=0x7fc90800be50 comp rx=0 tx=0).stop 2024-04-25T14:03:41.928 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:41.254+0000 7fc91e273700 1 -- 172.21.15.96:0/2265570608 shutdown_connections 2024-04-25T14:03:41.928 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:41.254+0000 7fc91e273700 1 --2- 172.21.15.96:0/2265570608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc918108850 0x7fc918108c60 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.928 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:41.254+0000 7fc91e273700 1 -- 172.21.15.96:0/2265570608 >> 172.21.15.96:0/2265570608 conn(0x7fc918103ef0 msgr2=0x7fc918106340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:41.928 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:41.254+0000 7fc91e273700 1 -- 172.21.15.96:0/2265570608 shutdown_connections 2024-04-25T14:03:41.928 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:41.254+0000 7fc91e273700 1 -- 172.21.15.96:0/2265570608 wait complete. 2024-04-25T14:03:41.928 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:41.254+0000 7fc91e273700 1 Processor -- start 2024-04-25T14:03:41.928 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:41.254+0000 7fc91e273700 1 -- start start 2024-04-25T14:03:41.928 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:41.255+0000 7fc91e273700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc918108850 0x7fc91819ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.928 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:41.255+0000 7fc91e273700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc91807c0d0 con 0x7fc918108850 2024-04-25T14:03:41.928 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:41.255+0000 7fc917fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc918108850 0x7fc91819ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.928 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:41.255+0000 7fc917fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc918108850 0x7fc91819ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44326/0 (socket says 172.21.15.96:44326) 2024-04-25T14:03:41.929 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:41.255+0000 7fc917fff700 1 -- 172.21.15.96:0/1411721614 learned_addr learned my addr 172.21.15.96:0/1411721614 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:03:41.929 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:41.255+0000 7fc917fff700 1 -- 172.21.15.96:0/1411721614 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc91819bfa0 con 0x7fc918108850 2024-04-25T14:03:41.929 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:41.255+0000 7fc917fff700 1 --2- 172.21.15.96:0/1411721614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc918108850 0x7fc91819ba60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc908003c30 tx=0x7fc908003d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:41.929 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:41.255+0000 7fc9157fa700 1 -- 172.21.15.96:0/1411721614 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc908027070 con 0x7fc918108850 2024-04-25T14:03:41.929 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:41.256+0000 7fc91e273700 1 -- 172.21.15.96:0/1411721614 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc91819c230 con 0x7fc918108850 2024-04-25T14:03:41.929 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:41.256+0000 7fc9157fa700 1 -- 172.21.15.96:0/1411721614 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc908004300 con 0x7fc918108850 2024-04-25T14:03:41.929 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:41.256+0000 7fc91e273700 1 -- 172.21.15.96:0/1411721614 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc91819ee40 con 0x7fc918108850 2024-04-25T14:03:41.929 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:41.256+0000 7fc9157fa700 1 -- 172.21.15.96:0/1411721614 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9080229d0 con 0x7fc918108850 2024-04-25T14:03:41.929 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:41.257+0000 7fc91e273700 1 -- 172.21.15.96:0/1411721614 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc918064cf0 con 0x7fc918108850 2024-04-25T14:03:41.929 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:41.257+0000 7fc9157fa700 1 -- 172.21.15.96:0/1411721614 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 2) ==== 49272+0+0 (secure 0 0 0) 0x7fc90802a070 con 0x7fc918108850 2024-04-25T14:03:41.929 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:41.258+0000 7fc9157fa700 1 -- 172.21.15.96:0/1411721614 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc90801f070 con 0x7fc918108850 2024-04-25T14:03:41.929 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:41.262+0000 7fc9157fa700 1 -- 172.21.15.96:0/1411721614 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fc908015070 con 0x7fc918108850 2024-04-25T14:03:41.929 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:41.334+0000 7fc91e273700 1 -- 172.21.15.96:0/1411721614 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc918194cc0 con 0x7fc918108850 2024-04-25T14:03:41.929 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:41.335+0000 7fc9157fa700 1 -- 172.21.15.96:0/1411721614 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fc918194cc0 con 0x7fc918108850 2024-04-25T14:03:41.930 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:41.335+0000 7fc91e273700 1 -- 172.21.15.96:0/1411721614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc918108850 msgr2=0x7fc91819ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:41.930 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:41.335+0000 7fc91e273700 1 --2- 172.21.15.96:0/1411721614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc918108850 0x7fc91819ba60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc908003c30 tx=0x7fc908003d10 comp rx=0 tx=0).stop 2024-04-25T14:03:41.930 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:41.335+0000 7fc91e273700 1 -- 172.21.15.96:0/1411721614 shutdown_connections 2024-04-25T14:03:41.930 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:41.335+0000 7fc91e273700 1 --2- 172.21.15.96:0/1411721614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc918108850 0x7fc91819ba60 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.930 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:41.336+0000 7fc91e273700 1 -- 172.21.15.96:0/1411721614 >> 172.21.15.96:0/1411721614 conn(0x7fc918103ef0 msgr2=0x7fc918191d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:41.930 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:41.336+0000 7fc91e273700 1 -- 172.21.15.96:0/1411721614 shutdown_connections 2024-04-25T14:03:41.930 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:41.336+0000 7fc91e273700 1 -- 172.21.15.96:0/1411721614 wait complete. 2024-04-25T14:03:41.930 INFO:teuthology.orchestra.run.smithi096.stdout:mgr not available, waiting (3/15)... 2024-04-25T14:03:42.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:42 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/1411721614' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T14:03:42.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:42 smithi096 ceph-mon[29207]: mgrmap e3: x(active, since 1.00871s) 2024-04-25T14:03:44.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:44 smithi096 ceph-mon[29207]: mgrmap e4: x(active, since 2s) 2024-04-25T14:03:45.272 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-04-25T14:03:45.272 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout { 2024-04-25T14:03:45.272 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "fsid": "59334182-030c-11ef-bc93-c7b262605968", 2024-04-25T14:03:45.272 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T14:03:45.272 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T14:03:45.272 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T14:03:45.272 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T14:03:45.272 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:03:45.273 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T14:03:45.273 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T14:03:45.273 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 0 2024-04-25T14:03:45.273 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:03:45.273 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T14:03:45.273 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T14:03:45.273 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:03:45.273 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-04-25T14:03:45.273 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T14:03:45.273 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:03:45.273 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T14:03:45.273 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T14:03:45.273 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:03:45.273 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T14:03:45.274 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:03:45.274 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T14:03:45.274 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T14:03:45.274 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T14:03:45.274 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T14:03:45.274 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T14:03:45.274 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T14:03:45.274 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:03:45.274 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T14:03:45.274 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T14:03:45.274 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T14:03:45.274 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T14:03:45.275 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T14:03:45.275 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T14:03:45.275 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T14:03:45.275 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T14:03:45.275 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T14:03:45.275 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:03:45.275 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T14:03:45.275 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:03:45.275 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T14:03:45.275 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T14:03:45.275 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:03:45.275 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T14:03:45.275 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T14:03:45.275 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T14:03:45.276 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T14:03:45.276 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T14:03:45.276 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T14:03:45.276 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T14:03:45.276 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:03:45.276 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T14:03:45.276 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:03:45.276 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T14:03:45.276 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:03:45.276 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T14:03:25.681351+0000", 2024-04-25T14:03:45.276 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T14:03:45.276 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:03:45.276 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T14:03:45.277 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout } 2024-04-25T14:03:45.277 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.387+0000 7f01b3ce7700 1 Processor -- start 2024-04-25T14:03:45.277 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.387+0000 7f01b3ce7700 1 -- start start 2024-04-25T14:03:45.277 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.387+0000 7f01b3ce7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01ac109980 0x7f01ac109d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:45.277 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.387+0000 7f01b3ce7700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac10a380 con 0x7f01ac109980 2024-04-25T14:03:45.277 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.387+0000 7f01b1a83700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01ac109980 0x7f01ac109d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:45.277 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.387+0000 7f01b1a83700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01ac109980 0x7f01ac109d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57922/0 (socket says 172.21.15.96:57922) 2024-04-25T14:03:45.277 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.387+0000 7f01b1a83700 1 -- 172.21.15.96:0/2278928450 learned_addr learned my addr 172.21.15.96:0/2278928450 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:03:45.277 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.388+0000 7f01b1a83700 1 -- 172.21.15.96:0/2278928450 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01ac10ab50 con 0x7f01ac109980 2024-04-25T14:03:45.277 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.388+0000 7f01b1a83700 1 --2- 172.21.15.96:0/2278928450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01ac109980 0x7f01ac109d90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f01a80098a0 tx=0x7f01a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f16340159082e79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:45.277 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.388+0000 7f01b0a81700 1 -- 172.21.15.96:0/2278928450 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01a8004160 con 0x7f01ac109980 2024-04-25T14:03:45.277 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.388+0000 7f01b0a81700 1 -- 172.21.15.96:0/2278928450 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f01a8004300 con 0x7f01ac109980 2024-04-25T14:03:45.277 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.388+0000 7f01b0a81700 1 -- 172.21.15.96:0/2278928450 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01a8011af0 con 0x7f01ac109980 2024-04-25T14:03:45.278 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.389+0000 7f01b3ce7700 1 -- 172.21.15.96:0/2278928450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01ac109980 msgr2=0x7f01ac109d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:45.278 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.389+0000 7f01b3ce7700 1 --2- 172.21.15.96:0/2278928450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01ac109980 0x7f01ac109d90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f01a80098a0 tx=0x7f01a8009bb0 comp rx=0 tx=0).stop 2024-04-25T14:03:45.278 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.389+0000 7f01b3ce7700 1 -- 172.21.15.96:0/2278928450 shutdown_connections 2024-04-25T14:03:45.278 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.389+0000 7f01b3ce7700 1 --2- 172.21.15.96:0/2278928450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01ac109980 0x7f01ac109d90 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:45.278 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.389+0000 7f01b3ce7700 1 -- 172.21.15.96:0/2278928450 >> 172.21.15.96:0/2278928450 conn(0x7f01ac105080 msgr2=0x7f01ac1074d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:45.278 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.389+0000 7f01b3ce7700 1 -- 172.21.15.96:0/2278928450 shutdown_connections 2024-04-25T14:03:45.278 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.389+0000 7f01b3ce7700 1 -- 172.21.15.96:0/2278928450 wait complete. 2024-04-25T14:03:45.278 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.389+0000 7f01b3ce7700 1 Processor -- start 2024-04-25T14:03:45.278 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.390+0000 7f01b3ce7700 1 -- start start 2024-04-25T14:03:45.278 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.390+0000 7f01b3ce7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01ac109980 0x7f01ac19dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:45.278 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.390+0000 7f01b3ce7700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac10df00 con 0x7f01ac109980 2024-04-25T14:03:45.278 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.390+0000 7f01b1a83700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01ac109980 0x7f01ac19dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:45.278 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.390+0000 7f01b1a83700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01ac109980 0x7f01ac19dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57924/0 (socket says 172.21.15.96:57924) 2024-04-25T14:03:45.279 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.390+0000 7f01b1a83700 1 -- 172.21.15.96:0/307526456 learned_addr learned my addr 172.21.15.96:0/307526456 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:03:45.279 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.390+0000 7f01b1a83700 1 -- 172.21.15.96:0/307526456 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01ac19e270 con 0x7f01ac109980 2024-04-25T14:03:45.279 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.390+0000 7f01b1a83700 1 --2- 172.21.15.96:0/307526456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01ac109980 0x7f01ac19dd30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f01a8011890 tx=0x7f01a8011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:45.279 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.390+0000 7f01a2ffd700 1 -- 172.21.15.96:0/307526456 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01a80039e0 con 0x7f01ac109980 2024-04-25T14:03:45.279 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.390+0000 7f01a2ffd700 1 -- 172.21.15.96:0/307526456 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f01a8003b80 con 0x7f01ac109980 2024-04-25T14:03:45.279 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.390+0000 7f01a2ffd700 1 -- 172.21.15.96:0/307526456 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01a80246e0 con 0x7f01ac109980 2024-04-25T14:03:45.279 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.390+0000 7f01b3ce7700 1 -- 172.21.15.96:0/307526456 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01ac19e500 con 0x7f01ac109980 2024-04-25T14:03:45.279 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.391+0000 7f01b3ce7700 1 -- 172.21.15.96:0/307526456 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01ac1a1100 con 0x7f01ac109980 2024-04-25T14:03:45.279 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.391+0000 7f01a2ffd700 1 -- 172.21.15.96:0/307526456 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 4) ==== 49505+0+0 (secure 0 0 0) 0x7f01a8022020 con 0x7f01ac109980 2024-04-25T14:03:45.279 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.392+0000 7f01b3ce7700 1 -- 172.21.15.96:0/307526456 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01ac064cf0 con 0x7f01ac109980 2024-04-25T14:03:45.279 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.392+0000 7f01a2ffd700 1 --2- 172.21.15.96:0/307526456 >> [v2:172.21.15.96:6800/2348397481,v1:172.21.15.96:6801/2348397481] conn(0x7f019803d630 0x7f019803fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:45.279 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.392+0000 7f01a2ffd700 1 -- 172.21.15.96:0/307526456 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f01a8052310 con 0x7f01ac109980 2024-04-25T14:03:45.279 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.392+0000 7f01b1282700 1 --2- 172.21.15.96:0/307526456 >> [v2:172.21.15.96:6800/2348397481,v1:172.21.15.96:6801/2348397481] conn(0x7f019803d630 0x7f019803fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:45.280 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.392+0000 7f01b1282700 1 --2- 172.21.15.96:0/307526456 >> [v2:172.21.15.96:6800/2348397481,v1:172.21.15.96:6801/2348397481] conn(0x7f019803d630 0x7f019803fb40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f019c006fd0 tx=0x7f019c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:45.280 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.397+0000 7f01a2ffd700 1 -- 172.21.15.96:0/307526456 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f01a800b3e0 con 0x7f01ac109980 2024-04-25T14:03:45.280 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.672+0000 7f01b3ce7700 1 -- 172.21.15.96:0/307526456 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f01ac1a1320 con 0x7f01ac109980 2024-04-25T14:03:45.280 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.673+0000 7f01a2ffd700 1 -- 172.21.15.96:0/307526456 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f01ac1a1320 con 0x7f01ac109980 2024-04-25T14:03:45.280 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.676+0000 7f01b3ce7700 1 -- 172.21.15.96:0/307526456 >> [v2:172.21.15.96:6800/2348397481,v1:172.21.15.96:6801/2348397481] conn(0x7f019803d630 msgr2=0x7f019803fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:45.280 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.676+0000 7f01b3ce7700 1 --2- 172.21.15.96:0/307526456 >> [v2:172.21.15.96:6800/2348397481,v1:172.21.15.96:6801/2348397481] conn(0x7f019803d630 0x7f019803fb40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f019c006fd0 tx=0x7f019c006e40 comp rx=0 tx=0).stop 2024-04-25T14:03:45.280 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.676+0000 7f01b3ce7700 1 -- 172.21.15.96:0/307526456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01ac109980 msgr2=0x7f01ac19dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:45.280 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.676+0000 7f01b3ce7700 1 --2- 172.21.15.96:0/307526456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01ac109980 0x7f01ac19dd30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f01a8011890 tx=0x7f01a8011970 comp rx=0 tx=0).stop 2024-04-25T14:03:45.280 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.676+0000 7f01b3ce7700 1 -- 172.21.15.96:0/307526456 shutdown_connections 2024-04-25T14:03:45.280 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.676+0000 7f01b3ce7700 1 --2- 172.21.15.96:0/307526456 >> [v2:172.21.15.96:6800/2348397481,v1:172.21.15.96:6801/2348397481] conn(0x7f019803d630 0x7f019803fb40 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:45.280 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.676+0000 7f01b3ce7700 1 --2- 172.21.15.96:0/307526456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01ac109980 0x7f01ac19dd30 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:45.280 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.676+0000 7f01b3ce7700 1 -- 172.21.15.96:0/307526456 >> 172.21.15.96:0/307526456 conn(0x7f01ac105080 msgr2=0x7f01ac106900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:45.280 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.676+0000 7f01b3ce7700 1 -- 172.21.15.96:0/307526456 shutdown_connections 2024-04-25T14:03:45.280 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:44.676+0000 7f01b3ce7700 1 -- 172.21.15.96:0/307526456 wait complete. 2024-04-25T14:03:45.281 INFO:teuthology.orchestra.run.smithi096.stdout:mgr is available 2024-04-25T14:03:45.563 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:45 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/307526456' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T14:03:46.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:46 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/2215273266' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T14:03:46.449 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-04-25T14:03:46.449 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T14:03:46.449 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout fsid = 59334182-030c-11ef-bc93-c7b262605968 2024-04-25T14:03:46.450 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T14:03:46.450 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T14:03:46.450 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T14:03:46.450 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T14:03:46.450 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T14:03:46.450 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-04-25T14:03:46.450 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T14:03:46.450 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T14:03:46.450 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-04-25T14:03:46.450 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T14:03:46.450 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T14:03:46.450 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T14:03:46.450 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T14:03:46.450 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T14:03:46.450 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T14:03:46.451 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.697+0000 7f0dda870700 1 Processor -- start 2024-04-25T14:03:46.451 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.697+0000 7f0dda870700 1 -- start start 2024-04-25T14:03:46.451 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.697+0000 7f0dda870700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0dd41099c0 0x7f0dd4109dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:46.451 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.697+0000 7f0dda870700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd410a3c0 con 0x7f0dd41099c0 2024-04-25T14:03:46.451 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.698+0000 7f0dd3fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0dd41099c0 0x7f0dd4109dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:46.451 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.698+0000 7f0dd3fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0dd41099c0 0x7f0dd4109dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57926/0 (socket says 172.21.15.96:57926) 2024-04-25T14:03:46.451 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.698+0000 7f0dd3fff700 1 -- 172.21.15.96:0/1800421806 learned_addr learned my addr 172.21.15.96:0/1800421806 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:03:46.451 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.698+0000 7f0dd3fff700 1 -- 172.21.15.96:0/1800421806 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dd410ab90 con 0x7f0dd41099c0 2024-04-25T14:03:46.451 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.698+0000 7f0dd3fff700 1 --2- 172.21.15.96:0/1800421806 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0dd41099c0 0x7f0dd4109dd0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0dc40098a0 tx=0x7f0dc4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca00e332d8882570 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:46.451 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.698+0000 7f0dd2ffd700 1 -- 172.21.15.96:0/1800421806 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0dc4004160 con 0x7f0dd41099c0 2024-04-25T14:03:46.451 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.698+0000 7f0dd2ffd700 1 -- 172.21.15.96:0/1800421806 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0dc4004300 con 0x7f0dd41099c0 2024-04-25T14:03:46.451 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.698+0000 7f0dd2ffd700 1 -- 172.21.15.96:0/1800421806 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0dc4011af0 con 0x7f0dd41099c0 2024-04-25T14:03:46.451 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.699+0000 7f0dda870700 1 -- 172.21.15.96:0/1800421806 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0dd41099c0 msgr2=0x7f0dd4109dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:46.452 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.699+0000 7f0dda870700 1 --2- 172.21.15.96:0/1800421806 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0dd41099c0 0x7f0dd4109dd0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0dc40098a0 tx=0x7f0dc4009bb0 comp rx=0 tx=0).stop 2024-04-25T14:03:46.452 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.699+0000 7f0dda870700 1 -- 172.21.15.96:0/1800421806 shutdown_connections 2024-04-25T14:03:46.452 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.699+0000 7f0dda870700 1 --2- 172.21.15.96:0/1800421806 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0dd41099c0 0x7f0dd4109dd0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:46.452 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.699+0000 7f0dda870700 1 -- 172.21.15.96:0/1800421806 >> 172.21.15.96:0/1800421806 conn(0x7f0dd41050e0 msgr2=0x7f0dd4107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:46.452 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.699+0000 7f0dda870700 1 -- 172.21.15.96:0/1800421806 shutdown_connections 2024-04-25T14:03:46.452 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.699+0000 7f0dda870700 1 -- 172.21.15.96:0/1800421806 wait complete. 2024-04-25T14:03:46.452 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.700+0000 7f0dda870700 1 Processor -- start 2024-04-25T14:03:46.452 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.700+0000 7f0dda870700 1 -- start start 2024-04-25T14:03:46.452 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.700+0000 7f0dda870700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0dd41099c0 0x7f0dd419dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:46.452 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.700+0000 7f0dda870700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd410df40 con 0x7f0dd41099c0 2024-04-25T14:03:46.452 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.700+0000 7f0dd3fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0dd41099c0 0x7f0dd419dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:46.452 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.700+0000 7f0dd3fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0dd41099c0 0x7f0dd419dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57928/0 (socket says 172.21.15.96:57928) 2024-04-25T14:03:46.452 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.700+0000 7f0dd3fff700 1 -- 172.21.15.96:0/2215273266 learned_addr learned my addr 172.21.15.96:0/2215273266 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:03:46.452 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.701+0000 7f0dd3fff700 1 -- 172.21.15.96:0/2215273266 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dd419e2b0 con 0x7f0dd41099c0 2024-04-25T14:03:46.452 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.701+0000 7f0dd3fff700 1 --2- 172.21.15.96:0/2215273266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0dd41099c0 0x7f0dd419dd70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4009fb0 tx=0x7f0dc4011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:46.453 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.701+0000 7f0dd17fa700 1 -- 172.21.15.96:0/2215273266 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0dc40036a0 con 0x7f0dd41099c0 2024-04-25T14:03:46.453 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.701+0000 7f0dd17fa700 1 -- 172.21.15.96:0/2215273266 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0dc4003cd0 con 0x7f0dd41099c0 2024-04-25T14:03:46.453 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.701+0000 7f0dd17fa700 1 -- 172.21.15.96:0/2215273266 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0dc401a430 con 0x7f0dd41099c0 2024-04-25T14:03:46.453 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.701+0000 7f0dda870700 1 -- 172.21.15.96:0/2215273266 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dd419e540 con 0x7f0dd41099c0 2024-04-25T14:03:46.453 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.701+0000 7f0dda870700 1 -- 172.21.15.96:0/2215273266 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dd41a1150 con 0x7f0dd41099c0 2024-04-25T14:03:46.453 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.702+0000 7f0dd17fa700 1 -- 172.21.15.96:0/2215273266 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 4) ==== 49505+0+0 (secure 0 0 0) 0x7f0dc401a6a0 con 0x7f0dd41099c0 2024-04-25T14:03:46.453 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.702+0000 7f0dda870700 1 -- 172.21.15.96:0/2215273266 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dd4064cf0 con 0x7f0dd41099c0 2024-04-25T14:03:46.453 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.702+0000 7f0dd17fa700 1 --2- 172.21.15.96:0/2215273266 >> [v2:172.21.15.96:6800/2348397481,v1:172.21.15.96:6801/2348397481] conn(0x7f0dbc03d5e0 0x7f0dbc03faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:46.453 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.702+0000 7f0dd17fa700 1 -- 172.21.15.96:0/2215273266 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0dc40523e0 con 0x7f0dd41099c0 2024-04-25T14:03:46.453 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.702+0000 7f0dd37fe700 1 --2- 172.21.15.96:0/2215273266 >> [v2:172.21.15.96:6800/2348397481,v1:172.21.15.96:6801/2348397481] conn(0x7f0dbc03d5e0 0x7f0dbc03faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:46.453 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.703+0000 7f0dd37fe700 1 --2- 172.21.15.96:0/2215273266 >> [v2:172.21.15.96:6800/2348397481,v1:172.21.15.96:6801/2348397481] conn(0x7f0dbc03d5e0 0x7f0dbc03faf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8006fd0 tx=0x7f0dc8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:46.453 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.707+0000 7f0dd17fa700 1 -- 172.21.15.96:0/2215273266 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0dc4020240 con 0x7f0dd41099c0 2024-04-25T14:03:46.453 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.903+0000 7f0dda870700 1 -- 172.21.15.96:0/2215273266 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f0dd41a1370 con 0x7f0dd41099c0 2024-04-25T14:03:46.453 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.905+0000 7f0dd17fa700 1 -- 172.21.15.96:0/2215273266 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7f0dd41a1370 con 0x7f0dd41099c0 2024-04-25T14:03:46.453 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.907+0000 7f0dda870700 1 -- 172.21.15.96:0/2215273266 >> [v2:172.21.15.96:6800/2348397481,v1:172.21.15.96:6801/2348397481] conn(0x7f0dbc03d5e0 msgr2=0x7f0dbc03faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:46.454 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.907+0000 7f0dda870700 1 --2- 172.21.15.96:0/2215273266 >> [v2:172.21.15.96:6800/2348397481,v1:172.21.15.96:6801/2348397481] conn(0x7f0dbc03d5e0 0x7f0dbc03faf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8006fd0 tx=0x7f0dc8006e40 comp rx=0 tx=0).stop 2024-04-25T14:03:46.454 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.907+0000 7f0dda870700 1 -- 172.21.15.96:0/2215273266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0dd41099c0 msgr2=0x7f0dd419dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:46.454 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.907+0000 7f0dda870700 1 --2- 172.21.15.96:0/2215273266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0dd41099c0 0x7f0dd419dd70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4009fb0 tx=0x7f0dc4011890 comp rx=0 tx=0).stop 2024-04-25T14:03:46.454 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.907+0000 7f0dda870700 1 -- 172.21.15.96:0/2215273266 shutdown_connections 2024-04-25T14:03:46.454 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.907+0000 7f0dda870700 1 --2- 172.21.15.96:0/2215273266 >> [v2:172.21.15.96:6800/2348397481,v1:172.21.15.96:6801/2348397481] conn(0x7f0dbc03d5e0 0x7f0dbc03faf0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:46.454 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.907+0000 7f0dda870700 1 --2- 172.21.15.96:0/2215273266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0dd41099c0 0x7f0dd419dd70 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:46.454 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.907+0000 7f0dda870700 1 -- 172.21.15.96:0/2215273266 >> 172.21.15.96:0/2215273266 conn(0x7f0dd41050e0 msgr2=0x7f0dd4106940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:46.454 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.907+0000 7f0dda870700 1 -- 172.21.15.96:0/2215273266 shutdown_connections 2024-04-25T14:03:46.454 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:45.907+0000 7f0dda870700 1 -- 172.21.15.96:0/2215273266 wait complete. 2024-04-25T14:03:46.454 INFO:teuthology.orchestra.run.smithi096.stdout:Enabling cephadm module... 2024-04-25T14:03:47.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:47 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/71093988' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-25T14:03:47.609 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:47 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: ignoring --setuser ceph since I am not root 2024-04-25T14:03:47.609 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:47 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: ignoring --setgroup ceph since I am not root 2024-04-25T14:03:47.609 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:47 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:47.571+0000 7f26bd992200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:03:47.972 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:47 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:47.759+0000 7f26bd992200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:03:47.972 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:47 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:47.861+0000 7f26bd992200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:03:47.975 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.067+0000 7f5590e80700 1 Processor -- start 2024-04-25T14:03:47.981 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.067+0000 7f5590e80700 1 -- start start 2024-04-25T14:03:47.981 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.068+0000 7f5590e80700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f558c07f660 0x7f558c07da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:47.981 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.068+0000 7f5590e80700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f558c07e060 con 0x7f558c07f660 2024-04-25T14:03:47.981 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.068+0000 7f558a59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f558c07f660 0x7f558c07da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:47.981 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.068+0000 7f558a59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f558c07f660 0x7f558c07da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57930/0 (socket says 172.21.15.96:57930) 2024-04-25T14:03:47.981 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.068+0000 7f558a59c700 1 -- 172.21.15.96:0/1261481164 learned_addr learned my addr 172.21.15.96:0/1261481164 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:03:47.981 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.068+0000 7f558a59c700 1 -- 172.21.15.96:0/1261481164 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f558c07e1e0 con 0x7f558c07f660 2024-04-25T14:03:47.981 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.068+0000 7f558a59c700 1 --2- 172.21.15.96:0/1261481164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f558c07f660 0x7f558c07da90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f55800098a0 tx=0x7f5580009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8bbb7d8d559a859 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:47.982 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.068+0000 7f558959a700 1 -- 172.21.15.96:0/1261481164 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5580004160 con 0x7f558c07f660 2024-04-25T14:03:47.982 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.068+0000 7f558959a700 1 -- 172.21.15.96:0/1261481164 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5580004300 con 0x7f558c07f660 2024-04-25T14:03:47.982 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.068+0000 7f558959a700 1 -- 172.21.15.96:0/1261481164 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5580011af0 con 0x7f558c07f660 2024-04-25T14:03:47.982 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.069+0000 7f5590e80700 1 -- 172.21.15.96:0/1261481164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f558c07f660 msgr2=0x7f558c07da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:47.982 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.069+0000 7f5590e80700 1 --2- 172.21.15.96:0/1261481164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f558c07f660 0x7f558c07da90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f55800098a0 tx=0x7f5580009bb0 comp rx=0 tx=0).stop 2024-04-25T14:03:47.982 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.069+0000 7f5590e80700 1 -- 172.21.15.96:0/1261481164 shutdown_connections 2024-04-25T14:03:47.982 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.069+0000 7f5590e80700 1 --2- 172.21.15.96:0/1261481164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f558c07f660 0x7f558c07da90 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:47.982 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.069+0000 7f5590e80700 1 -- 172.21.15.96:0/1261481164 >> 172.21.15.96:0/1261481164 conn(0x7f558c105080 msgr2=0x7f558c1074d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:47.982 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.069+0000 7f5590e80700 1 -- 172.21.15.96:0/1261481164 shutdown_connections 2024-04-25T14:03:47.982 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.069+0000 7f5590e80700 1 -- 172.21.15.96:0/1261481164 wait complete. 2024-04-25T14:03:47.982 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.070+0000 7f5590e80700 1 Processor -- start 2024-04-25T14:03:47.982 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.070+0000 7f5590e80700 1 -- start start 2024-04-25T14:03:47.982 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.070+0000 7f5590e80700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f558c07f660 0x7f558c19ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:47.983 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.070+0000 7f5590e80700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f558c10ff80 con 0x7f558c07f660 2024-04-25T14:03:47.983 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.070+0000 7f558a59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f558c07f660 0x7f558c19ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:47.983 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.070+0000 7f558a59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f558c07f660 0x7f558c19ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57932/0 (socket says 172.21.15.96:57932) 2024-04-25T14:03:47.983 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.070+0000 7f558a59c700 1 -- 172.21.15.96:0/71093988 learned_addr learned my addr 172.21.15.96:0/71093988 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:03:47.983 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.071+0000 7f558a59c700 1 -- 172.21.15.96:0/71093988 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f558c1a0440 con 0x7f558c07f660 2024-04-25T14:03:47.983 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.071+0000 7f558a59c700 1 --2- 172.21.15.96:0/71093988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f558c07f660 0x7f558c19ff00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5580009fb0 tx=0x7f5580011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:47.983 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.071+0000 7f557b7fe700 1 -- 172.21.15.96:0/71093988 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55800036a0 con 0x7f558c07f660 2024-04-25T14:03:47.983 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.071+0000 7f557b7fe700 1 -- 172.21.15.96:0/71093988 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f558001a430 con 0x7f558c07f660 2024-04-25T14:03:47.983 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.071+0000 7f5590e80700 1 -- 172.21.15.96:0/71093988 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f558c1a06d0 con 0x7f558c07f660 2024-04-25T14:03:47.983 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.071+0000 7f557b7fe700 1 -- 172.21.15.96:0/71093988 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5580019550 con 0x7f558c07f660 2024-04-25T14:03:47.983 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.071+0000 7f5590e80700 1 -- 172.21.15.96:0/71093988 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f558c1a32e0 con 0x7f558c07f660 2024-04-25T14:03:47.983 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.072+0000 7f557b7fe700 1 -- 172.21.15.96:0/71093988 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 4) ==== 49505+0+0 (secure 0 0 0) 0x7f5580019850 con 0x7f558c07f660 2024-04-25T14:03:47.983 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.072+0000 7f5590e80700 1 -- 172.21.15.96:0/71093988 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f558c064cf0 con 0x7f558c07f660 2024-04-25T14:03:47.983 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.072+0000 7f557b7fe700 1 --2- 172.21.15.96:0/71093988 >> [v2:172.21.15.96:6800/2348397481,v1:172.21.15.96:6801/2348397481] conn(0x7f557403d590 0x7f557403faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:47.983 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.072+0000 7f557b7fe700 1 -- 172.21.15.96:0/71093988 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5580052680 con 0x7f558c07f660 2024-04-25T14:03:47.984 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.072+0000 7f5589d9b700 1 --2- 172.21.15.96:0/71093988 >> [v2:172.21.15.96:6800/2348397481,v1:172.21.15.96:6801/2348397481] conn(0x7f557403d590 0x7f557403faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:47.984 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.073+0000 7f5589d9b700 1 --2- 172.21.15.96:0/71093988 >> [v2:172.21.15.96:6800/2348397481,v1:172.21.15.96:6801/2348397481] conn(0x7f557403d590 0x7f557403faa0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f557c006fd0 tx=0x7f557c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:47.984 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.078+0000 7f557b7fe700 1 -- 172.21.15.96:0/71093988 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5580017e90 con 0x7f558c07f660 2024-04-25T14:03:47.984 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.329+0000 7f5590e80700 1 -- 172.21.15.96:0/71093988 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f558c1a3500 con 0x7f558c07f660 2024-04-25T14:03:47.984 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.381+0000 7f557b7fe700 1 -- 172.21.15.96:0/71093988 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f5580023390 con 0x7f558c07f660 2024-04-25T14:03:47.984 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.383+0000 7f557b7fe700 1 -- 172.21.15.96:0/71093988 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mgrmap(e 5) ==== 49516+0+0 (secure 0 0 0) 0x7f558000fd80 con 0x7f558c07f660 2024-04-25T14:03:47.984 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.383+0000 7f5590e80700 1 -- 172.21.15.96:0/71093988 >> [v2:172.21.15.96:6800/2348397481,v1:172.21.15.96:6801/2348397481] conn(0x7f557403d590 msgr2=0x7f557403faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:47.984 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.383+0000 7f5590e80700 1 --2- 172.21.15.96:0/71093988 >> [v2:172.21.15.96:6800/2348397481,v1:172.21.15.96:6801/2348397481] conn(0x7f557403d590 0x7f557403faa0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f557c006fd0 tx=0x7f557c006e40 comp rx=0 tx=0).stop 2024-04-25T14:03:47.984 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.384+0000 7f5590e80700 1 -- 172.21.15.96:0/71093988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f558c07f660 msgr2=0x7f558c19ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:47.984 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.384+0000 7f5590e80700 1 --2- 172.21.15.96:0/71093988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f558c07f660 0x7f558c19ff00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5580009fb0 tx=0x7f5580011940 comp rx=0 tx=0).stop 2024-04-25T14:03:47.984 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.384+0000 7f5590e80700 1 -- 172.21.15.96:0/71093988 shutdown_connections 2024-04-25T14:03:47.984 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.384+0000 7f5590e80700 1 --2- 172.21.15.96:0/71093988 >> [v2:172.21.15.96:6800/2348397481,v1:172.21.15.96:6801/2348397481] conn(0x7f557403d590 0x7f557403faa0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:47.984 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.384+0000 7f5590e80700 1 --2- 172.21.15.96:0/71093988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f558c07f660 0x7f558c19ff00 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:47.984 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.384+0000 7f5590e80700 1 -- 172.21.15.96:0/71093988 >> 172.21.15.96:0/71093988 conn(0x7f558c105080 msgr2=0x7f558c106980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:47.984 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.384+0000 7f5590e80700 1 -- 172.21.15.96:0/71093988 shutdown_connections 2024-04-25T14:03:47.985 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:47.384+0000 7f5590e80700 1 -- 172.21.15.96:0/71093988 wait complete. 2024-04-25T14:03:48.273 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:48 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:48.184+0000 7f26bd992200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:03:48.273 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:48 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:48.273+0000 7f26bd992200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:03:48.608 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:48 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/71093988' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-25T14:03:48.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:48 smithi096 ceph-mon[29207]: mgrmap e5: x(active, since 6s) 2024-04-25T14:03:49.320 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout { 2024-04-25T14:03:49.321 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-25T14:03:49.321 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T14:03:49.321 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T14:03:49.321 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T14:03:49.321 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout } 2024-04-25T14:03:49.321 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.444+0000 7fe871034700 1 Processor -- start 2024-04-25T14:03:49.321 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.444+0000 7fe871034700 1 -- start start 2024-04-25T14:03:49.321 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.444+0000 7fe871034700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe86c107c40 0x7fe86c108050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:49.321 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.444+0000 7fe871034700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe86c108620 con 0x7fe86c107c40 2024-04-25T14:03:49.321 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.444+0000 7fe86ad9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe86c107c40 0x7fe86c108050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:49.321 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.444+0000 7fe86ad9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe86c107c40 0x7fe86c108050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57942/0 (socket says 172.21.15.96:57942) 2024-04-25T14:03:49.321 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.444+0000 7fe86ad9d700 1 -- 172.21.15.96:0/808472898 learned_addr learned my addr 172.21.15.96:0/808472898 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:03:49.322 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.444+0000 7fe86ad9d700 1 -- 172.21.15.96:0/808472898 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe86c10ac40 con 0x7fe86c107c40 2024-04-25T14:03:49.322 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.445+0000 7fe86ad9d700 1 --2- 172.21.15.96:0/808472898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe86c107c40 0x7fe86c108050 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe85c0098a0 tx=0x7fe85c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8fb7e144a3a0c6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:49.322 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.445+0000 7fe869d9b700 1 -- 172.21.15.96:0/808472898 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe85c004160 con 0x7fe86c107c40 2024-04-25T14:03:49.322 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.445+0000 7fe869d9b700 1 -- 172.21.15.96:0/808472898 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe85c004300 con 0x7fe86c107c40 2024-04-25T14:03:49.322 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.445+0000 7fe869d9b700 1 -- 172.21.15.96:0/808472898 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe85c011af0 con 0x7fe86c107c40 2024-04-25T14:03:49.322 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.446+0000 7fe871034700 1 -- 172.21.15.96:0/808472898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe86c107c40 msgr2=0x7fe86c108050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:49.322 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.446+0000 7fe871034700 1 --2- 172.21.15.96:0/808472898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe86c107c40 0x7fe86c108050 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe85c0098a0 tx=0x7fe85c009bb0 comp rx=0 tx=0).stop 2024-04-25T14:03:49.322 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.446+0000 7fe871034700 1 -- 172.21.15.96:0/808472898 shutdown_connections 2024-04-25T14:03:49.322 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.446+0000 7fe871034700 1 --2- 172.21.15.96:0/808472898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe86c107c40 0x7fe86c108050 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:49.322 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.446+0000 7fe871034700 1 -- 172.21.15.96:0/808472898 >> 172.21.15.96:0/808472898 conn(0x7fe86c103670 msgr2=0x7fe86c105ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:49.322 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.446+0000 7fe871034700 1 -- 172.21.15.96:0/808472898 shutdown_connections 2024-04-25T14:03:49.322 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.446+0000 7fe871034700 1 -- 172.21.15.96:0/808472898 wait complete. 2024-04-25T14:03:49.323 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.447+0000 7fe871034700 1 Processor -- start 2024-04-25T14:03:49.323 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.447+0000 7fe871034700 1 -- start start 2024-04-25T14:03:49.323 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.447+0000 7fe871034700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe86c107c40 0x7fe86c10a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:49.323 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.447+0000 7fe871034700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe86c10de70 con 0x7fe86c107c40 2024-04-25T14:03:49.323 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.447+0000 7fe86ad9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe86c107c40 0x7fe86c10a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:49.323 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.447+0000 7fe86ad9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe86c107c40 0x7fe86c10a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57946/0 (socket says 172.21.15.96:57946) 2024-04-25T14:03:49.323 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.447+0000 7fe86ad9d700 1 -- 172.21.15.96:0/2562950975 learned_addr learned my addr 172.21.15.96:0/2562950975 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:03:49.323 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.447+0000 7fe86ad9d700 1 -- 172.21.15.96:0/2562950975 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe86c10a900 con 0x7fe86c107c40 2024-04-25T14:03:49.323 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.448+0000 7fe86ad9d700 1 --2- 172.21.15.96:0/2562950975 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe86c107c40 0x7fe86c10a3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe85c009fb0 tx=0x7fe85c011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:49.323 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.448+0000 7fe853fff700 1 -- 172.21.15.96:0/2562950975 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe85c0036a0 con 0x7fe86c107c40 2024-04-25T14:03:49.323 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.448+0000 7fe871034700 1 -- 172.21.15.96:0/2562950975 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe86c108b50 con 0x7fe86c107c40 2024-04-25T14:03:49.323 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.448+0000 7fe853fff700 1 -- 172.21.15.96:0/2562950975 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe85c024440 con 0x7fe86c107c40 2024-04-25T14:03:49.323 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.448+0000 7fe853fff700 1 -- 172.21.15.96:0/2562950975 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe85c01a5c0 con 0x7fe86c107c40 2024-04-25T14:03:49.323 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.448+0000 7fe871034700 1 -- 172.21.15.96:0/2562950975 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe86c109030 con 0x7fe86c107c40 2024-04-25T14:03:49.324 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.449+0000 7fe853fff700 1 -- 172.21.15.96:0/2562950975 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 49516+0+0 (secure 0 0 0) 0x7fe85c01a880 con 0x7fe86c107c40 2024-04-25T14:03:49.324 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.449+0000 7fe871034700 1 -- 172.21.15.96:0/2562950975 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe86c064cf0 con 0x7fe86c107c40 2024-04-25T14:03:49.324 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.449+0000 7fe853fff700 1 --2- 172.21.15.96:0/2562950975 >> [v2:172.21.15.96:6800/2348397481,v1:172.21.15.96:6801/2348397481] conn(0x7fe85403d5e0 0x7fe85403faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:49.324 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.449+0000 7fe853fff700 1 -- 172.21.15.96:0/2562950975 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe85c0526f0 con 0x7fe86c107c40 2024-04-25T14:03:49.324 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.449+0000 7fe86a59c700 1 -- 172.21.15.96:0/2562950975 >> [v2:172.21.15.96:6800/2348397481,v1:172.21.15.96:6801/2348397481] conn(0x7fe85403d5e0 msgr2=0x7fe85403faf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/2348397481 2024-04-25T14:03:49.324 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.449+0000 7fe86a59c700 1 --2- 172.21.15.96:0/2562950975 >> [v2:172.21.15.96:6800/2348397481,v1:172.21.15.96:6801/2348397481] conn(0x7fe85403d5e0 0x7fe85403faf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:03:49.324 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.455+0000 7fe853fff700 1 -- 172.21.15.96:0/2562950975 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe85c0245f0 con 0x7fe86c107c40 2024-04-25T14:03:49.324 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.650+0000 7fe86a59c700 1 -- 172.21.15.96:0/2562950975 >> [v2:172.21.15.96:6800/2348397481,v1:172.21.15.96:6801/2348397481] conn(0x7fe85403d5e0 msgr2=0x7fe85403faf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/2348397481 2024-04-25T14:03:49.324 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.650+0000 7fe86a59c700 1 --2- 172.21.15.96:0/2562950975 >> [v2:172.21.15.96:6800/2348397481,v1:172.21.15.96:6801/2348397481] conn(0x7fe85403d5e0 0x7fe85403faf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T14:03:49.324 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.750+0000 7fe871034700 1 -- 172.21.15.96:0/2562950975 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fe86c1096f0 con 0x7fe86c107c40 2024-04-25T14:03:49.324 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.750+0000 7fe853fff700 1 -- 172.21.15.96:0/2562950975 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fe85c02c350 con 0x7fe86c107c40 2024-04-25T14:03:49.324 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.752+0000 7fe871034700 1 -- 172.21.15.96:0/2562950975 >> [v2:172.21.15.96:6800/2348397481,v1:172.21.15.96:6801/2348397481] conn(0x7fe85403d5e0 msgr2=0x7fe85403faf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:03:49.324 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.752+0000 7fe871034700 1 --2- 172.21.15.96:0/2562950975 >> [v2:172.21.15.96:6800/2348397481,v1:172.21.15.96:6801/2348397481] conn(0x7fe85403d5e0 0x7fe85403faf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:49.324 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.752+0000 7fe871034700 1 -- 172.21.15.96:0/2562950975 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe86c107c40 msgr2=0x7fe86c10a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:49.325 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.752+0000 7fe871034700 1 --2- 172.21.15.96:0/2562950975 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe86c107c40 0x7fe86c10a3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe85c009fb0 tx=0x7fe85c011890 comp rx=0 tx=0).stop 2024-04-25T14:03:49.325 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.752+0000 7fe871034700 1 -- 172.21.15.96:0/2562950975 shutdown_connections 2024-04-25T14:03:49.325 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.752+0000 7fe871034700 1 --2- 172.21.15.96:0/2562950975 >> [v2:172.21.15.96:6800/2348397481,v1:172.21.15.96:6801/2348397481] conn(0x7fe85403d5e0 0x7fe85403faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:49.325 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.752+0000 7fe871034700 1 --2- 172.21.15.96:0/2562950975 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe86c107c40 0x7fe86c10a3c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:49.325 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.752+0000 7fe871034700 1 -- 172.21.15.96:0/2562950975 >> 172.21.15.96:0/2562950975 conn(0x7fe86c103670 msgr2=0x7fe86c103f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:49.325 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.752+0000 7fe871034700 1 -- 172.21.15.96:0/2562950975 shutdown_connections 2024-04-25T14:03:49.325 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:48.752+0000 7fe871034700 1 -- 172.21.15.96:0/2562950975 wait complete. 2024-04-25T14:03:49.325 INFO:teuthology.orchestra.run.smithi096.stdout:Waiting for the mgr to restart... 2024-04-25T14:03:49.325 INFO:teuthology.orchestra.run.smithi096.stdout:Waiting for mgr epoch 5... 2024-04-25T14:03:49.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:49 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/2562950975' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T14:03:49.687 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:49 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:49.506+0000 7f26bd992200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:03:49.688 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:49 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:49.687+0000 7f26bd992200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:03:50.109 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:49 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:49.859+0000 7f26bd992200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:03:50.439 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:50 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:50.147+0000 7f26bd992200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:03:50.439 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:50 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:50.238+0000 7f26bd992200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:03:50.859 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:50 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:50.439+0000 7f26bd992200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:03:50.859 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:50 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:50.631+0000 7f26bd992200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:03:51.609 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:51 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:51.225+0000 7f26bd992200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T14:03:51.945 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:51 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:51.648+0000 7f26bd992200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:03:51.946 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:51 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:51.745+0000 7f26bd992200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:03:51.946 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:51 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:51.945+0000 7f26bd992200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T14:03:52.735 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:52 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:52.462+0000 7f26bd992200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:03:52.736 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:52 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:52.556+0000 7f26bd992200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:03:53.027 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:52 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:52.800+0000 7f26bd992200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:03:53.027 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:52 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:52.931+0000 7f26bd992200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:03:53.027 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:53 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:53.027+0000 7f26bd992200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:03:53.359 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:53 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:53.143+0000 7f26bd992200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:03:53.859 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:53 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:53.441+0000 7f26bd992200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:03:54.299 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:54 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:54.032+0000 7f26bd992200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:03:54.299 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:54 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:03:54.247+0000 7f26bd992200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:03:54.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:54 smithi096 ceph-mon[29207]: Active manager daemon x restarted 2024-04-25T14:03:54.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:54 smithi096 ceph-mon[29207]: Activating manager daemon x 2024-04-25T14:03:54.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:54 smithi096 ceph-mon[29207]: osdmap e2: 0 total, 0 up, 0 in 2024-04-25T14:03:54.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:54 smithi096 ceph-mon[29207]: mgrmap e6: x(active, starting, since 0.00391852s) 2024-04-25T14:03:54.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:54 smithi096 ceph-mon[29207]: from='mgr.14120 172.21.15.96:0/4269437437' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:03:54.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:54 smithi096 ceph-mon[29207]: from='mgr.14120 172.21.15.96:0/4269437437' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T14:03:54.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:54 smithi096 ceph-mon[29207]: from='mgr.14120 172.21.15.96:0/4269437437' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T14:03:54.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:54 smithi096 ceph-mon[29207]: from='mgr.14120 172.21.15.96:0/4269437437' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T14:03:54.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:54 smithi096 ceph-mon[29207]: from='mgr.14120 172.21.15.96:0/4269437437' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T14:03:54.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:54 smithi096 ceph-mon[29207]: Manager daemon x is now available 2024-04-25T14:03:54.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:54 smithi096 ceph-mon[29207]: from='mgr.14120 172.21.15.96:0/4269437437' entity='mgr.x' 2024-04-25T14:03:54.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:54 smithi096 ceph-mon[29207]: from='mgr.14120 172.21.15.96:0/4269437437' entity='mgr.x' 2024-04-25T14:03:54.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:54 smithi096 ceph-mon[29207]: from='mgr.14120 172.21.15.96:0/4269437437' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:03:54.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:54 smithi096 ceph-mon[29207]: from='mgr.14120 172.21.15.96:0/4269437437' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:03:55.575 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:55 smithi096 ceph-mon[29207]: Found migration_current of "None". Setting to last migration. 2024-04-25T14:03:55.575 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:55 smithi096 ceph-mon[29207]: from='mgr.14120 172.21.15.96:0/4269437437' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T14:03:55.576 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:55 smithi096 ceph-mon[29207]: from='mgr.14120 172.21.15.96:0/4269437437' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T14:03:55.576 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:55 smithi096 ceph-mon[29207]: from='mgr.14120 172.21.15.96:0/4269437437' entity='mgr.x' 2024-04-25T14:03:55.576 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:55 smithi096 ceph-mon[29207]: from='mgr.14120 172.21.15.96:0/4269437437' entity='mgr.x' 2024-04-25T14:03:55.576 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:55 smithi096 ceph-mon[29207]: from='mgr.14120 172.21.15.96:0/4269437437' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:03:55.576 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:55 smithi096 ceph-mon[29207]: mgrmap e7: x(active, since 1.00693s) 2024-04-25T14:03:55.870 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout { 2024-04-25T14:03:55.870 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-25T14:03:55.870 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T14:03:55.870 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout } 2024-04-25T14:03:55.870 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:49.727+0000 7f8508ade700 1 Processor -- start 2024-04-25T14:03:55.870 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:49.727+0000 7f8508ade700 1 -- start start 2024-04-25T14:03:55.870 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:49.727+0000 7f8508ade700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8504107c70 0x7f8504108080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:55.870 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:49.727+0000 7f8508ade700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8504108650 con 0x7f8504107c70 2024-04-25T14:03:55.871 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:49.727+0000 7f850259c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8504107c70 0x7f8504108080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:55.871 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:49.727+0000 7f850259c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8504107c70 0x7f8504108080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57960/0 (socket says 172.21.15.96:57960) 2024-04-25T14:03:55.871 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:49.727+0000 7f850259c700 1 -- 172.21.15.96:0/1842742690 learned_addr learned my addr 172.21.15.96:0/1842742690 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:03:55.871 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:49.728+0000 7f850259c700 1 -- 172.21.15.96:0/1842742690 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f850410ac70 con 0x7f8504107c70 2024-04-25T14:03:55.871 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:49.728+0000 7f850259c700 1 --2- 172.21.15.96:0/1842742690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8504107c70 0x7f8504108080 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f84f40098a0 tx=0x7f84f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b072523d8137bf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:55.871 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:49.728+0000 7f850159a700 1 -- 172.21.15.96:0/1842742690 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84f4004160 con 0x7f8504107c70 2024-04-25T14:03:55.871 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:49.728+0000 7f850159a700 1 -- 172.21.15.96:0/1842742690 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f84f4004300 con 0x7f8504107c70 2024-04-25T14:03:55.871 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:49.728+0000 7f850159a700 1 -- 172.21.15.96:0/1842742690 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84f4011af0 con 0x7f8504107c70 2024-04-25T14:03:55.871 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:49.729+0000 7f8508ade700 1 -- 172.21.15.96:0/1842742690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8504107c70 msgr2=0x7f8504108080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:55.871 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:49.729+0000 7f8508ade700 1 --2- 172.21.15.96:0/1842742690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8504107c70 0x7f8504108080 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f84f40098a0 tx=0x7f84f4009bb0 comp rx=0 tx=0).stop 2024-04-25T14:03:55.871 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:49.729+0000 7f8508ade700 1 -- 172.21.15.96:0/1842742690 shutdown_connections 2024-04-25T14:03:55.871 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:49.729+0000 7f8508ade700 1 --2- 172.21.15.96:0/1842742690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8504107c70 0x7f8504108080 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:55.871 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:49.729+0000 7f8508ade700 1 -- 172.21.15.96:0/1842742690 >> 172.21.15.96:0/1842742690 conn(0x7f85041036a0 msgr2=0x7f8504105af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:55.871 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:49.729+0000 7f8508ade700 1 -- 172.21.15.96:0/1842742690 shutdown_connections 2024-04-25T14:03:55.872 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:49.729+0000 7f8508ade700 1 -- 172.21.15.96:0/1842742690 wait complete. 2024-04-25T14:03:55.872 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:49.730+0000 7f8508ade700 1 Processor -- start 2024-04-25T14:03:55.872 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:49.730+0000 7f8508ade700 1 -- start start 2024-04-25T14:03:55.872 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:49.730+0000 7f8508ade700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8504107c70 0x7f850419df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:55.872 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:49.730+0000 7f8508ade700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f850410dea0 con 0x7f8504107c70 2024-04-25T14:03:55.872 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:49.730+0000 7f850259c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8504107c70 0x7f850419df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:55.872 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:49.730+0000 7f850259c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8504107c70 0x7f850419df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57972/0 (socket says 172.21.15.96:57972) 2024-04-25T14:03:55.872 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:49.730+0000 7f850259c700 1 -- 172.21.15.96:0/1430036006 learned_addr learned my addr 172.21.15.96:0/1430036006 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:03:55.872 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:49.731+0000 7f850259c700 1 -- 172.21.15.96:0/1430036006 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f850419e4b0 con 0x7f8504107c70 2024-04-25T14:03:55.872 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:49.731+0000 7f850259c700 1 --2- 172.21.15.96:0/1430036006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8504107c70 0x7f850419df70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f84f4011890 tx=0x7f84f4011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:55.872 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:49.731+0000 7f84f37fe700 1 -- 172.21.15.96:0/1430036006 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84f40039e0 con 0x7f8504107c70 2024-04-25T14:03:55.872 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:49.731+0000 7f84f37fe700 1 -- 172.21.15.96:0/1430036006 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f84f4003b80 con 0x7f8504107c70 2024-04-25T14:03:55.872 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:49.731+0000 7f84f37fe700 1 -- 172.21.15.96:0/1430036006 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84f4024750 con 0x7f8504107c70 2024-04-25T14:03:55.873 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:49.731+0000 7f8508ade700 1 -- 172.21.15.96:0/1430036006 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f850419e740 con 0x7f8504107c70 2024-04-25T14:03:55.873 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:49.731+0000 7f8508ade700 1 -- 172.21.15.96:0/1430036006 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85041a1350 con 0x7f8504107c70 2024-04-25T14:03:55.873 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:49.732+0000 7f84f37fe700 1 -- 172.21.15.96:0/1430036006 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 49516+0+0 (secure 0 0 0) 0x7f84f4022020 con 0x7f8504107c70 2024-04-25T14:03:55.873 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:49.732+0000 7f84f37fe700 1 --2- 172.21.15.96:0/1430036006 >> [v2:172.21.15.96:6800/2348397481,v1:172.21.15.96:6801/2348397481] conn(0x7f84ec03d680 0x7f84ec03fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:55.873 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:49.732+0000 7f84f37fe700 1 -- 172.21.15.96:0/1430036006 --> [v2:172.21.15.96:6800/2348397481,v1:172.21.15.96:6801/2348397481] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f84ec0402a0 con 0x7f84ec03d680 2024-04-25T14:03:55.873 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:49.732+0000 7f84f37fe700 1 -- 172.21.15.96:0/1430036006 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f84f4053230 con 0x7f8504107c70 2024-04-25T14:03:55.873 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:49.732+0000 7f8501d9b700 1 -- 172.21.15.96:0/1430036006 >> [v2:172.21.15.96:6800/2348397481,v1:172.21.15.96:6801/2348397481] conn(0x7f84ec03d680 msgr2=0x7f84ec03fb90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/2348397481 2024-04-25T14:03:55.873 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:49.732+0000 7f8501d9b700 1 --2- 172.21.15.96:0/1430036006 >> [v2:172.21.15.96:6800/2348397481,v1:172.21.15.96:6801/2348397481] conn(0x7f84ec03d680 0x7f84ec03fb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:03:55.873 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:49.933+0000 7f8501d9b700 1 -- 172.21.15.96:0/1430036006 >> [v2:172.21.15.96:6800/2348397481,v1:172.21.15.96:6801/2348397481] conn(0x7f84ec03d680 msgr2=0x7f84ec03fb90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/2348397481 2024-04-25T14:03:55.873 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:49.933+0000 7f8501d9b700 1 --2- 172.21.15.96:0/1430036006 >> [v2:172.21.15.96:6800/2348397481,v1:172.21.15.96:6801/2348397481] conn(0x7f84ec03d680 0x7f84ec03fb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T14:03:55.873 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:50.334+0000 7f8501d9b700 1 -- 172.21.15.96:0/1430036006 >> [v2:172.21.15.96:6800/2348397481,v1:172.21.15.96:6801/2348397481] conn(0x7f84ec03d680 msgr2=0x7f84ec03fb90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/2348397481 2024-04-25T14:03:55.873 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:50.334+0000 7f8501d9b700 1 --2- 172.21.15.96:0/1430036006 >> [v2:172.21.15.96:6800/2348397481,v1:172.21.15.96:6801/2348397481] conn(0x7f84ec03d680 0x7f84ec03fb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T14:03:55.873 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:51.135+0000 7f8501d9b700 1 -- 172.21.15.96:0/1430036006 >> [v2:172.21.15.96:6800/2348397481,v1:172.21.15.96:6801/2348397481] conn(0x7f84ec03d680 msgr2=0x7f84ec03fb90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/2348397481 2024-04-25T14:03:55.873 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:51.135+0000 7f8501d9b700 1 --2- 172.21.15.96:0/1430036006 >> [v2:172.21.15.96:6800/2348397481,v1:172.21.15.96:6801/2348397481] conn(0x7f84ec03d680 0x7f84ec03fb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T14:03:55.874 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:52.736+0000 7f8501d9b700 1 -- 172.21.15.96:0/1430036006 >> [v2:172.21.15.96:6800/2348397481,v1:172.21.15.96:6801/2348397481] conn(0x7f84ec03d680 msgr2=0x7f84ec03fb90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/2348397481 2024-04-25T14:03:55.874 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:52.736+0000 7f8501d9b700 1 --2- 172.21.15.96:0/1430036006 >> [v2:172.21.15.96:6800/2348397481,v1:172.21.15.96:6801/2348397481] conn(0x7f84ec03d680 0x7f84ec03fb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T14:03:55.874 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:54.251+0000 7f84f37fe700 1 -- 172.21.15.96:0/1430036006 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mgrmap(e 6) ==== 49283+0+0 (secure 0 0 0) 0x7f84f4018350 con 0x7f8504107c70 2024-04-25T14:03:55.874 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:54.251+0000 7f84f37fe700 1 -- 172.21.15.96:0/1430036006 >> [v2:172.21.15.96:6800/2348397481,v1:172.21.15.96:6801/2348397481] conn(0x7f84ec03d680 msgr2=0x7f84ec03fb90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:03:55.874 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:54.251+0000 7f84f37fe700 1 --2- 172.21.15.96:0/1430036006 >> [v2:172.21.15.96:6800/2348397481,v1:172.21.15.96:6801/2348397481] conn(0x7f84ec03d680 0x7f84ec03fb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:55.874 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:55.252+0000 7f84f37fe700 1 -- 172.21.15.96:0/1430036006 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 7) ==== 49410+0+0 (secure 0 0 0) 0x7f84f4053da0 con 0x7f8504107c70 2024-04-25T14:03:55.874 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:55.253+0000 7f84f37fe700 1 --2- 172.21.15.96:0/1430036006 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f84ec040fc0 0x7f84ec0433e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:55.874 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:55.253+0000 7f84f37fe700 1 -- 172.21.15.96:0/1430036006 --> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f84f4029e50 con 0x7f84ec040fc0 2024-04-25T14:03:55.874 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:55.253+0000 7f8501d9b700 1 --2- 172.21.15.96:0/1430036006 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f84ec040fc0 0x7f84ec0433e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:55.874 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:55.254+0000 7f8501d9b700 1 --2- 172.21.15.96:0/1430036006 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f84ec040fc0 0x7f84ec0433e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f84f8003a10 tx=0x7f84f80092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:55.874 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:55.254+0000 7f84f37fe700 1 -- 172.21.15.96:0/1430036006 <== mgr.14120 v2:172.21.15.96:6800/433591062 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f84f4029e50 con 0x7f84ec040fc0 2024-04-25T14:03:55.874 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:55.263+0000 7f8508ade700 1 -- 172.21.15.96:0/1430036006 --> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f8504064cf0 con 0x7f84ec040fc0 2024-04-25T14:03:55.874 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:55.263+0000 7f84f37fe700 1 -- 172.21.15.96:0/1430036006 <== mgr.14120 v2:172.21.15.96:6800/433591062 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f8504064cf0 con 0x7f84ec040fc0 2024-04-25T14:03:55.875 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:55.263+0000 7f8508ade700 1 -- 172.21.15.96:0/1430036006 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f84ec040fc0 msgr2=0x7f84ec0433e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:55.875 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:55.263+0000 7f8508ade700 1 --2- 172.21.15.96:0/1430036006 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f84ec040fc0 0x7f84ec0433e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f84f8003a10 tx=0x7f84f80092b0 comp rx=0 tx=0).stop 2024-04-25T14:03:55.875 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:55.263+0000 7f8508ade700 1 -- 172.21.15.96:0/1430036006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8504107c70 msgr2=0x7f850419df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:55.875 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:55.263+0000 7f8508ade700 1 --2- 172.21.15.96:0/1430036006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8504107c70 0x7f850419df70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f84f4011890 tx=0x7f84f4011970 comp rx=0 tx=0).stop 2024-04-25T14:03:55.875 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:55.263+0000 7f8508ade700 1 -- 172.21.15.96:0/1430036006 shutdown_connections 2024-04-25T14:03:55.875 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:55.263+0000 7f8508ade700 1 --2- 172.21.15.96:0/1430036006 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f84ec040fc0 0x7f84ec0433e0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:55.875 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:55.263+0000 7f8508ade700 1 --2- 172.21.15.96:0/1430036006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8504107c70 0x7f850419df70 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:55.875 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:55.263+0000 7f8508ade700 1 --2- 172.21.15.96:0/1430036006 >> [v2:172.21.15.96:6800/2348397481,v1:172.21.15.96:6801/2348397481] conn(0x7f84ec03d680 0x7f84ec03fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:55.875 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:55.263+0000 7f8508ade700 1 -- 172.21.15.96:0/1430036006 >> 172.21.15.96:0/1430036006 conn(0x7f85041036a0 msgr2=0x7f8504103f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:55.875 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:55.263+0000 7f8508ade700 1 -- 172.21.15.96:0/1430036006 shutdown_connections 2024-04-25T14:03:55.875 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:55.263+0000 7f8508ade700 1 -- 172.21.15.96:0/1430036006 wait complete. 2024-04-25T14:03:55.875 INFO:teuthology.orchestra.run.smithi096.stdout:mgr epoch 5 is available 2024-04-25T14:03:55.875 INFO:teuthology.orchestra.run.smithi096.stdout:Setting orchestrator backend to cephadm... 2024-04-25T14:03:56.514 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:56 smithi096 ceph-mon[29207]: [25/Apr/2024:14:03:54] ENGINE Bus STARTING 2024-04-25T14:03:56.514 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:56 smithi096 ceph-mon[29207]: [25/Apr/2024:14:03:55] ENGINE Serving on http://172.21.15.96:8765 2024-04-25T14:03:56.514 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:56 smithi096 ceph-mon[29207]: [25/Apr/2024:14:03:55] ENGINE Serving on https://172.21.15.96:7150 2024-04-25T14:03:56.514 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:56 smithi096 ceph-mon[29207]: [25/Apr/2024:14:03:55] ENGINE Bus STARTED 2024-04-25T14:03:56.514 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:56 smithi096 ceph-mon[29207]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T14:03:56.514 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:56 smithi096 ceph-mon[29207]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T14:03:57.066 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.281+0000 7fb45701d700 1 Processor -- start 2024-04-25T14:03:57.066 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.281+0000 7fb45701d700 1 -- start start 2024-04-25T14:03:57.066 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.281+0000 7fb45701d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb45010a470 0x7fb45010c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:57.066 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.281+0000 7fb45701d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb45007cdc0 con 0x7fb45010a470 2024-04-25T14:03:57.066 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.281+0000 7fb454db9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb45010a470 0x7fb45010c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:57.066 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.282+0000 7fb454db9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb45010a470 0x7fb45010c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57788/0 (socket says 172.21.15.96:57788) 2024-04-25T14:03:57.066 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.282+0000 7fb454db9700 1 -- 172.21.15.96:0/2618009620 learned_addr learned my addr 172.21.15.96:0/2618009620 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:03:57.066 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.282+0000 7fb454db9700 1 -- 172.21.15.96:0/2618009620 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb45010cdd0 con 0x7fb45010a470 2024-04-25T14:03:57.066 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.282+0000 7fb454db9700 1 --2- 172.21.15.96:0/2618009620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb45010a470 0x7fb45010c890 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb4440098a0 tx=0x7fb444009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d195ca293bc313f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:57.067 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.282+0000 7fb44f7fe700 1 -- 172.21.15.96:0/2618009620 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb444004160 con 0x7fb45010a470 2024-04-25T14:03:57.067 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.282+0000 7fb44f7fe700 1 -- 172.21.15.96:0/2618009620 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb444004300 con 0x7fb45010a470 2024-04-25T14:03:57.067 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.282+0000 7fb44f7fe700 1 -- 172.21.15.96:0/2618009620 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb444011af0 con 0x7fb45010a470 2024-04-25T14:03:57.067 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.283+0000 7fb45701d700 1 -- 172.21.15.96:0/2618009620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb45010a470 msgr2=0x7fb45010c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:57.067 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.283+0000 7fb45701d700 1 --2- 172.21.15.96:0/2618009620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb45010a470 0x7fb45010c890 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb4440098a0 tx=0x7fb444009bb0 comp rx=0 tx=0).stop 2024-04-25T14:03:57.067 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.283+0000 7fb45701d700 1 -- 172.21.15.96:0/2618009620 shutdown_connections 2024-04-25T14:03:57.067 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.283+0000 7fb45701d700 1 --2- 172.21.15.96:0/2618009620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb45010a470 0x7fb45010c890 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:57.067 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.283+0000 7fb45701d700 1 -- 172.21.15.96:0/2618009620 >> 172.21.15.96:0/2618009620 conn(0x7fb450103ef0 msgr2=0x7fb450106340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:57.067 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.283+0000 7fb45701d700 1 -- 172.21.15.96:0/2618009620 shutdown_connections 2024-04-25T14:03:57.067 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.283+0000 7fb45701d700 1 -- 172.21.15.96:0/2618009620 wait complete. 2024-04-25T14:03:57.067 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.284+0000 7fb45701d700 1 Processor -- start 2024-04-25T14:03:57.067 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.284+0000 7fb45701d700 1 -- start start 2024-04-25T14:03:57.067 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.284+0000 7fb45701d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb45010a470 0x7fb450109bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:57.067 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.284+0000 7fb45701d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb45010dd30 con 0x7fb45010a470 2024-04-25T14:03:57.068 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.284+0000 7fb454db9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb45010a470 0x7fb450109bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:57.068 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.284+0000 7fb454db9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb45010a470 0x7fb450109bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57794/0 (socket says 172.21.15.96:57794) 2024-04-25T14:03:57.068 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.284+0000 7fb454db9700 1 -- 172.21.15.96:0/349998055 learned_addr learned my addr 172.21.15.96:0/349998055 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:03:57.068 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.284+0000 7fb454db9700 1 -- 172.21.15.96:0/349998055 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb45010a130 con 0x7fb45010a470 2024-04-25T14:03:57.068 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.285+0000 7fb454db9700 1 --2- 172.21.15.96:0/349998055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb45010a470 0x7fb450109bf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb444009fb0 tx=0x7fb444011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:57.068 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.285+0000 7fb44dffb700 1 -- 172.21.15.96:0/349998055 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4440036a0 con 0x7fb45010a470 2024-04-25T14:03:57.068 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.285+0000 7fb44dffb700 1 -- 172.21.15.96:0/349998055 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb44401a430 con 0x7fb45010a470 2024-04-25T14:03:57.068 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.285+0000 7fb45701d700 1 -- 172.21.15.96:0/349998055 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb450108380 con 0x7fb45010a470 2024-04-25T14:03:57.068 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.285+0000 7fb44dffb700 1 -- 172.21.15.96:0/349998055 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb444019550 con 0x7fb45010a470 2024-04-25T14:03:57.068 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.285+0000 7fb45701d700 1 -- 172.21.15.96:0/349998055 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb450108860 con 0x7fb45010a470 2024-04-25T14:03:57.068 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.286+0000 7fb44dffb700 1 -- 172.21.15.96:0/349998055 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 7) ==== 49410+0+0 (secure 0 0 0) 0x7fb444019800 con 0x7fb45010a470 2024-04-25T14:03:57.068 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.286+0000 7fb44dffb700 1 --2- 172.21.15.96:0/349998055 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7fb44003d500 0x7fb44003fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:57.069 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.286+0000 7fb44dffb700 1 -- 172.21.15.96:0/349998055 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb444053490 con 0x7fb45010a470 2024-04-25T14:03:57.069 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.286+0000 7fb44ffff700 1 --2- 172.21.15.96:0/349998055 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7fb44003d500 0x7fb44003fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:57.069 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.286+0000 7fb45701d700 1 -- 172.21.15.96:0/349998055 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb450064cf0 con 0x7fb45010a470 2024-04-25T14:03:57.069 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.287+0000 7fb44ffff700 1 --2- 172.21.15.96:0/349998055 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7fb44003d500 0x7fb44003fa10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb43c006fd0 tx=0x7fb43c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:57.069 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.292+0000 7fb44dffb700 1 -- 172.21.15.96:0/349998055 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb444017e90 con 0x7fb45010a470 2024-04-25T14:03:57.069 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.307+0000 7fb44dffb700 1 -- 172.21.15.96:0/349998055 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7fb44400fd80 con 0x7fb45010a470 2024-04-25T14:03:57.069 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.514+0000 7fb45701d700 1 -- 172.21.15.96:0/349998055 --> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fb450108e70 con 0x7fb44003d500 2024-04-25T14:03:57.069 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.519+0000 7fb44dffb700 1 -- 172.21.15.96:0/349998055 <== mgr.14120 v2:172.21.15.96:6800/433591062 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb450108e70 con 0x7fb44003d500 2024-04-25T14:03:57.069 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.521+0000 7fb45701d700 1 -- 172.21.15.96:0/349998055 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7fb44003d500 msgr2=0x7fb44003fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:57.069 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.521+0000 7fb45701d700 1 --2- 172.21.15.96:0/349998055 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7fb44003d500 0x7fb44003fa10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb43c006fd0 tx=0x7fb43c006e40 comp rx=0 tx=0).stop 2024-04-25T14:03:57.069 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.521+0000 7fb45701d700 1 -- 172.21.15.96:0/349998055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb45010a470 msgr2=0x7fb450109bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:57.069 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.521+0000 7fb45701d700 1 --2- 172.21.15.96:0/349998055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb45010a470 0x7fb450109bf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb444009fb0 tx=0x7fb444011940 comp rx=0 tx=0).stop 2024-04-25T14:03:57.069 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.521+0000 7fb45701d700 1 -- 172.21.15.96:0/349998055 shutdown_connections 2024-04-25T14:03:57.070 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.521+0000 7fb45701d700 1 --2- 172.21.15.96:0/349998055 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7fb44003d500 0x7fb44003fa10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:57.070 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.521+0000 7fb45701d700 1 --2- 172.21.15.96:0/349998055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb45010a470 0x7fb450109bf0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:57.070 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.521+0000 7fb45701d700 1 -- 172.21.15.96:0/349998055 >> 172.21.15.96:0/349998055 conn(0x7fb450103ef0 msgr2=0x7fb450104820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:57.070 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.522+0000 7fb45701d700 1 -- 172.21.15.96:0/349998055 shutdown_connections 2024-04-25T14:03:57.070 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:56.522+0000 7fb45701d700 1 -- 172.21.15.96:0/349998055 wait complete. 2024-04-25T14:03:57.485 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:57 smithi096 ceph-mon[29207]: mgrmap e8: x(active, since 2s) 2024-04-25T14:03:57.485 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:57 smithi096 ceph-mon[29207]: from='mgr.14120 172.21.15.96:0/4269437437' entity='mgr.x' 2024-04-25T14:03:57.485 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:57 smithi096 ceph-mon[29207]: from='mgr.14120 172.21.15.96:0/4269437437' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:03:58.356 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-25T14:03:58.356 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.520+0000 7ff20f1d1700 1 Processor -- start 2024-04-25T14:03:58.356 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.520+0000 7ff20f1d1700 1 -- start start 2024-04-25T14:03:58.356 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.520+0000 7ff20f1d1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff20810a4a0 0x7ff20810c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:58.356 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.520+0000 7ff20f1d1700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff20807cdc0 con 0x7ff20810a4a0 2024-04-25T14:03:58.356 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.521+0000 7ff20cf6d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff20810a4a0 0x7ff20810c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:58.356 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.521+0000 7ff20cf6d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff20810a4a0 0x7ff20810c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57800/0 (socket says 172.21.15.96:57800) 2024-04-25T14:03:58.356 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.521+0000 7ff20cf6d700 1 -- 172.21.15.96:0/494296872 learned_addr learned my addr 172.21.15.96:0/494296872 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:03:58.356 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.521+0000 7ff20cf6d700 1 -- 172.21.15.96:0/494296872 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff20810ce00 con 0x7ff20810a4a0 2024-04-25T14:03:58.357 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.521+0000 7ff20cf6d700 1 --2- 172.21.15.96:0/494296872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff20810a4a0 0x7ff20810c8c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8009e80 tx=0x7ff1f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6af438ffe3f0034 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:58.357 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.521+0000 7ff2077fe700 1 -- 172.21.15.96:0/494296872 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1f8017070 con 0x7ff20810a4a0 2024-04-25T14:03:58.357 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.521+0000 7ff2077fe700 1 -- 172.21.15.96:0/494296872 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff1f800b8c0 con 0x7ff20810a4a0 2024-04-25T14:03:58.357 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.521+0000 7ff2077fe700 1 -- 172.21.15.96:0/494296872 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1f8004490 con 0x7ff20810a4a0 2024-04-25T14:03:58.357 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.522+0000 7ff20f1d1700 1 -- 172.21.15.96:0/494296872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff20810a4a0 msgr2=0x7ff20810c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:58.357 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.522+0000 7ff20f1d1700 1 --2- 172.21.15.96:0/494296872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff20810a4a0 0x7ff20810c8c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8009e80 tx=0x7ff1f800b330 comp rx=0 tx=0).stop 2024-04-25T14:03:58.357 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.522+0000 7ff20f1d1700 1 -- 172.21.15.96:0/494296872 shutdown_connections 2024-04-25T14:03:58.357 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.522+0000 7ff20f1d1700 1 --2- 172.21.15.96:0/494296872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff20810a4a0 0x7ff20810c8c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.357 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.522+0000 7ff20f1d1700 1 -- 172.21.15.96:0/494296872 >> 172.21.15.96:0/494296872 conn(0x7ff208103f20 msgr2=0x7ff208106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:58.357 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.522+0000 7ff20f1d1700 1 -- 172.21.15.96:0/494296872 shutdown_connections 2024-04-25T14:03:58.357 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.522+0000 7ff20f1d1700 1 -- 172.21.15.96:0/494296872 wait complete. 2024-04-25T14:03:58.357 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.523+0000 7ff20f1d1700 1 Processor -- start 2024-04-25T14:03:58.357 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.523+0000 7ff20f1d1700 1 -- start start 2024-04-25T14:03:58.357 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.523+0000 7ff20f1d1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff20810a4a0 0x7ff20819dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:58.358 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.523+0000 7ff20f1d1700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff20810dd60 con 0x7ff20810a4a0 2024-04-25T14:03:58.358 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.523+0000 7ff20cf6d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff20810a4a0 0x7ff20819dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:58.358 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.523+0000 7ff20cf6d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff20810a4a0 0x7ff20819dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57810/0 (socket says 172.21.15.96:57810) 2024-04-25T14:03:58.358 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.523+0000 7ff20cf6d700 1 -- 172.21.15.96:0/965359380 learned_addr learned my addr 172.21.15.96:0/965359380 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:03:58.358 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.523+0000 7ff20cf6d700 1 -- 172.21.15.96:0/965359380 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff20819e1d0 con 0x7ff20810a4a0 2024-04-25T14:03:58.358 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.523+0000 7ff20cf6d700 1 --2- 172.21.15.96:0/965359380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff20810a4a0 0x7ff20819dc90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff1f800b7b0 tx=0x7ff1f800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:58.358 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.524+0000 7ff205ffb700 1 -- 172.21.15.96:0/965359380 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1f8020020 con 0x7ff20810a4a0 2024-04-25T14:03:58.358 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.524+0000 7ff205ffb700 1 -- 172.21.15.96:0/965359380 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff1f8012430 con 0x7ff20810a4a0 2024-04-25T14:03:58.358 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.524+0000 7ff205ffb700 1 -- 172.21.15.96:0/965359380 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1f8017040 con 0x7ff20810a4a0 2024-04-25T14:03:58.358 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.524+0000 7ff20f1d1700 1 -- 172.21.15.96:0/965359380 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff20819e460 con 0x7ff20810a4a0 2024-04-25T14:03:58.358 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.524+0000 7ff20f1d1700 1 -- 172.21.15.96:0/965359380 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2081a1070 con 0x7ff20810a4a0 2024-04-25T14:03:58.358 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.525+0000 7ff205ffb700 1 -- 172.21.15.96:0/965359380 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7ff1f8023020 con 0x7ff20810a4a0 2024-04-25T14:03:58.358 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.525+0000 7ff20f1d1700 1 -- 172.21.15.96:0/965359380 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff208064cf0 con 0x7ff20810a4a0 2024-04-25T14:03:58.358 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.525+0000 7ff205ffb700 1 --2- 172.21.15.96:0/965359380 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7ff1f403d630 0x7ff1f403fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:58.359 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.525+0000 7ff205ffb700 1 -- 172.21.15.96:0/965359380 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff1f800f050 con 0x7ff20810a4a0 2024-04-25T14:03:58.359 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.525+0000 7ff207fff700 1 --2- 172.21.15.96:0/965359380 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7ff1f403d630 0x7ff1f403fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:58.359 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.526+0000 7ff207fff700 1 --2- 172.21.15.96:0/965359380 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7ff1f403d630 0x7ff1f403fb40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0006fd0 tx=0x7ff1f0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:58.359 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.530+0000 7ff205ffb700 1 -- 172.21.15.96:0/965359380 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff1f802a020 con 0x7ff20810a4a0 2024-04-25T14:03:58.359 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.733+0000 7ff20f1d1700 1 -- 172.21.15.96:0/965359380 --> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7ff208104de0 con 0x7ff1f403d630 2024-04-25T14:03:58.359 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.734+0000 7ff205ffb700 1 -- 172.21.15.96:0/965359380 <== mgr.14120 v2:172.21.15.96:6800/433591062 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7ff208104de0 con 0x7ff1f403d630 2024-04-25T14:03:58.359 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.736+0000 7ff20f1d1700 1 -- 172.21.15.96:0/965359380 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7ff1f403d630 msgr2=0x7ff1f403fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:58.359 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.736+0000 7ff20f1d1700 1 --2- 172.21.15.96:0/965359380 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7ff1f403d630 0x7ff1f403fb40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0006fd0 tx=0x7ff1f0006e40 comp rx=0 tx=0).stop 2024-04-25T14:03:58.359 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.736+0000 7ff20f1d1700 1 -- 172.21.15.96:0/965359380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff20810a4a0 msgr2=0x7ff20819dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:58.359 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.736+0000 7ff20f1d1700 1 --2- 172.21.15.96:0/965359380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff20810a4a0 0x7ff20819dc90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff1f800b7b0 tx=0x7ff1f800bd50 comp rx=0 tx=0).stop 2024-04-25T14:03:58.359 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.736+0000 7ff20f1d1700 1 -- 172.21.15.96:0/965359380 shutdown_connections 2024-04-25T14:03:58.359 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.736+0000 7ff20f1d1700 1 --2- 172.21.15.96:0/965359380 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7ff1f403d630 0x7ff1f403fb40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.359 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.736+0000 7ff20f1d1700 1 --2- 172.21.15.96:0/965359380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff20810a4a0 0x7ff20819dc90 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.360 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.736+0000 7ff20f1d1700 1 -- 172.21.15.96:0/965359380 >> 172.21.15.96:0/965359380 conn(0x7ff208103f20 msgr2=0x7ff2081046d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:58.360 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.736+0000 7ff20f1d1700 1 -- 172.21.15.96:0/965359380 shutdown_connections 2024-04-25T14:03:58.360 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:57.736+0000 7ff20f1d1700 1 -- 172.21.15.96:0/965359380 wait complete. 2024-04-25T14:03:58.360 INFO:teuthology.orchestra.run.smithi096.stdout:Generating ssh key... 2024-04-25T14:03:58.560 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:58 smithi096 ceph-mon[29207]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:03:59.545 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:59 smithi096 ceph-mon[29207]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:03:59.545 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:59 smithi096 ceph-mon[29207]: from='mgr.14120 172.21.15.96:0/4269437437' entity='mgr.x' 2024-04-25T14:03:59.545 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:03:59 smithi096 ceph-mon[29207]: from='mgr.14120 172.21.15.96:0/4269437437' entity='mgr.x' 2024-04-25T14:03:59.545 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:59 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: Generating public/private rsa key pair. 2024-04-25T14:03:59.545 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:59 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: Your identification has been saved in /tmp/tmpy0e7o8uo/key. 2024-04-25T14:03:59.545 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:59 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: Your public key has been saved in /tmp/tmpy0e7o8uo/key.pub. 2024-04-25T14:03:59.545 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:59 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: The key fingerprint is: 2024-04-25T14:03:59.545 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:59 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: SHA256:4AwyttTRftv61XvH1piOVkhr38B38o5yO6BKqONCMRY ceph-59334182-030c-11ef-bc93-c7b262605968 2024-04-25T14:03:59.545 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:59 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: The key's randomart image is: 2024-04-25T14:03:59.546 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:59 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: +---[RSA 3072]----+ 2024-04-25T14:03:59.546 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:59 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: | .. | 2024-04-25T14:03:59.546 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:59 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: | E. .. | 2024-04-25T14:03:59.546 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:59 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: | =.o.. | 2024-04-25T14:03:59.546 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:59 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: | o++ +... . | 2024-04-25T14:03:59.546 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:59 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: | ..o o.So . + | 2024-04-25T14:03:59.546 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:59 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: | . .. . =.= o| 2024-04-25T14:03:59.546 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:59 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: | . . .. o.+.Xo| 2024-04-25T14:03:59.546 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:59 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: | . .. .. ..o.*oO| 2024-04-25T14:03:59.546 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:59 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: | oo. .o...+=*o| 2024-04-25T14:03:59.546 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:03:59 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: +----[SHA256]-----+ 2024-04-25T14:03:59.847 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:58.771+0000 7fea4ca9a700 1 Processor -- start 2024-04-25T14:03:59.848 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:58.771+0000 7fea4ca9a700 1 -- start start 2024-04-25T14:03:59.848 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:58.771+0000 7fea4ca9a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea48108920 0x7fea48108d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:59.848 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:58.771+0000 7fea4ca9a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fea48109320 con 0x7fea48108920 2024-04-25T14:03:59.848 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:58.771+0000 7fea4659c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea48108920 0x7fea48108d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:59.848 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:58.771+0000 7fea4659c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea48108920 0x7fea48108d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57812/0 (socket says 172.21.15.96:57812) 2024-04-25T14:03:59.848 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:58.771+0000 7fea4659c700 1 -- 172.21.15.96:0/834640728 learned_addr learned my addr 172.21.15.96:0/834640728 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:03:59.848 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:58.772+0000 7fea4659c700 1 -- 172.21.15.96:0/834640728 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea48109af0 con 0x7fea48108920 2024-04-25T14:03:59.848 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:58.772+0000 7fea4659c700 1 --2- 172.21.15.96:0/834640728 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea48108920 0x7fea48108d30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fea380098a0 tx=0x7fea38009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1998028e2f7d0cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:59.848 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:58.772+0000 7fea4559a700 1 -- 172.21.15.96:0/834640728 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea38004160 con 0x7fea48108920 2024-04-25T14:03:59.848 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:58.772+0000 7fea4559a700 1 -- 172.21.15.96:0/834640728 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fea38004300 con 0x7fea48108920 2024-04-25T14:03:59.848 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:58.772+0000 7fea4559a700 1 -- 172.21.15.96:0/834640728 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea38011af0 con 0x7fea48108920 2024-04-25T14:03:59.848 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:58.773+0000 7fea4ca9a700 1 -- 172.21.15.96:0/834640728 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea48108920 msgr2=0x7fea48108d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:59.848 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:58.773+0000 7fea4ca9a700 1 --2- 172.21.15.96:0/834640728 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea48108920 0x7fea48108d30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fea380098a0 tx=0x7fea38009bb0 comp rx=0 tx=0).stop 2024-04-25T14:03:59.849 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:58.773+0000 7fea4ca9a700 1 -- 172.21.15.96:0/834640728 shutdown_connections 2024-04-25T14:03:59.849 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:58.773+0000 7fea4ca9a700 1 --2- 172.21.15.96:0/834640728 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea48108920 0x7fea48108d30 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:59.849 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:58.773+0000 7fea4ca9a700 1 -- 172.21.15.96:0/834640728 >> 172.21.15.96:0/834640728 conn(0x7fea48103fe0 msgr2=0x7fea48106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:59.849 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:58.773+0000 7fea4ca9a700 1 -- 172.21.15.96:0/834640728 shutdown_connections 2024-04-25T14:03:59.849 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:58.773+0000 7fea4ca9a700 1 -- 172.21.15.96:0/834640728 wait complete. 2024-04-25T14:03:59.849 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:58.774+0000 7fea4ca9a700 1 Processor -- start 2024-04-25T14:03:59.849 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:58.774+0000 7fea4ca9a700 1 -- start start 2024-04-25T14:03:59.849 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:58.774+0000 7fea4ca9a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea48108920 0x7fea4819dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:59.850 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:58.774+0000 7fea4ca9a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fea4810cd20 con 0x7fea48108920 2024-04-25T14:03:59.850 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:58.774+0000 7fea4659c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea48108920 0x7fea4819dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:59.850 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:58.774+0000 7fea4659c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea48108920 0x7fea4819dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57828/0 (socket says 172.21.15.96:57828) 2024-04-25T14:03:59.850 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:58.774+0000 7fea4659c700 1 -- 172.21.15.96:0/3069828804 learned_addr learned my addr 172.21.15.96:0/3069828804 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:03:59.850 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:58.774+0000 7fea4659c700 1 -- 172.21.15.96:0/3069828804 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea4819e2a0 con 0x7fea48108920 2024-04-25T14:03:59.851 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:58.775+0000 7fea4659c700 1 --2- 172.21.15.96:0/3069828804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea48108920 0x7fea4819dd60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fea38009fb0 tx=0x7fea38011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:59.851 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:58.775+0000 7fea377fe700 1 -- 172.21.15.96:0/3069828804 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea380036a0 con 0x7fea48108920 2024-04-25T14:03:59.851 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:58.775+0000 7fea377fe700 1 -- 172.21.15.96:0/3069828804 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fea38003cd0 con 0x7fea48108920 2024-04-25T14:03:59.851 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:58.775+0000 7fea4ca9a700 1 -- 172.21.15.96:0/3069828804 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea4819e530 con 0x7fea48108920 2024-04-25T14:03:59.851 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:58.775+0000 7fea377fe700 1 -- 172.21.15.96:0/3069828804 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea3801a430 con 0x7fea48108920 2024-04-25T14:03:59.851 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:58.775+0000 7fea4ca9a700 1 -- 172.21.15.96:0/3069828804 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea481a1130 con 0x7fea48108920 2024-04-25T14:03:59.851 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:58.776+0000 7fea4ca9a700 1 -- 172.21.15.96:0/3069828804 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea48064cf0 con 0x7fea48108920 2024-04-25T14:03:59.851 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:58.776+0000 7fea377fe700 1 -- 172.21.15.96:0/3069828804 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7fea3801a6f0 con 0x7fea48108920 2024-04-25T14:03:59.851 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:58.776+0000 7fea377fe700 1 --2- 172.21.15.96:0/3069828804 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7fea3003d5e0 0x7fea3003faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:59.851 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:58.776+0000 7fea377fe700 1 -- 172.21.15.96:0/3069828804 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fea38052510 con 0x7fea48108920 2024-04-25T14:03:59.851 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:58.776+0000 7fea45d9b700 1 --2- 172.21.15.96:0/3069828804 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7fea3003d5e0 0x7fea3003faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:59.851 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:58.777+0000 7fea45d9b700 1 --2- 172.21.15.96:0/3069828804 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7fea3003d5e0 0x7fea3003faf0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fea3c006fd0 tx=0x7fea3c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:59.852 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:58.781+0000 7fea377fe700 1 -- 172.21.15.96:0/3069828804 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fea38017e90 con 0x7fea48108920 2024-04-25T14:03:59.852 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:58.982+0000 7fea4ca9a700 1 -- 172.21.15.96:0/3069828804 --> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fea4802e880 con 0x7fea3003d5e0 2024-04-25T14:03:59.852 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:59.239+0000 7fea377fe700 1 -- 172.21.15.96:0/3069828804 <== mgr.14120 v2:172.21.15.96:6800/433591062 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fea4802e880 con 0x7fea3003d5e0 2024-04-25T14:03:59.852 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:59.241+0000 7fea4ca9a700 1 -- 172.21.15.96:0/3069828804 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7fea3003d5e0 msgr2=0x7fea3003faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:59.852 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:59.241+0000 7fea4ca9a700 1 --2- 172.21.15.96:0/3069828804 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7fea3003d5e0 0x7fea3003faf0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fea3c006fd0 tx=0x7fea3c006e40 comp rx=0 tx=0).stop 2024-04-25T14:03:59.852 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:59.241+0000 7fea4ca9a700 1 -- 172.21.15.96:0/3069828804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea48108920 msgr2=0x7fea4819dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:59.852 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:59.241+0000 7fea4ca9a700 1 --2- 172.21.15.96:0/3069828804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea48108920 0x7fea4819dd60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fea38009fb0 tx=0x7fea38011890 comp rx=0 tx=0).stop 2024-04-25T14:03:59.852 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:59.241+0000 7fea4ca9a700 1 -- 172.21.15.96:0/3069828804 shutdown_connections 2024-04-25T14:03:59.852 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:59.241+0000 7fea4ca9a700 1 --2- 172.21.15.96:0/3069828804 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7fea3003d5e0 0x7fea3003faf0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:59.852 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:59.241+0000 7fea4ca9a700 1 --2- 172.21.15.96:0/3069828804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea48108920 0x7fea4819dd60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:59.852 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:59.241+0000 7fea4ca9a700 1 -- 172.21.15.96:0/3069828804 >> 172.21.15.96:0/3069828804 conn(0x7fea48103fe0 msgr2=0x7fea48194310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:59.852 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:59.241+0000 7fea4ca9a700 1 -- 172.21.15.96:0/3069828804 shutdown_connections 2024-04-25T14:03:59.852 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:03:59.241+0000 7fea4ca9a700 1 -- 172.21.15.96:0/3069828804 wait complete. 2024-04-25T14:04:00.478 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:00 smithi096 ceph-mon[29207]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:04:00.479 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:00 smithi096 ceph-mon[29207]: Generating ssh key... 2024-04-25T14:04:01.135 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDoqiYS3qctb73u6iSODcS9ylsh58ZGbXlGKnerDCuVOT1IINo8UUyVOv2+vTycoBZpsN233QmYjfjMerHSYnqLd6CTCezZzHoL78p8VNFQQ+nwy5WSakeFUKndNJnfQRsxe+FVDXROZrDNrKX92AAi6XaiKUjgUZlveBiHGazKcJhCx021RtEDQPKIMYDcwAo8LDtRPg/73Qh3rcGoAIpCXhilD34kvqAseQTqmeAnm+fqqgAcoQSGFu1UtdPS6qwxZiNwu0au3r06iLrqMTZptqG8wIqDaUdpnLz64pDgCG9tO+jkHuoeF5Wk39O1Vnh+fdokf25CeATWWyIlQQJXr1uwHWcr3qPM4aWgsGicf9shJBH+MQT10bRkdbyoFZ84IBpyBti6D3kfs6fV8H7V5RUL2/wWBw0Z8tLy8asn61rPjubnf0bpkn8Mq9GLX/Y/iS6emFCGfqU9CdJHSdjblT1/cmCVsQIiMWp0S+DLwjonOtxM2nPSfjpmvOoh1vc= ceph-59334182-030c-11ef-bc93-c7b262605968 2024-04-25T14:04:01.135 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.266+0000 7f4bc7bc5700 1 Processor -- start 2024-04-25T14:04:01.136 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.266+0000 7f4bc7bc5700 1 -- start start 2024-04-25T14:04:01.136 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.266+0000 7f4bc7bc5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4bc01099a0 0x7f4bc0109db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:01.136 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.266+0000 7f4bc7bc5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc010a3a0 con 0x7f4bc01099a0 2024-04-25T14:04:01.136 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.266+0000 7f4bc5961700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4bc01099a0 0x7f4bc0109db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:01.136 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.266+0000 7f4bc5961700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4bc01099a0 0x7f4bc0109db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57836/0 (socket says 172.21.15.96:57836) 2024-04-25T14:04:01.136 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.267+0000 7f4bc5961700 1 -- 172.21.15.96:0/936285289 learned_addr learned my addr 172.21.15.96:0/936285289 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:04:01.136 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.267+0000 7f4bc5961700 1 -- 172.21.15.96:0/936285289 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bc010ab70 con 0x7f4bc01099a0 2024-04-25T14:04:01.136 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.267+0000 7f4bc5961700 1 --2- 172.21.15.96:0/936285289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4bc01099a0 0x7f4bc0109db0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc0098a0 tx=0x7f4bbc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=878479e8f6afcee7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:01.136 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.267+0000 7f4bc495f700 1 -- 172.21.15.96:0/936285289 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4bbc004160 con 0x7f4bc01099a0 2024-04-25T14:04:01.136 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.267+0000 7f4bc495f700 1 -- 172.21.15.96:0/936285289 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4bbc004300 con 0x7f4bc01099a0 2024-04-25T14:04:01.136 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.267+0000 7f4bc495f700 1 -- 172.21.15.96:0/936285289 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4bbc011af0 con 0x7f4bc01099a0 2024-04-25T14:04:01.137 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.268+0000 7f4bc7bc5700 1 -- 172.21.15.96:0/936285289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4bc01099a0 msgr2=0x7f4bc0109db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:01.137 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.268+0000 7f4bc7bc5700 1 --2- 172.21.15.96:0/936285289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4bc01099a0 0x7f4bc0109db0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc0098a0 tx=0x7f4bbc009bb0 comp rx=0 tx=0).stop 2024-04-25T14:04:01.137 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.268+0000 7f4bc7bc5700 1 -- 172.21.15.96:0/936285289 shutdown_connections 2024-04-25T14:04:01.137 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.268+0000 7f4bc7bc5700 1 --2- 172.21.15.96:0/936285289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4bc01099a0 0x7f4bc0109db0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:01.137 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.268+0000 7f4bc7bc5700 1 -- 172.21.15.96:0/936285289 >> 172.21.15.96:0/936285289 conn(0x7f4bc0104fc0 msgr2=0x7f4bc0107490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:01.137 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.268+0000 7f4bc7bc5700 1 -- 172.21.15.96:0/936285289 shutdown_connections 2024-04-25T14:04:01.137 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.268+0000 7f4bc7bc5700 1 -- 172.21.15.96:0/936285289 wait complete. 2024-04-25T14:04:01.137 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.269+0000 7f4bc7bc5700 1 Processor -- start 2024-04-25T14:04:01.137 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.269+0000 7f4bc7bc5700 1 -- start start 2024-04-25T14:04:01.137 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.269+0000 7f4bc7bc5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4bc01099a0 0x7f4bc019dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:01.137 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.269+0000 7f4bc7bc5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc010df20 con 0x7f4bc01099a0 2024-04-25T14:04:01.137 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.269+0000 7f4bc5961700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4bc01099a0 0x7f4bc019dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:01.137 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.269+0000 7f4bc5961700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4bc01099a0 0x7f4bc019dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57842/0 (socket says 172.21.15.96:57842) 2024-04-25T14:04:01.137 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.269+0000 7f4bc5961700 1 -- 172.21.15.96:0/1992183787 learned_addr learned my addr 172.21.15.96:0/1992183787 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:04:01.138 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.269+0000 7f4bc5961700 1 -- 172.21.15.96:0/1992183787 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bc019e2b0 con 0x7f4bc01099a0 2024-04-25T14:04:01.138 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.269+0000 7f4bc5961700 1 --2- 172.21.15.96:0/1992183787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4bc01099a0 0x7f4bc019dd70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc009fb0 tx=0x7f4bbc011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:01.138 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.269+0000 7f4bb6ffd700 1 -- 172.21.15.96:0/1992183787 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4bbc0036a0 con 0x7f4bc01099a0 2024-04-25T14:04:01.138 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.269+0000 7f4bb6ffd700 1 -- 172.21.15.96:0/1992183787 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4bbc003cd0 con 0x7f4bc01099a0 2024-04-25T14:04:01.138 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.269+0000 7f4bc7bc5700 1 -- 172.21.15.96:0/1992183787 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bc019e540 con 0x7f4bc01099a0 2024-04-25T14:04:01.138 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.269+0000 7f4bb6ffd700 1 -- 172.21.15.96:0/1992183787 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4bbc01a430 con 0x7f4bc01099a0 2024-04-25T14:04:01.138 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.269+0000 7f4bc7bc5700 1 -- 172.21.15.96:0/1992183787 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bc01a1150 con 0x7f4bc01099a0 2024-04-25T14:04:01.138 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.270+0000 7f4bb6ffd700 1 -- 172.21.15.96:0/1992183787 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f4bbc01a6f0 con 0x7f4bc01099a0 2024-04-25T14:04:01.138 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.271+0000 7f4bb6ffd700 1 --2- 172.21.15.96:0/1992183787 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f4bac03d5e0 0x7f4bac03faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:01.138 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.271+0000 7f4bb6ffd700 1 -- 172.21.15.96:0/1992183787 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4bbc0534b0 con 0x7f4bc01099a0 2024-04-25T14:04:01.138 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.271+0000 7f4bc5160700 1 --2- 172.21.15.96:0/1992183787 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f4bac03d5e0 0x7f4bac03faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:01.138 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.271+0000 7f4bc7bc5700 1 -- 172.21.15.96:0/1992183787 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bc0064cf0 con 0x7f4bc01099a0 2024-04-25T14:04:01.138 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.271+0000 7f4bc5160700 1 --2- 172.21.15.96:0/1992183787 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f4bac03d5e0 0x7f4bac03faf0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0006fd0 tx=0x7f4bb0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:01.139 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.276+0000 7f4bb6ffd700 1 -- 172.21.15.96:0/1992183787 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4bbc017e90 con 0x7f4bc01099a0 2024-04-25T14:04:01.139 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.478+0000 7f4bc7bc5700 1 -- 172.21.15.96:0/1992183787 --> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f4bc0105f10 con 0x7f4bac03d5e0 2024-04-25T14:04:01.139 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.479+0000 7f4bb6ffd700 1 -- 172.21.15.96:0/1992183787 <== mgr.14120 v2:172.21.15.96:6800/433591062 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f4bc0105f10 con 0x7f4bac03d5e0 2024-04-25T14:04:01.139 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.481+0000 7f4bc7bc5700 1 -- 172.21.15.96:0/1992183787 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f4bac03d5e0 msgr2=0x7f4bac03faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:01.139 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.481+0000 7f4bc7bc5700 1 --2- 172.21.15.96:0/1992183787 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f4bac03d5e0 0x7f4bac03faf0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0006fd0 tx=0x7f4bb0006e40 comp rx=0 tx=0).stop 2024-04-25T14:04:01.139 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.481+0000 7f4bc7bc5700 1 -- 172.21.15.96:0/1992183787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4bc01099a0 msgr2=0x7f4bc019dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:01.139 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.481+0000 7f4bc7bc5700 1 --2- 172.21.15.96:0/1992183787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4bc01099a0 0x7f4bc019dd70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc009fb0 tx=0x7f4bbc011890 comp rx=0 tx=0).stop 2024-04-25T14:04:01.139 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.481+0000 7f4bc7bc5700 1 -- 172.21.15.96:0/1992183787 shutdown_connections 2024-04-25T14:04:01.139 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.481+0000 7f4bc7bc5700 1 --2- 172.21.15.96:0/1992183787 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f4bac03d5e0 0x7f4bac03faf0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:01.139 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.481+0000 7f4bc7bc5700 1 --2- 172.21.15.96:0/1992183787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4bc01099a0 0x7f4bc019dd70 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:01.139 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.481+0000 7f4bc7bc5700 1 -- 172.21.15.96:0/1992183787 >> 172.21.15.96:0/1992183787 conn(0x7f4bc0104fc0 msgr2=0x7f4bc0105800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:01.139 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.482+0000 7f4bc7bc5700 1 -- 172.21.15.96:0/1992183787 shutdown_connections 2024-04-25T14:04:01.139 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:00.482+0000 7f4bc7bc5700 1 -- 172.21.15.96:0/1992183787 wait complete. 2024-04-25T14:04:01.139 INFO:teuthology.orchestra.run.smithi096.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-25T14:04:01.140 INFO:teuthology.orchestra.run.smithi096.stdout:Adding key to root@localhost authorized_keys... 2024-04-25T14:04:01.140 INFO:teuthology.orchestra.run.smithi096.stdout:Adding host smithi096... 2024-04-25T14:04:01.538 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:01 smithi096 ceph-mon[29207]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:04:02.681 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:02 smithi096 ceph-mon[29207]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi096", "addr": "172.21.15.96", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:04:02.681 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:02 smithi096 ceph-mon[29207]: Deploying cephadm binary to smithi096 2024-04-25T14:04:06.585 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout Added host 'smithi096' with addr '172.21.15.96' 2024-04-25T14:04:06.585 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:01.590+0000 7f919e94e700 1 Processor -- start 2024-04-25T14:04:06.585 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:01.590+0000 7f919e94e700 1 -- start start 2024-04-25T14:04:06.585 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:01.590+0000 7f919e94e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9198109990 0x7f9198109da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.585 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:01.590+0000 7f919e94e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f919810a390 con 0x7f9198109990 2024-04-25T14:04:06.585 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:01.590+0000 7f9197fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9198109990 0x7f9198109da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.585 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:01.590+0000 7f9197fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9198109990 0x7f9198109da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57220/0 (socket says 172.21.15.96:57220) 2024-04-25T14:04:06.585 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:01.590+0000 7f9197fff700 1 -- 172.21.15.96:0/2646943025 learned_addr learned my addr 172.21.15.96:0/2646943025 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:04:06.585 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:01.591+0000 7f9197fff700 1 -- 172.21.15.96:0/2646943025 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f919810ab60 con 0x7f9198109990 2024-04-25T14:04:06.585 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:01.591+0000 7f9197fff700 1 --2- 172.21.15.96:0/2646943025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9198109990 0x7f9198109da0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f91880098a0 tx=0x7f9188009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc8d6ae35e972574 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:06.585 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:01.591+0000 7f9196ffd700 1 -- 172.21.15.96:0/2646943025 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9188004160 con 0x7f9198109990 2024-04-25T14:04:06.586 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:01.591+0000 7f9196ffd700 1 -- 172.21.15.96:0/2646943025 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9188004300 con 0x7f9198109990 2024-04-25T14:04:06.586 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:01.591+0000 7f9196ffd700 1 -- 172.21.15.96:0/2646943025 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9188011af0 con 0x7f9198109990 2024-04-25T14:04:06.586 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:01.592+0000 7f919e94e700 1 -- 172.21.15.96:0/2646943025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9198109990 msgr2=0x7f9198109da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:06.586 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:01.592+0000 7f919e94e700 1 --2- 172.21.15.96:0/2646943025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9198109990 0x7f9198109da0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f91880098a0 tx=0x7f9188009bb0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.586 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:01.592+0000 7f919e94e700 1 -- 172.21.15.96:0/2646943025 shutdown_connections 2024-04-25T14:04:06.586 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:01.592+0000 7f919e94e700 1 --2- 172.21.15.96:0/2646943025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9198109990 0x7f9198109da0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.586 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:01.592+0000 7f919e94e700 1 -- 172.21.15.96:0/2646943025 >> 172.21.15.96:0/2646943025 conn(0x7f9198105050 msgr2=0x7f91981074a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:06.586 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:01.592+0000 7f919e94e700 1 -- 172.21.15.96:0/2646943025 shutdown_connections 2024-04-25T14:04:06.586 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:01.592+0000 7f919e94e700 1 -- 172.21.15.96:0/2646943025 wait complete. 2024-04-25T14:04:06.586 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:01.593+0000 7f919e94e700 1 Processor -- start 2024-04-25T14:04:06.586 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:01.593+0000 7f919e94e700 1 -- start start 2024-04-25T14:04:06.586 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:01.593+0000 7f919e94e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f919819dd70 0x7f919819e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.586 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:01.593+0000 7f919e94e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f919810df10 con 0x7f919819dd70 2024-04-25T14:04:06.586 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:01.593+0000 7f9197fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f919819dd70 0x7f919819e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.587 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:01.593+0000 7f9197fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f919819dd70 0x7f919819e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57234/0 (socket says 172.21.15.96:57234) 2024-04-25T14:04:06.587 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:01.593+0000 7f9197fff700 1 -- 172.21.15.96:0/2463884735 learned_addr learned my addr 172.21.15.96:0/2463884735 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:04:06.587 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:01.593+0000 7f9197fff700 1 -- 172.21.15.96:0/2463884735 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f919819e720 con 0x7f919819dd70 2024-04-25T14:04:06.587 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:01.594+0000 7f9197fff700 1 --2- 172.21.15.96:0/2463884735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f919819dd70 0x7f919819e1a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f91880117b0 tx=0x7f9188012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:06.587 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:01.594+0000 7f91957fa700 1 -- 172.21.15.96:0/2463884735 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9188012eb0 con 0x7f919819dd70 2024-04-25T14:04:06.587 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:01.594+0000 7f91957fa700 1 -- 172.21.15.96:0/2463884735 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9188003b30 con 0x7f919819dd70 2024-04-25T14:04:06.587 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:01.594+0000 7f919e94e700 1 -- 172.21.15.96:0/2463884735 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f919819e9b0 con 0x7f919819dd70 2024-04-25T14:04:06.587 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:01.594+0000 7f91957fa700 1 -- 172.21.15.96:0/2463884735 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f918801ad80 con 0x7f919819dd70 2024-04-25T14:04:06.587 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:01.594+0000 7f919e94e700 1 -- 172.21.15.96:0/2463884735 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91981a14e0 con 0x7f919819dd70 2024-04-25T14:04:06.587 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:01.595+0000 7f91957fa700 1 -- 172.21.15.96:0/2463884735 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f918802d440 con 0x7f919819dd70 2024-04-25T14:04:06.587 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:01.595+0000 7f919e94e700 1 -- 172.21.15.96:0/2463884735 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9198064cf0 con 0x7f919819dd70 2024-04-25T14:04:06.587 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:01.595+0000 7f91957fa700 1 --2- 172.21.15.96:0/2463884735 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f918003d630 0x7f918003fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.587 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:01.595+0000 7f91957fa700 1 -- 172.21.15.96:0/2463884735 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9188052520 con 0x7f919819dd70 2024-04-25T14:04:06.588 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:01.595+0000 7f91977fe700 1 --2- 172.21.15.96:0/2463884735 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f918003d630 0x7f918003fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.588 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:01.595+0000 7f91977fe700 1 --2- 172.21.15.96:0/2463884735 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f918003d630 0x7f918003fb40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f918c006fd0 tx=0x7f918c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:06.588 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:01.600+0000 7f91957fa700 1 -- 172.21.15.96:0/2463884735 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9188052280 con 0x7f919819dd70 2024-04-25T14:04:06.588 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:01.809+0000 7f919e94e700 1 -- 172.21.15.96:0/2463884735 --> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi096", "addr": "172.21.15.96", "target": ["mon-mgr", ""]}) -- 0x7f9198107040 con 0x7f918003d630 2024-04-25T14:04:06.588 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:05.947+0000 7f91957fa700 1 -- 172.21.15.96:0/2463884735 <== mgr.14120 v2:172.21.15.96:6800/433591062 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f9198107040 con 0x7f918003d630 2024-04-25T14:04:06.588 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:05.949+0000 7f919e94e700 1 -- 172.21.15.96:0/2463884735 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f918003d630 msgr2=0x7f918003fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:06.588 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:05.949+0000 7f919e94e700 1 --2- 172.21.15.96:0/2463884735 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f918003d630 0x7f918003fb40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f918c006fd0 tx=0x7f918c006e40 comp rx=0 tx=0).stop 2024-04-25T14:04:06.588 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:05.950+0000 7f919e94e700 1 -- 172.21.15.96:0/2463884735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f919819dd70 msgr2=0x7f919819e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:06.588 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:05.950+0000 7f919e94e700 1 --2- 172.21.15.96:0/2463884735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f919819dd70 0x7f919819e1a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f91880117b0 tx=0x7f9188012b60 comp rx=0 tx=0).stop 2024-04-25T14:04:06.588 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:05.950+0000 7f919e94e700 1 -- 172.21.15.96:0/2463884735 shutdown_connections 2024-04-25T14:04:06.588 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:05.950+0000 7f919e94e700 1 --2- 172.21.15.96:0/2463884735 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f918003d630 0x7f918003fb40 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.588 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:05.950+0000 7f919e94e700 1 --2- 172.21.15.96:0/2463884735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f919819dd70 0x7f919819e1a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.588 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:05.950+0000 7f919e94e700 1 -- 172.21.15.96:0/2463884735 >> 172.21.15.96:0/2463884735 conn(0x7f9198105050 msgr2=0x7f9198106930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:06.588 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:05.950+0000 7f919e94e700 1 -- 172.21.15.96:0/2463884735 shutdown_connections 2024-04-25T14:04:06.589 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:05.950+0000 7f919e94e700 1 -- 172.21.15.96:0/2463884735 wait complete. 2024-04-25T14:04:06.589 INFO:teuthology.orchestra.run.smithi096.stdout:Deploying unmanaged mon service... 2024-04-25T14:04:07.234 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:06 smithi096 ceph-mon[29207]: from='mgr.14120 172.21.15.96:0/4269437437' entity='mgr.x' 2024-04-25T14:04:07.234 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:06 smithi096 ceph-mon[29207]: Added host smithi096 2024-04-25T14:04:07.234 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:06 smithi096 ceph-mon[29207]: from='mgr.14120 172.21.15.96:0/4269437437' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:04:08.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:08 smithi096 ceph-mon[29207]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:04:08.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:08 smithi096 ceph-mon[29207]: Saving service mon spec with placement count:5 2024-04-25T14:04:08.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:08 smithi096 ceph-mon[29207]: from='mgr.14120 172.21.15.96:0/4269437437' entity='mgr.x' 2024-04-25T14:04:08.901 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-25T14:04:08.901 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.100+0000 7fa3c4a1d700 1 Processor -- start 2024-04-25T14:04:08.901 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.100+0000 7fa3c4a1d700 1 -- start start 2024-04-25T14:04:08.901 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.100+0000 7fa3c4a1d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3c010a4e0 0x7fa3c010c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:08.902 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.100+0000 7fa3c4a1d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c007d610 con 0x7fa3c010a4e0 2024-04-25T14:04:08.902 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.100+0000 7fa3be59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3c010a4e0 0x7fa3c010c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:08.902 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.100+0000 7fa3be59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3c010a4e0 0x7fa3c010c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57242/0 (socket says 172.21.15.96:57242) 2024-04-25T14:04:08.902 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.100+0000 7fa3be59c700 1 -- 172.21.15.96:0/339785364 learned_addr learned my addr 172.21.15.96:0/339785364 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:04:08.902 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.101+0000 7fa3be59c700 1 -- 172.21.15.96:0/339785364 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c010ce40 con 0x7fa3c010a4e0 2024-04-25T14:04:08.902 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.101+0000 7fa3be59c700 1 --2- 172.21.15.96:0/339785364 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3c010a4e0 0x7fa3c010c900 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa3b00098a0 tx=0x7fa3b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=972c4bf15daceeeb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:08.902 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.101+0000 7fa3bd59a700 1 -- 172.21.15.96:0/339785364 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3b0004160 con 0x7fa3c010a4e0 2024-04-25T14:04:08.902 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.101+0000 7fa3bd59a700 1 -- 172.21.15.96:0/339785364 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa3b0004300 con 0x7fa3c010a4e0 2024-04-25T14:04:08.902 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.101+0000 7fa3bd59a700 1 -- 172.21.15.96:0/339785364 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3b0011af0 con 0x7fa3c010a4e0 2024-04-25T14:04:08.902 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.102+0000 7fa3c4a1d700 1 -- 172.21.15.96:0/339785364 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3c010a4e0 msgr2=0x7fa3c010c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:08.902 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.102+0000 7fa3c4a1d700 1 --2- 172.21.15.96:0/339785364 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3c010a4e0 0x7fa3c010c900 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa3b00098a0 tx=0x7fa3b0009bb0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.902 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.102+0000 7fa3c4a1d700 1 -- 172.21.15.96:0/339785364 shutdown_connections 2024-04-25T14:04:08.903 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.102+0000 7fa3c4a1d700 1 --2- 172.21.15.96:0/339785364 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3c010a4e0 0x7fa3c010c900 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.903 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.102+0000 7fa3c4a1d700 1 -- 172.21.15.96:0/339785364 >> 172.21.15.96:0/339785364 conn(0x7fa3c0103f40 msgr2=0x7fa3c0106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:08.903 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.102+0000 7fa3c4a1d700 1 -- 172.21.15.96:0/339785364 shutdown_connections 2024-04-25T14:04:08.903 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.102+0000 7fa3c4a1d700 1 -- 172.21.15.96:0/339785364 wait complete. 2024-04-25T14:04:08.903 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.103+0000 7fa3c4a1d700 1 Processor -- start 2024-04-25T14:04:08.903 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.103+0000 7fa3c4a1d700 1 -- start start 2024-04-25T14:04:08.903 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.103+0000 7fa3c4a1d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3c010a4e0 0x7fa3c019dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:08.903 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.103+0000 7fa3c4a1d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c010dda0 con 0x7fa3c010a4e0 2024-04-25T14:04:08.903 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.103+0000 7fa3be59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3c010a4e0 0x7fa3c019dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:08.903 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.103+0000 7fa3be59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3c010a4e0 0x7fa3c019dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57248/0 (socket says 172.21.15.96:57248) 2024-04-25T14:04:08.903 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.103+0000 7fa3be59c700 1 -- 172.21.15.96:0/1903365061 learned_addr learned my addr 172.21.15.96:0/1903365061 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:04:08.903 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.103+0000 7fa3be59c700 1 -- 172.21.15.96:0/1903365061 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c019e1e0 con 0x7fa3c010a4e0 2024-04-25T14:04:08.903 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.103+0000 7fa3be59c700 1 --2- 172.21.15.96:0/1903365061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3c010a4e0 0x7fa3c019dca0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0009fb0 tx=0x7fa3b0011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:08.903 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.103+0000 7fa3af7fe700 1 -- 172.21.15.96:0/1903365061 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3b00036a0 con 0x7fa3c010a4e0 2024-04-25T14:04:08.904 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.103+0000 7fa3af7fe700 1 -- 172.21.15.96:0/1903365061 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa3b001a430 con 0x7fa3c010a4e0 2024-04-25T14:04:08.904 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.104+0000 7fa3c4a1d700 1 -- 172.21.15.96:0/1903365061 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3c019e470 con 0x7fa3c010a4e0 2024-04-25T14:04:08.904 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.104+0000 7fa3af7fe700 1 -- 172.21.15.96:0/1903365061 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3b0019550 con 0x7fa3c010a4e0 2024-04-25T14:04:08.904 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.104+0000 7fa3c4a1d700 1 -- 172.21.15.96:0/1903365061 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3c01a1080 con 0x7fa3c010a4e0 2024-04-25T14:04:08.904 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.105+0000 7fa3af7fe700 1 -- 172.21.15.96:0/1903365061 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7fa3b001a8e0 con 0x7fa3c010a4e0 2024-04-25T14:04:08.904 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.105+0000 7fa3c4a1d700 1 -- 172.21.15.96:0/1903365061 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3c0064cf0 con 0x7fa3c010a4e0 2024-04-25T14:04:08.904 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.105+0000 7fa3af7fe700 1 --2- 172.21.15.96:0/1903365061 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7fa3a803d5e0 0x7fa3a803faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:08.904 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.105+0000 7fa3af7fe700 1 -- 172.21.15.96:0/1903365061 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa3b00526a0 con 0x7fa3c010a4e0 2024-04-25T14:04:08.904 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.105+0000 7fa3bdd9b700 1 --2- 172.21.15.96:0/1903365061 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7fa3a803d5e0 0x7fa3a803faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:08.904 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.105+0000 7fa3bdd9b700 1 --2- 172.21.15.96:0/1903365061 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7fa3a803d5e0 0x7fa3a803faf0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4006fd0 tx=0x7fa3b4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:08.904 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.110+0000 7fa3af7fe700 1 -- 172.21.15.96:0/1903365061 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa3b000b3e0 con 0x7fa3c010a4e0 2024-04-25T14:04:08.904 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.336+0000 7fa3c4a1d700 1 -- 172.21.15.96:0/1903365061 --> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fa3c0105e70 con 0x7fa3a803d5e0 2024-04-25T14:04:08.904 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.341+0000 7fa3af7fe700 1 -- 172.21.15.96:0/1903365061 <== mgr.14120 v2:172.21.15.96:6800/433591062 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fa3c0105e70 con 0x7fa3a803d5e0 2024-04-25T14:04:08.904 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.343+0000 7fa3c4a1d700 1 -- 172.21.15.96:0/1903365061 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7fa3a803d5e0 msgr2=0x7fa3a803faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:08.904 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.343+0000 7fa3c4a1d700 1 --2- 172.21.15.96:0/1903365061 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7fa3a803d5e0 0x7fa3a803faf0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4006fd0 tx=0x7fa3b4006e40 comp rx=0 tx=0).stop 2024-04-25T14:04:08.905 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.343+0000 7fa3c4a1d700 1 -- 172.21.15.96:0/1903365061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3c010a4e0 msgr2=0x7fa3c019dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:08.905 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.343+0000 7fa3c4a1d700 1 --2- 172.21.15.96:0/1903365061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3c010a4e0 0x7fa3c019dca0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0009fb0 tx=0x7fa3b0011940 comp rx=0 tx=0).stop 2024-04-25T14:04:08.905 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.343+0000 7fa3c4a1d700 1 -- 172.21.15.96:0/1903365061 shutdown_connections 2024-04-25T14:04:08.905 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.343+0000 7fa3c4a1d700 1 --2- 172.21.15.96:0/1903365061 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7fa3a803d5e0 0x7fa3a803faf0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.905 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.343+0000 7fa3c4a1d700 1 --2- 172.21.15.96:0/1903365061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3c010a4e0 0x7fa3c019dca0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.905 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.343+0000 7fa3c4a1d700 1 -- 172.21.15.96:0/1903365061 >> 172.21.15.96:0/1903365061 conn(0x7fa3c0103f40 msgr2=0x7fa3c0105760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:08.905 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.343+0000 7fa3c4a1d700 1 -- 172.21.15.96:0/1903365061 shutdown_connections 2024-04-25T14:04:08.905 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:07.343+0000 7fa3c4a1d700 1 -- 172.21.15.96:0/1903365061 wait complete. 2024-04-25T14:04:08.905 INFO:teuthology.orchestra.run.smithi096.stdout:Deploying unmanaged mgr service... 2024-04-25T14:04:09.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:09 smithi096 ceph-mon[29207]: from='mgr.14120 172.21.15.96:0/4269437437' entity='mgr.x' 2024-04-25T14:04:09.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:09 smithi096 ceph-mon[29207]: from='mgr.14120 172.21.15.96:0/4269437437' entity='mgr.x' 2024-04-25T14:04:10.208 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-25T14:04:10.208 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.351+0000 7fa699a5b700 1 Processor -- start 2024-04-25T14:04:10.208 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.351+0000 7fa699a5b700 1 -- start start 2024-04-25T14:04:10.208 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.352+0000 7fa699a5b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa694108880 0x7fa694108c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:10.208 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.352+0000 7fa699a5b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa694109280 con 0x7fa694108880 2024-04-25T14:04:10.208 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.352+0000 7fa692ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa694108880 0x7fa694108c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:10.208 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.352+0000 7fa692ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa694108880 0x7fa694108c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57254/0 (socket says 172.21.15.96:57254) 2024-04-25T14:04:10.208 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.352+0000 7fa692ffd700 1 -- 172.21.15.96:0/3310992010 learned_addr learned my addr 172.21.15.96:0/3310992010 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:04:10.208 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.352+0000 7fa692ffd700 1 -- 172.21.15.96:0/3310992010 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa694109a50 con 0x7fa694108880 2024-04-25T14:04:10.209 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.352+0000 7fa692ffd700 1 --2- 172.21.15.96:0/3310992010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa694108880 0x7fa694108c90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa6880098a0 tx=0x7fa688009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6269069ab0bea559 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:10.209 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.353+0000 7fa691ffb700 1 -- 172.21.15.96:0/3310992010 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa688004160 con 0x7fa694108880 2024-04-25T14:04:10.209 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.353+0000 7fa691ffb700 1 -- 172.21.15.96:0/3310992010 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa688004300 con 0x7fa694108880 2024-04-25T14:04:10.209 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.353+0000 7fa691ffb700 1 -- 172.21.15.96:0/3310992010 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa688011a20 con 0x7fa694108880 2024-04-25T14:04:10.209 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.354+0000 7fa699a5b700 1 -- 172.21.15.96:0/3310992010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa694108880 msgr2=0x7fa694108c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:10.209 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.354+0000 7fa699a5b700 1 --2- 172.21.15.96:0/3310992010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa694108880 0x7fa694108c90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa6880098a0 tx=0x7fa688009bb0 comp rx=0 tx=0).stop 2024-04-25T14:04:10.209 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.354+0000 7fa699a5b700 1 -- 172.21.15.96:0/3310992010 shutdown_connections 2024-04-25T14:04:10.209 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.354+0000 7fa699a5b700 1 --2- 172.21.15.96:0/3310992010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa694108880 0x7fa694108c90 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:10.209 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.354+0000 7fa699a5b700 1 -- 172.21.15.96:0/3310992010 >> 172.21.15.96:0/3310992010 conn(0x7fa694103f20 msgr2=0x7fa694106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:10.209 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.354+0000 7fa699a5b700 1 -- 172.21.15.96:0/3310992010 shutdown_connections 2024-04-25T14:04:10.209 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.354+0000 7fa699a5b700 1 -- 172.21.15.96:0/3310992010 wait complete. 2024-04-25T14:04:10.209 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.355+0000 7fa699a5b700 1 Processor -- start 2024-04-25T14:04:10.209 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.355+0000 7fa699a5b700 1 -- start start 2024-04-25T14:04:10.210 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.355+0000 7fa699a5b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa694108880 0x7fa6941939e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:10.210 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.355+0000 7fa699a5b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa69410ce00 con 0x7fa694108880 2024-04-25T14:04:10.210 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.355+0000 7fa692ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa694108880 0x7fa6941939e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:10.210 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.355+0000 7fa692ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa694108880 0x7fa6941939e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57262/0 (socket says 172.21.15.96:57262) 2024-04-25T14:04:10.210 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.355+0000 7fa692ffd700 1 -- 172.21.15.96:0/358953709 learned_addr learned my addr 172.21.15.96:0/358953709 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:04:10.210 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.355+0000 7fa692ffd700 1 -- 172.21.15.96:0/358953709 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa694193f20 con 0x7fa694108880 2024-04-25T14:04:10.210 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.356+0000 7fa692ffd700 1 --2- 172.21.15.96:0/358953709 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa694108880 0x7fa6941939e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa688003e70 tx=0x7fa688011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:10.210 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.356+0000 7fa698a59700 1 -- 172.21.15.96:0/358953709 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa688003aa0 con 0x7fa694108880 2024-04-25T14:04:10.210 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.356+0000 7fa699a5b700 1 -- 172.21.15.96:0/358953709 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa694192170 con 0x7fa694108880 2024-04-25T14:04:10.210 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.356+0000 7fa698a59700 1 -- 172.21.15.96:0/358953709 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa68801a8e0 con 0x7fa694108880 2024-04-25T14:04:10.210 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.356+0000 7fa698a59700 1 -- 172.21.15.96:0/358953709 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa688024870 con 0x7fa694108880 2024-04-25T14:04:10.210 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.356+0000 7fa699a5b700 1 -- 172.21.15.96:0/358953709 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa694192650 con 0x7fa694108880 2024-04-25T14:04:10.210 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.357+0000 7fa698a59700 1 -- 172.21.15.96:0/358953709 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7fa688022020 con 0x7fa694108880 2024-04-25T14:04:10.211 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.357+0000 7fa698a59700 1 --2- 172.21.15.96:0/358953709 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7fa67c03d630 0x7fa67c03fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:10.211 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.357+0000 7fa699a5b700 1 -- 172.21.15.96:0/358953709 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa694064cf0 con 0x7fa694108880 2024-04-25T14:04:10.211 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.357+0000 7fa698a59700 1 -- 172.21.15.96:0/358953709 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa688053600 con 0x7fa694108880 2024-04-25T14:04:10.211 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.357+0000 7fa6927fc700 1 --2- 172.21.15.96:0/358953709 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7fa67c03d630 0x7fa67c03fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:10.211 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.358+0000 7fa6927fc700 1 --2- 172.21.15.96:0/358953709 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7fa67c03d630 0x7fa67c03fb40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa684006fd0 tx=0x7fa684006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:10.211 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.367+0000 7fa698a59700 1 -- 172.21.15.96:0/358953709 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa68800e260 con 0x7fa694108880 2024-04-25T14:04:10.211 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.602+0000 7fa699a5b700 1 -- 172.21.15.96:0/358953709 --> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fa694192d70 con 0x7fa67c03d630 2024-04-25T14:04:10.211 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.608+0000 7fa698a59700 1 -- 172.21.15.96:0/358953709 <== mgr.14120 v2:172.21.15.96:6800/433591062 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fa694192d70 con 0x7fa67c03d630 2024-04-25T14:04:10.211 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.611+0000 7fa699a5b700 1 -- 172.21.15.96:0/358953709 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7fa67c03d630 msgr2=0x7fa67c03fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:10.211 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.611+0000 7fa699a5b700 1 --2- 172.21.15.96:0/358953709 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7fa67c03d630 0x7fa67c03fb40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa684006fd0 tx=0x7fa684006e40 comp rx=0 tx=0).stop 2024-04-25T14:04:10.211 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.611+0000 7fa699a5b700 1 -- 172.21.15.96:0/358953709 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa694108880 msgr2=0x7fa6941939e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:10.211 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.611+0000 7fa699a5b700 1 --2- 172.21.15.96:0/358953709 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa694108880 0x7fa6941939e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa688003e70 tx=0x7fa688011aa0 comp rx=0 tx=0).stop 2024-04-25T14:04:10.211 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.611+0000 7fa699a5b700 1 -- 172.21.15.96:0/358953709 shutdown_connections 2024-04-25T14:04:10.211 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.611+0000 7fa699a5b700 1 --2- 172.21.15.96:0/358953709 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7fa67c03d630 0x7fa67c03fb40 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:10.212 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.611+0000 7fa699a5b700 1 --2- 172.21.15.96:0/358953709 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa694108880 0x7fa6941939e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:10.212 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.611+0000 7fa699a5b700 1 -- 172.21.15.96:0/358953709 >> 172.21.15.96:0/358953709 conn(0x7fa694103f20 msgr2=0x7fa694104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:10.212 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.611+0000 7fa699a5b700 1 -- 172.21.15.96:0/358953709 shutdown_connections 2024-04-25T14:04:10.212 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:09.611+0000 7fa699a5b700 1 -- 172.21.15.96:0/358953709 wait complete. 2024-04-25T14:04:10.796 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:10 smithi096 ceph-mon[29207]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:04:10.796 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:10 smithi096 ceph-mon[29207]: Saving service mgr spec with placement count:2 2024-04-25T14:04:10.796 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:10 smithi096 ceph-mon[29207]: from='mgr.14120 172.21.15.96:0/4269437437' entity='mgr.x' 2024-04-25T14:04:12.151 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.330+0000 7f252a7d1700 1 Processor -- start 2024-04-25T14:04:12.151 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.330+0000 7f252a7d1700 1 -- start start 2024-04-25T14:04:12.151 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.330+0000 7f252a7d1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f252410a460 0x7f252410c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.151 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.330+0000 7f252a7d1700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f252407ce00 con 0x7f252410a460 2024-04-25T14:04:12.151 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.330+0000 7f2523fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f252410a460 0x7f252410c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.151 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.331+0000 7f2523fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f252410a460 0x7f252410c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57278/0 (socket says 172.21.15.96:57278) 2024-04-25T14:04:12.151 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.331+0000 7f2523fff700 1 -- 172.21.15.96:0/1861551939 learned_addr learned my addr 172.21.15.96:0/1861551939 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:04:12.151 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.331+0000 7f2523fff700 1 -- 172.21.15.96:0/1861551939 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f252410cdc0 con 0x7f252410a460 2024-04-25T14:04:12.151 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.331+0000 7f2523fff700 1 --2- 172.21.15.96:0/1861551939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f252410a460 0x7f252410c880 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f25180098a0 tx=0x7f2518009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b26c4974fcfcecd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:12.151 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.331+0000 7f2522ffd700 1 -- 172.21.15.96:0/1861551939 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2518004160 con 0x7f252410a460 2024-04-25T14:04:12.151 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.331+0000 7f2522ffd700 1 -- 172.21.15.96:0/1861551939 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2518004300 con 0x7f252410a460 2024-04-25T14:04:12.152 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.331+0000 7f2522ffd700 1 -- 172.21.15.96:0/1861551939 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2518011a20 con 0x7f252410a460 2024-04-25T14:04:12.152 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.332+0000 7f252a7d1700 1 -- 172.21.15.96:0/1861551939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f252410a460 msgr2=0x7f252410c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.152 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.332+0000 7f252a7d1700 1 --2- 172.21.15.96:0/1861551939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f252410a460 0x7f252410c880 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f25180098a0 tx=0x7f2518009bb0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.152 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.332+0000 7f252a7d1700 1 -- 172.21.15.96:0/1861551939 shutdown_connections 2024-04-25T14:04:12.152 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.332+0000 7f252a7d1700 1 --2- 172.21.15.96:0/1861551939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f252410a460 0x7f252410c880 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.152 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.332+0000 7f252a7d1700 1 -- 172.21.15.96:0/1861551939 >> 172.21.15.96:0/1861551939 conn(0x7f2524103f20 msgr2=0x7f2524106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:12.152 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.332+0000 7f252a7d1700 1 -- 172.21.15.96:0/1861551939 shutdown_connections 2024-04-25T14:04:12.152 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.332+0000 7f252a7d1700 1 -- 172.21.15.96:0/1861551939 wait complete. 2024-04-25T14:04:12.152 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.333+0000 7f252a7d1700 1 Processor -- start 2024-04-25T14:04:12.152 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.333+0000 7f252a7d1700 1 -- start start 2024-04-25T14:04:12.152 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.333+0000 7f252a7d1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f252410a460 0x7f252419fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.152 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.333+0000 7f252a7d1700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f252410ddc0 con 0x7f252410a460 2024-04-25T14:04:12.152 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.333+0000 7f2523fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f252410a460 0x7f252419fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.152 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.333+0000 7f2523fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f252410a460 0x7f252419fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57290/0 (socket says 172.21.15.96:57290) 2024-04-25T14:04:12.152 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.333+0000 7f2523fff700 1 -- 172.21.15.96:0/251796558 learned_addr learned my addr 172.21.15.96:0/251796558 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:04:12.153 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.333+0000 7f2523fff700 1 -- 172.21.15.96:0/251796558 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25241a0420 con 0x7f252410a460 2024-04-25T14:04:12.153 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.334+0000 7f2523fff700 1 --2- 172.21.15.96:0/251796558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f252410a460 0x7f252419fee0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f25180119f0 tx=0x7f2518011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:12.153 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.334+0000 7f25217fa700 1 -- 172.21.15.96:0/251796558 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25180039e0 con 0x7f252410a460 2024-04-25T14:04:12.153 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.334+0000 7f25217fa700 1 -- 172.21.15.96:0/251796558 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f251801b8f0 con 0x7f252410a460 2024-04-25T14:04:12.153 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.334+0000 7f252a7d1700 1 -- 172.21.15.96:0/251796558 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25241a06b0 con 0x7f252410a460 2024-04-25T14:04:12.153 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.334+0000 7f252a7d1700 1 -- 172.21.15.96:0/251796558 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25241a32c0 con 0x7f252410a460 2024-04-25T14:04:12.153 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.334+0000 7f25217fa700 1 -- 172.21.15.96:0/251796558 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25180237a0 con 0x7f252410a460 2024-04-25T14:04:12.153 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.335+0000 7f25217fa700 1 -- 172.21.15.96:0/251796558 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f251802c3f0 con 0x7f252410a460 2024-04-25T14:04:12.153 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.335+0000 7f252a7d1700 1 -- 172.21.15.96:0/251796558 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2524064cf0 con 0x7f252410a460 2024-04-25T14:04:12.153 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.335+0000 7f25217fa700 1 --2- 172.21.15.96:0/251796558 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f250c03d5e0 0x7f250c03faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.153 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.335+0000 7f25217fa700 1 -- 172.21.15.96:0/251796558 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2518052c50 con 0x7f252410a460 2024-04-25T14:04:12.153 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.335+0000 7f25237fe700 1 --2- 172.21.15.96:0/251796558 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f250c03d5e0 0x7f250c03faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.153 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.336+0000 7f25237fe700 1 --2- 172.21.15.96:0/251796558 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f250c03d5e0 0x7f250c03faf0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2514006fd0 tx=0x7f2514006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:12.153 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.340+0000 7f25217fa700 1 -- 172.21.15.96:0/251796558 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2518058020 con 0x7f252410a460 2024-04-25T14:04:12.154 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.544+0000 7f252a7d1700 1 -- 172.21.15.96:0/251796558 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f252402e7e0 con 0x7f252410a460 2024-04-25T14:04:12.154 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.548+0000 7f25217fa700 1 -- 172.21.15.96:0/251796558 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f251801f030 con 0x7f252410a460 2024-04-25T14:04:12.154 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.551+0000 7f252a7d1700 1 -- 172.21.15.96:0/251796558 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f250c03d5e0 msgr2=0x7f250c03faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.154 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.551+0000 7f252a7d1700 1 --2- 172.21.15.96:0/251796558 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f250c03d5e0 0x7f250c03faf0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2514006fd0 tx=0x7f2514006e40 comp rx=0 tx=0).stop 2024-04-25T14:04:12.154 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.551+0000 7f252a7d1700 1 -- 172.21.15.96:0/251796558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f252410a460 msgr2=0x7f252419fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.154 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.551+0000 7f252a7d1700 1 --2- 172.21.15.96:0/251796558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f252410a460 0x7f252419fee0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f25180119f0 tx=0x7f2518011ad0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.154 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.551+0000 7f252a7d1700 1 -- 172.21.15.96:0/251796558 shutdown_connections 2024-04-25T14:04:12.154 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.551+0000 7f252a7d1700 1 --2- 172.21.15.96:0/251796558 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f250c03d5e0 0x7f250c03faf0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.154 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.551+0000 7f252a7d1700 1 --2- 172.21.15.96:0/251796558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f252410a460 0x7f252419fee0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.154 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.551+0000 7f252a7d1700 1 -- 172.21.15.96:0/251796558 >> 172.21.15.96:0/251796558 conn(0x7f2524103f20 msgr2=0x7f2524108420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:12.154 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.551+0000 7f252a7d1700 1 -- 172.21.15.96:0/251796558 shutdown_connections 2024-04-25T14:04:12.154 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:11.551+0000 7f252a7d1700 1 -- 172.21.15.96:0/251796558 wait complete. 2024-04-25T14:04:12.463 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:12 smithi096 ceph-mon[29207]: from='mgr.14120 172.21.15.96:0/4269437437' entity='mgr.x' 2024-04-25T14:04:12.463 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:12 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/251796558' entity='client.admin' 2024-04-25T14:04:13.416 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.582+0000 7fa2c8449700 1 Processor -- start 2024-04-25T14:04:13.416 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.582+0000 7fa2c8449700 1 -- start start 2024-04-25T14:04:13.417 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.582+0000 7fa2c8449700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2c0108850 0x7fa2c0108c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.417 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.582+0000 7fa2c8449700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c0109250 con 0x7fa2c0108850 2024-04-25T14:04:13.417 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.582+0000 7fa2c61e5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2c0108850 0x7fa2c0108c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.417 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.582+0000 7fa2c61e5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2c0108850 0x7fa2c0108c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34034/0 (socket says 172.21.15.96:34034) 2024-04-25T14:04:13.417 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.582+0000 7fa2c61e5700 1 -- 172.21.15.96:0/1515125939 learned_addr learned my addr 172.21.15.96:0/1515125939 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:04:13.417 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.583+0000 7fa2c61e5700 1 -- 172.21.15.96:0/1515125939 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c0109a20 con 0x7fa2c0108850 2024-04-25T14:04:13.417 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.583+0000 7fa2c61e5700 1 --2- 172.21.15.96:0/1515125939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2c0108850 0x7fa2c0108c60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc00bb40 tx=0x7fa2bc00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77346971c71d9379 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:13.417 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.583+0000 7fa2c51e3700 1 -- 172.21.15.96:0/1515125939 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2bc00dbe0 con 0x7fa2c0108850 2024-04-25T14:04:13.417 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.583+0000 7fa2c51e3700 1 -- 172.21.15.96:0/1515125939 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2bc0044e0 con 0x7fa2c0108850 2024-04-25T14:04:13.417 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.583+0000 7fa2c51e3700 1 -- 172.21.15.96:0/1515125939 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2bc010470 con 0x7fa2c0108850 2024-04-25T14:04:13.417 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.584+0000 7fa2c8449700 1 -- 172.21.15.96:0/1515125939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2c0108850 msgr2=0x7fa2c0108c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.417 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.584+0000 7fa2c8449700 1 --2- 172.21.15.96:0/1515125939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2c0108850 0x7fa2c0108c60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc00bb40 tx=0x7fa2bc00be50 comp rx=0 tx=0).stop 2024-04-25T14:04:13.418 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.584+0000 7fa2c8449700 1 -- 172.21.15.96:0/1515125939 shutdown_connections 2024-04-25T14:04:13.418 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.584+0000 7fa2c8449700 1 --2- 172.21.15.96:0/1515125939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2c0108850 0x7fa2c0108c60 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.418 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.584+0000 7fa2c8449700 1 -- 172.21.15.96:0/1515125939 >> 172.21.15.96:0/1515125939 conn(0x7fa2c0103ef0 msgr2=0x7fa2c0106340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:13.418 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.584+0000 7fa2c8449700 1 -- 172.21.15.96:0/1515125939 shutdown_connections 2024-04-25T14:04:13.418 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.584+0000 7fa2c8449700 1 -- 172.21.15.96:0/1515125939 wait complete. 2024-04-25T14:04:13.418 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.585+0000 7fa2c8449700 1 Processor -- start 2024-04-25T14:04:13.418 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.585+0000 7fa2c8449700 1 -- start start 2024-04-25T14:04:13.418 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.586+0000 7fa2c8449700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2c0108850 0x7fa2c019ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.418 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.586+0000 7fa2c8449700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c007c0d0 con 0x7fa2c0108850 2024-04-25T14:04:13.418 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.586+0000 7fa2c61e5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2c0108850 0x7fa2c019ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.418 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.586+0000 7fa2c61e5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2c0108850 0x7fa2c019ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34048/0 (socket says 172.21.15.96:34048) 2024-04-25T14:04:13.418 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.586+0000 7fa2c61e5700 1 -- 172.21.15.96:0/80168346 learned_addr learned my addr 172.21.15.96:0/80168346 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:04:13.418 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.586+0000 7fa2c61e5700 1 -- 172.21.15.96:0/80168346 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c019bfd0 con 0x7fa2c0108850 2024-04-25T14:04:13.419 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.586+0000 7fa2c61e5700 1 --2- 172.21.15.96:0/80168346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2c0108850 0x7fa2c019ba90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc003c30 tx=0x7fa2bc003980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:13.419 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.586+0000 7fa2b77fe700 1 -- 172.21.15.96:0/80168346 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2bc027070 con 0x7fa2c0108850 2024-04-25T14:04:13.419 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.586+0000 7fa2b77fe700 1 -- 172.21.15.96:0/80168346 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2bc0042d0 con 0x7fa2c0108850 2024-04-25T14:04:13.419 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.586+0000 7fa2c8449700 1 -- 172.21.15.96:0/80168346 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2c019c260 con 0x7fa2c0108850 2024-04-25T14:04:13.419 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.587+0000 7fa2b77fe700 1 -- 172.21.15.96:0/80168346 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2bc0229b0 con 0x7fa2c0108850 2024-04-25T14:04:13.419 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.587+0000 7fa2c8449700 1 -- 172.21.15.96:0/80168346 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2c019ee70 con 0x7fa2c0108850 2024-04-25T14:04:13.419 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.587+0000 7fa2b77fe700 1 -- 172.21.15.96:0/80168346 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7fa2bc02a070 con 0x7fa2c0108850 2024-04-25T14:04:13.419 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.588+0000 7fa2c8449700 1 -- 172.21.15.96:0/80168346 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2c0064cf0 con 0x7fa2c0108850 2024-04-25T14:04:13.419 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.588+0000 7fa2b77fe700 1 --2- 172.21.15.96:0/80168346 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7fa2ac03d630 0x7fa2ac03fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.419 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.588+0000 7fa2b77fe700 1 -- 172.21.15.96:0/80168346 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa2bc01f070 con 0x7fa2c0108850 2024-04-25T14:04:13.419 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.588+0000 7fa2c59e4700 1 --2- 172.21.15.96:0/80168346 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7fa2ac03d630 0x7fa2ac03fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.419 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.588+0000 7fa2c59e4700 1 --2- 172.21.15.96:0/80168346 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7fa2ac03d630 0x7fa2ac03fb40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0006fd0 tx=0x7fa2b0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:13.419 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.593+0000 7fa2b77fe700 1 -- 172.21.15.96:0/80168346 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa2bc015070 con 0x7fa2c0108850 2024-04-25T14:04:13.420 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.800+0000 7fa2c8449700 1 -- 172.21.15.96:0/80168346 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fa2c0194ca0 con 0x7fa2c0108850 2024-04-25T14:04:13.420 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.804+0000 7fa2b77fe700 1 -- 172.21.15.96:0/80168346 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fa2c0194ca0 con 0x7fa2c0108850 2024-04-25T14:04:13.420 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.807+0000 7fa2c8449700 1 -- 172.21.15.96:0/80168346 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7fa2ac03d630 msgr2=0x7fa2ac03fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.420 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.807+0000 7fa2c8449700 1 --2- 172.21.15.96:0/80168346 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7fa2ac03d630 0x7fa2ac03fb40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0006fd0 tx=0x7fa2b0006e40 comp rx=0 tx=0).stop 2024-04-25T14:04:13.420 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.807+0000 7fa2c8449700 1 -- 172.21.15.96:0/80168346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2c0108850 msgr2=0x7fa2c019ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.420 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.807+0000 7fa2c8449700 1 --2- 172.21.15.96:0/80168346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2c0108850 0x7fa2c019ba90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc003c30 tx=0x7fa2bc003980 comp rx=0 tx=0).stop 2024-04-25T14:04:13.420 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.807+0000 7fa2c8449700 1 -- 172.21.15.96:0/80168346 shutdown_connections 2024-04-25T14:04:13.420 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.807+0000 7fa2c8449700 1 --2- 172.21.15.96:0/80168346 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7fa2ac03d630 0x7fa2ac03fb40 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.420 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.807+0000 7fa2c8449700 1 --2- 172.21.15.96:0/80168346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2c0108850 0x7fa2c019ba90 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.420 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.807+0000 7fa2c8449700 1 -- 172.21.15.96:0/80168346 >> 172.21.15.96:0/80168346 conn(0x7fa2c0103ef0 msgr2=0x7fa2c0191d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:13.420 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.807+0000 7fa2c8449700 1 -- 172.21.15.96:0/80168346 shutdown_connections 2024-04-25T14:04:13.420 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:12.807+0000 7fa2c8449700 1 -- 172.21.15.96:0/80168346 wait complete. 2024-04-25T14:04:13.420 INFO:teuthology.orchestra.run.smithi096.stdout:Enabling the dashboard module... 2024-04-25T14:04:13.719 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:13 smithi096 ceph-mon[29207]: from='mgr.14120 172.21.15.96:0/4269437437' entity='mgr.x' 2024-04-25T14:04:13.720 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:13 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/80168346' entity='client.admin' 2024-04-25T14:04:14.804 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:14 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/4118617406' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-25T14:04:14.804 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:04:14 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: ignoring --setuser ceph since I am not root 2024-04-25T14:04:14.804 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:04:14 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: ignoring --setgroup ceph since I am not root 2024-04-25T14:04:14.805 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:04:14 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:04:14.627+0000 7fedfba33200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:04:15.108 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:04:14 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:04:14.818+0000 7fedfba33200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:04:15.109 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:04:14 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:04:14.921+0000 7fedfba33200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:04:15.197 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:13.830+0000 7f64d71df700 1 Processor -- start 2024-04-25T14:04:15.202 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:13.830+0000 7f64d71df700 1 -- start start 2024-04-25T14:04:15.203 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:13.831+0000 7f64d71df700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64d007f620 0x7f64d007da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.203 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:13.831+0000 7f64d71df700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f64d007df90 con 0x7f64d007f620 2024-04-25T14:04:15.203 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:13.831+0000 7f64d4f7b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64d007f620 0x7f64d007da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.203 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:13.831+0000 7f64d4f7b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64d007f620 0x7f64d007da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34062/0 (socket says 172.21.15.96:34062) 2024-04-25T14:04:15.203 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:13.831+0000 7f64d4f7b700 1 -- 172.21.15.96:0/235185798 learned_addr learned my addr 172.21.15.96:0/235185798 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:04:15.203 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:13.831+0000 7f64d4f7b700 1 -- 172.21.15.96:0/235185798 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64d007e110 con 0x7f64d007f620 2024-04-25T14:04:15.203 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:13.831+0000 7f64d4f7b700 1 --2- 172.21.15.96:0/235185798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64d007f620 0x7f64d007da50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f64c40098a0 tx=0x7f64c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3612a47aab8f3752 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:15.203 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:13.831+0000 7f64cf7fe700 1 -- 172.21.15.96:0/235185798 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f64c4004160 con 0x7f64d007f620 2024-04-25T14:04:15.203 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:13.832+0000 7f64cf7fe700 1 -- 172.21.15.96:0/235185798 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64c4004300 con 0x7f64d007f620 2024-04-25T14:04:15.203 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:13.832+0000 7f64cf7fe700 1 -- 172.21.15.96:0/235185798 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f64c4011a20 con 0x7f64d007f620 2024-04-25T14:04:15.203 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:13.832+0000 7f64d71df700 1 -- 172.21.15.96:0/235185798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64d007f620 msgr2=0x7f64d007da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:15.203 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:13.832+0000 7f64d71df700 1 --2- 172.21.15.96:0/235185798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64d007f620 0x7f64d007da50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f64c40098a0 tx=0x7f64c4009bb0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.203 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:13.832+0000 7f64d71df700 1 -- 172.21.15.96:0/235185798 shutdown_connections 2024-04-25T14:04:15.203 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:13.832+0000 7f64d71df700 1 --2- 172.21.15.96:0/235185798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64d007f620 0x7f64d007da50 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.204 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:13.832+0000 7f64d71df700 1 -- 172.21.15.96:0/235185798 >> 172.21.15.96:0/235185798 conn(0x7f64d0105080 msgr2=0x7f64d01074d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:15.204 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:13.832+0000 7f64d71df700 1 -- 172.21.15.96:0/235185798 shutdown_connections 2024-04-25T14:04:15.204 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:13.832+0000 7f64d71df700 1 -- 172.21.15.96:0/235185798 wait complete. 2024-04-25T14:04:15.204 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:13.833+0000 7f64d71df700 1 Processor -- start 2024-04-25T14:04:15.204 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:13.833+0000 7f64d71df700 1 -- start start 2024-04-25T14:04:15.204 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:13.833+0000 7f64d71df700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64d019fee0 0x7f64d01a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.204 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:13.833+0000 7f64d71df700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f64d010ffa0 con 0x7f64d019fee0 2024-04-25T14:04:15.204 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:13.833+0000 7f64d4f7b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64d019fee0 0x7f64d01a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.204 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:13.833+0000 7f64d4f7b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64d019fee0 0x7f64d01a0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34064/0 (socket says 172.21.15.96:34064) 2024-04-25T14:04:15.204 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:13.833+0000 7f64d4f7b700 1 -- 172.21.15.96:0/4118617406 learned_addr learned my addr 172.21.15.96:0/4118617406 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:04:15.204 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:13.833+0000 7f64d4f7b700 1 -- 172.21.15.96:0/4118617406 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64d01a0890 con 0x7f64d019fee0 2024-04-25T14:04:15.204 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:13.834+0000 7f64d4f7b700 1 --2- 172.21.15.96:0/4118617406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64d019fee0 0x7f64d01a0310 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f64c4015000 tx=0x7f64c40117c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:15.204 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:13.834+0000 7f64cdffb700 1 -- 172.21.15.96:0/4118617406 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f64c4011ad0 con 0x7f64d019fee0 2024-04-25T14:04:15.204 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:13.834+0000 7f64cdffb700 1 -- 172.21.15.96:0/4118617406 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64c4003b50 con 0x7f64d019fee0 2024-04-25T14:04:15.204 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:13.834+0000 7f64d71df700 1 -- 172.21.15.96:0/4118617406 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64d01a0b20 con 0x7f64d019fee0 2024-04-25T14:04:15.204 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:13.834+0000 7f64cdffb700 1 -- 172.21.15.96:0/4118617406 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f64c401be10 con 0x7f64d019fee0 2024-04-25T14:04:15.205 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:13.834+0000 7f64d71df700 1 -- 172.21.15.96:0/4118617406 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64d01a3650 con 0x7f64d019fee0 2024-04-25T14:04:15.205 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:13.835+0000 7f64d71df700 1 -- 172.21.15.96:0/4118617406 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64d0064cf0 con 0x7f64d019fee0 2024-04-25T14:04:15.205 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:13.835+0000 7f64cdffb700 1 -- 172.21.15.96:0/4118617406 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f64c401c470 con 0x7f64d019fee0 2024-04-25T14:04:15.205 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:13.835+0000 7f64cdffb700 1 --2- 172.21.15.96:0/4118617406 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f64c003d540 0x7f64c003fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.205 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:13.835+0000 7f64cdffb700 1 -- 172.21.15.96:0/4118617406 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f64c4052b10 con 0x7f64d019fee0 2024-04-25T14:04:15.205 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:13.835+0000 7f64cffff700 1 --2- 172.21.15.96:0/4118617406 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f64c003d540 0x7f64c003fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.205 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:13.836+0000 7f64cffff700 1 --2- 172.21.15.96:0/4118617406 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f64c003d540 0x7f64c003fa50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f64bc006fd0 tx=0x7f64bc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:15.205 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:13.840+0000 7f64cdffb700 1 -- 172.21.15.96:0/4118617406 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f64c400f2e0 con 0x7f64d019fee0 2024-04-25T14:04:15.205 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:14.115+0000 7f64d71df700 1 -- 172.21.15.96:0/4118617406 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f64d01a3f20 con 0x7f64d019fee0 2024-04-25T14:04:15.205 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:14.424+0000 7f64cdffb700 1 -- 172.21.15.96:0/4118617406 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f64c4052870 con 0x7f64d019fee0 2024-04-25T14:04:15.205 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:14.426+0000 7f64cdffb700 1 -- 172.21.15.96:0/4118617406 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mgrmap(e 9) ==== 49529+0+0 (secure 0 0 0) 0x7f64c401acd0 con 0x7f64d019fee0 2024-04-25T14:04:15.205 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:14.427+0000 7f64d71df700 1 -- 172.21.15.96:0/4118617406 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f64c003d540 msgr2=0x7f64c003fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:15.205 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:14.427+0000 7f64d71df700 1 --2- 172.21.15.96:0/4118617406 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f64c003d540 0x7f64c003fa50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f64bc006fd0 tx=0x7f64bc006e40 comp rx=0 tx=0).stop 2024-04-25T14:04:15.205 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:14.427+0000 7f64d71df700 1 -- 172.21.15.96:0/4118617406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64d019fee0 msgr2=0x7f64d01a0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:15.205 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:14.427+0000 7f64d71df700 1 --2- 172.21.15.96:0/4118617406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64d019fee0 0x7f64d01a0310 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f64c4015000 tx=0x7f64c40117c0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.206 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:14.427+0000 7f64d71df700 1 -- 172.21.15.96:0/4118617406 shutdown_connections 2024-04-25T14:04:15.206 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:14.427+0000 7f64d71df700 1 --2- 172.21.15.96:0/4118617406 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f64c003d540 0x7f64c003fa50 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.206 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:14.427+0000 7f64d71df700 1 --2- 172.21.15.96:0/4118617406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64d019fee0 0x7f64d01a0310 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.206 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:14.427+0000 7f64d71df700 1 -- 172.21.15.96:0/4118617406 >> 172.21.15.96:0/4118617406 conn(0x7f64d0105080 msgr2=0x7f64d01069e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:15.206 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:14.427+0000 7f64d71df700 1 -- 172.21.15.96:0/4118617406 shutdown_connections 2024-04-25T14:04:15.206 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:14.427+0000 7f64d71df700 1 -- 172.21.15.96:0/4118617406 wait complete. 2024-04-25T14:04:15.446 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:15 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/4118617406' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-25T14:04:15.447 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:15 smithi096 ceph-mon[29207]: mgrmap e9: x(active, since 20s) 2024-04-25T14:04:15.447 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:04:15 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:04:15.232+0000 7fedfba33200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:04:15.447 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:04:15 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:04:15.317+0000 7fedfba33200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:04:16.656 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout { 2024-04-25T14:04:16.657 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-25T14:04:16.657 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T14:04:16.657 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T14:04:16.657 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T14:04:16.657 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout } 2024-04-25T14:04:16.657 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.650+0000 7f901eb37700 1 Processor -- start 2024-04-25T14:04:16.657 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.650+0000 7f901eb37700 1 -- start start 2024-04-25T14:04:16.657 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.650+0000 7f901eb37700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90181082f0 0x7f9018108700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:16.657 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.650+0000 7f901eb37700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f901807d5f0 con 0x7f90181082f0 2024-04-25T14:04:16.657 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.651+0000 7f901c8d3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90181082f0 0x7f9018108700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:16.657 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.651+0000 7f901c8d3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90181082f0 0x7f9018108700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34098/0 (socket says 172.21.15.96:34098) 2024-04-25T14:04:16.657 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.651+0000 7f901c8d3700 1 -- 172.21.15.96:0/2698554839 learned_addr learned my addr 172.21.15.96:0/2698554839 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:04:16.658 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.651+0000 7f901c8d3700 1 -- 172.21.15.96:0/2698554839 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9018108c40 con 0x7f90181082f0 2024-04-25T14:04:16.658 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.651+0000 7f901c8d3700 1 --2- 172.21.15.96:0/2698554839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90181082f0 0x7f9018108700 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f900c0098a0 tx=0x7f900c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba489648fde7225 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:16.658 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.651+0000 7f90177fe700 1 -- 172.21.15.96:0/2698554839 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f900c004160 con 0x7f90181082f0 2024-04-25T14:04:16.658 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.651+0000 7f90177fe700 1 -- 172.21.15.96:0/2698554839 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f900c004300 con 0x7f90181082f0 2024-04-25T14:04:16.658 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.651+0000 7f90177fe700 1 -- 172.21.15.96:0/2698554839 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f900c011a20 con 0x7f90181082f0 2024-04-25T14:04:16.658 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.652+0000 7f901eb37700 1 -- 172.21.15.96:0/2698554839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90181082f0 msgr2=0x7f9018108700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:16.658 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.652+0000 7f901eb37700 1 --2- 172.21.15.96:0/2698554839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90181082f0 0x7f9018108700 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f900c0098a0 tx=0x7f900c009bb0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.658 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.652+0000 7f901eb37700 1 -- 172.21.15.96:0/2698554839 shutdown_connections 2024-04-25T14:04:16.658 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.652+0000 7f901eb37700 1 --2- 172.21.15.96:0/2698554839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90181082f0 0x7f9018108700 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.658 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.652+0000 7f901eb37700 1 -- 172.21.15.96:0/2698554839 >> 172.21.15.96:0/2698554839 conn(0x7f9018103f40 msgr2=0x7f9018106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:16.658 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.652+0000 7f901eb37700 1 -- 172.21.15.96:0/2698554839 shutdown_connections 2024-04-25T14:04:16.658 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.652+0000 7f901eb37700 1 -- 172.21.15.96:0/2698554839 wait complete. 2024-04-25T14:04:16.658 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.653+0000 7f901eb37700 1 Processor -- start 2024-04-25T14:04:16.659 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.653+0000 7f901eb37700 1 -- start start 2024-04-25T14:04:16.659 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.653+0000 7f901eb37700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90181082f0 0x7f901819fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:16.659 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.653+0000 7f901eb37700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9018109b80 con 0x7f90181082f0 2024-04-25T14:04:16.659 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.653+0000 7f901c8d3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90181082f0 0x7f901819fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:16.659 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.653+0000 7f901c8d3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90181082f0 0x7f901819fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34102/0 (socket says 172.21.15.96:34102) 2024-04-25T14:04:16.659 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.653+0000 7f901c8d3700 1 -- 172.21.15.96:0/821772558 learned_addr learned my addr 172.21.15.96:0/821772558 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:04:16.659 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.654+0000 7f901c8d3700 1 -- 172.21.15.96:0/821772558 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90181a03c0 con 0x7f90181082f0 2024-04-25T14:04:16.659 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.654+0000 7f901c8d3700 1 --2- 172.21.15.96:0/821772558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90181082f0 0x7f901819fe80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f900c0119f0 tx=0x7f900c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:16.659 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.654+0000 7f9015ffb700 1 -- 172.21.15.96:0/821772558 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f900c0039e0 con 0x7f90181082f0 2024-04-25T14:04:16.659 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.654+0000 7f901eb37700 1 -- 172.21.15.96:0/821772558 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90181a0650 con 0x7f90181082f0 2024-04-25T14:04:16.659 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.654+0000 7f9015ffb700 1 -- 172.21.15.96:0/821772558 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f900c01b8f0 con 0x7f90181082f0 2024-04-25T14:04:16.659 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.654+0000 7f9015ffb700 1 -- 172.21.15.96:0/821772558 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f900c0237a0 con 0x7f90181082f0 2024-04-25T14:04:16.659 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.655+0000 7f901eb37700 1 -- 172.21.15.96:0/821772558 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90181a3260 con 0x7f90181082f0 2024-04-25T14:04:16.659 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.656+0000 7f9015ffb700 1 -- 172.21.15.96:0/821772558 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 9) ==== 49529+0+0 (secure 0 0 0) 0x7f900c02c3f0 con 0x7f90181082f0 2024-04-25T14:04:16.660 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.656+0000 7f901eb37700 1 -- 172.21.15.96:0/821772558 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9018064cf0 con 0x7f90181082f0 2024-04-25T14:04:16.660 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.656+0000 7f9015ffb700 1 --2- 172.21.15.96:0/821772558 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f900803d690 0x7f900803fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:16.660 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.656+0000 7f9015ffb700 1 -- 172.21.15.96:0/821772558 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f900c01baa0 con 0x7f90181082f0 2024-04-25T14:04:16.660 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.656+0000 7f9017fff700 1 -- 172.21.15.96:0/821772558 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f900803d690 msgr2=0x7f900803fba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/433591062 2024-04-25T14:04:16.660 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.656+0000 7f9017fff700 1 --2- 172.21.15.96:0/821772558 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f900803d690 0x7f900803fba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:04:16.660 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.661+0000 7f9015ffb700 1 -- 172.21.15.96:0/821772558 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f900c01f030 con 0x7f90181082f0 2024-04-25T14:04:16.660 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.857+0000 7f9017fff700 1 -- 172.21.15.96:0/821772558 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f900803d690 msgr2=0x7f900803fba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/433591062 2024-04-25T14:04:16.660 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.857+0000 7f9017fff700 1 --2- 172.21.15.96:0/821772558 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f900803d690 0x7f900803fba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T14:04:16.660 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.959+0000 7f901eb37700 1 -- 172.21.15.96:0/821772558 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f901802e7e0 con 0x7f90181082f0 2024-04-25T14:04:16.660 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.959+0000 7f9015ffb700 1 -- 172.21.15.96:0/821772558 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f900c016df0 con 0x7f90181082f0 2024-04-25T14:04:16.660 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.961+0000 7f901eb37700 1 -- 172.21.15.96:0/821772558 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f900803d690 msgr2=0x7f900803fba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:04:16.660 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.961+0000 7f901eb37700 1 --2- 172.21.15.96:0/821772558 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f900803d690 0x7f900803fba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.660 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.961+0000 7f901eb37700 1 -- 172.21.15.96:0/821772558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90181082f0 msgr2=0x7f901819fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:16.661 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.961+0000 7f901eb37700 1 --2- 172.21.15.96:0/821772558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90181082f0 0x7f901819fe80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f900c0119f0 tx=0x7f900c011ad0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.661 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.961+0000 7f901eb37700 1 -- 172.21.15.96:0/821772558 shutdown_connections 2024-04-25T14:04:16.661 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.961+0000 7f901eb37700 1 --2- 172.21.15.96:0/821772558 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f900803d690 0x7f900803fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.661 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.961+0000 7f901eb37700 1 --2- 172.21.15.96:0/821772558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90181082f0 0x7f901819fe80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.661 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.961+0000 7f901eb37700 1 -- 172.21.15.96:0/821772558 >> 172.21.15.96:0/821772558 conn(0x7f9018103f40 msgr2=0x7f9018193f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:16.661 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.962+0000 7f901eb37700 1 -- 172.21.15.96:0/821772558 shutdown_connections 2024-04-25T14:04:16.661 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:15.962+0000 7f901eb37700 1 -- 172.21.15.96:0/821772558 wait complete. 2024-04-25T14:04:16.661 INFO:teuthology.orchestra.run.smithi096.stdout:Waiting for the mgr to restart... 2024-04-25T14:04:16.661 INFO:teuthology.orchestra.run.smithi096.stdout:Waiting for mgr epoch 9... 2024-04-25T14:04:16.722 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:16 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/821772558' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T14:04:16.723 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:04:16 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:04:16.538+0000 7fedfba33200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:04:16.723 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:04:16 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:04:16.724+0000 7fedfba33200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:04:17.016 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:04:16 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:04:16.895+0000 7fedfba33200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:04:17.291 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:04:17 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:04:17.199+0000 7fedfba33200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:04:17.609 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:04:17 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:04:17.291+0000 7fedfba33200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:04:17.609 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:04:17 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:04:17.493+0000 7fedfba33200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:04:18.109 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:04:17 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:04:17.686+0000 7fedfba33200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:04:18.536 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:04:18 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:04:18.279+0000 7fedfba33200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T14:04:18.798 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:04:18 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:04:18.702+0000 7fedfba33200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:04:18.798 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:04:18 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:04:18.798+0000 7fedfba33200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:04:19.109 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:04:19 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:04:19.001+0000 7fedfba33200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T14:04:19.851 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:04:19 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:04:19.515+0000 7fedfba33200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:04:19.851 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:04:19 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:04:19.607+0000 7fedfba33200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:04:20.109 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:04:19 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:04:19.851+0000 7fedfba33200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:04:20.109 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:04:19 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:04:19.984+0000 7fedfba33200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:04:20.109 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:04:20 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:04:20.080+0000 7fedfba33200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:04:20.491 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:04:20 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:04:20.195+0000 7fedfba33200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:04:20.491 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:04:20 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:04:20.491+0000 7fedfba33200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:04:21.359 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:04:21 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:04:21.088+0000 7fedfba33200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:04:21.359 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:04:21 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:04:21.305+0000 7fedfba33200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:04:21.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:21 smithi096 ceph-mon[29207]: Active manager daemon x restarted 2024-04-25T14:04:21.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:21 smithi096 ceph-mon[29207]: Activating manager daemon x 2024-04-25T14:04:21.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:21 smithi096 ceph-mon[29207]: osdmap e3: 0 total, 0 up, 0 in 2024-04-25T14:04:21.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:21 smithi096 ceph-mon[29207]: mgrmap e10: x(active, starting, since 0.00634283s) 2024-04-25T14:04:21.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:21 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:04:21.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:21 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T14:04:21.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:21 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T14:04:21.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:21 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T14:04:21.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:21 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T14:04:21.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:21 smithi096 ceph-mon[29207]: Manager daemon x is now available 2024-04-25T14:04:21.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:21 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:04:22.531 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:22 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T14:04:22.531 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:22 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T14:04:22.531 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:22 smithi096 ceph-mon[29207]: mgrmap e11: x(active, since 1.00727s) 2024-04-25T14:04:23.208 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout { 2024-04-25T14:04:23.208 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-25T14:04:23.208 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T14:04:23.208 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout } 2024-04-25T14:04:23.208 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:17.129+0000 7f9a2d96c700 1 Processor -- start 2024-04-25T14:04:23.208 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:17.129+0000 7f9a2d96c700 1 -- start start 2024-04-25T14:04:23.208 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:17.129+0000 7f9a2d96c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a28108880 0x7f9a28108c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:23.209 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:17.129+0000 7f9a2d96c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9a28109280 con 0x7f9a28108880 2024-04-25T14:04:23.209 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:17.129+0000 7f9a26ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a28108880 0x7f9a28108c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:23.209 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:17.129+0000 7f9a26ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a28108880 0x7f9a28108c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34118/0 (socket says 172.21.15.96:34118) 2024-04-25T14:04:23.209 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:17.129+0000 7f9a26ffd700 1 -- 172.21.15.96:0/303878645 learned_addr learned my addr 172.21.15.96:0/303878645 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:04:23.209 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:17.130+0000 7f9a26ffd700 1 -- 172.21.15.96:0/303878645 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a28109a50 con 0x7f9a28108880 2024-04-25T14:04:23.209 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:17.130+0000 7f9a26ffd700 1 --2- 172.21.15.96:0/303878645 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a28108880 0x7f9a28108c90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c0098a0 tx=0x7f9a1c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45bd5c528efc357b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:23.209 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:17.130+0000 7f9a25ffb700 1 -- 172.21.15.96:0/303878645 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a1c004160 con 0x7f9a28108880 2024-04-25T14:04:23.209 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:17.130+0000 7f9a25ffb700 1 -- 172.21.15.96:0/303878645 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a1c004300 con 0x7f9a28108880 2024-04-25T14:04:23.209 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:17.130+0000 7f9a25ffb700 1 -- 172.21.15.96:0/303878645 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a1c011a20 con 0x7f9a28108880 2024-04-25T14:04:23.209 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:17.131+0000 7f9a2d96c700 1 -- 172.21.15.96:0/303878645 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a28108880 msgr2=0x7f9a28108c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:23.209 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:17.131+0000 7f9a2d96c700 1 --2- 172.21.15.96:0/303878645 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a28108880 0x7f9a28108c90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c0098a0 tx=0x7f9a1c009bb0 comp rx=0 tx=0).stop 2024-04-25T14:04:23.209 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:17.131+0000 7f9a2d96c700 1 -- 172.21.15.96:0/303878645 shutdown_connections 2024-04-25T14:04:23.209 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:17.131+0000 7f9a2d96c700 1 --2- 172.21.15.96:0/303878645 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a28108880 0x7f9a28108c90 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:23.209 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:17.131+0000 7f9a2d96c700 1 -- 172.21.15.96:0/303878645 >> 172.21.15.96:0/303878645 conn(0x7f9a28103f20 msgr2=0x7f9a28106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:23.210 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:17.131+0000 7f9a2d96c700 1 -- 172.21.15.96:0/303878645 shutdown_connections 2024-04-25T14:04:23.210 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:17.131+0000 7f9a2d96c700 1 -- 172.21.15.96:0/303878645 wait complete. 2024-04-25T14:04:23.210 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:17.132+0000 7f9a2d96c700 1 Processor -- start 2024-04-25T14:04:23.210 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:17.132+0000 7f9a2d96c700 1 -- start start 2024-04-25T14:04:23.210 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:17.132+0000 7f9a2d96c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a28108880 0x7f9a2819dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:23.210 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:17.132+0000 7f9a2d96c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2810ce00 con 0x7f9a28108880 2024-04-25T14:04:23.210 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:17.132+0000 7f9a26ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a28108880 0x7f9a2819dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:23.210 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:17.132+0000 7f9a26ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a28108880 0x7f9a2819dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34130/0 (socket says 172.21.15.96:34130) 2024-04-25T14:04:23.210 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:17.132+0000 7f9a26ffd700 1 -- 172.21.15.96:0/3667493633 learned_addr learned my addr 172.21.15.96:0/3667493633 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:04:23.210 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:17.132+0000 7f9a26ffd700 1 -- 172.21.15.96:0/3667493633 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a2819e240 con 0x7f9a28108880 2024-04-25T14:04:23.210 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:17.133+0000 7f9a26ffd700 1 --2- 172.21.15.96:0/3667493633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a28108880 0x7f9a2819dd00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c003e70 tx=0x7f9a1c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:23.210 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:17.133+0000 7f9a2c96a700 1 -- 172.21.15.96:0/3667493633 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a1c003af0 con 0x7f9a28108880 2024-04-25T14:04:23.210 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:17.133+0000 7f9a2c96a700 1 -- 172.21.15.96:0/3667493633 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a1c01a8e0 con 0x7f9a28108880 2024-04-25T14:04:23.210 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:17.133+0000 7f9a2c96a700 1 -- 172.21.15.96:0/3667493633 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a1c024870 con 0x7f9a28108880 2024-04-25T14:04:23.211 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:17.133+0000 7f9a2d96c700 1 -- 172.21.15.96:0/3667493633 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a2819e4d0 con 0x7f9a28108880 2024-04-25T14:04:23.211 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:17.133+0000 7f9a2d96c700 1 -- 172.21.15.96:0/3667493633 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a281a10e0 con 0x7f9a28108880 2024-04-25T14:04:23.211 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:17.134+0000 7f9a2c96a700 1 -- 172.21.15.96:0/3667493633 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 9) ==== 49529+0+0 (secure 0 0 0) 0x7f9a1c022020 con 0x7f9a28108880 2024-04-25T14:04:23.211 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:17.134+0000 7f9a2c96a700 1 --2- 172.21.15.96:0/3667493633 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f9a1003d630 0x7f9a1003fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:23.211 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:17.134+0000 7f9a2c96a700 1 -- 172.21.15.96:0/3667493633 --> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9a10040250 con 0x7f9a1003d630 2024-04-25T14:04:23.211 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:17.134+0000 7f9a2c96a700 1 -- 172.21.15.96:0/3667493633 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9a1c053640 con 0x7f9a28108880 2024-04-25T14:04:23.211 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:17.134+0000 7f9a267fc700 1 -- 172.21.15.96:0/3667493633 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f9a1003d630 msgr2=0x7f9a1003fb40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/433591062 2024-04-25T14:04:23.211 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:17.134+0000 7f9a267fc700 1 --2- 172.21.15.96:0/3667493633 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f9a1003d630 0x7f9a1003fb40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:04:23.211 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:17.335+0000 7f9a267fc700 1 -- 172.21.15.96:0/3667493633 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f9a1003d630 msgr2=0x7f9a1003fb40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/433591062 2024-04-25T14:04:23.211 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:17.335+0000 7f9a267fc700 1 --2- 172.21.15.96:0/3667493633 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f9a1003d630 0x7f9a1003fb40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T14:04:23.211 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:17.736+0000 7f9a267fc700 1 -- 172.21.15.96:0/3667493633 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f9a1003d630 msgr2=0x7f9a1003fb40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/433591062 2024-04-25T14:04:23.211 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:17.736+0000 7f9a267fc700 1 --2- 172.21.15.96:0/3667493633 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f9a1003d630 0x7f9a1003fb40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T14:04:23.211 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:18.537+0000 7f9a267fc700 1 -- 172.21.15.96:0/3667493633 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f9a1003d630 msgr2=0x7f9a1003fb40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/433591062 2024-04-25T14:04:23.212 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:18.537+0000 7f9a267fc700 1 --2- 172.21.15.96:0/3667493633 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f9a1003d630 0x7f9a1003fb40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T14:04:23.212 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:20.138+0000 7f9a267fc700 1 -- 172.21.15.96:0/3667493633 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f9a1003d630 msgr2=0x7f9a1003fb40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/433591062 2024-04-25T14:04:23.212 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:20.138+0000 7f9a267fc700 1 --2- 172.21.15.96:0/3667493633 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f9a1003d630 0x7f9a1003fb40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T14:04:23.212 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:21.311+0000 7f9a2c96a700 1 -- 172.21.15.96:0/3667493633 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mgrmap(e 10) ==== 49296+0+0 (secure 0 0 0) 0x7f9a1c02d5d0 con 0x7f9a28108880 2024-04-25T14:04:23.212 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:21.311+0000 7f9a2c96a700 1 -- 172.21.15.96:0/3667493633 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f9a1003d630 msgr2=0x7f9a1003fb40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:04:23.212 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:21.311+0000 7f9a2c96a700 1 --2- 172.21.15.96:0/3667493633 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f9a1003d630 0x7f9a1003fb40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:23.212 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:22.311+0000 7f9a2c96a700 1 -- 172.21.15.96:0/3667493633 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 11) ==== 49423+0+0 (secure 0 0 0) 0x7f9a1c00bd00 con 0x7f9a28108880 2024-04-25T14:04:23.212 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:22.311+0000 7f9a2c96a700 1 --2- 172.21.15.96:0/3667493633 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f9a1c0531d0 0x7f9a10042fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:23.212 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:22.311+0000 7f9a2c96a700 1 -- 172.21.15.96:0/3667493633 --> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9a1c018890 con 0x7f9a1c0531d0 2024-04-25T14:04:23.212 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:22.312+0000 7f9a267fc700 1 --2- 172.21.15.96:0/3667493633 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f9a1c0531d0 0x7f9a10042fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:23.212 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:22.312+0000 7f9a267fc700 1 --2- 172.21.15.96:0/3667493633 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f9a1c0531d0 0x7f9a10042fd0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9a18003a10 tx=0x7f9a180092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:23.212 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:22.313+0000 7f9a2c96a700 1 -- 172.21.15.96:0/3667493633 <== mgr.14152 v2:172.21.15.96:6800/536332425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f9a1c018890 con 0x7f9a1c0531d0 2024-04-25T14:04:23.212 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:22.322+0000 7f9a0e7fc700 1 -- 172.21.15.96:0/3667493633 --> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f9a28064cf0 con 0x7f9a1c0531d0 2024-04-25T14:04:23.212 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:22.322+0000 7f9a2c96a700 1 -- 172.21.15.96:0/3667493633 <== mgr.14152 v2:172.21.15.96:6800/536332425 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f9a28064cf0 con 0x7f9a1c0531d0 2024-04-25T14:04:23.213 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:22.322+0000 7f9a0e7fc700 1 -- 172.21.15.96:0/3667493633 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f9a1c0531d0 msgr2=0x7f9a10042fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:23.213 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:22.322+0000 7f9a0e7fc700 1 --2- 172.21.15.96:0/3667493633 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f9a1c0531d0 0x7f9a10042fd0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9a18003a10 tx=0x7f9a180092b0 comp rx=0 tx=0).stop 2024-04-25T14:04:23.213 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:22.322+0000 7f9a0e7fc700 1 -- 172.21.15.96:0/3667493633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a28108880 msgr2=0x7f9a2819dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:23.213 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:22.322+0000 7f9a0e7fc700 1 --2- 172.21.15.96:0/3667493633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a28108880 0x7f9a2819dd00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c003e70 tx=0x7f9a1c011aa0 comp rx=0 tx=0).stop 2024-04-25T14:04:23.213 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:22.322+0000 7f9a0e7fc700 1 -- 172.21.15.96:0/3667493633 shutdown_connections 2024-04-25T14:04:23.213 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:22.322+0000 7f9a0e7fc700 1 --2- 172.21.15.96:0/3667493633 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f9a1c0531d0 0x7f9a10042fd0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:23.213 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:22.322+0000 7f9a0e7fc700 1 --2- 172.21.15.96:0/3667493633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a28108880 0x7f9a2819dd00 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:23.213 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:22.322+0000 7f9a0e7fc700 1 --2- 172.21.15.96:0/3667493633 >> [v2:172.21.15.96:6800/433591062,v1:172.21.15.96:6801/433591062] conn(0x7f9a1003d630 0x7f9a1003fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:23.213 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:22.322+0000 7f9a0e7fc700 1 -- 172.21.15.96:0/3667493633 >> 172.21.15.96:0/3667493633 conn(0x7f9a28103f20 msgr2=0x7f9a28194290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:23.213 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:22.323+0000 7f9a0e7fc700 1 -- 172.21.15.96:0/3667493633 shutdown_connections 2024-04-25T14:04:23.213 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:22.323+0000 7f9a0e7fc700 1 -- 172.21.15.96:0/3667493633 wait complete. 2024-04-25T14:04:23.213 INFO:teuthology.orchestra.run.smithi096.stdout:mgr epoch 9 is available 2024-04-25T14:04:23.213 INFO:teuthology.orchestra.run.smithi096.stdout:Generating a dashboard self-signed certificate... 2024-04-25T14:04:23.482 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:23 smithi096 ceph-mon[29207]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T14:04:23.483 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:23 smithi096 ceph-mon[29207]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T14:04:23.483 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:23 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:04:24.570 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-25T14:04:24.571 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:23.715+0000 7f2601233700 1 Processor -- start 2024-04-25T14:04:24.571 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:23.715+0000 7f2601233700 1 -- start start 2024-04-25T14:04:24.571 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:23.715+0000 7f2601233700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25fc10a470 0x7f25fc10c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:24.571 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:23.715+0000 7f2601233700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f25fc07cdc0 con 0x7f25fc10a470 2024-04-25T14:04:24.571 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:23.715+0000 7f25fad9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25fc10a470 0x7f25fc10c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:24.571 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:23.715+0000 7f25fad9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25fc10a470 0x7f25fc10c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53086/0 (socket says 172.21.15.96:53086) 2024-04-25T14:04:24.571 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:23.715+0000 7f25fad9d700 1 -- 172.21.15.96:0/1784103739 learned_addr learned my addr 172.21.15.96:0/1784103739 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:04:24.571 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:23.716+0000 7f25fad9d700 1 -- 172.21.15.96:0/1784103739 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25fc10cdd0 con 0x7f25fc10a470 2024-04-25T14:04:24.571 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:23.716+0000 7f25fad9d700 1 --2- 172.21.15.96:0/1784103739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25fc10a470 0x7f25fc10c890 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f25f00098a0 tx=0x7f25f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1efc1a63f662f9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:24.571 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:23.716+0000 7f25f9d9b700 1 -- 172.21.15.96:0/1784103739 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25f0004160 con 0x7f25fc10a470 2024-04-25T14:04:24.571 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:23.716+0000 7f25f9d9b700 1 -- 172.21.15.96:0/1784103739 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25f0004300 con 0x7f25fc10a470 2024-04-25T14:04:24.572 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:23.716+0000 7f25f9d9b700 1 -- 172.21.15.96:0/1784103739 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25f0011a20 con 0x7f25fc10a470 2024-04-25T14:04:24.572 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:23.717+0000 7f2601233700 1 -- 172.21.15.96:0/1784103739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25fc10a470 msgr2=0x7f25fc10c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:24.572 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:23.717+0000 7f2601233700 1 --2- 172.21.15.96:0/1784103739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25fc10a470 0x7f25fc10c890 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f25f00098a0 tx=0x7f25f0009bb0 comp rx=0 tx=0).stop 2024-04-25T14:04:24.572 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:23.717+0000 7f2601233700 1 -- 172.21.15.96:0/1784103739 shutdown_connections 2024-04-25T14:04:24.572 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:23.717+0000 7f2601233700 1 --2- 172.21.15.96:0/1784103739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25fc10a470 0x7f25fc10c890 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:24.572 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:23.717+0000 7f2601233700 1 -- 172.21.15.96:0/1784103739 >> 172.21.15.96:0/1784103739 conn(0x7f25fc103ef0 msgr2=0x7f25fc106340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:24.572 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:23.717+0000 7f2601233700 1 -- 172.21.15.96:0/1784103739 shutdown_connections 2024-04-25T14:04:24.572 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:23.717+0000 7f2601233700 1 -- 172.21.15.96:0/1784103739 wait complete. 2024-04-25T14:04:24.572 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:23.718+0000 7f2601233700 1 Processor -- start 2024-04-25T14:04:24.572 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:23.718+0000 7f2601233700 1 -- start start 2024-04-25T14:04:24.572 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:23.718+0000 7f2601233700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25fc10a470 0x7f25fc19dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:24.572 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:23.718+0000 7f2601233700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f25fc10dd70 con 0x7f25fc10a470 2024-04-25T14:04:24.572 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:23.718+0000 7f25fad9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25fc10a470 0x7f25fc19dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:24.573 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:23.718+0000 7f25fad9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25fc10a470 0x7f25fc19dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53088/0 (socket says 172.21.15.96:53088) 2024-04-25T14:04:24.573 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:23.718+0000 7f25fad9d700 1 -- 172.21.15.96:0/2105404236 learned_addr learned my addr 172.21.15.96:0/2105404236 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:04:24.573 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:23.719+0000 7f25fad9d700 1 -- 172.21.15.96:0/2105404236 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25fc19e160 con 0x7f25fc10a470 2024-04-25T14:04:24.573 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:23.719+0000 7f25fad9d700 1 --2- 172.21.15.96:0/2105404236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25fc10a470 0x7f25fc19dc20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f25f00119f0 tx=0x7f25f0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:24.573 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:23.719+0000 7f25e3fff700 1 -- 172.21.15.96:0/2105404236 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25f00039e0 con 0x7f25fc10a470 2024-04-25T14:04:24.573 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:23.719+0000 7f25e3fff700 1 -- 172.21.15.96:0/2105404236 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25f001b8f0 con 0x7f25fc10a470 2024-04-25T14:04:24.573 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:23.719+0000 7f2601233700 1 -- 172.21.15.96:0/2105404236 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25fc19e3f0 con 0x7f25fc10a470 2024-04-25T14:04:24.573 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:23.719+0000 7f25e3fff700 1 -- 172.21.15.96:0/2105404236 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25f00237a0 con 0x7f25fc10a470 2024-04-25T14:04:24.573 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:23.719+0000 7f2601233700 1 -- 172.21.15.96:0/2105404236 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25fc1a1000 con 0x7f25fc10a470 2024-04-25T14:04:24.573 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:23.720+0000 7f25e3fff700 1 -- 172.21.15.96:0/2105404236 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 11) ==== 49423+0+0 (secure 0 0 0) 0x7f25f002c3f0 con 0x7f25fc10a470 2024-04-25T14:04:24.573 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:23.720+0000 7f2601233700 1 -- 172.21.15.96:0/2105404236 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25fc064cf0 con 0x7f25fc10a470 2024-04-25T14:04:24.573 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:23.720+0000 7f25e3fff700 1 --2- 172.21.15.96:0/2105404236 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f25e403d570 0x7f25e403fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:24.573 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:23.720+0000 7f25e3fff700 1 -- 172.21.15.96:0/2105404236 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f25f0053ba0 con 0x7f25fc10a470 2024-04-25T14:04:24.574 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:23.720+0000 7f25fa59c700 1 --2- 172.21.15.96:0/2105404236 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f25e403d570 0x7f25e403fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:24.574 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:23.721+0000 7f25fa59c700 1 --2- 172.21.15.96:0/2105404236 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f25e403d570 0x7f25e403fa80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f25ec006fd0 tx=0x7f25ec006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:24.574 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:23.726+0000 7f25e3fff700 1 -- 172.21.15.96:0/2105404236 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f25f001f030 con 0x7f25fc10a470 2024-04-25T14:04:24.574 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:23.871+0000 7f25e3fff700 1 -- 172.21.15.96:0/2105404236 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 12) ==== 49529+0+0 (secure 0 0 0) 0x7f25f002bca0 con 0x7f25fc10a470 2024-04-25T14:04:24.574 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:23.936+0000 7f2601233700 1 -- 172.21.15.96:0/2105404236 --> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f25fc104e70 con 0x7f25e403d570 2024-04-25T14:04:24.574 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:23.999+0000 7f25e3fff700 1 -- 172.21.15.96:0/2105404236 <== mgr.14152 v2:172.21.15.96:6800/536332425 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f25fc104e70 con 0x7f25e403d570 2024-04-25T14:04:24.574 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:24.001+0000 7f2601233700 1 -- 172.21.15.96:0/2105404236 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f25e403d570 msgr2=0x7f25e403fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:24.574 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:24.001+0000 7f2601233700 1 --2- 172.21.15.96:0/2105404236 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f25e403d570 0x7f25e403fa80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f25ec006fd0 tx=0x7f25ec006e40 comp rx=0 tx=0).stop 2024-04-25T14:04:24.574 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:24.001+0000 7f2601233700 1 -- 172.21.15.96:0/2105404236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25fc10a470 msgr2=0x7f25fc19dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:24.574 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:24.001+0000 7f2601233700 1 --2- 172.21.15.96:0/2105404236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25fc10a470 0x7f25fc19dc20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f25f00119f0 tx=0x7f25f0011ad0 comp rx=0 tx=0).stop 2024-04-25T14:04:24.574 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:24.001+0000 7f2601233700 1 -- 172.21.15.96:0/2105404236 shutdown_connections 2024-04-25T14:04:24.574 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:24.001+0000 7f2601233700 1 --2- 172.21.15.96:0/2105404236 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f25e403d570 0x7f25e403fa80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:24.574 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:24.001+0000 7f2601233700 1 --2- 172.21.15.96:0/2105404236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25fc10a470 0x7f25fc19dc20 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:24.574 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:24.001+0000 7f2601233700 1 -- 172.21.15.96:0/2105404236 >> 172.21.15.96:0/2105404236 conn(0x7f25fc103ef0 msgr2=0x7f25fc104760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:24.575 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:24.001+0000 7f2601233700 1 -- 172.21.15.96:0/2105404236 shutdown_connections 2024-04-25T14:04:24.575 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:24.001+0000 7f2601233700 1 -- 172.21.15.96:0/2105404236 wait complete. 2024-04-25T14:04:24.575 INFO:teuthology.orchestra.run.smithi096.stdout:Creating initial admin user... 2024-04-25T14:04:24.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:24 smithi096 ceph-mon[29207]: [25/Apr/2024:14:04:22] ENGINE Bus STARTING 2024-04-25T14:04:24.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:24 smithi096 ceph-mon[29207]: [25/Apr/2024:14:04:22] ENGINE Serving on https://172.21.15.96:7150 2024-04-25T14:04:24.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:24 smithi096 ceph-mon[29207]: [25/Apr/2024:14:04:22] ENGINE Serving on http://172.21.15.96:8765 2024-04-25T14:04:24.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:24 smithi096 ceph-mon[29207]: [25/Apr/2024:14:04:22] ENGINE Bus STARTED 2024-04-25T14:04:24.689 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:24 smithi096 ceph-mon[29207]: mgrmap e12: x(active, since 2s) 2024-04-25T14:04:24.689 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:24 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:04:24.689 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:24 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:04:25.505 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:25 smithi096 ceph-mon[29207]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:04:25.506 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:25 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:04:26.413 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$stOXikWRfXl8vfQWcYmYI.YmLyX.VO66/qWN0K2a5epf5jFupzSiu", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714053865, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-25T14:04:26.413 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.014+0000 7faa5783d700 1 Processor -- start 2024-04-25T14:04:26.414 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.015+0000 7faa5783d700 1 -- start start 2024-04-25T14:04:26.414 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.015+0000 7faa5783d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa50107c40 0x7faa50108050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:26.414 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.015+0000 7faa5783d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faa50108620 con 0x7faa50107c40 2024-04-25T14:04:26.414 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.015+0000 7faa555d9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa50107c40 0x7faa50108050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:26.414 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.015+0000 7faa555d9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa50107c40 0x7faa50108050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53100/0 (socket says 172.21.15.96:53100) 2024-04-25T14:04:26.414 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.015+0000 7faa555d9700 1 -- 172.21.15.96:0/941453953 learned_addr learned my addr 172.21.15.96:0/941453953 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:04:26.414 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.015+0000 7faa555d9700 1 -- 172.21.15.96:0/941453953 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa5010ac40 con 0x7faa50107c40 2024-04-25T14:04:26.414 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.015+0000 7faa555d9700 1 --2- 172.21.15.96:0/941453953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa50107c40 0x7faa50108050 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7faa400098a0 tx=0x7faa40009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=683836d2e450cd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:26.414 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.016+0000 7faa47fff700 1 -- 172.21.15.96:0/941453953 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa40004160 con 0x7faa50107c40 2024-04-25T14:04:26.414 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.016+0000 7faa47fff700 1 -- 172.21.15.96:0/941453953 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa40004300 con 0x7faa50107c40 2024-04-25T14:04:26.414 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.016+0000 7faa47fff700 1 -- 172.21.15.96:0/941453953 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa40011a20 con 0x7faa50107c40 2024-04-25T14:04:26.415 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.016+0000 7faa5783d700 1 -- 172.21.15.96:0/941453953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa50107c40 msgr2=0x7faa50108050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:26.415 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.016+0000 7faa5783d700 1 --2- 172.21.15.96:0/941453953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa50107c40 0x7faa50108050 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7faa400098a0 tx=0x7faa40009bb0 comp rx=0 tx=0).stop 2024-04-25T14:04:26.415 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.017+0000 7faa5783d700 1 -- 172.21.15.96:0/941453953 shutdown_connections 2024-04-25T14:04:26.415 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.017+0000 7faa5783d700 1 --2- 172.21.15.96:0/941453953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa50107c40 0x7faa50108050 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:26.415 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.017+0000 7faa5783d700 1 -- 172.21.15.96:0/941453953 >> 172.21.15.96:0/941453953 conn(0x7faa50103670 msgr2=0x7faa50105ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:26.415 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.017+0000 7faa5783d700 1 -- 172.21.15.96:0/941453953 shutdown_connections 2024-04-25T14:04:26.415 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.017+0000 7faa5783d700 1 -- 172.21.15.96:0/941453953 wait complete. 2024-04-25T14:04:26.415 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.017+0000 7faa5783d700 1 Processor -- start 2024-04-25T14:04:26.416 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.017+0000 7faa5783d700 1 -- start start 2024-04-25T14:04:26.416 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.017+0000 7faa5783d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa50107c40 0x7faa5010a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:26.416 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.018+0000 7faa5783d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faa5010de70 con 0x7faa50107c40 2024-04-25T14:04:26.416 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.018+0000 7faa555d9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa50107c40 0x7faa5010a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:26.416 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.018+0000 7faa555d9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa50107c40 0x7faa5010a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53102/0 (socket says 172.21.15.96:53102) 2024-04-25T14:04:26.416 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.018+0000 7faa555d9700 1 -- 172.21.15.96:0/379447744 learned_addr learned my addr 172.21.15.96:0/379447744 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:04:26.416 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.018+0000 7faa555d9700 1 -- 172.21.15.96:0/379447744 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa5010a900 con 0x7faa50107c40 2024-04-25T14:04:26.416 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.018+0000 7faa555d9700 1 --2- 172.21.15.96:0/379447744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa50107c40 0x7faa5010a3c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7faa40011a20 tx=0x7faa40011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:26.416 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.018+0000 7faa467fc700 1 -- 172.21.15.96:0/379447744 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa40003af0 con 0x7faa50107c40 2024-04-25T14:04:26.416 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.018+0000 7faa467fc700 1 -- 172.21.15.96:0/379447744 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa4001b8f0 con 0x7faa50107c40 2024-04-25T14:04:26.416 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.018+0000 7faa467fc700 1 -- 172.21.15.96:0/379447744 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa40023890 con 0x7faa50107c40 2024-04-25T14:04:26.416 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.018+0000 7faa5783d700 1 -- 172.21.15.96:0/379447744 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa50108b50 con 0x7faa50107c40 2024-04-25T14:04:26.416 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.018+0000 7faa5783d700 1 -- 172.21.15.96:0/379447744 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa50109030 con 0x7faa50107c40 2024-04-25T14:04:26.416 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.019+0000 7faa5783d700 1 -- 172.21.15.96:0/379447744 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa50064cf0 con 0x7faa50107c40 2024-04-25T14:04:26.417 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.019+0000 7faa467fc700 1 -- 172.21.15.96:0/379447744 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 12) ==== 49529+0+0 (secure 0 0 0) 0x7faa4002c3f0 con 0x7faa50107c40 2024-04-25T14:04:26.417 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.020+0000 7faa467fc700 1 --2- 172.21.15.96:0/379447744 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7faa3c03d630 0x7faa3c03fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:26.417 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.020+0000 7faa467fc700 1 -- 172.21.15.96:0/379447744 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7faa40053130 con 0x7faa50107c40 2024-04-25T14:04:26.417 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.020+0000 7faa54dd8700 1 --2- 172.21.15.96:0/379447744 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7faa3c03d630 0x7faa3c03fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:26.417 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.020+0000 7faa54dd8700 1 --2- 172.21.15.96:0/379447744 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7faa3c03d630 0x7faa3c03fb40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7faa4c006fd0 tx=0x7faa4c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:26.417 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.025+0000 7faa467fc700 1 -- 172.21.15.96:0/379447744 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faa4001f030 con 0x7faa50107c40 2024-04-25T14:04:26.417 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.252+0000 7faa5783d700 1 -- 172.21.15.96:0/379447744 --> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7faa501095e0 con 0x7faa3c03d630 2024-04-25T14:04:26.417 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.477+0000 7faa467fc700 1 -- 172.21.15.96:0/379447744 <== mgr.14152 v2:172.21.15.96:6800/536332425 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7faa501095e0 con 0x7faa3c03d630 2024-04-25T14:04:26.417 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.479+0000 7faa5783d700 1 -- 172.21.15.96:0/379447744 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7faa3c03d630 msgr2=0x7faa3c03fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:26.417 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.479+0000 7faa5783d700 1 --2- 172.21.15.96:0/379447744 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7faa3c03d630 0x7faa3c03fb40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7faa4c006fd0 tx=0x7faa4c006e40 comp rx=0 tx=0).stop 2024-04-25T14:04:26.417 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.479+0000 7faa5783d700 1 -- 172.21.15.96:0/379447744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa50107c40 msgr2=0x7faa5010a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:26.417 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.479+0000 7faa5783d700 1 --2- 172.21.15.96:0/379447744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa50107c40 0x7faa5010a3c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7faa40011a20 tx=0x7faa40011b00 comp rx=0 tx=0).stop 2024-04-25T14:04:26.417 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.479+0000 7faa5783d700 1 -- 172.21.15.96:0/379447744 shutdown_connections 2024-04-25T14:04:26.418 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.479+0000 7faa5783d700 1 --2- 172.21.15.96:0/379447744 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7faa3c03d630 0x7faa3c03fb40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:26.418 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.479+0000 7faa5783d700 1 --2- 172.21.15.96:0/379447744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa50107c40 0x7faa5010a3c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:26.418 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.479+0000 7faa5783d700 1 -- 172.21.15.96:0/379447744 >> 172.21.15.96:0/379447744 conn(0x7faa50103670 msgr2=0x7faa50103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:26.418 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.479+0000 7faa5783d700 1 -- 172.21.15.96:0/379447744 shutdown_connections 2024-04-25T14:04:26.418 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:25.479+0000 7faa5783d700 1 -- 172.21.15.96:0/379447744 wait complete. 2024-04-25T14:04:26.418 INFO:teuthology.orchestra.run.smithi096.stdout:Fetching dashboard port number... 2024-04-25T14:04:26.671 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:26 smithi096 ceph-mon[29207]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:04:26.672 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:26 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:04:28.164 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 8443 2024-04-25T14:04:28.164 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.330+0000 7f1277e59700 1 Processor -- start 2024-04-25T14:04:28.164 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.330+0000 7f1277e59700 1 -- start start 2024-04-25T14:04:28.164 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.330+0000 7f1277e59700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12701088a0 0x7f1270108cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:28.164 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.330+0000 7f1277e59700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f12701092a0 con 0x7f12701088a0 2024-04-25T14:04:28.164 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.330+0000 7f1275bf5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12701088a0 0x7f1270108cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:28.164 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.330+0000 7f1275bf5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12701088a0 0x7f1270108cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53112/0 (socket says 172.21.15.96:53112) 2024-04-25T14:04:28.164 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.330+0000 7f1275bf5700 1 -- 172.21.15.96:0/1316132182 learned_addr learned my addr 172.21.15.96:0/1316132182 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:04:28.164 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.331+0000 7f1275bf5700 1 -- 172.21.15.96:0/1316132182 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1270109a70 con 0x7f12701088a0 2024-04-25T14:04:28.165 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.331+0000 7f1275bf5700 1 --2- 172.21.15.96:0/1316132182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12701088a0 0x7f1270108cb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f126c0098a0 tx=0x7f126c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d305101616b5514d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:28.165 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.331+0000 7f1274bf3700 1 -- 172.21.15.96:0/1316132182 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f126c004160 con 0x7f12701088a0 2024-04-25T14:04:28.165 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.331+0000 7f1274bf3700 1 -- 172.21.15.96:0/1316132182 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f126c004300 con 0x7f12701088a0 2024-04-25T14:04:28.165 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.331+0000 7f1274bf3700 1 -- 172.21.15.96:0/1316132182 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f126c011a20 con 0x7f12701088a0 2024-04-25T14:04:28.165 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.332+0000 7f1277e59700 1 -- 172.21.15.96:0/1316132182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12701088a0 msgr2=0x7f1270108cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:28.165 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.332+0000 7f1277e59700 1 --2- 172.21.15.96:0/1316132182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12701088a0 0x7f1270108cb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f126c0098a0 tx=0x7f126c009bb0 comp rx=0 tx=0).stop 2024-04-25T14:04:28.165 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.332+0000 7f1277e59700 1 -- 172.21.15.96:0/1316132182 shutdown_connections 2024-04-25T14:04:28.165 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.332+0000 7f1277e59700 1 --2- 172.21.15.96:0/1316132182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12701088a0 0x7f1270108cb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:28.165 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.332+0000 7f1277e59700 1 -- 172.21.15.96:0/1316132182 >> 172.21.15.96:0/1316132182 conn(0x7f1270103f60 msgr2=0x7f12701063b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:28.165 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.332+0000 7f1277e59700 1 -- 172.21.15.96:0/1316132182 shutdown_connections 2024-04-25T14:04:28.165 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.332+0000 7f1277e59700 1 -- 172.21.15.96:0/1316132182 wait complete. 2024-04-25T14:04:28.165 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.333+0000 7f1277e59700 1 Processor -- start 2024-04-25T14:04:28.165 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.333+0000 7f1277e59700 1 -- start start 2024-04-25T14:04:28.165 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.333+0000 7f1277e59700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12701088a0 0x7f127007d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:28.166 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.333+0000 7f1277e59700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f127010ce20 con 0x7f12701088a0 2024-04-25T14:04:28.166 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.333+0000 7f1275bf5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12701088a0 0x7f127007d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:28.166 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.333+0000 7f1275bf5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12701088a0 0x7f127007d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53118/0 (socket says 172.21.15.96:53118) 2024-04-25T14:04:28.166 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.333+0000 7f1275bf5700 1 -- 172.21.15.96:0/305210805 learned_addr learned my addr 172.21.15.96:0/305210805 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:04:28.166 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.333+0000 7f1275bf5700 1 -- 172.21.15.96:0/305210805 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f127007d700 con 0x7f12701088a0 2024-04-25T14:04:28.166 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.333+0000 7f1275bf5700 1 --2- 172.21.15.96:0/305210805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12701088a0 0x7f127007d1c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f126c003e70 tx=0x7f126c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:28.166 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.333+0000 7f1266ffd700 1 -- 172.21.15.96:0/305210805 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f126c003af0 con 0x7f12701088a0 2024-04-25T14:04:28.166 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.334+0000 7f1266ffd700 1 -- 172.21.15.96:0/305210805 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f126c01a8e0 con 0x7f12701088a0 2024-04-25T14:04:28.166 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.334+0000 7f1277e59700 1 -- 172.21.15.96:0/305210805 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12701a0a40 con 0x7f12701088a0 2024-04-25T14:04:28.166 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.334+0000 7f1266ffd700 1 -- 172.21.15.96:0/305210805 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f126c024870 con 0x7f12701088a0 2024-04-25T14:04:28.166 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.334+0000 7f1277e59700 1 -- 172.21.15.96:0/305210805 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12701a0f20 con 0x7f12701088a0 2024-04-25T14:04:28.166 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.334+0000 7f1277e59700 1 -- 172.21.15.96:0/305210805 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1270064cf0 con 0x7f12701088a0 2024-04-25T14:04:28.166 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.335+0000 7f1266ffd700 1 -- 172.21.15.96:0/305210805 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 12) ==== 49529+0+0 (secure 0 0 0) 0x7f126c022020 con 0x7f12701088a0 2024-04-25T14:04:28.167 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.335+0000 7f1266ffd700 1 --2- 172.21.15.96:0/305210805 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f125c03d690 0x7f125c03fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:28.167 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.335+0000 7f1266ffd700 1 -- 172.21.15.96:0/305210805 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f126c052840 con 0x7f12701088a0 2024-04-25T14:04:28.167 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.335+0000 7f12753f4700 1 --2- 172.21.15.96:0/305210805 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f125c03d690 0x7f125c03fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:28.167 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.335+0000 7f12753f4700 1 --2- 172.21.15.96:0/305210805 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f125c03d690 0x7f125c03fba0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1260006fd0 tx=0x7f1260006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:28.167 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.340+0000 7f1266ffd700 1 -- 172.21.15.96:0/305210805 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f126c00e260 con 0x7f12701088a0 2024-04-25T14:04:28.167 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.478+0000 7f1266ffd700 1 -- 172.21.15.96:0/305210805 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f126c02dc90 con 0x7f12701088a0 2024-04-25T14:04:28.167 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.541+0000 7f1277e59700 1 -- 172.21.15.96:0/305210805 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f12701a12d0 con 0x7f12701088a0 2024-04-25T14:04:28.167 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.542+0000 7f1266ffd700 1 -- 172.21.15.96:0/305210805 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f126c04a020 con 0x7f12701088a0 2024-04-25T14:04:28.167 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.544+0000 7f1277e59700 1 -- 172.21.15.96:0/305210805 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f125c03d690 msgr2=0x7f125c03fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:28.167 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.544+0000 7f1277e59700 1 --2- 172.21.15.96:0/305210805 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f125c03d690 0x7f125c03fba0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1260006fd0 tx=0x7f1260006e40 comp rx=0 tx=0).stop 2024-04-25T14:04:28.167 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.544+0000 7f1277e59700 1 -- 172.21.15.96:0/305210805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12701088a0 msgr2=0x7f127007d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:28.168 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.544+0000 7f1277e59700 1 --2- 172.21.15.96:0/305210805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12701088a0 0x7f127007d1c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f126c003e70 tx=0x7f126c011aa0 comp rx=0 tx=0).stop 2024-04-25T14:04:28.168 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.544+0000 7f1277e59700 1 -- 172.21.15.96:0/305210805 shutdown_connections 2024-04-25T14:04:28.168 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.544+0000 7f1277e59700 1 --2- 172.21.15.96:0/305210805 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f125c03d690 0x7f125c03fba0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:28.168 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.544+0000 7f1277e59700 1 --2- 172.21.15.96:0/305210805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12701088a0 0x7f127007d1c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:28.168 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.544+0000 7f1277e59700 1 -- 172.21.15.96:0/305210805 >> 172.21.15.96:0/305210805 conn(0x7f1270103f60 msgr2=0x7f1270105760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:28.168 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.544+0000 7f1277e59700 1 -- 172.21.15.96:0/305210805 shutdown_connections 2024-04-25T14:04:28.168 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:27.544+0000 7f1277e59700 1 -- 172.21.15.96:0/305210805 wait complete. 2024-04-25T14:04:28.171 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T14:04:28.171 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout disabled 2024-04-25T14:04:28.178 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T14:04:28.178 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout inactive 2024-04-25T14:04:28.178 INFO:teuthology.orchestra.run.smithi096.stdout:firewalld.service is not enabled 2024-04-25T14:04:28.178 INFO:teuthology.orchestra.run.smithi096.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-25T14:04:28.179 INFO:teuthology.orchestra.run.smithi096.stdout:Ceph Dashboard is now available at: 2024-04-25T14:04:28.179 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:04:28.179 INFO:teuthology.orchestra.run.smithi096.stdout: URL: https://smithi096:8443/ 2024-04-25T14:04:28.179 INFO:teuthology.orchestra.run.smithi096.stdout: User: admin 2024-04-25T14:04:28.180 INFO:teuthology.orchestra.run.smithi096.stdout: Password: jum9c65yma 2024-04-25T14:04:28.180 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:04:28.180 INFO:teuthology.orchestra.run.smithi096.stdout:Saving cluster configuration to /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/config directory 2024-04-25T14:04:28.180 INFO:teuthology.orchestra.run.smithi096.stdout:Enabling autotune for osd_memory_target 2024-04-25T14:04:28.811 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:28 smithi096 ceph-mon[29207]: mgrmap e13: x(active, since 6s) 2024-04-25T14:04:28.811 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:28 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/305210805' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-25T14:04:29.403 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.606+0000 7f198334d700 1 Processor -- start 2024-04-25T14:04:29.403 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.606+0000 7f198334d700 1 -- start start 2024-04-25T14:04:29.403 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.606+0000 7f198334d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f197c10a4c0 0x7f197c10c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:29.403 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.606+0000 7f198334d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f197c07d630 con 0x7f197c10a4c0 2024-04-25T14:04:29.404 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.607+0000 7f19810e9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f197c10a4c0 0x7f197c10c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:29.404 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.607+0000 7f19810e9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f197c10a4c0 0x7f197c10c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53124/0 (socket says 172.21.15.96:53124) 2024-04-25T14:04:29.404 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.607+0000 7f19810e9700 1 -- 172.21.15.96:0/3853728340 learned_addr learned my addr 172.21.15.96:0/3853728340 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:04:29.404 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.607+0000 7f19810e9700 1 -- 172.21.15.96:0/3853728340 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f197c10ce20 con 0x7f197c10a4c0 2024-04-25T14:04:29.404 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.607+0000 7f19810e9700 1 --2- 172.21.15.96:0/3853728340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f197c10a4c0 0x7f197c10c8e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1974009e80 tx=0x7f197400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=590f530095e8a198 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:29.404 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.607+0000 7f1973fff700 1 -- 172.21.15.96:0/3853728340 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1974017070 con 0x7f197c10a4c0 2024-04-25T14:04:29.404 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.607+0000 7f1973fff700 1 -- 172.21.15.96:0/3853728340 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f197400b8c0 con 0x7f197c10a4c0 2024-04-25T14:04:29.404 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.607+0000 7f1973fff700 1 -- 172.21.15.96:0/3853728340 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1974004490 con 0x7f197c10a4c0 2024-04-25T14:04:29.404 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.608+0000 7f198334d700 1 -- 172.21.15.96:0/3853728340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f197c10a4c0 msgr2=0x7f197c10c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:29.404 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.608+0000 7f198334d700 1 --2- 172.21.15.96:0/3853728340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f197c10a4c0 0x7f197c10c8e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1974009e80 tx=0x7f197400b330 comp rx=0 tx=0).stop 2024-04-25T14:04:29.404 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.608+0000 7f198334d700 1 -- 172.21.15.96:0/3853728340 shutdown_connections 2024-04-25T14:04:29.404 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.608+0000 7f198334d700 1 --2- 172.21.15.96:0/3853728340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f197c10a4c0 0x7f197c10c8e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:29.405 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.608+0000 7f198334d700 1 -- 172.21.15.96:0/3853728340 >> 172.21.15.96:0/3853728340 conn(0x7f197c103f00 msgr2=0x7f197c106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:29.405 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.608+0000 7f198334d700 1 -- 172.21.15.96:0/3853728340 shutdown_connections 2024-04-25T14:04:29.405 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.608+0000 7f198334d700 1 -- 172.21.15.96:0/3853728340 wait complete. 2024-04-25T14:04:29.405 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.609+0000 7f198334d700 1 Processor -- start 2024-04-25T14:04:29.405 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.609+0000 7f198334d700 1 -- start start 2024-04-25T14:04:29.405 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.609+0000 7f198334d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f197c10a4c0 0x7f197c109c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:29.405 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.609+0000 7f198334d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f197c10ddc0 con 0x7f197c10a4c0 2024-04-25T14:04:29.405 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.609+0000 7f19810e9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f197c10a4c0 0x7f197c109c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:29.405 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.609+0000 7f19810e9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f197c10a4c0 0x7f197c109c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53130/0 (socket says 172.21.15.96:53130) 2024-04-25T14:04:29.405 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.609+0000 7f19810e9700 1 -- 172.21.15.96:0/3048043909 learned_addr learned my addr 172.21.15.96:0/3048043909 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:04:29.405 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.609+0000 7f19810e9700 1 -- 172.21.15.96:0/3048043909 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f197c10a180 con 0x7f197c10a4c0 2024-04-25T14:04:29.405 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.609+0000 7f19810e9700 1 --2- 172.21.15.96:0/3048043909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f197c10a4c0 0x7f197c109c40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1974009e50 tx=0x7f197400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:29.405 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.609+0000 7f19727fc700 1 -- 172.21.15.96:0/3048043909 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1974020020 con 0x7f197c10a4c0 2024-04-25T14:04:29.405 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.610+0000 7f19727fc700 1 -- 172.21.15.96:0/3048043909 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1974012430 con 0x7f197c10a4c0 2024-04-25T14:04:29.405 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.610+0000 7f198334d700 1 -- 172.21.15.96:0/3048043909 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f197c1083d0 con 0x7f197c10a4c0 2024-04-25T14:04:29.406 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.610+0000 7f19727fc700 1 -- 172.21.15.96:0/3048043909 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1974017040 con 0x7f197c10a4c0 2024-04-25T14:04:29.406 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.610+0000 7f198334d700 1 -- 172.21.15.96:0/3048043909 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f197c108910 con 0x7f197c10a4c0 2024-04-25T14:04:29.406 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.610+0000 7f19727fc700 1 -- 172.21.15.96:0/3048043909 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f1974023020 con 0x7f197c10a4c0 2024-04-25T14:04:29.406 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.610+0000 7f198334d700 1 -- 172.21.15.96:0/3048043909 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f197c064cf0 con 0x7f197c10a4c0 2024-04-25T14:04:29.406 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.611+0000 7f19727fc700 1 --2- 172.21.15.96:0/3048043909 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f196803d6d0 0x7f196803fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:29.406 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.611+0000 7f19727fc700 1 -- 172.21.15.96:0/3048043909 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f197400f050 con 0x7f197c10a4c0 2024-04-25T14:04:29.406 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.611+0000 7f19808e8700 1 --2- 172.21.15.96:0/3048043909 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f196803d6d0 0x7f196803fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:29.406 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.611+0000 7f19808e8700 1 --2- 172.21.15.96:0/3048043909 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f196803d6d0 0x7f196803fbe0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f196c006fd0 tx=0x7f196c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:29.406 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.616+0000 7f19727fc700 1 -- 172.21.15.96:0/3048043909 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f197402a020 con 0x7f197c10a4c0 2024-04-25T14:04:29.406 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.811+0000 7f198334d700 1 -- 172.21.15.96:0/3048043909 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f197c1090a0 con 0x7f197c10a4c0 2024-04-25T14:04:29.406 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.812+0000 7f19727fc700 1 -- 172.21.15.96:0/3048043909 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f197402dd60 con 0x7f197c10a4c0 2024-04-25T14:04:29.406 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.814+0000 7f198334d700 1 -- 172.21.15.96:0/3048043909 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f196803d6d0 msgr2=0x7f196803fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:29.406 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.814+0000 7f198334d700 1 --2- 172.21.15.96:0/3048043909 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f196803d6d0 0x7f196803fbe0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f196c006fd0 tx=0x7f196c006e40 comp rx=0 tx=0).stop 2024-04-25T14:04:29.407 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.814+0000 7f198334d700 1 -- 172.21.15.96:0/3048043909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f197c10a4c0 msgr2=0x7f197c109c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:29.407 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.814+0000 7f198334d700 1 --2- 172.21.15.96:0/3048043909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f197c10a4c0 0x7f197c109c40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1974009e50 tx=0x7f197400bd50 comp rx=0 tx=0).stop 2024-04-25T14:04:29.407 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.814+0000 7f198334d700 1 -- 172.21.15.96:0/3048043909 shutdown_connections 2024-04-25T14:04:29.407 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.814+0000 7f198334d700 1 --2- 172.21.15.96:0/3048043909 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f196803d6d0 0x7f196803fbe0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:29.407 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.814+0000 7f198334d700 1 --2- 172.21.15.96:0/3048043909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f197c10a4c0 0x7f197c109c40 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:29.407 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.814+0000 7f198334d700 1 -- 172.21.15.96:0/3048043909 >> 172.21.15.96:0/3048043909 conn(0x7f197c103f00 msgr2=0x7f197c1046c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:29.407 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.814+0000 7f198334d700 1 -- 172.21.15.96:0/3048043909 shutdown_connections 2024-04-25T14:04:29.407 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:28.814+0000 7f198334d700 1 -- 172.21.15.96:0/3048043909 wait complete. 2024-04-25T14:04:30.742 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:29.828+0000 7f34bb00b700 1 Processor -- start 2024-04-25T14:04:30.743 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:29.828+0000 7f34bb00b700 1 -- start start 2024-04-25T14:04:30.743 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:29.828+0000 7f34bb00b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34b4108310 0x7f34b4108720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:30.743 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:29.828+0000 7f34bb00b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f34b407d610 con 0x7f34b4108310 2024-04-25T14:04:30.743 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:29.828+0000 7f34b8da7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34b4108310 0x7f34b4108720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:30.743 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:29.828+0000 7f34b8da7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34b4108310 0x7f34b4108720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53138/0 (socket says 172.21.15.96:53138) 2024-04-25T14:04:30.743 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:29.828+0000 7f34b8da7700 1 -- 172.21.15.96:0/1271689330 learned_addr learned my addr 172.21.15.96:0/1271689330 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:04:30.743 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:29.829+0000 7f34b8da7700 1 -- 172.21.15.96:0/1271689330 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b4108c60 con 0x7f34b4108310 2024-04-25T14:04:30.743 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:29.829+0000 7f34b8da7700 1 --2- 172.21.15.96:0/1271689330 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34b4108310 0x7f34b4108720 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f34a80098a0 tx=0x7f34a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79f267554af4648c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:30.743 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:29.829+0000 7f34b37fe700 1 -- 172.21.15.96:0/1271689330 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f34a8004160 con 0x7f34b4108310 2024-04-25T14:04:30.743 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:29.829+0000 7f34b37fe700 1 -- 172.21.15.96:0/1271689330 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34a8004300 con 0x7f34b4108310 2024-04-25T14:04:30.743 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:29.829+0000 7f34b37fe700 1 -- 172.21.15.96:0/1271689330 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f34a8011a20 con 0x7f34b4108310 2024-04-25T14:04:30.743 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:29.830+0000 7f34bb00b700 1 -- 172.21.15.96:0/1271689330 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34b4108310 msgr2=0x7f34b4108720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:30.743 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:29.830+0000 7f34bb00b700 1 --2- 172.21.15.96:0/1271689330 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34b4108310 0x7f34b4108720 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f34a80098a0 tx=0x7f34a8009bb0 comp rx=0 tx=0).stop 2024-04-25T14:04:30.743 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:29.830+0000 7f34bb00b700 1 -- 172.21.15.96:0/1271689330 shutdown_connections 2024-04-25T14:04:30.744 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:29.830+0000 7f34bb00b700 1 --2- 172.21.15.96:0/1271689330 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34b4108310 0x7f34b4108720 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:30.744 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:29.830+0000 7f34bb00b700 1 -- 172.21.15.96:0/1271689330 >> 172.21.15.96:0/1271689330 conn(0x7f34b4103fe0 msgr2=0x7f34b4106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:30.744 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:29.830+0000 7f34bb00b700 1 -- 172.21.15.96:0/1271689330 shutdown_connections 2024-04-25T14:04:30.744 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:29.830+0000 7f34bb00b700 1 -- 172.21.15.96:0/1271689330 wait complete. 2024-04-25T14:04:30.744 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:29.830+0000 7f34bb00b700 1 Processor -- start 2024-04-25T14:04:30.744 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:29.830+0000 7f34bb00b700 1 -- start start 2024-04-25T14:04:30.744 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:29.830+0000 7f34bb00b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34b4108310 0x7f34b419fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:30.744 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:29.830+0000 7f34bb00b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f34b4109d20 con 0x7f34b4108310 2024-04-25T14:04:30.744 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:29.831+0000 7f34b8da7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34b4108310 0x7f34b419fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:30.744 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:29.831+0000 7f34b8da7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34b4108310 0x7f34b419fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53150/0 (socket says 172.21.15.96:53150) 2024-04-25T14:04:30.744 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:29.831+0000 7f34b8da7700 1 -- 172.21.15.96:0/2991453326 learned_addr learned my addr 172.21.15.96:0/2991453326 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:04:30.744 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:29.831+0000 7f34b8da7700 1 -- 172.21.15.96:0/2991453326 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b41a0400 con 0x7f34b4108310 2024-04-25T14:04:30.744 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:29.831+0000 7f34b8da7700 1 --2- 172.21.15.96:0/2991453326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34b4108310 0x7f34b419fec0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f34a8000c00 tx=0x7f34a8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:30.744 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:29.831+0000 7f34b1ffb700 1 -- 172.21.15.96:0/2991453326 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f34a8003aa0 con 0x7f34b4108310 2024-04-25T14:04:30.745 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:29.831+0000 7f34bb00b700 1 -- 172.21.15.96:0/2991453326 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34b41a0690 con 0x7f34b4108310 2024-04-25T14:04:30.745 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:29.831+0000 7f34b1ffb700 1 -- 172.21.15.96:0/2991453326 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34a801b8f0 con 0x7f34b4108310 2024-04-25T14:04:30.745 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:29.831+0000 7f34b1ffb700 1 -- 172.21.15.96:0/2991453326 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f34a8023890 con 0x7f34b4108310 2024-04-25T14:04:30.745 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:29.831+0000 7f34bb00b700 1 -- 172.21.15.96:0/2991453326 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34b41a32a0 con 0x7f34b4108310 2024-04-25T14:04:30.745 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:29.832+0000 7f34bb00b700 1 -- 172.21.15.96:0/2991453326 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34b4064cf0 con 0x7f34b4108310 2024-04-25T14:04:30.745 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:29.832+0000 7f34b1ffb700 1 -- 172.21.15.96:0/2991453326 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f34a802c3f0 con 0x7f34b4108310 2024-04-25T14:04:30.745 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:29.832+0000 7f34b1ffb700 1 --2- 172.21.15.96:0/2991453326 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f34a403d720 0x7f34a403fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:30.745 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:29.833+0000 7f34b1ffb700 1 -- 172.21.15.96:0/2991453326 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f34a8052df0 con 0x7f34b4108310 2024-04-25T14:04:30.745 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:29.833+0000 7f34b3fff700 1 --2- 172.21.15.96:0/2991453326 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f34a403d720 0x7f34a403fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:30.745 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:29.833+0000 7f34b3fff700 1 --2- 172.21.15.96:0/2991453326 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f34a403d720 0x7f34a403fc30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f34a0006fd0 tx=0x7f34a0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:30.745 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:29.838+0000 7f34b1ffb700 1 -- 172.21.15.96:0/2991453326 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f34a801f030 con 0x7f34b4108310 2024-04-25T14:04:30.745 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:30.146+0000 7f34bb00b700 1 -- 172.21.15.96:0/2991453326 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f34b402e810 con 0x7f34b4108310 2024-04-25T14:04:30.745 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:30.149+0000 7f34b1ffb700 1 -- 172.21.15.96:0/2991453326 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f34a8016df0 con 0x7f34b4108310 2024-04-25T14:04:30.745 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:30.151+0000 7f34bb00b700 1 -- 172.21.15.96:0/2991453326 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f34a403d720 msgr2=0x7f34a403fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:30.745 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:30.151+0000 7f34bb00b700 1 --2- 172.21.15.96:0/2991453326 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f34a403d720 0x7f34a403fc30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f34a0006fd0 tx=0x7f34a0006e40 comp rx=0 tx=0).stop 2024-04-25T14:04:30.746 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:30.151+0000 7f34bb00b700 1 -- 172.21.15.96:0/2991453326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34b4108310 msgr2=0x7f34b419fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:30.746 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:30.151+0000 7f34bb00b700 1 --2- 172.21.15.96:0/2991453326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34b4108310 0x7f34b419fec0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f34a8000c00 tx=0x7f34a8011aa0 comp rx=0 tx=0).stop 2024-04-25T14:04:30.746 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:30.151+0000 7f34bb00b700 1 -- 172.21.15.96:0/2991453326 shutdown_connections 2024-04-25T14:04:30.746 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:30.151+0000 7f34bb00b700 1 --2- 172.21.15.96:0/2991453326 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f34a403d720 0x7f34a403fc30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:30.746 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:30.151+0000 7f34bb00b700 1 --2- 172.21.15.96:0/2991453326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34b4108310 0x7f34b419fec0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:30.746 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:30.151+0000 7f34bb00b700 1 -- 172.21.15.96:0/2991453326 >> 172.21.15.96:0/2991453326 conn(0x7f34b4103fe0 msgr2=0x7f34b4193e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:30.746 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:30.151+0000 7f34bb00b700 1 -- 172.21.15.96:0/2991453326 shutdown_connections 2024-04-25T14:04:30.746 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T14:04:30.151+0000 7f34bb00b700 1 -- 172.21.15.96:0/2991453326 wait complete. 2024-04-25T14:04:30.746 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-25T14:04:30.746 INFO:teuthology.orchestra.run.smithi096.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-25T14:04:30.746 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:04:30.746 INFO:teuthology.orchestra.run.smithi096.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 59334182-030c-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-25T14:04:30.746 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:04:30.746 INFO:teuthology.orchestra.run.smithi096.stdout:Or, if you are only running a single cluster on this host: 2024-04-25T14:04:30.746 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:04:30.747 INFO:teuthology.orchestra.run.smithi096.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-25T14:04:30.747 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:04:30.747 INFO:teuthology.orchestra.run.smithi096.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-25T14:04:30.747 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:04:30.747 INFO:teuthology.orchestra.run.smithi096.stdout: ceph telemetry on 2024-04-25T14:04:30.747 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:04:30.747 INFO:teuthology.orchestra.run.smithi096.stdout:For more information see: 2024-04-25T14:04:30.747 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:04:30.747 INFO:teuthology.orchestra.run.smithi096.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-25T14:04:30.747 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:04:30.747 INFO:teuthology.orchestra.run.smithi096.stdout:Bootstrap complete. 2024-04-25T14:04:30.777 INFO:tasks.cephadm:Fetching config... 2024-04-25T14:04:30.778 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T14:04:30.778 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-25T14:04:30.793 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-25T14:04:30.793 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T14:04:30.793 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-25T14:04:30.848 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-25T14:04:30.848 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T14:04:30.848 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-25T14:04:30.912 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-25T14:04:30.912 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T14:04:30.912 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-25T14:04:30.967 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-25T14:04:30.967 DEBUG:teuthology.orchestra.run.smithi096:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDoqiYS3qctb73u6iSODcS9ylsh58ZGbXlGKnerDCuVOT1IINo8UUyVOv2+vTycoBZpsN233QmYjfjMerHSYnqLd6CTCezZzHoL78p8VNFQQ+nwy5WSakeFUKndNJnfQRsxe+FVDXROZrDNrKX92AAi6XaiKUjgUZlveBiHGazKcJhCx021RtEDQPKIMYDcwAo8LDtRPg/73Qh3rcGoAIpCXhilD34kvqAseQTqmeAnm+fqqgAcoQSGFu1UtdPS6qwxZiNwu0au3r06iLrqMTZptqG8wIqDaUdpnLz64pDgCG9tO+jkHuoeF5Wk39O1Vnh+fdokf25CeATWWyIlQQJXr1uwHWcr3qPM4aWgsGicf9shJBH+MQT10bRkdbyoFZ84IBpyBti6D3kfs6fV8H7V5RUL2/wWBw0Z8tLy8asn61rPjubnf0bpkn8Mq9GLX/Y/iS6emFCGfqU9CdJHSdjblT1/cmCVsQIiMWp0S+DLwjonOtxM2nPSfjpmvOoh1vc= ceph-59334182-030c-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T14:04:31.039 INFO:teuthology.orchestra.run.smithi096.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDoqiYS3qctb73u6iSODcS9ylsh58ZGbXlGKnerDCuVOT1IINo8UUyVOv2+vTycoBZpsN233QmYjfjMerHSYnqLd6CTCezZzHoL78p8VNFQQ+nwy5WSakeFUKndNJnfQRsxe+FVDXROZrDNrKX92AAi6XaiKUjgUZlveBiHGazKcJhCx021RtEDQPKIMYDcwAo8LDtRPg/73Qh3rcGoAIpCXhilD34kvqAseQTqmeAnm+fqqgAcoQSGFu1UtdPS6qwxZiNwu0au3r06iLrqMTZptqG8wIqDaUdpnLz64pDgCG9tO+jkHuoeF5Wk39O1Vnh+fdokf25CeATWWyIlQQJXr1uwHWcr3qPM4aWgsGicf9shJBH+MQT10bRkdbyoFZ84IBpyBti6D3kfs6fV8H7V5RUL2/wWBw0Z8tLy8asn61rPjubnf0bpkn8Mq9GLX/Y/iS6emFCGfqU9CdJHSdjblT1/cmCVsQIiMWp0S+DLwjonOtxM2nPSfjpmvOoh1vc= ceph-59334182-030c-11ef-bc93-c7b262605968 2024-04-25T14:04:31.051 DEBUG:teuthology.orchestra.run.smithi097:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDoqiYS3qctb73u6iSODcS9ylsh58ZGbXlGKnerDCuVOT1IINo8UUyVOv2+vTycoBZpsN233QmYjfjMerHSYnqLd6CTCezZzHoL78p8VNFQQ+nwy5WSakeFUKndNJnfQRsxe+FVDXROZrDNrKX92AAi6XaiKUjgUZlveBiHGazKcJhCx021RtEDQPKIMYDcwAo8LDtRPg/73Qh3rcGoAIpCXhilD34kvqAseQTqmeAnm+fqqgAcoQSGFu1UtdPS6qwxZiNwu0au3r06iLrqMTZptqG8wIqDaUdpnLz64pDgCG9tO+jkHuoeF5Wk39O1Vnh+fdokf25CeATWWyIlQQJXr1uwHWcr3qPM4aWgsGicf9shJBH+MQT10bRkdbyoFZ84IBpyBti6D3kfs6fV8H7V5RUL2/wWBw0Z8tLy8asn61rPjubnf0bpkn8Mq9GLX/Y/iS6emFCGfqU9CdJHSdjblT1/cmCVsQIiMWp0S+DLwjonOtxM2nPSfjpmvOoh1vc= ceph-59334182-030c-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T14:04:31.089 INFO:teuthology.orchestra.run.smithi097.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDoqiYS3qctb73u6iSODcS9ylsh58ZGbXlGKnerDCuVOT1IINo8UUyVOv2+vTycoBZpsN233QmYjfjMerHSYnqLd6CTCezZzHoL78p8VNFQQ+nwy5WSakeFUKndNJnfQRsxe+FVDXROZrDNrKX92AAi6XaiKUjgUZlveBiHGazKcJhCx021RtEDQPKIMYDcwAo8LDtRPg/73Qh3rcGoAIpCXhilD34kvqAseQTqmeAnm+fqqgAcoQSGFu1UtdPS6qwxZiNwu0au3r06iLrqMTZptqG8wIqDaUdpnLz64pDgCG9tO+jkHuoeF5Wk39O1Vnh+fdokf25CeATWWyIlQQJXr1uwHWcr3qPM4aWgsGicf9shJBH+MQT10bRkdbyoFZ84IBpyBti6D3kfs6fV8H7V5RUL2/wWBw0Z8tLy8asn61rPjubnf0bpkn8Mq9GLX/Y/iS6emFCGfqU9CdJHSdjblT1/cmCVsQIiMWp0S+DLwjonOtxM2nPSfjpmvOoh1vc= ceph-59334182-030c-11ef-bc93-c7b262605968 2024-04-25T14:04:31.101 DEBUG:teuthology.orchestra.run.smithi151:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDoqiYS3qctb73u6iSODcS9ylsh58ZGbXlGKnerDCuVOT1IINo8UUyVOv2+vTycoBZpsN233QmYjfjMerHSYnqLd6CTCezZzHoL78p8VNFQQ+nwy5WSakeFUKndNJnfQRsxe+FVDXROZrDNrKX92AAi6XaiKUjgUZlveBiHGazKcJhCx021RtEDQPKIMYDcwAo8LDtRPg/73Qh3rcGoAIpCXhilD34kvqAseQTqmeAnm+fqqgAcoQSGFu1UtdPS6qwxZiNwu0au3r06iLrqMTZptqG8wIqDaUdpnLz64pDgCG9tO+jkHuoeF5Wk39O1Vnh+fdokf25CeATWWyIlQQJXr1uwHWcr3qPM4aWgsGicf9shJBH+MQT10bRkdbyoFZ84IBpyBti6D3kfs6fV8H7V5RUL2/wWBw0Z8tLy8asn61rPjubnf0bpkn8Mq9GLX/Y/iS6emFCGfqU9CdJHSdjblT1/cmCVsQIiMWp0S+DLwjonOtxM2nPSfjpmvOoh1vc= ceph-59334182-030c-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T14:04:31.140 INFO:teuthology.orchestra.run.smithi151.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDoqiYS3qctb73u6iSODcS9ylsh58ZGbXlGKnerDCuVOT1IINo8UUyVOv2+vTycoBZpsN233QmYjfjMerHSYnqLd6CTCezZzHoL78p8VNFQQ+nwy5WSakeFUKndNJnfQRsxe+FVDXROZrDNrKX92AAi6XaiKUjgUZlveBiHGazKcJhCx021RtEDQPKIMYDcwAo8LDtRPg/73Qh3rcGoAIpCXhilD34kvqAseQTqmeAnm+fqqgAcoQSGFu1UtdPS6qwxZiNwu0au3r06iLrqMTZptqG8wIqDaUdpnLz64pDgCG9tO+jkHuoeF5Wk39O1Vnh+fdokf25CeATWWyIlQQJXr1uwHWcr3qPM4aWgsGicf9shJBH+MQT10bRkdbyoFZ84IBpyBti6D3kfs6fV8H7V5RUL2/wWBw0Z8tLy8asn61rPjubnf0bpkn8Mq9GLX/Y/iS6emFCGfqU9CdJHSdjblT1/cmCVsQIiMWp0S+DLwjonOtxM2nPSfjpmvOoh1vc= ceph-59334182-030c-11ef-bc93-c7b262605968 2024-04-25T14:04:31.152 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-25T14:04:31.173 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:31 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/2991453326' entity='client.admin' 2024-04-25T14:04:31.389 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:04:34.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:34.132+0000 7fa825bc7700 1 -- 172.21.15.96:0/1763115825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa820103a90 msgr2=0x7fa820103ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:34.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:34.132+0000 7fa825bc7700 1 --2- 172.21.15.96:0/1763115825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa820103a90 0x7fa820103ea0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa810009910 tx=0x7fa810009c20 comp rx=0 tx=0).stop 2024-04-25T14:04:34.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:34.132+0000 7fa825bc7700 1 -- 172.21.15.96:0/1763115825 shutdown_connections 2024-04-25T14:04:34.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:34.132+0000 7fa825bc7700 1 --2- 172.21.15.96:0/1763115825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa820103a90 0x7fa820103ea0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:34.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:34.132+0000 7fa825bc7700 1 -- 172.21.15.96:0/1763115825 >> 172.21.15.96:0/1763115825 conn(0x7fa8200ff0d0 msgr2=0x7fa820101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:34.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:34.132+0000 7fa825bc7700 1 -- 172.21.15.96:0/1763115825 shutdown_connections 2024-04-25T14:04:34.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:34.132+0000 7fa825bc7700 1 -- 172.21.15.96:0/1763115825 wait complete. 2024-04-25T14:04:34.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:34.133+0000 7fa825bc7700 1 Processor -- start 2024-04-25T14:04:34.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:34.133+0000 7fa825bc7700 1 -- start start 2024-04-25T14:04:34.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:34.133+0000 7fa825bc7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa820103a90 0x7fa820077250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:34.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:34.133+0000 7fa825bc7700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa820109a50 con 0x7fa820103a90 2024-04-25T14:04:34.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:34.133+0000 7fa81f7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa820103a90 0x7fa820077250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:34.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:34.133+0000 7fa81f7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa820103a90 0x7fa820077250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42924/0 (socket says 172.21.15.96:42924) 2024-04-25T14:04:34.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:34.133+0000 7fa81f7fe700 1 -- 172.21.15.96:0/3232126477 learned_addr learned my addr 172.21.15.96:0/3232126477 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:04:34.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:34.134+0000 7fa81f7fe700 1 -- 172.21.15.96:0/3232126477 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa820077790 con 0x7fa820103a90 2024-04-25T14:04:34.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:34.134+0000 7fa81f7fe700 1 --2- 172.21.15.96:0/3232126477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa820103a90 0x7fa820077250 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa810006010 tx=0x7fa810004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:34.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:34.134+0000 7fa81cff9700 1 -- 172.21.15.96:0/3232126477 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa810026070 con 0x7fa820103a90 2024-04-25T14:04:34.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:34.134+0000 7fa81cff9700 1 -- 172.21.15.96:0/3232126477 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa81000bc00 con 0x7fa820103a90 2024-04-25T14:04:34.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:34.134+0000 7fa825bc7700 1 -- 172.21.15.96:0/3232126477 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8200759e0 con 0x7fa820103a90 2024-04-25T14:04:34.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:34.134+0000 7fa81cff9700 1 -- 172.21.15.96:0/3232126477 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa810003970 con 0x7fa820103a90 2024-04-25T14:04:34.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:34.134+0000 7fa825bc7700 1 -- 172.21.15.96:0/3232126477 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa820075ec0 con 0x7fa820103a90 2024-04-25T14:04:34.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:34.135+0000 7fa825bc7700 1 -- 172.21.15.96:0/3232126477 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa82010a1d0 con 0x7fa820103a90 2024-04-25T14:04:34.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:34.135+0000 7fa81cff9700 1 -- 172.21.15.96:0/3232126477 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7fa810029070 con 0x7fa820103a90 2024-04-25T14:04:34.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:34.135+0000 7fa81cff9700 1 --2- 172.21.15.96:0/3232126477 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa80803d670 0x7fa80803fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:34.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:34.136+0000 7fa81cff9700 1 -- 172.21.15.96:0/3232126477 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa81001e070 con 0x7fa820103a90 2024-04-25T14:04:34.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:34.136+0000 7fa81effd700 1 --2- 172.21.15.96:0/3232126477 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa80803d670 0x7fa80803fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:34.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:34.136+0000 7fa81effd700 1 --2- 172.21.15.96:0/3232126477 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa80803d670 0x7fa80803fb80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa814006fd0 tx=0x7fa814006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:34.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:34.141+0000 7fa81cff9700 1 -- 172.21.15.96:0/3232126477 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa810013070 con 0x7fa820103a90 2024-04-25T14:04:34.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:34.364+0000 7fa825bc7700 1 -- 172.21.15.96:0/3232126477 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fa8201923f0 con 0x7fa820103a90 2024-04-25T14:04:34.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:34.367+0000 7fa81cff9700 1 -- 172.21.15.96:0/3232126477 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fa8100300c0 con 0x7fa820103a90 2024-04-25T14:04:34.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:34.369+0000 7fa825bc7700 1 -- 172.21.15.96:0/3232126477 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa80803d670 msgr2=0x7fa80803fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:34.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:34.369+0000 7fa825bc7700 1 --2- 172.21.15.96:0/3232126477 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa80803d670 0x7fa80803fb80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa814006fd0 tx=0x7fa814006e40 comp rx=0 tx=0).stop 2024-04-25T14:04:34.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:34.369+0000 7fa825bc7700 1 -- 172.21.15.96:0/3232126477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa820103a90 msgr2=0x7fa820077250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:34.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:34.369+0000 7fa825bc7700 1 --2- 172.21.15.96:0/3232126477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa820103a90 0x7fa820077250 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa810006010 tx=0x7fa810004fa0 comp rx=0 tx=0).stop 2024-04-25T14:04:34.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:34.369+0000 7fa825bc7700 1 -- 172.21.15.96:0/3232126477 shutdown_connections 2024-04-25T14:04:34.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:34.369+0000 7fa825bc7700 1 --2- 172.21.15.96:0/3232126477 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa80803d670 0x7fa80803fb80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:34.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:34.369+0000 7fa825bc7700 1 --2- 172.21.15.96:0/3232126477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa820103a90 0x7fa820077250 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:34.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:34.369+0000 7fa825bc7700 1 -- 172.21.15.96:0/3232126477 >> 172.21.15.96:0/3232126477 conn(0x7fa8200ff0d0 msgr2=0x7fa8200ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:34.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:34.369+0000 7fa825bc7700 1 -- 172.21.15.96:0/3232126477 shutdown_connections 2024-04-25T14:04:34.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:34.369+0000 7fa825bc7700 1 -- 172.21.15.96:0/3232126477 wait complete. 2024-04-25T14:04:35.538 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-25T14:04:35.539 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-25T14:04:35.710 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:35 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/3232126477' entity='client.admin' 2024-04-25T14:04:35.711 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:35 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:04:35.711 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:35 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:04:35.711 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:35 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi096", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:04:35.711 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:35 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:04:35.711 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:35 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:04:35.711 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:35 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:04:35.813 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:04:37.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:37.387+0000 7fdf164a7700 1 -- 172.21.15.96:0/1155655132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf10104640 msgr2=0x7fdf10106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:37.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:37.387+0000 7fdf164a7700 1 --2- 172.21.15.96:0/1155655132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf10104640 0x7fdf10106a60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdf00009910 tx=0x7fdf00009c20 comp rx=0 tx=0).stop 2024-04-25T14:04:37.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:37.387+0000 7fdf164a7700 1 -- 172.21.15.96:0/1155655132 shutdown_connections 2024-04-25T14:04:37.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:37.387+0000 7fdf164a7700 1 --2- 172.21.15.96:0/1155655132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf10104640 0x7fdf10106a60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:37.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:37.387+0000 7fdf164a7700 1 -- 172.21.15.96:0/1155655132 >> 172.21.15.96:0/1155655132 conn(0x7fdf100fdfe0 msgr2=0x7fdf101004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:37.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:37.387+0000 7fdf164a7700 1 -- 172.21.15.96:0/1155655132 shutdown_connections 2024-04-25T14:04:37.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:37.387+0000 7fdf164a7700 1 -- 172.21.15.96:0/1155655132 wait complete. 2024-04-25T14:04:37.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:37.388+0000 7fdf164a7700 1 Processor -- start 2024-04-25T14:04:37.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:37.388+0000 7fdf164a7700 1 -- start start 2024-04-25T14:04:37.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:37.388+0000 7fdf164a7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf10104640 0x7fdf101995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:37.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:37.388+0000 7fdf164a7700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdf10109a80 con 0x7fdf10104640 2024-04-25T14:04:37.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:37.388+0000 7fdf0ffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf10104640 0x7fdf101995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:37.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:37.388+0000 7fdf0ffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf10104640 0x7fdf101995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42942/0 (socket says 172.21.15.96:42942) 2024-04-25T14:04:37.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:37.388+0000 7fdf0ffff700 1 -- 172.21.15.96:0/605441406 learned_addr learned my addr 172.21.15.96:0/605441406 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:04:37.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:37.389+0000 7fdf0ffff700 1 -- 172.21.15.96:0/605441406 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf10199af0 con 0x7fdf10104640 2024-04-25T14:04:37.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:37.389+0000 7fdf0ffff700 1 --2- 172.21.15.96:0/605441406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf10104640 0x7fdf101995b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdf000050d0 tx=0x7fdf00009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:37.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:37.389+0000 7fdf0d7fa700 1 -- 172.21.15.96:0/605441406 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf00027070 con 0x7fdf10104640 2024-04-25T14:04:37.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:37.389+0000 7fdf0d7fa700 1 -- 172.21.15.96:0/605441406 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf0000be10 con 0x7fdf10104640 2024-04-25T14:04:37.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:37.389+0000 7fdf0d7fa700 1 -- 172.21.15.96:0/605441406 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf00022d80 con 0x7fdf10104640 2024-04-25T14:04:37.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:37.389+0000 7fdf164a7700 1 -- 172.21.15.96:0/605441406 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf10199d80 con 0x7fdf10104640 2024-04-25T14:04:37.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:37.389+0000 7fdf164a7700 1 -- 172.21.15.96:0/605441406 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf1019c990 con 0x7fdf10104640 2024-04-25T14:04:37.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:37.390+0000 7fdf0d7fa700 1 -- 172.21.15.96:0/605441406 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7fdf0002a070 con 0x7fdf10104640 2024-04-25T14:04:37.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:37.390+0000 7fdf164a7700 1 -- 172.21.15.96:0/605441406 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf1010a200 con 0x7fdf10104640 2024-04-25T14:04:37.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:37.390+0000 7fdf0d7fa700 1 --2- 172.21.15.96:0/605441406 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fdef803d5c0 0x7fdef803fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:37.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:37.390+0000 7fdf0d7fa700 1 -- 172.21.15.96:0/605441406 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdf0001e030 con 0x7fdf10104640 2024-04-25T14:04:37.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:37.390+0000 7fdf0f7fe700 1 --2- 172.21.15.96:0/605441406 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fdef803d5c0 0x7fdef803fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:37.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:37.391+0000 7fdf0f7fe700 1 --2- 172.21.15.96:0/605441406 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fdef803d5c0 0x7fdef803fad0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fdf04006fd0 tx=0x7fdf04006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:37.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:37.395+0000 7fdf0d7fa700 1 -- 172.21.15.96:0/605441406 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdf00013070 con 0x7fdf10104640 2024-04-25T14:04:37.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:37.596+0000 7fdf164a7700 1 -- 172.21.15.96:0/605441406 --> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fdf1002e620 con 0x7fdef803d5c0 2024-04-25T14:04:37.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:37.600+0000 7fdf0d7fa700 1 -- 172.21.15.96:0/605441406 <== mgr.14152 v2:172.21.15.96:6800/536332425 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdf1002e620 con 0x7fdef803d5c0 2024-04-25T14:04:37.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:37.602+0000 7fdf164a7700 1 -- 172.21.15.96:0/605441406 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fdef803d5c0 msgr2=0x7fdef803fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:37.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:37.602+0000 7fdf164a7700 1 --2- 172.21.15.96:0/605441406 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fdef803d5c0 0x7fdef803fad0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fdf04006fd0 tx=0x7fdf04006e40 comp rx=0 tx=0).stop 2024-04-25T14:04:37.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:37.602+0000 7fdf164a7700 1 -- 172.21.15.96:0/605441406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf10104640 msgr2=0x7fdf101995b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:37.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:37.602+0000 7fdf164a7700 1 --2- 172.21.15.96:0/605441406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf10104640 0x7fdf101995b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdf000050d0 tx=0x7fdf00009f80 comp rx=0 tx=0).stop 2024-04-25T14:04:37.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:37.602+0000 7fdf164a7700 1 -- 172.21.15.96:0/605441406 shutdown_connections 2024-04-25T14:04:37.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:37.602+0000 7fdf164a7700 1 --2- 172.21.15.96:0/605441406 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fdef803d5c0 0x7fdef803fad0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:37.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:37.602+0000 7fdf164a7700 1 --2- 172.21.15.96:0/605441406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf10104640 0x7fdf101995b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:37.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:37.602+0000 7fdf164a7700 1 -- 172.21.15.96:0/605441406 >> 172.21.15.96:0/605441406 conn(0x7fdf100fdfe0 msgr2=0x7fdf100fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:37.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:37.602+0000 7fdf164a7700 1 -- 172.21.15.96:0/605441406 shutdown_connections 2024-04-25T14:04:37.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:37.602+0000 7fdf164a7700 1 -- 172.21.15.96:0/605441406 wait complete. 2024-04-25T14:04:38.356 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi097 2024-04-25T14:04:38.356 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T14:04:38.356 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/etc/ceph/ceph.conf 2024-04-25T14:04:38.375 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T14:04:38.375 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:04:38.433 INFO:tasks.cephadm:Adding host smithi097 to orchestrator... 2024-04-25T14:04:38.433 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph orch host add smithi097 2024-04-25T14:04:38.681 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:04:38.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:38 smithi096 ceph-mon[29207]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:04:38.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:38 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:04:38.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:38 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:04:38.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:38 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:04:38.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:38 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:04:38.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:38 smithi096 ceph-mon[29207]: Updating smithi096:/etc/ceph/ceph.conf 2024-04-25T14:04:38.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:38 smithi096 ceph-mon[29207]: Updating smithi096:/var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:04:38.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:38 smithi096 ceph-mon[29207]: Updating smithi096:/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:04:38.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:38 smithi096 ceph-mon[29207]: Updating smithi096:/var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T14:04:38.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:38 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:04:38.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:38 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:04:38.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:38 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:04:40.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:40.272+0000 7f9b2891a700 1 -- 172.21.15.96:0/1606075983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b24109a40 msgr2=0x7f9b24109e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:40.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:40.272+0000 7f9b2891a700 1 --2- 172.21.15.96:0/1606075983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b24109a40 0x7f9b24109e50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9b14009910 tx=0x7f9b14009c20 comp rx=0 tx=0).stop 2024-04-25T14:04:40.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:40.272+0000 7f9b2891a700 1 -- 172.21.15.96:0/1606075983 shutdown_connections 2024-04-25T14:04:40.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:40.272+0000 7f9b2891a700 1 --2- 172.21.15.96:0/1606075983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b24109a40 0x7f9b24109e50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:40.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:40.272+0000 7f9b2891a700 1 -- 172.21.15.96:0/1606075983 >> 172.21.15.96:0/1606075983 conn(0x7f9b240ff550 msgr2=0x7f9b241019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:40.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:40.272+0000 7f9b2891a700 1 -- 172.21.15.96:0/1606075983 shutdown_connections 2024-04-25T14:04:40.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:40.272+0000 7f9b2891a700 1 -- 172.21.15.96:0/1606075983 wait complete. 2024-04-25T14:04:40.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:40.273+0000 7f9b2891a700 1 Processor -- start 2024-04-25T14:04:40.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:40.273+0000 7f9b2891a700 1 -- start start 2024-04-25T14:04:40.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:40.273+0000 7f9b2891a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b24109a40 0x7f9b2419f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:40.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:40.273+0000 7f9b2891a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2410df90 con 0x7f9b24109a40 2024-04-25T14:04:40.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:40.273+0000 7f9b2259c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b24109a40 0x7f9b2419f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:40.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:40.273+0000 7f9b2259c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b24109a40 0x7f9b2419f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42972/0 (socket says 172.21.15.96:42972) 2024-04-25T14:04:40.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:40.273+0000 7f9b2259c700 1 -- 172.21.15.96:0/3600265892 learned_addr learned my addr 172.21.15.96:0/3600265892 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:04:40.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:40.273+0000 7f9b2259c700 1 -- 172.21.15.96:0/3600265892 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b2419fa30 con 0x7f9b24109a40 2024-04-25T14:04:40.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:40.273+0000 7f9b2259c700 1 --2- 172.21.15.96:0/3600265892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b24109a40 0x7f9b2419f4f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9b140056d0 tx=0x7f9b140057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:40.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:40.274+0000 7f9b137fe700 1 -- 172.21.15.96:0/3600265892 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b14027070 con 0x7f9b24109a40 2024-04-25T14:04:40.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:40.274+0000 7f9b2891a700 1 -- 172.21.15.96:0/3600265892 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b2419fd20 con 0x7f9b24109a40 2024-04-25T14:04:40.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:40.274+0000 7f9b137fe700 1 -- 172.21.15.96:0/3600265892 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b1400be10 con 0x7f9b24109a40 2024-04-25T14:04:40.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:40.274+0000 7f9b137fe700 1 -- 172.21.15.96:0/3600265892 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b14022d20 con 0x7f9b24109a40 2024-04-25T14:04:40.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:40.274+0000 7f9b2891a700 1 -- 172.21.15.96:0/3600265892 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b24199850 con 0x7f9b24109a40 2024-04-25T14:04:40.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:40.275+0000 7f9b2891a700 1 -- 172.21.15.96:0/3600265892 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b24105150 con 0x7f9b24109a40 2024-04-25T14:04:40.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:40.275+0000 7f9b137fe700 1 -- 172.21.15.96:0/3600265892 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f9b1402a070 con 0x7f9b24109a40 2024-04-25T14:04:40.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:40.275+0000 7f9b137fe700 1 --2- 172.21.15.96:0/3600265892 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f9b0c03d610 0x7f9b0c03fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:40.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:40.275+0000 7f9b137fe700 1 -- 172.21.15.96:0/3600265892 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9b1401e030 con 0x7f9b24109a40 2024-04-25T14:04:40.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:40.275+0000 7f9b21d9b700 1 --2- 172.21.15.96:0/3600265892 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f9b0c03d610 0x7f9b0c03fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:40.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:40.276+0000 7f9b21d9b700 1 --2- 172.21.15.96:0/3600265892 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f9b0c03d610 0x7f9b0c03fb20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9b18006fd0 tx=0x7f9b18006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:40.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:40.280+0000 7f9b137fe700 1 -- 172.21.15.96:0/3600265892 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9b14013070 con 0x7f9b24109a40 2024-04-25T14:04:40.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:40.481+0000 7f9b2891a700 1 -- 172.21.15.96:0/3600265892 --> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi097", "target": ["mon-mgr", ""]}) -- 0x7f9b2419ab90 con 0x7f9b0c03d610 2024-04-25T14:04:40.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:40 smithi096 ceph-mon[29207]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi097", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:04:41.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:41 smithi096 ceph-mon[29207]: Deploying cephadm binary to smithi097 2024-04-25T14:04:41.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:41 smithi096 ceph-mon[29207]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:04:44.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:44 smithi096 ceph-mon[29207]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:04:44.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:44.662+0000 7f9b137fe700 1 -- 172.21.15.96:0/3600265892 <== mgr.14152 v2:172.21.15.96:6800/536332425 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f9b2419ab90 con 0x7f9b0c03d610 2024-04-25T14:04:44.662 INFO:teuthology.orchestra.run.smithi096.stdout:Added host 'smithi097' with addr '172.21.15.97' 2024-04-25T14:04:44.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:44.664+0000 7f9b2891a700 1 -- 172.21.15.96:0/3600265892 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f9b0c03d610 msgr2=0x7f9b0c03fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:44.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:44.664+0000 7f9b2891a700 1 --2- 172.21.15.96:0/3600265892 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f9b0c03d610 0x7f9b0c03fb20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9b18006fd0 tx=0x7f9b18006e40 comp rx=0 tx=0).stop 2024-04-25T14:04:44.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:44.664+0000 7f9b2891a700 1 -- 172.21.15.96:0/3600265892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b24109a40 msgr2=0x7f9b2419f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:44.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:44.664+0000 7f9b2891a700 1 --2- 172.21.15.96:0/3600265892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b24109a40 0x7f9b2419f4f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9b140056d0 tx=0x7f9b140057b0 comp rx=0 tx=0).stop 2024-04-25T14:04:44.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:44.664+0000 7f9b2891a700 1 -- 172.21.15.96:0/3600265892 shutdown_connections 2024-04-25T14:04:44.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:44.664+0000 7f9b2891a700 1 --2- 172.21.15.96:0/3600265892 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f9b0c03d610 0x7f9b0c03fb20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:44.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:44.664+0000 7f9b2891a700 1 --2- 172.21.15.96:0/3600265892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b24109a40 0x7f9b2419f4f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:44.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:44.664+0000 7f9b2891a700 1 -- 172.21.15.96:0/3600265892 >> 172.21.15.96:0/3600265892 conn(0x7f9b240ff550 msgr2=0x7f9b24100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:44.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:44.664+0000 7f9b2891a700 1 -- 172.21.15.96:0/3600265892 shutdown_connections 2024-04-25T14:04:44.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:44.664+0000 7f9b2891a700 1 -- 172.21.15.96:0/3600265892 wait complete. 2024-04-25T14:04:45.257 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T14:04:45.489 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:04:45.940 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:45 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:04:45.940 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:45 smithi096 ceph-mon[29207]: Added host smithi097 2024-04-25T14:04:45.941 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:45 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:04:45.941 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:45 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:04:45.941 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:45 smithi096 ceph-mon[29207]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:04:46.820 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:46 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:04:47.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:47.048+0000 7f5fc9fbb700 1 -- 172.21.15.96:0/1419791241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5fc4108930 msgr2=0x7f5fc4108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:47.048+0000 7f5fc9fbb700 1 --2- 172.21.15.96:0/1419791241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5fc4108930 0x7f5fc4108d40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4009910 tx=0x7f5fb4009c20 comp rx=0 tx=0).stop 2024-04-25T14:04:47.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:47.048+0000 7f5fc9fbb700 1 -- 172.21.15.96:0/1419791241 shutdown_connections 2024-04-25T14:04:47.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:47.048+0000 7f5fc9fbb700 1 --2- 172.21.15.96:0/1419791241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5fc4108930 0x7f5fc4108d40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:47.048+0000 7f5fc9fbb700 1 -- 172.21.15.96:0/1419791241 >> 172.21.15.96:0/1419791241 conn(0x7f5fc40fe480 msgr2=0x7f5fc41008d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:47.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:47.048+0000 7f5fc9fbb700 1 -- 172.21.15.96:0/1419791241 shutdown_connections 2024-04-25T14:04:47.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:47.048+0000 7f5fc9fbb700 1 -- 172.21.15.96:0/1419791241 wait complete. 2024-04-25T14:04:47.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:47.049+0000 7f5fc9fbb700 1 Processor -- start 2024-04-25T14:04:47.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:47.049+0000 7f5fc9fbb700 1 -- start start 2024-04-25T14:04:47.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:47.049+0000 7f5fc9fbb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5fc4108930 0x7f5fc419f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:47.049+0000 7f5fc9fbb700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc410ce80 con 0x7f5fc4108930 2024-04-25T14:04:47.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:47.050+0000 7f5fc37fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5fc4108930 0x7f5fc419f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:47.050+0000 7f5fc37fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5fc4108930 0x7f5fc419f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54708/0 (socket says 172.21.15.96:54708) 2024-04-25T14:04:47.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:47.050+0000 7f5fc37fe700 1 -- 172.21.15.96:0/3375541806 learned_addr learned my addr 172.21.15.96:0/3375541806 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:04:47.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:47.050+0000 7f5fc37fe700 1 -- 172.21.15.96:0/3375541806 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fc419fa00 con 0x7f5fc4108930 2024-04-25T14:04:47.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:47.050+0000 7f5fc37fe700 1 --2- 172.21.15.96:0/3375541806 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5fc4108930 0x7f5fc419f4c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5fb40056d0 tx=0x7f5fb40057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:47.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:47.050+0000 7f5fc17fa700 1 -- 172.21.15.96:0/3375541806 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fb4027070 con 0x7f5fc4108930 2024-04-25T14:04:47.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:47.050+0000 7f5fc17fa700 1 -- 172.21.15.96:0/3375541806 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fb400be10 con 0x7f5fc4108930 2024-04-25T14:04:47.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:47.050+0000 7f5fc9fbb700 1 -- 172.21.15.96:0/3375541806 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fc419fcf0 con 0x7f5fc4108930 2024-04-25T14:04:47.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:47.050+0000 7f5fc17fa700 1 -- 172.21.15.96:0/3375541806 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fb4022d20 con 0x7f5fc4108930 2024-04-25T14:04:47.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:47.050+0000 7f5fc9fbb700 1 -- 172.21.15.96:0/3375541806 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fc4199820 con 0x7f5fc4108930 2024-04-25T14:04:47.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:47.051+0000 7f5fc9fbb700 1 -- 172.21.15.96:0/3375541806 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fc4104040 con 0x7f5fc4108930 2024-04-25T14:04:47.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:47.051+0000 7f5fc17fa700 1 -- 172.21.15.96:0/3375541806 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f5fb402a070 con 0x7f5fc4108930 2024-04-25T14:04:47.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:47.051+0000 7f5fc17fa700 1 --2- 172.21.15.96:0/3375541806 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f5fb003d610 0x7f5fb003fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:47.051+0000 7f5fc17fa700 1 -- 172.21.15.96:0/3375541806 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5fb401e030 con 0x7f5fc4108930 2024-04-25T14:04:47.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:47.051+0000 7f5fbaffd700 1 --2- 172.21.15.96:0/3375541806 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f5fb003d610 0x7f5fb003fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:47.052+0000 7f5fbaffd700 1 --2- 172.21.15.96:0/3375541806 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f5fb003d610 0x7f5fb003fb20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5fac006fd0 tx=0x7f5fac006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:47.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:47.057+0000 7f5fc17fa700 1 -- 172.21.15.96:0/3375541806 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5fb4013070 con 0x7f5fc4108930 2024-04-25T14:04:47.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:47.259+0000 7f5fc9fbb700 1 -- 172.21.15.96:0/3375541806 --> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5fc419ab90 con 0x7f5fb003d610 2024-04-25T14:04:47.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:47.260+0000 7f5fc17fa700 1 -- 172.21.15.96:0/3375541806 <== mgr.14152 v2:172.21.15.96:6800/536332425 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f5fc419ab90 con 0x7f5fb003d610 2024-04-25T14:04:47.260 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:04:47.260 INFO:teuthology.orchestra.run.smithi096.stdout:[{"addr": "172.21.15.96", "hostname": "smithi096", "labels": [], "status": ""}, {"addr": "172.21.15.97", "hostname": "smithi097", "labels": [], "status": ""}] 2024-04-25T14:04:47.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:47.262+0000 7f5fc9fbb700 1 -- 172.21.15.96:0/3375541806 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f5fb003d610 msgr2=0x7f5fb003fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:47.262+0000 7f5fc9fbb700 1 --2- 172.21.15.96:0/3375541806 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f5fb003d610 0x7f5fb003fb20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5fac006fd0 tx=0x7f5fac006e40 comp rx=0 tx=0).stop 2024-04-25T14:04:47.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:47.262+0000 7f5fc9fbb700 1 -- 172.21.15.96:0/3375541806 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5fc4108930 msgr2=0x7f5fc419f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:47.262+0000 7f5fc9fbb700 1 --2- 172.21.15.96:0/3375541806 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5fc4108930 0x7f5fc419f4c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5fb40056d0 tx=0x7f5fb40057b0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:47.262+0000 7f5fc9fbb700 1 -- 172.21.15.96:0/3375541806 shutdown_connections 2024-04-25T14:04:47.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:47.262+0000 7f5fc9fbb700 1 --2- 172.21.15.96:0/3375541806 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f5fb003d610 0x7f5fb003fb20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:47.262+0000 7f5fc9fbb700 1 --2- 172.21.15.96:0/3375541806 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5fc4108930 0x7f5fc419f4c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:47.262+0000 7f5fc9fbb700 1 -- 172.21.15.96:0/3375541806 >> 172.21.15.96:0/3375541806 conn(0x7f5fc40fe480 msgr2=0x7f5fc4106410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:47.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:47.263+0000 7f5fc9fbb700 1 -- 172.21.15.96:0/3375541806 shutdown_connections 2024-04-25T14:04:47.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:47.263+0000 7f5fc9fbb700 1 -- 172.21.15.96:0/3375541806 wait complete. 2024-04-25T14:04:47.909 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi151 2024-04-25T14:04:47.910 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-25T14:04:47.910 DEBUG:teuthology.orchestra.run.smithi151:> dd of=/etc/ceph/ceph.conf 2024-04-25T14:04:47.929 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-25T14:04:47.929 DEBUG:teuthology.orchestra.run.smithi151:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:04:47.986 INFO:tasks.cephadm:Adding host smithi151 to orchestrator... 2024-04-25T14:04:47.986 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph orch host add smithi151 2024-04-25T14:04:48.223 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:04:48.561 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:48 smithi096 ceph-mon[29207]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:04:48.561 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:48 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:04:48.561 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:48 smithi096 ceph-mon[29207]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:04:49.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:49.653+0000 7f701f22f700 1 -- 172.21.15.96:0/154339046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7018109a10 msgr2=0x7f7018109e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:49.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:49.653+0000 7f701f22f700 1 --2- 172.21.15.96:0/154339046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7018109a10 0x7f7018109e20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f700c009910 tx=0x7f700c009c20 comp rx=0 tx=0).stop 2024-04-25T14:04:49.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:49.653+0000 7f701f22f700 1 -- 172.21.15.96:0/154339046 shutdown_connections 2024-04-25T14:04:49.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:49.653+0000 7f701f22f700 1 --2- 172.21.15.96:0/154339046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7018109a10 0x7f7018109e20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:49.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:49.653+0000 7f701f22f700 1 -- 172.21.15.96:0/154339046 >> 172.21.15.96:0/154339046 conn(0x7f70180ff560 msgr2=0x7f70181019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:49.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:49.653+0000 7f701f22f700 1 -- 172.21.15.96:0/154339046 shutdown_connections 2024-04-25T14:04:49.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:49.653+0000 7f701f22f700 1 -- 172.21.15.96:0/154339046 wait complete. 2024-04-25T14:04:49.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:49.653+0000 7f701f22f700 1 Processor -- start 2024-04-25T14:04:49.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:49.653+0000 7f701f22f700 1 -- start start 2024-04-25T14:04:49.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:49.654+0000 7f701f22f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7018109a10 0x7f70181997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:49.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:49.654+0000 7f701f22f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f701810df60 con 0x7f7018109a10 2024-04-25T14:04:49.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:49.654+0000 7f701cfcb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7018109a10 0x7f70181997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:49.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:49.654+0000 7f701cfcb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7018109a10 0x7f70181997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54718/0 (socket says 172.21.15.96:54718) 2024-04-25T14:04:49.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:49.654+0000 7f701cfcb700 1 -- 172.21.15.96:0/3773633149 learned_addr learned my addr 172.21.15.96:0/3773633149 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:04:49.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:49.654+0000 7f701cfcb700 1 -- 172.21.15.96:0/3773633149 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7018199d20 con 0x7f7018109a10 2024-04-25T14:04:49.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:49.654+0000 7f701cfcb700 1 --2- 172.21.15.96:0/3773633149 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7018109a10 0x7f70181997e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f700c0056d0 tx=0x7f700c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:49.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:49.654+0000 7f7015ffb700 1 -- 172.21.15.96:0/3773633149 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f700c027070 con 0x7f7018109a10 2024-04-25T14:04:49.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:49.654+0000 7f7015ffb700 1 -- 172.21.15.96:0/3773633149 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f700c00be10 con 0x7f7018109a10 2024-04-25T14:04:49.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:49.654+0000 7f701f22f700 1 -- 172.21.15.96:0/3773633149 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f701819a010 con 0x7f7018109a10 2024-04-25T14:04:49.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:49.654+0000 7f7015ffb700 1 -- 172.21.15.96:0/3773633149 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f700c022d20 con 0x7f7018109a10 2024-04-25T14:04:49.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:49.655+0000 7f701f22f700 1 -- 172.21.15.96:0/3773633149 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f701819e010 con 0x7f7018109a10 2024-04-25T14:04:49.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:49.655+0000 7f7015ffb700 1 -- 172.21.15.96:0/3773633149 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f700c02a070 con 0x7f7018109a10 2024-04-25T14:04:49.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:49.655+0000 7f701f22f700 1 -- 172.21.15.96:0/3773633149 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7018105120 con 0x7f7018109a10 2024-04-25T14:04:49.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:49.656+0000 7f7015ffb700 1 --2- 172.21.15.96:0/3773633149 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f700803d670 0x7f700803fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:49.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:49.656+0000 7f7015ffb700 1 -- 172.21.15.96:0/3773633149 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f700c01e030 con 0x7f7018109a10 2024-04-25T14:04:49.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:49.656+0000 7f7017fff700 1 --2- 172.21.15.96:0/3773633149 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f700803d670 0x7f700803fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:49.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:49.656+0000 7f7017fff700 1 --2- 172.21.15.96:0/3773633149 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f700803d670 0x7f700803fb80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7004006fd0 tx=0x7f7004006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:49.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:49.661+0000 7f7015ffb700 1 -- 172.21.15.96:0/3773633149 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f700c013070 con 0x7f7018109a10 2024-04-25T14:04:49.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:49.862+0000 7f701f22f700 1 -- 172.21.15.96:0/3773633149 --> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi151", "target": ["mon-mgr", ""]}) -- 0x7f701819ab40 con 0x7f700803d670 2024-04-25T14:04:50.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:50 smithi096 ceph-mon[29207]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:04:51.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:51 smithi096 ceph-mon[29207]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi151", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:04:51.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:51 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:04:51.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:51 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:04:51.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:51 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:04:51.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:51 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:04:51.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:51 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi097", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:04:51.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:51 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:04:51.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:51 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:04:52.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:52 smithi096 ceph-mon[29207]: Deploying cephadm binary to smithi151 2024-04-25T14:04:52.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:52 smithi096 ceph-mon[29207]: Updating smithi097:/etc/ceph/ceph.conf 2024-04-25T14:04:52.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:52 smithi096 ceph-mon[29207]: Updating smithi097:/var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:04:52.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:52 smithi096 ceph-mon[29207]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:04:54.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:54.051+0000 7f7015ffb700 1 -- 172.21.15.96:0/3773633149 <== mgr.14152 v2:172.21.15.96:6800/536332425 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f701819ab40 con 0x7f700803d670 2024-04-25T14:04:54.051 INFO:teuthology.orchestra.run.smithi096.stdout:Added host 'smithi151' with addr '172.21.15.151' 2024-04-25T14:04:54.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:54.054+0000 7f701f22f700 1 -- 172.21.15.96:0/3773633149 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f700803d670 msgr2=0x7f700803fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:54.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:54.054+0000 7f701f22f700 1 --2- 172.21.15.96:0/3773633149 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f700803d670 0x7f700803fb80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7004006fd0 tx=0x7f7004006e40 comp rx=0 tx=0).stop 2024-04-25T14:04:54.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:54.054+0000 7f701f22f700 1 -- 172.21.15.96:0/3773633149 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7018109a10 msgr2=0x7f70181997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:54.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:54.054+0000 7f701f22f700 1 --2- 172.21.15.96:0/3773633149 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7018109a10 0x7f70181997e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f700c0056d0 tx=0x7f700c0057b0 comp rx=0 tx=0).stop 2024-04-25T14:04:54.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:54.054+0000 7f701f22f700 1 -- 172.21.15.96:0/3773633149 shutdown_connections 2024-04-25T14:04:54.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:54.054+0000 7f701f22f700 1 --2- 172.21.15.96:0/3773633149 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f700803d670 0x7f700803fb80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:54.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:54.054+0000 7f701f22f700 1 --2- 172.21.15.96:0/3773633149 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7018109a10 0x7f70181997e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:54.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:54.054+0000 7f701f22f700 1 -- 172.21.15.96:0/3773633149 >> 172.21.15.96:0/3773633149 conn(0x7f70180ff560 msgr2=0x7f7018101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:54.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:54.054+0000 7f701f22f700 1 -- 172.21.15.96:0/3773633149 shutdown_connections 2024-04-25T14:04:54.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:54.054+0000 7f701f22f700 1 -- 172.21.15.96:0/3773633149 wait complete. 2024-04-25T14:04:54.678 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:54 smithi096 ceph-mon[29207]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:04:54.678 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:54 smithi096 ceph-mon[29207]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:04:54.678 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:54 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:04:54.678 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:54 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:04:54.678 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:54 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:04:54.678 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:54 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:04:54.678 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:54 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:04:54.704 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T14:04:54.934 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:04:55.717 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:55 smithi096 ceph-mon[29207]: Updating smithi097:/var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T14:04:55.717 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:55 smithi096 ceph-mon[29207]: Added host smithi151 2024-04-25T14:04:55.717 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:55 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:04:56.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:56.431+0000 7fb23cc10700 1 -- 172.21.15.96:0/728589472 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2381021e0 msgr2=0x7fb2381025f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:56.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:56.431+0000 7fb23cc10700 1 --2- 172.21.15.96:0/728589472 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2381021e0 0x7fb2381025f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb22800b0a0 tx=0x7fb22800b3b0 comp rx=0 tx=0).stop 2024-04-25T14:04:56.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:56.431+0000 7fb23cc10700 1 -- 172.21.15.96:0/728589472 shutdown_connections 2024-04-25T14:04:56.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:56.431+0000 7fb23cc10700 1 --2- 172.21.15.96:0/728589472 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2381021e0 0x7fb2381025f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:56.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:56.431+0000 7fb23cc10700 1 -- 172.21.15.96:0/728589472 >> 172.21.15.96:0/728589472 conn(0x7fb2380fdd00 msgr2=0x7fb238100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:56.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:56.431+0000 7fb23cc10700 1 -- 172.21.15.96:0/728589472 shutdown_connections 2024-04-25T14:04:56.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:56.431+0000 7fb23cc10700 1 -- 172.21.15.96:0/728589472 wait complete. 2024-04-25T14:04:56.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:56.432+0000 7fb23cc10700 1 Processor -- start 2024-04-25T14:04:56.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:56.432+0000 7fb23cc10700 1 -- start start 2024-04-25T14:04:56.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:56.432+0000 7fb23cc10700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2381021e0 0x7fb238103130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:56.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:56.432+0000 7fb23cc10700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb23810def0 con 0x7fb2381021e0 2024-04-25T14:04:56.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:56.432+0000 7fb23659c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2381021e0 0x7fb238103130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:56.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:56.432+0000 7fb23659c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2381021e0 0x7fb238103130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50770/0 (socket says 172.21.15.96:50770) 2024-04-25T14:04:56.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:56.432+0000 7fb23659c700 1 -- 172.21.15.96:0/4258167634 learned_addr learned my addr 172.21.15.96:0/4258167634 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:04:56.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:56.433+0000 7fb23659c700 1 -- 172.21.15.96:0/4258167634 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb238104f80 con 0x7fb2381021e0 2024-04-25T14:04:56.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:56.433+0000 7fb23659c700 1 --2- 172.21.15.96:0/4258167634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2381021e0 0x7fb238103130 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb228015040 tx=0x7fb228009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:56.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:56.433+0000 7fb22f7fe700 1 -- 172.21.15.96:0/4258167634 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb228021020 con 0x7fb2381021e0 2024-04-25T14:04:56.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:56.433+0000 7fb22f7fe700 1 -- 172.21.15.96:0/4258167634 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb228007970 con 0x7fb2381021e0 2024-04-25T14:04:56.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:56.433+0000 7fb23cc10700 1 -- 172.21.15.96:0/4258167634 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb238103670 con 0x7fb2381021e0 2024-04-25T14:04:56.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:56.433+0000 7fb22f7fe700 1 -- 172.21.15.96:0/4258167634 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb228017040 con 0x7fb2381021e0 2024-04-25T14:04:56.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:56.433+0000 7fb23cc10700 1 -- 172.21.15.96:0/4258167634 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb238103b30 con 0x7fb2381021e0 2024-04-25T14:04:56.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:56.434+0000 7fb22f7fe700 1 -- 172.21.15.96:0/4258167634 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7fb228024020 con 0x7fb2381021e0 2024-04-25T14:04:56.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:56.434+0000 7fb22f7fe700 1 --2- 172.21.15.96:0/4258167634 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb22403d610 0x7fb22403fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:56.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:56.434+0000 7fb23cc10700 1 -- 172.21.15.96:0/4258167634 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb238064cf0 con 0x7fb2381021e0 2024-04-25T14:04:56.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:56.434+0000 7fb22f7fe700 1 -- 172.21.15.96:0/4258167634 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb22800f050 con 0x7fb2381021e0 2024-04-25T14:04:56.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:56.434+0000 7fb235d9b700 1 --2- 172.21.15.96:0/4258167634 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb22403d610 0x7fb22403fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:56.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:56.435+0000 7fb235d9b700 1 --2- 172.21.15.96:0/4258167634 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb22403d610 0x7fb22403fb20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb220006fd0 tx=0x7fb220006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:56.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:56.440+0000 7fb22f7fe700 1 -- 172.21.15.96:0/4258167634 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb22802b020 con 0x7fb2381021e0 2024-04-25T14:04:56.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:56.644+0000 7fb23cc10700 1 -- 172.21.15.96:0/4258167634 --> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb2381052b0 con 0x7fb22403d610 2024-04-25T14:04:56.644 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:56 smithi096 ceph-mon[29207]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:04:56.644 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:56 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:04:56.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:56.645+0000 7fb22f7fe700 1 -- 172.21.15.96:0/4258167634 <== mgr.14152 v2:172.21.15.96:6800/536332425 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7fb2381052b0 con 0x7fb22403d610 2024-04-25T14:04:56.644 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:04:56.645 INFO:teuthology.orchestra.run.smithi096.stdout:[{"addr": "172.21.15.96", "hostname": "smithi096", "labels": [], "status": ""}, {"addr": "172.21.15.97", "hostname": "smithi097", "labels": [], "status": ""}, {"addr": "172.21.15.151", "hostname": "smithi151", "labels": [], "status": ""}] 2024-04-25T14:04:56.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:56.647+0000 7fb23cc10700 1 -- 172.21.15.96:0/4258167634 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb22403d610 msgr2=0x7fb22403fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:56.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:56.647+0000 7fb23cc10700 1 --2- 172.21.15.96:0/4258167634 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb22403d610 0x7fb22403fb20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb220006fd0 tx=0x7fb220006e40 comp rx=0 tx=0).stop 2024-04-25T14:04:56.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:56.647+0000 7fb23cc10700 1 -- 172.21.15.96:0/4258167634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2381021e0 msgr2=0x7fb238103130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:56.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:56.647+0000 7fb23cc10700 1 --2- 172.21.15.96:0/4258167634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2381021e0 0x7fb238103130 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb228015040 tx=0x7fb228009790 comp rx=0 tx=0).stop 2024-04-25T14:04:56.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:56.648+0000 7fb23cc10700 1 -- 172.21.15.96:0/4258167634 shutdown_connections 2024-04-25T14:04:56.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:56.648+0000 7fb23cc10700 1 --2- 172.21.15.96:0/4258167634 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb22403d610 0x7fb22403fb20 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:56.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:56.648+0000 7fb23cc10700 1 --2- 172.21.15.96:0/4258167634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2381021e0 0x7fb238103130 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:56.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:56.648+0000 7fb23cc10700 1 -- 172.21.15.96:0/4258167634 >> 172.21.15.96:0/4258167634 conn(0x7fb2380fdd00 msgr2=0x7fb238109820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:56.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:56.648+0000 7fb23cc10700 1 -- 172.21.15.96:0/4258167634 shutdown_connections 2024-04-25T14:04:56.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:56.648+0000 7fb23cc10700 1 -- 172.21.15.96:0/4258167634 wait complete. 2024-04-25T14:04:57.319 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-25T14:04:57.319 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-25T14:04:57.552 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:04:57.934 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:57 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:04:57.934 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:57 smithi096 ceph-mon[29207]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:04:57.934 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:57 smithi096 ceph-mon[29207]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:04:59.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:59.045+0000 7fcdaec61700 1 -- 172.21.15.96:0/3472531057 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcda81032d0 msgr2=0x7fcda81036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:59.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:59.045+0000 7fcdaec61700 1 --2- 172.21.15.96:0/3472531057 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcda81032d0 0x7fcda81036e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c009910 tx=0x7fcd9c009c20 comp rx=0 tx=0).stop 2024-04-25T14:04:59.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:59.045+0000 7fcdaec61700 1 -- 172.21.15.96:0/3472531057 shutdown_connections 2024-04-25T14:04:59.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:59.045+0000 7fcdaec61700 1 --2- 172.21.15.96:0/3472531057 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcda81032d0 0x7fcda81036e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:59.045+0000 7fcdaec61700 1 -- 172.21.15.96:0/3472531057 >> 172.21.15.96:0/3472531057 conn(0x7fcda80fdd00 msgr2=0x7fcda8100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:59.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:59.045+0000 7fcdaec61700 1 -- 172.21.15.96:0/3472531057 shutdown_connections 2024-04-25T14:04:59.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:59.045+0000 7fcdaec61700 1 -- 172.21.15.96:0/3472531057 wait complete. 2024-04-25T14:04:59.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:59.045+0000 7fcdaec61700 1 Processor -- start 2024-04-25T14:04:59.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:59.045+0000 7fcdaec61700 1 -- start start 2024-04-25T14:04:59.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:59.046+0000 7fcdaec61700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcda81032d0 0x7fcda819f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:59.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:59.046+0000 7fcdaec61700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcda810d0a0 con 0x7fcda81032d0 2024-04-25T14:04:59.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:59.046+0000 7fcdac9fd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcda81032d0 0x7fcda819f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:59.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:59.046+0000 7fcdac9fd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcda81032d0 0x7fcda819f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50790/0 (socket says 172.21.15.96:50790) 2024-04-25T14:04:59.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:59.046+0000 7fcdac9fd700 1 -- 172.21.15.96:0/144008411 learned_addr learned my addr 172.21.15.96:0/144008411 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:04:59.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:59.046+0000 7fcdac9fd700 1 -- 172.21.15.96:0/144008411 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcda819fba0 con 0x7fcda81032d0 2024-04-25T14:04:59.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:59.046+0000 7fcdac9fd700 1 --2- 172.21.15.96:0/144008411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcda81032d0 0x7fcda819f660 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c0056d0 tx=0x7fcd9c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:59.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:59.046+0000 7fcda5ffb700 1 -- 172.21.15.96:0/144008411 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd9c026070 con 0x7fcda81032d0 2024-04-25T14:04:59.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:59.046+0000 7fcda5ffb700 1 -- 172.21.15.96:0/144008411 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd9c00bba0 con 0x7fcda81032d0 2024-04-25T14:04:59.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:59.046+0000 7fcda5ffb700 1 -- 172.21.15.96:0/144008411 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd9c003920 con 0x7fcda81032d0 2024-04-25T14:04:59.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:59.046+0000 7fcdaec61700 1 -- 172.21.15.96:0/144008411 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcda819fe90 con 0x7fcda81032d0 2024-04-25T14:04:59.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:59.047+0000 7fcdaec61700 1 -- 172.21.15.96:0/144008411 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcda8199880 con 0x7fcda81032d0 2024-04-25T14:04:59.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:59.047+0000 7fcda5ffb700 1 -- 172.21.15.96:0/144008411 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7fcd9c029070 con 0x7fcda81032d0 2024-04-25T14:04:59.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:59.047+0000 7fcdaec61700 1 -- 172.21.15.96:0/144008411 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcda81042f0 con 0x7fcda81032d0 2024-04-25T14:04:59.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:59.048+0000 7fcda5ffb700 1 --2- 172.21.15.96:0/144008411 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fcd9803d610 0x7fcd9803fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:59.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:59.048+0000 7fcda5ffb700 1 -- 172.21.15.96:0/144008411 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcd9c01e070 con 0x7fcda81032d0 2024-04-25T14:04:59.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:59.048+0000 7fcda7fff700 1 --2- 172.21.15.96:0/144008411 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fcd9803d610 0x7fcd9803fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:59.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:59.048+0000 7fcda7fff700 1 --2- 172.21.15.96:0/144008411 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fcd9803d610 0x7fcd9803fb20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcd94006fd0 tx=0x7fcd94006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:59.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:59.053+0000 7fcda5ffb700 1 -- 172.21.15.96:0/144008411 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcd9c013070 con 0x7fcda81032d0 2024-04-25T14:04:59.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:59.256+0000 7fcdaec61700 1 -- 172.21.15.96:0/144008411 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fcda8064cf0 con 0x7fcda81032d0 2024-04-25T14:04:59.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:59.308+0000 7fcda5ffb700 1 -- 172.21.15.96:0/144008411 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fcd9c00bd50 con 0x7fcda81032d0 2024-04-25T14:04:59.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:59.310+0000 7fcdaec61700 1 -- 172.21.15.96:0/144008411 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fcd9803d610 msgr2=0x7fcd9803fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:59.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:59.310+0000 7fcdaec61700 1 --2- 172.21.15.96:0/144008411 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fcd9803d610 0x7fcd9803fb20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcd94006fd0 tx=0x7fcd94006e40 comp rx=0 tx=0).stop 2024-04-25T14:04:59.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:59.310+0000 7fcdaec61700 1 -- 172.21.15.96:0/144008411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcda81032d0 msgr2=0x7fcda819f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:59.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:59.310+0000 7fcdaec61700 1 --2- 172.21.15.96:0/144008411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcda81032d0 0x7fcda819f660 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c0056d0 tx=0x7fcd9c0057b0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:59.310+0000 7fcdaec61700 1 -- 172.21.15.96:0/144008411 shutdown_connections 2024-04-25T14:04:59.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:59.310+0000 7fcdaec61700 1 --2- 172.21.15.96:0/144008411 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fcd9803d610 0x7fcd9803fb20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:59.310+0000 7fcdaec61700 1 --2- 172.21.15.96:0/144008411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcda81032d0 0x7fcda819f660 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:59.310+0000 7fcdaec61700 1 -- 172.21.15.96:0/144008411 >> 172.21.15.96:0/144008411 conn(0x7fcda80fdd00 msgr2=0x7fcda8100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:59.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:59.310+0000 7fcdaec61700 1 -- 172.21.15.96:0/144008411 shutdown_connections 2024-04-25T14:04:59.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:04:59.310+0000 7fcdaec61700 1 -- 172.21.15.96:0/144008411 wait complete. 2024-04-25T14:04:59.310 INFO:teuthology.orchestra.run.smithi096.stderr:adjusted tunables profile to default 2024-04-25T14:04:59.608 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:04:59 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/144008411' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T14:04:59.996 INFO:tasks.cephadm:Adding mon.a on smithi096 2024-04-25T14:04:59.997 INFO:tasks.cephadm:Adding mon.b on smithi097 2024-04-25T14:04:59.997 INFO:tasks.cephadm:Adding mon.c on smithi151 2024-04-25T14:04:59.997 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi096:172.21.15.96=a;smithi097:172.21.15.97=b;smithi151:172.21.15.151=c' 2024-04-25T14:05:00.233 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:05:00.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:00 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/144008411' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T14:05:00.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:00 smithi096 ceph-mon[29207]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T14:05:00.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:00 smithi096 ceph-mon[29207]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:00.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:00 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:00.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:00 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:00.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:00 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:00.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:00 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:00.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:00 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi151", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:05:00.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:00 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:00.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:00 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:05:01.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:01 smithi096 ceph-mon[29207]: Updating smithi151:/etc/ceph/ceph.conf 2024-04-25T14:05:01.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:01 smithi096 ceph-mon[29207]: Updating smithi151:/var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:05:01.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:01 smithi096 ceph-mon[29207]: Updating smithi151:/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:05:01.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:01 smithi096 ceph-mon[29207]: Updating smithi151:/var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T14:05:01.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:01 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:01.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:01 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:01.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:01 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:01.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:01.739+0000 7f76dbe15700 1 -- 172.21.15.151:0/3820374499 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76d4107990 msgr2=0x7f76d4107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:01.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:01.739+0000 7f76dbe15700 1 --2- 172.21.15.151:0/3820374499 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76d4107990 0x7f76d4107da0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f76d0009910 tx=0x7f76d0009c20 comp rx=0 tx=0).stop 2024-04-25T14:05:01.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:01.739+0000 7f76dbe15700 1 -- 172.21.15.151:0/3820374499 shutdown_connections 2024-04-25T14:05:01.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:01.739+0000 7f76dbe15700 1 --2- 172.21.15.151:0/3820374499 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76d4107990 0x7f76d4107da0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:01.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:01.739+0000 7f76dbe15700 1 -- 172.21.15.151:0/3820374499 >> 172.21.15.151:0/3820374499 conn(0x7f76d4078c30 msgr2=0x7f76d4079070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:01.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:01.739+0000 7f76dbe15700 1 -- 172.21.15.151:0/3820374499 shutdown_connections 2024-04-25T14:05:01.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:01.739+0000 7f76dbe15700 1 -- 172.21.15.151:0/3820374499 wait complete. 2024-04-25T14:05:01.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:01.740+0000 7f76dbe15700 1 Processor -- start 2024-04-25T14:05:01.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:01.740+0000 7f76dbe15700 1 -- start start 2024-04-25T14:05:01.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:01.740+0000 7f76dbe15700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76d4107990 0x7f76d419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:01.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:01.740+0000 7f76dbe15700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f76d410cf20 con 0x7f76d4107990 2024-04-25T14:05:01.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:01.740+0000 7f76d9bb1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76d4107990 0x7f76d419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:01.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:01.741+0000 7f76d9bb1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76d4107990 0x7f76d419a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.151:49322/0 (socket says 172.21.15.151:49322) 2024-04-25T14:05:01.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:01.741+0000 7f76d9bb1700 1 -- 172.21.15.151:0/2550897578 learned_addr learned my addr 172.21.15.151:0/2550897578 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-25T14:05:01.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:01.741+0000 7f76d9bb1700 1 -- 172.21.15.151:0/2550897578 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76d4102e50 con 0x7f76d4107990 2024-04-25T14:05:01.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:01.741+0000 7f76d9bb1700 1 --2- 172.21.15.151:0/2550897578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76d4107990 0x7f76d419a6a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f76d000b5c0 tx=0x7f76d0005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:01.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:01.741+0000 7f76caffd700 1 -- 172.21.15.151:0/2550897578 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76d0026070 con 0x7f76d4107990 2024-04-25T14:05:01.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:01.741+0000 7f76dbe15700 1 -- 172.21.15.151:0/2550897578 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76d419ac40 con 0x7f76d4107990 2024-04-25T14:05:01.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:01.741+0000 7f76caffd700 1 -- 172.21.15.151:0/2550897578 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76d000b750 con 0x7f76d4107990 2024-04-25T14:05:01.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:01.741+0000 7f76caffd700 1 -- 172.21.15.151:0/2550897578 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76d00037d0 con 0x7f76d4107990 2024-04-25T14:05:01.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:01.741+0000 7f76dbe15700 1 -- 172.21.15.151:0/2550897578 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76d419b100 con 0x7f76d4107990 2024-04-25T14:05:01.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:01.742+0000 7f76dbe15700 1 -- 172.21.15.151:0/2550897578 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76d4104270 con 0x7f76d4107990 2024-04-25T14:05:01.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:01.742+0000 7f76caffd700 1 -- 172.21.15.151:0/2550897578 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f76d0029070 con 0x7f76d4107990 2024-04-25T14:05:01.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:01.743+0000 7f76caffd700 1 --2- 172.21.15.151:0/2550897578 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f76c003d610 0x7f76c003fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:01.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:01.743+0000 7f76caffd700 1 -- 172.21.15.151:0/2550897578 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f76d001e070 con 0x7f76d4107990 2024-04-25T14:05:01.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:01.743+0000 7f76d93b0700 1 --2- 172.21.15.151:0/2550897578 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f76c003d610 0x7f76c003fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:01.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:01.743+0000 7f76d93b0700 1 --2- 172.21.15.151:0/2550897578 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f76c003d610 0x7f76c003fb20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f76c4006fd0 tx=0x7f76c4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:01.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:01.748+0000 7f76caffd700 1 -- 172.21.15.151:0/2550897578 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f76d0013070 con 0x7f76d4107990 2024-04-25T14:05:01.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:01.961+0000 7f76dbe15700 1 -- 172.21.15.151:0/2550897578 --> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi096:172.21.15.96=a;smithi097:172.21.15.97=b;smithi151:172.21.15.151=c", "target": ["mon-mgr", ""]}) -- 0x7f76d419b440 con 0x7f76c003d610 2024-04-25T14:05:01.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:01.968+0000 7f76caffd700 1 -- 172.21.15.151:0/2550897578 <== mgr.14152 v2:172.21.15.96:6800/536332425 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f76d419b440 con 0x7f76c003d610 2024-04-25T14:05:01.968 INFO:teuthology.orchestra.run.smithi151.stdout:Scheduled mon update... 2024-04-25T14:05:01.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:01.970+0000 7f76dbe15700 1 -- 172.21.15.151:0/2550897578 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f76c003d610 msgr2=0x7f76c003fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:01.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:01.970+0000 7f76dbe15700 1 --2- 172.21.15.151:0/2550897578 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f76c003d610 0x7f76c003fb20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f76c4006fd0 tx=0x7f76c4006e40 comp rx=0 tx=0).stop 2024-04-25T14:05:01.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:01.970+0000 7f76dbe15700 1 -- 172.21.15.151:0/2550897578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76d4107990 msgr2=0x7f76d419a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:01.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:01.970+0000 7f76dbe15700 1 --2- 172.21.15.151:0/2550897578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76d4107990 0x7f76d419a6a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f76d000b5c0 tx=0x7f76d0005780 comp rx=0 tx=0).stop 2024-04-25T14:05:01.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:01.970+0000 7f76dbe15700 1 -- 172.21.15.151:0/2550897578 shutdown_connections 2024-04-25T14:05:01.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:01.970+0000 7f76dbe15700 1 --2- 172.21.15.151:0/2550897578 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f76c003d610 0x7f76c003fb20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:01.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:01.970+0000 7f76dbe15700 1 --2- 172.21.15.151:0/2550897578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76d4107990 0x7f76d419a6a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:01.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:01.970+0000 7f76dbe15700 1 -- 172.21.15.151:0/2550897578 >> 172.21.15.151:0/2550897578 conn(0x7f76d4078c30 msgr2=0x7f76d40febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:01.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:01.970+0000 7f76dbe15700 1 -- 172.21.15.151:0/2550897578 shutdown_connections 2024-04-25T14:05:01.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:01.970+0000 7f76dbe15700 1 -- 172.21.15.151:0/2550897578 wait complete. 2024-04-25T14:05:02.605 DEBUG:teuthology.orchestra.run.smithi097:mon.b> sudo journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@mon.b.service 2024-04-25T14:05:02.608 DEBUG:teuthology.orchestra.run.smithi151:mon.c> sudo journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@mon.c.service 2024-04-25T14:05:02.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:02 smithi096 ceph-mon[29207]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:02.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:02 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:02.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:02 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:05:02.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:02 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:02.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:02 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:05:02.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:02 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:02.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:02 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:05:02.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:02 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:02.611 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T14:05:02.611 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T14:05:03.312 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:05:03.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:03 smithi096 ceph-mon[29207]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi096:172.21.15.96=a;smithi097:172.21.15.97=b;smithi151:172.21.15.151=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:05:03.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:03 smithi096 ceph-mon[29207]: Saving service mon spec with placement smithi096:172.21.15.96=a;smithi097:172.21.15.97=b;smithi151:172.21.15.151=c;count:3 2024-04-25T14:05:03.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:03 smithi096 ceph-mon[29207]: Deploying daemon mon.c on smithi151 2024-04-25T14:05:04.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:04 smithi096 ceph-mon[29207]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:06.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:06 smithi096 ceph-mon[29207]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:06.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:06.635+0000 7fad234c4700 1 -- 172.21.15.151:0/145435879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad1c108940 msgr2=0x7fad1c108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:06.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:06.635+0000 7fad234c4700 1 --2- 172.21.15.151:0/145435879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad1c108940 0x7fad1c108d50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fad1400b0a0 tx=0x7fad1400b3b0 comp rx=0 tx=0).stop 2024-04-25T14:05:06.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:06.635+0000 7fad234c4700 1 -- 172.21.15.151:0/145435879 shutdown_connections 2024-04-25T14:05:06.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:06.635+0000 7fad234c4700 1 --2- 172.21.15.151:0/145435879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad1c108940 0x7fad1c108d50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:06.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:06.635+0000 7fad234c4700 1 -- 172.21.15.151:0/145435879 >> 172.21.15.151:0/145435879 conn(0x7fad1c078460 msgr2=0x7fad1c0788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:06.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:06.635+0000 7fad234c4700 1 -- 172.21.15.151:0/145435879 shutdown_connections 2024-04-25T14:05:06.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:06.636+0000 7fad234c4700 1 -- 172.21.15.151:0/145435879 wait complete. 2024-04-25T14:05:06.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:06.636+0000 7fad234c4700 1 Processor -- start 2024-04-25T14:05:06.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:06.636+0000 7fad234c4700 1 -- start start 2024-04-25T14:05:06.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:06.637+0000 7fad234c4700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad1c108940 0x7fad1c198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:06.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:06.637+0000 7fad234c4700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fad1c10ce90 con 0x7fad1c108940 2024-04-25T14:05:06.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:06.637+0000 7fad21260700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad1c108940 0x7fad1c198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:06.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:06.637+0000 7fad21260700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad1c108940 0x7fad1c198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.151:37162/0 (socket says 172.21.15.151:37162) 2024-04-25T14:05:06.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:06.637+0000 7fad21260700 1 -- 172.21.15.151:0/880607812 learned_addr learned my addr 172.21.15.151:0/880607812 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-25T14:05:06.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:06.638+0000 7fad21260700 1 -- 172.21.15.151:0/880607812 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad1c198bb0 con 0x7fad1c108940 2024-04-25T14:05:06.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:06.638+0000 7fad21260700 1 --2- 172.21.15.151:0/880607812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad1c108940 0x7fad1c198670 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fad14015040 tx=0x7fad14009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:06.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:06.638+0000 7fad1a7fc700 1 -- 172.21.15.151:0/880607812 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad14021020 con 0x7fad1c108940 2024-04-25T14:05:06.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:06.638+0000 7fad1a7fc700 1 -- 172.21.15.151:0/880607812 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad14007970 con 0x7fad1c108940 2024-04-25T14:05:06.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:06.638+0000 7fad234c4700 1 -- 172.21.15.151:0/880607812 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad1c198e40 con 0x7fad1c108940 2024-04-25T14:05:06.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:06.638+0000 7fad1a7fc700 1 -- 172.21.15.151:0/880607812 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad14017040 con 0x7fad1c108940 2024-04-25T14:05:06.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:06.638+0000 7fad234c4700 1 -- 172.21.15.151:0/880607812 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad1c19cea0 con 0x7fad1c108940 2024-04-25T14:05:06.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:06.639+0000 7fad234c4700 1 -- 172.21.15.151:0/880607812 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad1c104050 con 0x7fad1c108940 2024-04-25T14:05:06.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:06.640+0000 7fad1a7fc700 1 -- 172.21.15.151:0/880607812 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7fad14024020 con 0x7fad1c108940 2024-04-25T14:05:06.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:06.640+0000 7fad1a7fc700 1 --2- 172.21.15.151:0/880607812 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fad0403d670 0x7fad0403fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:06.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:06.640+0000 7fad1a7fc700 1 -- 172.21.15.151:0/880607812 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fad1400f050 con 0x7fad1c108940 2024-04-25T14:05:06.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:06.640+0000 7fad20a5f700 1 --2- 172.21.15.151:0/880607812 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fad0403d670 0x7fad0403fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:06.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:06.641+0000 7fad20a5f700 1 --2- 172.21.15.151:0/880607812 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fad0403d670 0x7fad0403fb80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fad10006fd0 tx=0x7fad10006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:06.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:06.646+0000 7fad1a7fc700 1 -- 172.21.15.151:0/880607812 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fad1402b020 con 0x7fad1c108940 2024-04-25T14:05:06.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:06.941+0000 7fad234c4700 1 -- 172.21.15.151:0/880607812 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fad1c064cf0 con 0x7fad1c108940 2024-04-25T14:05:06.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:06.942+0000 7fad1a7fc700 1 -- 172.21.15.151:0/880607812 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fad1402e9b0 con 0x7fad1c108940 2024-04-25T14:05:06.942 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-25T14:05:06.942 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":1,"fsid":"59334182-030c-11ef-bc93-c7b262605968","modified":"2024-04-25T14:03:12.788276Z","created":"2024-04-25T14:03:12.788276Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:3300","nonce":0},{"type":"v1","addr":"172.21.15.96:6789","nonce":0}]},"addr":"172.21.15.96:6789/0","public_addr":"172.21.15.96:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T14:05:06.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:06.944+0000 7fad234c4700 1 -- 172.21.15.151:0/880607812 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fad0403d670 msgr2=0x7fad0403fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:06.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:06.944+0000 7fad234c4700 1 --2- 172.21.15.151:0/880607812 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fad0403d670 0x7fad0403fb80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fad10006fd0 tx=0x7fad10006e40 comp rx=0 tx=0).stop 2024-04-25T14:05:06.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:06.944+0000 7fad234c4700 1 -- 172.21.15.151:0/880607812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad1c108940 msgr2=0x7fad1c198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:06.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:06.944+0000 7fad234c4700 1 --2- 172.21.15.151:0/880607812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad1c108940 0x7fad1c198670 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fad14015040 tx=0x7fad14009790 comp rx=0 tx=0).stop 2024-04-25T14:05:06.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:06.944+0000 7fad234c4700 1 -- 172.21.15.151:0/880607812 shutdown_connections 2024-04-25T14:05:06.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:06.944+0000 7fad234c4700 1 --2- 172.21.15.151:0/880607812 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fad0403d670 0x7fad0403fb80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:06.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:06.944+0000 7fad234c4700 1 --2- 172.21.15.151:0/880607812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad1c108940 0x7fad1c198670 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:06.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:06.944+0000 7fad234c4700 1 -- 172.21.15.151:0/880607812 >> 172.21.15.151:0/880607812 conn(0x7fad1c078460 msgr2=0x7fad1c101000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:06.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:06.944+0000 7fad234c4700 1 -- 172.21.15.151:0/880607812 shutdown_connections 2024-04-25T14:05:06.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:06.945+0000 7fad234c4700 1 -- 172.21.15.151:0/880607812 wait complete. 2024-04-25T14:05:06.945 INFO:teuthology.orchestra.run.smithi151.stderr:dumped monmap epoch 1 2024-04-25T14:05:07.090 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 podman[32959]: 2024-04-25 14:05:07.055910823 +0000 UTC m=+0.358662234 container start 412fdde6b2cb4f22fdaf2512bea08a63805959d5ce8da1a7faf300847598c535 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mon-c, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-25T14:05:07.090 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T14:05:07.090 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev), process ceph-mon, pid 2 2024-04-25T14:05:07.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: pidfile_write: ignore empty --pid-file 2024-04-25T14:05:07.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: load: jerasure load: lrc 2024-04-25T14:05:07.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: RocksDB version: 7.9.2 2024-04-25T14:05:07.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Git sha 0 2024-04-25T14:05:07.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Compile date 2024-04-25 02:57:34 2024-04-25T14:05:07.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: DB SUMMARY 2024-04-25T14:05:07.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: DB Session ID: Z75TJGWZDG3J4P0B07CF 2024-04-25T14:05:07.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: CURRENT file: CURRENT 2024-04-25T14:05:07.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T14:05:07.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-25T14:05:07.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-25T14:05:07.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-25T14:05:07.440 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.error_if_exists: 0 2024-04-25T14:05:07.440 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.create_if_missing: 0 2024-04-25T14:05:07.440 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.paranoid_checks: 1 2024-04-25T14:05:07.440 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T14:05:07.440 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T14:05:07.440 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T14:05:07.440 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.env: 0x55dfa09ae500 2024-04-25T14:05:07.440 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T14:05:07.440 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.info_log: 0x55dfa19b1fc0 2024-04-25T14:05:07.440 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T14:05:07.440 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.statistics: (nil) 2024-04-25T14:05:07.440 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.use_fsync: 0 2024-04-25T14:05:07.440 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.max_log_file_size: 0 2024-04-25T14:05:07.440 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T14:05:07.441 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T14:05:07.441 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T14:05:07.441 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T14:05:07.441 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.allow_fallocate: 1 2024-04-25T14:05:07.441 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T14:05:07.441 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T14:05:07.441 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.use_direct_reads: 0 2024-04-25T14:05:07.441 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T14:05:07.441 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T14:05:07.441 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.db_log_dir: 2024-04-25T14:05:07.441 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.wal_dir: 2024-04-25T14:05:07.441 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T14:05:07.442 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T14:05:07.442 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T14:05:07.442 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T14:05:07.442 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T14:05:07.442 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T14:05:07.442 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T14:05:07.442 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T14:05:07.442 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.write_buffer_manager: 0x55dfa19828c0 2024-04-25T14:05:07.442 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T14:05:07.442 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T14:05:07.442 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T14:05:07.442 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T14:05:07.442 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T14:05:07.442 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T14:05:07.443 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T14:05:07.443 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T14:05:07.443 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.unordered_write: 0 2024-04-25T14:05:07.443 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T14:05:07.443 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T14:05:07.443 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T14:05:07.443 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T14:05:07.443 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.row_cache: None 2024-04-25T14:05:07.443 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.wal_filter: None 2024-04-25T14:05:07.443 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T14:05:07.443 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T14:05:07.443 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.two_write_queues: 0 2024-04-25T14:05:07.443 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T14:05:07.443 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.wal_compression: 0 2024-04-25T14:05:07.444 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.atomic_flush: 0 2024-04-25T14:05:07.444 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T14:05:07.444 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T14:05:07.444 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T14:05:07.444 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.log_readahead_size: 0 2024-04-25T14:05:07.444 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T14:05:07.444 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T14:05:07.444 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T14:05:07.444 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T14:05:07.444 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T14:05:07.444 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T14:05:07.444 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T14:05:07.444 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.max_background_jobs: 2 2024-04-25T14:05:07.444 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.max_background_compactions: -1 2024-04-25T14:05:07.446 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.max_subcompactions: 1 2024-04-25T14:05:07.446 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T14:05:07.446 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T14:05:07.446 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T14:05:07.446 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T14:05:07.446 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T14:05:07.446 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T14:05:07.446 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T14:05:07.447 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T14:05:07.447 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.max_open_files: -1 2024-04-25T14:05:07.447 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T14:05:07.447 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T14:05:07.447 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T14:05:07.447 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T14:05:07.447 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.max_background_flushes: -1 2024-04-25T14:05:07.447 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Compression algorithms supported: 2024-04-25T14:05:07.447 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T14:05:07.447 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: kZSTD supported: 0 2024-04-25T14:05:07.447 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: kXpressCompression supported: 0 2024-04-25T14:05:07.447 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T14:05:07.447 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: kZlibCompression supported: 1 2024-04-25T14:05:07.447 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: kSnappyCompression supported: 1 2024-04-25T14:05:07.448 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: kLZ4Compression supported: 1 2024-04-25T14:05:07.448 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: kBZip2Compression supported: 0 2024-04-25T14:05:07.448 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T14:05:07.448 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T14:05:07.448 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-25T14:05:07.448 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T14:05:07.448 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T14:05:07.448 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.merge_operator: 2024-04-25T14:05:07.448 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.compaction_filter: None 2024-04-25T14:05:07.448 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T14:05:07.448 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T14:05:07.448 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T14:05:07.448 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T14:05:07.448 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55dfa1ab81a0) 2024-04-25T14:05:07.448 INFO:journalctl@ceph.mon.c.smithi151.stdout: cache_index_and_filter_blocks: 1 2024-04-25T14:05:07.449 INFO:journalctl@ceph.mon.c.smithi151.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T14:05:07.449 INFO:journalctl@ceph.mon.c.smithi151.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T14:05:07.449 INFO:journalctl@ceph.mon.c.smithi151.stdout: pin_top_level_index_and_filter: 1 2024-04-25T14:05:07.449 INFO:journalctl@ceph.mon.c.smithi151.stdout: index_type: 0 2024-04-25T14:05:07.449 INFO:journalctl@ceph.mon.c.smithi151.stdout: data_block_index_type: 0 2024-04-25T14:05:07.449 INFO:journalctl@ceph.mon.c.smithi151.stdout: index_shortening: 1 2024-04-25T14:05:07.449 INFO:journalctl@ceph.mon.c.smithi151.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T14:05:07.449 INFO:journalctl@ceph.mon.c.smithi151.stdout: checksum: 4 2024-04-25T14:05:07.449 INFO:journalctl@ceph.mon.c.smithi151.stdout: no_block_cache: 0 2024-04-25T14:05:07.449 INFO:journalctl@ceph.mon.c.smithi151.stdout: block_cache: 0x55dfa19b51f0 2024-04-25T14:05:07.449 INFO:journalctl@ceph.mon.c.smithi151.stdout: block_cache_name: BinnedLRUCache 2024-04-25T14:05:07.449 INFO:journalctl@ceph.mon.c.smithi151.stdout: block_cache_options: 2024-04-25T14:05:07.449 INFO:journalctl@ceph.mon.c.smithi151.stdout: capacity : 536870912 2024-04-25T14:05:07.449 INFO:journalctl@ceph.mon.c.smithi151.stdout: num_shard_bits : 4 2024-04-25T14:05:07.449 INFO:journalctl@ceph.mon.c.smithi151.stdout: strict_capacity_limit : 0 2024-04-25T14:05:07.450 INFO:journalctl@ceph.mon.c.smithi151.stdout: high_pri_pool_ratio: 0.000 2024-04-25T14:05:07.450 INFO:journalctl@ceph.mon.c.smithi151.stdout: block_cache_compressed: (nil) 2024-04-25T14:05:07.450 INFO:journalctl@ceph.mon.c.smithi151.stdout: persistent_cache: (nil) 2024-04-25T14:05:07.450 INFO:journalctl@ceph.mon.c.smithi151.stdout: block_size: 4096 2024-04-25T14:05:07.450 INFO:journalctl@ceph.mon.c.smithi151.stdout: block_size_deviation: 10 2024-04-25T14:05:07.450 INFO:journalctl@ceph.mon.c.smithi151.stdout: block_restart_interval: 16 2024-04-25T14:05:07.450 INFO:journalctl@ceph.mon.c.smithi151.stdout: index_block_restart_interval: 1 2024-04-25T14:05:07.450 INFO:journalctl@ceph.mon.c.smithi151.stdout: metadata_block_size: 4096 2024-04-25T14:05:07.450 INFO:journalctl@ceph.mon.c.smithi151.stdout: partition_filters: 0 2024-04-25T14:05:07.450 INFO:journalctl@ceph.mon.c.smithi151.stdout: use_delta_encoding: 1 2024-04-25T14:05:07.450 INFO:journalctl@ceph.mon.c.smithi151.stdout: filter_policy: bloomfilter 2024-04-25T14:05:07.450 INFO:journalctl@ceph.mon.c.smithi151.stdout: whole_key_filtering: 1 2024-04-25T14:05:07.450 INFO:journalctl@ceph.mon.c.smithi151.stdout: verify_compression: 0 2024-04-25T14:05:07.450 INFO:journalctl@ceph.mon.c.smithi151.stdout: read_amp_bytes_per_bit: 0 2024-04-25T14:05:07.451 INFO:journalctl@ceph.mon.c.smithi151.stdout: format_version: 5 2024-04-25T14:05:07.451 INFO:journalctl@ceph.mon.c.smithi151.stdout: enable_index_compression: 1 2024-04-25T14:05:07.451 INFO:journalctl@ceph.mon.c.smithi151.stdout: block_align: 0 2024-04-25T14:05:07.451 INFO:journalctl@ceph.mon.c.smithi151.stdout: max_auto_readahead_size: 262144 2024-04-25T14:05:07.451 INFO:journalctl@ceph.mon.c.smithi151.stdout: prepopulate_block_cache: 0 2024-04-25T14:05:07.451 INFO:journalctl@ceph.mon.c.smithi151.stdout: initial_auto_readahead_size: 8192 2024-04-25T14:05:07.451 INFO:journalctl@ceph.mon.c.smithi151.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T14:05:07.451 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T14:05:07.451 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T14:05:07.451 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.compression: NoCompression 2024-04-25T14:05:07.451 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T14:05:07.451 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T14:05:07.451 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T14:05:07.451 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.num_levels: 7 2024-04-25T14:05:07.451 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T14:05:07.451 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T14:05:07.452 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T14:05:07.452 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T14:05:07.452 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T14:05:07.452 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T14:05:07.452 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T14:05:07.452 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T14:05:07.452 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T14:05:07.452 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T14:05:07.452 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T14:05:07.452 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T14:05:07.452 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T14:05:07.452 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T14:05:07.452 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T14:05:07.452 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T14:05:07.453 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T14:05:07.453 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T14:05:07.453 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T14:05:07.453 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T14:05:07.453 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T14:05:07.453 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T14:05:07.453 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T14:05:07.453 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T14:05:07.453 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T14:05:07.453 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T14:05:07.453 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T14:05:07.453 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T14:05:07.453 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T14:05:07.453 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T14:05:07.454 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T14:05:07.454 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T14:05:07.454 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T14:05:07.454 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T14:05:07.454 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T14:05:07.454 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T14:05:07.454 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T14:05:07.454 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T14:05:07.454 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T14:05:07.454 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T14:05:07.454 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T14:05:07.454 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T14:05:07.454 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T14:05:07.454 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T14:05:07.455 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T14:05:07.455 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T14:05:07.455 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T14:05:07.455 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T14:05:07.455 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T14:05:07.455 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T14:05:07.455 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T14:05:07.455 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T14:05:07.455 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T14:05:07.455 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T14:05:07.455 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.inplace_update_support: 0 2024-04-25T14:05:07.455 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T14:05:07.455 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T14:05:07.455 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T14:05:07.456 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T14:05:07.456 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.bloom_locality: 0 2024-04-25T14:05:07.456 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.max_successive_merges: 0 2024-04-25T14:05:07.456 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T14:05:07.456 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T14:05:07.456 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T14:05:07.456 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T14:05:07.456 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.ttl: 2592000 2024-04-25T14:05:07.456 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T14:05:07.456 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T14:05:07.456 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T14:05:07.456 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.enable_blob_files: false 2024-04-25T14:05:07.456 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.min_blob_size: 0 2024-04-25T14:05:07.456 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T14:05:07.456 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T14:05:07.457 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T14:05:07.457 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T14:05:07.457 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T14:05:07.457 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T14:05:07.457 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T14:05:07.457 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T14:05:07.457 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-25T14:05:07.457 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-25T14:05:07.457 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: c879b494-821d-455c-80e4-162d83fcf0a8 2024-04-25T14:05:07.457 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714053907097701, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-25T14:05:07.457 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-25T14:05:07.457 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714053907098299, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714053907, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "c879b494-821d-455c-80e4-162d83fcf0a8", "db_session_id": "Z75TJGWZDG3J4P0B07CF", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-25T14:05:07.457 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714053907098404, "job": 1, "event": "recovery_finished"} 2024-04-25T14:05:07.457 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-25T14:05:07.458 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T14:05:07.458 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55dfa1999c00 2024-04-25T14:05:07.458 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: DB pointer 0x55dfa678c000 2024-04-25T14:05:07.458 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T14:05:07.458 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T14:05:07.458 INFO:journalctl@ceph.mon.c.smithi151.stdout: ** DB Stats ** 2024-04-25T14:05:07.458 INFO:journalctl@ceph.mon.c.smithi151.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T14:05:07.458 INFO:journalctl@ceph.mon.c.smithi151.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T14:05:07.458 INFO:journalctl@ceph.mon.c.smithi151.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T14:05:07.458 INFO:journalctl@ceph.mon.c.smithi151.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T14:05:07.458 INFO:journalctl@ceph.mon.c.smithi151.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T14:05:07.458 INFO:journalctl@ceph.mon.c.smithi151.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T14:05:07.458 INFO:journalctl@ceph.mon.c.smithi151.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T14:05:07.458 INFO:journalctl@ceph.mon.c.smithi151.stdout: 2024-04-25T14:05:07.459 INFO:journalctl@ceph.mon.c.smithi151.stdout: ** Compaction Stats [default] ** 2024-04-25T14:05:07.459 INFO:journalctl@ceph.mon.c.smithi151.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T14:05:07.459 INFO:journalctl@ceph.mon.c.smithi151.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T14:05:07.459 INFO:journalctl@ceph.mon.c.smithi151.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T14:05:07.459 INFO:journalctl@ceph.mon.c.smithi151.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T14:05:07.459 INFO:journalctl@ceph.mon.c.smithi151.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T14:05:07.459 INFO:journalctl@ceph.mon.c.smithi151.stdout: 2024-04-25T14:05:07.459 INFO:journalctl@ceph.mon.c.smithi151.stdout: ** Compaction Stats [default] ** 2024-04-25T14:05:07.459 INFO:journalctl@ceph.mon.c.smithi151.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T14:05:07.459 INFO:journalctl@ceph.mon.c.smithi151.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T14:05:07.459 INFO:journalctl@ceph.mon.c.smithi151.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T14:05:07.459 INFO:journalctl@ceph.mon.c.smithi151.stdout: 2024-04-25T14:05:07.459 INFO:journalctl@ceph.mon.c.smithi151.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T14:05:07.460 INFO:journalctl@ceph.mon.c.smithi151.stdout: 2024-04-25T14:05:07.460 INFO:journalctl@ceph.mon.c.smithi151.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T14:05:07.460 INFO:journalctl@ceph.mon.c.smithi151.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T14:05:07.460 INFO:journalctl@ceph.mon.c.smithi151.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T14:05:07.460 INFO:journalctl@ceph.mon.c.smithi151.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T14:05:07.460 INFO:journalctl@ceph.mon.c.smithi151.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T14:05:07.460 INFO:journalctl@ceph.mon.c.smithi151.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T14:05:07.460 INFO:journalctl@ceph.mon.c.smithi151.stdout: Cumulative compaction: 0.00 GB write, 0.46 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T14:05:07.460 INFO:journalctl@ceph.mon.c.smithi151.stdout: Interval compaction: 0.00 GB write, 0.46 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T14:05:07.460 INFO:journalctl@ceph.mon.c.smithi151.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T14:05:07.460 INFO:journalctl@ceph.mon.c.smithi151.stdout: Block cache BinnedLRUCache@0x55dfa19b51f0#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-04-25T14:05:07.460 INFO:journalctl@ceph.mon.c.smithi151.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-25T14:05:07.460 INFO:journalctl@ceph.mon.c.smithi151.stdout: 2024-04-25T14:05:07.460 INFO:journalctl@ceph.mon.c.smithi151.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T14:05:07.461 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-25T14:05:07.461 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: using public_addr v2:172.21.15.151:0/0 -> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] 2024-04-25T14:05:07.461 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: starting mon.c rank -1 at public addrs [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] at bind addrs [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 59334182-030c-11ef-bc93-c7b262605968 2024-04-25T14:05:07.461 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: mon.c@-1(???) e0 preinit fsid 59334182-030c-11ef-bc93-c7b262605968 2024-04-25T14:05:07.461 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: mon.c@-1(synchronizing).mds e1 new map 2024-04-25T14:05:07.461 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-25T14:05:07.461 INFO:journalctl@ceph.mon.c.smithi151.stdout: e1 2024-04-25T14:05:07.461 INFO:journalctl@ceph.mon.c.smithi151.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T14:05:07.461 INFO:journalctl@ceph.mon.c.smithi151.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T14:05:07.461 INFO:journalctl@ceph.mon.c.smithi151.stdout: legacy client fscid: -1 2024-04-25T14:05:07.461 INFO:journalctl@ceph.mon.c.smithi151.stdout: 2024-04-25T14:05:07.461 INFO:journalctl@ceph.mon.c.smithi151.stdout: No filesystems configured 2024-04-25T14:05:07.461 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-25T14:05:07.462 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-25T14:05:07.462 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-25T14:05:07.462 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-25T14:05:07.462 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-25T14:05:07.462 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-25T14:05:07.462 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T14:05:07.462 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T14:05:07.462 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T14:05:07.462 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T14:05:07.462 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:07.462 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi151", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:05:07.462 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:07.462 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:07.462 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:07.463 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:07.463 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi097", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:05:07.463 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:07.463 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:05:07.463 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: Deploying cephadm binary to smithi151 2024-04-25T14:05:07.463 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: Updating smithi097:/etc/ceph/ceph.conf 2024-04-25T14:05:07.463 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: Updating smithi097:/var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:05:07.463 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:07.463 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:07.463 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:05:07.463 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:07.463 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:07.463 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:07.463 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:07.464 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:05:07.464 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: Updating smithi097:/var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T14:05:07.464 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: Added host smithi151 2024-04-25T14:05:07.464 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:07.464 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:07.464 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:07.464 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:07.464 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:05:07.464 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:07.464 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/144008411' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T14:05:07.464 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/144008411' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T14:05:07.464 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T14:05:07.464 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:07.465 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:07.465 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:07.465 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:07.465 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:07.465 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi151", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:05:07.465 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:07.465 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:05:07.465 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: Updating smithi151:/etc/ceph/ceph.conf 2024-04-25T14:05:07.465 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: Updating smithi151:/var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:05:07.465 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: Updating smithi151:/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:05:07.465 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: Updating smithi151:/var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T14:05:07.465 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:07.465 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:07.465 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:07.465 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:07.466 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:07.466 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:05:07.466 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:07.466 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:05:07.466 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:07.466 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:05:07.466 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:07.466 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi096:172.21.15.96=a;smithi097:172.21.15.97=b;smithi151:172.21.15.151=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:05:07.466 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: Saving service mon spec with placement smithi096:172.21.15.96=a;smithi097:172.21.15.97=b;smithi151:172.21.15.151=c;count:3 2024-04-25T14:05:07.466 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: Deploying daemon mon.c on smithi151 2024-04-25T14:05:07.466 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:07.466 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:07.466 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 ceph-mon[32990]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-25T14:05:07.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 bash[32959]: 412fdde6b2cb4f22fdaf2512bea08a63805959d5ce8da1a7faf300847598c535 2024-04-25T14:05:07.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:07 smithi151 systemd[1]: Started Ceph mon.c for 59334182-030c-11ef-bc93-c7b262605968. 2024-04-25T14:05:08.570 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T14:05:08.570 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T14:05:08.814 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:05:12.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:12 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:05:12.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:12 smithi096 ceph-mon[29207]: mon.a calling monitor election 2024-04-25T14:05:12.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:12 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:05:12.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:12 smithi096 ceph-mon[29207]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:12.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:12 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:05:12.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:12 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:05:12.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:12 smithi096 ceph-mon[29207]: mon.c calling monitor election 2024-04-25T14:05:12.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:12 smithi096 ceph-mon[29207]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:12.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:12 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:05:12.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:12 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:05:12.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:12 smithi096 ceph-mon[29207]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:12.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:12 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:05:12.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:12 smithi096 ceph-mon[29207]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T14:05:12.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:12 smithi096 ceph-mon[29207]: monmap epoch 2 2024-04-25T14:05:12.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:12 smithi096 ceph-mon[29207]: fsid 59334182-030c-11ef-bc93-c7b262605968 2024-04-25T14:05:12.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:12 smithi096 ceph-mon[29207]: last_changed 2024-04-25T14:05:07.125083+0000 2024-04-25T14:05:12.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:12 smithi096 ceph-mon[29207]: created 2024-04-25T14:03:12.788276+0000 2024-04-25T14:05:12.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:12 smithi096 ceph-mon[29207]: min_mon_release 19 (squid) 2024-04-25T14:05:12.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:12 smithi096 ceph-mon[29207]: election_strategy: 1 2024-04-25T14:05:12.611 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:12 smithi096 ceph-mon[29207]: 0: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.a 2024-04-25T14:05:12.611 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:12 smithi096 ceph-mon[29207]: 1: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.c 2024-04-25T14:05:12.611 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:12 smithi096 ceph-mon[29207]: fsmap 2024-04-25T14:05:12.611 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:12 smithi096 ceph-mon[29207]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T14:05:12.611 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:12 smithi096 ceph-mon[29207]: mgrmap e13: x(active, since 50s) 2024-04-25T14:05:12.611 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:12 smithi096 ceph-mon[29207]: overall HEALTH_OK 2024-04-25T14:05:12.611 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:12 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:12.611 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:12 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:12.611 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:12 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:12.611 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:12 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:05:12.611 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:12 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:12.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:12 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:05:12.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:12 smithi151 ceph-mon[32990]: mon.a calling monitor election 2024-04-25T14:05:12.689 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:12 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:05:12.689 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:12 smithi151 ceph-mon[32990]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:12.689 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:12 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:05:12.689 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:12 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:05:12.689 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:12 smithi151 ceph-mon[32990]: mon.c calling monitor election 2024-04-25T14:05:12.689 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:12 smithi151 ceph-mon[32990]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:12.690 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:12 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:05:12.690 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:12 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:05:12.690 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:12 smithi151 ceph-mon[32990]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:12.690 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:12 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:05:12.690 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:12 smithi151 ceph-mon[32990]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T14:05:12.690 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:12 smithi151 ceph-mon[32990]: monmap epoch 2 2024-04-25T14:05:12.690 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:12 smithi151 ceph-mon[32990]: fsid 59334182-030c-11ef-bc93-c7b262605968 2024-04-25T14:05:12.691 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:12 smithi151 ceph-mon[32990]: last_changed 2024-04-25T14:05:07.125083+0000 2024-04-25T14:05:12.691 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:12 smithi151 ceph-mon[32990]: created 2024-04-25T14:03:12.788276+0000 2024-04-25T14:05:12.691 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:12 smithi151 ceph-mon[32990]: min_mon_release 19 (squid) 2024-04-25T14:05:12.691 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:12 smithi151 ceph-mon[32990]: election_strategy: 1 2024-04-25T14:05:12.691 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:12 smithi151 ceph-mon[32990]: 0: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.a 2024-04-25T14:05:12.691 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:12 smithi151 ceph-mon[32990]: 1: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.c 2024-04-25T14:05:12.691 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:12 smithi151 ceph-mon[32990]: fsmap 2024-04-25T14:05:12.691 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:12 smithi151 ceph-mon[32990]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T14:05:12.691 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:12 smithi151 ceph-mon[32990]: mgrmap e13: x(active, since 50s) 2024-04-25T14:05:12.692 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:12 smithi151 ceph-mon[32990]: overall HEALTH_OK 2024-04-25T14:05:12.692 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:12 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:12.694 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:12 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:12.694 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:12 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:12.694 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:12 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:05:12.694 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:12 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:13.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.181+0000 7f6465c87700 1 -- 172.21.15.151:0/1569606013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f644c0035c0 msgr2=0x7f644c005aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:13.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.181+0000 7f6465c87700 1 --2- 172.21.15.151:0/1569606013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f644c0035c0 0x7f644c005aa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6460103a20 tx=0x7f645000feb0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.181+0000 7f6465c87700 1 -- 172.21.15.151:0/1569606013 shutdown_connections 2024-04-25T14:05:13.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.181+0000 7f6465c87700 1 --2- 172.21.15.151:0/1569606013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f644c0035c0 0x7f644c005aa0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.181+0000 7f6465c87700 1 -- 172.21.15.151:0/1569606013 >> 172.21.15.151:0/1569606013 conn(0x7f64600ff650 msgr2=0x7f6460101aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:13.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.181+0000 7f6465c87700 1 -- 172.21.15.151:0/1569606013 shutdown_connections 2024-04-25T14:05:13.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.181+0000 7f6465c87700 1 -- 172.21.15.151:0/1569606013 wait complete. 2024-04-25T14:05:13.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.182+0000 7f6465c87700 1 Processor -- start 2024-04-25T14:05:13.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.182+0000 7f6465c87700 1 -- start start 2024-04-25T14:05:13.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.183+0000 7f6465c87700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f644c0035c0 0x7f646019f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.183+0000 7f6465c87700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f646019fae0 0x7f6460199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.183+0000 7f6465c87700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f646010bad0 con 0x7f646019fae0 2024-04-25T14:05:13.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.183+0000 7f6465c87700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f646010b950 con 0x7f644c0035c0 2024-04-25T14:05:13.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.183+0000 7f645f7fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f644c0035c0 0x7f646019f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.183+0000 7f645f7fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f644c0035c0 0x7f646019f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39734/0 (socket says 172.21.15.151:39734) 2024-04-25T14:05:13.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.183+0000 7f645f7fe700 1 -- 172.21.15.151:0/1944139603 learned_addr learned my addr 172.21.15.151:0/1944139603 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-25T14:05:13.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.183+0000 7f6457fff700 1 --2- 172.21.15.151:0/1944139603 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f646019fae0 0x7f6460199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.183+0000 7f645f7fe700 1 -- 172.21.15.151:0/1944139603 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f644c0035c0 msgr2=0x7f646019f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-25T14:05:13.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.183+0000 7f645f7fe700 1 -- 172.21.15.151:0/1944139603 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f644c0035c0 msgr2=0x7f646019f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T14:05:13.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.183+0000 7f645f7fe700 1 --2- 172.21.15.151:0/1944139603 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f644c0035c0 0x7f646019f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T14:05:13.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.183+0000 7f645f7fe700 1 --2- 172.21.15.151:0/1944139603 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f644c0035c0 0x7f646019f5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:05:13.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.183+0000 7f6457fff700 1 -- 172.21.15.151:0/1944139603 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f644c0035c0 msgr2=0x7f646019f5a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:05:13.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.183+0000 7f6457fff700 1 --2- 172.21.15.151:0/1944139603 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f644c0035c0 0x7f646019f5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.183+0000 7f6457fff700 1 -- 172.21.15.151:0/1944139603 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64601a0080 con 0x7f646019fae0 2024-04-25T14:05:13.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.184+0000 7f6457fff700 1 --2- 172.21.15.151:0/1944139603 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f646019fae0 0x7f6460199730 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f644800b4d0 tx=0x7f644800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:13.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.184+0000 7f645d7fa700 1 -- 172.21.15.151:0/1944139603 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f6448011630 con 0x7f646019fae0 2024-04-25T14:05:13.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.184+0000 7f645d7fa700 1 -- 172.21.15.151:0/1944139603 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6448011c80 con 0x7f646019fae0 2024-04-25T14:05:13.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.184+0000 7f6465c87700 1 -- 172.21.15.151:0/1944139603 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6460199d30 con 0x7f646019fae0 2024-04-25T14:05:13.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.184+0000 7f645d7fa700 1 -- 172.21.15.151:0/1944139603 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f6448010da0 con 0x7f646019fae0 2024-04-25T14:05:13.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.184+0000 7f6465c87700 1 -- 172.21.15.151:0/1944139603 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f646019a2c0 con 0x7f646019fae0 2024-04-25T14:05:13.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.185+0000 7f645d7fa700 1 -- 172.21.15.151:0/1944139603 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f6448018020 con 0x7f646019fae0 2024-04-25T14:05:13.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.185+0000 7f6465c87700 1 -- 172.21.15.151:0/1944139603 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f646019a5d0 con 0x7f646019fae0 2024-04-25T14:05:13.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.185+0000 7f645d7fa700 1 --2- 172.21.15.151:0/1944139603 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f644003d3b0 0x7f644003f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.185+0000 7f645d7fa700 1 -- 172.21.15.151:0/1944139603 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6448051070 con 0x7f646019fae0 2024-04-25T14:05:13.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.185+0000 7f645f7fe700 1 --2- 172.21.15.151:0/1944139603 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f644003d3b0 0x7f644003f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.186+0000 7f645f7fe700 1 --2- 172.21.15.151:0/1944139603 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f644003d3b0 0x7f644003f8c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f645000fa60 tx=0x7f6450005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:13.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.191+0000 7f645d7fa700 1 -- 172.21.15.151:0/1944139603 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f644801f020 con 0x7f646019fae0 2024-04-25T14:05:13.437 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:13 smithi151 ceph-mon[32990]: Deploying daemon mon.b on smithi097 2024-04-25T14:05:13.437 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:13 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:05:13.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.484+0000 7f6465c87700 1 -- 172.21.15.151:0/1944139603 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f6460064d20 con 0x7f646019fae0 2024-04-25T14:05:13.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.485+0000 7f645d7fa700 1 -- 172.21.15.151:0/1944139603 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f6460064d20 con 0x7f646019fae0 2024-04-25T14:05:13.484 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-25T14:05:13.485 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":2,"fsid":"59334182-030c-11ef-bc93-c7b262605968","modified":"2024-04-25T14:05:07.125083Z","created":"2024-04-25T14:03:12.788276Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:3300","nonce":0},{"type":"v1","addr":"172.21.15.96:6789","nonce":0}]},"addr":"172.21.15.96:6789/0","public_addr":"172.21.15.96:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:3300","nonce":0},{"type":"v1","addr":"172.21.15.151:6789","nonce":0}]},"addr":"172.21.15.151:6789/0","public_addr":"172.21.15.151:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T14:05:13.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.487+0000 7f6465c87700 1 -- 172.21.15.151:0/1944139603 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f644003d3b0 msgr2=0x7f644003f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:13.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.487+0000 7f6465c87700 1 --2- 172.21.15.151:0/1944139603 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f644003d3b0 0x7f644003f8c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f645000fa60 tx=0x7f6450005960 comp rx=0 tx=0).stop 2024-04-25T14:05:13.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.487+0000 7f6465c87700 1 -- 172.21.15.151:0/1944139603 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f646019fae0 msgr2=0x7f6460199730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:13.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.487+0000 7f6465c87700 1 --2- 172.21.15.151:0/1944139603 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f646019fae0 0x7f6460199730 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f644800b4d0 tx=0x7f644800b890 comp rx=0 tx=0).stop 2024-04-25T14:05:13.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.487+0000 7f6465c87700 1 -- 172.21.15.151:0/1944139603 shutdown_connections 2024-04-25T14:05:13.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.487+0000 7f6465c87700 1 --2- 172.21.15.151:0/1944139603 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f644003d3b0 0x7f644003f8c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.488 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.487+0000 7f6465c87700 1 --2- 172.21.15.151:0/1944139603 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f644c0035c0 0x7f646019f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.488 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.487+0000 7f6465c87700 1 --2- 172.21.15.151:0/1944139603 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f646019fae0 0x7f6460199730 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.488 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.487+0000 7f6465c87700 1 -- 172.21.15.151:0/1944139603 >> 172.21.15.151:0/1944139603 conn(0x7f64600ff650 msgr2=0x7f64601070d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:13.488 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.487+0000 7f6465c87700 1 -- 172.21.15.151:0/1944139603 shutdown_connections 2024-04-25T14:05:13.488 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:13.487+0000 7f6465c87700 1 -- 172.21.15.151:0/1944139603 wait complete. 2024-04-25T14:05:13.488 INFO:teuthology.orchestra.run.smithi151.stderr:dumped monmap epoch 2 2024-04-25T14:05:13.608 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:13 smithi096 ceph-mon[29207]: Deploying daemon mon.b on smithi097 2024-04-25T14:05:13.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:13 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:05:14.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:14 smithi151 ceph-mon[32990]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:14.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:14 smithi151 ceph-mon[32990]: from='client.? 172.21.15.151:0/1944139603' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T14:05:14.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:14 smithi096 ceph-mon[29207]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:14.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:14 smithi096 ceph-mon[29207]: from='client.? 172.21.15.151:0/1944139603' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T14:05:15.093 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T14:05:15.093 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T14:05:15.337 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:05:15.987 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 podman[32700]: 2024-04-25 14:05:15.954038961 +0000 UTC m=+0.342732712 container start affb5a16a9fcac0a7229a99273711c799f934e2c4e9ee9523c6a3cc25df5d153 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mon-b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-25T14:05:16.286 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T14:05:16.286 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev), process ceph-mon, pid 2 2024-04-25T14:05:16.286 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: pidfile_write: ignore empty --pid-file 2024-04-25T14:05:16.287 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: load: jerasure load: lrc 2024-04-25T14:05:16.287 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: RocksDB version: 7.9.2 2024-04-25T14:05:16.287 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Git sha 0 2024-04-25T14:05:16.287 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Compile date 2024-04-25 02:57:34 2024-04-25T14:05:16.287 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: DB SUMMARY 2024-04-25T14:05:16.287 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: DB Session ID: WIR8HRWWKJ3V76G6HWOU 2024-04-25T14:05:16.287 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: CURRENT file: CURRENT 2024-04-25T14:05:16.287 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T14:05:16.287 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-25T14:05:16.287 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-04-25T14:05:16.287 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-04-25T14:05:16.287 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.error_if_exists: 0 2024-04-25T14:05:16.287 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.create_if_missing: 0 2024-04-25T14:05:16.287 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.paranoid_checks: 1 2024-04-25T14:05:16.288 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T14:05:16.288 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T14:05:16.288 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T14:05:16.288 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.env: 0x55d1d0ae1500 2024-04-25T14:05:16.288 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T14:05:16.288 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.info_log: 0x55d1d30a7fc0 2024-04-25T14:05:16.288 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T14:05:16.288 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.statistics: (nil) 2024-04-25T14:05:16.288 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.use_fsync: 0 2024-04-25T14:05:16.288 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.max_log_file_size: 0 2024-04-25T14:05:16.288 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T14:05:16.288 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T14:05:16.288 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T14:05:16.289 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T14:05:16.289 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.allow_fallocate: 1 2024-04-25T14:05:16.289 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T14:05:16.289 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T14:05:16.289 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.use_direct_reads: 0 2024-04-25T14:05:16.289 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T14:05:16.289 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T14:05:16.289 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.db_log_dir: 2024-04-25T14:05:16.289 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.wal_dir: 2024-04-25T14:05:16.289 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T14:05:16.291 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T14:05:16.291 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T14:05:16.291 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T14:05:16.291 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T14:05:16.291 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T14:05:16.291 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T14:05:16.291 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T14:05:16.291 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.write_buffer_manager: 0x55d1d30788c0 2024-04-25T14:05:16.291 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T14:05:16.291 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T14:05:16.291 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T14:05:16.291 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T14:05:16.291 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T14:05:16.291 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T14:05:16.292 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T14:05:16.292 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T14:05:16.292 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.unordered_write: 0 2024-04-25T14:05:16.292 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T14:05:16.292 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T14:05:16.292 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T14:05:16.292 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T14:05:16.292 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.row_cache: None 2024-04-25T14:05:16.292 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.wal_filter: None 2024-04-25T14:05:16.292 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T14:05:16.292 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T14:05:16.292 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.two_write_queues: 0 2024-04-25T14:05:16.292 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T14:05:16.293 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.wal_compression: 0 2024-04-25T14:05:16.293 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.atomic_flush: 0 2024-04-25T14:05:16.293 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T14:05:16.293 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T14:05:16.293 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T14:05:16.293 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.log_readahead_size: 0 2024-04-25T14:05:16.293 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T14:05:16.293 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T14:05:16.293 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T14:05:16.293 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T14:05:16.293 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T14:05:16.293 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T14:05:16.293 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T14:05:16.293 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.max_background_jobs: 2 2024-04-25T14:05:16.293 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.max_background_compactions: -1 2024-04-25T14:05:16.294 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.max_subcompactions: 1 2024-04-25T14:05:16.294 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T14:05:16.294 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T14:05:16.294 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T14:05:16.294 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T14:05:16.294 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T14:05:16.294 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T14:05:16.294 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T14:05:16.294 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T14:05:16.294 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.max_open_files: -1 2024-04-25T14:05:16.294 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T14:05:16.294 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T14:05:16.294 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T14:05:16.295 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T14:05:16.295 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.max_background_flushes: -1 2024-04-25T14:05:16.295 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Compression algorithms supported: 2024-04-25T14:05:16.295 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T14:05:16.295 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: kZSTD supported: 0 2024-04-25T14:05:16.295 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: kXpressCompression supported: 0 2024-04-25T14:05:16.295 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T14:05:16.295 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: kZlibCompression supported: 1 2024-04-25T14:05:16.295 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: kSnappyCompression supported: 1 2024-04-25T14:05:16.295 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: kLZ4Compression supported: 1 2024-04-25T14:05:16.295 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: kBZip2Compression supported: 0 2024-04-25T14:05:16.295 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T14:05:16.295 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T14:05:16.295 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-04-25T14:05:16.296 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T14:05:16.296 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T14:05:16.296 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.merge_operator: 2024-04-25T14:05:16.296 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.compaction_filter: None 2024-04-25T14:05:16.296 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T14:05:16.296 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T14:05:16.296 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T14:05:16.296 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T14:05:16.296 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d1d31ae1a0) 2024-04-25T14:05:16.296 INFO:journalctl@ceph.mon.b.smithi097.stdout: cache_index_and_filter_blocks: 1 2024-04-25T14:05:16.296 INFO:journalctl@ceph.mon.b.smithi097.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T14:05:16.296 INFO:journalctl@ceph.mon.b.smithi097.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T14:05:16.296 INFO:journalctl@ceph.mon.b.smithi097.stdout: pin_top_level_index_and_filter: 1 2024-04-25T14:05:16.296 INFO:journalctl@ceph.mon.b.smithi097.stdout: index_type: 0 2024-04-25T14:05:16.297 INFO:journalctl@ceph.mon.b.smithi097.stdout: data_block_index_type: 0 2024-04-25T14:05:16.297 INFO:journalctl@ceph.mon.b.smithi097.stdout: index_shortening: 1 2024-04-25T14:05:16.297 INFO:journalctl@ceph.mon.b.smithi097.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T14:05:16.297 INFO:journalctl@ceph.mon.b.smithi097.stdout: checksum: 4 2024-04-25T14:05:16.297 INFO:journalctl@ceph.mon.b.smithi097.stdout: no_block_cache: 0 2024-04-25T14:05:16.297 INFO:journalctl@ceph.mon.b.smithi097.stdout: block_cache: 0x55d1d30ab1f0 2024-04-25T14:05:16.297 INFO:journalctl@ceph.mon.b.smithi097.stdout: block_cache_name: BinnedLRUCache 2024-04-25T14:05:16.297 INFO:journalctl@ceph.mon.b.smithi097.stdout: block_cache_options: 2024-04-25T14:05:16.297 INFO:journalctl@ceph.mon.b.smithi097.stdout: capacity : 536870912 2024-04-25T14:05:16.297 INFO:journalctl@ceph.mon.b.smithi097.stdout: num_shard_bits : 4 2024-04-25T14:05:16.297 INFO:journalctl@ceph.mon.b.smithi097.stdout: strict_capacity_limit : 0 2024-04-25T14:05:16.297 INFO:journalctl@ceph.mon.b.smithi097.stdout: high_pri_pool_ratio: 0.000 2024-04-25T14:05:16.297 INFO:journalctl@ceph.mon.b.smithi097.stdout: block_cache_compressed: (nil) 2024-04-25T14:05:16.297 INFO:journalctl@ceph.mon.b.smithi097.stdout: persistent_cache: (nil) 2024-04-25T14:05:16.297 INFO:journalctl@ceph.mon.b.smithi097.stdout: block_size: 4096 2024-04-25T14:05:16.298 INFO:journalctl@ceph.mon.b.smithi097.stdout: block_size_deviation: 10 2024-04-25T14:05:16.298 INFO:journalctl@ceph.mon.b.smithi097.stdout: block_restart_interval: 16 2024-04-25T14:05:16.298 INFO:journalctl@ceph.mon.b.smithi097.stdout: index_block_restart_interval: 1 2024-04-25T14:05:16.298 INFO:journalctl@ceph.mon.b.smithi097.stdout: metadata_block_size: 4096 2024-04-25T14:05:16.298 INFO:journalctl@ceph.mon.b.smithi097.stdout: partition_filters: 0 2024-04-25T14:05:16.298 INFO:journalctl@ceph.mon.b.smithi097.stdout: use_delta_encoding: 1 2024-04-25T14:05:16.298 INFO:journalctl@ceph.mon.b.smithi097.stdout: filter_policy: bloomfilter 2024-04-25T14:05:16.298 INFO:journalctl@ceph.mon.b.smithi097.stdout: whole_key_filtering: 1 2024-04-25T14:05:16.298 INFO:journalctl@ceph.mon.b.smithi097.stdout: verify_compression: 0 2024-04-25T14:05:16.298 INFO:journalctl@ceph.mon.b.smithi097.stdout: read_amp_bytes_per_bit: 0 2024-04-25T14:05:16.298 INFO:journalctl@ceph.mon.b.smithi097.stdout: format_version: 5 2024-04-25T14:05:16.298 INFO:journalctl@ceph.mon.b.smithi097.stdout: enable_index_compression: 1 2024-04-25T14:05:16.298 INFO:journalctl@ceph.mon.b.smithi097.stdout: block_align: 0 2024-04-25T14:05:16.298 INFO:journalctl@ceph.mon.b.smithi097.stdout: max_auto_readahead_size: 262144 2024-04-25T14:05:16.298 INFO:journalctl@ceph.mon.b.smithi097.stdout: prepopulate_block_cache: 0 2024-04-25T14:05:16.299 INFO:journalctl@ceph.mon.b.smithi097.stdout: initial_auto_readahead_size: 8192 2024-04-25T14:05:16.299 INFO:journalctl@ceph.mon.b.smithi097.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T14:05:16.299 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T14:05:16.299 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T14:05:16.299 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.compression: NoCompression 2024-04-25T14:05:16.299 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T14:05:16.299 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T14:05:16.299 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T14:05:16.299 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.num_levels: 7 2024-04-25T14:05:16.299 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T14:05:16.299 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T14:05:16.299 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T14:05:16.299 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T14:05:16.299 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T14:05:16.299 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T14:05:16.300 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T14:05:16.300 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T14:05:16.300 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T14:05:16.300 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T14:05:16.300 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T14:05:16.300 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T14:05:16.300 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T14:05:16.300 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T14:05:16.300 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T14:05:16.300 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T14:05:16.300 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T14:05:16.300 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T14:05:16.300 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T14:05:16.300 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T14:05:16.301 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T14:05:16.301 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T14:05:16.301 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T14:05:16.301 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T14:05:16.301 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T14:05:16.301 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T14:05:16.301 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T14:05:16.301 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T14:05:16.301 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T14:05:16.301 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T14:05:16.301 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T14:05:16.301 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T14:05:16.301 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T14:05:16.301 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T14:05:16.301 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T14:05:16.301 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T14:05:16.302 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T14:05:16.302 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T14:05:16.302 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T14:05:16.302 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T14:05:16.302 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T14:05:16.302 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T14:05:16.302 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T14:05:16.302 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T14:05:16.302 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T14:05:16.302 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T14:05:16.302 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T14:05:16.302 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T14:05:16.302 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T14:05:16.302 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T14:05:16.303 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T14:05:16.303 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T14:05:16.303 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T14:05:16.303 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T14:05:16.303 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.inplace_update_support: 0 2024-04-25T14:05:16.303 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T14:05:16.303 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T14:05:16.303 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T14:05:16.303 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T14:05:16.303 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.bloom_locality: 0 2024-04-25T14:05:16.303 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.max_successive_merges: 0 2024-04-25T14:05:16.303 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T14:05:16.303 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T14:05:16.303 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T14:05:16.303 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T14:05:16.303 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.ttl: 2592000 2024-04-25T14:05:16.304 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T14:05:16.304 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T14:05:16.304 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T14:05:16.304 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.enable_blob_files: false 2024-04-25T14:05:16.304 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.min_blob_size: 0 2024-04-25T14:05:16.304 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T14:05:16.304 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T14:05:16.304 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T14:05:16.304 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T14:05:16.304 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T14:05:16.304 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T14:05:16.304 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T14:05:16.304 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T14:05:16.304 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-25T14:05:16.304 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-25T14:05:16.304 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 954c879c-5449-4fd4-a48f-9f9811fb1ebd 2024-04-25T14:05:16.305 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714053915994618, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-25T14:05:16.305 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-25T14:05:16.305 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714053915995044, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714053915, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "954c879c-5449-4fd4-a48f-9f9811fb1ebd", "db_session_id": "WIR8HRWWKJ3V76G6HWOU", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-25T14:05:16.305 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714053915995123, "job": 1, "event": "recovery_finished"} 2024-04-25T14:05:16.305 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-25T14:05:16.305 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T14:05:16.305 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55d1d308fc00 2024-04-25T14:05:16.305 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: DB pointer 0x55d1d7e82000 2024-04-25T14:05:16.305 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T14:05:16.305 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T14:05:16.305 INFO:journalctl@ceph.mon.b.smithi097.stdout: ** DB Stats ** 2024-04-25T14:05:16.305 INFO:journalctl@ceph.mon.b.smithi097.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T14:05:16.305 INFO:journalctl@ceph.mon.b.smithi097.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T14:05:16.305 INFO:journalctl@ceph.mon.b.smithi097.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T14:05:16.305 INFO:journalctl@ceph.mon.b.smithi097.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T14:05:16.305 INFO:journalctl@ceph.mon.b.smithi097.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T14:05:16.306 INFO:journalctl@ceph.mon.b.smithi097.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T14:05:16.306 INFO:journalctl@ceph.mon.b.smithi097.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T14:05:16.306 INFO:journalctl@ceph.mon.b.smithi097.stdout: 2024-04-25T14:05:16.306 INFO:journalctl@ceph.mon.b.smithi097.stdout: ** Compaction Stats [default] ** 2024-04-25T14:05:16.306 INFO:journalctl@ceph.mon.b.smithi097.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T14:05:16.306 INFO:journalctl@ceph.mon.b.smithi097.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T14:05:16.306 INFO:journalctl@ceph.mon.b.smithi097.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T14:05:16.306 INFO:journalctl@ceph.mon.b.smithi097.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T14:05:16.306 INFO:journalctl@ceph.mon.b.smithi097.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T14:05:16.306 INFO:journalctl@ceph.mon.b.smithi097.stdout: 2024-04-25T14:05:16.306 INFO:journalctl@ceph.mon.b.smithi097.stdout: ** Compaction Stats [default] ** 2024-04-25T14:05:16.306 INFO:journalctl@ceph.mon.b.smithi097.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T14:05:16.306 INFO:journalctl@ceph.mon.b.smithi097.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T14:05:16.306 INFO:journalctl@ceph.mon.b.smithi097.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T14:05:16.307 INFO:journalctl@ceph.mon.b.smithi097.stdout: 2024-04-25T14:05:16.307 INFO:journalctl@ceph.mon.b.smithi097.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T14:05:16.307 INFO:journalctl@ceph.mon.b.smithi097.stdout: 2024-04-25T14:05:16.307 INFO:journalctl@ceph.mon.b.smithi097.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T14:05:16.307 INFO:journalctl@ceph.mon.b.smithi097.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T14:05:16.307 INFO:journalctl@ceph.mon.b.smithi097.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T14:05:16.307 INFO:journalctl@ceph.mon.b.smithi097.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T14:05:16.307 INFO:journalctl@ceph.mon.b.smithi097.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T14:05:16.307 INFO:journalctl@ceph.mon.b.smithi097.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T14:05:16.307 INFO:journalctl@ceph.mon.b.smithi097.stdout: Cumulative compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T14:05:16.307 INFO:journalctl@ceph.mon.b.smithi097.stdout: Interval compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T14:05:16.307 INFO:journalctl@ceph.mon.b.smithi097.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T14:05:16.307 INFO:journalctl@ceph.mon.b.smithi097.stdout: Block cache BinnedLRUCache@0x55d1d30ab1f0#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-25T14:05:16.307 INFO:journalctl@ceph.mon.b.smithi097.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-25T14:05:16.307 INFO:journalctl@ceph.mon.b.smithi097.stdout: 2024-04-25T14:05:16.307 INFO:journalctl@ceph.mon.b.smithi097.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T14:05:16.308 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-04-25T14:05:16.308 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: using public_addr v2:172.21.15.97:0/0 -> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] 2024-04-25T14:05:16.308 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: starting mon.b rank -1 at public addrs [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] at bind addrs [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 59334182-030c-11ef-bc93-c7b262605968 2024-04-25T14:05:16.308 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:15 smithi097 ceph-mon[32720]: mon.b@-1(???) e0 preinit fsid 59334182-030c-11ef-bc93-c7b262605968 2024-04-25T14:05:16.308 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: mon.b@-1(synchronizing).mds e1 new map 2024-04-25T14:05:16.308 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: mon.b@-1(synchronizing).mds e1 print_map 2024-04-25T14:05:16.308 INFO:journalctl@ceph.mon.b.smithi097.stdout: e1 2024-04-25T14:05:16.308 INFO:journalctl@ceph.mon.b.smithi097.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T14:05:16.308 INFO:journalctl@ceph.mon.b.smithi097.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T14:05:16.308 INFO:journalctl@ceph.mon.b.smithi097.stdout: legacy client fscid: -1 2024-04-25T14:05:16.308 INFO:journalctl@ceph.mon.b.smithi097.stdout: 2024-04-25T14:05:16.308 INFO:journalctl@ceph.mon.b.smithi097.stdout: No filesystems configured 2024-04-25T14:05:16.308 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-25T14:05:16.308 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-25T14:05:16.308 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-25T14:05:16.309 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-25T14:05:16.309 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-25T14:05:16.309 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-25T14:05:16.309 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T14:05:16.309 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T14:05:16.309 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T14:05:16.309 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T14:05:16.309 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:16.309 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi151", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:05:16.309 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:16.309 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:16.309 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:16.309 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:16.309 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi097", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:05:16.309 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:16.310 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:05:16.310 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: Deploying cephadm binary to smithi151 2024-04-25T14:05:16.310 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: Updating smithi097:/etc/ceph/ceph.conf 2024-04-25T14:05:16.310 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: Updating smithi097:/var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:05:16.310 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:16.310 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:16.310 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:05:16.310 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:16.310 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:16.311 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:16.311 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:16.311 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:05:16.311 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: Updating smithi097:/var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T14:05:16.311 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: Added host smithi151 2024-04-25T14:05:16.311 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:16.311 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:16.311 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:16.311 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:16.311 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:05:16.311 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:16.312 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/144008411' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T14:05:16.312 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/144008411' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T14:05:16.312 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T14:05:16.312 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:16.312 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:16.312 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:16.312 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:16.312 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:16.312 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi151", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:05:16.312 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:16.312 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:05:16.312 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: Updating smithi151:/etc/ceph/ceph.conf 2024-04-25T14:05:16.312 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: Updating smithi151:/var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:05:16.312 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: Updating smithi151:/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:05:16.312 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: Updating smithi151:/var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T14:05:16.313 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:16.313 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:16.313 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:16.313 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:16.313 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:16.313 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:05:16.313 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:16.313 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:05:16.313 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:16.313 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:05:16.313 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:16.313 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi096:172.21.15.96=a;smithi097:172.21.15.97=b;smithi151:172.21.15.151=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:05:16.313 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: Saving service mon spec with placement smithi096:172.21.15.96=a;smithi097:172.21.15.97=b;smithi151:172.21.15.151=c;count:3 2024-04-25T14:05:16.313 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: Deploying daemon mon.c on smithi151 2024-04-25T14:05:16.313 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:16.314 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:16.314 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:05:16.314 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: mon.a calling monitor election 2024-04-25T14:05:16.314 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:05:16.314 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:16.314 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:05:16.314 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:05:16.314 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: mon.c calling monitor election 2024-04-25T14:05:16.314 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:16.314 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:05:16.314 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:05:16.314 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:16.314 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:05:16.315 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T14:05:16.315 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: monmap epoch 2 2024-04-25T14:05:16.315 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: fsid 59334182-030c-11ef-bc93-c7b262605968 2024-04-25T14:05:16.315 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: last_changed 2024-04-25T14:05:07.125083+0000 2024-04-25T14:05:16.315 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: created 2024-04-25T14:03:12.788276+0000 2024-04-25T14:05:16.315 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: min_mon_release 19 (squid) 2024-04-25T14:05:16.315 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: election_strategy: 1 2024-04-25T14:05:16.315 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: 0: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.a 2024-04-25T14:05:16.315 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: 1: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.c 2024-04-25T14:05:16.315 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: fsmap 2024-04-25T14:05:16.315 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T14:05:16.315 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: mgrmap e13: x(active, since 50s) 2024-04-25T14:05:16.315 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: overall HEALTH_OK 2024-04-25T14:05:16.315 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:16.315 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:16.315 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:16.316 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:05:16.316 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:16.316 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: Deploying daemon mon.b on smithi097 2024-04-25T14:05:16.316 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:05:16.316 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:16.316 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: from='client.? 172.21.15.151:0/1944139603' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T14:05:16.316 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 ceph-mon[32720]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-25T14:05:16.316 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 bash[32700]: affb5a16a9fcac0a7229a99273711c799f934e2c4e9ee9523c6a3cc25df5d153 2024-04-25T14:05:16.316 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:16 smithi097 systemd[1]: Started Ceph mon.b for 59334182-030c-11ef-bc93-c7b262605968. 2024-04-25T14:05:21.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.313+0000 7fbed066f700 1 -- 172.21.15.151:0/426886344 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbeb40035c0 msgr2=0x7fbeb4005aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:21.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.313+0000 7fbed066f700 1 --2- 172.21.15.151:0/426886344 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbeb40035c0 0x7fbeb4005aa0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbec8103a30 tx=0x7fbeb800fe60 comp rx=0 tx=0).stop 2024-04-25T14:05:21.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.313+0000 7fbed066f700 1 -- 172.21.15.151:0/426886344 shutdown_connections 2024-04-25T14:05:21.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.313+0000 7fbed066f700 1 --2- 172.21.15.151:0/426886344 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbeb40035c0 0x7fbeb4005aa0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.313+0000 7fbed066f700 1 -- 172.21.15.151:0/426886344 >> 172.21.15.151:0/426886344 conn(0x7fbec80ff640 msgr2=0x7fbec8101a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:21.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.313+0000 7fbed066f700 1 -- 172.21.15.151:0/426886344 shutdown_connections 2024-04-25T14:05:21.315 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.313+0000 7fbed066f700 1 -- 172.21.15.151:0/426886344 wait complete. 2024-04-25T14:05:21.315 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.314+0000 7fbed066f700 1 Processor -- start 2024-04-25T14:05:21.315 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.314+0000 7fbed066f700 1 -- start start 2024-04-25T14:05:21.315 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.315+0000 7fbed066f700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbeb40035c0 0x7fbec8075990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.315 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.315+0000 7fbed066f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbec8075ed0 0x7fbec80762e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.315 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.315+0000 7fbed066f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbec81a5cd0 0x7fbec81a80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.315 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.315+0000 7fbed066f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbec810ba90 con 0x7fbec81a5cd0 2024-04-25T14:05:21.315 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.315+0000 7fbed066f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbec810b910 con 0x7fbec8075ed0 2024-04-25T14:05:21.315 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.315+0000 7fbed066f700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbec810bc10 con 0x7fbeb40035c0 2024-04-25T14:05:21.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.315+0000 7fbecec0c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbec81a5cd0 0x7fbec81a80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.315+0000 7fbecec0c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbec81a5cd0 0x7fbec81a80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.151:44056/0 (socket says 172.21.15.151:44056) 2024-04-25T14:05:21.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.315+0000 7fbecec0c700 1 -- 172.21.15.151:0/1976340542 learned_addr learned my addr 172.21.15.151:0/1976340542 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-25T14:05:21.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.315+0000 7fbece40b700 1 --2- 172.21.15.151:0/1976340542 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbeb40035c0 0x7fbec8075990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.315+0000 7fbecdc0a700 1 --2- 172.21.15.151:0/1976340542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbec8075ed0 0x7fbec80762e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.315+0000 7fbece40b700 1 -- 172.21.15.151:0/1976340542 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbeb40035c0 msgr2=0x7fbec8075990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-25T14:05:21.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.315+0000 7fbece40b700 1 -- 172.21.15.151:0/1976340542 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbeb40035c0 msgr2=0x7fbec8075990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T14:05:21.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.315+0000 7fbece40b700 1 --2- 172.21.15.151:0/1976340542 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbeb40035c0 0x7fbec8075990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T14:05:21.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.315+0000 7fbecdc0a700 1 -- 172.21.15.151:0/1976340542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbec8075ed0 msgr2=0x7fbec80762e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-25T14:05:21.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.315+0000 7fbecdc0a700 1 -- 172.21.15.151:0/1976340542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbec8075ed0 msgr2=0x7fbec80762e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T14:05:21.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.315+0000 7fbecdc0a700 1 --2- 172.21.15.151:0/1976340542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbec8075ed0 0x7fbec80762e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T14:05:21.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.315+0000 7fbecdc0a700 1 --2- 172.21.15.151:0/1976340542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbec8075ed0 0x7fbec80762e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:05:21.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.315+0000 7fbece40b700 1 --2- 172.21.15.151:0/1976340542 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbeb40035c0 0x7fbec8075990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:05:21.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.315+0000 7fbecec0c700 1 -- 172.21.15.151:0/1976340542 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbeb40035c0 msgr2=0x7fbec8075990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:05:21.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.315+0000 7fbecec0c700 1 --2- 172.21.15.151:0/1976340542 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbeb40035c0 0x7fbec8075990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.315+0000 7fbecec0c700 1 -- 172.21.15.151:0/1976340542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbec8075ed0 msgr2=0x7fbec80762e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:05:21.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.315+0000 7fbecec0c700 1 --2- 172.21.15.151:0/1976340542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbec8075ed0 0x7fbec80762e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.315+0000 7fbecec0c700 1 -- 172.21.15.151:0/1976340542 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbec81a8770 con 0x7fbec81a5cd0 2024-04-25T14:05:21.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.316+0000 7fbecec0c700 1 --2- 172.21.15.151:0/1976340542 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbec81a5cd0 0x7fbec81a80f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbec000ba50 tx=0x7fbec000be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:21.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.316+0000 7fbebf7fe700 1 -- 172.21.15.151:0/1976340542 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec0019070 con 0x7fbec81a5cd0 2024-04-25T14:05:21.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.316+0000 7fbebf7fe700 1 -- 172.21.15.151:0/1976340542 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbec0014440 con 0x7fbec81a5cd0 2024-04-25T14:05:21.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.316+0000 7fbed066f700 1 -- 172.21.15.151:0/1976340542 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbec81a8a60 con 0x7fbec81a5cd0 2024-04-25T14:05:21.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.316+0000 7fbebf7fe700 1 -- 172.21.15.151:0/1976340542 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec0007590 con 0x7fbec81a5cd0 2024-04-25T14:05:21.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.316+0000 7fbed066f700 1 -- 172.21.15.151:0/1976340542 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbec81a8f50 con 0x7fbec81a5cd0 2024-04-25T14:05:21.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.317+0000 7fbebf7fe700 1 -- 172.21.15.151:0/1976340542 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7fbec001c070 con 0x7fbec81a5cd0 2024-04-25T14:05:21.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.317+0000 7fbed066f700 1 -- 172.21.15.151:0/1976340542 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbec8064cf0 con 0x7fbec81a5cd0 2024-04-25T14:05:21.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.317+0000 7fbebf7fe700 1 --2- 172.21.15.151:0/1976340542 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fbeac03daa0 0x7fbeac03ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.317+0000 7fbebf7fe700 1 -- 172.21.15.151:0/1976340542 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fbec0051cf0 con 0x7fbec81a5cd0 2024-04-25T14:05:21.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.318+0000 7fbece40b700 1 --2- 172.21.15.151:0/1976340542 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fbeac03daa0 0x7fbeac03ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.318+0000 7fbece40b700 1 --2- 172.21.15.151:0/1976340542 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fbeac03daa0 0x7fbeac03ffb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbeb800fa10 tx=0x7fbeb800f4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:21.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.323+0000 7fbebf7fe700 1 -- 172.21.15.151:0/1976340542 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbec0014910 con 0x7fbec81a5cd0 2024-04-25T14:05:21.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:21 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:05:21.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:21 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:05:21.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:21 smithi096 ceph-mon[29207]: mon.a calling monitor election 2024-04-25T14:05:21.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:21 smithi096 ceph-mon[29207]: mon.c calling monitor election 2024-04-25T14:05:21.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:21 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:05:21.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:21 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:05:21.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:21 smithi096 ceph-mon[29207]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:21.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:21 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:05:21.360 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:21 smithi096 ceph-mon[29207]: mon.b calling monitor election 2024-04-25T14:05:21.360 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:21 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:05:21.360 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:21 smithi096 ceph-mon[29207]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:21.360 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:21 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:05:21.360 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:21 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:05:21.360 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:21 smithi096 ceph-mon[29207]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T14:05:21.360 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:21 smithi096 ceph-mon[29207]: monmap epoch 3 2024-04-25T14:05:21.360 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:21 smithi096 ceph-mon[29207]: fsid 59334182-030c-11ef-bc93-c7b262605968 2024-04-25T14:05:21.360 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:21 smithi096 ceph-mon[29207]: last_changed 2024-04-25T14:05:16.020008+0000 2024-04-25T14:05:21.360 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:21 smithi096 ceph-mon[29207]: created 2024-04-25T14:03:12.788276+0000 2024-04-25T14:05:21.360 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:21 smithi096 ceph-mon[29207]: min_mon_release 19 (squid) 2024-04-25T14:05:21.360 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:21 smithi096 ceph-mon[29207]: election_strategy: 1 2024-04-25T14:05:21.360 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:21 smithi096 ceph-mon[29207]: 0: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.a 2024-04-25T14:05:21.360 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:21 smithi096 ceph-mon[29207]: 1: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.c 2024-04-25T14:05:21.360 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:21 smithi096 ceph-mon[29207]: 2: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.b 2024-04-25T14:05:21.360 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:21 smithi096 ceph-mon[29207]: fsmap 2024-04-25T14:05:21.361 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:21 smithi096 ceph-mon[29207]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T14:05:21.361 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:21 smithi096 ceph-mon[29207]: mgrmap e13: x(active, since 59s) 2024-04-25T14:05:21.361 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:21 smithi096 ceph-mon[29207]: overall HEALTH_OK 2024-04-25T14:05:21.361 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:21 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:21.361 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:21 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:21.361 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:21 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:21.361 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:21 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:21.422 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:21 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:05:21.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:21 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:05:21.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:21 smithi097 ceph-mon[32720]: mon.a calling monitor election 2024-04-25T14:05:21.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:21 smithi097 ceph-mon[32720]: mon.c calling monitor election 2024-04-25T14:05:21.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:21 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:05:21.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:21 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:05:21.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:21 smithi097 ceph-mon[32720]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:21.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:21 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:05:21.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:21 smithi097 ceph-mon[32720]: mon.b calling monitor election 2024-04-25T14:05:21.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:21 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:05:21.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:21 smithi097 ceph-mon[32720]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:21.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:21 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:05:21.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:21 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:05:21.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:21 smithi097 ceph-mon[32720]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T14:05:21.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:21 smithi097 ceph-mon[32720]: monmap epoch 3 2024-04-25T14:05:21.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:21 smithi097 ceph-mon[32720]: fsid 59334182-030c-11ef-bc93-c7b262605968 2024-04-25T14:05:21.424 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:21 smithi097 ceph-mon[32720]: last_changed 2024-04-25T14:05:16.020008+0000 2024-04-25T14:05:21.424 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:21 smithi097 ceph-mon[32720]: created 2024-04-25T14:03:12.788276+0000 2024-04-25T14:05:21.424 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:21 smithi097 ceph-mon[32720]: min_mon_release 19 (squid) 2024-04-25T14:05:21.424 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:21 smithi097 ceph-mon[32720]: election_strategy: 1 2024-04-25T14:05:21.424 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:21 smithi097 ceph-mon[32720]: 0: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.a 2024-04-25T14:05:21.424 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:21 smithi097 ceph-mon[32720]: 1: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.c 2024-04-25T14:05:21.424 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:21 smithi097 ceph-mon[32720]: 2: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.b 2024-04-25T14:05:21.424 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:21 smithi097 ceph-mon[32720]: fsmap 2024-04-25T14:05:21.424 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:21 smithi097 ceph-mon[32720]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T14:05:21.424 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:21 smithi097 ceph-mon[32720]: mgrmap e13: x(active, since 59s) 2024-04-25T14:05:21.424 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:21 smithi097 ceph-mon[32720]: overall HEALTH_OK 2024-04-25T14:05:21.424 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:21 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:21.424 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:21 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:21.424 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:21 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:21.424 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:21 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:21.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:21 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:05:21.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:21 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:05:21.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:21 smithi151 ceph-mon[32990]: mon.a calling monitor election 2024-04-25T14:05:21.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:21 smithi151 ceph-mon[32990]: mon.c calling monitor election 2024-04-25T14:05:21.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:21 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:05:21.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:21 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:05:21.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:21 smithi151 ceph-mon[32990]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:21.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:21 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:05:21.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:21 smithi151 ceph-mon[32990]: mon.b calling monitor election 2024-04-25T14:05:21.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:21 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:05:21.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:21 smithi151 ceph-mon[32990]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:21.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:21 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:05:21.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:21 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:05:21.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:21 smithi151 ceph-mon[32990]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T14:05:21.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:21 smithi151 ceph-mon[32990]: monmap epoch 3 2024-04-25T14:05:21.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:21 smithi151 ceph-mon[32990]: fsid 59334182-030c-11ef-bc93-c7b262605968 2024-04-25T14:05:21.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:21 smithi151 ceph-mon[32990]: last_changed 2024-04-25T14:05:16.020008+0000 2024-04-25T14:05:21.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:21 smithi151 ceph-mon[32990]: created 2024-04-25T14:03:12.788276+0000 2024-04-25T14:05:21.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:21 smithi151 ceph-mon[32990]: min_mon_release 19 (squid) 2024-04-25T14:05:21.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:21 smithi151 ceph-mon[32990]: election_strategy: 1 2024-04-25T14:05:21.440 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:21 smithi151 ceph-mon[32990]: 0: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.a 2024-04-25T14:05:21.440 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:21 smithi151 ceph-mon[32990]: 1: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.c 2024-04-25T14:05:21.440 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:21 smithi151 ceph-mon[32990]: 2: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.b 2024-04-25T14:05:21.440 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:21 smithi151 ceph-mon[32990]: fsmap 2024-04-25T14:05:21.440 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:21 smithi151 ceph-mon[32990]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T14:05:21.440 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:21 smithi151 ceph-mon[32990]: mgrmap e13: x(active, since 59s) 2024-04-25T14:05:21.440 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:21 smithi151 ceph-mon[32990]: overall HEALTH_OK 2024-04-25T14:05:21.440 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:21 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:21.440 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:21 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:21.440 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:21 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:21.440 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:21 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:21.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.609+0000 7fbed066f700 1 -- 172.21.15.151:0/1976340542 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fbec802e6d0 con 0x7fbec81a5cd0 2024-04-25T14:05:21.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.610+0000 7fbebf7fe700 1 -- 172.21.15.151:0/1976340542 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1281 (secure 0 0 0) 0x7fbec00145f0 con 0x7fbec81a5cd0 2024-04-25T14:05:21.609 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-25T14:05:21.610 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":3,"fsid":"59334182-030c-11ef-bc93-c7b262605968","modified":"2024-04-25T14:05:16.020008Z","created":"2024-04-25T14:03:12.788276Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:3300","nonce":0},{"type":"v1","addr":"172.21.15.96:6789","nonce":0}]},"addr":"172.21.15.96:6789/0","public_addr":"172.21.15.96:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:3300","nonce":0},{"type":"v1","addr":"172.21.15.151:6789","nonce":0}]},"addr":"172.21.15.151:6789/0","public_addr":"172.21.15.151:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:3300","nonce":0},{"type":"v1","addr":"172.21.15.97:6789","nonce":0}]},"addr":"172.21.15.97:6789/0","public_addr":"172.21.15.97:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-04-25T14:05:21.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.612+0000 7fbed066f700 1 -- 172.21.15.151:0/1976340542 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fbeac03daa0 msgr2=0x7fbeac03ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:21.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.612+0000 7fbed066f700 1 --2- 172.21.15.151:0/1976340542 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fbeac03daa0 0x7fbeac03ffb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbeb800fa10 tx=0x7fbeb800f4a0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.612+0000 7fbed066f700 1 -- 172.21.15.151:0/1976340542 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbec81a5cd0 msgr2=0x7fbec81a80f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:21.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.612+0000 7fbed066f700 1 --2- 172.21.15.151:0/1976340542 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbec81a5cd0 0x7fbec81a80f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbec000ba50 tx=0x7fbec000be10 comp rx=0 tx=0).stop 2024-04-25T14:05:21.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.612+0000 7fbed066f700 1 -- 172.21.15.151:0/1976340542 shutdown_connections 2024-04-25T14:05:21.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.612+0000 7fbed066f700 1 --2- 172.21.15.151:0/1976340542 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fbeac03daa0 0x7fbeac03ffb0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.612+0000 7fbed066f700 1 --2- 172.21.15.151:0/1976340542 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbec81a5cd0 0x7fbec81a80f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.612+0000 7fbed066f700 1 --2- 172.21.15.151:0/1976340542 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbeb40035c0 0x7fbec8075990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.612+0000 7fbed066f700 1 --2- 172.21.15.151:0/1976340542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbec8075ed0 0x7fbec80762e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.612+0000 7fbed066f700 1 -- 172.21.15.151:0/1976340542 >> 172.21.15.151:0/1976340542 conn(0x7fbec80ff640 msgr2=0x7fbec8107400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:21.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.612+0000 7fbed066f700 1 -- 172.21.15.151:0/1976340542 shutdown_connections 2024-04-25T14:05:21.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:21.612+0000 7fbed066f700 1 -- 172.21.15.151:0/1976340542 wait complete. 2024-04-25T14:05:21.613 INFO:teuthology.orchestra.run.smithi151.stderr:dumped monmap epoch 3 2024-04-25T14:05:22.187 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:22 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:05:22.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:22 smithi151 ceph-mon[32990]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:22.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:22 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:22.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:22 smithi151 ceph-mon[32990]: from='client.? 172.21.15.151:0/1976340542' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T14:05:22.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:22 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:05:22.243 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-25T14:05:22.244 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-25T14:05:22.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:22 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:05:22.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:22 smithi096 ceph-mon[29207]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:22.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:22 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:22.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:22 smithi096 ceph-mon[29207]: from='client.? 172.21.15.151:0/1976340542' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T14:05:22.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:22 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:05:22.422 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:22 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:05:22.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:22 smithi097 ceph-mon[32720]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:22.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:22 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:22.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:22 smithi097 ceph-mon[32720]: from='client.? 172.21.15.151:0/1976340542' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T14:05:22.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:22 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:05:22.493 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:05:23.770 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:23 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:23.770 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:23 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:23.770 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:23 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:23.770 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:23 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:23.770 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:23 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:23.770 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:23 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:05:23.771 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:23 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:23.771 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:23 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:23.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:23 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:23.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:23 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:23.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:23 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:23.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:23 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:23.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:23 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:23.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:23 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:05:23.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:23 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:23.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:23 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:23.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:23 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:23.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:23 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:23.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:23 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:23.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:23 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:23.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:23 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:23.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:23 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:05:23.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:23 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:23.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:23 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:24.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.126+0000 7f1bc191a700 1 -- 172.21.15.96:0/2997442237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1bbc108a60 msgr2=0x7f1bbc108e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:24.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.126+0000 7f1bc191a700 1 --2- 172.21.15.96:0/2997442237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1bbc108a60 0x7f1bbc108e70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1bac009910 tx=0x7f1bac009c20 comp rx=0 tx=0).stop 2024-04-25T14:05:24.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.126+0000 7f1bc191a700 1 -- 172.21.15.96:0/2997442237 shutdown_connections 2024-04-25T14:05:24.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.126+0000 7f1bc191a700 1 --2- 172.21.15.96:0/2997442237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1bbc108a60 0x7f1bbc108e70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:24.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.126+0000 7f1bc191a700 1 -- 172.21.15.96:0/2997442237 >> 172.21.15.96:0/2997442237 conn(0x7f1bbc0fdd00 msgr2=0x7f1bbc100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:24.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.126+0000 7f1bc191a700 1 -- 172.21.15.96:0/2997442237 shutdown_connections 2024-04-25T14:05:24.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.126+0000 7f1bc191a700 1 -- 172.21.15.96:0/2997442237 wait complete. 2024-04-25T14:05:24.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.127+0000 7f1bc191a700 1 Processor -- start 2024-04-25T14:05:24.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.127+0000 7f1bc191a700 1 -- start start 2024-04-25T14:05:24.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.128+0000 7f1bc191a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bbc108a60 0x7f1bbc19f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:24.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.128+0000 7f1bc191a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1bbc19fdb0 0x7f1bbc199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:24.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.128+0000 7f1bc191a700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1bbc199fd0 0x7f1bbc19a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:24.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.128+0000 7f1bc191a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1bbc10e0e0 con 0x7f1bbc19fdb0 2024-04-25T14:05:24.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.128+0000 7f1bc191a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1bbc10df60 con 0x7f1bbc108a60 2024-04-25T14:05:24.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.128+0000 7f1bc191a700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1bbc10e260 con 0x7f1bbc199fd0 2024-04-25T14:05:24.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.128+0000 7f1bba7fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1bbc19fdb0 0x7f1bbc199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:24.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.128+0000 7f1bba7fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1bbc19fdb0 0x7f1bbc199a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50034/0 (socket says 172.21.15.96:50034) 2024-04-25T14:05:24.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.128+0000 7f1bba7fc700 1 -- 172.21.15.96:0/3684554124 learned_addr learned my addr 172.21.15.96:0/3684554124 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:05:24.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.128+0000 7f1bbaffd700 1 --2- 172.21.15.96:0/3684554124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bbc108a60 0x7f1bbc19f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:24.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.128+0000 7f1bbb7fe700 1 --2- 172.21.15.96:0/3684554124 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1bbc199fd0 0x7f1bbc19a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:24.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.128+0000 7f1bba7fc700 1 -- 172.21.15.96:0/3684554124 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1bbc199fd0 msgr2=0x7f1bbc19a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:24.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.128+0000 7f1bba7fc700 1 --2- 172.21.15.96:0/3684554124 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1bbc199fd0 0x7f1bbc19a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:24.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.128+0000 7f1bba7fc700 1 -- 172.21.15.96:0/3684554124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bbc108a60 msgr2=0x7f1bbc19f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:24.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.128+0000 7f1bba7fc700 1 --2- 172.21.15.96:0/3684554124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bbc108a60 0x7f1bbc19f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:24.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.128+0000 7f1bba7fc700 1 -- 172.21.15.96:0/3684554124 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bbc1a6700 con 0x7f1bbc19fdb0 2024-04-25T14:05:24.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.128+0000 7f1bbb7fe700 1 --2- 172.21.15.96:0/3684554124 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1bbc199fd0 0x7f1bbc19a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:24.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.128+0000 7f1bba7fc700 1 --2- 172.21.15.96:0/3684554124 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1bbc19fdb0 0x7f1bbc199a90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1bb000b780 tx=0x7f1bb000ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:24.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.128+0000 7f1bc0918700 1 -- 172.21.15.96:0/3684554124 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb0004230 con 0x7f1bbc19fdb0 2024-04-25T14:05:24.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.129+0000 7f1bc0918700 1 -- 172.21.15.96:0/3684554124 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bb00043d0 con 0x7f1bbc19fdb0 2024-04-25T14:05:24.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.129+0000 7f1bc0918700 1 -- 172.21.15.96:0/3684554124 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb0011b20 con 0x7f1bbc19fdb0 2024-04-25T14:05:24.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.129+0000 7f1bc191a700 1 -- 172.21.15.96:0/3684554124 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bbc1a6990 con 0x7f1bbc19fdb0 2024-04-25T14:05:24.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.129+0000 7f1bc191a700 1 -- 172.21.15.96:0/3684554124 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bbc1a6ef0 con 0x7f1bbc19fdb0 2024-04-25T14:05:24.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.130+0000 7f1bc191a700 1 -- 172.21.15.96:0/3684554124 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bbc064cf0 con 0x7f1bbc19fdb0 2024-04-25T14:05:24.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.130+0000 7f1bc0918700 1 -- 172.21.15.96:0/3684554124 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f1bb0011cc0 con 0x7f1bbc19fdb0 2024-04-25T14:05:24.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.130+0000 7f1bc0918700 1 --2- 172.21.15.96:0/3684554124 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1ba403daa0 0x7f1ba403ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:24.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.130+0000 7f1bc0918700 1 -- 172.21.15.96:0/3684554124 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1bb00510e0 con 0x7f1bbc19fdb0 2024-04-25T14:05:24.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.130+0000 7f1bbaffd700 1 --2- 172.21.15.96:0/3684554124 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1ba403daa0 0x7f1ba403ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:24.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.131+0000 7f1bbaffd700 1 --2- 172.21.15.96:0/3684554124 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1ba403daa0 0x7f1ba403ffb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1bac005e00 tx=0x7f1bac01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:24.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.135+0000 7f1bc0918700 1 -- 172.21.15.96:0/3684554124 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1bb0017d50 con 0x7f1bbc19fdb0 2024-04-25T14:05:24.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.344+0000 7f1bc191a700 1 -- 172.21.15.96:0/3684554124 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f1bbc1a71e0 con 0x7f1bbc19fdb0 2024-04-25T14:05:24.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.345+0000 7f1bc0918700 1 -- 172.21.15.96:0/3684554124 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7f1bb0053130 con 0x7f1bbc19fdb0 2024-04-25T14:05:24.344 INFO:teuthology.orchestra.run.smithi096.stdout:# minimal ceph.conf for 59334182-030c-11ef-bc93-c7b262605968 2024-04-25T14:05:24.345 INFO:teuthology.orchestra.run.smithi096.stdout:[global] 2024-04-25T14:05:24.345 INFO:teuthology.orchestra.run.smithi096.stdout: fsid = 59334182-030c-11ef-bc93-c7b262605968 2024-04-25T14:05:24.345 INFO:teuthology.orchestra.run.smithi096.stdout: mon_host = [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] 2024-04-25T14:05:24.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.347+0000 7f1bc191a700 1 -- 172.21.15.96:0/3684554124 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1ba403daa0 msgr2=0x7f1ba403ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:24.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.347+0000 7f1bc191a700 1 --2- 172.21.15.96:0/3684554124 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1ba403daa0 0x7f1ba403ffb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1bac005e00 tx=0x7f1bac01a040 comp rx=0 tx=0).stop 2024-04-25T14:05:24.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.347+0000 7f1bc191a700 1 -- 172.21.15.96:0/3684554124 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1bbc19fdb0 msgr2=0x7f1bbc199a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:24.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.347+0000 7f1bc191a700 1 --2- 172.21.15.96:0/3684554124 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1bbc19fdb0 0x7f1bbc199a90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1bb000b780 tx=0x7f1bb000ba90 comp rx=0 tx=0).stop 2024-04-25T14:05:24.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.347+0000 7f1bc191a700 1 -- 172.21.15.96:0/3684554124 shutdown_connections 2024-04-25T14:05:24.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.347+0000 7f1bc191a700 1 --2- 172.21.15.96:0/3684554124 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1ba403daa0 0x7f1ba403ffb0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:24.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.347+0000 7f1bc191a700 1 --2- 172.21.15.96:0/3684554124 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1bbc199fd0 0x7f1bbc19a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:24.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.347+0000 7f1bc191a700 1 --2- 172.21.15.96:0/3684554124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bbc108a60 0x7f1bbc19f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:24.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.347+0000 7f1bc191a700 1 --2- 172.21.15.96:0/3684554124 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1bbc19fdb0 0x7f1bbc199a90 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:24.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.347+0000 7f1bc191a700 1 -- 172.21.15.96:0/3684554124 >> 172.21.15.96:0/3684554124 conn(0x7f1bbc0fdd00 msgr2=0x7f1bbc105660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:24.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.347+0000 7f1bc191a700 1 -- 172.21.15.96:0/3684554124 shutdown_connections 2024-04-25T14:05:24.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:24.347+0000 7f1bc191a700 1 -- 172.21.15.96:0/3684554124 wait complete. 2024-04-25T14:05:24.630 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:24 smithi096 ceph-mon[29207]: Updating smithi096:/etc/ceph/ceph.conf 2024-04-25T14:05:24.630 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:24 smithi096 ceph-mon[29207]: Updating smithi097:/etc/ceph/ceph.conf 2024-04-25T14:05:24.630 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:24 smithi096 ceph-mon[29207]: Updating smithi151:/etc/ceph/ceph.conf 2024-04-25T14:05:24.630 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:24 smithi096 ceph-mon[29207]: Updating smithi096:/var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:05:24.630 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:24 smithi096 ceph-mon[29207]: Updating smithi151:/var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:05:24.630 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:24 smithi096 ceph-mon[29207]: Updating smithi097:/var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:05:24.630 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:24 smithi096 ceph-mon[29207]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:24.630 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:24 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:24.630 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:24 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:24.630 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:24 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:24.630 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:24 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:24.631 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:24 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:24.631 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:24 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:24.631 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:24 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:24.631 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:24 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:24.631 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:24 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:24.631 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:24 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:05:24.631 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:24 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T14:05:24.631 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:24 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:24.631 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:24 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/3684554124' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:24.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:24 smithi097 ceph-mon[32720]: Updating smithi096:/etc/ceph/ceph.conf 2024-04-25T14:05:24.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:24 smithi097 ceph-mon[32720]: Updating smithi097:/etc/ceph/ceph.conf 2024-04-25T14:05:24.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:24 smithi097 ceph-mon[32720]: Updating smithi151:/etc/ceph/ceph.conf 2024-04-25T14:05:24.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:24 smithi097 ceph-mon[32720]: Updating smithi096:/var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:05:24.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:24 smithi097 ceph-mon[32720]: Updating smithi151:/var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:05:24.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:24 smithi097 ceph-mon[32720]: Updating smithi097:/var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:05:24.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:24 smithi097 ceph-mon[32720]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:24.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:24 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:24.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:24 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:24.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:24 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:24.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:24 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:24.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:24 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:24.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:24 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:24.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:24 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:24.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:24 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:24.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:24 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:24.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:24 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:05:24.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:24 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T14:05:24.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:24 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:24.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:24 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/3684554124' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:24.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:24 smithi151 ceph-mon[32990]: Updating smithi096:/etc/ceph/ceph.conf 2024-04-25T14:05:24.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:24 smithi151 ceph-mon[32990]: Updating smithi097:/etc/ceph/ceph.conf 2024-04-25T14:05:24.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:24 smithi151 ceph-mon[32990]: Updating smithi151:/etc/ceph/ceph.conf 2024-04-25T14:05:24.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:24 smithi151 ceph-mon[32990]: Updating smithi096:/var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:05:24.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:24 smithi151 ceph-mon[32990]: Updating smithi151:/var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:05:24.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:24 smithi151 ceph-mon[32990]: Updating smithi097:/var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:05:24.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:24 smithi151 ceph-mon[32990]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:24.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:24 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:24.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:24 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:24.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:24 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:24.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:24 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:24.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:24 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:24.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:24 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:24.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:24 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:24.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:24 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:24.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:24 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:24.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:24 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:05:24.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:24 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T14:05:24.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:24 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:24.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:24 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/3684554124' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:25.036 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-25T14:05:25.036 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T14:05:25.036 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T14:05:25.062 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T14:05:25.062 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:05:25.125 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T14:05:25.125 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T14:05:25.156 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T14:05:25.157 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:05:25.226 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-25T14:05:25.226 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T14:05:25.257 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-25T14:05:25.257 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:05:25.325 INFO:tasks.cephadm:Adding mgr.x on smithi096 2024-04-25T14:05:25.325 INFO:tasks.cephadm:Adding mgr.y on smithi097 2024-04-25T14:05:25.325 INFO:tasks.cephadm:Adding mgr.z on smithi151 2024-04-25T14:05:25.325 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi096=x;smithi097=y;smithi151=z' 2024-04-25T14:05:25.586 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:25 smithi096 ceph-mon[29207]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T14:05:25.586 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:25 smithi096 ceph-mon[29207]: Reconfiguring daemon mon.a on smithi096 2024-04-25T14:05:25.599 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:05:25.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:25 smithi151 ceph-mon[32990]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T14:05:25.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:25 smithi151 ceph-mon[32990]: Reconfiguring daemon mon.a on smithi096 2024-04-25T14:05:25.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:25 smithi097 ceph-mon[32720]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T14:05:25.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:25 smithi097 ceph-mon[32720]: Reconfiguring daemon mon.a on smithi096 2024-04-25T14:05:26.761 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:26 smithi097 ceph-mon[32720]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:26.761 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:26 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:26.761 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:26 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:26.761 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:26 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:05:26.761 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:26 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T14:05:26.761 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:26 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:26.825 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:26 smithi151 ceph-mon[32990]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:26.825 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:26 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:26.825 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:26 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:26.825 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:26 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:05:26.825 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:26 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T14:05:26.826 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:26 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:26.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:26 smithi096 ceph-mon[29207]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:26.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:26 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:26.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:26 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:26.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:26 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:05:26.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:26 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T14:05:26.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:26 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:27.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.154+0000 7f27192b0700 1 -- 172.21.15.151:0/898799019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2714101e50 msgr2=0x7f2714102260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:27.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.155+0000 7f27192b0700 1 --2- 172.21.15.151:0/898799019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2714101e50 0x7f2714102260 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f270400b0a0 tx=0x7f270400b3b0 comp rx=0 tx=0).stop 2024-04-25T14:05:27.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.155+0000 7f27192b0700 1 -- 172.21.15.151:0/898799019 shutdown_connections 2024-04-25T14:05:27.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.155+0000 7f27192b0700 1 --2- 172.21.15.151:0/898799019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2714101e50 0x7f2714102260 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:27.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.155+0000 7f27192b0700 1 -- 172.21.15.151:0/898799019 >> 172.21.15.151:0/898799019 conn(0x7f27140fd830 msgr2=0x7f27140ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:27.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.155+0000 7f27192b0700 1 -- 172.21.15.151:0/898799019 shutdown_connections 2024-04-25T14:05:27.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.155+0000 7f27192b0700 1 -- 172.21.15.151:0/898799019 wait complete. 2024-04-25T14:05:27.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.155+0000 7f27192b0700 1 Processor -- start 2024-04-25T14:05:27.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.156+0000 7f27192b0700 1 -- start start 2024-04-25T14:05:27.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.156+0000 7f27192b0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2714101e50 0x7f2714199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:27.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.156+0000 7f27192b0700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2714199dd0 0x7f271419ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:27.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.156+0000 7f27192b0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f271419f2a0 0x7f27141a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:27.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.156+0000 7f27192b0700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2714109b00 con 0x7f271419f2a0 2024-04-25T14:05:27.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.156+0000 7f27192b0700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2714109980 con 0x7f2714101e50 2024-04-25T14:05:27.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.156+0000 7f27192b0700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2714109c80 con 0x7f2714199dd0 2024-04-25T14:05:27.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.156+0000 7f27127fc700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2714199dd0 0x7f271419ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:27.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.156+0000 7f27127fc700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2714199dd0 0x7f271419ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:45510/0 (socket says 172.21.15.151:45510) 2024-04-25T14:05:27.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.156+0000 7f27127fc700 1 -- 172.21.15.151:0/2256656439 learned_addr learned my addr 172.21.15.151:0/2256656439 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-25T14:05:27.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.156+0000 7f27137fe700 1 --2- 172.21.15.151:0/2256656439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f271419f2a0 0x7f27141a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:27.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.156+0000 7f2712ffd700 1 --2- 172.21.15.151:0/2256656439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2714101e50 0x7f2714199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:27.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.156+0000 7f27137fe700 1 -- 172.21.15.151:0/2256656439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2714199dd0 msgr2=0x7f271419ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:27.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.156+0000 7f27137fe700 1 --2- 172.21.15.151:0/2256656439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2714199dd0 0x7f271419ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:27.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.156+0000 7f27137fe700 1 -- 172.21.15.151:0/2256656439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2714101e50 msgr2=0x7f2714199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:27.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.156+0000 7f27137fe700 1 --2- 172.21.15.151:0/2256656439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2714101e50 0x7f2714199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:27.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.156+0000 7f27137fe700 1 -- 172.21.15.151:0/2256656439 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27141a1c30 con 0x7f271419f2a0 2024-04-25T14:05:27.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.156+0000 7f27127fc700 1 --2- 172.21.15.151:0/2256656439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2714199dd0 0x7f271419ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:27.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.157+0000 7f27137fe700 1 --2- 172.21.15.151:0/2256656439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f271419f2a0 0x7f27141a16c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f270c006fd0 tx=0x7f270c00b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:27.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.157+0000 7f270bfff700 1 -- 172.21.15.151:0/2256656439 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270c015020 con 0x7f271419f2a0 2024-04-25T14:05:27.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.157+0000 7f270bfff700 1 -- 172.21.15.151:0/2256656439 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f270c00be30 con 0x7f271419f2a0 2024-04-25T14:05:27.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.157+0000 7f27192b0700 1 -- 172.21.15.151:0/2256656439 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27141a1f20 con 0x7f271419f2a0 2024-04-25T14:05:27.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.157+0000 7f270bfff700 1 -- 172.21.15.151:0/2256656439 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270c010be0 con 0x7f271419f2a0 2024-04-25T14:05:27.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.157+0000 7f27192b0700 1 -- 172.21.15.151:0/2256656439 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27141a2340 con 0x7f271419f2a0 2024-04-25T14:05:27.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.158+0000 7f27192b0700 1 -- 172.21.15.151:0/2256656439 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27141a2600 con 0x7f271419f2a0 2024-04-25T14:05:27.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.158+0000 7f270bfff700 1 -- 172.21.15.151:0/2256656439 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f270c018020 con 0x7f271419f2a0 2024-04-25T14:05:27.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.158+0000 7f270bfff700 1 --2- 172.21.15.151:0/2256656439 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f270003daa0 0x7f270003ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:27.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.158+0000 7f270bfff700 1 -- 172.21.15.151:0/2256656439 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f270c050f00 con 0x7f271419f2a0 2024-04-25T14:05:27.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.158+0000 7f2712ffd700 1 --2- 172.21.15.151:0/2256656439 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f270003daa0 0x7f270003ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:27.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.159+0000 7f2712ffd700 1 --2- 172.21.15.151:0/2256656439 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f270003daa0 0x7f270003ffb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f270400b830 tx=0x7f2704006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:27.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.164+0000 7f270bfff700 1 -- 172.21.15.151:0/2256656439 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f270c01f020 con 0x7f271419f2a0 2024-04-25T14:05:27.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.371+0000 7f27192b0700 1 -- 172.21.15.151:0/2256656439 --> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi096=x;smithi097=y;smithi151=z", "target": ["mon-mgr", ""]}) -- 0x7f27140ff7b0 con 0x7f270003daa0 2024-04-25T14:05:27.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.377+0000 7f270bfff700 1 -- 172.21.15.151:0/2256656439 <== mgr.14152 v2:172.21.15.96:6800/536332425 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f27140ff7b0 con 0x7f270003daa0 2024-04-25T14:05:27.377 INFO:teuthology.orchestra.run.smithi151.stdout:Scheduled mgr update... 2024-04-25T14:05:27.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.381+0000 7f27192b0700 1 -- 172.21.15.151:0/2256656439 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f270003daa0 msgr2=0x7f270003ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:27.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.381+0000 7f27192b0700 1 --2- 172.21.15.151:0/2256656439 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f270003daa0 0x7f270003ffb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f270400b830 tx=0x7f2704006210 comp rx=0 tx=0).stop 2024-04-25T14:05:27.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.381+0000 7f27192b0700 1 -- 172.21.15.151:0/2256656439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f271419f2a0 msgr2=0x7f27141a16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:27.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.381+0000 7f27192b0700 1 --2- 172.21.15.151:0/2256656439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f271419f2a0 0x7f27141a16c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f270c006fd0 tx=0x7f270c00b730 comp rx=0 tx=0).stop 2024-04-25T14:05:27.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.381+0000 7f27192b0700 1 -- 172.21.15.151:0/2256656439 shutdown_connections 2024-04-25T14:05:27.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.381+0000 7f27192b0700 1 --2- 172.21.15.151:0/2256656439 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f270003daa0 0x7f270003ffb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:27.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.381+0000 7f27192b0700 1 --2- 172.21.15.151:0/2256656439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f271419f2a0 0x7f27141a16c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:27.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.381+0000 7f27192b0700 1 --2- 172.21.15.151:0/2256656439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2714101e50 0x7f2714199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:27.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.381+0000 7f27192b0700 1 --2- 172.21.15.151:0/2256656439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2714199dd0 0x7f271419ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:27.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.381+0000 7f27192b0700 1 -- 172.21.15.151:0/2256656439 >> 172.21.15.151:0/2256656439 conn(0x7f27140fd830 msgr2=0x7f27140ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:27.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.381+0000 7f27192b0700 1 -- 172.21.15.151:0/2256656439 shutdown_connections 2024-04-25T14:05:27.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:05:27.381+0000 7f27192b0700 1 -- 172.21.15.151:0/2256656439 wait complete. 2024-04-25T14:05:27.541 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:27 smithi097 ceph-mon[32720]: Reconfiguring mon.b (monmap changed)... 2024-04-25T14:05:27.541 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:27 smithi097 ceph-mon[32720]: Reconfiguring daemon mon.b on smithi097 2024-04-25T14:05:27.541 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:27 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:27.541 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:27 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:27.541 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:27 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:27.541 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:27 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:05:27.542 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:27 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T14:05:27.542 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:27 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:27.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:27 smithi151 ceph-mon[32990]: Reconfiguring mon.b (monmap changed)... 2024-04-25T14:05:27.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:27 smithi151 ceph-mon[32990]: Reconfiguring daemon mon.b on smithi097 2024-04-25T14:05:27.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:27 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:27.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:27 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:27.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:27 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:27.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:27 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:05:27.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:27 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T14:05:27.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:27 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:27.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:27 smithi096 ceph-mon[29207]: Reconfiguring mon.b (monmap changed)... 2024-04-25T14:05:27.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:27 smithi096 ceph-mon[29207]: Reconfiguring daemon mon.b on smithi097 2024-04-25T14:05:27.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:27 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:27.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:27 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:27.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:27 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:27.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:27 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:05:27.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:27 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T14:05:27.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:27 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:27.941 DEBUG:teuthology.orchestra.run.smithi097:mgr.y> sudo journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@mgr.y.service 2024-04-25T14:05:27.943 DEBUG:teuthology.orchestra.run.smithi151:mgr.z> sudo journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@mgr.z.service 2024-04-25T14:05:27.945 INFO:tasks.cephadm:Deploying OSDs... 2024-04-25T14:05:27.946 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T14:05:27.946 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T14:05:27.965 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T14:05:27.965 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_1 2024-04-25T14:05:28.023 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T14:05:28.023 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:05:28.023 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-25T14:05:28.023 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:05:28.023 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:05:28.024 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-04-25 14:04:34.010637204 +0000 2024-04-25T14:05:28.024 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-04-25 13:59:23.980734482 +0000 2024-04-25T14:05:28.024 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-04-25 13:59:23.980734482 +0000 2024-04-25T14:05:28.024 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-04-25 13:59:23.980734482 +0000 2024-04-25T14:05:28.024 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T14:05:28.089 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-04-25T14:05:28.089 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-04-25T14:05:28.089 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.00022562 s, 2.3 MB/s 2024-04-25T14:05:28.090 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T14:05:28.148 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_2 2024-04-25T14:05:28.203 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T14:05:28.203 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:05:28.203 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-25T14:05:28.203 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:05:28.203 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:05:28.203 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-04-25 14:04:34.038636611 +0000 2024-04-25T14:05:28.203 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-04-25 13:59:24.319726813 +0000 2024-04-25T14:05:28.203 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-04-25 13:59:24.319726813 +0000 2024-04-25T14:05:28.204 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-04-25 13:59:24.319726813 +0000 2024-04-25T14:05:28.204 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T14:05:28.266 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-04-25T14:05:28.266 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-04-25T14:05:28.266 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000185624 s, 2.8 MB/s 2024-04-25T14:05:28.268 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T14:05:28.325 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_3 2024-04-25T14:05:28.381 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T14:05:28.381 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:05:28.381 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-04-25T14:05:28.381 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:05:28.381 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:05:28.381 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-04-25 14:04:34.075635827 +0000 2024-04-25T14:05:28.381 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-04-25 13:59:24.662719054 +0000 2024-04-25T14:05:28.381 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-04-25 13:59:24.662719054 +0000 2024-04-25T14:05:28.381 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-04-25 13:59:24.662719054 +0000 2024-04-25T14:05:28.382 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T14:05:28.446 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-04-25T14:05:28.446 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-04-25T14:05:28.446 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000175945 s, 2.9 MB/s 2024-04-25T14:05:28.447 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T14:05:28.503 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_4 2024-04-25T14:05:28.559 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T14:05:28.559 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:05:28.559 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-25T14:05:28.559 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:05:28.559 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:05:28.560 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-04-25 14:04:34.102635255 +0000 2024-04-25T14:05:28.560 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-04-25 13:59:24.992711589 +0000 2024-04-25T14:05:28.560 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-04-25 13:59:24.992711589 +0000 2024-04-25T14:05:28.560 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-04-25 13:59:24.992711589 +0000 2024-04-25T14:05:28.560 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T14:05:28.623 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-04-25T14:05:28.623 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-04-25T14:05:28.623 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.00023617 s, 2.2 MB/s 2024-04-25T14:05:28.624 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T14:05:28.680 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T14:05:28.680 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T14:05:28.689 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:28 smithi151 ceph-mon[32990]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:28.689 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:28 smithi151 ceph-mon[32990]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi096=x;smithi097=y;smithi151=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:05:28.689 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:28 smithi151 ceph-mon[32990]: Saving service mgr spec with placement smithi096=x;smithi097=y;smithi151=z;count:3 2024-04-25T14:05:28.689 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:28 smithi151 ceph-mon[32990]: Reconfiguring mon.c (monmap changed)... 2024-04-25T14:05:28.689 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:28 smithi151 ceph-mon[32990]: Reconfiguring daemon mon.c on smithi151 2024-04-25T14:05:28.699 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T14:05:28.699 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_1 2024-04-25T14:05:28.756 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T14:05:28.757 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:05:28.757 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-25T14:05:28.757 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:05:28.757 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:05:28.757 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-04-25 14:05:15.420676033 +0000 2024-04-25T14:05:28.757 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-04-25 13:59:43.890046555 +0000 2024-04-25T14:05:28.757 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-04-25 13:59:43.890046555 +0000 2024-04-25T14:05:28.757 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-04-25 13:59:43.890046555 +0000 2024-04-25T14:05:28.757 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T14:05:28.821 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:28 smithi097 ceph-mon[32720]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:28.821 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:28 smithi097 ceph-mon[32720]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi096=x;smithi097=y;smithi151=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:05:28.822 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:28 smithi097 ceph-mon[32720]: Saving service mgr spec with placement smithi096=x;smithi097=y;smithi151=z;count:3 2024-04-25T14:05:28.822 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:28 smithi097 ceph-mon[32720]: Reconfiguring mon.c (monmap changed)... 2024-04-25T14:05:28.822 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:28 smithi097 ceph-mon[32720]: Reconfiguring daemon mon.c on smithi151 2024-04-25T14:05:28.824 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T14:05:28.824 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T14:05:28.824 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000262242 s, 2.0 MB/s 2024-04-25T14:05:28.825 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T14:05:28.858 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:28 smithi096 ceph-mon[29207]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:28.858 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:28 smithi096 ceph-mon[29207]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi096=x;smithi097=y;smithi151=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:05:28.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:28 smithi096 ceph-mon[29207]: Saving service mgr spec with placement smithi096=x;smithi097=y;smithi151=z;count:3 2024-04-25T14:05:28.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:28 smithi096 ceph-mon[29207]: Reconfiguring mon.c (monmap changed)... 2024-04-25T14:05:28.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:28 smithi096 ceph-mon[29207]: Reconfiguring daemon mon.c on smithi151 2024-04-25T14:05:28.884 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_2 2024-04-25T14:05:28.943 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T14:05:28.943 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:05:28.943 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-25T14:05:28.943 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:05:28.943 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:05:28.943 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-04-25 14:05:15.420676033 +0000 2024-04-25T14:05:28.943 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-04-25 13:59:44.224040349 +0000 2024-04-25T14:05:28.943 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-04-25 13:59:44.224040349 +0000 2024-04-25T14:05:28.943 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-04-25 13:59:44.224040349 +0000 2024-04-25T14:05:28.944 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T14:05:29.009 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T14:05:29.009 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T14:05:29.010 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000849298 s, 603 kB/s 2024-04-25T14:05:29.011 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T14:05:29.068 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_3 2024-04-25T14:05:29.124 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T14:05:29.124 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:05:29.124 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-25T14:05:29.124 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:05:29.124 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:05:29.124 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-04-25 14:05:15.421676016 +0000 2024-04-25T14:05:29.125 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-04-25 13:59:44.569033938 +0000 2024-04-25T14:05:29.125 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-04-25 13:59:44.569033938 +0000 2024-04-25T14:05:29.125 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-04-25 13:59:44.569033938 +0000 2024-04-25T14:05:29.125 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T14:05:29.187 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T14:05:29.187 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T14:05:29.188 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000270364 s, 1.9 MB/s 2024-04-25T14:05:29.189 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T14:05:29.246 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_4 2024-04-25T14:05:29.302 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T14:05:29.302 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:05:29.302 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-25T14:05:29.302 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:05:29.302 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:05:29.302 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-04-25 14:05:15.421676016 +0000 2024-04-25T14:05:29.302 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-04-25 13:59:44.911027583 +0000 2024-04-25T14:05:29.302 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-04-25 13:59:44.911027583 +0000 2024-04-25T14:05:29.302 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-04-25 13:59:44.911027583 +0000 2024-04-25T14:05:29.302 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T14:05:29.365 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T14:05:29.365 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T14:05:29.365 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.00020003 s, 2.6 MB/s 2024-04-25T14:05:29.366 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T14:05:29.422 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-25T14:05:29.422 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T14:05:29.440 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T14:05:29.440 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_1 2024-04-25T14:05:29.497 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T14:05:29.497 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:05:29.497 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-25T14:05:29.497 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:05:29.497 INFO:teuthology.orchestra.run.smithi151.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:05:29.497 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-04-25 14:05:06.323370498 +0000 2024-04-25T14:05:29.497 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-04-25 13:59:43.916660905 +0000 2024-04-25T14:05:29.498 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-04-25 13:59:43.916660905 +0000 2024-04-25T14:05:29.498 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: 2024-04-25 13:59:43.916660905 +0000 2024-04-25T14:05:29.498 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T14:05:29.564 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-04-25T14:05:29.564 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-04-25T14:05:29.564 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000282616 s, 1.8 MB/s 2024-04-25T14:05:29.565 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T14:05:29.622 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_2 2024-04-25T14:05:29.677 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T14:05:29.677 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:05:29.677 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-25T14:05:29.677 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:05:29.677 INFO:teuthology.orchestra.run.smithi151.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:05:29.677 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-04-25 14:05:06.323370498 +0000 2024-04-25T14:05:29.678 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-04-25 13:59:44.256656083 +0000 2024-04-25T14:05:29.678 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-04-25 13:59:44.256656083 +0000 2024-04-25T14:05:29.678 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: 2024-04-25 13:59:44.256656083 +0000 2024-04-25T14:05:29.678 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T14:05:29.740 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-04-25T14:05:29.740 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-04-25T14:05:29.740 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000214876 s, 2.4 MB/s 2024-04-25T14:05:29.741 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T14:05:29.797 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_3 2024-04-25T14:05:29.852 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T14:05:29.852 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:05:29.852 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-25T14:05:29.852 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:05:29.852 INFO:teuthology.orchestra.run.smithi151.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:05:29.852 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-04-25 14:05:06.324370484 +0000 2024-04-25T14:05:29.852 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-04-25 13:59:44.585651417 +0000 2024-04-25T14:05:29.852 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-04-25 13:59:44.585651417 +0000 2024-04-25T14:05:29.853 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: 2024-04-25 13:59:44.585651417 +0000 2024-04-25T14:05:29.853 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T14:05:29.914 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-04-25T14:05:29.915 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-04-25T14:05:29.915 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000200354 s, 2.6 MB/s 2024-04-25T14:05:29.916 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T14:05:29.974 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_4 2024-04-25T14:05:30.030 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T14:05:30.030 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:05:30.030 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-25T14:05:30.030 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:05:30.030 INFO:teuthology.orchestra.run.smithi151.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:05:30.030 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-04-25 14:05:06.324370484 +0000 2024-04-25T14:05:30.030 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-04-25 13:59:44.932646496 +0000 2024-04-25T14:05:30.031 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-04-25 13:59:44.932646496 +0000 2024-04-25T14:05:30.031 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: 2024-04-25 13:59:44.932646496 +0000 2024-04-25T14:05:30.031 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T14:05:30.094 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-04-25T14:05:30.094 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-04-25T14:05:30.094 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.00019148 s, 2.7 MB/s 2024-04-25T14:05:30.095 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T14:05:30.150 INFO:tasks.cephadm:Deploying osd.0 on smithi096 with /dev/vg_nvme/lv_4... 2024-04-25T14:05:30.151 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T14:05:30.176 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:29 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:30.176 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:29 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:30.176 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:29 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:05:30.176 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:29 smithi151 ceph-mon[32990]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:30.176 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:29 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:30.176 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:29 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:05:30.177 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:29 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:30.177 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:29 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T14:05:30.177 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:29 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T14:05:30.177 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:29 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T14:05:30.177 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:29 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:30.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:29 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:30.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:29 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:30.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:29 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:05:30.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:29 smithi096 ceph-mon[29207]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:30.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:29 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:30.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:29 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:05:30.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:29 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:30.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:29 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T14:05:30.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:29 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T14:05:30.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:29 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T14:05:30.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:29 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:30.388 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:05:30.422 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:29 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:30.422 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:29 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:30.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:29 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:05:30.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:29 smithi097 ceph-mon[32720]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:30.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:29 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:30.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:29 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:05:30.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:29 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:30.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:29 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T14:05:30.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:29 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T14:05:30.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:29 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T14:05:30.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:29 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:31.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:30 smithi096 ceph-mon[29207]: Deploying daemon mgr.z on smithi151 2024-04-25T14:05:31.329 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:30 smithi151 ceph-mon[32990]: Deploying daemon mgr.z on smithi151 2024-04-25T14:05:31.422 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:30 smithi097 ceph-mon[32720]: Deploying daemon mgr.z on smithi151 2024-04-25T14:05:31.688 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Apr 25 14:05:31 smithi151 systemd[1]: Starting Ceph mgr.z for 59334182-030c-11ef-bc93-c7b262605968... 2024-04-25T14:05:32.014 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:31 smithi151 ceph-mon[32990]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:32.014 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Apr 25 14:05:31 smithi151 podman[34825]: 2024-04-25 14:05:31.666169082 +0000 UTC m=+0.017901458 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:05:32.014 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Apr 25 14:05:31 smithi151 podman[34825]: 2024-04-25 14:05:31.800710944 +0000 UTC m=+0.152443291 container create e616504caf4aa5e4095e97a508a2adc42b3608fab4c219c9b49318395656ac60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=) 2024-04-25T14:05:32.014 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Apr 25 14:05:32 smithi151 podman[34825]: 2024-04-25 14:05:32.011759504 +0000 UTC m=+0.363491856 container init e616504caf4aa5e4095e97a508a2adc42b3608fab4c219c9b49318395656ac60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-25T14:05:32.014 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Apr 25 14:05:32 smithi151 podman[34825]: 2024-04-25 14:05:32.015124772 +0000 UTC m=+0.366857119 container start e616504caf4aa5e4095e97a508a2adc42b3608fab4c219c9b49318395656ac60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:05:32.268 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:31 smithi096 ceph-mon[29207]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:32.268 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T14:05:32.269 INFO:teuthology.orchestra.run.smithi096.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T14:05:32.269 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10+0 records in 2024-04-25T14:05:32.269 INFO:teuthology.orchestra.run.smithi096.stderr:10+0 records out 2024-04-25T14:05:32.269 INFO:teuthology.orchestra.run.smithi096.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0132617 s, 791 MB/s 2024-04-25T14:05:32.269 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping successful for: 2024-04-25T14:05:32.318 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Apr 25 14:05:32 smithi151 ceph-mgr[34844]: -- 172.21.15.151:0/2011600252 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55809dc785a0 con 0x5580a2a02000 2024-04-25T14:05:32.318 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Apr 25 14:05:32 smithi151 bash[34825]: e616504caf4aa5e4095e97a508a2adc42b3608fab4c219c9b49318395656ac60 2024-04-25T14:05:32.318 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Apr 25 14:05:32 smithi151 systemd[1]: Started Ceph mgr.z for 59334182-030c-11ef-bc93-c7b262605968. 2024-04-25T14:05:32.318 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Apr 25 14:05:32 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-z[34840]: 2024-04-25T14:05:32.183+0000 7f7c7049a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:05:32.324 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:31 smithi097 ceph-mon[32720]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:32.687 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Apr 25 14:05:32 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-z[34840]: 2024-04-25T14:05:32.375+0000 7f7c7049a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:05:32.688 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Apr 25 14:05:32 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-z[34840]: 2024-04-25T14:05:32.486+0000 7f7c7049a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:05:32.927 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi096:vg_nvme/lv_4 2024-04-25T14:05:33.161 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:05:33.187 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Apr 25 14:05:32 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-z[34840]: 2024-04-25T14:05:32.809+0000 7f7c7049a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:05:33.188 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Apr 25 14:05:32 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-z[34840]: 2024-04-25T14:05:32.893+0000 7f7c7049a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:05:33.422 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:33 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:33.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:33 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:33.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:33 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:33.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:33 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T14:05:33.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:33 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T14:05:33.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:33 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T14:05:33.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:33 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:33.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:33 smithi097 ceph-mon[32720]: Deploying daemon mgr.y on smithi097 2024-04-25T14:05:33.446 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:33 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:33.446 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:33 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:33.446 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:33 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:33.446 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:33 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T14:05:33.446 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:33 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T14:05:33.446 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:33 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T14:05:33.446 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:33 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:33.447 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:33 smithi096 ceph-mon[29207]: Deploying daemon mgr.y on smithi097 2024-04-25T14:05:33.499 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:33 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:33.499 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:33 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:33.499 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:33 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:33.499 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:33 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T14:05:33.500 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:33 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T14:05:33.500 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:33 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T14:05:33.500 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:33 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:33.500 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:33 smithi151 ceph-mon[32990]: Deploying daemon mgr.y on smithi097 2024-04-25T14:05:34.391 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:34 smithi096 ceph-mon[29207]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:34.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:34 smithi097 ceph-mon[32720]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:34.423 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 14:05:34 smithi097 systemd[1]: Starting Ceph mgr.y for 59334182-030c-11ef-bc93-c7b262605968... 2024-04-25T14:05:34.426 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:34 smithi151 ceph-mon[32990]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:34.426 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Apr 25 14:05:34 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-z[34840]: 2024-04-25T14:05:34.084+0000 7f7c7049a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:05:34.426 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Apr 25 14:05:34 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-z[34840]: 2024-04-25T14:05:34.254+0000 7f7c7049a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:05:34.426 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Apr 25 14:05:34 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-z[34840]: 2024-04-25T14:05:34.425+0000 7f7c7049a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:05:34.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:34.637+0000 7f51a34c9700 1 -- 172.21.15.96:0/1765985735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f519c104210 msgr2=0x7f519c104620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:34.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:34.637+0000 7f51a34c9700 1 --2- 172.21.15.96:0/1765985735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f519c104210 0x7f519c104620 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f518c009990 tx=0x7f518c009ca0 comp rx=0 tx=0).stop 2024-04-25T14:05:34.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:34.637+0000 7f51a34c9700 1 -- 172.21.15.96:0/1765985735 shutdown_connections 2024-04-25T14:05:34.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:34.637+0000 7f51a34c9700 1 --2- 172.21.15.96:0/1765985735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f519c104bb0 0x7f519c10b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:34.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:34.637+0000 7f51a34c9700 1 --2- 172.21.15.96:0/1765985735 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f519c1030e0 0x7f519c1034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:34.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:34.637+0000 7f51a34c9700 1 --2- 172.21.15.96:0/1765985735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f519c104210 0x7f519c104620 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:34.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:34.637+0000 7f51a34c9700 1 -- 172.21.15.96:0/1765985735 >> 172.21.15.96:0/1765985735 conn(0x7f519c0fe740 msgr2=0x7f519c100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:34.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:34.637+0000 7f51a34c9700 1 -- 172.21.15.96:0/1765985735 shutdown_connections 2024-04-25T14:05:34.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:34.637+0000 7f51a34c9700 1 -- 172.21.15.96:0/1765985735 wait complete. 2024-04-25T14:05:34.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:34.638+0000 7f51a34c9700 1 Processor -- start 2024-04-25T14:05:34.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:34.638+0000 7f51a34c9700 1 -- start start 2024-04-25T14:05:34.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:34.638+0000 7f51a34c9700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f519c1030e0 0x7f519c19a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:34.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:34.638+0000 7f51a34c9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f519c104210 0x7f519c19ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:34.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:34.638+0000 7f51a34c9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f519c104bb0 0x7f519c1a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:34.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:34.638+0000 7f51a34c9700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f519c10eb60 con 0x7f519c104210 2024-04-25T14:05:34.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:34.638+0000 7f51a34c9700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f519c10e9e0 con 0x7f519c104bb0 2024-04-25T14:05:34.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:34.638+0000 7f51a34c9700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f519c10ece0 con 0x7f519c1030e0 2024-04-25T14:05:34.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:34.638+0000 7f51a0a64700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f519c104210 0x7f519c19ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:34.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:34.638+0000 7f51a0a64700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f519c104210 0x7f519c19ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50468/0 (socket says 172.21.15.96:50468) 2024-04-25T14:05:34.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:34.638+0000 7f51a0a64700 1 -- 172.21.15.96:0/4289105337 learned_addr learned my addr 172.21.15.96:0/4289105337 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:05:34.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:34.638+0000 7f51a1265700 1 --2- 172.21.15.96:0/4289105337 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f519c1030e0 0x7f519c19a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:34.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:34.638+0000 7f51a1a66700 1 --2- 172.21.15.96:0/4289105337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f519c104bb0 0x7f519c1a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:34.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:34.639+0000 7f51a0a64700 1 -- 172.21.15.96:0/4289105337 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f519c1030e0 msgr2=0x7f519c19a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:34.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:34.639+0000 7f51a0a64700 1 --2- 172.21.15.96:0/4289105337 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f519c1030e0 0x7f519c19a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:34.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:34.639+0000 7f51a0a64700 1 -- 172.21.15.96:0/4289105337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f519c104bb0 msgr2=0x7f519c1a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:34.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:34.639+0000 7f51a0a64700 1 --2- 172.21.15.96:0/4289105337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f519c104bb0 0x7f519c1a1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:34.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:34.639+0000 7f51a0a64700 1 -- 172.21.15.96:0/4289105337 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f519c1a2370 con 0x7f519c104210 2024-04-25T14:05:34.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:34.639+0000 7f51a0a64700 1 --2- 172.21.15.96:0/4289105337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f519c104210 0x7f519c19ad50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f518c0098e0 tx=0x7f518c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:34.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:34.639+0000 7f51927fc700 1 -- 172.21.15.96:0/4289105337 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518c026070 con 0x7f519c104210 2024-04-25T14:05:34.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:34.639+0000 7f51927fc700 1 -- 172.21.15.96:0/4289105337 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f518c021410 con 0x7f519c104210 2024-04-25T14:05:34.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:34.639+0000 7f51a34c9700 1 -- 172.21.15.96:0/4289105337 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f519c1a2600 con 0x7f519c104210 2024-04-25T14:05:34.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:34.639+0000 7f51927fc700 1 -- 172.21.15.96:0/4289105337 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518c017650 con 0x7f519c104210 2024-04-25T14:05:34.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:34.639+0000 7f51a34c9700 1 -- 172.21.15.96:0/4289105337 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f519c1a2a50 con 0x7f519c104210 2024-04-25T14:05:34.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:34.640+0000 7f51a34c9700 1 -- 172.21.15.96:0/4289105337 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f519c051190 con 0x7f519c104210 2024-04-25T14:05:34.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:34.640+0000 7f51927fc700 1 -- 172.21.15.96:0/4289105337 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f518c029070 con 0x7f519c104210 2024-04-25T14:05:34.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:34.640+0000 7f51927fc700 1 --2- 172.21.15.96:0/4289105337 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f518803d850 0x7f518803fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:34.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:34.641+0000 7f51927fc700 1 -- 172.21.15.96:0/4289105337 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f518c0527c0 con 0x7f519c104210 2024-04-25T14:05:34.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:34.641+0000 7f51a1265700 1 --2- 172.21.15.96:0/4289105337 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f518803d850 0x7f518803fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:34.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:34.641+0000 7f51a1265700 1 --2- 172.21.15.96:0/4289105337 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f518803d850 0x7f518803fd60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f519c104110 tx=0x7f5198009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:34.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:34.646+0000 7f51927fc700 1 -- 172.21.15.96:0/4289105337 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f518c013070 con 0x7f519c104210 2024-04-25T14:05:34.767 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 14:05:34 smithi097 podman[34102]: 2024-04-25 14:05:34.495338012 +0000 UTC m=+0.093731664 container create 38308e63f649ec1908770fae5052036b5a046f75404b34cafd5a11fb87b23253 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-y, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:05:34.768 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 14:05:34 smithi097 podman[34102]: 2024-04-25 14:05:34.419747056 +0000 UTC m=+0.018140731 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:05:34.768 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 14:05:34 smithi097 podman[34102]: 2024-04-25 14:05:34.731252997 +0000 UTC m=+0.329646648 container init 38308e63f649ec1908770fae5052036b5a046f75404b34cafd5a11fb87b23253 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-y, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T14:05:34.768 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 14:05:34 smithi097 podman[34102]: 2024-04-25 14:05:34.733742862 +0000 UTC m=+0.332136514 container start 38308e63f649ec1908770fae5052036b5a046f75404b34cafd5a11fb87b23253 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-y, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:05:34.768 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 14:05:34 smithi097 ceph-mgr[34121]: -- 172.21.15.97:0/1677046255 <== mon.2 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b548e1a5a0 con 0x55b54dba4900 2024-04-25T14:05:34.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:34.850+0000 7f51a34c9700 1 -- 172.21.15.96:0/4289105337 --> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f519c100770 con 0x7f518803d850 2024-04-25T14:05:35.008 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Apr 25 14:05:34 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-z[34840]: 2024-04-25T14:05:34.711+0000 7f7c7049a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:05:35.008 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Apr 25 14:05:34 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-z[34840]: 2024-04-25T14:05:34.802+0000 7f7c7049a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:05:35.009 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Apr 25 14:05:35 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-z[34840]: 2024-04-25T14:05:35.008+0000 7f7c7049a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:05:35.093 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 14:05:34 smithi097 bash[34102]: 38308e63f649ec1908770fae5052036b5a046f75404b34cafd5a11fb87b23253 2024-04-25T14:05:35.093 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 14:05:34 smithi097 systemd[1]: Started Ceph mgr.y for 59334182-030c-11ef-bc93-c7b262605968. 2024-04-25T14:05:35.094 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 14:05:34 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-y[34117]: 2024-04-25T14:05:34.904+0000 7f2665c80200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:05:35.094 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 14:05:35 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-y[34117]: 2024-04-25T14:05:35.094+0000 7f2665c80200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:05:35.422 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 14:05:35 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-y[34117]: 2024-04-25T14:05:35.200+0000 7f2665c80200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:05:35.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:35 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:05:35.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:35 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:05:35.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:35 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:35.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:35 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:35.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:35 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:35.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:35 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:35.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:35 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:35.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:35 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:05:35.437 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Apr 25 14:05:35 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-z[34840]: 2024-04-25T14:05:35.208+0000 7f7c7049a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:05:35.437 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:35 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:05:35.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:35 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:05:35.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:35 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:35.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:35 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:35.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:35 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:35.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:35 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:35.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:35 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:35.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:35 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:05:35.462 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:35 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:05:35.462 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:35 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:05:35.462 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:35 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:35.462 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:35 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:35.462 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:35 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:35.462 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:35 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:35.462 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:35 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:35.462 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:35 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:05:35.809 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 14:05:35 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-y[34117]: 2024-04-25T14:05:35.523+0000 7f2665c80200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:05:35.809 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 14:05:35 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-y[34117]: 2024-04-25T14:05:35.619+0000 7f2665c80200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:05:36.106 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Apr 25 14:05:35 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-z[34840]: 2024-04-25T14:05:35.856+0000 7f7c7049a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T14:05:36.397 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:36 smithi151 ceph-mon[32990]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:05:36.397 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:36 smithi151 ceph-mon[32990]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:36.397 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Apr 25 14:05:36 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-z[34840]: 2024-04-25T14:05:36.290+0000 7f7c7049a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:05:36.397 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Apr 25 14:05:36 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-z[34840]: 2024-04-25T14:05:36.396+0000 7f7c7049a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:05:36.469 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:36 smithi096 ceph-mon[29207]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:05:36.469 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:36 smithi096 ceph-mon[29207]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:36.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:36 smithi097 ceph-mon[32720]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:05:36.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:36 smithi097 ceph-mon[32720]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:36.687 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Apr 25 14:05:36 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-z[34840]: 2024-04-25T14:05:36.602+0000 7f7c7049a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T14:05:37.161 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 14:05:36 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-y[34117]: 2024-04-25T14:05:36.830+0000 7f2665c80200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:05:37.161 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 14:05:36 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-y[34117]: 2024-04-25T14:05:36.995+0000 7f2665c80200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:05:37.161 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 14:05:37 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-y[34117]: 2024-04-25T14:05:37.161+0000 7f2665c80200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:05:37.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:37 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:37.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:37 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:37.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:37 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:37.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:37 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:37.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:37 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:37.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:37 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:37.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:37 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:05:37.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:37 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:37.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:37 smithi097 ceph-mon[32720]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T14:05:37.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:37 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T14:05:37.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:37 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T14:05:37.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:37 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:37.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:37 smithi097 ceph-mon[32720]: Reconfiguring daemon mgr.x on smithi096 2024-04-25T14:05:37.437 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Apr 25 14:05:37 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-z[34840]: 2024-04-25T14:05:37.124+0000 7f7c7049a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:05:37.438 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Apr 25 14:05:37 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-z[34840]: 2024-04-25T14:05:37.216+0000 7f7c7049a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:05:37.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:37 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:37.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:37 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:37.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:37 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:37.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:37 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:37.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:37 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:37.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:37 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:37.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:37 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:05:37.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:37 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:37.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:37 smithi151 ceph-mon[32990]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T14:05:37.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:37 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T14:05:37.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:37 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T14:05:37.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:37 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:37.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:37 smithi151 ceph-mon[32990]: Reconfiguring daemon mgr.x on smithi096 2024-04-25T14:05:37.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:37 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:37.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:37 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:37.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:37 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:37.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:37 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:37.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:37 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:37.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:37 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:37.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:37 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:05:37.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:37 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:37.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:37 smithi096 ceph-mon[29207]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T14:05:37.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:37 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T14:05:37.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:37 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T14:05:37.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:37 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:37.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:37 smithi096 ceph-mon[29207]: Reconfiguring daemon mgr.x on smithi096 2024-04-25T14:05:37.728 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 14:05:37 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-y[34117]: 2024-04-25T14:05:37.441+0000 7f2665c80200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:05:37.728 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 14:05:37 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-y[34117]: 2024-04-25T14:05:37.530+0000 7f2665c80200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:05:37.728 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 14:05:37 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-y[34117]: 2024-04-25T14:05:37.728+0000 7f2665c80200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:05:37.802 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Apr 25 14:05:37 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-z[34840]: 2024-04-25T14:05:37.459+0000 7f7c7049a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:05:37.802 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Apr 25 14:05:37 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-z[34840]: 2024-04-25T14:05:37.592+0000 7f7c7049a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:05:37.802 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Apr 25 14:05:37 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-z[34840]: 2024-04-25T14:05:37.685+0000 7f7c7049a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:05:37.802 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Apr 25 14:05:37 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-z[34840]: 2024-04-25T14:05:37.801+0000 7f7c7049a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:05:38.172 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 14:05:37 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-y[34117]: 2024-04-25T14:05:37.916+0000 7f2665c80200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:05:38.438 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Apr 25 14:05:38 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-z[34840]: 2024-04-25T14:05:38.105+0000 7f7c7049a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:05:38.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:38 smithi151 ceph-mon[32990]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:38.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:38 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/2955320243' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf4ba3ac-142e-4e8e-96d4-410d77157eb0"}]: dispatch 2024-04-25T14:05:38.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:38 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/2955320243' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cf4ba3ac-142e-4e8e-96d4-410d77157eb0"}]': finished 2024-04-25T14:05:38.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:38 smithi151 ceph-mon[32990]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T14:05:38.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:38 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:05:38.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:38 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/3737080654' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:05:38.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:38 smithi096 ceph-mon[29207]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:38.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:38 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/2955320243' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf4ba3ac-142e-4e8e-96d4-410d77157eb0"}]: dispatch 2024-04-25T14:05:38.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:38 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/2955320243' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cf4ba3ac-142e-4e8e-96d4-410d77157eb0"}]': finished 2024-04-25T14:05:38.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:38 smithi096 ceph-mon[29207]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T14:05:38.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:38 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:05:38.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:38 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/3737080654' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:05:38.672 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 14:05:38 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-y[34117]: 2024-04-25T14:05:38.501+0000 7f2665c80200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T14:05:38.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:38 smithi097 ceph-mon[32720]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:38.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:38 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/2955320243' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf4ba3ac-142e-4e8e-96d4-410d77157eb0"}]: dispatch 2024-04-25T14:05:38.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:38 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/2955320243' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cf4ba3ac-142e-4e8e-96d4-410d77157eb0"}]': finished 2024-04-25T14:05:38.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:38 smithi097 ceph-mon[32720]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T14:05:38.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:38 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:05:38.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:38 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/3737080654' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:05:39.172 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 14:05:38 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-y[34117]: 2024-04-25T14:05:38.912+0000 7f2665c80200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:05:39.172 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 14:05:39 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-y[34117]: 2024-04-25T14:05:39.004+0000 7f2665c80200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:05:39.188 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Apr 25 14:05:38 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-z[34840]: 2024-04-25T14:05:38.706+0000 7f7c7049a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:05:39.188 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Apr 25 14:05:38 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-z[34840]: 2024-04-25T14:05:38.924+0000 7f7c7049a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:05:39.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:39.310+0000 7f51927fc700 1 -- 172.21.15.96:0/4289105337 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 14) ==== 98518+0+0 (secure 0 0 0) 0x7f518c009d10 con 0x7f519c104210 2024-04-25T14:05:39.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:39 smithi096 ceph-mon[29207]: Standby manager daemon z started 2024-04-25T14:05:39.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:39 smithi096 ceph-mon[29207]: from='mgr.? 172.21.15.151:0/586808641' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T14:05:39.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:39 smithi096 ceph-mon[29207]: from='mgr.? 172.21.15.151:0/586808641' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T14:05:39.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:39 smithi096 ceph-mon[29207]: from='mgr.? 172.21.15.151:0/586808641' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T14:05:39.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:39 smithi096 ceph-mon[29207]: from='mgr.? 172.21.15.151:0/586808641' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T14:05:39.672 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 14:05:39 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-y[34117]: 2024-04-25T14:05:39.200+0000 7f2665c80200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T14:05:39.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:39 smithi097 ceph-mon[32720]: Standby manager daemon z started 2024-04-25T14:05:39.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:39 smithi097 ceph-mon[32720]: from='mgr.? 172.21.15.151:0/586808641' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T14:05:39.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:39 smithi097 ceph-mon[32720]: from='mgr.? 172.21.15.151:0/586808641' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T14:05:39.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:39 smithi097 ceph-mon[32720]: from='mgr.? 172.21.15.151:0/586808641' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T14:05:39.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:39 smithi097 ceph-mon[32720]: from='mgr.? 172.21.15.151:0/586808641' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T14:05:39.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:39 smithi151 ceph-mon[32990]: Standby manager daemon z started 2024-04-25T14:05:39.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:39 smithi151 ceph-mon[32990]: from='mgr.? 172.21.15.151:0/586808641' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T14:05:39.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:39 smithi151 ceph-mon[32990]: from='mgr.? 172.21.15.151:0/586808641' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T14:05:39.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:39 smithi151 ceph-mon[32990]: from='mgr.? 172.21.15.151:0/586808641' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T14:05:39.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:39 smithi151 ceph-mon[32990]: from='mgr.? 172.21.15.151:0/586808641' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T14:05:40.032 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 14:05:39 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-y[34117]: 2024-04-25T14:05:39.709+0000 7f2665c80200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:05:40.032 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 14:05:39 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-y[34117]: 2024-04-25T14:05:39.798+0000 7f2665c80200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:05:40.321 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 14:05:40 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-y[34117]: 2024-04-25T14:05:40.032+0000 7f2665c80200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:05:40.321 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 14:05:40 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-y[34117]: 2024-04-25T14:05:40.162+0000 7f2665c80200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:05:40.321 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 14:05:40 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-y[34117]: 2024-04-25T14:05:40.253+0000 7f2665c80200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:05:40.608 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:40 smithi096 ceph-mon[29207]: mgrmap e14: x(active, since 78s), standbys: z 2024-04-25T14:05:40.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:40 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T14:05:40.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:40 smithi096 ceph-mon[29207]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:40.661 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:40 smithi097 ceph-mon[32720]: mgrmap e14: x(active, since 78s), standbys: z 2024-04-25T14:05:40.661 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:40 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T14:05:40.661 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:40 smithi097 ceph-mon[32720]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:40.661 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 14:05:40 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-y[34117]: 2024-04-25T14:05:40.367+0000 7f2665c80200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:05:40.661 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 14:05:40 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-y[34117]: 2024-04-25T14:05:40.661+0000 7f2665c80200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:05:40.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:40 smithi151 ceph-mon[32990]: mgrmap e14: x(active, since 78s), standbys: z 2024-04-25T14:05:40.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:40 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T14:05:40.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:40 smithi151 ceph-mon[32990]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:41.672 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 14:05:41 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-y[34117]: 2024-04-25T14:05:41.246+0000 7f2665c80200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:05:41.672 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 14:05:41 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-y[34117]: 2024-04-25T14:05:41.454+0000 7f2665c80200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:05:42.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:42.390+0000 7f51927fc700 1 -- 172.21.15.96:0/4289105337 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mgrmap(e 15) ==== 147464+0+0 (secure 0 0 0) 0x7f518c02acc0 con 0x7f519c104210 2024-04-25T14:05:42.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:42 smithi096 ceph-mon[29207]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:42.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:42 smithi096 ceph-mon[29207]: Standby manager daemon y started 2024-04-25T14:05:42.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:42 smithi096 ceph-mon[29207]: from='mgr.? 172.21.15.97:0/1242175176' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T14:05:42.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:42 smithi096 ceph-mon[29207]: from='mgr.? 172.21.15.97:0/1242175176' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T14:05:42.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:42 smithi096 ceph-mon[29207]: from='mgr.? 172.21.15.97:0/1242175176' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T14:05:42.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:42 smithi096 ceph-mon[29207]: from='mgr.? 172.21.15.97:0/1242175176' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T14:05:42.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:42 smithi097 ceph-mon[32720]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:42.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:42 smithi097 ceph-mon[32720]: Standby manager daemon y started 2024-04-25T14:05:42.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:42 smithi097 ceph-mon[32720]: from='mgr.? 172.21.15.97:0/1242175176' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T14:05:42.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:42 smithi097 ceph-mon[32720]: from='mgr.? 172.21.15.97:0/1242175176' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T14:05:42.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:42 smithi097 ceph-mon[32720]: from='mgr.? 172.21.15.97:0/1242175176' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T14:05:42.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:42 smithi097 ceph-mon[32720]: from='mgr.? 172.21.15.97:0/1242175176' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T14:05:42.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:42 smithi151 ceph-mon[32990]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:42.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:42 smithi151 ceph-mon[32990]: Standby manager daemon y started 2024-04-25T14:05:42.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:42 smithi151 ceph-mon[32990]: from='mgr.? 172.21.15.97:0/1242175176' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T14:05:42.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:42 smithi151 ceph-mon[32990]: from='mgr.? 172.21.15.97:0/1242175176' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T14:05:42.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:42 smithi151 ceph-mon[32990]: from='mgr.? 172.21.15.97:0/1242175176' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T14:05:42.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:42 smithi151 ceph-mon[32990]: from='mgr.? 172.21.15.97:0/1242175176' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T14:05:43.530 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:43 smithi096 ceph-mon[29207]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-04-25T14:05:43.530 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:43 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T14:05:43.530 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:43 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:43.530 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:43 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:43.530 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:43 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:05:43.530 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:43 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:43.531 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:43 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:05:43.531 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:43 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:43.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:43 smithi097 ceph-mon[32720]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-04-25T14:05:43.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:43 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T14:05:43.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:43 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:43.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:43 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:43.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:43 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:05:43.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:43 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:43.674 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:43 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:05:43.674 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:43 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:43.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:43 smithi151 ceph-mon[32990]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-04-25T14:05:43.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:43 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T14:05:43.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:43 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:43.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:43 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:43.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:43 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:05:43.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:43 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:43.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:43 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:05:43.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:43 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:44.662 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:44 smithi096 ceph-mon[29207]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:44.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:44 smithi097 ceph-mon[32720]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:44.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:44 smithi151 ceph-mon[32990]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:46.485 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:46 smithi096 ceph-mon[29207]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:46.485 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:46 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T14:05:46.485 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:46 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:46.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:46 smithi097 ceph-mon[32720]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:46.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:46 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T14:05:46.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:46 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:46.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:46 smithi151 ceph-mon[32990]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:46.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:46 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T14:05:46.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:46 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:47.480 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:47 smithi096 ceph-mon[29207]: Deploying daemon osd.0 on smithi096 2024-04-25T14:05:47.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:47 smithi097 ceph-mon[32720]: Deploying daemon osd.0 on smithi096 2024-04-25T14:05:47.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:47 smithi151 ceph-mon[32990]: Deploying daemon osd.0 on smithi096 2024-04-25T14:05:48.510 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:48 smithi096 ceph-mon[29207]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:48.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:48 smithi097 ceph-mon[32720]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:48.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:48 smithi151 ceph-mon[32990]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:50.654 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:50 smithi096 ceph-mon[29207]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:50.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:50 smithi097 ceph-mon[32720]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:50.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:50 smithi151 ceph-mon[32990]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:52.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:52 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:52.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:52 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:52.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:52 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:05:52.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:52 smithi096 ceph-mon[29207]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:52.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:52 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:52.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:52 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:52.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:52 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:05:52.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:52 smithi097 ceph-mon[32720]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:52.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:52 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:52.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:52 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:52.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:52 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:05:52.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:52 smithi151 ceph-mon[32990]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:54.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:54 smithi097 ceph-mon[32720]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:54.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:54 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:54.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:54 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:54.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:54 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:54.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:54 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:05:54.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:54 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:54.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:54 smithi151 ceph-mon[32990]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:54.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:54 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:54.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:54 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:54.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:54 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:54.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:54 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:05:54.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:54 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:54.754 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:54 smithi096 ceph-mon[29207]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:54.754 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:54 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:54.754 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:54 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:54.754 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:54 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:54.754 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:54 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:05:54.754 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:54 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:54.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:54.806+0000 7f51927fc700 1 -- 172.21.15.96:0/4289105337 <== mgr.14152 v2:172.21.15.96:6800/536332425 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f519c100770 con 0x7f518803d850 2024-04-25T14:05:54.806 INFO:teuthology.orchestra.run.smithi096.stdout:Created osd(s) 0 on host 'smithi096' 2024-04-25T14:05:54.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:54.808+0000 7f51a34c9700 1 -- 172.21.15.96:0/4289105337 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f518803d850 msgr2=0x7f518803fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:54.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:54.808+0000 7f51a34c9700 1 --2- 172.21.15.96:0/4289105337 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f518803d850 0x7f518803fd60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f519c104110 tx=0x7f5198009260 comp rx=0 tx=0).stop 2024-04-25T14:05:54.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:54.809+0000 7f51a34c9700 1 -- 172.21.15.96:0/4289105337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f519c104210 msgr2=0x7f519c19ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:54.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:54.809+0000 7f51a34c9700 1 --2- 172.21.15.96:0/4289105337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f519c104210 0x7f519c19ad50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f518c0098e0 tx=0x7f518c0047b0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:54.809+0000 7f51a34c9700 1 -- 172.21.15.96:0/4289105337 shutdown_connections 2024-04-25T14:05:54.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:54.809+0000 7f51a34c9700 1 --2- 172.21.15.96:0/4289105337 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f518803d850 0x7f518803fd60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:54.809+0000 7f51a34c9700 1 --2- 172.21.15.96:0/4289105337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f519c104bb0 0x7f519c1a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:54.809+0000 7f51a34c9700 1 --2- 172.21.15.96:0/4289105337 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f519c1030e0 0x7f519c19a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:54.809+0000 7f51a34c9700 1 --2- 172.21.15.96:0/4289105337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f519c104210 0x7f519c19ad50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:54.809+0000 7f51a34c9700 1 -- 172.21.15.96:0/4289105337 >> 172.21.15.96:0/4289105337 conn(0x7f519c0fe740 msgr2=0x7f519c100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:54.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:54.809+0000 7f51a34c9700 1 -- 172.21.15.96:0/4289105337 shutdown_connections 2024-04-25T14:05:54.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:05:54.809+0000 7f51a34c9700 1 -- 172.21.15.96:0/4289105337 wait complete. 2024-04-25T14:05:55.528 DEBUG:teuthology.orchestra.run.smithi096:osd.0> sudo journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@osd.0.service 2024-04-25T14:05:55.530 INFO:tasks.cephadm:Deploying osd.1 on smithi097 with /dev/vg_nvme/lv_4... 2024-04-25T14:05:55.530 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T14:05:55.771 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T14:05:55.894 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:55 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:55.894 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:55 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:05:55.894 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:55 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:55.894 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:55 smithi097 ceph-mon[32720]: from='osd.0 [v2:172.21.15.96:6802/3832506168,v1:172.21.15.96:6803/3832506168]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T14:05:55.894 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:55 smithi097 ceph-mon[32720]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:56.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:55 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:56.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:55 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:05:56.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:55 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:56.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:55 smithi096 ceph-mon[29207]: from='osd.0 [v2:172.21.15.96:6802/3832506168,v1:172.21.15.96:6803/3832506168]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T14:05:56.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:55 smithi096 ceph-mon[29207]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:56.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:55 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:56.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:55 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:05:56.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:55 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:56.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:55 smithi151 ceph-mon[32990]: from='osd.0 [v2:172.21.15.96:6802/3832506168,v1:172.21.15.96:6803/3832506168]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T14:05:56.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:55 smithi151 ceph-mon[32990]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:57.012 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:56 smithi097 ceph-mon[32720]: from='osd.0 [v2:172.21.15.96:6802/3832506168,v1:172.21.15.96:6803/3832506168]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T14:05:57.012 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:56 smithi097 ceph-mon[32720]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T14:05:57.012 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:56 smithi097 ceph-mon[32720]: from='osd.0 [v2:172.21.15.96:6802/3832506168,v1:172.21.15.96:6803/3832506168]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-04-25T14:05:57.013 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:56 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:05:57.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:56 smithi096 ceph-mon[29207]: from='osd.0 [v2:172.21.15.96:6802/3832506168,v1:172.21.15.96:6803/3832506168]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T14:05:57.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:56 smithi096 ceph-mon[29207]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T14:05:57.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:56 smithi096 ceph-mon[29207]: from='osd.0 [v2:172.21.15.96:6802/3832506168,v1:172.21.15.96:6803/3832506168]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-04-25T14:05:57.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:56 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:05:57.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:56 smithi151 ceph-mon[32990]: from='osd.0 [v2:172.21.15.96:6802/3832506168,v1:172.21.15.96:6803/3832506168]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T14:05:57.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:56 smithi151 ceph-mon[32990]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T14:05:57.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:56 smithi151 ceph-mon[32990]: from='osd.0 [v2:172.21.15.96:6802/3832506168,v1:172.21.15.96:6803/3832506168]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-04-25T14:05:57.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:56 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:05:57.360 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 14:05:57 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-0[44132]: 2024-04-25T14:05:57.278+0000 7f9e9fbeb700 -1 osd.0 0 waiting for initial osdmap 2024-04-25T14:05:57.360 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 14:05:57 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-0[44132]: 2024-04-25T14:05:57.282+0000 7f9e97197700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:05:57.598 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T14:05:57.598 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T14:05:57.598 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2024-04-25T14:05:57.598 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2024-04-25T14:05:57.598 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125341 s, 837 MB/s 2024-04-25T14:05:57.599 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2024-04-25T14:05:58.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:57 smithi096 ceph-mon[29207]: from='osd.0 [v2:172.21.15.96:6802/3832506168,v1:172.21.15.96:6803/3832506168]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-04-25T14:05:58.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:57 smithi096 ceph-mon[29207]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T14:05:58.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:57 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:05:58.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:57 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:05:58.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:57 smithi096 ceph-mon[29207]: from='osd.0 [v2:172.21.15.96:6802/3832506168,v1:172.21.15.96:6803/3832506168]' entity='osd.0' 2024-04-25T14:05:58.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:57 smithi096 ceph-mon[29207]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:58.172 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:57 smithi097 ceph-mon[32720]: from='osd.0 [v2:172.21.15.96:6802/3832506168,v1:172.21.15.96:6803/3832506168]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-04-25T14:05:58.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:57 smithi097 ceph-mon[32720]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T14:05:58.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:57 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:05:58.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:57 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:05:58.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:57 smithi097 ceph-mon[32720]: from='osd.0 [v2:172.21.15.96:6802/3832506168,v1:172.21.15.96:6803/3832506168]' entity='osd.0' 2024-04-25T14:05:58.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:57 smithi097 ceph-mon[32720]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:58.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:57 smithi151 ceph-mon[32990]: from='osd.0 [v2:172.21.15.96:6802/3832506168,v1:172.21.15.96:6803/3832506168]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-04-25T14:05:58.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:57 smithi151 ceph-mon[32990]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T14:05:58.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:57 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:05:58.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:57 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:05:58.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:57 smithi151 ceph-mon[32990]: from='osd.0 [v2:172.21.15.96:6802/3832506168,v1:172.21.15.96:6803/3832506168]' entity='osd.0' 2024-04-25T14:05:58.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:57 smithi151 ceph-mon[32990]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:05:58.205 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi097:vg_nvme/lv_4 2024-04-25T14:05:58.455 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T14:05:58.925 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:58 smithi096 ceph-mon[29207]: purged_snaps scrub starts 2024-04-25T14:05:58.925 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:58 smithi096 ceph-mon[29207]: purged_snaps scrub ok 2024-04-25T14:05:58.925 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:58 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:05:58.925 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:58 smithi096 ceph-mon[29207]: osd.0 [v2:172.21.15.96:6802/3832506168,v1:172.21.15.96:6803/3832506168] boot 2024-04-25T14:05:58.925 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:58 smithi096 ceph-mon[29207]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T14:05:58.925 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:58 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:05:58.925 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:58 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:58.925 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:58 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:58.925 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:58 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:05:58.926 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:58 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:58.926 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:58 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:58.926 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:58 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:05:58.926 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:58 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:59.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:58 smithi097 ceph-mon[32720]: purged_snaps scrub starts 2024-04-25T14:05:59.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:58 smithi097 ceph-mon[32720]: purged_snaps scrub ok 2024-04-25T14:05:59.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:58 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:05:59.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:58 smithi097 ceph-mon[32720]: osd.0 [v2:172.21.15.96:6802/3832506168,v1:172.21.15.96:6803/3832506168] boot 2024-04-25T14:05:59.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:58 smithi097 ceph-mon[32720]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T14:05:59.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:58 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:05:59.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:58 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:59.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:58 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:59.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:58 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:05:59.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:58 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:59.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:58 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:59.174 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:58 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:05:59.174 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:58 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:59.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:58 smithi151 ceph-mon[32990]: purged_snaps scrub starts 2024-04-25T14:05:59.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:58 smithi151 ceph-mon[32990]: purged_snaps scrub ok 2024-04-25T14:05:59.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:58 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:05:59.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:58 smithi151 ceph-mon[32990]: osd.0 [v2:172.21.15.96:6802/3832506168,v1:172.21.15.96:6803/3832506168] boot 2024-04-25T14:05:59.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:58 smithi151 ceph-mon[32990]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T14:05:59.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:58 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:05:59.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:58 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:59.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:58 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:59.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:58 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:05:59.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:58 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:05:59.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:58 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:59.189 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:58 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:05:59.189 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:58 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:00.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:00.023+0000 7fb6fa704700 1 -- 172.21.15.97:0/2704851635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6f4107670 msgr2=0x7fb6f4109a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:00.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:00.023+0000 7fb6fa704700 1 --2- 172.21.15.97:0/2704851635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6f4107670 0x7fb6f4109a90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8009990 tx=0x7fb6e8009ca0 comp rx=0 tx=0).stop 2024-04-25T14:06:00.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:00.023+0000 7fb6fa704700 1 -- 172.21.15.97:0/2704851635 shutdown_connections 2024-04-25T14:06:00.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:00.023+0000 7fb6fa704700 1 --2- 172.21.15.97:0/2704851635 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb6f4109fd0 0x7fb6f410c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:00.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:00.023+0000 7fb6fa704700 1 --2- 172.21.15.97:0/2704851635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6f4104d10 0x7fb6f4107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:00.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:00.023+0000 7fb6fa704700 1 --2- 172.21.15.97:0/2704851635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6f4107670 0x7fb6f4109a90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:00.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:00.023+0000 7fb6fa704700 1 -- 172.21.15.97:0/2704851635 >> 172.21.15.97:0/2704851635 conn(0x7fb6f40fe750 msgr2=0x7fb6f4100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:00.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:00.023+0000 7fb6fa704700 1 -- 172.21.15.97:0/2704851635 shutdown_connections 2024-04-25T14:06:00.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:00.023+0000 7fb6fa704700 1 -- 172.21.15.97:0/2704851635 wait complete. 2024-04-25T14:06:00.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:00.024+0000 7fb6fa704700 1 Processor -- start 2024-04-25T14:06:00.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:00.024+0000 7fb6fa704700 1 -- start start 2024-04-25T14:06:00.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:00.024+0000 7fb6fa704700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6f4104d10 0x7fb6f419a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:00.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:00.024+0000 7fb6fa704700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb6f4107670 0x7fb6f419add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:00.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:00.024+0000 7fb6fa704700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6f4109fd0 0x7fb6f41a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:00.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:00.024+0000 7fb6fa704700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f410fce0 con 0x7fb6f4109fd0 2024-04-25T14:06:00.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:00.024+0000 7fb6fa704700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f410fb60 con 0x7fb6f4104d10 2024-04-25T14:06:00.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:00.024+0000 7fb6fa704700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f410fe60 con 0x7fb6f4107670 2024-04-25T14:06:00.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:00.024+0000 7fb6f3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6f4104d10 0x7fb6f419a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:00.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:00.024+0000 7fb6f3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6f4104d10 0x7fb6f419a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41590/0 (socket says 172.21.15.97:41590) 2024-04-25T14:06:00.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:00.024+0000 7fb6f3fff700 1 -- 172.21.15.97:0/3527728888 learned_addr learned my addr 172.21.15.97:0/3527728888 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T14:06:00.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:00.025+0000 7fb6f37fe700 1 --2- 172.21.15.97:0/3527728888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb6f4107670 0x7fb6f419add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:00.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:00.025+0000 7fb6f8ca1700 1 --2- 172.21.15.97:0/3527728888 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6f4109fd0 0x7fb6f41a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:00.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:00.025+0000 7fb6f3fff700 1 -- 172.21.15.97:0/3527728888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb6f4107670 msgr2=0x7fb6f419add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:00.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:00.025+0000 7fb6f3fff700 1 --2- 172.21.15.97:0/3527728888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb6f4107670 0x7fb6f419add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:00.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:00.025+0000 7fb6f3fff700 1 -- 172.21.15.97:0/3527728888 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6f4109fd0 msgr2=0x7fb6f41a1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:00.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:00.025+0000 7fb6f3fff700 1 --2- 172.21.15.97:0/3527728888 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6f4109fd0 0x7fb6f41a1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:00.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:00.025+0000 7fb6f3fff700 1 -- 172.21.15.97:0/3527728888 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f41a2360 con 0x7fb6f4104d10 2024-04-25T14:06:00.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:00.025+0000 7fb6f8ca1700 1 --2- 172.21.15.97:0/3527728888 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6f4109fd0 0x7fb6f41a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:00.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:00.025+0000 7fb6f3fff700 1 --2- 172.21.15.97:0/3527728888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6f4104d10 0x7fb6f419a890 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb6e400b500 tx=0x7fb6e400b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:00.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:00.025+0000 7fb6f17fa700 1 -- 172.21.15.97:0/3527728888 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e4015020 con 0x7fb6f4104d10 2024-04-25T14:06:00.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:00.025+0000 7fb6f17fa700 1 -- 172.21.15.97:0/3527728888 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6e4011ce0 con 0x7fb6f4104d10 2024-04-25T14:06:00.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:00.025+0000 7fb6fa704700 1 -- 172.21.15.97:0/3527728888 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6f41a2650 con 0x7fb6f4104d10 2024-04-25T14:06:00.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:00.025+0000 7fb6f17fa700 1 -- 172.21.15.97:0/3527728888 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e4010d40 con 0x7fb6f4104d10 2024-04-25T14:06:00.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:00.025+0000 7fb6fa704700 1 -- 172.21.15.97:0/3527728888 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6f41a2c60 con 0x7fb6f4104d10 2024-04-25T14:06:00.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:00.026+0000 7fb6fa704700 1 -- 172.21.15.97:0/3527728888 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6f4051190 con 0x7fb6f4104d10 2024-04-25T14:06:00.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:00.028+0000 7fb6f17fa700 1 -- 172.21.15.97:0/3527728888 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 15) ==== 147464+0+0 (secure 0 0 0) 0x7fb6e4018020 con 0x7fb6f4104d10 2024-04-25T14:06:00.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:00.029+0000 7fb6f17fa700 1 --2- 172.21.15.97:0/3527728888 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb6dc0afb70 0x7fb6dc0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:00.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:00.029+0000 7fb6f17fa700 1 -- 172.21.15.97:0/3527728888 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fb6e40dce80 con 0x7fb6f4104d10 2024-04-25T14:06:00.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:00.029+0000 7fb6f37fe700 1 --2- 172.21.15.97:0/3527728888 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb6dc0afb70 0x7fb6dc0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:00.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:00.030+0000 7fb6f37fe700 1 --2- 172.21.15.97:0/3527728888 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb6dc0afb70 0x7fb6dc0b2080 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb6e80038a0 tx=0x7fb6e801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:00.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:00.033+0000 7fb6f17fa700 1 -- 172.21.15.97:0/3527728888 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb6e401f020 con 0x7fb6f4104d10 2024-04-25T14:06:00.108 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:59 smithi096 ceph-mon[29207]: Detected new or changed devices on smithi096 2024-04-25T14:06:00.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:59 smithi096 ceph-mon[29207]: Adjusting osd_memory_target on smithi096 to 17088M 2024-04-25T14:06:00.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:05:59 smithi096 ceph-mon[29207]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:06:00.172 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:59 smithi097 ceph-mon[32720]: Detected new or changed devices on smithi096 2024-04-25T14:06:00.172 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:59 smithi097 ceph-mon[32720]: Adjusting osd_memory_target on smithi096 to 17088M 2024-04-25T14:06:00.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:05:59 smithi097 ceph-mon[32720]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:06:00.187 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:59 smithi151 ceph-mon[32990]: Detected new or changed devices on smithi096 2024-04-25T14:06:00.187 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:59 smithi151 ceph-mon[32990]: Adjusting osd_memory_target on smithi096 to 17088M 2024-04-25T14:06:00.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:05:59 smithi151 ceph-mon[32990]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:06:00.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:00.236+0000 7fb6fa704700 1 -- 172.21.15.97:0/3527728888 --> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fb6f4063ad0 con 0x7fb6dc0afb70 2024-04-25T14:06:01.107 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:00 smithi151 ceph-mon[32990]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T14:06:01.107 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:00 smithi151 ceph-mon[32990]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:06:01.107 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:00 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:06:01.107 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:00 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:06:01.107 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:00 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:06:01.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:00 smithi096 ceph-mon[29207]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T14:06:01.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:00 smithi096 ceph-mon[29207]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:06:01.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:00 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:06:01.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:00 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:06:01.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:00 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:06:01.139 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:00 smithi097 ceph-mon[32720]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T14:06:01.139 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:00 smithi097 ceph-mon[32720]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:06:01.139 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:00 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:06:01.139 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:00 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:06:01.139 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:00 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:06:02.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:01 smithi096 ceph-mon[29207]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:06:02.121 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:01 smithi097 ceph-mon[32720]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:06:02.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:01 smithi151 ceph-mon[32990]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:06:03.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:02 smithi096 ceph-mon[29207]: from='client.? 172.21.15.97:0/1140735935' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "866e73b9-4a26-48ba-badf-704fe22f71cc"}]: dispatch 2024-04-25T14:06:03.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:02 smithi096 ceph-mon[29207]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "866e73b9-4a26-48ba-badf-704fe22f71cc"}]: dispatch 2024-04-25T14:06:03.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:02 smithi096 ceph-mon[29207]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "866e73b9-4a26-48ba-badf-704fe22f71cc"}]': finished 2024-04-25T14:06:03.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:02 smithi096 ceph-mon[29207]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T14:06:03.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:02 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:06:03.130 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:02 smithi097 ceph-mon[32720]: from='client.? 172.21.15.97:0/1140735935' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "866e73b9-4a26-48ba-badf-704fe22f71cc"}]: dispatch 2024-04-25T14:06:03.130 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:02 smithi097 ceph-mon[32720]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "866e73b9-4a26-48ba-badf-704fe22f71cc"}]: dispatch 2024-04-25T14:06:03.130 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:02 smithi097 ceph-mon[32720]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "866e73b9-4a26-48ba-badf-704fe22f71cc"}]': finished 2024-04-25T14:06:03.130 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:02 smithi097 ceph-mon[32720]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T14:06:03.130 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:02 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:06:03.187 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:02 smithi151 ceph-mon[32990]: from='client.? 172.21.15.97:0/1140735935' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "866e73b9-4a26-48ba-badf-704fe22f71cc"}]: dispatch 2024-04-25T14:06:03.187 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:02 smithi151 ceph-mon[32990]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "866e73b9-4a26-48ba-badf-704fe22f71cc"}]: dispatch 2024-04-25T14:06:03.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:02 smithi151 ceph-mon[32990]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "866e73b9-4a26-48ba-badf-704fe22f71cc"}]': finished 2024-04-25T14:06:03.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:02 smithi151 ceph-mon[32990]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T14:06:03.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:02 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:06:04.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:03 smithi096 ceph-mon[29207]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:06:04.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:03 smithi096 ceph-mon[29207]: from='client.? 172.21.15.97:0/2432864923' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:06:04.172 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:03 smithi097 ceph-mon[32720]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:06:04.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:03 smithi097 ceph-mon[32720]: from='client.? 172.21.15.97:0/2432864923' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:06:04.187 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:03 smithi151 ceph-mon[32990]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:06:04.187 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:03 smithi151 ceph-mon[32990]: from='client.? 172.21.15.97:0/2432864923' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:06:06.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:06 smithi097 ceph-mon[32720]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:06:06.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:06 smithi151 ceph-mon[32990]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:06:06.858 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:06 smithi096 ceph-mon[29207]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:06:08.466 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:08 smithi097 ceph-mon[32720]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:06:08.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:08 smithi151 ceph-mon[32990]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:06:08.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:08 smithi096 ceph-mon[29207]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:06:10.500 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:10 smithi097 ceph-mon[32720]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:06:10.500 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:10 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T14:06:10.500 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:10 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:06:10.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:10 smithi151 ceph-mon[32990]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:06:10.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:10 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T14:06:10.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:10 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:06:10.858 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:10 smithi096 ceph-mon[29207]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:06:10.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:10 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T14:06:10.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:10 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:06:11.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:11 smithi097 ceph-mon[32720]: Deploying daemon osd.1 on smithi097 2024-04-25T14:06:11.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:11 smithi151 ceph-mon[32990]: Deploying daemon osd.1 on smithi097 2024-04-25T14:06:11.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:11 smithi096 ceph-mon[29207]: Deploying daemon osd.1 on smithi097 2024-04-25T14:06:12.620 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:12 smithi097 ceph-mon[32720]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:06:12.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:12 smithi151 ceph-mon[32990]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:06:12.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:12 smithi096 ceph-mon[29207]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:06:14.410 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:14 smithi097 ceph-mon[32720]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:06:14.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:14 smithi151 ceph-mon[32990]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:06:14.858 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:14 smithi096 ceph-mon[29207]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:06:16.187 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:15 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:16.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:15 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:16.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:15 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:06:16.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:15 smithi151 ceph-mon[32990]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:06:16.336 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:15 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:16.336 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:15 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:16.336 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:15 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:06:16.336 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:15 smithi097 ceph-mon[32720]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:06:16.358 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:15 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:16.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:15 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:16.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:15 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:06:16.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:15 smithi096 ceph-mon[29207]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:06:18.454 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:18 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:18.454 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:18 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:18.455 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:18 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:06:18.455 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:18 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:06:18.455 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:18 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:18.455 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:18 smithi097 ceph-mon[32720]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:06:18.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:18.504+0000 7fb6f17fa700 1 -- 172.21.15.97:0/3527728888 <== mgr.14152 v2:172.21.15.96:6800/536332425 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb6f4063ad0 con 0x7fb6dc0afb70 2024-04-25T14:06:18.504 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 1 on host 'smithi097' 2024-04-25T14:06:18.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:18.506+0000 7fb6fa704700 1 -- 172.21.15.97:0/3527728888 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb6dc0afb70 msgr2=0x7fb6dc0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:18.506+0000 7fb6fa704700 1 --2- 172.21.15.97:0/3527728888 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb6dc0afb70 0x7fb6dc0b2080 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb6e80038a0 tx=0x7fb6e801a040 comp rx=0 tx=0).stop 2024-04-25T14:06:18.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:18.506+0000 7fb6fa704700 1 -- 172.21.15.97:0/3527728888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6f4104d10 msgr2=0x7fb6f419a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:18.506+0000 7fb6fa704700 1 --2- 172.21.15.97:0/3527728888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6f4104d10 0x7fb6f419a890 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb6e400b500 tx=0x7fb6e400b8c0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:18.506+0000 7fb6fa704700 1 -- 172.21.15.97:0/3527728888 shutdown_connections 2024-04-25T14:06:18.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:18.506+0000 7fb6fa704700 1 --2- 172.21.15.97:0/3527728888 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb6dc0afb70 0x7fb6dc0b2080 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:18.506+0000 7fb6fa704700 1 --2- 172.21.15.97:0/3527728888 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6f4109fd0 0x7fb6f41a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:18.506+0000 7fb6fa704700 1 --2- 172.21.15.97:0/3527728888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6f4104d10 0x7fb6f419a890 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:18.506+0000 7fb6fa704700 1 --2- 172.21.15.97:0/3527728888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb6f4107670 0x7fb6f419add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:18.506+0000 7fb6fa704700 1 -- 172.21.15.97:0/3527728888 >> 172.21.15.97:0/3527728888 conn(0x7fb6f40fe750 msgr2=0x7fb6f4105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:18.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:18.506+0000 7fb6fa704700 1 -- 172.21.15.97:0/3527728888 shutdown_connections 2024-04-25T14:06:18.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:06:18.506+0000 7fb6fa704700 1 -- 172.21.15.97:0/3527728888 wait complete. 2024-04-25T14:06:18.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:18 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:18.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:18 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:18.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:18 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:06:18.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:18 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:06:18.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:18 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:18.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:18 smithi096 ceph-mon[29207]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:06:18.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:18 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:18.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:18 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:18.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:18 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:06:18.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:18 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:06:18.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:18 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:18.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:18 smithi151 ceph-mon[32990]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:06:19.083 DEBUG:teuthology.orchestra.run.smithi097:osd.1> sudo journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@osd.1.service 2024-04-25T14:06:19.085 INFO:tasks.cephadm:Deploying osd.2 on smithi151 with /dev/vg_nvme/lv_4... 2024-04-25T14:06:19.085 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T14:06:19.332 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:06:19.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:19 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:19.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:19 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:19.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:19 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:06:19.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:19 smithi097 ceph-mon[32720]: from='osd.1 [v2:172.21.15.97:6800/1083311494,v1:172.21.15.97:6801/1083311494]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T14:06:19.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:19 smithi097 ceph-mon[32720]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T14:06:19.760 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:19 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:19.760 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:19 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:19.760 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:19 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:06:19.760 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:19 smithi151 ceph-mon[32990]: from='osd.1 [v2:172.21.15.97:6800/1083311494,v1:172.21.15.97:6801/1083311494]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T14:06:19.760 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:19 smithi151 ceph-mon[32990]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T14:06:19.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:19 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:19.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:19 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:19.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:19 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:06:19.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:19 smithi096 ceph-mon[29207]: from='osd.1 [v2:172.21.15.97:6800/1083311494,v1:172.21.15.97:6801/1083311494]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T14:06:19.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:19 smithi096 ceph-mon[29207]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T14:06:20.792 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:20 smithi151 ceph-mon[32990]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:06:20.792 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:20 smithi151 ceph-mon[32990]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T14:06:20.792 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:20 smithi151 ceph-mon[32990]: from='osd.1 [v2:172.21.15.97:6800/1083311494,v1:172.21.15.97:6801/1083311494]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T14:06:20.792 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:20 smithi151 ceph-mon[32990]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T14:06:20.793 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:20 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:06:20.793 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:20 smithi151 ceph-mon[32990]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T14:06:20.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:20 smithi096 ceph-mon[29207]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:06:20.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:20 smithi096 ceph-mon[29207]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T14:06:20.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:20 smithi096 ceph-mon[29207]: from='osd.1 [v2:172.21.15.97:6800/1083311494,v1:172.21.15.97:6801/1083311494]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T14:06:20.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:20 smithi096 ceph-mon[29207]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T14:06:20.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:20 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:06:20.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:20 smithi096 ceph-mon[29207]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T14:06:20.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:20 smithi097 ceph-mon[32720]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:06:20.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:20 smithi097 ceph-mon[32720]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T14:06:20.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:20 smithi097 ceph-mon[32720]: from='osd.1 [v2:172.21.15.97:6800/1083311494,v1:172.21.15.97:6801/1083311494]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T14:06:20.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:20 smithi097 ceph-mon[32720]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T14:06:20.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:20 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:06:20.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:20 smithi097 ceph-mon[32720]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T14:06:20.923 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 14:06:20 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-1[37313]: 2024-04-25T14:06:20.504+0000 7f1f7e1fa700 -1 osd.1 0 waiting for initial osdmap 2024-04-25T14:06:20.923 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 14:06:20 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-1[37313]: 2024-04-25T14:06:20.510+0000 7f1f79faf700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:06:21.163 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T14:06:21.163 INFO:teuthology.orchestra.run.smithi151.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T14:06:21.163 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10+0 records in 2024-04-25T14:06:21.163 INFO:teuthology.orchestra.run.smithi151.stderr:10+0 records out 2024-04-25T14:06:21.163 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130809 s, 802 MB/s 2024-04-25T14:06:21.163 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping successful for: 2024-04-25T14:06:21.664 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:21 smithi097 ceph-mon[32720]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-25T14:06:21.664 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:21 smithi097 ceph-mon[32720]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T14:06:21.664 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:21 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:06:21.664 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:21 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:06:21.731 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:21 smithi151 ceph-mon[32990]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-25T14:06:21.731 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:21 smithi151 ceph-mon[32990]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T14:06:21.731 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:21 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:06:21.731 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:21 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:06:21.732 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi151:vg_nvme/lv_4 2024-04-25T14:06:21.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:21 smithi096 ceph-mon[29207]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-25T14:06:21.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:21 smithi096 ceph-mon[29207]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T14:06:21.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:21 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:06:21.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:21 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:06:21.975 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:06:22.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:22 smithi151 ceph-mon[32990]: purged_snaps scrub starts 2024-04-25T14:06:22.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:22 smithi151 ceph-mon[32990]: purged_snaps scrub ok 2024-04-25T14:06:22.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:22 smithi151 ceph-mon[32990]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:06:22.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:22 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:06:22.689 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:22 smithi151 ceph-mon[32990]: osd.1 [v2:172.21.15.97:6800/1083311494,v1:172.21.15.97:6801/1083311494] boot 2024-04-25T14:06:22.689 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:22 smithi151 ceph-mon[32990]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T14:06:22.689 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:22 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:06:22.689 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:22 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:22.689 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:22 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:22.689 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:22 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:06:22.689 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:22 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:22.689 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:22 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:06:22.689 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:22 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:06:22.689 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:22 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:22.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:22 smithi096 ceph-mon[29207]: purged_snaps scrub starts 2024-04-25T14:06:22.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:22 smithi096 ceph-mon[29207]: purged_snaps scrub ok 2024-04-25T14:06:22.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:22 smithi096 ceph-mon[29207]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:06:22.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:22 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:06:22.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:22 smithi096 ceph-mon[29207]: osd.1 [v2:172.21.15.97:6800/1083311494,v1:172.21.15.97:6801/1083311494] boot 2024-04-25T14:06:22.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:22 smithi096 ceph-mon[29207]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T14:06:22.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:22 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:06:22.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:22 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:22.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:22 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:22.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:22 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:06:22.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:22 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:22.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:22 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:06:22.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:22 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:06:22.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:22 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:22.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:22 smithi097 ceph-mon[32720]: purged_snaps scrub starts 2024-04-25T14:06:22.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:22 smithi097 ceph-mon[32720]: purged_snaps scrub ok 2024-04-25T14:06:22.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:22 smithi097 ceph-mon[32720]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:06:22.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:22 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:06:22.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:22 smithi097 ceph-mon[32720]: osd.1 [v2:172.21.15.97:6800/1083311494,v1:172.21.15.97:6801/1083311494] boot 2024-04-25T14:06:22.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:22 smithi097 ceph-mon[32720]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T14:06:22.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:22 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:06:22.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:22 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:22.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:22 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:22.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:22 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:06:22.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:22 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:22.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:22 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:06:22.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:22 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:06:22.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:22 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:23.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:23.515+0000 7fd35ef5e700 1 -- 172.21.15.151:0/356065855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd358105330 msgr2=0x7fd358105740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:23.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:23.515+0000 7fd35ef5e700 1 --2- 172.21.15.151:0/356065855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd358105330 0x7fd358105740 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd34c009990 tx=0x7fd34c009ca0 comp rx=0 tx=0).stop 2024-04-25T14:06:23.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:23.515+0000 7fd35ef5e700 1 -- 172.21.15.151:0/356065855 shutdown_connections 2024-04-25T14:06:23.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:23.515+0000 7fd35ef5e700 1 --2- 172.21.15.151:0/356065855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd358105cd0 0x7fd35810c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:23.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:23.515+0000 7fd35ef5e700 1 --2- 172.21.15.151:0/356065855 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd358104200 0x7fd358104610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:23.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:23.515+0000 7fd35ef5e700 1 --2- 172.21.15.151:0/356065855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd358105330 0x7fd358105740 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:23.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:23.515+0000 7fd35ef5e700 1 -- 172.21.15.151:0/356065855 >> 172.21.15.151:0/356065855 conn(0x7fd3580ff840 msgr2=0x7fd358101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:23.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:23.515+0000 7fd35ef5e700 1 -- 172.21.15.151:0/356065855 shutdown_connections 2024-04-25T14:06:23.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:23.515+0000 7fd35ef5e700 1 -- 172.21.15.151:0/356065855 wait complete. 2024-04-25T14:06:23.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:23.516+0000 7fd35ef5e700 1 Processor -- start 2024-04-25T14:06:23.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:23.516+0000 7fd35ef5e700 1 -- start start 2024-04-25T14:06:23.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:23.516+0000 7fd35ef5e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd358104200 0x7fd35819a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:23.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:23.516+0000 7fd35ef5e700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd358105330 0x7fd35819acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:23.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:23.516+0000 7fd35ef5e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd358105cd0 0x7fd3581a1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:23.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:23.516+0000 7fd35ef5e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd35810fc80 con 0x7fd358105cd0 2024-04-25T14:06:23.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:23.516+0000 7fd35ef5e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd35810fb00 con 0x7fd358104200 2024-04-25T14:06:23.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:23.516+0000 7fd35ef5e700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd35810fe00 con 0x7fd358105330 2024-04-25T14:06:23.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:23.516+0000 7fd357fff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd358105330 0x7fd35819acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:23.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:23.516+0000 7fd357fff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd358105330 0x7fd35819acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:35428/0 (socket says 172.21.15.151:35428) 2024-04-25T14:06:23.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:23.516+0000 7fd357fff700 1 -- 172.21.15.151:0/2865429658 learned_addr learned my addr 172.21.15.151:0/2865429658 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-25T14:06:23.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:23.516+0000 7fd35d4fb700 1 --2- 172.21.15.151:0/2865429658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd358105cd0 0x7fd3581a1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:23.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:23.516+0000 7fd35ccfa700 1 --2- 172.21.15.151:0/2865429658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd358104200 0x7fd35819a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:23.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:23.517+0000 7fd357fff700 1 -- 172.21.15.151:0/2865429658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd358104200 msgr2=0x7fd35819a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:23.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:23.517+0000 7fd357fff700 1 --2- 172.21.15.151:0/2865429658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd358104200 0x7fd35819a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:23.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:23.517+0000 7fd357fff700 1 -- 172.21.15.151:0/2865429658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd358105cd0 msgr2=0x7fd3581a1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:23.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:23.517+0000 7fd357fff700 1 --2- 172.21.15.151:0/2865429658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd358105cd0 0x7fd3581a1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:23.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:23.517+0000 7fd357fff700 1 -- 172.21.15.151:0/2865429658 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3581a22f0 con 0x7fd358105330 2024-04-25T14:06:23.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:23.517+0000 7fd35ccfa700 1 --2- 172.21.15.151:0/2865429658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd358104200 0x7fd35819a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:23.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:23.517+0000 7fd357fff700 1 --2- 172.21.15.151:0/2865429658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd358105330 0x7fd35819acd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd34c0098e0 tx=0x7fd34c0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:23.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:23.517+0000 7fd355ffb700 1 -- 172.21.15.151:0/2865429658 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34c026070 con 0x7fd358105330 2024-04-25T14:06:23.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:23.517+0000 7fd355ffb700 1 -- 172.21.15.151:0/2865429658 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd34c021410 con 0x7fd358105330 2024-04-25T14:06:23.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:23.517+0000 7fd35ef5e700 1 -- 172.21.15.151:0/2865429658 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3581a2580 con 0x7fd358105330 2024-04-25T14:06:23.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:23.517+0000 7fd355ffb700 1 -- 172.21.15.151:0/2865429658 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34c017650 con 0x7fd358105330 2024-04-25T14:06:23.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:23.517+0000 7fd35ef5e700 1 -- 172.21.15.151:0/2865429658 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3581a29d0 con 0x7fd358105330 2024-04-25T14:06:23.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:23.518+0000 7fd35ef5e700 1 -- 172.21.15.151:0/2865429658 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd358051190 con 0x7fd358105330 2024-04-25T14:06:23.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:23.520+0000 7fd355ffb700 1 -- 172.21.15.151:0/2865429658 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 15) ==== 147464+0+0 (secure 0 0 0) 0x7fd34c029070 con 0x7fd358105330 2024-04-25T14:06:23.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:23.520+0000 7fd355ffb700 1 --2- 172.21.15.151:0/2865429658 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd3400afaf0 0x7fd3400b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:23.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:23.520+0000 7fd355ffb700 1 -- 172.21.15.151:0/2865429658 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7fd34c0de5a0 con 0x7fd358105330 2024-04-25T14:06:23.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:23.520+0000 7fd35ccfa700 1 --2- 172.21.15.151:0/2865429658 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd3400afaf0 0x7fd3400b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:23.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:23.521+0000 7fd35ccfa700 1 --2- 172.21.15.151:0/2865429658 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd3400afaf0 0x7fd3400b2000 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd358105230 tx=0x7fd350009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:23.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:23.524+0000 7fd355ffb700 1 -- 172.21.15.151:0/2865429658 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd34c013070 con 0x7fd358105330 2024-04-25T14:06:23.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:23 smithi151 ceph-mon[32990]: Detected new or changed devices on smithi097 2024-04-25T14:06:23.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:23 smithi151 ceph-mon[32990]: Adjusting osd_memory_target on smithi097 to 17088M 2024-04-25T14:06:23.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:23 smithi151 ceph-mon[32990]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T14:06:23.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:23.731+0000 7fd35ef5e700 1 -- 172.21.15.151:0/2865429658 --> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fd358063ad0 con 0x7fd3400afaf0 2024-04-25T14:06:23.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:23 smithi096 ceph-mon[29207]: Detected new or changed devices on smithi097 2024-04-25T14:06:23.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:23 smithi096 ceph-mon[29207]: Adjusting osd_memory_target on smithi097 to 17088M 2024-04-25T14:06:23.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:23 smithi096 ceph-mon[29207]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T14:06:23.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:23 smithi097 ceph-mon[32720]: Detected new or changed devices on smithi097 2024-04-25T14:06:23.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:23 smithi097 ceph-mon[32720]: Adjusting osd_memory_target on smithi097 to 17088M 2024-04-25T14:06:23.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:23 smithi097 ceph-mon[32720]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T14:06:24.587 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:24 smithi151 ceph-mon[32990]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:06:24.588 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:24 smithi151 ceph-mon[32990]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:06:24.588 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:24 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:06:24.588 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:24 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:06:24.588 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:24 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:06:24.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:24 smithi096 ceph-mon[29207]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:06:24.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:24 smithi096 ceph-mon[29207]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:06:24.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:24 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:06:24.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:24 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:06:24.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:24 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:06:24.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:24 smithi097 ceph-mon[32720]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:06:24.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:24 smithi097 ceph-mon[32720]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:06:24.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:24 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:06:24.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:24 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:06:24.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:24 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:06:25.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:25 smithi151 ceph-mon[32990]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:06:25.858 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:25 smithi096 ceph-mon[29207]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:06:25.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:25 smithi097 ceph-mon[32720]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:06:26.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:26 smithi096 ceph-mon[29207]: from='client.? 172.21.15.151:0/1009496383' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45c79ecb-abc4-4dba-bae8-74804e4bc1bd"}]: dispatch 2024-04-25T14:06:26.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:26 smithi096 ceph-mon[29207]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45c79ecb-abc4-4dba-bae8-74804e4bc1bd"}]: dispatch 2024-04-25T14:06:26.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:26 smithi096 ceph-mon[29207]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "45c79ecb-abc4-4dba-bae8-74804e4bc1bd"}]': finished 2024-04-25T14:06:26.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:26 smithi096 ceph-mon[29207]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T14:06:26.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:26 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:06:26.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:26 smithi097 ceph-mon[32720]: from='client.? 172.21.15.151:0/1009496383' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45c79ecb-abc4-4dba-bae8-74804e4bc1bd"}]: dispatch 2024-04-25T14:06:26.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:26 smithi097 ceph-mon[32720]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45c79ecb-abc4-4dba-bae8-74804e4bc1bd"}]: dispatch 2024-04-25T14:06:26.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:26 smithi097 ceph-mon[32720]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "45c79ecb-abc4-4dba-bae8-74804e4bc1bd"}]': finished 2024-04-25T14:06:26.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:26 smithi097 ceph-mon[32720]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T14:06:26.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:26 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:06:26.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:26 smithi151 ceph-mon[32990]: from='client.? 172.21.15.151:0/1009496383' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45c79ecb-abc4-4dba-bae8-74804e4bc1bd"}]: dispatch 2024-04-25T14:06:26.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:26 smithi151 ceph-mon[32990]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45c79ecb-abc4-4dba-bae8-74804e4bc1bd"}]: dispatch 2024-04-25T14:06:26.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:26 smithi151 ceph-mon[32990]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "45c79ecb-abc4-4dba-bae8-74804e4bc1bd"}]': finished 2024-04-25T14:06:26.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:26 smithi151 ceph-mon[32990]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T14:06:26.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:26 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:06:27.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:27 smithi096 ceph-mon[29207]: from='client.? 172.21.15.151:0/1355884662' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:06:27.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:27 smithi096 ceph-mon[29207]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:06:27.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:27 smithi097 ceph-mon[32720]: from='client.? 172.21.15.151:0/1355884662' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:06:27.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:27 smithi097 ceph-mon[32720]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:06:27.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:27 smithi151 ceph-mon[32990]: from='client.? 172.21.15.151:0/1355884662' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:06:27.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:27 smithi151 ceph-mon[32990]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:06:30.485 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:30 smithi151 ceph-mon[32990]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:06:30.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:30 smithi097 ceph-mon[32720]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:06:30.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:30 smithi096 ceph-mon[29207]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:06:32.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:32 smithi151 ceph-mon[32990]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:06:32.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:32 smithi097 ceph-mon[32720]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:06:32.858 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:32 smithi096 ceph-mon[29207]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:06:33.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:33 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T14:06:33.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:33 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:06:33.671 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:33 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T14:06:33.671 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:33 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:06:33.760 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:33 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T14:06:33.760 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:33 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:06:34.626 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:34 smithi151 ceph-mon[32990]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:06:34.627 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:34 smithi151 ceph-mon[32990]: Deploying daemon osd.2 on smithi151 2024-04-25T14:06:34.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:34 smithi097 ceph-mon[32720]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:06:34.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:34 smithi097 ceph-mon[32720]: Deploying daemon osd.2 on smithi151 2024-04-25T14:06:34.858 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:34 smithi096 ceph-mon[29207]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:06:34.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:34 smithi096 ceph-mon[29207]: Deploying daemon osd.2 on smithi151 2024-04-25T14:06:36.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:36 smithi097 ceph-mon[32720]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:06:36.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:36 smithi151 ceph-mon[32990]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:06:36.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:36 smithi096 ceph-mon[29207]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:06:38.489 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:38 smithi151 ceph-mon[32990]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:06:38.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:38 smithi097 ceph-mon[32720]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:06:38.858 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:38 smithi096 ceph-mon[29207]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:06:39.708 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:39 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:39.708 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:39 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:39.708 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:39 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:06:39.708 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:39 smithi151 ceph-mon[32990]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:06:40.108 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:39 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:40.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:39 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:40.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:39 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:06:40.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:39 smithi096 ceph-mon[29207]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:06:40.172 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:39 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:40.172 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:39 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:40.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:39 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:06:40.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:39 smithi097 ceph-mon[32720]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:06:41.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:41.795+0000 7fd355ffb700 1 -- 172.21.15.151:0/2865429658 <== mgr.14152 v2:172.21.15.96:6800/536332425 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd358063ad0 con 0x7fd3400afaf0 2024-04-25T14:06:41.795 INFO:teuthology.orchestra.run.smithi151.stdout:Created osd(s) 2 on host 'smithi151' 2024-04-25T14:06:41.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:41.798+0000 7fd35ef5e700 1 -- 172.21.15.151:0/2865429658 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd3400afaf0 msgr2=0x7fd3400b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:41.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:41.798+0000 7fd35ef5e700 1 --2- 172.21.15.151:0/2865429658 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd3400afaf0 0x7fd3400b2000 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd358105230 tx=0x7fd350009040 comp rx=0 tx=0).stop 2024-04-25T14:06:41.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:41.798+0000 7fd35ef5e700 1 -- 172.21.15.151:0/2865429658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd358105330 msgr2=0x7fd35819acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:41.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:41.798+0000 7fd35ef5e700 1 --2- 172.21.15.151:0/2865429658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd358105330 0x7fd35819acd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd34c0098e0 tx=0x7fd34c0047b0 comp rx=0 tx=0).stop 2024-04-25T14:06:41.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:41.798+0000 7fd35ef5e700 1 -- 172.21.15.151:0/2865429658 shutdown_connections 2024-04-25T14:06:41.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:41.798+0000 7fd35ef5e700 1 --2- 172.21.15.151:0/2865429658 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd3400afaf0 0x7fd3400b2000 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:41.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:41.798+0000 7fd35ef5e700 1 --2- 172.21.15.151:0/2865429658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd358105cd0 0x7fd3581a1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:41.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:41.798+0000 7fd35ef5e700 1 --2- 172.21.15.151:0/2865429658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd358104200 0x7fd35819a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:41.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:41.798+0000 7fd35ef5e700 1 --2- 172.21.15.151:0/2865429658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd358105330 0x7fd35819acd0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:41.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:41.798+0000 7fd35ef5e700 1 -- 172.21.15.151:0/2865429658 >> 172.21.15.151:0/2865429658 conn(0x7fd3580ff840 msgr2=0x7fd358108560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:41.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:41.798+0000 7fd35ef5e700 1 -- 172.21.15.151:0/2865429658 shutdown_connections 2024-04-25T14:06:41.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:06:41.798+0000 7fd35ef5e700 1 -- 172.21.15.151:0/2865429658 wait complete. 2024-04-25T14:06:42.428 DEBUG:teuthology.orchestra.run.smithi151:osd.2> sudo journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@osd.2.service 2024-04-25T14:06:42.431 INFO:tasks.cephadm:Deploying osd.3 on smithi096 with /dev/vg_nvme/lv_3... 2024-04-25T14:06:42.431 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T14:06:42.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:42 smithi097 ceph-mon[32720]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:06:42.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:42 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:42.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:42 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:42.678 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:06:42.678 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:42 smithi096 ceph-mon[29207]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:06:42.679 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:42 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:42.679 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:42 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:42.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:42 smithi151 ceph-mon[32990]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:06:42.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:42 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:42.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:42 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:42.688 INFO:journalctl@ceph.osd.2.smithi151.stdout:Apr 25 14:06:42 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-2[38142]: 2024-04-25T14:06:42.613+0000 7f4cb67727c0 -1 osd.2 0 log_to_monitors true 2024-04-25T14:06:43.815 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:43 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:43.815 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:43 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:43.815 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:43 smithi096 ceph-mon[29207]: from='osd.2 [v2:172.21.15.151:6800/981902966,v1:172.21.15.151:6801/981902966]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T14:06:43.815 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:43 smithi096 ceph-mon[29207]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T14:06:43.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:43 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:43.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:43 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:43.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:43 smithi097 ceph-mon[32720]: from='osd.2 [v2:172.21.15.151:6800/981902966,v1:172.21.15.151:6801/981902966]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T14:06:43.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:43 smithi097 ceph-mon[32720]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T14:06:43.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:43 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:43.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:43 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:43.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:43 smithi151 ceph-mon[32990]: from='osd.2 [v2:172.21.15.151:6800/981902966,v1:172.21.15.151:6801/981902966]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T14:06:43.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:43 smithi151 ceph-mon[32990]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T14:06:44.688 INFO:journalctl@ceph.osd.2.smithi151.stdout:Apr 25 14:06:44 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-2[38142]: 2024-04-25T14:06:44.544+0000 7f4cabdd7700 -1 osd.2 0 waiting for initial osdmap 2024-04-25T14:06:44.688 INFO:journalctl@ceph.osd.2.smithi151.stdout:Apr 25 14:06:44 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-2[38142]: 2024-04-25T14:06:44.549+0000 7f4ca7b8c700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:06:44.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:44 smithi151 ceph-mon[32990]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:06:44.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:44 smithi151 ceph-mon[32990]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T14:06:44.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:44 smithi151 ceph-mon[32990]: from='osd.2 [v2:172.21.15.151:6800/981902966,v1:172.21.15.151:6801/981902966]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-04-25T14:06:44.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:44 smithi151 ceph-mon[32990]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T14:06:44.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:44 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:06:44.689 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:44 smithi151 ceph-mon[32990]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-04-25T14:06:44.709 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:44 smithi096 ceph-mon[29207]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:06:44.709 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:44 smithi096 ceph-mon[29207]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T14:06:44.709 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:44 smithi096 ceph-mon[29207]: from='osd.2 [v2:172.21.15.151:6800/981902966,v1:172.21.15.151:6801/981902966]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-04-25T14:06:44.709 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:44 smithi096 ceph-mon[29207]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T14:06:44.709 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:44 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:06:44.709 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:44 smithi096 ceph-mon[29207]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-04-25T14:06:44.710 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T14:06:44.710 INFO:teuthology.orchestra.run.smithi096.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T14:06:44.710 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10+0 records in 2024-04-25T14:06:44.710 INFO:teuthology.orchestra.run.smithi096.stderr:10+0 records out 2024-04-25T14:06:44.710 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.013144 s, 798 MB/s 2024-04-25T14:06:44.710 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping successful for: 2024-04-25T14:06:44.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:44 smithi097 ceph-mon[32720]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:06:44.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:44 smithi097 ceph-mon[32720]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T14:06:44.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:44 smithi097 ceph-mon[32720]: from='osd.2 [v2:172.21.15.151:6800/981902966,v1:172.21.15.151:6801/981902966]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-04-25T14:06:44.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:44 smithi097 ceph-mon[32720]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T14:06:44.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:44 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:06:44.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:44 smithi097 ceph-mon[32720]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-04-25T14:06:45.385 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi096:vg_nvme/lv_3 2024-04-25T14:06:45.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:45 smithi096 ceph-mon[29207]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-04-25T14:06:45.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:45 smithi096 ceph-mon[29207]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T14:06:45.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:45 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:06:45.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:45 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:06:45.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:45 smithi096 ceph-mon[29207]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:06:45.632 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:06:45.772 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:45 smithi151 ceph-mon[32990]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-04-25T14:06:45.772 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:45 smithi151 ceph-mon[32990]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T14:06:45.773 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:45 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:06:45.773 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:45 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:06:45.773 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:45 smithi151 ceph-mon[32990]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:06:45.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:45 smithi097 ceph-mon[32720]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-04-25T14:06:45.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:45 smithi097 ceph-mon[32720]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T14:06:45.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:45 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:06:45.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:45 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:06:45.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:45 smithi097 ceph-mon[32720]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:06:46.854 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:46 smithi096 ceph-mon[29207]: purged_snaps scrub starts 2024-04-25T14:06:46.855 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:46 smithi096 ceph-mon[29207]: purged_snaps scrub ok 2024-04-25T14:06:46.855 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:46 smithi096 ceph-mon[29207]: osd.2 [v2:172.21.15.151:6800/981902966,v1:172.21.15.151:6801/981902966] boot 2024-04-25T14:06:46.855 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:46 smithi096 ceph-mon[29207]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T14:06:46.855 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:46 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:06:46.855 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:46 smithi096 ceph-mon[29207]: Detected new or changed devices on smithi151 2024-04-25T14:06:46.855 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:46 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:46.855 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:46 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:46.855 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:46 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:06:46.855 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:46 smithi096 ceph-mon[29207]: Adjusting osd_memory_target on smithi151 to 17088M 2024-04-25T14:06:46.855 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:46 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:46.855 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:46 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:06:46.855 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:46 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:06:46.855 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:46 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:46.855 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:46 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:06:46.856 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:46 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:06:46.856 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:46 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:06:46.856 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:46 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:46.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:46 smithi097 ceph-mon[32720]: purged_snaps scrub starts 2024-04-25T14:06:46.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:46 smithi097 ceph-mon[32720]: purged_snaps scrub ok 2024-04-25T14:06:46.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:46 smithi097 ceph-mon[32720]: osd.2 [v2:172.21.15.151:6800/981902966,v1:172.21.15.151:6801/981902966] boot 2024-04-25T14:06:46.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:46 smithi097 ceph-mon[32720]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T14:06:46.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:46 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:06:46.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:46 smithi097 ceph-mon[32720]: Detected new or changed devices on smithi151 2024-04-25T14:06:46.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:46 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:46.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:46 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:46.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:46 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:06:46.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:46 smithi097 ceph-mon[32720]: Adjusting osd_memory_target on smithi151 to 17088M 2024-04-25T14:06:46.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:46 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:46.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:46 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:06:46.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:46 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:06:46.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:46 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:46.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:46 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:06:46.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:46 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:06:46.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:46 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:06:46.925 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:46 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:46.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:46 smithi151 ceph-mon[32990]: purged_snaps scrub starts 2024-04-25T14:06:46.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:46 smithi151 ceph-mon[32990]: purged_snaps scrub ok 2024-04-25T14:06:46.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:46 smithi151 ceph-mon[32990]: osd.2 [v2:172.21.15.151:6800/981902966,v1:172.21.15.151:6801/981902966] boot 2024-04-25T14:06:46.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:46 smithi151 ceph-mon[32990]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T14:06:46.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:46 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:06:46.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:46 smithi151 ceph-mon[32990]: Detected new or changed devices on smithi151 2024-04-25T14:06:46.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:46 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:46.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:46 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:46.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:46 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:06:46.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:46 smithi151 ceph-mon[32990]: Adjusting osd_memory_target on smithi151 to 17088M 2024-04-25T14:06:46.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:46 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:46.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:46 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:06:46.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:46 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:06:46.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:46 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:46.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:46 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:06:46.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:46 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:06:46.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:46 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:06:46.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:46 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:47.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:06:47.387+0000 7f4f4764e700 1 -- 172.21.15.96:0/3946147896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f401030e0 msgr2=0x7f4f401034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:47.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:06:47.387+0000 7f4f4764e700 1 --2- 172.21.15.96:0/3946147896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f401030e0 0x7f4f401034f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c009990 tx=0x7f4f3c009ca0 comp rx=0 tx=0).stop 2024-04-25T14:06:47.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:06:47.387+0000 7f4f4764e700 1 -- 172.21.15.96:0/3946147896 shutdown_connections 2024-04-25T14:06:47.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:06:47.387+0000 7f4f4764e700 1 --2- 172.21.15.96:0/3946147896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f40104bb0 0x7f4f4010b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:47.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:06:47.387+0000 7f4f4764e700 1 --2- 172.21.15.96:0/3946147896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f401030e0 0x7f4f401034f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:47.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:06:47.387+0000 7f4f4764e700 1 --2- 172.21.15.96:0/3946147896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f40104210 0x7f4f40104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:47.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:06:47.387+0000 7f4f4764e700 1 -- 172.21.15.96:0/3946147896 >> 172.21.15.96:0/3946147896 conn(0x7f4f400fe740 msgr2=0x7f4f40100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:47.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:06:47.388+0000 7f4f4764e700 1 -- 172.21.15.96:0/3946147896 shutdown_connections 2024-04-25T14:06:47.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:06:47.388+0000 7f4f4764e700 1 -- 172.21.15.96:0/3946147896 wait complete. 2024-04-25T14:06:47.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:06:47.388+0000 7f4f4764e700 1 Processor -- start 2024-04-25T14:06:47.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:06:47.388+0000 7f4f4764e700 1 -- start start 2024-04-25T14:06:47.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:06:47.389+0000 7f4f4764e700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f401030e0 0x7f4f401996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:47.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:06:47.389+0000 7f4f4764e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f40104210 0x7f4f40199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:47.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:06:47.389+0000 7f4f4764e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f40104bb0 0x7f4f401a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:47.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:06:47.389+0000 7f4f4764e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4010eb30 con 0x7f4f40104210 2024-04-25T14:06:47.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:06:47.389+0000 7f4f4764e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4010e9b0 con 0x7f4f40104bb0 2024-04-25T14:06:47.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:06:47.389+0000 7f4f4764e700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4010ecb0 con 0x7f4f401030e0 2024-04-25T14:06:47.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:06:47.389+0000 7f4f44be9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f40104210 0x7f4f40199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:47.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:06:47.389+0000 7f4f44be9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f40104210 0x7f4f40199c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44254/0 (socket says 172.21.15.96:44254) 2024-04-25T14:06:47.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:06:47.389+0000 7f4f44be9700 1 -- 172.21.15.96:0/637045266 learned_addr learned my addr 172.21.15.96:0/637045266 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:06:47.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:06:47.389+0000 7f4f453ea700 1 --2- 172.21.15.96:0/637045266 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f401030e0 0x7f4f401996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:47.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:06:47.389+0000 7f4f45beb700 1 --2- 172.21.15.96:0/637045266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f40104bb0 0x7f4f401a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:47.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:06:47.389+0000 7f4f44be9700 1 -- 172.21.15.96:0/637045266 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f401030e0 msgr2=0x7f4f401996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:47.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:06:47.389+0000 7f4f44be9700 1 --2- 172.21.15.96:0/637045266 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f401030e0 0x7f4f401996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:47.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:06:47.389+0000 7f4f44be9700 1 -- 172.21.15.96:0/637045266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f40104bb0 msgr2=0x7f4f401a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:47.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:06:47.389+0000 7f4f44be9700 1 --2- 172.21.15.96:0/637045266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f40104bb0 0x7f4f401a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:47.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:06:47.389+0000 7f4f44be9700 1 -- 172.21.15.96:0/637045266 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f401a11a0 con 0x7f4f40104210 2024-04-25T14:06:47.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:06:47.389+0000 7f4f45beb700 1 --2- 172.21.15.96:0/637045266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f40104bb0 0x7f4f401a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:47.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:06:47.389+0000 7f4f44be9700 1 --2- 172.21.15.96:0/637045266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f40104210 0x7f4f40199c10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4f3000c9b0 tx=0x7f4f3000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:47.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:06:47.389+0000 7f4f367fc700 1 -- 172.21.15.96:0/637045266 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f30007b00 con 0x7f4f40104210 2024-04-25T14:06:47.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:06:47.389+0000 7f4f367fc700 1 -- 172.21.15.96:0/637045266 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f30007ca0 con 0x7f4f40104210 2024-04-25T14:06:47.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:06:47.389+0000 7f4f367fc700 1 -- 172.21.15.96:0/637045266 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f30018660 con 0x7f4f40104210 2024-04-25T14:06:47.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:06:47.390+0000 7f4f4764e700 1 -- 172.21.15.96:0/637045266 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f401a1490 con 0x7f4f40104210 2024-04-25T14:06:47.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:06:47.390+0000 7f4f4764e700 1 -- 172.21.15.96:0/637045266 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f401a1980 con 0x7f4f40104210 2024-04-25T14:06:47.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:06:47.390+0000 7f4f4764e700 1 -- 172.21.15.96:0/637045266 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f40051190 con 0x7f4f40104210 2024-04-25T14:06:47.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:06:47.392+0000 7f4f367fc700 1 -- 172.21.15.96:0/637045266 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) ==== 147464+0+0 (secure 0 0 0) 0x7f4f3000f450 con 0x7f4f40104210 2024-04-25T14:06:47.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:06:47.392+0000 7f4f367fc700 1 --2- 172.21.15.96:0/637045266 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f4f2c0afaf0 0x7f4f2c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:47.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:06:47.393+0000 7f4f453ea700 1 --2- 172.21.15.96:0/637045266 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f4f2c0afaf0 0x7f4f2c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:47.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:06:47.393+0000 7f4f367fc700 1 -- 172.21.15.96:0/637045266 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 2851+0+0 (secure 0 0 0) 0x7f4f300dd880 con 0x7f4f40104210 2024-04-25T14:06:47.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:06:47.393+0000 7f4f453ea700 1 --2- 172.21.15.96:0/637045266 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f4f2c0afaf0 0x7f4f2c0b2000 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0038a0 tx=0x7f4f3c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:47.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:06:47.398+0000 7f4f367fc700 1 -- 172.21.15.96:0/637045266 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4f30014030 con 0x7f4f40104210 2024-04-25T14:06:47.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:06:47.609+0000 7f4f4764e700 1 -- 172.21.15.96:0/637045266 --> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f4f40063ad0 con 0x7f4f2c0afaf0 2024-04-25T14:06:47.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:47 smithi096 ceph-mon[29207]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T14:06:47.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:47 smithi096 ceph-mon[29207]: pgmap v82: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:06:47.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:47 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T14:06:47.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:47 smithi097 ceph-mon[32720]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T14:06:47.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:47 smithi097 ceph-mon[32720]: pgmap v82: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:06:47.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:47 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T14:06:47.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:47 smithi151 ceph-mon[32990]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T14:06:47.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:47 smithi151 ceph-mon[32990]: pgmap v82: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:06:47.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:47 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T14:06:48.815 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 14:06:48 smithi096 sudo[48519]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T14:06:48.815 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 14:06:48 smithi096 sudo[48519]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:06:48.815 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 14:06:48 smithi096 sudo[48519]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:06:48.815 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 14:06:48 smithi096 sudo[48519]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:06:48.816 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 14:06:48 smithi096 sudo[48522]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T14:06:48.816 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 14:06:48 smithi096 sudo[48522]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:06:48.816 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 14:06:48 smithi096 sudo[48522]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:06:48.816 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 14:06:48 smithi096 sudo[48522]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:06:48.816 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:48 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T14:06:48.816 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:48 smithi096 ceph-mon[29207]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T14:06:48.816 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:48 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T14:06:48.816 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:48 smithi096 ceph-mon[29207]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:06:48.816 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:48 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:06:48.816 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:48 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:06:48.816 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:48 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:06:48.852 INFO:journalctl@ceph.osd.2.smithi151.stdout:Apr 25 14:06:48 smithi151 sudo[41882]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T14:06:48.852 INFO:journalctl@ceph.osd.2.smithi151.stdout:Apr 25 14:06:48 smithi151 sudo[41882]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:06:48.852 INFO:journalctl@ceph.osd.2.smithi151.stdout:Apr 25 14:06:48 smithi151 sudo[41882]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:06:48.852 INFO:journalctl@ceph.osd.2.smithi151.stdout:Apr 25 14:06:48 smithi151 sudo[41882]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:06:48.852 INFO:journalctl@ceph.osd.2.smithi151.stdout:Apr 25 14:06:48 smithi151 sudo[41885]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T14:06:48.852 INFO:journalctl@ceph.osd.2.smithi151.stdout:Apr 25 14:06:48 smithi151 sudo[41885]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:06:48.852 INFO:journalctl@ceph.osd.2.smithi151.stdout:Apr 25 14:06:48 smithi151 sudo[41885]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:06:48.852 INFO:journalctl@ceph.osd.2.smithi151.stdout:Apr 25 14:06:48 smithi151 sudo[41885]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:06:48.853 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:48 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T14:06:48.853 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:48 smithi151 ceph-mon[32990]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T14:06:48.853 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:48 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T14:06:48.853 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:48 smithi151 ceph-mon[32990]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:06:48.853 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:48 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:06:48.853 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:48 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:06:48.853 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:48 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:06:48.853 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:48 smithi151 sudo[41888]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T14:06:48.889 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:48 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T14:06:48.889 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:48 smithi097 ceph-mon[32720]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T14:06:48.890 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:48 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T14:06:48.890 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:48 smithi097 ceph-mon[32720]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:06:48.890 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:48 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:06:48.890 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:48 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:06:48.890 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:48 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:06:48.890 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 14:06:48 smithi097 sudo[41440]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T14:06:48.890 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 14:06:48 smithi097 sudo[41440]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:06:48.890 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 14:06:48 smithi097 sudo[41440]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:06:48.890 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 14:06:48 smithi097 sudo[41440]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:06:48.890 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 14:06:48 smithi097 sudo[41443]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T14:06:48.890 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 14:06:48 smithi097 sudo[41443]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:06:48.891 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 14:06:48 smithi097 sudo[41443]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:06:48.891 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 14:06:48 smithi097 sudo[41443]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:06:49.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:48 smithi096 sudo[48525]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T14:06:49.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:48 smithi096 sudo[48525]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:06:49.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:48 smithi096 sudo[48525]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:06:49.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:48 smithi096 sudo[48525]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:06:49.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:48 smithi096 sudo[48529]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T14:06:49.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:48 smithi096 sudo[48529]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:06:49.110 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:48 smithi096 sudo[48529]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:06:49.110 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:48 smithi096 sudo[48529]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:06:49.172 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:48 smithi097 sudo[41446]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T14:06:49.172 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:48 smithi097 sudo[41446]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:06:49.172 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:48 smithi097 sudo[41446]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:06:49.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:48 smithi097 sudo[41446]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:06:49.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:48 smithi097 sudo[41449]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T14:06:49.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:48 smithi097 sudo[41449]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:06:49.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:48 smithi097 sudo[41449]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:06:49.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:48 smithi097 sudo[41449]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:06:49.187 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:48 smithi151 sudo[41888]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:06:49.187 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:48 smithi151 sudo[41888]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:06:49.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:48 smithi151 sudo[41888]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:06:49.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:48 smithi151 sudo[41891]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T14:06:49.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:48 smithi151 sudo[41891]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:06:49.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:48 smithi151 sudo[41891]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:06:49.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:48 smithi151 sudo[41891]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:06:49.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:06:49.817+0000 7f4f367fc700 1 -- 172.21.15.96:0/637045266 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f4f300a36b0 con 0x7f4f40104210 2024-04-25T14:06:49.818 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:49 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T14:06:49.818 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:49 smithi096 ceph-mon[29207]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T14:06:49.818 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:49 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:49.818 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:49 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:49.818 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:49 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:49.818 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:49 smithi096 ceph-mon[29207]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T14:06:49.818 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:49 smithi096 ceph-mon[29207]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T14:06:49.818 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:49 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:06:49.818 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:49 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:06:49.818 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:49 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:06:49.818 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:49 smithi096 ceph-mon[29207]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T14:06:49.818 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:49 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:06:49.819 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:49 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:06:49.819 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:49 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:06:49.819 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:49 smithi096 ceph-mon[29207]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T14:06:49.819 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:49 smithi096 ceph-mon[29207]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T14:06:49.819 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:49 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:06:49.819 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:49 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:06:49.819 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:49 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:06:49.819 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:49 smithi096 ceph-mon[29207]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T14:06:49.819 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:49 smithi096 ceph-mon[29207]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:06:49.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:49 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T14:06:49.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:49 smithi097 ceph-mon[32720]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T14:06:49.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:49 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:49.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:49 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:49.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:49 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:49.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:49 smithi097 ceph-mon[32720]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T14:06:49.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:49 smithi097 ceph-mon[32720]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T14:06:49.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:49 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:06:49.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:49 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:06:49.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:49 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:06:49.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:49 smithi097 ceph-mon[32720]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T14:06:49.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:49 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:06:49.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:49 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:06:49.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:49 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:06:49.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:49 smithi097 ceph-mon[32720]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T14:06:49.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:49 smithi097 ceph-mon[32720]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T14:06:49.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:49 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:06:49.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:49 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:06:49.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:49 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:06:49.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:49 smithi097 ceph-mon[32720]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T14:06:49.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:49 smithi097 ceph-mon[32720]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:06:49.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:49 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T14:06:49.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:49 smithi151 ceph-mon[32990]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T14:06:49.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:49 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:49.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:49 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:49.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:49 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:06:49.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:49 smithi151 ceph-mon[32990]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T14:06:49.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:49 smithi151 ceph-mon[32990]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T14:06:49.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:49 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:06:49.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:49 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:06:49.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:49 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:06:49.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:49 smithi151 ceph-mon[32990]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T14:06:49.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:49 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:06:49.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:49 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:06:49.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:49 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:06:49.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:49 smithi151 ceph-mon[32990]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T14:06:49.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:49 smithi151 ceph-mon[32990]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T14:06:49.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:49 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:06:49.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:49 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:06:49.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:49 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:06:49.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:49 smithi151 ceph-mon[32990]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T14:06:49.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:49 smithi151 ceph-mon[32990]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:06:50.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:50 smithi096 ceph-mon[29207]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T14:06:50.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:50 smithi096 ceph-mon[29207]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T14:06:50.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:50 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/3176633987' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9fc39468-a7ea-4ccf-b365-335ad2ea4c7a"}]: dispatch 2024-04-25T14:06:50.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:50 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/3176633987' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9fc39468-a7ea-4ccf-b365-335ad2ea4c7a"}]': finished 2024-04-25T14:06:50.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:50 smithi096 ceph-mon[29207]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T14:06:50.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:50 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:06:50.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:50 smithi097 ceph-mon[32720]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T14:06:50.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:50 smithi097 ceph-mon[32720]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T14:06:50.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:50 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/3176633987' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9fc39468-a7ea-4ccf-b365-335ad2ea4c7a"}]: dispatch 2024-04-25T14:06:50.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:50 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/3176633987' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9fc39468-a7ea-4ccf-b365-335ad2ea4c7a"}]': finished 2024-04-25T14:06:50.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:50 smithi097 ceph-mon[32720]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T14:06:50.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:50 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:06:50.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:50 smithi151 ceph-mon[32990]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T14:06:50.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:50 smithi151 ceph-mon[32990]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T14:06:50.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:50 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/3176633987' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9fc39468-a7ea-4ccf-b365-335ad2ea4c7a"}]: dispatch 2024-04-25T14:06:50.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:50 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/3176633987' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9fc39468-a7ea-4ccf-b365-335ad2ea4c7a"}]': finished 2024-04-25T14:06:50.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:50 smithi151 ceph-mon[32990]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T14:06:50.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:50 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:06:51.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:51 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/338481382' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:06:51.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:51 smithi096 ceph-mon[29207]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:06:51.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:51 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/338481382' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:06:51.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:51 smithi097 ceph-mon[32720]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:06:51.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:51 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/338481382' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:06:51.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:51 smithi151 ceph-mon[32990]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:06:54.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:54 smithi097 ceph-mon[32720]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:06:54.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:54 smithi151 ceph-mon[32990]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:06:54.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:54 smithi096 ceph-mon[29207]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:06:56.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:56 smithi097 ceph-mon[32720]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:06:56.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:56 smithi151 ceph-mon[32990]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:06:56.696 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:56 smithi096 ceph-mon[29207]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:06:58.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:58 smithi097 ceph-mon[32720]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:06:58.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:58 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T14:06:58.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:58 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:06:58.683 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:58 smithi096 ceph-mon[29207]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:06:58.683 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:58 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T14:06:58.683 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:58 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:06:58.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:58 smithi151 ceph-mon[32990]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:06:58.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:58 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T14:06:58.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:58 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:06:59.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:06:59 smithi097 ceph-mon[32720]: Deploying daemon osd.3 on smithi096 2024-04-25T14:06:59.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:06:59 smithi096 ceph-mon[29207]: Deploying daemon osd.3 on smithi096 2024-04-25T14:06:59.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:06:59 smithi151 ceph-mon[32990]: Deploying daemon osd.3 on smithi096 2024-04-25T14:07:00.674 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:00 smithi097 ceph-mon[32720]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:07:00.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:00 smithi096 ceph-mon[29207]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:07:00.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:00 smithi151 ceph-mon[32990]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:07:02.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:02 smithi097 ceph-mon[32720]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:07:02.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:02 smithi151 ceph-mon[32990]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:07:02.771 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:02 smithi096 ceph-mon[29207]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:07:04.660 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:04 smithi096 ceph-mon[29207]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:07:04.661 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:04 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:04.661 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:04 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:04.661 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:04 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:07:04.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:04 smithi097 ceph-mon[32720]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:07:04.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:04 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:04.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:04 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:04.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:04 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:07:04.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:04 smithi151 ceph-mon[32990]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:07:04.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:04 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:04.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:04 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:04.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:04 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:07:06.453 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:06 smithi096 ceph-mon[29207]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:07:06.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:06 smithi097 ceph-mon[32720]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:07:06.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:06 smithi151 ceph-mon[32990]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:07:07.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:07:07.666+0000 7f4f367fc700 1 -- 172.21.15.96:0/637045266 <== mgr.14152 v2:172.21.15.96:6800/536332425 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4f40063ad0 con 0x7f4f2c0afaf0 2024-04-25T14:07:07.666 INFO:teuthology.orchestra.run.smithi096.stdout:Created osd(s) 3 on host 'smithi096' 2024-04-25T14:07:07.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:07:07.668+0000 7f4f4764e700 1 -- 172.21.15.96:0/637045266 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f4f2c0afaf0 msgr2=0x7f4f2c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:07.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:07:07.668+0000 7f4f4764e700 1 --2- 172.21.15.96:0/637045266 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f4f2c0afaf0 0x7f4f2c0b2000 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0038a0 tx=0x7f4f3c01a040 comp rx=0 tx=0).stop 2024-04-25T14:07:07.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:07:07.668+0000 7f4f4764e700 1 -- 172.21.15.96:0/637045266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f40104210 msgr2=0x7f4f40199c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:07.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:07:07.668+0000 7f4f4764e700 1 --2- 172.21.15.96:0/637045266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f40104210 0x7f4f40199c10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4f3000c9b0 tx=0x7f4f3000ccc0 comp rx=0 tx=0).stop 2024-04-25T14:07:07.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:07:07.668+0000 7f4f4764e700 1 -- 172.21.15.96:0/637045266 shutdown_connections 2024-04-25T14:07:07.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:07:07.668+0000 7f4f4764e700 1 --2- 172.21.15.96:0/637045266 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f4f2c0afaf0 0x7f4f2c0b2000 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:07.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:07:07.668+0000 7f4f4764e700 1 --2- 172.21.15.96:0/637045266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f40104bb0 0x7f4f401a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:07.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:07:07.668+0000 7f4f4764e700 1 --2- 172.21.15.96:0/637045266 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f401030e0 0x7f4f401996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:07.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:07:07.668+0000 7f4f4764e700 1 --2- 172.21.15.96:0/637045266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f40104210 0x7f4f40199c10 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:07.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:07:07.668+0000 7f4f4764e700 1 -- 172.21.15.96:0/637045266 >> 172.21.15.96:0/637045266 conn(0x7f4f400fe740 msgr2=0x7f4f40107440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:07.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:07:07.668+0000 7f4f4764e700 1 -- 172.21.15.96:0/637045266 shutdown_connections 2024-04-25T14:07:07.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:07:07.668+0000 7f4f4764e700 1 -- 172.21.15.96:0/637045266 wait complete. 2024-04-25T14:07:07.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:07 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:07.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:07 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:07.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:07 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:07:07.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:07 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:07:07.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:07 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:07.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:07 smithi096 ceph-mon[29207]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:07:07.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:07 smithi096 ceph-mon[29207]: from='osd.3 [v2:172.21.15.96:6810/2274158748,v1:172.21.15.96:6811/2274158748]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T14:07:07.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:07 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:07.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:07 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:07.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:07 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:07:08.172 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:07 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:08.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:07 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:08.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:07 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:07:08.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:07 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:07:08.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:07 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:08.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:07 smithi097 ceph-mon[32720]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:07:08.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:07 smithi097 ceph-mon[32720]: from='osd.3 [v2:172.21.15.96:6810/2274158748,v1:172.21.15.96:6811/2274158748]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T14:07:08.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:07 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:08.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:07 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:08.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:07 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:07:08.187 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:07 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:08.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:07 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:08.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:07 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:07:08.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:07 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:07:08.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:07 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:08.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:07 smithi151 ceph-mon[32990]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:07:08.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:07 smithi151 ceph-mon[32990]: from='osd.3 [v2:172.21.15.96:6810/2274158748,v1:172.21.15.96:6811/2274158748]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T14:07:08.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:07 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:08.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:07 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:08.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:07 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:07:08.321 DEBUG:teuthology.orchestra.run.smithi096:osd.3> sudo journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@osd.3.service 2024-04-25T14:07:08.324 INFO:tasks.cephadm:Deploying osd.4 on smithi097 with /dev/vg_nvme/lv_3... 2024-04-25T14:07:08.324 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T14:07:08.568 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T14:07:08.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:08 smithi096 ceph-mon[29207]: from='osd.3 [v2:172.21.15.96:6810/2274158748,v1:172.21.15.96:6811/2274158748]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T14:07:08.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:08 smithi096 ceph-mon[29207]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T14:07:08.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:08 smithi096 ceph-mon[29207]: from='osd.3 [v2:172.21.15.96:6810/2274158748,v1:172.21.15.96:6811/2274158748]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-04-25T14:07:08.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:08 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:07:08.859 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 14:07:08 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-3[50623]: 2024-04-25T14:07:08.731+0000 7fcd5a120700 -1 osd.3 0 waiting for initial osdmap 2024-04-25T14:07:08.859 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 14:07:08 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-3[50623]: 2024-04-25T14:07:08.738+0000 7fcd55ed5700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:07:08.965 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:08 smithi097 ceph-mon[32720]: from='osd.3 [v2:172.21.15.96:6810/2274158748,v1:172.21.15.96:6811/2274158748]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T14:07:08.965 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:08 smithi097 ceph-mon[32720]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T14:07:08.965 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:08 smithi097 ceph-mon[32720]: from='osd.3 [v2:172.21.15.96:6810/2274158748,v1:172.21.15.96:6811/2274158748]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-04-25T14:07:08.965 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:08 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:07:09.187 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:08 smithi151 ceph-mon[32990]: from='osd.3 [v2:172.21.15.96:6810/2274158748,v1:172.21.15.96:6811/2274158748]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T14:07:09.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:08 smithi151 ceph-mon[32990]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T14:07:09.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:08 smithi151 ceph-mon[32990]: from='osd.3 [v2:172.21.15.96:6810/2274158748,v1:172.21.15.96:6811/2274158748]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-04-25T14:07:09.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:08 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:07:09.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:09 smithi096 ceph-mon[29207]: from='osd.3 [v2:172.21.15.96:6810/2274158748,v1:172.21.15.96:6811/2274158748]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-04-25T14:07:09.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:09 smithi096 ceph-mon[29207]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T14:07:09.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:09 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:07:09.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:09 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:07:09.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:09 smithi096 ceph-mon[29207]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:07:10.056 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:09 smithi097 ceph-mon[32720]: from='osd.3 [v2:172.21.15.96:6810/2274158748,v1:172.21.15.96:6811/2274158748]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-04-25T14:07:10.056 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:09 smithi097 ceph-mon[32720]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T14:07:10.056 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:09 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:07:10.056 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:09 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:07:10.056 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:09 smithi097 ceph-mon[32720]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:07:10.187 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:09 smithi151 ceph-mon[32990]: from='osd.3 [v2:172.21.15.96:6810/2274158748,v1:172.21.15.96:6811/2274158748]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-04-25T14:07:10.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:09 smithi151 ceph-mon[32990]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T14:07:10.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:09 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:07:10.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:09 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:07:10.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:09 smithi151 ceph-mon[32990]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:07:10.526 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T14:07:10.526 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T14:07:10.526 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2024-04-25T14:07:10.526 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2024-04-25T14:07:10.526 INFO:teuthology.orchestra.run.smithi097.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0133339 s, 786 MB/s 2024-04-25T14:07:10.526 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2024-04-25T14:07:11.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:10 smithi097 ceph-mon[32720]: purged_snaps scrub starts 2024-04-25T14:07:11.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:10 smithi097 ceph-mon[32720]: purged_snaps scrub ok 2024-04-25T14:07:11.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:10 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:07:11.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:10 smithi097 ceph-mon[32720]: osd.3 [v2:172.21.15.96:6810/2274158748,v1:172.21.15.96:6811/2274158748] boot 2024-04-25T14:07:11.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:10 smithi097 ceph-mon[32720]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T14:07:11.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:10 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:07:11.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:10 smithi096 ceph-mon[29207]: purged_snaps scrub starts 2024-04-25T14:07:11.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:10 smithi096 ceph-mon[29207]: purged_snaps scrub ok 2024-04-25T14:07:11.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:10 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:07:11.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:10 smithi096 ceph-mon[29207]: osd.3 [v2:172.21.15.96:6810/2274158748,v1:172.21.15.96:6811/2274158748] boot 2024-04-25T14:07:11.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:10 smithi096 ceph-mon[29207]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T14:07:11.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:10 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:07:11.119 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi097:vg_nvme/lv_3 2024-04-25T14:07:11.187 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:10 smithi151 ceph-mon[32990]: purged_snaps scrub starts 2024-04-25T14:07:11.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:10 smithi151 ceph-mon[32990]: purged_snaps scrub ok 2024-04-25T14:07:11.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:10 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:07:11.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:10 smithi151 ceph-mon[32990]: osd.3 [v2:172.21.15.96:6810/2274158748,v1:172.21.15.96:6811/2274158748] boot 2024-04-25T14:07:11.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:10 smithi151 ceph-mon[32990]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T14:07:11.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:10 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:07:11.355 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T14:07:12.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:11 smithi096 ceph-mon[29207]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T14:07:12.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:11 smithi096 ceph-mon[29207]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T14:07:12.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:11 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:12.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:11 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:12.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:11 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:07:12.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:11 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:07:12.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:11 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:12.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:11 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:07:12.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:11 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:07:12.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:11 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:12.110 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:11 smithi096 ceph-mon[29207]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T14:07:12.123 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:11 smithi097 ceph-mon[32720]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T14:07:12.123 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:11 smithi097 ceph-mon[32720]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T14:07:12.123 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:11 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:12.124 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:11 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:12.124 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:11 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:07:12.124 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:11 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:07:12.124 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:11 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:12.124 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:11 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:07:12.124 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:11 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:07:12.124 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:11 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:12.124 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:11 smithi097 ceph-mon[32720]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T14:07:12.187 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:11 smithi151 ceph-mon[32990]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T14:07:12.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:11 smithi151 ceph-mon[32990]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T14:07:12.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:11 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:12.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:11 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:12.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:11 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:07:12.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:11 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:07:12.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:11 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:12.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:11 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:07:12.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:11 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:07:12.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:11 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:12.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:11 smithi151 ceph-mon[32990]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T14:07:12.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:12.858+0000 7f8fe9771700 1 -- 172.21.15.97:0/3093544318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fe4104be0 msgr2=0x7f8fe410b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:12.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:12.858+0000 7f8fe9771700 1 --2- 172.21.15.97:0/3093544318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fe4104be0 0x7f8fe410b490 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc00b0a0 tx=0x7f8fdc00b3b0 comp rx=0 tx=0).stop 2024-04-25T14:07:12.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:12.858+0000 7f8fe9771700 1 -- 172.21.15.97:0/3093544318 shutdown_connections 2024-04-25T14:07:12.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:12.858+0000 7f8fe9771700 1 --2- 172.21.15.97:0/3093544318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fe4104be0 0x7f8fe410b490 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:12.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:12.858+0000 7f8fe9771700 1 --2- 172.21.15.97:0/3093544318 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fe4103110 0x7f8fe4103520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:12.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:12.858+0000 7f8fe9771700 1 --2- 172.21.15.97:0/3093544318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fe4104240 0x7f8fe4104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:12.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:12.858+0000 7f8fe9771700 1 -- 172.21.15.97:0/3093544318 >> 172.21.15.97:0/3093544318 conn(0x7f8fe40fe750 msgr2=0x7f8fe4100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:12.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:12.858+0000 7f8fe9771700 1 -- 172.21.15.97:0/3093544318 shutdown_connections 2024-04-25T14:07:12.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:12.858+0000 7f8fe9771700 1 -- 172.21.15.97:0/3093544318 wait complete. 2024-04-25T14:07:12.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:12.859+0000 7f8fe9771700 1 Processor -- start 2024-04-25T14:07:12.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:12.859+0000 7f8fe9771700 1 -- start start 2024-04-25T14:07:12.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:12.860+0000 7f8fe9771700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fe4103110 0x7f8fe41997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:12.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:12.860+0000 7f8fe9771700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fe4104240 0x7f8fe4199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:12.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:12.860+0000 7f8fe9771700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fe4104be0 0x7f8fe41a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:12.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:12.860+0000 7f8fe9771700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe410ec70 con 0x7f8fe4104be0 2024-04-25T14:07:12.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:12.860+0000 7f8fe9771700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe410eaf0 con 0x7f8fe4104240 2024-04-25T14:07:12.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:12.860+0000 7f8fe9771700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe410edf0 con 0x7f8fe4103110 2024-04-25T14:07:12.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:12.860+0000 7f8fe27fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fe4104240 0x7f8fe4199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:12.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:12.860+0000 7f8fe27fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fe4104240 0x7f8fe4199d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41896/0 (socket says 172.21.15.97:41896) 2024-04-25T14:07:12.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:12.860+0000 7f8fe27fc700 1 -- 172.21.15.97:0/2372074028 learned_addr learned my addr 172.21.15.97:0/2372074028 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T14:07:12.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:12.860+0000 7f8fe2ffd700 1 --2- 172.21.15.97:0/2372074028 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fe4103110 0x7f8fe41997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:12.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:12.860+0000 7f8fe37fe700 1 --2- 172.21.15.97:0/2372074028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fe4104be0 0x7f8fe41a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:12.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:12.860+0000 7f8fe27fc700 1 -- 172.21.15.97:0/2372074028 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fe4103110 msgr2=0x7f8fe41997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:12.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:12.860+0000 7f8fe27fc700 1 --2- 172.21.15.97:0/2372074028 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fe4103110 0x7f8fe41997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:12.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:12.860+0000 7f8fe27fc700 1 -- 172.21.15.97:0/2372074028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fe4104be0 msgr2=0x7f8fe41a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:12.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:12.860+0000 7f8fe27fc700 1 --2- 172.21.15.97:0/2372074028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fe4104be0 0x7f8fe41a0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:12.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:12.860+0000 7f8fe27fc700 1 -- 172.21.15.97:0/2372074028 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe41a1320 con 0x7f8fe4104240 2024-04-25T14:07:12.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:12.860+0000 7f8fe27fc700 1 --2- 172.21.15.97:0/2372074028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fe4104240 0x7f8fe4199d00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8fd800b530 tx=0x7f8fd800b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:12.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:12.860+0000 7f8fcbfff700 1 -- 172.21.15.97:0/2372074028 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8015020 con 0x7f8fe4104240 2024-04-25T14:07:12.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:12.860+0000 7f8fe9771700 1 -- 172.21.15.97:0/2372074028 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fe41a1610 con 0x7f8fe4104240 2024-04-25T14:07:12.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:12.860+0000 7f8fcbfff700 1 -- 172.21.15.97:0/2372074028 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fd8011ce0 con 0x7f8fe4104240 2024-04-25T14:07:12.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:12.860+0000 7f8fcbfff700 1 -- 172.21.15.97:0/2372074028 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8010e40 con 0x7f8fe4104240 2024-04-25T14:07:12.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:12.861+0000 7f8fe9771700 1 -- 172.21.15.97:0/2372074028 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fe41a1bd0 con 0x7f8fe4104240 2024-04-25T14:07:12.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:12.862+0000 7f8fe9771700 1 -- 172.21.15.97:0/2372074028 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fe4051190 con 0x7f8fe4104240 2024-04-25T14:07:12.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:12.863+0000 7f8fcbfff700 1 -- 172.21.15.97:0/2372074028 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f8fd8018020 con 0x7f8fe4104240 2024-04-25T14:07:12.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:12.863+0000 7f8fcbfff700 1 --2- 172.21.15.97:0/2372074028 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f8fcc0afb80 0x7f8fcc0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:12.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:12.864+0000 7f8fcbfff700 1 -- 172.21.15.97:0/2372074028 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3669+0+0 (secure 0 0 0) 0x7f8fd80dd250 con 0x7f8fe4104240 2024-04-25T14:07:12.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:12.864+0000 7f8fe2ffd700 1 --2- 172.21.15.97:0/2372074028 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f8fcc0afb80 0x7f8fcc0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:12.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:12.864+0000 7f8fe2ffd700 1 --2- 172.21.15.97:0/2372074028 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f8fcc0afb80 0x7f8fcc0b2090 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4104140 tx=0x7f8fd4009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:12.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:12.868+0000 7f8fcbfff700 1 -- 172.21.15.97:0/2372074028 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8fd801f020 con 0x7f8fe4104240 2024-04-25T14:07:12.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:12 smithi097 ceph-mon[32720]: Detected new or changed devices on smithi096 2024-04-25T14:07:12.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:12 smithi097 ceph-mon[32720]: Adjusting osd_memory_target on smithi096 to 8544M 2024-04-25T14:07:13.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:13.076+0000 7f8fe9771700 1 -- 172.21.15.97:0/2372074028 --> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f8fe4063ad0 con 0x7f8fcc0afb80 2024-04-25T14:07:13.108 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:12 smithi096 ceph-mon[29207]: Detected new or changed devices on smithi096 2024-04-25T14:07:13.108 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:12 smithi096 ceph-mon[29207]: Adjusting osd_memory_target on smithi096 to 8544M 2024-04-25T14:07:13.187 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:12 smithi151 ceph-mon[32990]: Detected new or changed devices on smithi096 2024-04-25T14:07:13.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:12 smithi151 ceph-mon[32990]: Adjusting osd_memory_target on smithi096 to 8544M 2024-04-25T14:07:14.031 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:13 smithi097 ceph-mon[32720]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:07:14.031 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:13 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:07:14.031 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:13 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:07:14.032 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:13 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:07:14.032 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:13 smithi097 ceph-mon[32720]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-04-25T14:07:14.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:13 smithi096 ceph-mon[29207]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:07:14.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:13 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:07:14.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:13 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:07:14.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:13 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:07:14.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:13 smithi096 ceph-mon[29207]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-04-25T14:07:14.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:13 smithi151 ceph-mon[32990]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:07:14.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:13 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:07:14.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:13 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:07:14.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:13 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:07:14.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:13 smithi151 ceph-mon[32990]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-04-25T14:07:16.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:16 smithi097 ceph-mon[32720]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T14:07:16.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:16 smithi097 ceph-mon[32720]: from='client.? 172.21.15.97:0/505218811' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "616582f1-12d2-456b-b7b3-e5517800e8a3"}]: dispatch 2024-04-25T14:07:16.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:16 smithi097 ceph-mon[32720]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "616582f1-12d2-456b-b7b3-e5517800e8a3"}]: dispatch 2024-04-25T14:07:16.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:16 smithi097 ceph-mon[32720]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "616582f1-12d2-456b-b7b3-e5517800e8a3"}]': finished 2024-04-25T14:07:16.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:16 smithi097 ceph-mon[32720]: osdmap e29: 5 total, 4 up, 5 in 2024-04-25T14:07:16.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:07:16.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:16 smithi151 ceph-mon[32990]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T14:07:16.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:16 smithi151 ceph-mon[32990]: from='client.? 172.21.15.97:0/505218811' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "616582f1-12d2-456b-b7b3-e5517800e8a3"}]: dispatch 2024-04-25T14:07:16.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:16 smithi151 ceph-mon[32990]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "616582f1-12d2-456b-b7b3-e5517800e8a3"}]: dispatch 2024-04-25T14:07:16.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:16 smithi151 ceph-mon[32990]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "616582f1-12d2-456b-b7b3-e5517800e8a3"}]': finished 2024-04-25T14:07:16.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:16 smithi151 ceph-mon[32990]: osdmap e29: 5 total, 4 up, 5 in 2024-04-25T14:07:16.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:16 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:07:16.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:16 smithi096 ceph-mon[29207]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T14:07:16.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:16 smithi096 ceph-mon[29207]: from='client.? 172.21.15.97:0/505218811' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "616582f1-12d2-456b-b7b3-e5517800e8a3"}]: dispatch 2024-04-25T14:07:16.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:16 smithi096 ceph-mon[29207]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "616582f1-12d2-456b-b7b3-e5517800e8a3"}]: dispatch 2024-04-25T14:07:16.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:16 smithi096 ceph-mon[29207]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "616582f1-12d2-456b-b7b3-e5517800e8a3"}]': finished 2024-04-25T14:07:16.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:16 smithi096 ceph-mon[29207]: osdmap e29: 5 total, 4 up, 5 in 2024-04-25T14:07:16.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:16 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:07:17.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:17 smithi097 ceph-mon[32720]: from='client.? 172.21.15.97:0/2848368236' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:07:17.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:17 smithi151 ceph-mon[32990]: from='client.? 172.21.15.97:0/2848368236' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:07:17.858 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:17 smithi096 ceph-mon[29207]: from='client.? 172.21.15.97:0/2848368236' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:07:18.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:18 smithi097 ceph-mon[32720]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-25T14:07:18.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:18 smithi151 ceph-mon[32990]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-25T14:07:18.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:18 smithi096 ceph-mon[29207]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-25T14:07:20.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:20 smithi097 ceph-mon[32720]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T14:07:20.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:20 smithi151 ceph-mon[32990]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T14:07:20.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:20 smithi096 ceph-mon[29207]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T14:07:22.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:22 smithi097 ceph-mon[32720]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T14:07:22.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:22 smithi151 ceph-mon[32990]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T14:07:22.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:22 smithi096 ceph-mon[29207]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T14:07:23.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:23 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T14:07:23.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:23 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:07:23.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:23 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T14:07:23.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:23 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:07:23.858 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:23 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T14:07:23.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:23 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:07:24.531 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:24 smithi097 ceph-mon[32720]: Deploying daemon osd.4 on smithi097 2024-04-25T14:07:24.531 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:24 smithi097 ceph-mon[32720]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T14:07:24.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:24 smithi151 ceph-mon[32990]: Deploying daemon osd.4 on smithi097 2024-04-25T14:07:24.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:24 smithi151 ceph-mon[32990]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T14:07:24.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:24 smithi096 ceph-mon[29207]: Deploying daemon osd.4 on smithi097 2024-04-25T14:07:24.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:24 smithi096 ceph-mon[29207]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T14:07:26.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:26 smithi151 ceph-mon[32990]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T14:07:26.786 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:26 smithi097 ceph-mon[32720]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T14:07:26.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:26 smithi096 ceph-mon[29207]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T14:07:28.435 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:28 smithi097 ceph-mon[32720]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T14:07:28.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:28 smithi151 ceph-mon[32990]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T14:07:28.858 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:28 smithi096 ceph-mon[29207]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T14:07:29.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:29 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:29.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:29 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:29.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:29 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:07:29.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:29 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:29.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:29 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:29.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:29 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:07:29.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:29 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:29.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:29 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:29.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:29 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:07:30.858 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:30 smithi096 ceph-mon[29207]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T14:07:30.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:30 smithi097 ceph-mon[32720]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T14:07:30.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:30 smithi151 ceph-mon[32990]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T14:07:32.039 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 4 on host 'smithi097' 2024-04-25T14:07:32.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:32.040+0000 7f8fcbfff700 1 -- 172.21.15.97:0/2372074028 <== mgr.14152 v2:172.21.15.96:6800/536332425 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8fe4063ad0 con 0x7f8fcc0afb80 2024-04-25T14:07:32.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:32.042+0000 7f8fe9771700 1 -- 172.21.15.97:0/2372074028 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f8fcc0afb80 msgr2=0x7f8fcc0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:32.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:32.042+0000 7f8fe9771700 1 --2- 172.21.15.97:0/2372074028 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f8fcc0afb80 0x7f8fcc0b2090 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4104140 tx=0x7f8fd4009190 comp rx=0 tx=0).stop 2024-04-25T14:07:32.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:32.042+0000 7f8fe9771700 1 -- 172.21.15.97:0/2372074028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fe4104240 msgr2=0x7f8fe4199d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:32.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:32.042+0000 7f8fe9771700 1 --2- 172.21.15.97:0/2372074028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fe4104240 0x7f8fe4199d00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8fd800b530 tx=0x7f8fd800b840 comp rx=0 tx=0).stop 2024-04-25T14:07:32.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:32.042+0000 7f8fe9771700 1 -- 172.21.15.97:0/2372074028 shutdown_connections 2024-04-25T14:07:32.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:32.042+0000 7f8fe9771700 1 --2- 172.21.15.97:0/2372074028 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f8fcc0afb80 0x7f8fcc0b2090 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:32.042+0000 7f8fe9771700 1 --2- 172.21.15.97:0/2372074028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fe4104be0 0x7f8fe41a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:32.042+0000 7f8fe9771700 1 --2- 172.21.15.97:0/2372074028 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fe4103110 0x7f8fe41997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:32.042+0000 7f8fe9771700 1 --2- 172.21.15.97:0/2372074028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fe4104240 0x7f8fe4199d00 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:32.042+0000 7f8fe9771700 1 -- 172.21.15.97:0/2372074028 >> 172.21.15.97:0/2372074028 conn(0x7f8fe40fe750 msgr2=0x7f8fe4107470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:32.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:32.042+0000 7f8fe9771700 1 -- 172.21.15.97:0/2372074028 shutdown_connections 2024-04-25T14:07:32.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:07:32.042+0000 7f8fe9771700 1 -- 172.21.15.97:0/2372074028 wait complete. 2024-04-25T14:07:32.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:32 smithi097 ceph-mon[32720]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T14:07:32.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:32 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:32.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:32 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:32.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:32 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:32.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:32 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:32.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:32 smithi097 ceph-mon[32720]: from='osd.4 [v2:172.21.15.97:6808/3727031962,v1:172.21.15.97:6809/3727031962]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T14:07:32.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:32 smithi097 ceph-mon[32720]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T14:07:32.725 DEBUG:teuthology.orchestra.run.smithi097:osd.4> sudo journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@osd.4.service 2024-04-25T14:07:32.727 INFO:tasks.cephadm:Deploying osd.5 on smithi151 with /dev/vg_nvme/lv_3... 2024-04-25T14:07:32.727 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T14:07:32.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:32 smithi096 ceph-mon[29207]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T14:07:32.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:32 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:32.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:32 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:32.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:32 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:32.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:32 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:32.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:32 smithi096 ceph-mon[29207]: from='osd.4 [v2:172.21.15.97:6808/3727031962,v1:172.21.15.97:6809/3727031962]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T14:07:32.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:32 smithi096 ceph-mon[29207]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T14:07:32.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:32 smithi151 ceph-mon[32990]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T14:07:32.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:32 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:32.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:32 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:32.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:32 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:32.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:32 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:32.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:32 smithi151 ceph-mon[32990]: from='osd.4 [v2:172.21.15.97:6808/3727031962,v1:172.21.15.97:6809/3727031962]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T14:07:32.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:32 smithi151 ceph-mon[32990]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T14:07:32.982 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:07:34.172 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:34 smithi097 ceph-mon[32720]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T14:07:34.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:34 smithi097 ceph-mon[32720]: osdmap e30: 5 total, 4 up, 5 in 2024-04-25T14:07:34.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:34 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:07:34.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:34 smithi097 ceph-mon[32720]: from='osd.4 [v2:172.21.15.97:6808/3727031962,v1:172.21.15.97:6809/3727031962]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T14:07:34.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:34 smithi097 ceph-mon[32720]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T14:07:34.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:34 smithi097 ceph-mon[32720]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T14:07:34.173 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 14:07:34 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-4[44233]: 2024-04-25T14:07:34.139+0000 7fb9a0359700 -1 osd.4 0 waiting for initial osdmap 2024-04-25T14:07:34.173 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 14:07:34 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-4[44233]: 2024-04-25T14:07:34.145+0000 7fb99c10e700 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:07:34.242 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:34 smithi151 ceph-mon[32990]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T14:07:34.243 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:34 smithi151 ceph-mon[32990]: osdmap e30: 5 total, 4 up, 5 in 2024-04-25T14:07:34.243 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:34 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:07:34.243 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:34 smithi151 ceph-mon[32990]: from='osd.4 [v2:172.21.15.97:6808/3727031962,v1:172.21.15.97:6809/3727031962]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T14:07:34.243 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:34 smithi151 ceph-mon[32990]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T14:07:34.243 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:34 smithi151 ceph-mon[32990]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T14:07:34.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:34 smithi096 ceph-mon[29207]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T14:07:34.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:34 smithi096 ceph-mon[29207]: osdmap e30: 5 total, 4 up, 5 in 2024-04-25T14:07:34.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:34 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:07:34.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:34 smithi096 ceph-mon[29207]: from='osd.4 [v2:172.21.15.97:6808/3727031962,v1:172.21.15.97:6809/3727031962]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T14:07:34.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:34 smithi096 ceph-mon[29207]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T14:07:34.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:34 smithi096 ceph-mon[29207]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T14:07:34.928 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T14:07:34.928 INFO:teuthology.orchestra.run.smithi151.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T14:07:34.928 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10+0 records in 2024-04-25T14:07:34.928 INFO:teuthology.orchestra.run.smithi151.stderr:10+0 records out 2024-04-25T14:07:34.928 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0129098 s, 812 MB/s 2024-04-25T14:07:34.928 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping successful for: 2024-04-25T14:07:35.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:35 smithi151 ceph-mon[32990]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-25T14:07:35.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:35 smithi151 ceph-mon[32990]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T14:07:35.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:35 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:07:35.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:35 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:07:35.405 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:35 smithi097 ceph-mon[32720]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-25T14:07:35.405 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:35 smithi097 ceph-mon[32720]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T14:07:35.405 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:35 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:07:35.405 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:35 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:07:35.537 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi151:vg_nvme/lv_3 2024-04-25T14:07:35.608 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:35 smithi096 ceph-mon[29207]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-25T14:07:35.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:35 smithi096 ceph-mon[29207]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T14:07:35.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:35 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:07:35.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:35 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:07:35.782 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:07:36.422 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:36 smithi097 ceph-mon[32720]: purged_snaps scrub starts 2024-04-25T14:07:36.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:36 smithi097 ceph-mon[32720]: purged_snaps scrub ok 2024-04-25T14:07:36.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:36 smithi097 ceph-mon[32720]: osd.4 [v2:172.21.15.97:6808/3727031962,v1:172.21.15.97:6809/3727031962] boot 2024-04-25T14:07:36.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:36 smithi097 ceph-mon[32720]: osdmap e32: 5 total, 5 up, 5 in 2024-04-25T14:07:36.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:36 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:07:36.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:36 smithi097 ceph-mon[32720]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T14:07:36.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:36 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:36.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:36 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:36.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:36 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:07:36.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:36 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:07:36.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:36 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:36.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:36 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:07:36.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:36 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:07:36.424 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:36 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:36.424 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:36 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:07:36.424 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:36 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:07:36.424 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:36 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:07:36.424 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:36 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:36.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:36 smithi151 ceph-mon[32990]: purged_snaps scrub starts 2024-04-25T14:07:36.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:36 smithi151 ceph-mon[32990]: purged_snaps scrub ok 2024-04-25T14:07:36.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:36 smithi151 ceph-mon[32990]: osd.4 [v2:172.21.15.97:6808/3727031962,v1:172.21.15.97:6809/3727031962] boot 2024-04-25T14:07:36.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:36 smithi151 ceph-mon[32990]: osdmap e32: 5 total, 5 up, 5 in 2024-04-25T14:07:36.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:36 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:07:36.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:36 smithi151 ceph-mon[32990]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T14:07:36.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:36 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:36.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:36 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:36.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:36 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:07:36.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:36 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:07:36.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:36 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:36.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:36 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:07:36.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:36 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:07:36.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:36 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:36.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:36 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:07:36.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:36 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:07:36.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:36 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:07:36.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:36 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:36.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:36 smithi096 ceph-mon[29207]: purged_snaps scrub starts 2024-04-25T14:07:36.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:36 smithi096 ceph-mon[29207]: purged_snaps scrub ok 2024-04-25T14:07:36.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:36 smithi096 ceph-mon[29207]: osd.4 [v2:172.21.15.97:6808/3727031962,v1:172.21.15.97:6809/3727031962] boot 2024-04-25T14:07:36.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:36 smithi096 ceph-mon[29207]: osdmap e32: 5 total, 5 up, 5 in 2024-04-25T14:07:36.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:36 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:07:36.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:36 smithi096 ceph-mon[29207]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T14:07:36.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:36 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:36.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:36 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:36.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:36 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:07:36.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:36 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:07:36.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:36 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:36.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:36 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:07:36.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:36 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:07:36.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:36 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:36.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:36 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:07:36.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:36 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:07:36.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:36 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:07:36.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:36 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:37.186 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:37 smithi151 ceph-mon[32990]: Detected new or changed devices on smithi097 2024-04-25T14:07:37.186 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:37 smithi151 ceph-mon[32990]: Adjusting osd_memory_target on smithi097 to 8544M 2024-04-25T14:07:37.186 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:37 smithi151 ceph-mon[32990]: osdmap e33: 5 total, 5 up, 5 in 2024-04-25T14:07:37.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:37.297+0000 7f6c6ba8e700 1 -- 172.21.15.151:0/627528285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c64109fd0 msgr2=0x7f6c6410c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:37.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:37.297+0000 7f6c6ba8e700 1 --2- 172.21.15.151:0/627528285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c64109fd0 0x7f6c6410c3f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c00b0a0 tx=0x7f6c5c00b3b0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:37.297+0000 7f6c6ba8e700 1 -- 172.21.15.151:0/627528285 shutdown_connections 2024-04-25T14:07:37.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:37.297+0000 7f6c6ba8e700 1 --2- 172.21.15.151:0/627528285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c64109fd0 0x7f6c6410c3f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:37.297+0000 7f6c6ba8e700 1 --2- 172.21.15.151:0/627528285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c64104d10 0x7f6c64107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:37.297+0000 7f6c6ba8e700 1 --2- 172.21.15.151:0/627528285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c64107670 0x7f6c64109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:37.297+0000 7f6c6ba8e700 1 -- 172.21.15.151:0/627528285 >> 172.21.15.151:0/627528285 conn(0x7f6c640fe750 msgr2=0x7f6c64100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:37.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:37.298+0000 7f6c6ba8e700 1 -- 172.21.15.151:0/627528285 shutdown_connections 2024-04-25T14:07:37.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:37.298+0000 7f6c6ba8e700 1 -- 172.21.15.151:0/627528285 wait complete. 2024-04-25T14:07:37.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:37.298+0000 7f6c6ba8e700 1 Processor -- start 2024-04-25T14:07:37.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:37.298+0000 7f6c6ba8e700 1 -- start start 2024-04-25T14:07:37.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:37.299+0000 7f6c6ba8e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c64104d10 0x7f6c6419a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:37.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:37.299+0000 7f6c6ba8e700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c64107670 0x7f6c6419ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:37.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:37.299+0000 7f6c6ba8e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c64109fd0 0x7f6c641a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:37.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:37.299+0000 7f6c6ba8e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6410fd10 con 0x7f6c64109fd0 2024-04-25T14:07:37.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:37.299+0000 7f6c6ba8e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6410fb90 con 0x7f6c64104d10 2024-04-25T14:07:37.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:37.299+0000 7f6c6ba8e700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6410fe90 con 0x7f6c64107670 2024-04-25T14:07:37.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:37.299+0000 7f6c69029700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c64107670 0x7f6c6419ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:37.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:37.299+0000 7f6c69029700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c64107670 0x7f6c6419ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51928/0 (socket says 172.21.15.151:51928) 2024-04-25T14:07:37.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:37.299+0000 7f6c69029700 1 -- 172.21.15.151:0/1290300450 learned_addr learned my addr 172.21.15.151:0/1290300450 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-25T14:07:37.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:37.299+0000 7f6c69029700 1 -- 172.21.15.151:0/1290300450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c64104d10 msgr2=0x7f6c6419a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:37.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:37.299+0000 7f6c69029700 1 --2- 172.21.15.151:0/1290300450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c64104d10 0x7f6c6419a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:37.299+0000 7f6c69029700 1 -- 172.21.15.151:0/1290300450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c64109fd0 msgr2=0x7f6c641a1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:37.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:37.299+0000 7f6c69029700 1 --2- 172.21.15.151:0/1290300450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c64109fd0 0x7f6c641a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:37.299+0000 7f6c69029700 1 -- 172.21.15.151:0/1290300450 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c641a2310 con 0x7f6c64107670 2024-04-25T14:07:37.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:37.299+0000 7f6c69029700 1 --2- 172.21.15.151:0/1290300450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c64107670 0x7f6c6419ad60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6c5400b530 tx=0x7f6c5400b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:37.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:37.299+0000 7f6c5affd700 1 -- 172.21.15.151:0/1290300450 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c54015020 con 0x7f6c64107670 2024-04-25T14:07:37.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:37.300+0000 7f6c5affd700 1 -- 172.21.15.151:0/1290300450 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c54011ce0 con 0x7f6c64107670 2024-04-25T14:07:37.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:37.300+0000 7f6c6ba8e700 1 -- 172.21.15.151:0/1290300450 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c641a2600 con 0x7f6c64107670 2024-04-25T14:07:37.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:37.300+0000 7f6c5affd700 1 -- 172.21.15.151:0/1290300450 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c54010d20 con 0x7f6c64107670 2024-04-25T14:07:37.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:37.300+0000 7f6c6ba8e700 1 -- 172.21.15.151:0/1290300450 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c641a2bc0 con 0x7f6c64107670 2024-04-25T14:07:37.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:37.300+0000 7f6c6ba8e700 1 -- 172.21.15.151:0/1290300450 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c64051190 con 0x7f6c64107670 2024-04-25T14:07:37.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:37.302+0000 7f6c5affd700 1 -- 172.21.15.151:0/1290300450 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f6c54018020 con 0x7f6c64107670 2024-04-25T14:07:37.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:37.302+0000 7f6c5affd700 1 --2- 172.21.15.151:0/1290300450 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f6c500afb80 0x7f6c500b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:37.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:37.303+0000 7f6c5affd700 1 -- 172.21.15.151:0/1290300450 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4101+0+0 (secure 0 0 0) 0x7f6c540dd920 con 0x7f6c64107670 2024-04-25T14:07:37.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:37.303+0000 7f6c6982a700 1 --2- 172.21.15.151:0/1290300450 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f6c500afb80 0x7f6c500b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:37.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:37.303+0000 7f6c6982a700 1 --2- 172.21.15.151:0/1290300450 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f6c500afb80 0x7f6c500b2090 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6c6406bf00 tx=0x7f6c60005c80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:37.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:37.306+0000 7f6c5affd700 1 -- 172.21.15.151:0/1290300450 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6c5401f020 con 0x7f6c64107670 2024-04-25T14:07:37.422 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:37 smithi097 ceph-mon[32720]: Detected new or changed devices on smithi097 2024-04-25T14:07:37.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:37 smithi097 ceph-mon[32720]: Adjusting osd_memory_target on smithi097 to 8544M 2024-04-25T14:07:37.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:37 smithi097 ceph-mon[32720]: osdmap e33: 5 total, 5 up, 5 in 2024-04-25T14:07:37.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:37.516+0000 7f6c6ba8e700 1 -- 172.21.15.151:0/1290300450 --> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f6c64063ad0 con 0x7f6c500afb80 2024-04-25T14:07:37.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:37 smithi096 ceph-mon[29207]: Detected new or changed devices on smithi097 2024-04-25T14:07:37.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:37 smithi096 ceph-mon[29207]: Adjusting osd_memory_target on smithi097 to 8544M 2024-04-25T14:07:37.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:37 smithi096 ceph-mon[29207]: osdmap e33: 5 total, 5 up, 5 in 2024-04-25T14:07:38.377 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:38 smithi151 ceph-mon[32990]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:07:38.377 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:38 smithi151 ceph-mon[32990]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:07:38.378 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:38 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:07:38.378 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:38 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:07:38.378 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:38 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:07:38.422 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:38 smithi097 ceph-mon[32720]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:07:38.422 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:38 smithi097 ceph-mon[32720]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:07:38.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:38 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:07:38.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:38 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:07:38.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:38 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:07:38.608 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:38 smithi096 ceph-mon[29207]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:07:38.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:38 smithi096 ceph-mon[29207]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:07:38.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:38 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:07:38.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:38 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:07:38.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:38 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:07:40.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:40 smithi097 ceph-mon[32720]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:07:40.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:40 smithi097 ceph-mon[32720]: from='client.? 172.21.15.151:0/352724550' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d277784-ab0f-4c90-81a1-2a4034c52a27"}]: dispatch 2024-04-25T14:07:40.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:40 smithi097 ceph-mon[32720]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d277784-ab0f-4c90-81a1-2a4034c52a27"}]: dispatch 2024-04-25T14:07:40.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:40 smithi097 ceph-mon[32720]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3d277784-ab0f-4c90-81a1-2a4034c52a27"}]': finished 2024-04-25T14:07:40.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:40 smithi097 ceph-mon[32720]: osdmap e34: 6 total, 5 up, 6 in 2024-04-25T14:07:40.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:40 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:07:40.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:40 smithi151 ceph-mon[32990]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:07:40.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:40 smithi151 ceph-mon[32990]: from='client.? 172.21.15.151:0/352724550' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d277784-ab0f-4c90-81a1-2a4034c52a27"}]: dispatch 2024-04-25T14:07:40.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:40 smithi151 ceph-mon[32990]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d277784-ab0f-4c90-81a1-2a4034c52a27"}]: dispatch 2024-04-25T14:07:40.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:40 smithi151 ceph-mon[32990]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3d277784-ab0f-4c90-81a1-2a4034c52a27"}]': finished 2024-04-25T14:07:40.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:40 smithi151 ceph-mon[32990]: osdmap e34: 6 total, 5 up, 6 in 2024-04-25T14:07:40.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:40 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:07:40.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:40 smithi096 ceph-mon[29207]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:07:40.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:40 smithi096 ceph-mon[29207]: from='client.? 172.21.15.151:0/352724550' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d277784-ab0f-4c90-81a1-2a4034c52a27"}]: dispatch 2024-04-25T14:07:40.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:40 smithi096 ceph-mon[29207]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d277784-ab0f-4c90-81a1-2a4034c52a27"}]: dispatch 2024-04-25T14:07:40.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:40 smithi096 ceph-mon[29207]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3d277784-ab0f-4c90-81a1-2a4034c52a27"}]': finished 2024-04-25T14:07:40.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:40 smithi096 ceph-mon[29207]: osdmap e34: 6 total, 5 up, 6 in 2024-04-25T14:07:40.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:40 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:07:41.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:41 smithi097 ceph-mon[32720]: from='client.? 172.21.15.151:0/251826218' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:07:41.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:41 smithi151 ceph-mon[32990]: from='client.? 172.21.15.151:0/251826218' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:07:41.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:41 smithi096 ceph-mon[29207]: from='client.? 172.21.15.151:0/251826218' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:07:42.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:42 smithi097 ceph-mon[32720]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:07:42.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:42 smithi151 ceph-mon[32990]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:07:42.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:42 smithi096 ceph-mon[29207]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:07:44.556 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:44 smithi151 ceph-mon[32990]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:07:44.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:44 smithi097 ceph-mon[32720]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:07:44.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:44 smithi096 ceph-mon[29207]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:07:46.431 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:46 smithi151 ceph-mon[32990]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:07:46.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:46 smithi097 ceph-mon[32720]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:07:46.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:46 smithi096 ceph-mon[29207]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:07:48.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:48 smithi097 ceph-mon[32720]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:07:48.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:48 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T14:07:48.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:48 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:07:48.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:48 smithi097 ceph-mon[32720]: Deploying daemon osd.5 on smithi151 2024-04-25T14:07:48.689 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:48 smithi151 ceph-mon[32990]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:07:48.689 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:48 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T14:07:48.690 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:48 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:07:48.690 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:48 smithi151 ceph-mon[32990]: Deploying daemon osd.5 on smithi151 2024-04-25T14:07:48.858 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:48 smithi096 ceph-mon[29207]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:07:48.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:48 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T14:07:48.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:48 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:07:48.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:48 smithi096 ceph-mon[29207]: Deploying daemon osd.5 on smithi151 2024-04-25T14:07:50.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:50 smithi097 ceph-mon[32720]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:07:50.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:50 smithi151 ceph-mon[32990]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:07:50.858 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:50 smithi096 ceph-mon[29207]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:07:52.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:52 smithi151 ceph-mon[32990]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:07:52.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:52 smithi096 ceph-mon[29207]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:07:52.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:52 smithi097 ceph-mon[32720]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:07:54.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:54 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:54.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:54 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:07:54.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:54 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:54.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:54 smithi151 ceph-mon[32990]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:07:54.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:54 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:54.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:54 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:07:54.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:54 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:54.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:54 smithi096 ceph-mon[29207]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:07:54.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:54 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:54.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:54 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:07:54.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:54 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:54.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:54 smithi097 ceph-mon[32720]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:07:56.601 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:56 smithi151 ceph-mon[32990]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:07:56.601 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:56 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:56.601 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:56 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:56.601 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:56 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:07:56.602 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:56 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:07:56.602 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:56 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:56.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:56 smithi097 ceph-mon[32720]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:07:56.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:56 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:56.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:56 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:56.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:56 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:07:56.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:56 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:07:56.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:56 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:56.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:56 smithi096 ceph-mon[29207]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:07:56.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:56 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:56.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:56 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:56.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:56 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:07:56.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:56 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:07:56.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:56 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:56.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:56.932+0000 7f6c5affd700 1 -- 172.21.15.151:0/1290300450 <== mgr.14152 v2:172.21.15.96:6800/536332425 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6c64063ad0 con 0x7f6c500afb80 2024-04-25T14:07:56.932 INFO:teuthology.orchestra.run.smithi151.stdout:Created osd(s) 5 on host 'smithi151' 2024-04-25T14:07:56.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:56.935+0000 7f6c6ba8e700 1 -- 172.21.15.151:0/1290300450 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f6c500afb80 msgr2=0x7f6c500b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:56.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:56.935+0000 7f6c6ba8e700 1 --2- 172.21.15.151:0/1290300450 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f6c500afb80 0x7f6c500b2090 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6c6406bf00 tx=0x7f6c60005c80 comp rx=0 tx=0).stop 2024-04-25T14:07:56.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:56.935+0000 7f6c6ba8e700 1 -- 172.21.15.151:0/1290300450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c64107670 msgr2=0x7f6c6419ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:56.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:56.935+0000 7f6c6ba8e700 1 --2- 172.21.15.151:0/1290300450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c64107670 0x7f6c6419ad60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6c5400b530 tx=0x7f6c5400b8f0 comp rx=0 tx=0).stop 2024-04-25T14:07:56.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:56.935+0000 7f6c6ba8e700 1 -- 172.21.15.151:0/1290300450 shutdown_connections 2024-04-25T14:07:56.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:56.935+0000 7f6c6ba8e700 1 --2- 172.21.15.151:0/1290300450 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f6c500afb80 0x7f6c500b2090 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:56.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:56.935+0000 7f6c6ba8e700 1 --2- 172.21.15.151:0/1290300450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c64109fd0 0x7f6c641a1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:56.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:56.935+0000 7f6c6ba8e700 1 --2- 172.21.15.151:0/1290300450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c64104d10 0x7f6c6419a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:56.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:56.935+0000 7f6c6ba8e700 1 --2- 172.21.15.151:0/1290300450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c64107670 0x7f6c6419ad60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:56.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:56.935+0000 7f6c6ba8e700 1 -- 172.21.15.151:0/1290300450 >> 172.21.15.151:0/1290300450 conn(0x7f6c640fe750 msgr2=0x7f6c64105840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:56.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:56.935+0000 7f6c6ba8e700 1 -- 172.21.15.151:0/1290300450 shutdown_connections 2024-04-25T14:07:56.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:07:56.935+0000 7f6c6ba8e700 1 -- 172.21.15.151:0/1290300450 wait complete. 2024-04-25T14:07:57.566 DEBUG:teuthology.orchestra.run.smithi151:osd.5> sudo journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@osd.5.service 2024-04-25T14:07:57.568 INFO:tasks.cephadm:Deploying osd.6 on smithi096 with /dev/vg_nvme/lv_2... 2024-04-25T14:07:57.568 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T14:07:57.814 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:07:58.172 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:57 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:58.172 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:57 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:58.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:57 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:07:58.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:57 smithi097 ceph-mon[32720]: from='osd.5 [v2:172.21.15.151:6808/223882546,v1:172.21.15.151:6809/223882546]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T14:07:58.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:57 smithi097 ceph-mon[32720]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T14:07:58.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:57 smithi097 ceph-mon[32720]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:07:58.181 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:57 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:58.181 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:57 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:58.181 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:57 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:07:58.181 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:57 smithi096 ceph-mon[29207]: from='osd.5 [v2:172.21.15.151:6808/223882546,v1:172.21.15.151:6809/223882546]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T14:07:58.181 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:57 smithi096 ceph-mon[29207]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T14:07:58.181 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:57 smithi096 ceph-mon[29207]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:07:58.187 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:57 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:58.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:57 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:07:58.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:57 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:07:58.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:57 smithi151 ceph-mon[32990]: from='osd.5 [v2:172.21.15.151:6808/223882546,v1:172.21.15.151:6809/223882546]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T14:07:58.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:57 smithi151 ceph-mon[32990]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T14:07:58.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:57 smithi151 ceph-mon[32990]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:07:58.907 INFO:journalctl@ceph.osd.5.smithi151.stdout:Apr 25 14:07:58 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-5[44759]: 2024-04-25T14:07:58.901+0000 7fb23677b700 -1 osd.5 0 waiting for initial osdmap 2024-04-25T14:07:58.908 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:58 smithi151 ceph-mon[32990]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T14:07:58.908 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:58 smithi151 ceph-mon[32990]: osdmap e35: 6 total, 5 up, 6 in 2024-04-25T14:07:58.908 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:58 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:07:58.908 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:58 smithi151 ceph-mon[32990]: from='osd.5 [v2:172.21.15.151:6808/223882546,v1:172.21.15.151:6809/223882546]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-04-25T14:07:58.908 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:58 smithi151 ceph-mon[32990]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-04-25T14:07:59.172 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:58 smithi097 ceph-mon[32720]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T14:07:59.172 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:58 smithi097 ceph-mon[32720]: osdmap e35: 6 total, 5 up, 6 in 2024-04-25T14:07:59.172 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:58 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:07:59.172 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:58 smithi097 ceph-mon[32720]: from='osd.5 [v2:172.21.15.151:6808/223882546,v1:172.21.15.151:6809/223882546]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-04-25T14:07:59.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:58 smithi097 ceph-mon[32720]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-04-25T14:07:59.187 INFO:journalctl@ceph.osd.5.smithi151.stdout:Apr 25 14:07:58 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-5[44759]: 2024-04-25T14:07:58.909+0000 7fb232530700 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:07:59.236 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:58 smithi096 ceph-mon[29207]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T14:07:59.236 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:58 smithi096 ceph-mon[29207]: osdmap e35: 6 total, 5 up, 6 in 2024-04-25T14:07:59.236 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:58 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:07:59.236 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:58 smithi096 ceph-mon[29207]: from='osd.5 [v2:172.21.15.151:6808/223882546,v1:172.21.15.151:6809/223882546]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-04-25T14:07:59.236 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:58 smithi096 ceph-mon[29207]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-04-25T14:07:59.816 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T14:07:59.816 INFO:teuthology.orchestra.run.smithi096.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T14:07:59.816 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10+0 records in 2024-04-25T14:07:59.816 INFO:teuthology.orchestra.run.smithi096.stderr:10+0 records out 2024-04-25T14:07:59.816 INFO:teuthology.orchestra.run.smithi096.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112978 s, 928 MB/s 2024-04-25T14:07:59.816 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 2024-04-25T14:07:59.816 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping successful for: 2024-04-25T14:08:00.119 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:59 smithi151 ceph-mon[32990]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-04-25T14:08:00.119 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:59 smithi151 ceph-mon[32990]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T14:08:00.119 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:59 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:08:00.119 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:59 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:08:00.119 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:07:59 smithi151 ceph-mon[32990]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:08:00.172 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:59 smithi097 ceph-mon[32720]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-04-25T14:08:00.172 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:59 smithi097 ceph-mon[32720]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T14:08:00.172 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:59 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:08:00.172 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:59 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:08:00.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:07:59 smithi097 ceph-mon[32720]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:08:00.216 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:59 smithi096 ceph-mon[29207]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-04-25T14:08:00.217 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:59 smithi096 ceph-mon[29207]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T14:08:00.217 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:59 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:08:00.217 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:59 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:08:00.217 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:07:59 smithi096 ceph-mon[29207]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:08:00.546 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi096:vg_nvme/lv_2 2024-04-25T14:08:00.797 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:08:01.078 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:00 smithi096 ceph-mon[29207]: purged_snaps scrub starts 2024-04-25T14:08:01.078 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:00 smithi096 ceph-mon[29207]: purged_snaps scrub ok 2024-04-25T14:08:01.078 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:00 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:08:01.078 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:00 smithi096 ceph-mon[29207]: osd.5 [v2:172.21.15.151:6808/223882546,v1:172.21.15.151:6809/223882546] boot 2024-04-25T14:08:01.078 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:00 smithi096 ceph-mon[29207]: osdmap e37: 6 total, 6 up, 6 in 2024-04-25T14:08:01.078 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:00 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:08:01.078 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:00 smithi096 ceph-mon[29207]: Detected new or changed devices on smithi151 2024-04-25T14:08:01.078 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:00 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:01.079 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:00 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:01.079 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:00 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:08:01.079 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:00 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:08:01.079 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:00 smithi096 ceph-mon[29207]: Adjusting osd_memory_target on smithi151 to 8544M 2024-04-25T14:08:01.079 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:00 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:01.079 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:00 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:08:01.079 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:00 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:08:01.079 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:00 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:01.172 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:00 smithi097 ceph-mon[32720]: purged_snaps scrub starts 2024-04-25T14:08:01.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:00 smithi097 ceph-mon[32720]: purged_snaps scrub ok 2024-04-25T14:08:01.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:00 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:08:01.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:00 smithi097 ceph-mon[32720]: osd.5 [v2:172.21.15.151:6808/223882546,v1:172.21.15.151:6809/223882546] boot 2024-04-25T14:08:01.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:00 smithi097 ceph-mon[32720]: osdmap e37: 6 total, 6 up, 6 in 2024-04-25T14:08:01.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:00 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:08:01.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:00 smithi097 ceph-mon[32720]: Detected new or changed devices on smithi151 2024-04-25T14:08:01.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:00 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:01.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:00 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:01.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:00 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:08:01.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:00 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:08:01.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:00 smithi097 ceph-mon[32720]: Adjusting osd_memory_target on smithi151 to 8544M 2024-04-25T14:08:01.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:00 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:01.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:00 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:08:01.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:00 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:08:01.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:00 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:01.187 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:00 smithi151 ceph-mon[32990]: purged_snaps scrub starts 2024-04-25T14:08:01.187 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:00 smithi151 ceph-mon[32990]: purged_snaps scrub ok 2024-04-25T14:08:01.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:00 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:08:01.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:00 smithi151 ceph-mon[32990]: osd.5 [v2:172.21.15.151:6808/223882546,v1:172.21.15.151:6809/223882546] boot 2024-04-25T14:08:01.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:00 smithi151 ceph-mon[32990]: osdmap e37: 6 total, 6 up, 6 in 2024-04-25T14:08:01.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:00 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:08:01.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:00 smithi151 ceph-mon[32990]: Detected new or changed devices on smithi151 2024-04-25T14:08:01.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:00 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:01.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:00 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:01.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:00 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:08:01.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:00 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:08:01.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:00 smithi151 ceph-mon[32990]: Adjusting osd_memory_target on smithi151 to 8544M 2024-04-25T14:08:01.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:00 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:01.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:00 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:08:01.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:00 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:08:01.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:00 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:02.187 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:01 smithi151 ceph-mon[32990]: osdmap e38: 6 total, 6 up, 6 in 2024-04-25T14:08:02.187 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:01 smithi151 ceph-mon[32990]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:08:02.261 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:01 smithi096 ceph-mon[29207]: osdmap e38: 6 total, 6 up, 6 in 2024-04-25T14:08:02.261 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:01 smithi096 ceph-mon[29207]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:08:02.348 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:01 smithi097 ceph-mon[32720]: osdmap e38: 6 total, 6 up, 6 in 2024-04-25T14:08:02.348 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:01 smithi097 ceph-mon[32720]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:08:02.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:02.494+0000 7f6ed5835700 1 -- 172.21.15.96:0/1987532598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ed0104210 msgr2=0x7f6ed0104620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:02.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:02.494+0000 7f6ed5835700 1 --2- 172.21.15.96:0/1987532598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ed0104210 0x7f6ed0104620 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4009990 tx=0x7f6ec4009ca0 comp rx=0 tx=0).stop 2024-04-25T14:08:02.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:02.494+0000 7f6ed5835700 1 -- 172.21.15.96:0/1987532598 shutdown_connections 2024-04-25T14:08:02.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:02.494+0000 7f6ed5835700 1 --2- 172.21.15.96:0/1987532598 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ed0104bb0 0x7f6ed010b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:02.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:02.494+0000 7f6ed5835700 1 --2- 172.21.15.96:0/1987532598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ed01030e0 0x7f6ed01034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:02.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:02.494+0000 7f6ed5835700 1 --2- 172.21.15.96:0/1987532598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ed0104210 0x7f6ed0104620 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:02.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:02.494+0000 7f6ed5835700 1 -- 172.21.15.96:0/1987532598 >> 172.21.15.96:0/1987532598 conn(0x7f6ed00fe740 msgr2=0x7f6ed0100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:02.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:02.494+0000 7f6ed5835700 1 -- 172.21.15.96:0/1987532598 shutdown_connections 2024-04-25T14:08:02.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:02.494+0000 7f6ed5835700 1 -- 172.21.15.96:0/1987532598 wait complete. 2024-04-25T14:08:02.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:02.495+0000 7f6ed5835700 1 Processor -- start 2024-04-25T14:08:02.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:02.495+0000 7f6ed5835700 1 -- start start 2024-04-25T14:08:02.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:02.495+0000 7f6ed5835700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ed01030e0 0x7f6ed0076340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:02.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:02.495+0000 7f6ed5835700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ed0104210 0x7f6ed019f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:02.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:02.495+0000 7f6ed5835700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ed0104bb0 0x7f6ed01a1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:02.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:02.495+0000 7f6ed5835700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed010ebd0 con 0x7f6ed0104bb0 2024-04-25T14:08:02.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:02.495+0000 7f6ed5835700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed010ea50 con 0x7f6ed0104210 2024-04-25T14:08:02.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:02.495+0000 7f6ed5835700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed010ed50 con 0x7f6ed01030e0 2024-04-25T14:08:02.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:02.495+0000 7f6ecf7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ed0104bb0 0x7f6ed01a1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:02.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:02.495+0000 7f6ecf7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ed0104bb0 0x7f6ed01a1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36912/0 (socket says 172.21.15.96:36912) 2024-04-25T14:08:02.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:02.495+0000 7f6ecf7fe700 1 -- 172.21.15.96:0/1087586532 learned_addr learned my addr 172.21.15.96:0/1087586532 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:08:02.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:02.495+0000 7f6ece7fc700 1 --2- 172.21.15.96:0/1087586532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ed0104210 0x7f6ed019f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:02.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:02.495+0000 7f6eceffd700 1 --2- 172.21.15.96:0/1087586532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ed01030e0 0x7f6ed0076340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:02.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:02.496+0000 7f6ecf7fe700 1 -- 172.21.15.96:0/1087586532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ed01030e0 msgr2=0x7f6ed0076340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:02.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:02.496+0000 7f6ecf7fe700 1 --2- 172.21.15.96:0/1087586532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ed01030e0 0x7f6ed0076340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:02.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:02.496+0000 7f6ecf7fe700 1 -- 172.21.15.96:0/1087586532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ed0104210 msgr2=0x7f6ed019f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:02.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:02.496+0000 7f6ecf7fe700 1 --2- 172.21.15.96:0/1087586532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ed0104210 0x7f6ed019f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:02.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:02.496+0000 7f6ecf7fe700 1 -- 172.21.15.96:0/1087586532 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed01a2370 con 0x7f6ed0104bb0 2024-04-25T14:08:02.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:02.496+0000 7f6ecf7fe700 1 --2- 172.21.15.96:0/1087586532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ed0104bb0 0x7f6ed01a1b40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6ec800b970 tx=0x7f6ec800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:02.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:02.496+0000 7f6ed4833700 1 -- 172.21.15.96:0/1087586532 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8019070 con 0x7f6ed0104bb0 2024-04-25T14:08:02.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:02.496+0000 7f6ed5835700 1 -- 172.21.15.96:0/1087586532 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ed01a2660 con 0x7f6ed0104bb0 2024-04-25T14:08:02.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:02.496+0000 7f6ed4833700 1 -- 172.21.15.96:0/1087586532 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ec800cde0 con 0x7f6ed0104bb0 2024-04-25T14:08:02.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:02.496+0000 7f6ed4833700 1 -- 172.21.15.96:0/1087586532 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8007370 con 0x7f6ed0104bb0 2024-04-25T14:08:02.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:02.496+0000 7f6ed5835700 1 -- 172.21.15.96:0/1087586532 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ed01a2bd0 con 0x7f6ed0104bb0 2024-04-25T14:08:02.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:02.497+0000 7f6ed5835700 1 -- 172.21.15.96:0/1087586532 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ed0051190 con 0x7f6ed0104bb0 2024-04-25T14:08:02.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:02.498+0000 7f6ed4833700 1 -- 172.21.15.96:0/1087586532 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f6ec801c070 con 0x7f6ed0104bb0 2024-04-25T14:08:02.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:02.499+0000 7f6ed4833700 1 --2- 172.21.15.96:0/1087586532 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f6eb80afb80 0x7f6eb80b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:02.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:02.499+0000 7f6eceffd700 1 --2- 172.21.15.96:0/1087586532 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f6eb80afb80 0x7f6eb80b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:02.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:02.499+0000 7f6ed4833700 1 -- 172.21.15.96:0/1087586532 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4533+0+0 (secure 0 0 0) 0x7f6ec80dd310 con 0x7f6ed0104bb0 2024-04-25T14:08:02.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:02.499+0000 7f6eceffd700 1 --2- 172.21.15.96:0/1087586532 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f6eb80afb80 0x7f6eb80b2090 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0005950 tx=0x7f6ec00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:02.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:02.503+0000 7f6ed4833700 1 -- 172.21.15.96:0/1087586532 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6ec80a6db0 con 0x7f6ed0104bb0 2024-04-25T14:08:02.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:02.712+0000 7f6ed5835700 1 -- 172.21.15.96:0/1087586532 --> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f6ed0063ad0 con 0x7f6eb80afb80 2024-04-25T14:08:03.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:02 smithi096 ceph-mon[29207]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T14:08:03.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:02 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:08:03.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:02 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:08:03.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:02 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:08:03.172 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:02 smithi097 ceph-mon[32720]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T14:08:03.172 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:02 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:08:03.172 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:02 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:08:03.172 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:02 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:08:03.187 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:02 smithi151 ceph-mon[32990]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T14:08:03.187 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:02 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:08:03.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:02 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:08:03.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:02 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:08:04.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:03 smithi151 ceph-mon[32990]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:08:04.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:03 smithi151 ceph-mon[32990]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-04-25T14:08:04.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:03 smithi096 ceph-mon[29207]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:08:04.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:03 smithi096 ceph-mon[29207]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-04-25T14:08:04.422 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:03 smithi097 ceph-mon[32720]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:08:04.422 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:03 smithi097 ceph-mon[32720]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-04-25T14:08:06.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:06 smithi097 ceph-mon[32720]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T14:08:06.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:06 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/2214541993' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a9ad585-cb1a-4239-9d18-055f59164049"}]: dispatch 2024-04-25T14:08:06.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:06 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/2214541993' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9a9ad585-cb1a-4239-9d18-055f59164049"}]': finished 2024-04-25T14:08:06.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:06 smithi097 ceph-mon[32720]: osdmap e40: 7 total, 6 up, 7 in 2024-04-25T14:08:06.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:06 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:08:06.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:06 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/1890524375' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:08:06.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:06 smithi151 ceph-mon[32990]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T14:08:06.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:06 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/2214541993' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a9ad585-cb1a-4239-9d18-055f59164049"}]: dispatch 2024-04-25T14:08:06.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:06 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/2214541993' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9a9ad585-cb1a-4239-9d18-055f59164049"}]': finished 2024-04-25T14:08:06.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:06 smithi151 ceph-mon[32990]: osdmap e40: 7 total, 6 up, 7 in 2024-04-25T14:08:06.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:06 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:08:06.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:06 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/1890524375' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:08:06.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:06 smithi096 ceph-mon[29207]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T14:08:06.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:06 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/2214541993' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a9ad585-cb1a-4239-9d18-055f59164049"}]: dispatch 2024-04-25T14:08:06.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:06 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/2214541993' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9a9ad585-cb1a-4239-9d18-055f59164049"}]': finished 2024-04-25T14:08:06.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:06 smithi096 ceph-mon[29207]: osdmap e40: 7 total, 6 up, 7 in 2024-04-25T14:08:06.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:06 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:08:06.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:06 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/1890524375' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:08:08.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:08 smithi097 ceph-mon[32720]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-04-25T14:08:08.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:08 smithi151 ceph-mon[32990]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-04-25T14:08:08.858 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:08 smithi096 ceph-mon[29207]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-04-25T14:08:10.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:10 smithi097 ceph-mon[32720]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T14:08:10.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:10 smithi151 ceph-mon[32990]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T14:08:10.722 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:10 smithi096 ceph-mon[29207]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T14:08:12.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:12 smithi096 ceph-mon[29207]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-25T14:08:12.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:12 smithi097 ceph-mon[32720]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-25T14:08:12.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:12 smithi151 ceph-mon[32990]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-25T14:08:13.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:13 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T14:08:13.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:13 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:08:13.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:13 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T14:08:13.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:13 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:08:13.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:13 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T14:08:13.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:13 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:08:14.608 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:14 smithi096 ceph-mon[29207]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:08:14.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:14 smithi096 ceph-mon[29207]: Deploying daemon osd.6 on smithi096 2024-04-25T14:08:14.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:14 smithi097 ceph-mon[32720]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:08:14.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:14 smithi097 ceph-mon[32720]: Deploying daemon osd.6 on smithi096 2024-04-25T14:08:14.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:14 smithi151 ceph-mon[32990]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:08:14.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:14 smithi151 ceph-mon[32990]: Deploying daemon osd.6 on smithi096 2024-04-25T14:08:16.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:16 smithi151 ceph-mon[32990]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:08:16.781 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:16 smithi096 ceph-mon[29207]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:08:16.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:16 smithi097 ceph-mon[32720]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:08:18.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:18 smithi096 ceph-mon[29207]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:08:18.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:18 smithi151 ceph-mon[32990]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:08:18.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:18 smithi097 ceph-mon[32720]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:08:20.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:20 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:20.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:20 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:20.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:20 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:08:20.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:20 smithi096 ceph-mon[29207]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:08:20.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:20 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:20.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:20 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:20.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:20 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:08:20.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:20 smithi097 ceph-mon[32720]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:08:20.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:20 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:20.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:20 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:20.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:20 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:08:20.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:20 smithi151 ceph-mon[32990]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:08:22.574 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:22 smithi096 ceph-mon[29207]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:08:22.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:22 smithi097 ceph-mon[32720]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:08:22.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:22 smithi151 ceph-mon[32990]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:08:22.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:22.919+0000 7f6ed4833700 1 -- 172.21.15.96:0/1087586532 <== mgr.14152 v2:172.21.15.96:6800/536332425 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6ed0063ad0 con 0x7f6eb80afb80 2024-04-25T14:08:22.919 INFO:teuthology.orchestra.run.smithi096.stdout:Created osd(s) 6 on host 'smithi096' 2024-04-25T14:08:22.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:22.921+0000 7f6ed5835700 1 -- 172.21.15.96:0/1087586532 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f6eb80afb80 msgr2=0x7f6eb80b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:22.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:22.921+0000 7f6ed5835700 1 --2- 172.21.15.96:0/1087586532 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f6eb80afb80 0x7f6eb80b2090 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0005950 tx=0x7f6ec00058e0 comp rx=0 tx=0).stop 2024-04-25T14:08:22.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:22.921+0000 7f6ed5835700 1 -- 172.21.15.96:0/1087586532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ed0104bb0 msgr2=0x7f6ed01a1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:22.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:22.921+0000 7f6ed5835700 1 --2- 172.21.15.96:0/1087586532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ed0104bb0 0x7f6ed01a1b40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6ec800b970 tx=0x7f6ec800bc80 comp rx=0 tx=0).stop 2024-04-25T14:08:22.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:22.922+0000 7f6ed5835700 1 -- 172.21.15.96:0/1087586532 shutdown_connections 2024-04-25T14:08:22.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:22.922+0000 7f6ed5835700 1 --2- 172.21.15.96:0/1087586532 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f6eb80afb80 0x7f6eb80b2090 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:22.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:22.922+0000 7f6ed5835700 1 --2- 172.21.15.96:0/1087586532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ed0104bb0 0x7f6ed01a1b40 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:22.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:22.922+0000 7f6ed5835700 1 --2- 172.21.15.96:0/1087586532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ed01030e0 0x7f6ed0076340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:22.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:22.922+0000 7f6ed5835700 1 --2- 172.21.15.96:0/1087586532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ed0104210 0x7f6ed019f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:22.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:22.922+0000 7f6ed5835700 1 -- 172.21.15.96:0/1087586532 >> 172.21.15.96:0/1087586532 conn(0x7f6ed00fe740 msgr2=0x7f6ed0107440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:22.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:22.922+0000 7f6ed5835700 1 -- 172.21.15.96:0/1087586532 shutdown_connections 2024-04-25T14:08:22.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:08:22.922+0000 7f6ed5835700 1 -- 172.21.15.96:0/1087586532 wait complete. 2024-04-25T14:08:23.707 DEBUG:teuthology.orchestra.run.smithi096:osd.6> sudo journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@osd.6.service 2024-04-25T14:08:23.709 INFO:tasks.cephadm:Deploying osd.7 on smithi097 with /dev/vg_nvme/lv_2... 2024-04-25T14:08:23.709 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T14:08:24.007 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:23 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:24.008 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:23 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:24.008 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:23 smithi096 ceph-mon[29207]: from='osd.6 [v2:172.21.15.96:6818/96197407,v1:172.21.15.96:6819/96197407]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T14:08:24.008 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:23 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:24.008 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:23 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:24.008 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:23 smithi096 ceph-mon[29207]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:08:24.059 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T14:08:24.172 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:23 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:24.172 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:23 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:24.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:23 smithi097 ceph-mon[32720]: from='osd.6 [v2:172.21.15.96:6818/96197407,v1:172.21.15.96:6819/96197407]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T14:08:24.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:23 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:24.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:23 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:24.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:23 smithi097 ceph-mon[32720]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:08:24.187 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:23 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:24.187 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:23 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:24.187 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:23 smithi151 ceph-mon[32990]: from='osd.6 [v2:172.21.15.96:6818/96197407,v1:172.21.15.96:6819/96197407]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T14:08:24.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:23 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:24.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:23 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:24.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:23 smithi151 ceph-mon[32990]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:08:25.187 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:24 smithi151 ceph-mon[32990]: from='osd.6 [v2:172.21.15.96:6818/96197407,v1:172.21.15.96:6819/96197407]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T14:08:25.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:24 smithi151 ceph-mon[32990]: osdmap e41: 7 total, 6 up, 7 in 2024-04-25T14:08:25.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:24 smithi151 ceph-mon[32990]: from='osd.6 [v2:172.21.15.96:6818/96197407,v1:172.21.15.96:6819/96197407]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-04-25T14:08:25.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:24 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:08:25.222 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:24 smithi096 ceph-mon[29207]: from='osd.6 [v2:172.21.15.96:6818/96197407,v1:172.21.15.96:6819/96197407]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T14:08:25.222 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:24 smithi096 ceph-mon[29207]: osdmap e41: 7 total, 6 up, 7 in 2024-04-25T14:08:25.222 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:24 smithi096 ceph-mon[29207]: from='osd.6 [v2:172.21.15.96:6818/96197407,v1:172.21.15.96:6819/96197407]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-04-25T14:08:25.222 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:24 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:08:25.225 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:24 smithi097 ceph-mon[32720]: from='osd.6 [v2:172.21.15.96:6818/96197407,v1:172.21.15.96:6819/96197407]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T14:08:25.225 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:24 smithi097 ceph-mon[32720]: osdmap e41: 7 total, 6 up, 7 in 2024-04-25T14:08:25.225 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:24 smithi097 ceph-mon[32720]: from='osd.6 [v2:172.21.15.96:6818/96197407,v1:172.21.15.96:6819/96197407]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-04-25T14:08:25.225 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:24 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:08:25.609 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:08:25 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:08:25.388+0000 7f7be4a4c700 -1 osd.6 0 waiting for initial osdmap 2024-04-25T14:08:25.609 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:08:25 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:08:25.402+0000 7f7bdc7f9700 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:08:26.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:25 smithi097 ceph-mon[32720]: from='osd.6 [v2:172.21.15.96:6818/96197407,v1:172.21.15.96:6819/96197407]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-04-25T14:08:26.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:25 smithi097 ceph-mon[32720]: osdmap e42: 7 total, 6 up, 7 in 2024-04-25T14:08:26.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:25 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:08:26.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:25 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:08:26.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:25 smithi097 ceph-mon[32720]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:08:26.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:25 smithi097 ceph-mon[32720]: from='osd.6 [v2:172.21.15.96:6818/96197407,v1:172.21.15.96:6819/96197407]' entity='osd.6' 2024-04-25T14:08:26.187 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:25 smithi151 ceph-mon[32990]: from='osd.6 [v2:172.21.15.96:6818/96197407,v1:172.21.15.96:6819/96197407]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-04-25T14:08:26.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:25 smithi151 ceph-mon[32990]: osdmap e42: 7 total, 6 up, 7 in 2024-04-25T14:08:26.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:25 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:08:26.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:25 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:08:26.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:25 smithi151 ceph-mon[32990]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:08:26.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:25 smithi151 ceph-mon[32990]: from='osd.6 [v2:172.21.15.96:6818/96197407,v1:172.21.15.96:6819/96197407]' entity='osd.6' 2024-04-25T14:08:26.200 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T14:08:26.200 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T14:08:26.200 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2024-04-25T14:08:26.200 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2024-04-25T14:08:26.200 INFO:teuthology.orchestra.run.smithi097.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112865 s, 929 MB/s 2024-04-25T14:08:26.200 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 2024-04-25T14:08:26.200 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2024-04-25T14:08:26.358 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:25 smithi096 ceph-mon[29207]: from='osd.6 [v2:172.21.15.96:6818/96197407,v1:172.21.15.96:6819/96197407]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-04-25T14:08:26.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:25 smithi096 ceph-mon[29207]: osdmap e42: 7 total, 6 up, 7 in 2024-04-25T14:08:26.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:25 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:08:26.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:25 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:08:26.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:25 smithi096 ceph-mon[29207]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:08:26.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:25 smithi096 ceph-mon[29207]: from='osd.6 [v2:172.21.15.96:6818/96197407,v1:172.21.15.96:6819/96197407]' entity='osd.6' 2024-04-25T14:08:26.839 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi097:vg_nvme/lv_2 2024-04-25T14:08:27.083 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T14:08:27.172 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:26 smithi097 ceph-mon[32720]: purged_snaps scrub starts 2024-04-25T14:08:27.172 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:26 smithi097 ceph-mon[32720]: purged_snaps scrub ok 2024-04-25T14:08:27.172 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:26 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:08:27.172 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:26 smithi097 ceph-mon[32720]: osd.6 [v2:172.21.15.96:6818/96197407,v1:172.21.15.96:6819/96197407] boot 2024-04-25T14:08:27.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:26 smithi097 ceph-mon[32720]: osdmap e43: 7 total, 7 up, 7 in 2024-04-25T14:08:27.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:26 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:08:27.187 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:26 smithi151 ceph-mon[32990]: purged_snaps scrub starts 2024-04-25T14:08:27.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:26 smithi151 ceph-mon[32990]: purged_snaps scrub ok 2024-04-25T14:08:27.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:26 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:08:27.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:26 smithi151 ceph-mon[32990]: osd.6 [v2:172.21.15.96:6818/96197407,v1:172.21.15.96:6819/96197407] boot 2024-04-25T14:08:27.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:26 smithi151 ceph-mon[32990]: osdmap e43: 7 total, 7 up, 7 in 2024-04-25T14:08:27.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:26 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:08:27.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:26 smithi096 ceph-mon[29207]: purged_snaps scrub starts 2024-04-25T14:08:27.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:26 smithi096 ceph-mon[29207]: purged_snaps scrub ok 2024-04-25T14:08:27.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:26 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:08:27.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:26 smithi096 ceph-mon[29207]: osd.6 [v2:172.21.15.96:6818/96197407,v1:172.21.15.96:6819/96197407] boot 2024-04-25T14:08:27.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:26 smithi096 ceph-mon[29207]: osdmap e43: 7 total, 7 up, 7 in 2024-04-25T14:08:27.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:26 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:08:28.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:28 smithi097 ceph-mon[32720]: Detected new or changed devices on smithi096 2024-04-25T14:08:28.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:28 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:28.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:28 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:28.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:28 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:08:28.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:28 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:08:28.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:28 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:08:28.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:28 smithi097 ceph-mon[32720]: Adjusting osd_memory_target on smithi096 to 5696M 2024-04-25T14:08:28.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:28 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:28.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:28 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:08:28.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:28 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:08:28.174 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:28 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:28.174 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:28 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:08:28.174 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:28 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:08:28.174 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:28 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:08:28.174 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:28 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:28.174 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:28 smithi097 ceph-mon[32720]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:08:28.174 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:28 smithi097 ceph-mon[32720]: osdmap e44: 7 total, 7 up, 7 in 2024-04-25T14:08:28.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:28 smithi151 ceph-mon[32990]: Detected new or changed devices on smithi096 2024-04-25T14:08:28.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:28 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:28.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:28 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:28.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:28 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:08:28.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:28 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:08:28.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:28 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:08:28.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:28 smithi151 ceph-mon[32990]: Adjusting osd_memory_target on smithi096 to 5696M 2024-04-25T14:08:28.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:28 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:28.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:28 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:08:28.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:28 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:08:28.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:28 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:28.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:28 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:08:28.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:28 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:08:28.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:28 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:08:28.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:28 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:28.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:28 smithi151 ceph-mon[32990]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:08:28.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:28 smithi151 ceph-mon[32990]: osdmap e44: 7 total, 7 up, 7 in 2024-04-25T14:08:28.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:28 smithi096 ceph-mon[29207]: Detected new or changed devices on smithi096 2024-04-25T14:08:28.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:28 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:28.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:28 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:28.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:28 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:08:28.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:28 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:08:28.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:28 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:08:28.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:28 smithi096 ceph-mon[29207]: Adjusting osd_memory_target on smithi096 to 5696M 2024-04-25T14:08:28.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:28 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:28.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:28 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:08:28.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:28 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:08:28.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:28 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:28.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:28 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:08:28.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:28 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:08:28.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:28 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:08:28.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:28 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:28.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:28 smithi096 ceph-mon[29207]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:08:28.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:28 smithi096 ceph-mon[29207]: osdmap e44: 7 total, 7 up, 7 in 2024-04-25T14:08:28.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:28.734+0000 7f8a90d75700 1 -- 172.21.15.97:0/581133468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a8c105340 msgr2=0x7f8a8c105750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:28.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:28.734+0000 7f8a90d75700 1 --2- 172.21.15.97:0/581133468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a8c105340 0x7f8a8c105750 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8a74009990 tx=0x7f8a74009ca0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:28.734+0000 7f8a90d75700 1 -- 172.21.15.97:0/581133468 shutdown_connections 2024-04-25T14:08:28.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:28.734+0000 7f8a90d75700 1 --2- 172.21.15.97:0/581133468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a8c105ce0 0x7f8a8c10c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:28.734+0000 7f8a90d75700 1 --2- 172.21.15.97:0/581133468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a8c104210 0x7f8a8c104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:28.734+0000 7f8a90d75700 1 --2- 172.21.15.97:0/581133468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a8c105340 0x7f8a8c105750 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:28.734+0000 7f8a90d75700 1 -- 172.21.15.97:0/581133468 >> 172.21.15.97:0/581133468 conn(0x7f8a8c0ff870 msgr2=0x7f8a8c101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:28.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:28.734+0000 7f8a90d75700 1 -- 172.21.15.97:0/581133468 shutdown_connections 2024-04-25T14:08:28.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:28.734+0000 7f8a90d75700 1 -- 172.21.15.97:0/581133468 wait complete. 2024-04-25T14:08:28.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:28.735+0000 7f8a90d75700 1 Processor -- start 2024-04-25T14:08:28.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:28.735+0000 7f8a90d75700 1 -- start start 2024-04-25T14:08:28.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:28.735+0000 7f8a90d75700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a8c104210 0x7f8a8c19a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:28.735+0000 7f8a90d75700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a8c105340 0x7f8a8c19acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:28.735+0000 7f8a90d75700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a8c105ce0 0x7f8a8c1a1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:28.735+0000 7f8a90d75700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8c10fc70 con 0x7f8a8c105340 2024-04-25T14:08:28.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:28.735+0000 7f8a90d75700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8c10faf0 con 0x7f8a8c104210 2024-04-25T14:08:28.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:28.735+0000 7f8a90d75700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8c10fdf0 con 0x7f8a8c105ce0 2024-04-25T14:08:28.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:28.735+0000 7f8a8a59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a8c104210 0x7f8a8c19a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:28.735+0000 7f8a8a59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a8c104210 0x7f8a8c19a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39004/0 (socket says 172.21.15.97:39004) 2024-04-25T14:08:28.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:28.735+0000 7f8a8a59c700 1 -- 172.21.15.97:0/1129596023 learned_addr learned my addr 172.21.15.97:0/1129596023 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T14:08:28.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:28.736+0000 7f8a81d9b700 1 --2- 172.21.15.97:0/1129596023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a8c105340 0x7f8a8c19acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:28.736+0000 7f8a8ad9d700 1 --2- 172.21.15.97:0/1129596023 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a8c105ce0 0x7f8a8c1a1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:28.736+0000 7f8a8a59c700 1 -- 172.21.15.97:0/1129596023 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a8c105ce0 msgr2=0x7f8a8c1a1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:28.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:28.736+0000 7f8a8a59c700 1 --2- 172.21.15.97:0/1129596023 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a8c105ce0 0x7f8a8c1a1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:28.736+0000 7f8a8a59c700 1 -- 172.21.15.97:0/1129596023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a8c105340 msgr2=0x7f8a8c19acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:28.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:28.736+0000 7f8a8a59c700 1 --2- 172.21.15.97:0/1129596023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a8c105340 0x7f8a8c19acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:28.736+0000 7f8a8a59c700 1 -- 172.21.15.97:0/1129596023 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a8c1a2250 con 0x7f8a8c104210 2024-04-25T14:08:28.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:28.736+0000 7f8a8a59c700 1 --2- 172.21.15.97:0/1129596023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a8c104210 0x7f8a8c19a780 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c00d950 tx=0x7f8a7c00dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:28.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:28.736+0000 7f8a83fff700 1 -- 172.21.15.97:0/1129596023 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c017070 con 0x7f8a8c104210 2024-04-25T14:08:28.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:28.736+0000 7f8a83fff700 1 -- 172.21.15.97:0/1129596023 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a7c00de30 con 0x7f8a8c104210 2024-04-25T14:08:28.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:28.736+0000 7f8a90d75700 1 -- 172.21.15.97:0/1129596023 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a8c1a2540 con 0x7f8a8c104210 2024-04-25T14:08:28.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:28.736+0000 7f8a83fff700 1 -- 172.21.15.97:0/1129596023 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c003b10 con 0x7f8a8c104210 2024-04-25T14:08:28.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:28.736+0000 7f8a90d75700 1 -- 172.21.15.97:0/1129596023 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a8c1a2a80 con 0x7f8a8c104210 2024-04-25T14:08:28.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:28.737+0000 7f8a90d75700 1 -- 172.21.15.97:0/1129596023 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a8c051190 con 0x7f8a8c104210 2024-04-25T14:08:28.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:28.739+0000 7f8a83fff700 1 -- 172.21.15.97:0/1129596023 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f8a7c003cb0 con 0x7f8a8c104210 2024-04-25T14:08:28.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:28.740+0000 7f8a83fff700 1 --2- 172.21.15.97:0/1129596023 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f8a780afb80 0x7f8a780b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:28.740+0000 7f8a83fff700 1 -- 172.21.15.97:0/1129596023 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4965+0+0 (secure 0 0 0) 0x7f8a7c0de310 con 0x7f8a8c104210 2024-04-25T14:08:28.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:28.740+0000 7f8a81d9b700 1 --2- 172.21.15.97:0/1129596023 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f8a780afb80 0x7f8a780b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:28.741+0000 7f8a81d9b700 1 --2- 172.21.15.97:0/1129596023 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f8a780afb80 0x7f8a780b2090 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8a740038a0 tx=0x7f8a7401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:28.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:28.744+0000 7f8a83fff700 1 -- 172.21.15.97:0/1129596023 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8a7c020030 con 0x7f8a8c104210 2024-04-25T14:08:28.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:28.952+0000 7f8a90d75700 1 -- 172.21.15.97:0/1129596023 --> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f8a8c063ad0 con 0x7f8a780afb80 2024-04-25T14:08:29.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:29 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:08:29.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:29 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:08:29.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:29 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:08:29.620 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:29 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:08:29.620 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:29 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:08:29.620 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:29 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:08:29.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:29 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:08:29.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:29 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:08:29.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:29 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:08:30.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:30 smithi096 ceph-mon[29207]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:08:30.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:30 smithi096 ceph-mon[29207]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:08:30.631 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:30 smithi097 ceph-mon[32720]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:08:30.631 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:30 smithi097 ceph-mon[32720]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:08:30.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:30 smithi151 ceph-mon[32990]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:08:30.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:30 smithi151 ceph-mon[32990]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:08:32.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:32 smithi097 ceph-mon[32720]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:08:32.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:32 smithi097 ceph-mon[32720]: from='client.? 172.21.15.97:0/3870231861' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "303551c8-6af0-42f2-844f-45bf8db5baa9"}]: dispatch 2024-04-25T14:08:32.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:32 smithi097 ceph-mon[32720]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "303551c8-6af0-42f2-844f-45bf8db5baa9"}]: dispatch 2024-04-25T14:08:32.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:32 smithi097 ceph-mon[32720]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "303551c8-6af0-42f2-844f-45bf8db5baa9"}]': finished 2024-04-25T14:08:32.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:32 smithi097 ceph-mon[32720]: osdmap e45: 8 total, 7 up, 8 in 2024-04-25T14:08:32.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:32 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:08:32.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:32 smithi097 ceph-mon[32720]: from='client.? 172.21.15.97:0/4210347553' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:08:32.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:32 smithi151 ceph-mon[32990]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:08:32.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:32 smithi151 ceph-mon[32990]: from='client.? 172.21.15.97:0/3870231861' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "303551c8-6af0-42f2-844f-45bf8db5baa9"}]: dispatch 2024-04-25T14:08:32.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:32 smithi151 ceph-mon[32990]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "303551c8-6af0-42f2-844f-45bf8db5baa9"}]: dispatch 2024-04-25T14:08:32.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:32 smithi151 ceph-mon[32990]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "303551c8-6af0-42f2-844f-45bf8db5baa9"}]': finished 2024-04-25T14:08:32.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:32 smithi151 ceph-mon[32990]: osdmap e45: 8 total, 7 up, 8 in 2024-04-25T14:08:32.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:32 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:08:32.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:32 smithi151 ceph-mon[32990]: from='client.? 172.21.15.97:0/4210347553' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:08:32.858 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:32 smithi096 ceph-mon[29207]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:08:32.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:32 smithi096 ceph-mon[29207]: from='client.? 172.21.15.97:0/3870231861' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "303551c8-6af0-42f2-844f-45bf8db5baa9"}]: dispatch 2024-04-25T14:08:32.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:32 smithi096 ceph-mon[29207]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "303551c8-6af0-42f2-844f-45bf8db5baa9"}]: dispatch 2024-04-25T14:08:32.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:32 smithi096 ceph-mon[29207]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "303551c8-6af0-42f2-844f-45bf8db5baa9"}]': finished 2024-04-25T14:08:32.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:32 smithi096 ceph-mon[29207]: osdmap e45: 8 total, 7 up, 8 in 2024-04-25T14:08:32.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:32 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:08:32.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:32 smithi096 ceph-mon[29207]: from='client.? 172.21.15.97:0/4210347553' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:08:34.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:34 smithi097 ceph-mon[32720]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:08:34.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:34 smithi151 ceph-mon[32990]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:08:34.858 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:34 smithi096 ceph-mon[29207]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:08:36.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:36 smithi097 ceph-mon[32720]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:08:36.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:36 smithi151 ceph-mon[32990]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:08:36.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:36 smithi096 ceph-mon[29207]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:08:38.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:38 smithi097 ceph-mon[32720]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:08:38.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:38 smithi151 ceph-mon[32990]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:08:38.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:38 smithi096 ceph-mon[29207]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:08:39.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:39 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T14:08:39.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:39 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:08:39.702 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:39 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T14:08:39.703 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:39 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:08:39.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:39 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T14:08:39.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:39 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:08:40.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:40 smithi151 ceph-mon[32990]: Deploying daemon osd.7 on smithi097 2024-04-25T14:08:40.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:40 smithi151 ceph-mon[32990]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:08:40.816 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:40 smithi097 ceph-mon[32720]: Deploying daemon osd.7 on smithi097 2024-04-25T14:08:40.817 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:40 smithi097 ceph-mon[32720]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:08:40.858 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:40 smithi096 ceph-mon[29207]: Deploying daemon osd.7 on smithi097 2024-04-25T14:08:40.858 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:40 smithi096 ceph-mon[29207]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:08:42.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:42 smithi151 ceph-mon[32990]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:08:42.773 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:42 smithi097 ceph-mon[32720]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:08:42.858 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:42 smithi096 ceph-mon[29207]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:08:44.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:44 smithi151 ceph-mon[32990]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:08:44.712 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:44 smithi097 ceph-mon[32720]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:08:44.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:44 smithi096 ceph-mon[29207]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:08:45.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:45 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:45.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:45 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:45.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:45 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:08:45.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:45 smithi097 ceph-mon[32720]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:08:46.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:45 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:46.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:45 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:46.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:45 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:08:46.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:45 smithi096 ceph-mon[29207]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:08:46.187 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:45 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:46.187 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:45 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:46.187 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:45 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:08:46.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:45 smithi151 ceph-mon[32990]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:08:48.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:48.472+0000 7f8a83fff700 1 -- 172.21.15.97:0/1129596023 <== mgr.14152 v2:172.21.15.96:6800/536332425 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8a8c063ad0 con 0x7f8a780afb80 2024-04-25T14:08:48.472 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 7 on host 'smithi097' 2024-04-25T14:08:48.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:48.474+0000 7f8a90d75700 1 -- 172.21.15.97:0/1129596023 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f8a780afb80 msgr2=0x7f8a780b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:48.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:48.474+0000 7f8a90d75700 1 --2- 172.21.15.97:0/1129596023 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f8a780afb80 0x7f8a780b2090 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8a740038a0 tx=0x7f8a7401a040 comp rx=0 tx=0).stop 2024-04-25T14:08:48.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:48.474+0000 7f8a90d75700 1 -- 172.21.15.97:0/1129596023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a8c104210 msgr2=0x7f8a8c19a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:48.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:48.474+0000 7f8a90d75700 1 --2- 172.21.15.97:0/1129596023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a8c104210 0x7f8a8c19a780 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c00d950 tx=0x7f8a7c00dd10 comp rx=0 tx=0).stop 2024-04-25T14:08:48.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:48.474+0000 7f8a90d75700 1 -- 172.21.15.97:0/1129596023 shutdown_connections 2024-04-25T14:08:48.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:48.474+0000 7f8a90d75700 1 --2- 172.21.15.97:0/1129596023 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f8a780afb80 0x7f8a780b2090 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:48.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:48.474+0000 7f8a90d75700 1 --2- 172.21.15.97:0/1129596023 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a8c105ce0 0x7f8a8c1a1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:48.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:48.474+0000 7f8a90d75700 1 --2- 172.21.15.97:0/1129596023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a8c104210 0x7f8a8c19a780 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:48.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:48.474+0000 7f8a90d75700 1 --2- 172.21.15.97:0/1129596023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a8c105340 0x7f8a8c19acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:48.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:48.474+0000 7f8a90d75700 1 -- 172.21.15.97:0/1129596023 >> 172.21.15.97:0/1129596023 conn(0x7f8a8c0ff870 msgr2=0x7f8a8c108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:48.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:48.474+0000 7f8a90d75700 1 -- 172.21.15.97:0/1129596023 shutdown_connections 2024-04-25T14:08:48.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:08:48.474+0000 7f8a90d75700 1 -- 172.21.15.97:0/1129596023 wait complete. 2024-04-25T14:08:48.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:48 smithi097 ceph-mon[32720]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:08:48.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:48 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:48.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:48 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:48.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:48 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:08:48.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:48 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:08:48.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:48 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:48.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:48 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:48.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:48 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:48.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:48 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:08:48.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:48 smithi096 ceph-mon[29207]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:08:48.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:48 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:48.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:48 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:48.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:48 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:08:48.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:48 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:08:48.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:48 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:48.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:48 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:48.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:48 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:48.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:48 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:08:48.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:48 smithi151 ceph-mon[32990]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:08:48.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:48 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:48.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:48 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:48.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:48 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:08:48.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:48 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:08:48.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:48 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:48.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:48 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:48.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:48 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:48.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:48 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:08:49.172 DEBUG:teuthology.orchestra.run.smithi097:osd.7> sudo journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@osd.7.service 2024-04-25T14:08:49.174 INFO:tasks.cephadm:Deploying osd.8 on smithi151 with /dev/vg_nvme/lv_2... 2024-04-25T14:08:49.175 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T14:08:49.422 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:08:49.770 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:49 smithi151 ceph-mon[32990]: from='osd.7 [v2:172.21.15.97:6816/2247216400,v1:172.21.15.97:6817/2247216400]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T14:08:49.770 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:49 smithi151 ceph-mon[32990]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T14:08:49.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:49 smithi096 ceph-mon[29207]: from='osd.7 [v2:172.21.15.97:6816/2247216400,v1:172.21.15.97:6817/2247216400]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T14:08:49.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:49 smithi096 ceph-mon[29207]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T14:08:49.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:49 smithi097 ceph-mon[32720]: from='osd.7 [v2:172.21.15.97:6816/2247216400,v1:172.21.15.97:6817/2247216400]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T14:08:49.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:49 smithi097 ceph-mon[32720]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T14:08:50.600 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:50 smithi097 ceph-mon[32720]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:08:50.600 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:50 smithi097 ceph-mon[32720]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T14:08:50.600 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:50 smithi097 ceph-mon[32720]: from='osd.7 [v2:172.21.15.97:6816/2247216400,v1:172.21.15.97:6817/2247216400]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T14:08:50.600 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:50 smithi097 ceph-mon[32720]: osdmap e46: 8 total, 7 up, 8 in 2024-04-25T14:08:50.600 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:50 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:08:50.600 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:50 smithi097 ceph-mon[32720]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T14:08:50.802 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:50 smithi151 ceph-mon[32990]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:08:50.802 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:50 smithi151 ceph-mon[32990]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T14:08:50.802 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:50 smithi151 ceph-mon[32990]: from='osd.7 [v2:172.21.15.97:6816/2247216400,v1:172.21.15.97:6817/2247216400]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T14:08:50.802 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:50 smithi151 ceph-mon[32990]: osdmap e46: 8 total, 7 up, 8 in 2024-04-25T14:08:50.802 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:50 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:08:50.802 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:50 smithi151 ceph-mon[32990]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T14:08:50.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:50 smithi096 ceph-mon[29207]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:08:50.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:50 smithi096 ceph-mon[29207]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T14:08:50.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:50 smithi096 ceph-mon[29207]: from='osd.7 [v2:172.21.15.97:6816/2247216400,v1:172.21.15.97:6817/2247216400]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T14:08:50.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:50 smithi096 ceph-mon[29207]: osdmap e46: 8 total, 7 up, 8 in 2024-04-25T14:08:50.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:50 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:08:50.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:50 smithi096 ceph-mon[29207]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T14:08:51.401 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:08:50 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:08:50.963+0000 7fe1aae65700 -1 osd.7 0 waiting for initial osdmap 2024-04-25T14:08:51.401 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:08:50 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:08:50.974+0000 7fe1a2c12700 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:08:51.526 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T14:08:51.527 INFO:teuthology.orchestra.run.smithi151.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T14:08:51.527 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10+0 records in 2024-04-25T14:08:51.527 INFO:teuthology.orchestra.run.smithi151.stderr:10+0 records out 2024-04-25T14:08:51.527 INFO:teuthology.orchestra.run.smithi151.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114042 s, 919 MB/s 2024-04-25T14:08:51.527 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping successful for: 2024-04-25T14:08:51.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:51 smithi097 ceph-mon[32720]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-25T14:08:51.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:51 smithi097 ceph-mon[32720]: osdmap e47: 8 total, 7 up, 8 in 2024-04-25T14:08:51.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:51 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:08:51.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:51 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:08:51.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:51 smithi097 ceph-mon[32720]: from='osd.7 ' entity='osd.7' 2024-04-25T14:08:51.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:51 smithi097 ceph-mon[32720]: osd.7 [v2:172.21.15.97:6816/2247216400,v1:172.21.15.97:6817/2247216400] boot 2024-04-25T14:08:51.831 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:51 smithi151 ceph-mon[32990]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-25T14:08:51.831 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:51 smithi151 ceph-mon[32990]: osdmap e47: 8 total, 7 up, 8 in 2024-04-25T14:08:51.831 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:51 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:08:51.831 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:51 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:08:51.831 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:51 smithi151 ceph-mon[32990]: from='osd.7 ' entity='osd.7' 2024-04-25T14:08:51.831 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:51 smithi151 ceph-mon[32990]: osd.7 [v2:172.21.15.97:6816/2247216400,v1:172.21.15.97:6817/2247216400] boot 2024-04-25T14:08:51.858 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:51 smithi096 ceph-mon[29207]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-25T14:08:51.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:51 smithi096 ceph-mon[29207]: osdmap e47: 8 total, 7 up, 8 in 2024-04-25T14:08:51.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:51 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:08:51.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:51 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:08:51.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:51 smithi096 ceph-mon[29207]: from='osd.7 ' entity='osd.7' 2024-04-25T14:08:51.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:51 smithi096 ceph-mon[29207]: osd.7 [v2:172.21.15.97:6816/2247216400,v1:172.21.15.97:6817/2247216400] boot 2024-04-25T14:08:52.183 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi151:vg_nvme/lv_2 2024-04-25T14:08:52.432 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:08:52.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:52 smithi097 ceph-mon[32720]: purged_snaps scrub starts 2024-04-25T14:08:52.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:52 smithi097 ceph-mon[32720]: purged_snaps scrub ok 2024-04-25T14:08:52.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:52 smithi097 ceph-mon[32720]: pgmap v172: 1 pgs: 1 unknown; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:08:52.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:52 smithi097 ceph-mon[32720]: osdmap e48: 8 total, 8 up, 8 in 2024-04-25T14:08:52.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:52 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:08:52.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:52 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:52.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:52 smithi097 ceph-mon[32720]: osdmap e49: 8 total, 8 up, 8 in 2024-04-25T14:08:52.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:52 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:52.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:52 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:08:52.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:52 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:08:52.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:52 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:08:52.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:52 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:52.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:52 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:08:52.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:52 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:08:52.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:52 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:52.816 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:52 smithi151 ceph-mon[32990]: purged_snaps scrub starts 2024-04-25T14:08:52.816 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:52 smithi151 ceph-mon[32990]: purged_snaps scrub ok 2024-04-25T14:08:52.816 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:52 smithi151 ceph-mon[32990]: pgmap v172: 1 pgs: 1 unknown; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:08:52.816 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:52 smithi151 ceph-mon[32990]: osdmap e48: 8 total, 8 up, 8 in 2024-04-25T14:08:52.816 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:52 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:08:52.816 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:52 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:52.816 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:52 smithi151 ceph-mon[32990]: osdmap e49: 8 total, 8 up, 8 in 2024-04-25T14:08:52.816 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:52 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:52.817 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:52 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:08:52.817 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:52 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:08:52.817 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:52 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:08:52.817 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:52 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:52.817 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:52 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:08:52.817 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:52 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:08:52.817 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:52 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:52.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:52 smithi096 ceph-mon[29207]: purged_snaps scrub starts 2024-04-25T14:08:52.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:52 smithi096 ceph-mon[29207]: purged_snaps scrub ok 2024-04-25T14:08:52.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:52 smithi096 ceph-mon[29207]: pgmap v172: 1 pgs: 1 unknown; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:08:52.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:52 smithi096 ceph-mon[29207]: osdmap e48: 8 total, 8 up, 8 in 2024-04-25T14:08:52.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:52 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:08:52.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:52 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:52.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:52 smithi096 ceph-mon[29207]: osdmap e49: 8 total, 8 up, 8 in 2024-04-25T14:08:52.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:52 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:52.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:52 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:08:52.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:52 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:08:52.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:52 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:08:52.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:52 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:52.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:52 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:08:52.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:52 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:08:52.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:52 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:08:53.821 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:53 smithi151 ceph-mon[32990]: Detected new or changed devices on smithi097 2024-04-25T14:08:53.821 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:53 smithi151 ceph-mon[32990]: Adjusting osd_memory_target on smithi097 to 5696M 2024-04-25T14:08:53.858 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:53 smithi096 ceph-mon[29207]: Detected new or changed devices on smithi097 2024-04-25T14:08:53.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:53 smithi096 ceph-mon[29207]: Adjusting osd_memory_target on smithi097 to 5696M 2024-04-25T14:08:53.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:53 smithi097 ceph-mon[32720]: Detected new or changed devices on smithi097 2024-04-25T14:08:53.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:53 smithi097 ceph-mon[32720]: Adjusting osd_memory_target on smithi097 to 5696M 2024-04-25T14:08:54.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:08:54.064+0000 7f921342b700 1 -- 172.21.15.151:0/3118255146 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f920c107330 msgr2=0x7f920c1097f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:54.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:08:54.064+0000 7f921342b700 1 --2- 172.21.15.151:0/3118255146 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f920c107330 0x7f920c1097f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f91fc009990 tx=0x7f91fc009ca0 comp rx=0 tx=0).stop 2024-04-25T14:08:54.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:08:54.064+0000 7f921342b700 1 -- 172.21.15.151:0/3118255146 shutdown_connections 2024-04-25T14:08:54.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:08:54.064+0000 7f921342b700 1 --2- 172.21.15.151:0/3118255146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f920c109e80 0x7f920c10c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:54.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:08:54.064+0000 7f921342b700 1 --2- 172.21.15.151:0/3118255146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f920c06cbb0 0x7f920c106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:54.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:08:54.064+0000 7f921342b700 1 --2- 172.21.15.151:0/3118255146 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f920c107330 0x7f920c1097f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:54.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:08:54.064+0000 7f921342b700 1 -- 172.21.15.151:0/3118255146 >> 172.21.15.151:0/3118255146 conn(0x7f920c0fe710 msgr2=0x7f920c100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:54.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:08:54.064+0000 7f921342b700 1 -- 172.21.15.151:0/3118255146 shutdown_connections 2024-04-25T14:08:54.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:08:54.064+0000 7f921342b700 1 -- 172.21.15.151:0/3118255146 wait complete. 2024-04-25T14:08:54.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:08:54.065+0000 7f921342b700 1 Processor -- start 2024-04-25T14:08:54.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:08:54.065+0000 7f921342b700 1 -- start start 2024-04-25T14:08:54.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:08:54.065+0000 7f921342b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f920c06cbb0 0x7f920c1045b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:54.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:08:54.065+0000 7f921342b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f920c107330 0x7f920c102c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:54.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:08:54.065+0000 7f921342b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f920c109e80 0x7f920c1031a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:54.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:08:54.065+0000 7f921342b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f920c10fc40 con 0x7f920c06cbb0 2024-04-25T14:08:54.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:08:54.065+0000 7f921342b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f920c10fac0 con 0x7f920c109e80 2024-04-25T14:08:54.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:08:54.065+0000 7f921342b700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f920c10fdc0 con 0x7f920c107330 2024-04-25T14:08:54.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:08:54.065+0000 7f92109c6700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f920c107330 0x7f920c102c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:54.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:08:54.065+0000 7f92109c6700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f920c107330 0x7f920c102c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54678/0 (socket says 172.21.15.151:54678) 2024-04-25T14:08:54.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:08:54.065+0000 7f92109c6700 1 -- 172.21.15.151:0/3653247355 learned_addr learned my addr 172.21.15.151:0/3653247355 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-25T14:08:54.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:08:54.065+0000 7f92111c7700 1 --2- 172.21.15.151:0/3653247355 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f920c06cbb0 0x7f920c1045b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:54.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:08:54.065+0000 7f92119c8700 1 --2- 172.21.15.151:0/3653247355 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f920c109e80 0x7f920c1031a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:54.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:08:54.065+0000 7f92109c6700 1 -- 172.21.15.151:0/3653247355 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f920c109e80 msgr2=0x7f920c1031a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:54.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:08:54.065+0000 7f92109c6700 1 --2- 172.21.15.151:0/3653247355 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f920c109e80 0x7f920c1031a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:54.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:08:54.065+0000 7f92109c6700 1 -- 172.21.15.151:0/3653247355 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f920c06cbb0 msgr2=0x7f920c1045b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:54.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:08:54.065+0000 7f92109c6700 1 --2- 172.21.15.151:0/3653247355 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f920c06cbb0 0x7f920c1045b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:54.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:08:54.066+0000 7f92109c6700 1 -- 172.21.15.151:0/3653247355 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f920c103900 con 0x7f920c107330 2024-04-25T14:08:54.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:08:54.066+0000 7f92119c8700 1 --2- 172.21.15.151:0/3653247355 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f920c109e80 0x7f920c1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:08:54.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:08:54.066+0000 7f92109c6700 1 --2- 172.21.15.151:0/3653247355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f920c107330 0x7f920c102c60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f91fc006010 tx=0x7f91fc0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:54.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:08:54.066+0000 7f92027fc700 1 -- 172.21.15.151:0/3653247355 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91fc027070 con 0x7f920c107330 2024-04-25T14:08:54.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:08:54.066+0000 7f921342b700 1 -- 172.21.15.151:0/3653247355 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f920c1a4880 con 0x7f920c107330 2024-04-25T14:08:54.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:08:54.066+0000 7f92027fc700 1 -- 172.21.15.151:0/3653247355 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91fc0040c0 con 0x7f920c107330 2024-04-25T14:08:54.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:08:54.066+0000 7f92027fc700 1 -- 172.21.15.151:0/3653247355 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91fc022d90 con 0x7f920c107330 2024-04-25T14:08:54.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:08:54.066+0000 7f921342b700 1 -- 172.21.15.151:0/3653247355 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f920c1a4c70 con 0x7f920c107330 2024-04-25T14:08:54.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:08:54.067+0000 7f921342b700 1 -- 172.21.15.151:0/3653247355 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f920c051190 con 0x7f920c107330 2024-04-25T14:08:54.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:08:54.068+0000 7f92027fc700 1 -- 172.21.15.151:0/3653247355 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f91fc02a070 con 0x7f920c107330 2024-04-25T14:08:54.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:08:54.069+0000 7f92027fc700 1 --2- 172.21.15.151:0/3653247355 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f91f80afb80 0x7f91f80b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:54.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:08:54.069+0000 7f92027fc700 1 -- 172.21.15.151:0/3653247355 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5397+0+0 (secure 0 0 0) 0x7f91fc0df220 con 0x7f920c107330 2024-04-25T14:08:54.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:08:54.069+0000 7f92111c7700 1 --2- 172.21.15.151:0/3653247355 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f91f80afb80 0x7f91f80b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:54.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:08:54.070+0000 7f92111c7700 1 --2- 172.21.15.151:0/3653247355 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f91f80afb80 0x7f91f80b2090 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9208009550 tx=0x7f92080091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:54.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:08:54.073+0000 7f92027fc700 1 -- 172.21.15.151:0/3653247355 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f91fc013070 con 0x7f920c107330 2024-04-25T14:08:54.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:08:54.282+0000 7f921342b700 1 -- 172.21.15.151:0/3653247355 --> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f920c02e8f0 con 0x7f91f80afb80 2024-04-25T14:08:54.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:54 smithi096 ceph-mon[29207]: pgmap v175: 1 pgs: 1 unknown; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:08:54.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:54 smithi096 ceph-mon[29207]: osdmap e50: 8 total, 8 up, 8 in 2024-04-25T14:08:54.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:54 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:08:54.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:54 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:08:54.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:54 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:08:54.891 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:54 smithi151 ceph-mon[32990]: pgmap v175: 1 pgs: 1 unknown; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:08:54.891 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:54 smithi151 ceph-mon[32990]: osdmap e50: 8 total, 8 up, 8 in 2024-04-25T14:08:54.891 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:54 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:08:54.891 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:54 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:08:54.891 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:54 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:08:54.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:54 smithi097 ceph-mon[32720]: pgmap v175: 1 pgs: 1 unknown; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:08:54.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:54 smithi097 ceph-mon[32720]: osdmap e50: 8 total, 8 up, 8 in 2024-04-25T14:08:54.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:54 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:08:54.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:54 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:08:54.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:54 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:08:55.806 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:55 smithi151 ceph-mon[32990]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:08:55.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:55 smithi096 ceph-mon[29207]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:08:55.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:55 smithi097 ceph-mon[32720]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:08:56.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:56 smithi096 ceph-mon[29207]: pgmap v177: 1 pgs: 1 unknown; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:08:56.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:56 smithi097 ceph-mon[32720]: pgmap v177: 1 pgs: 1 unknown; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:08:56.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:56 smithi151 ceph-mon[32990]: pgmap v177: 1 pgs: 1 unknown; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:08:57.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:57 smithi151 ceph-mon[32990]: from='client.? 172.21.15.151:0/3038351754' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "81f6cc66-e4ca-430c-b336-b6678d968818"}]: dispatch 2024-04-25T14:08:57.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:57 smithi151 ceph-mon[32990]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "81f6cc66-e4ca-430c-b336-b6678d968818"}]: dispatch 2024-04-25T14:08:57.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:57 smithi151 ceph-mon[32990]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "81f6cc66-e4ca-430c-b336-b6678d968818"}]': finished 2024-04-25T14:08:57.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:57 smithi151 ceph-mon[32990]: osdmap e51: 9 total, 8 up, 9 in 2024-04-25T14:08:57.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:57 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:08:57.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:57 smithi096 ceph-mon[29207]: from='client.? 172.21.15.151:0/3038351754' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "81f6cc66-e4ca-430c-b336-b6678d968818"}]: dispatch 2024-04-25T14:08:57.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:57 smithi096 ceph-mon[29207]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "81f6cc66-e4ca-430c-b336-b6678d968818"}]: dispatch 2024-04-25T14:08:57.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:57 smithi096 ceph-mon[29207]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "81f6cc66-e4ca-430c-b336-b6678d968818"}]': finished 2024-04-25T14:08:57.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:57 smithi096 ceph-mon[29207]: osdmap e51: 9 total, 8 up, 9 in 2024-04-25T14:08:57.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:57 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:08:57.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:57 smithi097 ceph-mon[32720]: from='client.? 172.21.15.151:0/3038351754' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "81f6cc66-e4ca-430c-b336-b6678d968818"}]: dispatch 2024-04-25T14:08:57.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:57 smithi097 ceph-mon[32720]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "81f6cc66-e4ca-430c-b336-b6678d968818"}]: dispatch 2024-04-25T14:08:57.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:57 smithi097 ceph-mon[32720]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "81f6cc66-e4ca-430c-b336-b6678d968818"}]': finished 2024-04-25T14:08:57.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:57 smithi097 ceph-mon[32720]: osdmap e51: 9 total, 8 up, 9 in 2024-04-25T14:08:57.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:57 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:08:58.858 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:58 smithi096 ceph-mon[29207]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 197 KiB/s, 0 objects/s recovering 2024-04-25T14:08:58.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:58 smithi096 ceph-mon[29207]: from='client.? 172.21.15.151:0/1950495675' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:08:58.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:58 smithi097 ceph-mon[32720]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 197 KiB/s, 0 objects/s recovering 2024-04-25T14:08:58.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:58 smithi097 ceph-mon[32720]: from='client.? 172.21.15.151:0/1950495675' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:08:58.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:58 smithi151 ceph-mon[32990]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 197 KiB/s, 0 objects/s recovering 2024-04-25T14:08:58.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:58 smithi151 ceph-mon[32990]: from='client.? 172.21.15.151:0/1950495675' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:08:59.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:08:59 smithi096 ceph-mon[29207]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 167 KiB/s, 0 objects/s recovering 2024-04-25T14:08:59.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:08:59 smithi097 ceph-mon[32720]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 167 KiB/s, 0 objects/s recovering 2024-04-25T14:08:59.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:08:59 smithi151 ceph-mon[32990]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 167 KiB/s, 0 objects/s recovering 2024-04-25T14:09:02.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:02 smithi151 ceph-mon[32990]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T14:09:02.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:02 smithi097 ceph-mon[32720]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T14:09:02.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:02 smithi096 ceph-mon[29207]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T14:09:04.437 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:04 smithi151 ceph-mon[32990]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-04-25T14:09:04.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:04 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T14:09:04.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:04 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:09:04.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:04 smithi097 ceph-mon[32720]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-04-25T14:09:04.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:04 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T14:09:04.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:04 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:09:04.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:04 smithi096 ceph-mon[29207]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-04-25T14:09:04.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:04 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T14:09:04.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:04 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:09:05.581 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:05 smithi151 ceph-mon[32990]: Deploying daemon osd.8 on smithi151 2024-04-25T14:09:05.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:05 smithi096 ceph-mon[29207]: Deploying daemon osd.8 on smithi151 2024-04-25T14:09:05.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:05 smithi097 ceph-mon[32720]: Deploying daemon osd.8 on smithi151 2024-04-25T14:09:06.677 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:06 smithi151 ceph-mon[32990]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-25T14:09:06.858 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:06 smithi096 ceph-mon[29207]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-25T14:09:06.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:06 smithi097 ceph-mon[32720]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-25T14:09:08.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:08 smithi151 ceph-mon[32990]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:09:08.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:08 smithi096 ceph-mon[29207]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:09:08.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:08 smithi097 ceph-mon[32720]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:09:10.689 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:10 smithi151 ceph-mon[32990]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:09:10.689 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:10 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:10.689 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:10 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:10.689 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:10 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:09:10.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:10 smithi096 ceph-mon[29207]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:09:10.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:10 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:10.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:10 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:10.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:10 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:09:10.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:10 smithi097 ceph-mon[32720]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:09:10.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:10 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:10.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:10 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:10.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:10 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:09:12.630 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:12 smithi151 ceph-mon[32990]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:09:12.858 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:12 smithi096 ceph-mon[29207]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:09:12.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:12 smithi097 ceph-mon[32720]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:09:13.911 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:13 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:13.911 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:13 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:13.911 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:13 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:09:13.911 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:13 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:09:13.911 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:13 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:13.912 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:13 smithi151 ceph-mon[32990]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:09:14.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:09:14.002+0000 7f92027fc700 1 -- 172.21.15.151:0/3653247355 <== mgr.14152 v2:172.21.15.96:6800/536332425 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f920c02e8f0 con 0x7f91f80afb80 2024-04-25T14:09:14.002 INFO:teuthology.orchestra.run.smithi151.stdout:Created osd(s) 8 on host 'smithi151' 2024-04-25T14:09:14.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:09:14.004+0000 7f921342b700 1 -- 172.21.15.151:0/3653247355 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f91f80afb80 msgr2=0x7f91f80b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:14.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:09:14.004+0000 7f921342b700 1 --2- 172.21.15.151:0/3653247355 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f91f80afb80 0x7f91f80b2090 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9208009550 tx=0x7f92080091b0 comp rx=0 tx=0).stop 2024-04-25T14:09:14.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:09:14.005+0000 7f921342b700 1 -- 172.21.15.151:0/3653247355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f920c107330 msgr2=0x7f920c102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:14.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:09:14.005+0000 7f921342b700 1 --2- 172.21.15.151:0/3653247355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f920c107330 0x7f920c102c60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f91fc006010 tx=0x7f91fc0047b0 comp rx=0 tx=0).stop 2024-04-25T14:09:14.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:09:14.005+0000 7f921342b700 1 -- 172.21.15.151:0/3653247355 shutdown_connections 2024-04-25T14:09:14.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:09:14.005+0000 7f921342b700 1 --2- 172.21.15.151:0/3653247355 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f91f80afb80 0x7f91f80b2090 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:14.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:09:14.005+0000 7f921342b700 1 --2- 172.21.15.151:0/3653247355 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f920c109e80 0x7f920c1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:14.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:09:14.005+0000 7f921342b700 1 --2- 172.21.15.151:0/3653247355 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f920c06cbb0 0x7f920c1045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:14.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:09:14.005+0000 7f921342b700 1 --2- 172.21.15.151:0/3653247355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f920c107330 0x7f920c102c60 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:14.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:09:14.005+0000 7f921342b700 1 -- 172.21.15.151:0/3653247355 >> 172.21.15.151:0/3653247355 conn(0x7f920c0fe710 msgr2=0x7f920c1054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:14.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:09:14.005+0000 7f921342b700 1 -- 172.21.15.151:0/3653247355 shutdown_connections 2024-04-25T14:09:14.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:09:14.005+0000 7f921342b700 1 -- 172.21.15.151:0/3653247355 wait complete. 2024-04-25T14:09:14.108 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:13 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:14.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:13 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:14.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:13 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:09:14.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:13 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:09:14.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:13 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:14.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:13 smithi096 ceph-mon[29207]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:09:14.172 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:13 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:14.172 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:13 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:14.172 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:13 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:09:14.172 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:13 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:09:14.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:13 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:14.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:13 smithi097 ceph-mon[32720]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:09:14.654 DEBUG:teuthology.orchestra.run.smithi151:osd.8> sudo journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@osd.8.service 2024-04-25T14:09:14.656 INFO:tasks.cephadm:Deploying osd.9 on smithi096 with /dev/vg_nvme/lv_1... 2024-04-25T14:09:14.657 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T14:09:14.903 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:09:15.172 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:14 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:15.172 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:14 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:15.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:14 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:09:15.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:14 smithi097 ceph-mon[32720]: from='osd.8 [v2:172.21.15.151:6816/2549822185,v1:172.21.15.151:6817/2549822185]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T14:09:15.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:14 smithi097 ceph-mon[32720]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T14:09:15.187 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:14 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:15.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:14 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:15.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:14 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:09:15.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:14 smithi151 ceph-mon[32990]: from='osd.8 [v2:172.21.15.151:6816/2549822185,v1:172.21.15.151:6817/2549822185]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T14:09:15.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:14 smithi151 ceph-mon[32990]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T14:09:15.234 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:14 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:15.234 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:14 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:15.234 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:14 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:09:15.234 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:14 smithi096 ceph-mon[29207]: from='osd.8 [v2:172.21.15.151:6816/2549822185,v1:172.21.15.151:6817/2549822185]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T14:09:15.234 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:14 smithi096 ceph-mon[29207]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T14:09:16.182 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:15 smithi151 ceph-mon[32990]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T14:09:16.182 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:15 smithi151 ceph-mon[32990]: osdmap e52: 9 total, 8 up, 9 in 2024-04-25T14:09:16.183 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:15 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:09:16.183 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:15 smithi151 ceph-mon[32990]: from='osd.8 [v2:172.21.15.151:6816/2549822185,v1:172.21.15.151:6817/2549822185]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-04-25T14:09:16.183 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:15 smithi151 ceph-mon[32990]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-04-25T14:09:16.183 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:15 smithi151 ceph-mon[32990]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:09:16.281 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:15 smithi096 ceph-mon[29207]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T14:09:16.281 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:15 smithi096 ceph-mon[29207]: osdmap e52: 9 total, 8 up, 9 in 2024-04-25T14:09:16.281 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:15 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:09:16.281 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:15 smithi096 ceph-mon[29207]: from='osd.8 [v2:172.21.15.151:6816/2549822185,v1:172.21.15.151:6817/2549822185]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-04-25T14:09:16.281 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:15 smithi096 ceph-mon[29207]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-04-25T14:09:16.281 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:15 smithi096 ceph-mon[29207]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:09:16.422 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:15 smithi097 ceph-mon[32720]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T14:09:16.422 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:15 smithi097 ceph-mon[32720]: osdmap e52: 9 total, 8 up, 9 in 2024-04-25T14:09:16.422 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:15 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:09:16.422 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:15 smithi097 ceph-mon[32720]: from='osd.8 [v2:172.21.15.151:6816/2549822185,v1:172.21.15.151:6817/2549822185]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-04-25T14:09:16.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:15 smithi097 ceph-mon[32720]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-04-25T14:09:16.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:15 smithi097 ceph-mon[32720]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:09:16.877 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:09:16 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:09:16.458+0000 7f81ac15d700 -1 osd.8 0 waiting for initial osdmap 2024-04-25T14:09:16.877 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:09:16 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:09:16.471+0000 7f81a3f0a700 -1 osd.8 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:09:17.179 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:16 smithi096 ceph-mon[29207]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-04-25T14:09:17.179 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:16 smithi096 ceph-mon[29207]: osdmap e53: 9 total, 8 up, 9 in 2024-04-25T14:09:17.179 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:16 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:09:17.179 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:16 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:09:17.179 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:16 smithi096 ceph-mon[29207]: from='osd.8 ' entity='osd.8' 2024-04-25T14:09:17.180 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T14:09:17.180 INFO:teuthology.orchestra.run.smithi096.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T14:09:17.180 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10+0 records in 2024-04-25T14:09:17.180 INFO:teuthology.orchestra.run.smithi096.stderr:10+0 records out 2024-04-25T14:09:17.180 INFO:teuthology.orchestra.run.smithi096.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0134708 s, 778 MB/s 2024-04-25T14:09:17.180 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping successful for: 2024-04-25T14:09:17.187 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:16 smithi151 ceph-mon[32990]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-04-25T14:09:17.187 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:16 smithi151 ceph-mon[32990]: osdmap e53: 9 total, 8 up, 9 in 2024-04-25T14:09:17.187 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:16 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:09:17.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:16 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:09:17.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:16 smithi151 ceph-mon[32990]: from='osd.8 ' entity='osd.8' 2024-04-25T14:09:17.422 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:16 smithi097 ceph-mon[32720]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-04-25T14:09:17.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:16 smithi097 ceph-mon[32720]: osdmap e53: 9 total, 8 up, 9 in 2024-04-25T14:09:17.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:09:17.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:09:17.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:16 smithi097 ceph-mon[32720]: from='osd.8 ' entity='osd.8' 2024-04-25T14:09:18.027 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi096:vg_nvme/lv_1 2024-04-25T14:09:18.275 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:09:18.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:17 smithi096 ceph-mon[29207]: purged_snaps scrub starts 2024-04-25T14:09:18.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:17 smithi096 ceph-mon[29207]: purged_snaps scrub ok 2024-04-25T14:09:18.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:17 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:09:18.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:17 smithi096 ceph-mon[29207]: osd.8 [v2:172.21.15.151:6816/2549822185,v1:172.21.15.151:6817/2549822185] boot 2024-04-25T14:09:18.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:17 smithi096 ceph-mon[29207]: osdmap e54: 9 total, 9 up, 9 in 2024-04-25T14:09:18.361 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:17 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:09:18.361 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:17 smithi096 ceph-mon[29207]: pgmap v192: 1 pgs: 1 peering; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:09:18.361 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:17 smithi096 ceph-mon[29207]: osdmap e55: 9 total, 9 up, 9 in 2024-04-25T14:09:18.361 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:17 smithi096 ceph-mon[29207]: Detected new or changed devices on smithi151 2024-04-25T14:09:18.361 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:17 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:18.361 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:17 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:18.361 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:17 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:09:18.361 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:17 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:09:18.361 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:17 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:09:18.361 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:17 smithi096 ceph-mon[29207]: Adjusting osd_memory_target on smithi151 to 5696M 2024-04-25T14:09:18.361 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:17 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:18.361 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:17 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:09:18.361 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:17 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:09:18.361 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:17 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:18.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:17 smithi097 ceph-mon[32720]: purged_snaps scrub starts 2024-04-25T14:09:18.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:17 smithi097 ceph-mon[32720]: purged_snaps scrub ok 2024-04-25T14:09:18.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:17 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:09:18.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:17 smithi097 ceph-mon[32720]: osd.8 [v2:172.21.15.151:6816/2549822185,v1:172.21.15.151:6817/2549822185] boot 2024-04-25T14:09:18.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:17 smithi097 ceph-mon[32720]: osdmap e54: 9 total, 9 up, 9 in 2024-04-25T14:09:18.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:17 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:09:18.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:17 smithi097 ceph-mon[32720]: pgmap v192: 1 pgs: 1 peering; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:09:18.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:17 smithi097 ceph-mon[32720]: osdmap e55: 9 total, 9 up, 9 in 2024-04-25T14:09:18.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:17 smithi097 ceph-mon[32720]: Detected new or changed devices on smithi151 2024-04-25T14:09:18.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:17 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:18.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:17 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:18.424 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:17 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:09:18.424 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:17 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:09:18.424 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:17 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:09:18.424 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:17 smithi097 ceph-mon[32720]: Adjusting osd_memory_target on smithi151 to 5696M 2024-04-25T14:09:18.424 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:17 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:18.424 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:17 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:09:18.424 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:17 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:09:18.424 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:17 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:18.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:17 smithi151 ceph-mon[32990]: purged_snaps scrub starts 2024-04-25T14:09:18.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:17 smithi151 ceph-mon[32990]: purged_snaps scrub ok 2024-04-25T14:09:18.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:17 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:09:18.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:17 smithi151 ceph-mon[32990]: osd.8 [v2:172.21.15.151:6816/2549822185,v1:172.21.15.151:6817/2549822185] boot 2024-04-25T14:09:18.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:17 smithi151 ceph-mon[32990]: osdmap e54: 9 total, 9 up, 9 in 2024-04-25T14:09:18.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:17 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:09:18.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:17 smithi151 ceph-mon[32990]: pgmap v192: 1 pgs: 1 peering; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:09:18.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:17 smithi151 ceph-mon[32990]: osdmap e55: 9 total, 9 up, 9 in 2024-04-25T14:09:18.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:17 smithi151 ceph-mon[32990]: Detected new or changed devices on smithi151 2024-04-25T14:09:18.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:17 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:18.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:17 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:18.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:17 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:09:18.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:17 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:09:18.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:17 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:09:18.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:17 smithi151 ceph-mon[32990]: Adjusting osd_memory_target on smithi151 to 5696M 2024-04-25T14:09:18.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:17 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:18.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:17 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:09:18.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:17 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:09:18.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:17 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:19.773 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:19 smithi096 ceph-mon[29207]: osdmap e56: 9 total, 9 up, 9 in 2024-04-25T14:09:19.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:19 smithi097 ceph-mon[32720]: osdmap e56: 9 total, 9 up, 9 in 2024-04-25T14:09:19.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:19 smithi151 ceph-mon[32990]: osdmap e56: 9 total, 9 up, 9 in 2024-04-25T14:09:19.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:19.972+0000 7f036afa0700 1 -- 172.21.15.96:0/2573158717 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0364104910 msgr2=0x7f036410b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:19.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:19.972+0000 7f036afa0700 1 --2- 172.21.15.96:0/2573158717 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0364104910 0x7f036410b2a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f035c00b0a0 tx=0x7f035c00b3b0 comp rx=0 tx=0).stop 2024-04-25T14:09:19.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:19.973+0000 7f036afa0700 1 -- 172.21.15.96:0/2573158717 shutdown_connections 2024-04-25T14:09:19.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:19.973+0000 7f036afa0700 1 --2- 172.21.15.96:0/2573158717 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0364104910 0x7f036410b2a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:19.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:19.973+0000 7f036afa0700 1 --2- 172.21.15.96:0/2573158717 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0364102e70 0x7f03641032a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:19.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:19.973+0000 7f036afa0700 1 --2- 172.21.15.96:0/2573158717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0364103fc0 0x7f03641043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:19.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:19.973+0000 7f036afa0700 1 -- 172.21.15.96:0/2573158717 >> 172.21.15.96:0/2573158717 conn(0x7f03640fe570 msgr2=0x7f0364100a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:19.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:19.973+0000 7f036afa0700 1 -- 172.21.15.96:0/2573158717 shutdown_connections 2024-04-25T14:09:19.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:19.973+0000 7f036afa0700 1 -- 172.21.15.96:0/2573158717 wait complete. 2024-04-25T14:09:19.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:19.973+0000 7f036afa0700 1 Processor -- start 2024-04-25T14:09:19.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:19.973+0000 7f036afa0700 1 -- start start 2024-04-25T14:09:19.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:19.973+0000 7f036afa0700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0364102e70 0x7f036406c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:19.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:19.973+0000 7f036afa0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0364103fc0 0x7f036419f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:19.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:19.973+0000 7f036afa0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0364104910 0x7f03641a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:19.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:19.973+0000 7f036afa0700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f036410ea30 con 0x7f0364104910 2024-04-25T14:09:19.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:19.974+0000 7f036afa0700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f036410e8b0 con 0x7f0364103fc0 2024-04-25T14:09:19.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:19.974+0000 7f036afa0700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f036410ebb0 con 0x7f0364102e70 2024-04-25T14:09:19.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:19.974+0000 7f036a79f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0364104910 0x7f03641a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:19.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:19.974+0000 7f036a79f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0364104910 0x7f03641a1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38506/0 (socket says 172.21.15.96:38506) 2024-04-25T14:09:19.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:19.974+0000 7f036a79f700 1 -- 172.21.15.96:0/4094247756 learned_addr learned my addr 172.21.15.96:0/4094247756 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:09:19.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:19.974+0000 7f0369f9e700 1 --2- 172.21.15.96:0/4094247756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0364102e70 0x7f036406c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:19.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:19.974+0000 7f036979d700 1 --2- 172.21.15.96:0/4094247756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0364103fc0 0x7f036419f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:19.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:19.974+0000 7f036a79f700 1 -- 172.21.15.96:0/4094247756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0364102e70 msgr2=0x7f036406c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:19.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:19.974+0000 7f036a79f700 1 --2- 172.21.15.96:0/4094247756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0364102e70 0x7f036406c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:19.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:19.974+0000 7f036a79f700 1 -- 172.21.15.96:0/4094247756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0364103fc0 msgr2=0x7f036419f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:19.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:19.974+0000 7f036a79f700 1 --2- 172.21.15.96:0/4094247756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0364103fc0 0x7f036419f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:19.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:19.974+0000 7f036a79f700 1 -- 172.21.15.96:0/4094247756 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03641a2140 con 0x7f0364104910 2024-04-25T14:09:19.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:19.974+0000 7f036a79f700 1 --2- 172.21.15.96:0/4094247756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0364104910 0x7f03641a1910 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f035c006130 tx=0x7f035c004ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:19.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:19.974+0000 7f035affd700 1 -- 172.21.15.96:0/4094247756 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f035c021070 con 0x7f0364104910 2024-04-25T14:09:19.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:19.974+0000 7f036afa0700 1 -- 172.21.15.96:0/4094247756 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03641a23d0 con 0x7f0364104910 2024-04-25T14:09:19.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:19.974+0000 7f035affd700 1 -- 172.21.15.96:0/4094247756 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f035c004020 con 0x7f0364104910 2024-04-25T14:09:19.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:19.974+0000 7f035affd700 1 -- 172.21.15.96:0/4094247756 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f035c01d6b0 con 0x7f0364104910 2024-04-25T14:09:19.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:19.974+0000 7f036afa0700 1 -- 172.21.15.96:0/4094247756 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03641a2950 con 0x7f0364104910 2024-04-25T14:09:19.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:19.975+0000 7f036afa0700 1 -- 172.21.15.96:0/4094247756 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0364051190 con 0x7f0364104910 2024-04-25T14:09:19.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:19.977+0000 7f035affd700 1 -- 172.21.15.96:0/4094247756 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f035c0044f0 con 0x7f0364104910 2024-04-25T14:09:19.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:19.978+0000 7f035affd700 1 --2- 172.21.15.96:0/4094247756 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f03500afb80 0x7f03500b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:19.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:19.978+0000 7f0369f9e700 1 --2- 172.21.15.96:0/4094247756 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f03500afb80 0x7f03500b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:19.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:19.978+0000 7f035affd700 1 -- 172.21.15.96:0/4094247756 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5829+0+0 (secure 0 0 0) 0x7f035c00f020 con 0x7f0364104910 2024-04-25T14:09:19.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:19.978+0000 7f0369f9e700 1 --2- 172.21.15.96:0/4094247756 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f03500afb80 0x7f03500b2090 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0360006fd0 tx=0x7f0360008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:19.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:19.982+0000 7f035affd700 1 -- 172.21.15.96:0/4094247756 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f035c02a020 con 0x7f0364104910 2024-04-25T14:09:20.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:20.190+0000 7f036afa0700 1 -- 172.21.15.96:0/4094247756 --> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f0364063ad0 con 0x7f03500afb80 2024-04-25T14:09:20.800 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:20 smithi096 ceph-mon[29207]: pgmap v195: 1 pgs: 1 peering; 577 KiB data, 258 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:09:20.800 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:20 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:09:20.800 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:20 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:09:20.800 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:20 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:09:20.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:20 smithi097 ceph-mon[32720]: pgmap v195: 1 pgs: 1 peering; 577 KiB data, 258 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:09:20.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:20 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:09:20.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:20 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:09:20.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:20 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:09:20.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:20 smithi151 ceph-mon[32990]: pgmap v195: 1 pgs: 1 peering; 577 KiB data, 258 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:09:20.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:20 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:09:20.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:20 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:09:20.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:20 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:09:21.825 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:21 smithi096 ceph-mon[29207]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:09:21.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:21 smithi097 ceph-mon[32720]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:09:21.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:21 smithi151 ceph-mon[32990]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:09:22.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:22 smithi096 ceph-mon[29207]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 258 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:09:22.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:22 smithi097 ceph-mon[32720]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 258 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:09:22.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:22 smithi151 ceph-mon[32990]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 258 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:09:23.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:23 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/976936661' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29d4da16-feab-4d1c-9d1d-de4f799bd724"}]: dispatch 2024-04-25T14:09:23.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:23 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/976936661' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "29d4da16-feab-4d1c-9d1d-de4f799bd724"}]': finished 2024-04-25T14:09:23.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:23 smithi096 ceph-mon[29207]: osdmap e57: 10 total, 9 up, 10 in 2024-04-25T14:09:23.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:23 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:09:23.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:23 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/976936661' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29d4da16-feab-4d1c-9d1d-de4f799bd724"}]: dispatch 2024-04-25T14:09:23.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:23 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/976936661' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "29d4da16-feab-4d1c-9d1d-de4f799bd724"}]': finished 2024-04-25T14:09:23.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:23 smithi097 ceph-mon[32720]: osdmap e57: 10 total, 9 up, 10 in 2024-04-25T14:09:23.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:23 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:09:23.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:23 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/976936661' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29d4da16-feab-4d1c-9d1d-de4f799bd724"}]: dispatch 2024-04-25T14:09:23.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:23 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/976936661' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "29d4da16-feab-4d1c-9d1d-de4f799bd724"}]': finished 2024-04-25T14:09:23.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:23 smithi151 ceph-mon[32990]: osdmap e57: 10 total, 9 up, 10 in 2024-04-25T14:09:23.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:23 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:09:24.858 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:24 smithi096 ceph-mon[29207]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T14:09:24.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:24 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/828411526' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:09:24.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:24 smithi097 ceph-mon[32720]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T14:09:24.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:24 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/828411526' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:09:24.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:24 smithi151 ceph-mon[32990]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T14:09:24.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:24 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/828411526' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:09:26.759 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:26 smithi096 ceph-mon[29207]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-25T14:09:26.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:26 smithi097 ceph-mon[32720]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-25T14:09:26.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:26 smithi151 ceph-mon[32990]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-25T14:09:28.565 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:28 smithi096 ceph-mon[29207]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-25T14:09:28.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:28 smithi097 ceph-mon[32720]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-25T14:09:28.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:28 smithi151 ceph-mon[32990]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-25T14:09:30.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:30 smithi096 ceph-mon[29207]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T14:09:30.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:30 smithi097 ceph-mon[32720]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T14:09:30.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:30 smithi151 ceph-mon[32990]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T14:09:31.833 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:31 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T14:09:31.833 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:31 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:09:31.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:31 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T14:09:31.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:31 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:09:31.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:31 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T14:09:31.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:31 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:09:32.824 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:32 smithi097 ceph-mon[32720]: Deploying daemon osd.9 on smithi096 2024-04-25T14:09:32.824 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:32 smithi097 ceph-mon[32720]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T14:09:32.839 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:32 smithi151 ceph-mon[32990]: Deploying daemon osd.9 on smithi096 2024-04-25T14:09:32.839 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:32 smithi151 ceph-mon[32990]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T14:09:32.858 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:32 smithi096 ceph-mon[29207]: Deploying daemon osd.9 on smithi096 2024-04-25T14:09:32.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:32 smithi096 ceph-mon[29207]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T14:09:34.769 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:34 smithi096 ceph-mon[29207]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:09:34.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:34 smithi097 ceph-mon[32720]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:09:34.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:34 smithi151 ceph-mon[32990]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:09:36.593 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:36 smithi096 ceph-mon[29207]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:09:36.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:36 smithi097 ceph-mon[32720]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:09:36.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:36 smithi151 ceph-mon[32990]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:09:38.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:37 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:38.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:37 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:38.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:37 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:09:38.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:37 smithi096 ceph-mon[29207]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:09:38.422 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:37 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:38.422 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:37 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:38.422 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:37 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:09:38.422 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:37 smithi097 ceph-mon[32720]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:09:38.437 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:37 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:38.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:37 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:38.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:37 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:09:38.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:37 smithi151 ceph-mon[32990]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:09:40.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:40 smithi096 ceph-mon[29207]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:09:40.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:40 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:40.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:40 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:40.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:40 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:09:40.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:40 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:09:40.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:40 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:40.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:40 smithi151 ceph-mon[32990]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:09:40.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:40 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:40.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:40 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:40.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:40 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:09:40.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:40 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:09:40.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:40 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:40.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:40 smithi097 ceph-mon[32720]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:09:40.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:40 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:40.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:40 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:40.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:40 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:09:40.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:40 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:09:40.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:40 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:41.202 INFO:teuthology.orchestra.run.smithi096.stdout:Created osd(s) 9 on host 'smithi096' 2024-04-25T14:09:41.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:41.201+0000 7f035affd700 1 -- 172.21.15.96:0/4094247756 <== mgr.14152 v2:172.21.15.96:6800/536332425 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0364063ad0 con 0x7f03500afb80 2024-04-25T14:09:41.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:41.204+0000 7f036afa0700 1 -- 172.21.15.96:0/4094247756 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f03500afb80 msgr2=0x7f03500b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:41.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:41.204+0000 7f036afa0700 1 --2- 172.21.15.96:0/4094247756 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f03500afb80 0x7f03500b2090 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0360006fd0 tx=0x7f0360008040 comp rx=0 tx=0).stop 2024-04-25T14:09:41.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:41.204+0000 7f036afa0700 1 -- 172.21.15.96:0/4094247756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0364104910 msgr2=0x7f03641a1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:41.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:41.204+0000 7f036afa0700 1 --2- 172.21.15.96:0/4094247756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0364104910 0x7f03641a1910 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f035c006130 tx=0x7f035c004ef0 comp rx=0 tx=0).stop 2024-04-25T14:09:41.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:41.204+0000 7f036afa0700 1 -- 172.21.15.96:0/4094247756 shutdown_connections 2024-04-25T14:09:41.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:41.204+0000 7f036afa0700 1 --2- 172.21.15.96:0/4094247756 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f03500afb80 0x7f03500b2090 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:41.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:41.204+0000 7f036afa0700 1 --2- 172.21.15.96:0/4094247756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0364104910 0x7f03641a1910 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:41.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:41.204+0000 7f036afa0700 1 --2- 172.21.15.96:0/4094247756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0364102e70 0x7f036406c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:41.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:41.204+0000 7f036afa0700 1 --2- 172.21.15.96:0/4094247756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0364103fc0 0x7f036419f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:41.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:41.204+0000 7f036afa0700 1 -- 172.21.15.96:0/4094247756 >> 172.21.15.96:0/4094247756 conn(0x7f03640fe570 msgr2=0x7f0364107280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:41.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:41.204+0000 7f036afa0700 1 -- 172.21.15.96:0/4094247756 shutdown_connections 2024-04-25T14:09:41.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:09:41.204+0000 7f036afa0700 1 -- 172.21.15.96:0/4094247756 wait complete. 2024-04-25T14:09:41.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:41 smithi151 ceph-mon[32990]: from='osd.9 [v2:172.21.15.96:6826/1099399643,v1:172.21.15.96:6827/1099399643]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T14:09:41.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:41 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:41.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:41 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:41.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:41 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:09:41.858 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:41 smithi096 ceph-mon[29207]: from='osd.9 [v2:172.21.15.96:6826/1099399643,v1:172.21.15.96:6827/1099399643]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T14:09:41.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:41 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:41.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:41 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:41.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:41 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:09:41.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:41 smithi097 ceph-mon[32720]: from='osd.9 [v2:172.21.15.96:6826/1099399643,v1:172.21.15.96:6827/1099399643]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T14:09:41.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:41 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:41.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:41 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:41.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:41 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:09:41.950 DEBUG:teuthology.orchestra.run.smithi096:osd.9> sudo journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@osd.9.service 2024-04-25T14:09:41.952 INFO:tasks.cephadm:Deploying osd.10 on smithi097 with /dev/vg_nvme/lv_1... 2024-04-25T14:09:41.952 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T14:09:42.203 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T14:09:42.577 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:42 smithi097 ceph-mon[32720]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:09:42.577 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:42 smithi097 ceph-mon[32720]: from='osd.9 [v2:172.21.15.96:6826/1099399643,v1:172.21.15.96:6827/1099399643]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T14:09:42.577 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:42 smithi097 ceph-mon[32720]: osdmap e58: 10 total, 9 up, 10 in 2024-04-25T14:09:42.577 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:42 smithi097 ceph-mon[32720]: from='osd.9 [v2:172.21.15.96:6826/1099399643,v1:172.21.15.96:6827/1099399643]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-04-25T14:09:42.577 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:42 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:09:42.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:42 smithi096 ceph-mon[29207]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:09:42.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:42 smithi096 ceph-mon[29207]: from='osd.9 [v2:172.21.15.96:6826/1099399643,v1:172.21.15.96:6827/1099399643]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T14:09:42.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:42 smithi096 ceph-mon[29207]: osdmap e58: 10 total, 9 up, 10 in 2024-04-25T14:09:42.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:42 smithi096 ceph-mon[29207]: from='osd.9 [v2:172.21.15.96:6826/1099399643,v1:172.21.15.96:6827/1099399643]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-04-25T14:09:42.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:42 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:09:42.609 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:09:42 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:09:42.442+0000 7f316d7fe700 -1 osd.9 0 waiting for initial osdmap 2024-04-25T14:09:42.610 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:09:42 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:09:42.454+0000 7f31695b3700 -1 osd.9 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:09:42.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:42 smithi151 ceph-mon[32990]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:09:42.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:42 smithi151 ceph-mon[32990]: from='osd.9 [v2:172.21.15.96:6826/1099399643,v1:172.21.15.96:6827/1099399643]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T14:09:42.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:42 smithi151 ceph-mon[32990]: osdmap e58: 10 total, 9 up, 10 in 2024-04-25T14:09:42.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:42 smithi151 ceph-mon[32990]: from='osd.9 [v2:172.21.15.96:6826/1099399643,v1:172.21.15.96:6827/1099399643]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-04-25T14:09:42.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:42 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:09:43.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:43 smithi097 ceph-mon[32720]: from='osd.9 [v2:172.21.15.96:6826/1099399643,v1:172.21.15.96:6827/1099399643]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-04-25T14:09:43.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:43 smithi097 ceph-mon[32720]: osdmap e59: 10 total, 9 up, 10 in 2024-04-25T14:09:43.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:43 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:09:43.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:43 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:09:43.790 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:43 smithi096 ceph-mon[29207]: from='osd.9 [v2:172.21.15.96:6826/1099399643,v1:172.21.15.96:6827/1099399643]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-04-25T14:09:43.790 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:43 smithi096 ceph-mon[29207]: osdmap e59: 10 total, 9 up, 10 in 2024-04-25T14:09:43.790 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:43 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:09:43.790 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:43 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:09:43.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:43 smithi151 ceph-mon[32990]: from='osd.9 [v2:172.21.15.96:6826/1099399643,v1:172.21.15.96:6827/1099399643]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-04-25T14:09:43.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:43 smithi151 ceph-mon[32990]: osdmap e59: 10 total, 9 up, 10 in 2024-04-25T14:09:43.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:43 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:09:43.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:43 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:09:44.256 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T14:09:44.256 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T14:09:44.257 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2024-04-25T14:09:44.257 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2024-04-25T14:09:44.257 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0131097 s, 800 MB/s 2024-04-25T14:09:44.257 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2024-04-25T14:09:44.819 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:44 smithi096 ceph-mon[29207]: purged_snaps scrub starts 2024-04-25T14:09:44.819 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:44 smithi096 ceph-mon[29207]: purged_snaps scrub ok 2024-04-25T14:09:44.819 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:44 smithi096 ceph-mon[29207]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:09:44.819 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:44 smithi096 ceph-mon[29207]: osd.9 [v2:172.21.15.96:6826/1099399643,v1:172.21.15.96:6827/1099399643] boot 2024-04-25T14:09:44.819 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:44 smithi096 ceph-mon[29207]: osdmap e60: 10 total, 10 up, 10 in 2024-04-25T14:09:44.819 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:44 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:09:44.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:44 smithi097 ceph-mon[32720]: purged_snaps scrub starts 2024-04-25T14:09:44.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:44 smithi097 ceph-mon[32720]: purged_snaps scrub ok 2024-04-25T14:09:44.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:44 smithi097 ceph-mon[32720]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:09:44.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:44 smithi097 ceph-mon[32720]: osd.9 [v2:172.21.15.96:6826/1099399643,v1:172.21.15.96:6827/1099399643] boot 2024-04-25T14:09:44.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:44 smithi097 ceph-mon[32720]: osdmap e60: 10 total, 10 up, 10 in 2024-04-25T14:09:44.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:44 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:09:44.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:44 smithi151 ceph-mon[32990]: purged_snaps scrub starts 2024-04-25T14:09:44.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:44 smithi151 ceph-mon[32990]: purged_snaps scrub ok 2024-04-25T14:09:44.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:44 smithi151 ceph-mon[32990]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:09:44.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:44 smithi151 ceph-mon[32990]: osd.9 [v2:172.21.15.96:6826/1099399643,v1:172.21.15.96:6827/1099399643] boot 2024-04-25T14:09:44.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:44 smithi151 ceph-mon[32990]: osdmap e60: 10 total, 10 up, 10 in 2024-04-25T14:09:44.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:44 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:09:44.988 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi097:vg_nvme/lv_1 2024-04-25T14:09:45.227 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T14:09:45.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:45 smithi096 ceph-mon[29207]: osdmap e61: 10 total, 10 up, 10 in 2024-04-25T14:09:45.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:45 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:45.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:45 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:45.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:45 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:09:45.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:45 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:09:45.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:45 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:09:45.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:45 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:09:45.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:45 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:45.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:45 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:09:45.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:45 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:09:45.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:45 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:45.667 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:45 smithi097 ceph-mon[32720]: osdmap e61: 10 total, 10 up, 10 in 2024-04-25T14:09:45.667 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:45 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:45.667 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:45 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:45.668 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:45 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:09:45.668 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:45 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:09:45.668 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:45 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:09:45.668 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:45 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:09:45.668 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:45 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:45.668 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:45 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:09:45.668 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:45 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:09:45.668 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:45 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:45.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:45 smithi151 ceph-mon[32990]: osdmap e61: 10 total, 10 up, 10 in 2024-04-25T14:09:45.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:45 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:45.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:45 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:45.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:45 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:09:45.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:45 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:09:45.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:45 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:09:45.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:45 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:09:45.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:45 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:45.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:45 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:09:45.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:45 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:09:45.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:45 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:09:46.601 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:46 smithi097 ceph-mon[32720]: Detected new or changed devices on smithi096 2024-04-25T14:09:46.602 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:46 smithi097 ceph-mon[32720]: Adjusting osd_memory_target on smithi096 to 4272M 2024-04-25T14:09:46.602 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:46 smithi097 ceph-mon[32720]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:09:46.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:09:46.831+0000 7ffb75cb4700 1 -- 172.21.15.97:0/2166303832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb70104d00 msgr2=0x7ffb70107120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:46.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:09:46.831+0000 7ffb75cb4700 1 --2- 172.21.15.97:0/2166303832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb70104d00 0x7ffb70107120 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ffb60009990 tx=0x7ffb60009ca0 comp rx=0 tx=0).stop 2024-04-25T14:09:46.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:09:46.831+0000 7ffb75cb4700 1 -- 172.21.15.97:0/2166303832 shutdown_connections 2024-04-25T14:09:46.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:09:46.831+0000 7ffb75cb4700 1 --2- 172.21.15.97:0/2166303832 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb70109fc0 0x7ffb7010c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:46.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:09:46.831+0000 7ffb75cb4700 1 --2- 172.21.15.97:0/2166303832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb70104d00 0x7ffb70107120 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:46.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:09:46.831+0000 7ffb75cb4700 1 --2- 172.21.15.97:0/2166303832 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb70107660 0x7ffb70109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:46.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:09:46.831+0000 7ffb75cb4700 1 -- 172.21.15.97:0/2166303832 >> 172.21.15.97:0/2166303832 conn(0x7ffb700fe740 msgr2=0x7ffb70100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:46.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:09:46.831+0000 7ffb75cb4700 1 -- 172.21.15.97:0/2166303832 shutdown_connections 2024-04-25T14:09:46.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:09:46.831+0000 7ffb75cb4700 1 -- 172.21.15.97:0/2166303832 wait complete. 2024-04-25T14:09:46.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:09:46.832+0000 7ffb75cb4700 1 Processor -- start 2024-04-25T14:09:46.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:09:46.832+0000 7ffb75cb4700 1 -- start start 2024-04-25T14:09:46.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:09:46.832+0000 7ffb75cb4700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb70104d00 0x7ffb7019a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:46.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:09:46.832+0000 7ffb75cb4700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb70107660 0x7ffb7019ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:46.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:09:46.832+0000 7ffb75cb4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb70109fc0 0x7ffb701a1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:46.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:09:46.832+0000 7ffb75cb4700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7010fc40 con 0x7ffb70107660 2024-04-25T14:09:46.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:09:46.832+0000 7ffb75cb4700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7010fac0 con 0x7ffb70109fc0 2024-04-25T14:09:46.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:09:46.832+0000 7ffb75cb4700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7010fdc0 con 0x7ffb70104d00 2024-04-25T14:09:46.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:09:46.832+0000 7ffb6ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb70109fc0 0x7ffb701a1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:46.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:09:46.832+0000 7ffb6ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb70109fc0 0x7ffb701a1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43196/0 (socket says 172.21.15.97:43196) 2024-04-25T14:09:46.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:09:46.832+0000 7ffb6ffff700 1 -- 172.21.15.97:0/3471654929 learned_addr learned my addr 172.21.15.97:0/3471654929 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T14:09:46.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:09:46.833+0000 7ffb6effd700 1 --2- 172.21.15.97:0/3471654929 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb70107660 0x7ffb7019ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:46.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:09:46.833+0000 7ffb6f7fe700 1 --2- 172.21.15.97:0/3471654929 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb70104d00 0x7ffb7019a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:46.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:09:46.833+0000 7ffb6ffff700 1 -- 172.21.15.97:0/3471654929 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb70104d00 msgr2=0x7ffb7019a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:46.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:09:46.833+0000 7ffb6ffff700 1 --2- 172.21.15.97:0/3471654929 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb70104d00 0x7ffb7019a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:46.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:09:46.833+0000 7ffb6ffff700 1 -- 172.21.15.97:0/3471654929 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb70107660 msgr2=0x7ffb7019ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:46.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:09:46.833+0000 7ffb6ffff700 1 --2- 172.21.15.97:0/3471654929 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb70107660 0x7ffb7019ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:46.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:09:46.833+0000 7ffb6ffff700 1 -- 172.21.15.97:0/3471654929 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb701a2290 con 0x7ffb70109fc0 2024-04-25T14:09:46.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:09:46.833+0000 7ffb6ffff700 1 --2- 172.21.15.97:0/3471654929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb70109fc0 0x7ffb701a1c10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ffb6800ef30 tx=0x7ffb6800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:46.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:09:46.833+0000 7ffb6cff9700 1 -- 172.21.15.97:0/3471654929 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb680092e0 con 0x7ffb70109fc0 2024-04-25T14:09:46.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:09:46.833+0000 7ffb6cff9700 1 -- 172.21.15.97:0/3471654929 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb68004bb0 con 0x7ffb70109fc0 2024-04-25T14:09:46.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:09:46.833+0000 7ffb75cb4700 1 -- 172.21.15.97:0/3471654929 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb701a2520 con 0x7ffb70109fc0 2024-04-25T14:09:46.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:09:46.833+0000 7ffb6cff9700 1 -- 172.21.15.97:0/3471654929 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb68004280 con 0x7ffb70109fc0 2024-04-25T14:09:46.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:09:46.833+0000 7ffb75cb4700 1 -- 172.21.15.97:0/3471654929 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb701a2a60 con 0x7ffb70109fc0 2024-04-25T14:09:46.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:09:46.834+0000 7ffb75cb4700 1 -- 172.21.15.97:0/3471654929 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb70051190 con 0x7ffb70109fc0 2024-04-25T14:09:46.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:09:46.835+0000 7ffb6cff9700 1 -- 172.21.15.97:0/3471654929 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7ffb6801b030 con 0x7ffb70109fc0 2024-04-25T14:09:46.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:09:46.836+0000 7ffb6cff9700 1 --2- 172.21.15.97:0/3471654929 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ffb580afc00 0x7ffb580b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:46.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:09:46.837+0000 7ffb6cff9700 1 -- 172.21.15.97:0/3471654929 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6261+0+0 (secure 0 0 0) 0x7ffb680ddf00 con 0x7ffb70109fc0 2024-04-25T14:09:46.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:09:46.837+0000 7ffb6f7fe700 1 --2- 172.21.15.97:0/3471654929 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ffb580afc00 0x7ffb580b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:46.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:09:46.837+0000 7ffb6f7fe700 1 --2- 172.21.15.97:0/3471654929 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ffb580afc00 0x7ffb580b2110 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ffb6000b2e0 tx=0x7ffb60005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:46.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:09:46.841+0000 7ffb6cff9700 1 -- 172.21.15.97:0/3471654929 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffb68022020 con 0x7ffb70109fc0 2024-04-25T14:09:46.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:46 smithi096 ceph-mon[29207]: Detected new or changed devices on smithi096 2024-04-25T14:09:46.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:46 smithi096 ceph-mon[29207]: Adjusting osd_memory_target on smithi096 to 4272M 2024-04-25T14:09:46.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:46 smithi096 ceph-mon[29207]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:09:46.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:46 smithi151 ceph-mon[32990]: Detected new or changed devices on smithi096 2024-04-25T14:09:46.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:46 smithi151 ceph-mon[32990]: Adjusting osd_memory_target on smithi096 to 4272M 2024-04-25T14:09:46.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:46 smithi151 ceph-mon[32990]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:09:47.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:09:47.049+0000 7ffb75cb4700 1 -- 172.21.15.97:0/3471654929 --> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7ffb70063ad0 con 0x7ffb580afc00 2024-04-25T14:09:47.761 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:47 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:09:47.762 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:47 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:09:47.762 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:47 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:09:47.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:47 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:09:47.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:47 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:09:47.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:47 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:09:47.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:47 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:09:47.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:47 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:09:47.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:47 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:09:48.742 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:48 smithi097 ceph-mon[32720]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:09:48.742 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:48 smithi097 ceph-mon[32720]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:09:48.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:48 smithi096 ceph-mon[29207]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:09:48.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:48 smithi096 ceph-mon[29207]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:09:48.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:48 smithi151 ceph-mon[32990]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:09:48.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:48 smithi151 ceph-mon[32990]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:09:50.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:50 smithi097 ceph-mon[32720]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:09:50.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:50 smithi097 ceph-mon[32720]: from='client.? 172.21.15.97:0/791271025' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "197fee9b-eee4-4893-be1c-3d57f42250e9"}]: dispatch 2024-04-25T14:09:50.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:50 smithi097 ceph-mon[32720]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "197fee9b-eee4-4893-be1c-3d57f42250e9"}]: dispatch 2024-04-25T14:09:50.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:50 smithi097 ceph-mon[32720]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "197fee9b-eee4-4893-be1c-3d57f42250e9"}]': finished 2024-04-25T14:09:50.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:50 smithi097 ceph-mon[32720]: osdmap e62: 11 total, 10 up, 11 in 2024-04-25T14:09:50.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:50 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:09:50.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:50 smithi096 ceph-mon[29207]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:09:50.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:50 smithi096 ceph-mon[29207]: from='client.? 172.21.15.97:0/791271025' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "197fee9b-eee4-4893-be1c-3d57f42250e9"}]: dispatch 2024-04-25T14:09:50.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:50 smithi096 ceph-mon[29207]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "197fee9b-eee4-4893-be1c-3d57f42250e9"}]: dispatch 2024-04-25T14:09:50.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:50 smithi096 ceph-mon[29207]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "197fee9b-eee4-4893-be1c-3d57f42250e9"}]': finished 2024-04-25T14:09:50.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:50 smithi096 ceph-mon[29207]: osdmap e62: 11 total, 10 up, 11 in 2024-04-25T14:09:50.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:50 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:09:50.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:50 smithi151 ceph-mon[32990]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:09:50.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:50 smithi151 ceph-mon[32990]: from='client.? 172.21.15.97:0/791271025' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "197fee9b-eee4-4893-be1c-3d57f42250e9"}]: dispatch 2024-04-25T14:09:50.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:50 smithi151 ceph-mon[32990]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "197fee9b-eee4-4893-be1c-3d57f42250e9"}]: dispatch 2024-04-25T14:09:50.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:50 smithi151 ceph-mon[32990]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "197fee9b-eee4-4893-be1c-3d57f42250e9"}]': finished 2024-04-25T14:09:50.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:50 smithi151 ceph-mon[32990]: osdmap e62: 11 total, 10 up, 11 in 2024-04-25T14:09:50.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:50 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:09:51.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:51 smithi096 ceph-mon[29207]: from='client.? 172.21.15.97:0/1703096035' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:09:51.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:51 smithi097 ceph-mon[32720]: from='client.? 172.21.15.97:0/1703096035' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:09:51.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:51 smithi151 ceph-mon[32990]: from='client.? 172.21.15.97:0/1703096035' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:09:52.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:52 smithi096 ceph-mon[29207]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:09:52.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:52 smithi097 ceph-mon[32720]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:09:52.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:52 smithi151 ceph-mon[32990]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:09:54.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:54 smithi097 ceph-mon[32720]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:09:54.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:54 smithi096 ceph-mon[29207]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:09:54.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:54 smithi151 ceph-mon[32990]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:09:56.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:56 smithi097 ceph-mon[32720]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:09:56.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:56 smithi096 ceph-mon[29207]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:09:56.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:56 smithi151 ceph-mon[32990]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:09:58.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:58 smithi096 ceph-mon[29207]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:09:58.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:58 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T14:09:58.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:58 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:09:58.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:09:58 smithi096 ceph-mon[29207]: Deploying daemon osd.10 on smithi097 2024-04-25T14:09:58.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:58 smithi097 ceph-mon[32720]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:09:58.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:58 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T14:09:58.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:58 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:09:58.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:09:58 smithi097 ceph-mon[32720]: Deploying daemon osd.10 on smithi097 2024-04-25T14:09:58.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:58 smithi151 ceph-mon[32990]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:09:58.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:58 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T14:09:58.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:58 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:09:58.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:09:58 smithi151 ceph-mon[32990]: Deploying daemon osd.10 on smithi097 2024-04-25T14:10:00.831 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:00 smithi097 ceph-mon[32720]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:10:00.831 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:00 smithi097 ceph-mon[32720]: overall HEALTH_OK 2024-04-25T14:10:00.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:00 smithi096 ceph-mon[29207]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:10:00.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:00 smithi096 ceph-mon[29207]: overall HEALTH_OK 2024-04-25T14:10:00.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:00 smithi151 ceph-mon[32990]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:10:00.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:00 smithi151 ceph-mon[32990]: overall HEALTH_OK 2024-04-25T14:10:02.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:02 smithi097 ceph-mon[32720]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:10:02.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:02 smithi096 ceph-mon[29207]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:10:02.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:02 smithi151 ceph-mon[32990]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:10:04.320 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:04 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:04.320 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:04 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:04.320 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:04 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:10:04.320 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:04 smithi097 ceph-mon[32720]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:10:04.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:04 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:04.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:04 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:04.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:04 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:10:04.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:04 smithi096 ceph-mon[29207]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:10:04.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:04 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:04.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:04 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:04.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:04 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:10:04.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:04 smithi151 ceph-mon[32990]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:10:06.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:06 smithi097 ceph-mon[32720]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:10:06.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:06 smithi151 ceph-mon[32990]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:10:06.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:10:06.766+0000 7ffb6cff9700 1 -- 172.21.15.97:0/3471654929 <== mgr.14152 v2:172.21.15.96:6800/536332425 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7ffb70063ad0 con 0x7ffb580afc00 2024-04-25T14:10:06.766 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 10 on host 'smithi097' 2024-04-25T14:10:06.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:10:06.770+0000 7ffb75cb4700 1 -- 172.21.15.97:0/3471654929 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ffb580afc00 msgr2=0x7ffb580b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:06.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:10:06.770+0000 7ffb75cb4700 1 --2- 172.21.15.97:0/3471654929 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ffb580afc00 0x7ffb580b2110 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ffb6000b2e0 tx=0x7ffb60005fb0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:10:06.770+0000 7ffb75cb4700 1 -- 172.21.15.97:0/3471654929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb70109fc0 msgr2=0x7ffb701a1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:06.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:10:06.770+0000 7ffb75cb4700 1 --2- 172.21.15.97:0/3471654929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb70109fc0 0x7ffb701a1c10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ffb6800ef30 tx=0x7ffb6800c3a0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:10:06.770+0000 7ffb75cb4700 1 -- 172.21.15.97:0/3471654929 shutdown_connections 2024-04-25T14:10:06.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:10:06.770+0000 7ffb75cb4700 1 --2- 172.21.15.97:0/3471654929 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ffb580afc00 0x7ffb580b2110 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:10:06.770+0000 7ffb75cb4700 1 --2- 172.21.15.97:0/3471654929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb70109fc0 0x7ffb701a1c10 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:10:06.770+0000 7ffb75cb4700 1 --2- 172.21.15.97:0/3471654929 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb70104d00 0x7ffb7019a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:10:06.770+0000 7ffb75cb4700 1 --2- 172.21.15.97:0/3471654929 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb70107660 0x7ffb7019ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:10:06.770+0000 7ffb75cb4700 1 -- 172.21.15.97:0/3471654929 >> 172.21.15.97:0/3471654929 conn(0x7ffb700fe740 msgr2=0x7ffb701057d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:06.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:10:06.770+0000 7ffb75cb4700 1 -- 172.21.15.97:0/3471654929 shutdown_connections 2024-04-25T14:10:06.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T14:10:06.770+0000 7ffb75cb4700 1 -- 172.21.15.97:0/3471654929 wait complete. 2024-04-25T14:10:06.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:06 smithi096 ceph-mon[29207]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:10:07.389 DEBUG:teuthology.orchestra.run.smithi097:osd.10> sudo journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@osd.10.service 2024-04-25T14:10:07.392 INFO:tasks.cephadm:Deploying osd.11 on smithi151 with /dev/vg_nvme/lv_1... 2024-04-25T14:10:07.392 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T14:10:07.639 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:10:08.033 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:07 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:08.033 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:07 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:08.033 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:07 smithi151 ceph-mon[32990]: from='osd.10 [v2:172.21.15.97:6824/155754520,v1:172.21.15.97:6825/155754520]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T14:10:08.033 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:07 smithi151 ceph-mon[32990]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T14:10:08.033 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:07 smithi151 ceph-mon[32990]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:10:08.033 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:07 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:08.033 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:07 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:08.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:07 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:08.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:07 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:08.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:07 smithi096 ceph-mon[29207]: from='osd.10 [v2:172.21.15.97:6824/155754520,v1:172.21.15.97:6825/155754520]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T14:10:08.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:07 smithi096 ceph-mon[29207]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T14:10:08.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:07 smithi096 ceph-mon[29207]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:10:08.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:07 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:08.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:07 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:08.172 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:07 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:08.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:07 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:08.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:07 smithi097 ceph-mon[32720]: from='osd.10 [v2:172.21.15.97:6824/155754520,v1:172.21.15.97:6825/155754520]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T14:10:08.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:07 smithi097 ceph-mon[32720]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T14:10:08.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:07 smithi097 ceph-mon[32720]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:10:08.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:07 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:08.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:07 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:09.098 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:08 smithi151 ceph-mon[32990]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T14:10:09.099 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:08 smithi151 ceph-mon[32990]: osdmap e63: 11 total, 10 up, 11 in 2024-04-25T14:10:09.099 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:08 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:10:09.099 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:08 smithi151 ceph-mon[32990]: from='osd.10 [v2:172.21.15.97:6824/155754520,v1:172.21.15.97:6825/155754520]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T14:10:09.099 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:08 smithi151 ceph-mon[32990]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T14:10:09.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:08 smithi096 ceph-mon[29207]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T14:10:09.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:08 smithi096 ceph-mon[29207]: osdmap e63: 11 total, 10 up, 11 in 2024-04-25T14:10:09.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:08 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:10:09.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:08 smithi096 ceph-mon[29207]: from='osd.10 [v2:172.21.15.97:6824/155754520,v1:172.21.15.97:6825/155754520]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T14:10:09.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:08 smithi096 ceph-mon[29207]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T14:10:09.172 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:08 smithi097 ceph-mon[32720]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T14:10:09.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:08 smithi097 ceph-mon[32720]: osdmap e63: 11 total, 10 up, 11 in 2024-04-25T14:10:09.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:08 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:10:09.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:08 smithi097 ceph-mon[32720]: from='osd.10 [v2:172.21.15.97:6824/155754520,v1:172.21.15.97:6825/155754520]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T14:10:09.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:08 smithi097 ceph-mon[32720]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T14:10:09.535 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:10:09 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:10:09.199+0000 7fe9b56a3700 -1 osd.10 0 waiting for initial osdmap 2024-04-25T14:10:09.535 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:10:09 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:10:09.212+0000 7fe9ad450700 -1 osd.10 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:10:09.812 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T14:10:09.812 INFO:teuthology.orchestra.run.smithi151.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T14:10:09.812 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10+0 records in 2024-04-25T14:10:09.812 INFO:teuthology.orchestra.run.smithi151.stderr:10+0 records out 2024-04-25T14:10:09.812 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115481 s, 908 MB/s 2024-04-25T14:10:09.812 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping successful for: 2024-04-25T14:10:09.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:09 smithi097 ceph-mon[32720]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-25T14:10:09.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:09 smithi097 ceph-mon[32720]: osdmap e64: 11 total, 10 up, 11 in 2024-04-25T14:10:09.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:09 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:10:09.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:09 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:10:09.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:09 smithi097 ceph-mon[32720]: from='osd.10 ' entity='osd.10' 2024-04-25T14:10:09.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:09 smithi097 ceph-mon[32720]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:10:10.103 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:09 smithi151 ceph-mon[32990]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-25T14:10:10.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:09 smithi151 ceph-mon[32990]: osdmap e64: 11 total, 10 up, 11 in 2024-04-25T14:10:10.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:09 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:10:10.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:09 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:10:10.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:09 smithi151 ceph-mon[32990]: from='osd.10 ' entity='osd.10' 2024-04-25T14:10:10.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:09 smithi151 ceph-mon[32990]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:10:10.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:09 smithi096 ceph-mon[29207]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-25T14:10:10.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:09 smithi096 ceph-mon[29207]: osdmap e64: 11 total, 10 up, 11 in 2024-04-25T14:10:10.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:09 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:10:10.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:09 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:10:10.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:09 smithi096 ceph-mon[29207]: from='osd.10 ' entity='osd.10' 2024-04-25T14:10:10.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:09 smithi096 ceph-mon[29207]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:10:10.540 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi151:vg_nvme/lv_1 2024-04-25T14:10:10.786 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:10:10.819 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:10 smithi151 ceph-mon[32990]: purged_snaps scrub starts 2024-04-25T14:10:10.820 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:10 smithi151 ceph-mon[32990]: purged_snaps scrub ok 2024-04-25T14:10:10.820 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:10 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:10:10.820 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:10 smithi151 ceph-mon[32990]: osd.10 [v2:172.21.15.97:6824/155754520,v1:172.21.15.97:6825/155754520] boot 2024-04-25T14:10:10.820 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:10 smithi151 ceph-mon[32990]: osdmap e65: 11 total, 11 up, 11 in 2024-04-25T14:10:10.820 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:10 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:10:10.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:10 smithi097 ceph-mon[32720]: purged_snaps scrub starts 2024-04-25T14:10:10.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:10 smithi097 ceph-mon[32720]: purged_snaps scrub ok 2024-04-25T14:10:10.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:10 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:10:10.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:10 smithi097 ceph-mon[32720]: osd.10 [v2:172.21.15.97:6824/155754520,v1:172.21.15.97:6825/155754520] boot 2024-04-25T14:10:10.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:10 smithi097 ceph-mon[32720]: osdmap e65: 11 total, 11 up, 11 in 2024-04-25T14:10:10.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:10 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:10:11.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:10 smithi096 ceph-mon[29207]: purged_snaps scrub starts 2024-04-25T14:10:11.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:10 smithi096 ceph-mon[29207]: purged_snaps scrub ok 2024-04-25T14:10:11.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:10 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:10:11.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:10 smithi096 ceph-mon[29207]: osd.10 [v2:172.21.15.97:6824/155754520,v1:172.21.15.97:6825/155754520] boot 2024-04-25T14:10:11.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:10 smithi096 ceph-mon[29207]: osdmap e65: 11 total, 11 up, 11 in 2024-04-25T14:10:11.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:10 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:10:11.913 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:11 smithi151 ceph-mon[32990]: osdmap e66: 11 total, 11 up, 11 in 2024-04-25T14:10:11.913 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:11 smithi151 ceph-mon[32990]: Detected new or changed devices on smithi097 2024-04-25T14:10:11.913 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:11 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:11.913 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:11 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:11.913 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:11 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:10:11.913 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:11 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:10:11.914 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:11 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:10:11.914 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:11 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:10:11.914 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:11 smithi151 ceph-mon[32990]: Adjusting osd_memory_target on smithi097 to 4272M 2024-04-25T14:10:11.914 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:11 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:11.914 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:11 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:10:11.914 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:11 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:10:11.914 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:11 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:11.914 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:11 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:10:11.914 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:11 smithi151 ceph-mon[32990]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:10:12.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:11 smithi096 ceph-mon[29207]: osdmap e66: 11 total, 11 up, 11 in 2024-04-25T14:10:12.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:11 smithi096 ceph-mon[29207]: Detected new or changed devices on smithi097 2024-04-25T14:10:12.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:11 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:12.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:11 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:12.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:11 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:10:12.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:11 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:10:12.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:11 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:10:12.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:11 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:10:12.110 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:11 smithi096 ceph-mon[29207]: Adjusting osd_memory_target on smithi097 to 4272M 2024-04-25T14:10:12.110 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:11 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:12.110 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:11 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:10:12.110 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:11 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:10:12.110 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:11 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:12.110 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:11 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:10:12.110 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:11 smithi096 ceph-mon[29207]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:10:12.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:11 smithi097 ceph-mon[32720]: osdmap e66: 11 total, 11 up, 11 in 2024-04-25T14:10:12.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:11 smithi097 ceph-mon[32720]: Detected new or changed devices on smithi097 2024-04-25T14:10:12.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:11 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:12.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:11 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:12.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:11 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:10:12.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:11 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:10:12.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:11 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:10:12.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:11 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:10:12.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:11 smithi097 ceph-mon[32720]: Adjusting osd_memory_target on smithi097 to 4272M 2024-04-25T14:10:12.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:11 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:12.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:11 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:10:12.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:11 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:10:12.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:11 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:12.174 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:11 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:10:12.174 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:11 smithi097 ceph-mon[32720]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:10:12.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:12.466+0000 7f1be2972700 1 -- 172.21.15.151:0/3358221019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1bdc105ce0 msgr2=0x7f1bdc10c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:12.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:12.466+0000 7f1be2972700 1 --2- 172.21.15.151:0/3358221019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1bdc105ce0 0x7f1bdc10c590 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1bd400b0a0 tx=0x7f1bd400b3b0 comp rx=0 tx=0).stop 2024-04-25T14:10:12.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:12.466+0000 7f1be2972700 1 -- 172.21.15.151:0/3358221019 shutdown_connections 2024-04-25T14:10:12.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:12.466+0000 7f1be2972700 1 --2- 172.21.15.151:0/3358221019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1bdc105ce0 0x7f1bdc10c590 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:12.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:12.466+0000 7f1be2972700 1 --2- 172.21.15.151:0/3358221019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bdc104210 0x7f1bdc104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:12.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:12.466+0000 7f1be2972700 1 --2- 172.21.15.151:0/3358221019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1bdc105340 0x7f1bdc105750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:12.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:12.466+0000 7f1be2972700 1 -- 172.21.15.151:0/3358221019 >> 172.21.15.151:0/3358221019 conn(0x7f1bdc0ff870 msgr2=0x7f1bdc101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:12.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:12.466+0000 7f1be2972700 1 -- 172.21.15.151:0/3358221019 shutdown_connections 2024-04-25T14:10:12.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:12.466+0000 7f1be2972700 1 -- 172.21.15.151:0/3358221019 wait complete. 2024-04-25T14:10:12.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:12.466+0000 7f1be2972700 1 Processor -- start 2024-04-25T14:10:12.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:12.466+0000 7f1be2972700 1 -- start start 2024-04-25T14:10:12.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:12.467+0000 7f1be2972700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1bdc104210 0x7f1bdc19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:12.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:12.467+0000 7f1be2972700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1bdc105340 0x7f1bdc19ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:12.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:12.467+0000 7f1be2972700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bdc105ce0 0x7f1bdc1a1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:12.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:12.467+0000 7f1be2972700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1bdc10fc70 con 0x7f1bdc105340 2024-04-25T14:10:12.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:12.467+0000 7f1be2972700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1bdc10faf0 con 0x7f1bdc105ce0 2024-04-25T14:10:12.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:12.467+0000 7f1be2972700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1bdc10fdf0 con 0x7f1bdc104210 2024-04-25T14:10:12.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:12.467+0000 7f1bdbfff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1bdc104210 0x7f1bdc19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:12.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:12.467+0000 7f1bdbfff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1bdc104210 0x7f1bdc19a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42254/0 (socket says 172.21.15.151:42254) 2024-04-25T14:10:12.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:12.467+0000 7f1bdbfff700 1 -- 172.21.15.151:0/2139675271 learned_addr learned my addr 172.21.15.151:0/2139675271 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-25T14:10:12.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:12.467+0000 7f1bdb7fe700 1 --2- 172.21.15.151:0/2139675271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1bdc105340 0x7f1bdc19ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:12.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:12.467+0000 7f1be0f0f700 1 --2- 172.21.15.151:0/2139675271 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bdc105ce0 0x7f1bdc1a1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:12.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:12.467+0000 7f1bdb7fe700 1 -- 172.21.15.151:0/2139675271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1bdc104210 msgr2=0x7f1bdc19a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:12.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:12.467+0000 7f1bdb7fe700 1 --2- 172.21.15.151:0/2139675271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1bdc104210 0x7f1bdc19a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:12.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:12.467+0000 7f1bdb7fe700 1 -- 172.21.15.151:0/2139675271 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bdc105ce0 msgr2=0x7f1bdc1a1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:12.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:12.467+0000 7f1bdb7fe700 1 --2- 172.21.15.151:0/2139675271 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bdc105ce0 0x7f1bdc1a1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:12.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:12.467+0000 7f1bdb7fe700 1 -- 172.21.15.151:0/2139675271 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bdc1a2260 con 0x7f1bdc105340 2024-04-25T14:10:12.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:12.467+0000 7f1be0f0f700 1 --2- 172.21.15.151:0/2139675271 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bdc105ce0 0x7f1bdc1a1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:10:12.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:12.467+0000 7f1bdbfff700 1 --2- 172.21.15.151:0/2139675271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1bdc104210 0x7f1bdc19a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:10:12.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:12.468+0000 7f1bdb7fe700 1 --2- 172.21.15.151:0/2139675271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1bdc105340 0x7f1bdc19ac40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1bd000e9f0 tx=0x7f1bd000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:12.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:12.468+0000 7f1bd97fa700 1 -- 172.21.15.151:0/2139675271 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd000cbd0 con 0x7f1bdc105340 2024-04-25T14:10:12.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:12.468+0000 7f1be2972700 1 -- 172.21.15.151:0/2139675271 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bdc1a2550 con 0x7f1bdc105340 2024-04-25T14:10:12.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:12.468+0000 7f1bd97fa700 1 -- 172.21.15.151:0/2139675271 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bd000cd70 con 0x7f1bdc105340 2024-04-25T14:10:12.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:12.468+0000 7f1bd97fa700 1 -- 172.21.15.151:0/2139675271 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd0018680 con 0x7f1bdc105340 2024-04-25T14:10:12.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:12.468+0000 7f1be2972700 1 -- 172.21.15.151:0/2139675271 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bdc1a2ae0 con 0x7f1bdc105340 2024-04-25T14:10:12.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:12.469+0000 7f1be2972700 1 -- 172.21.15.151:0/2139675271 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bdc051190 con 0x7f1bdc105340 2024-04-25T14:10:12.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:12.470+0000 7f1bd97fa700 1 -- 172.21.15.151:0/2139675271 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f1bd0010430 con 0x7f1bdc105340 2024-04-25T14:10:12.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:12.471+0000 7f1bd97fa700 1 --2- 172.21.15.151:0/2139675271 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1bc40afb80 0x7f1bc40b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:12.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:12.471+0000 7f1bdbfff700 1 --2- 172.21.15.151:0/2139675271 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1bc40afb80 0x7f1bc40b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:12.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:12.471+0000 7f1bd97fa700 1 -- 172.21.15.151:0/2139675271 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6694+0+0 (secure 0 0 0) 0x7f1bd00de9f0 con 0x7f1bdc105340 2024-04-25T14:10:12.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:12.471+0000 7f1bdbfff700 1 --2- 172.21.15.151:0/2139675271 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1bc40afb80 0x7f1bc40b2090 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc00afd0 tx=0x7f1bcc00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:12.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:12.475+0000 7f1bd97fa700 1 -- 172.21.15.151:0/2139675271 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1bd00a7b80 con 0x7f1bdc105340 2024-04-25T14:10:12.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:12.679+0000 7f1be2972700 1 -- 172.21.15.151:0/2139675271 --> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f1bdc063ad0 con 0x7f1bc40afb80 2024-04-25T14:10:12.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:12 smithi151 ceph-mon[32990]: osdmap e67: 11 total, 11 up, 11 in 2024-04-25T14:10:12.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:12 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:10:12.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:12 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:10:12.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:12 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:12.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:12 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:10:12.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:12 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:10:12.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:12 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:10:13.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:12 smithi096 ceph-mon[29207]: osdmap e67: 11 total, 11 up, 11 in 2024-04-25T14:10:13.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:12 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:10:13.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:12 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:10:13.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:12 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:13.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:12 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:10:13.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:12 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:10:13.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:12 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:10:13.172 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:12 smithi097 ceph-mon[32720]: osdmap e67: 11 total, 11 up, 11 in 2024-04-25T14:10:13.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:12 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:10:13.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:12 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:10:13.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:12 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:13.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:12 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:10:13.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:12 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:10:13.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:12 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:10:13.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:13 smithi151 ceph-mon[32990]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:10:13.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:13 smithi151 ceph-mon[32990]: pgmap v233: 1 pgs: 1 peering; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:10:14.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:13 smithi096 ceph-mon[29207]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:10:14.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:13 smithi096 ceph-mon[29207]: pgmap v233: 1 pgs: 1 peering; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:10:14.172 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:13 smithi097 ceph-mon[32720]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:10:14.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:13 smithi097 ceph-mon[32720]: pgmap v233: 1 pgs: 1 peering; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:10:16.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:16 smithi151 ceph-mon[32990]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:10:16.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:16 smithi151 ceph-mon[32990]: from='client.? 172.21.15.151:0/3362956532' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9c1d0a83-fc12-41a2-8eb0-80c4abdd2770"}]: dispatch 2024-04-25T14:10:16.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:16 smithi151 ceph-mon[32990]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9c1d0a83-fc12-41a2-8eb0-80c4abdd2770"}]: dispatch 2024-04-25T14:10:16.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:16 smithi151 ceph-mon[32990]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9c1d0a83-fc12-41a2-8eb0-80c4abdd2770"}]': finished 2024-04-25T14:10:16.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:16 smithi151 ceph-mon[32990]: osdmap e68: 12 total, 11 up, 12 in 2024-04-25T14:10:16.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:16 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:10:16.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:16 smithi151 ceph-mon[32990]: from='client.? 172.21.15.151:0/1448516388' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:10:16.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:16 smithi096 ceph-mon[29207]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:10:16.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:16 smithi096 ceph-mon[29207]: from='client.? 172.21.15.151:0/3362956532' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9c1d0a83-fc12-41a2-8eb0-80c4abdd2770"}]: dispatch 2024-04-25T14:10:16.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:16 smithi096 ceph-mon[29207]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9c1d0a83-fc12-41a2-8eb0-80c4abdd2770"}]: dispatch 2024-04-25T14:10:16.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:16 smithi096 ceph-mon[29207]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9c1d0a83-fc12-41a2-8eb0-80c4abdd2770"}]': finished 2024-04-25T14:10:16.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:16 smithi096 ceph-mon[29207]: osdmap e68: 12 total, 11 up, 12 in 2024-04-25T14:10:16.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:16 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:10:16.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:16 smithi096 ceph-mon[29207]: from='client.? 172.21.15.151:0/1448516388' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:10:16.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:16 smithi097 ceph-mon[32720]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:10:16.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:16 smithi097 ceph-mon[32720]: from='client.? 172.21.15.151:0/3362956532' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9c1d0a83-fc12-41a2-8eb0-80c4abdd2770"}]: dispatch 2024-04-25T14:10:16.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:16 smithi097 ceph-mon[32720]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9c1d0a83-fc12-41a2-8eb0-80c4abdd2770"}]: dispatch 2024-04-25T14:10:16.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:16 smithi097 ceph-mon[32720]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9c1d0a83-fc12-41a2-8eb0-80c4abdd2770"}]': finished 2024-04-25T14:10:16.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:16 smithi097 ceph-mon[32720]: osdmap e68: 12 total, 11 up, 12 in 2024-04-25T14:10:16.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:10:16.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:16 smithi097 ceph-mon[32720]: from='client.? 172.21.15.151:0/1448516388' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:10:18.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:18 smithi151 ceph-mon[32990]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-25T14:10:18.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:18 smithi096 ceph-mon[29207]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-25T14:10:18.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:18 smithi097 ceph-mon[32720]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-25T14:10:20.695 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:20 smithi151 ceph-mon[32990]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T14:10:20.858 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:20 smithi096 ceph-mon[29207]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T14:10:20.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:20 smithi097 ceph-mon[32720]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T14:10:22.623 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:22 smithi151 ceph-mon[32990]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T14:10:22.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:22 smithi096 ceph-mon[29207]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T14:10:22.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:22 smithi097 ceph-mon[32720]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T14:10:23.704 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:23 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T14:10:23.704 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:23 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:10:23.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:23 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T14:10:23.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:23 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:10:23.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:23 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T14:10:23.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:23 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:10:24.548 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:24 smithi151 ceph-mon[32990]: Deploying daemon osd.11 on smithi151 2024-04-25T14:10:24.548 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:24 smithi151 ceph-mon[32990]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T14:10:24.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:24 smithi096 ceph-mon[29207]: Deploying daemon osd.11 on smithi151 2024-04-25T14:10:24.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:24 smithi096 ceph-mon[29207]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T14:10:24.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:24 smithi097 ceph-mon[32720]: Deploying daemon osd.11 on smithi151 2024-04-25T14:10:24.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:24 smithi097 ceph-mon[32720]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T14:10:26.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:26 smithi151 ceph-mon[32990]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T14:10:26.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:26 smithi096 ceph-mon[29207]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T14:10:26.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:26 smithi097 ceph-mon[32720]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T14:10:28.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:28 smithi151 ceph-mon[32990]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-04-25T14:10:28.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:28 smithi096 ceph-mon[29207]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-04-25T14:10:28.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:28 smithi097 ceph-mon[32720]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-04-25T14:10:30.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:29 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:30.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:29 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:30.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:29 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:10:30.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:29 smithi096 ceph-mon[29207]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:10:30.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:29 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:30.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:29 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:30.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:29 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:10:30.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:29 smithi151 ceph-mon[32990]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:10:30.172 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:29 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:30.172 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:29 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:30.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:29 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:10:30.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:29 smithi097 ceph-mon[32720]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:10:32.670 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:32 smithi151 ceph-mon[32990]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:10:32.670 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:32 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:32.670 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:32 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:32.670 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:32 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:10:32.670 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:32 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:10:32.670 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:32 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:32.745 INFO:teuthology.orchestra.run.smithi151.stdout:Created osd(s) 11 on host 'smithi151' 2024-04-25T14:10:32.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:32.743+0000 7f1bd97fa700 1 -- 172.21.15.151:0/2139675271 <== mgr.14152 v2:172.21.15.96:6800/536332425 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f1bdc063ad0 con 0x7f1bc40afb80 2024-04-25T14:10:32.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:32.746+0000 7f1be2972700 1 -- 172.21.15.151:0/2139675271 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1bc40afb80 msgr2=0x7f1bc40b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:32.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:32.746+0000 7f1be2972700 1 --2- 172.21.15.151:0/2139675271 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1bc40afb80 0x7f1bc40b2090 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc00afd0 tx=0x7f1bcc00a380 comp rx=0 tx=0).stop 2024-04-25T14:10:32.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:32.746+0000 7f1be2972700 1 -- 172.21.15.151:0/2139675271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1bdc105340 msgr2=0x7f1bdc19ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:32.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:32.746+0000 7f1be2972700 1 --2- 172.21.15.151:0/2139675271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1bdc105340 0x7f1bdc19ac40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1bd000e9f0 tx=0x7f1bd000ed00 comp rx=0 tx=0).stop 2024-04-25T14:10:32.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:32.746+0000 7f1be2972700 1 -- 172.21.15.151:0/2139675271 shutdown_connections 2024-04-25T14:10:32.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:32.746+0000 7f1be2972700 1 --2- 172.21.15.151:0/2139675271 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1bc40afb80 0x7f1bc40b2090 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:32.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:32.746+0000 7f1be2972700 1 --2- 172.21.15.151:0/2139675271 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bdc105ce0 0x7f1bdc1a1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:32.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:32.746+0000 7f1be2972700 1 --2- 172.21.15.151:0/2139675271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1bdc104210 0x7f1bdc19a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:32.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:32.746+0000 7f1be2972700 1 --2- 172.21.15.151:0/2139675271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1bdc105340 0x7f1bdc19ac40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:32.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:32.746+0000 7f1be2972700 1 -- 172.21.15.151:0/2139675271 >> 172.21.15.151:0/2139675271 conn(0x7f1bdc0ff870 msgr2=0x7f1bdc108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:32.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:32.746+0000 7f1be2972700 1 -- 172.21.15.151:0/2139675271 shutdown_connections 2024-04-25T14:10:32.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:32.746+0000 7f1be2972700 1 -- 172.21.15.151:0/2139675271 wait complete. 2024-04-25T14:10:32.858 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:32 smithi096 ceph-mon[29207]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:10:32.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:32 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:32.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:32 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:32.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:32 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:10:32.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:32 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:10:32.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:32 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:32.875 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:32 smithi097 ceph-mon[32720]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:10:32.875 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:32 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:32.876 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:32 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:32.876 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:32 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:10:32.876 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:32 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:10:32.876 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:32 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:33.341 DEBUG:teuthology.orchestra.run.smithi151:osd.11> sudo journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@osd.11.service 2024-04-25T14:10:33.344 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-25T14:10:33.344 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-25T14:10:33.594 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:10:33.676 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:33 smithi151 ceph-mon[32990]: from='osd.11 [v2:172.21.15.151:6824/2027958746,v1:172.21.15.151:6825/2027958746]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T14:10:33.676 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:33 smithi151 ceph-mon[32990]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T14:10:33.677 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:33 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:33.677 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:33 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:33.677 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:33 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:10:33.717 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:33 smithi096 ceph-mon[29207]: from='osd.11 [v2:172.21.15.151:6824/2027958746,v1:172.21.15.151:6825/2027958746]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T14:10:33.717 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:33 smithi096 ceph-mon[29207]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T14:10:33.717 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:33 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:33.717 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:33 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:33.717 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:33 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:10:33.891 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:33 smithi097 ceph-mon[32720]: from='osd.11 [v2:172.21.15.151:6824/2027958746,v1:172.21.15.151:6825/2027958746]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T14:10:33.891 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:33 smithi097 ceph-mon[32720]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T14:10:33.891 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:33 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:33.892 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:33 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:33.892 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:33 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:10:34.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:34 smithi096 ceph-mon[29207]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:10:34.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:34 smithi096 ceph-mon[29207]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T14:10:34.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:34 smithi096 ceph-mon[29207]: osdmap e69: 12 total, 11 up, 12 in 2024-04-25T14:10:34.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:34 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:10:34.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:34 smithi096 ceph-mon[29207]: from='osd.11 [v2:172.21.15.151:6824/2027958746,v1:172.21.15.151:6825/2027958746]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-04-25T14:10:34.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:34 smithi096 ceph-mon[29207]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-04-25T14:10:34.679 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:34 smithi151 ceph-mon[32990]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:10:34.679 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:34 smithi151 ceph-mon[32990]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T14:10:34.679 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:34 smithi151 ceph-mon[32990]: osdmap e69: 12 total, 11 up, 12 in 2024-04-25T14:10:34.679 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:34 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:10:34.679 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:34 smithi151 ceph-mon[32990]: from='osd.11 [v2:172.21.15.151:6824/2027958746,v1:172.21.15.151:6825/2027958746]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-04-25T14:10:34.679 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:34 smithi151 ceph-mon[32990]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-04-25T14:10:34.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:34 smithi097 ceph-mon[32720]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:10:34.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:34 smithi097 ceph-mon[32720]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T14:10:34.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:34 smithi097 ceph-mon[32720]: osdmap e69: 12 total, 11 up, 12 in 2024-04-25T14:10:34.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:34 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:10:34.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:34 smithi097 ceph-mon[32720]: from='osd.11 [v2:172.21.15.151:6824/2027958746,v1:172.21.15.151:6825/2027958746]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-04-25T14:10:34.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:34 smithi097 ceph-mon[32720]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-04-25T14:10:34.938 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:10:34 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:10:34.888+0000 7f0cab058700 -1 osd.11 0 waiting for initial osdmap 2024-04-25T14:10:34.938 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:10:34 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:10:34.903+0000 7f0ca2e05700 -1 osd.11 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:10:35.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.451+0000 7f369cc44700 1 -- 172.21.15.96:0/3013656936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3698105310 msgr2=0x7f3698105720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:35.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.451+0000 7f369cc44700 1 --2- 172.21.15.96:0/3013656936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3698105310 0x7f3698105720 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f368c009990 tx=0x7f368c009ca0 comp rx=0 tx=0).stop 2024-04-25T14:10:35.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.451+0000 7f369cc44700 1 -- 172.21.15.96:0/3013656936 shutdown_connections 2024-04-25T14:10:35.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.451+0000 7f369cc44700 1 --2- 172.21.15.96:0/3013656936 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3698105cb0 0x7f369810c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:35.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.451+0000 7f369cc44700 1 --2- 172.21.15.96:0/3013656936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36981041e0 0x7f36981045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:35.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.451+0000 7f369cc44700 1 --2- 172.21.15.96:0/3013656936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3698105310 0x7f3698105720 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:35.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.451+0000 7f369cc44700 1 -- 172.21.15.96:0/3013656936 >> 172.21.15.96:0/3013656936 conn(0x7f36980ff840 msgr2=0x7f3698101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:35.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.451+0000 7f369cc44700 1 -- 172.21.15.96:0/3013656936 shutdown_connections 2024-04-25T14:10:35.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.451+0000 7f369cc44700 1 -- 172.21.15.96:0/3013656936 wait complete. 2024-04-25T14:10:35.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.452+0000 7f369cc44700 1 Processor -- start 2024-04-25T14:10:35.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.452+0000 7f369cc44700 1 -- start start 2024-04-25T14:10:35.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.452+0000 7f369cc44700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36981041e0 0x7f369819a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:35.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.452+0000 7f369cc44700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3698105310 0x7f369819acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:35.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.452+0000 7f369cc44700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3698105cb0 0x7f36981a1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:35.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.452+0000 7f369cc44700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f369810fc40 con 0x7f36981041e0 2024-04-25T14:10:35.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.452+0000 7f369cc44700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f369810fac0 con 0x7f3698105cb0 2024-04-25T14:10:35.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.452+0000 7f369cc44700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f369810fdc0 con 0x7f3698105310 2024-04-25T14:10:35.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.453+0000 7f3695d9b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3698105310 0x7f369819acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:35.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.453+0000 7f3696d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3698105cb0 0x7f36981a1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:35.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.453+0000 7f3695d9b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3698105310 0x7f369819acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.96:39372/0 (socket says 172.21.15.96:39372) 2024-04-25T14:10:35.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.453+0000 7f3695d9b700 1 -- 172.21.15.96:0/3955667330 learned_addr learned my addr 172.21.15.96:0/3955667330 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:10:35.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.453+0000 7f369659c700 1 --2- 172.21.15.96:0/3955667330 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36981041e0 0x7f369819a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:35.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.453+0000 7f3695d9b700 1 -- 172.21.15.96:0/3955667330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3698105cb0 msgr2=0x7f36981a1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:35.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.453+0000 7f3695d9b700 1 --2- 172.21.15.96:0/3955667330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3698105cb0 0x7f36981a1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:35.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.453+0000 7f3695d9b700 1 -- 172.21.15.96:0/3955667330 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36981041e0 msgr2=0x7f369819a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:35.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.453+0000 7f3695d9b700 1 --2- 172.21.15.96:0/3955667330 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36981041e0 0x7f369819a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:35.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.453+0000 7f3695d9b700 1 -- 172.21.15.96:0/3955667330 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36981a22e0 con 0x7f3698105310 2024-04-25T14:10:35.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.453+0000 7f3695d9b700 1 --2- 172.21.15.96:0/3955667330 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3698105310 0x7f369819acc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f368c0098e0 tx=0x7f368c0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:35.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.457+0000 7f36877fe700 1 -- 172.21.15.96:0/3955667330 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368c026070 con 0x7f3698105310 2024-04-25T14:10:35.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.457+0000 7f36877fe700 1 -- 172.21.15.96:0/3955667330 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f368c021410 con 0x7f3698105310 2024-04-25T14:10:35.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.457+0000 7f369cc44700 1 -- 172.21.15.96:0/3955667330 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36981a2570 con 0x7f3698105310 2024-04-25T14:10:35.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.457+0000 7f36877fe700 1 -- 172.21.15.96:0/3955667330 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368c017650 con 0x7f3698105310 2024-04-25T14:10:35.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.457+0000 7f369cc44700 1 -- 172.21.15.96:0/3955667330 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36981a2a20 con 0x7f3698105310 2024-04-25T14:10:35.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.458+0000 7f369cc44700 1 -- 172.21.15.96:0/3955667330 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3698051190 con 0x7f3698105310 2024-04-25T14:10:35.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.460+0000 7f36877fe700 1 -- 172.21.15.96:0/3955667330 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f368c029070 con 0x7f3698105310 2024-04-25T14:10:35.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.460+0000 7f36877fe700 1 --2- 172.21.15.96:0/3955667330 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f36800afc00 0x7f36800b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:35.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.460+0000 7f369659c700 1 --2- 172.21.15.96:0/3955667330 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f36800afc00 0x7f36800b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:35.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.460+0000 7f36877fe700 1 -- 172.21.15.96:0/3955667330 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6847+0+0 (secure 0 0 0) 0x7f368c0df780 con 0x7f3698105310 2024-04-25T14:10:35.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.461+0000 7f369659c700 1 --2- 172.21.15.96:0/3955667330 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f36800afc00 0x7f36800b2110 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3698105210 tx=0x7f368800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:35.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.464+0000 7f36877fe700 1 -- 172.21.15.96:0/3955667330 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f368c013070 con 0x7f3698105310 2024-04-25T14:10:35.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:35 smithi096 ceph-mon[29207]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-04-25T14:10:35.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:35 smithi096 ceph-mon[29207]: osdmap e70: 12 total, 11 up, 12 in 2024-04-25T14:10:35.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:35 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:10:35.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:35 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:10:35.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:35 smithi096 ceph-mon[29207]: from='osd.11 ' entity='osd.11' 2024-04-25T14:10:35.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.662+0000 7f369cc44700 1 -- 172.21.15.96:0/3955667330 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f36980642a0 con 0x7f3698105310 2024-04-25T14:10:35.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.662+0000 7f36877fe700 1 -- 172.21.15.96:0/3955667330 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v70) ==== 74+0+133 (secure 0 0 0) 0x7f368c0a8990 con 0x7f3698105310 2024-04-25T14:10:35.662 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:10:35.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.665+0000 7f369cc44700 1 -- 172.21.15.96:0/3955667330 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f36800afc00 msgr2=0x7f36800b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:35.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.665+0000 7f369cc44700 1 --2- 172.21.15.96:0/3955667330 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f36800afc00 0x7f36800b2110 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3698105210 tx=0x7f368800a380 comp rx=0 tx=0).stop 2024-04-25T14:10:35.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.665+0000 7f369cc44700 1 -- 172.21.15.96:0/3955667330 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3698105310 msgr2=0x7f369819acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:35.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.665+0000 7f369cc44700 1 --2- 172.21.15.96:0/3955667330 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3698105310 0x7f369819acc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f368c0098e0 tx=0x7f368c0047b0 comp rx=0 tx=0).stop 2024-04-25T14:10:35.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.665+0000 7f369cc44700 1 -- 172.21.15.96:0/3955667330 shutdown_connections 2024-04-25T14:10:35.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.665+0000 7f369cc44700 1 --2- 172.21.15.96:0/3955667330 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f36800afc00 0x7f36800b2110 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:35.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.665+0000 7f369cc44700 1 --2- 172.21.15.96:0/3955667330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3698105cb0 0x7f36981a1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:35.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.665+0000 7f369cc44700 1 --2- 172.21.15.96:0/3955667330 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36981041e0 0x7f369819a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:35.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.665+0000 7f369cc44700 1 --2- 172.21.15.96:0/3955667330 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3698105310 0x7f369819acc0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:35.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.665+0000 7f369cc44700 1 -- 172.21.15.96:0/3955667330 >> 172.21.15.96:0/3955667330 conn(0x7f36980ff840 msgr2=0x7f3698108540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:35.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.665+0000 7f369cc44700 1 -- 172.21.15.96:0/3955667330 shutdown_connections 2024-04-25T14:10:35.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:35.665+0000 7f369cc44700 1 -- 172.21.15.96:0/3955667330 wait complete. 2024-04-25T14:10:35.915 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:35 smithi151 ceph-mon[32990]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-04-25T14:10:35.915 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:35 smithi151 ceph-mon[32990]: osdmap e70: 12 total, 11 up, 12 in 2024-04-25T14:10:35.915 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:35 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:10:35.915 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:35 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:10:35.916 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:35 smithi151 ceph-mon[32990]: from='osd.11 ' entity='osd.11' 2024-04-25T14:10:35.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:35 smithi097 ceph-mon[32720]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-04-25T14:10:35.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:35 smithi097 ceph-mon[32720]: osdmap e70: 12 total, 11 up, 12 in 2024-04-25T14:10:35.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:35 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:10:35.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:35 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:10:35.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:35 smithi097 ceph-mon[32720]: from='osd.11 ' entity='osd.11' 2024-04-25T14:10:36.431 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":70,"num_osds":12,"num_up_osds":11,"osd_up_since":1714054209,"num_in_osds":12,"osd_in_since":1714054215,"num_remapped_pgs":0} 2024-04-25T14:10:36.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:36 smithi151 ceph-mon[32990]: purged_snaps scrub starts 2024-04-25T14:10:36.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:36 smithi151 ceph-mon[32990]: purged_snaps scrub ok 2024-04-25T14:10:36.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:36 smithi151 ceph-mon[32990]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:10:36.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:36 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:10:36.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:36 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/3955667330' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T14:10:36.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:36 smithi151 ceph-mon[32990]: osd.11 [v2:172.21.15.151:6824/2027958746,v1:172.21.15.151:6825/2027958746] boot 2024-04-25T14:10:36.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:36 smithi151 ceph-mon[32990]: osdmap e71: 12 total, 12 up, 12 in 2024-04-25T14:10:36.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:36 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:10:36.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:36 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:36.689 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:36 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:36.689 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:36 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:10:36.689 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:36 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:10:36.689 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:36 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:10:36.689 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:36 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:10:36.689 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:36 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:36.689 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:36 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:10:36.689 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:36 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:10:36.689 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:36 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:36.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:36 smithi096 ceph-mon[29207]: purged_snaps scrub starts 2024-04-25T14:10:36.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:36 smithi096 ceph-mon[29207]: purged_snaps scrub ok 2024-04-25T14:10:36.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:36 smithi096 ceph-mon[29207]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:10:36.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:36 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:10:36.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:36 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/3955667330' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T14:10:36.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:36 smithi096 ceph-mon[29207]: osd.11 [v2:172.21.15.151:6824/2027958746,v1:172.21.15.151:6825/2027958746] boot 2024-04-25T14:10:36.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:36 smithi096 ceph-mon[29207]: osdmap e71: 12 total, 12 up, 12 in 2024-04-25T14:10:36.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:36 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:10:36.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:36 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:36.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:36 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:36.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:36 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:10:36.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:36 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:10:36.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:36 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:10:36.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:36 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:10:36.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:36 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:36.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:36 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:10:36.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:36 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:10:36.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:36 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:36.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:36 smithi097 ceph-mon[32720]: purged_snaps scrub starts 2024-04-25T14:10:36.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:36 smithi097 ceph-mon[32720]: purged_snaps scrub ok 2024-04-25T14:10:36.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:36 smithi097 ceph-mon[32720]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:10:36.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:36 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:10:36.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:36 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/3955667330' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T14:10:36.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:36 smithi097 ceph-mon[32720]: osd.11 [v2:172.21.15.151:6824/2027958746,v1:172.21.15.151:6825/2027958746] boot 2024-04-25T14:10:36.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:36 smithi097 ceph-mon[32720]: osdmap e71: 12 total, 12 up, 12 in 2024-04-25T14:10:36.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:36 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:10:36.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:36 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:36.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:36 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:36.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:36 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:10:36.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:36 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:10:36.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:36 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:10:36.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:36 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:10:36.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:36 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:36.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:36 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:10:36.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:36 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:10:36.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:36 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:37.433 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-25T14:10:37.686 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:10:37.807 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:37 smithi096 ceph-mon[29207]: Detected new or changed devices on smithi151 2024-04-25T14:10:37.808 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:37 smithi096 ceph-mon[29207]: Adjusting osd_memory_target on smithi151 to 4272M 2024-04-25T14:10:37.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:37 smithi097 ceph-mon[32720]: Detected new or changed devices on smithi151 2024-04-25T14:10:37.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:37 smithi097 ceph-mon[32720]: Adjusting osd_memory_target on smithi151 to 4272M 2024-04-25T14:10:37.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:37 smithi151 ceph-mon[32990]: Detected new or changed devices on smithi151 2024-04-25T14:10:37.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:37 smithi151 ceph-mon[32990]: Adjusting osd_memory_target on smithi151 to 4272M 2024-04-25T14:10:38.839 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:38 smithi096 ceph-mon[29207]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:10:38.840 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:38 smithi096 ceph-mon[29207]: osdmap e72: 12 total, 12 up, 12 in 2024-04-25T14:10:38.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:38 smithi097 ceph-mon[32720]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:10:38.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:38 smithi097 ceph-mon[32720]: osdmap e72: 12 total, 12 up, 12 in 2024-04-25T14:10:38.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:38 smithi151 ceph-mon[32990]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:10:38.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:38 smithi151 ceph-mon[32990]: osdmap e72: 12 total, 12 up, 12 in 2024-04-25T14:10:39.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.361+0000 7fc1077e1700 1 -- 172.21.15.96:0/3746956374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1001030d0 msgr2=0x7fc1001034e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:39.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.361+0000 7fc1077e1700 1 --2- 172.21.15.96:0/3746956374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1001030d0 0x7fc1001034e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc00b0a0 tx=0x7fc0fc00b3b0 comp rx=0 tx=0).stop 2024-04-25T14:10:39.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.361+0000 7fc1077e1700 1 -- 172.21.15.96:0/3746956374 shutdown_connections 2024-04-25T14:10:39.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.361+0000 7fc1077e1700 1 --2- 172.21.15.96:0/3746956374 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc100104ba0 0x7fc10010b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:39.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.361+0000 7fc1077e1700 1 --2- 172.21.15.96:0/3746956374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1001030d0 0x7fc1001034e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:39.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.361+0000 7fc1077e1700 1 --2- 172.21.15.96:0/3746956374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc100104200 0x7fc100104610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:39.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.361+0000 7fc1077e1700 1 -- 172.21.15.96:0/3746956374 >> 172.21.15.96:0/3746956374 conn(0x7fc1000fe710 msgr2=0x7fc100100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:39.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.361+0000 7fc1077e1700 1 -- 172.21.15.96:0/3746956374 shutdown_connections 2024-04-25T14:10:39.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.361+0000 7fc1077e1700 1 -- 172.21.15.96:0/3746956374 wait complete. 2024-04-25T14:10:39.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.362+0000 7fc1077e1700 1 Processor -- start 2024-04-25T14:10:39.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.362+0000 7fc1077e1700 1 -- start start 2024-04-25T14:10:39.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.362+0000 7fc1077e1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1001030d0 0x7fc10006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:39.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.362+0000 7fc1077e1700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc100104200 0x7fc10019f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:39.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.362+0000 7fc1077e1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc100104ba0 0x7fc1001a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:39.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.362+0000 7fc1077e1700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc10010eb20 con 0x7fc100104ba0 2024-04-25T14:10:39.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.362+0000 7fc1077e1700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc10010e9a0 con 0x7fc1001030d0 2024-04-25T14:10:39.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.362+0000 7fc1077e1700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc10010eca0 con 0x7fc100104200 2024-04-25T14:10:39.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.362+0000 7fc105d7e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc100104ba0 0x7fc1001a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:39.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.362+0000 7fc105d7e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc100104ba0 0x7fc1001a1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43872/0 (socket says 172.21.15.96:43872) 2024-04-25T14:10:39.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.362+0000 7fc105d7e700 1 -- 172.21.15.96:0/4209470516 learned_addr learned my addr 172.21.15.96:0/4209470516 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:10:39.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.363+0000 7fc104d7c700 1 --2- 172.21.15.96:0/4209470516 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc100104200 0x7fc10019f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:39.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.363+0000 7fc10557d700 1 --2- 172.21.15.96:0/4209470516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1001030d0 0x7fc10006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:39.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.363+0000 7fc105d7e700 1 -- 172.21.15.96:0/4209470516 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc100104200 msgr2=0x7fc10019f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:39.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.363+0000 7fc105d7e700 1 --2- 172.21.15.96:0/4209470516 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc100104200 0x7fc10019f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:39.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.363+0000 7fc105d7e700 1 -- 172.21.15.96:0/4209470516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1001030d0 msgr2=0x7fc10006c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:39.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.363+0000 7fc105d7e700 1 --2- 172.21.15.96:0/4209470516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1001030d0 0x7fc10006c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:39.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.363+0000 7fc105d7e700 1 -- 172.21.15.96:0/4209470516 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1001a2290 con 0x7fc100104ba0 2024-04-25T14:10:39.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.363+0000 7fc104d7c700 1 --2- 172.21.15.96:0/4209470516 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc100104200 0x7fc10019f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:39.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.363+0000 7fc10557d700 1 --2- 172.21.15.96:0/4209470516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1001030d0 0x7fc10006c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:39.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.363+0000 7fc105d7e700 1 --2- 172.21.15.96:0/4209470516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc100104ba0 0x7fc1001a1af0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc0f800c900 tx=0x7fc0f800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:39.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.363+0000 7fc0f67fc700 1 -- 172.21.15.96:0/4209470516 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8007b00 con 0x7fc100104ba0 2024-04-25T14:10:39.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.363+0000 7fc1077e1700 1 -- 172.21.15.96:0/4209470516 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1001a2580 con 0x7fc100104ba0 2024-04-25T14:10:39.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.363+0000 7fc0f67fc700 1 -- 172.21.15.96:0/4209470516 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0f8007ca0 con 0x7fc100104ba0 2024-04-25T14:10:39.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.363+0000 7fc0f67fc700 1 -- 172.21.15.96:0/4209470516 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8018660 con 0x7fc100104ba0 2024-04-25T14:10:39.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.363+0000 7fc1077e1700 1 -- 172.21.15.96:0/4209470516 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1001a2ac0 con 0x7fc100104ba0 2024-04-25T14:10:39.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.364+0000 7fc1077e1700 1 -- 172.21.15.96:0/4209470516 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc100051190 con 0x7fc100104ba0 2024-04-25T14:10:39.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.367+0000 7fc0f67fc700 1 -- 172.21.15.96:0/4209470516 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fc0f800f450 con 0x7fc100104ba0 2024-04-25T14:10:39.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.367+0000 7fc0f67fc700 1 --2- 172.21.15.96:0/4209470516 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fc0ec0afb80 0x7fc0ec0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:39.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.367+0000 7fc10557d700 1 --2- 172.21.15.96:0/4209470516 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fc0ec0afb80 0x7fc0ec0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:39.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.367+0000 7fc0f67fc700 1 -- 172.21.15.96:0/4209470516 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fc0f80de730 con 0x7fc100104ba0 2024-04-25T14:10:39.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.368+0000 7fc10557d700 1 --2- 172.21.15.96:0/4209470516 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fc0ec0afb80 0x7fc0ec0b2090 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc009fd0 tx=0x7fc0fc0098c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:39.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.371+0000 7fc0f67fc700 1 -- 172.21.15.96:0/4209470516 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc0f8014030 con 0x7fc100104ba0 2024-04-25T14:10:39.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.571+0000 7fc1077e1700 1 -- 172.21.15.96:0/4209470516 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fc10002e510 con 0x7fc100104ba0 2024-04-25T14:10:39.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.571+0000 7fc0f67fc700 1 -- 172.21.15.96:0/4209470516 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) ==== 74+0+133 (secure 0 0 0) 0x7fc0f80a77b0 con 0x7fc100104ba0 2024-04-25T14:10:39.572 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:10:39.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.573+0000 7fc1077e1700 1 -- 172.21.15.96:0/4209470516 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fc0ec0afb80 msgr2=0x7fc0ec0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:39.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.573+0000 7fc1077e1700 1 --2- 172.21.15.96:0/4209470516 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fc0ec0afb80 0x7fc0ec0b2090 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc009fd0 tx=0x7fc0fc0098c0 comp rx=0 tx=0).stop 2024-04-25T14:10:39.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.573+0000 7fc1077e1700 1 -- 172.21.15.96:0/4209470516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc100104ba0 msgr2=0x7fc1001a1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:39.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.573+0000 7fc1077e1700 1 --2- 172.21.15.96:0/4209470516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc100104ba0 0x7fc1001a1af0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc0f800c900 tx=0x7fc0f800ccc0 comp rx=0 tx=0).stop 2024-04-25T14:10:39.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.573+0000 7fc1077e1700 1 -- 172.21.15.96:0/4209470516 shutdown_connections 2024-04-25T14:10:39.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.573+0000 7fc1077e1700 1 --2- 172.21.15.96:0/4209470516 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fc0ec0afb80 0x7fc0ec0b2090 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:39.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.573+0000 7fc1077e1700 1 --2- 172.21.15.96:0/4209470516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc100104ba0 0x7fc1001a1af0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:39.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.573+0000 7fc1077e1700 1 --2- 172.21.15.96:0/4209470516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1001030d0 0x7fc10006c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:39.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.573+0000 7fc1077e1700 1 --2- 172.21.15.96:0/4209470516 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc100104200 0x7fc10019f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:39.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.574+0000 7fc1077e1700 1 -- 172.21.15.96:0/4209470516 >> 172.21.15.96:0/4209470516 conn(0x7fc1000fe710 msgr2=0x7fc100107430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:39.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.574+0000 7fc1077e1700 1 -- 172.21.15.96:0/4209470516 shutdown_connections 2024-04-25T14:10:39.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:39.574+0000 7fc1077e1700 1 -- 172.21.15.96:0/4209470516 wait complete. 2024-04-25T14:10:40.269 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":72,"num_osds":12,"num_up_osds":12,"osd_up_since":1714054235,"num_in_osds":12,"osd_in_since":1714054215,"num_remapped_pgs":0} 2024-04-25T14:10:40.269 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T14:10:40.504 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:10:40.608 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:40 smithi096 ceph-mon[29207]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:10:40.608 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:40 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/4209470516' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T14:10:40.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:40 smithi097 ceph-mon[32720]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:10:40.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:40 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/4209470516' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T14:10:40.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:40 smithi151 ceph-mon[32990]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:10:40.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:40 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/4209470516' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T14:10:42.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.243+0000 7f8c5e084700 1 -- 172.21.15.96:0/3449942809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c58108db0 msgr2=0x7f8c581091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:42.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.243+0000 7f8c5e084700 1 --2- 172.21.15.96:0/3449942809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c58108db0 0x7f8c581091c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8c480099c0 tx=0x7f8c48009cd0 comp rx=0 tx=0).stop 2024-04-25T14:10:42.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.243+0000 7f8c5e084700 1 -- 172.21.15.96:0/3449942809 shutdown_connections 2024-04-25T14:10:42.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.243+0000 7f8c5e084700 1 --2- 172.21.15.96:0/3449942809 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c58109700 0x7f8c58111060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:42.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.243+0000 7f8c5e084700 1 --2- 172.21.15.96:0/3449942809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c58104480 0x7f8c58104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:42.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.243+0000 7f8c5e084700 1 --2- 172.21.15.96:0/3449942809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c58108db0 0x7f8c581091c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:42.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.243+0000 7f8c5e084700 1 -- 172.21.15.96:0/3449942809 >> 172.21.15.96:0/3449942809 conn(0x7f8c580ffd70 msgr2=0x7f8c581021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:42.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.243+0000 7f8c5e084700 1 -- 172.21.15.96:0/3449942809 shutdown_connections 2024-04-25T14:10:42.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.243+0000 7f8c5e084700 1 -- 172.21.15.96:0/3449942809 wait complete. 2024-04-25T14:10:42.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.244+0000 7f8c5e084700 1 Processor -- start 2024-04-25T14:10:42.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.244+0000 7f8c5e084700 1 -- start start 2024-04-25T14:10:42.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.245+0000 7f8c5e084700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c58104480 0x7f8c5819adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:42.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.245+0000 7f8c5e084700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c58108db0 0x7f8c5819b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:42.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.245+0000 7f8c5e084700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c58109700 0x7f8c5819f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:42.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.245+0000 7f8c5e084700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8c58114450 con 0x7f8c58104480 2024-04-25T14:10:42.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.245+0000 7f8c5e084700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8c581142d0 con 0x7f8c58108db0 2024-04-25T14:10:42.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.245+0000 7f8c5e084700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8c581145d0 con 0x7f8c58109700 2024-04-25T14:10:42.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.245+0000 7f8c577fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c58104480 0x7f8c5819adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:42.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.245+0000 7f8c577fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c58104480 0x7f8c5819adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34138/0 (socket says 172.21.15.96:34138) 2024-04-25T14:10:42.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.245+0000 7f8c577fe700 1 -- 172.21.15.96:0/3563100194 learned_addr learned my addr 172.21.15.96:0/3563100194 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:10:42.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.245+0000 7f8c56ffd700 1 --2- 172.21.15.96:0/3563100194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c58108db0 0x7f8c5819b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:42.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.245+0000 7f8c57fff700 1 --2- 172.21.15.96:0/3563100194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c58109700 0x7f8c5819f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:42.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.245+0000 7f8c577fe700 1 -- 172.21.15.96:0/3563100194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c58109700 msgr2=0x7f8c5819f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:42.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.245+0000 7f8c577fe700 1 --2- 172.21.15.96:0/3563100194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c58109700 0x7f8c5819f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:42.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.245+0000 7f8c577fe700 1 -- 172.21.15.96:0/3563100194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c58108db0 msgr2=0x7f8c5819b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:42.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.245+0000 7f8c577fe700 1 --2- 172.21.15.96:0/3563100194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c58108db0 0x7f8c5819b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:42.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.245+0000 7f8c577fe700 1 -- 172.21.15.96:0/3563100194 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c5819fc90 con 0x7f8c58104480 2024-04-25T14:10:42.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.245+0000 7f8c57fff700 1 --2- 172.21.15.96:0/3563100194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c58109700 0x7f8c5819f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:42.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.245+0000 7f8c577fe700 1 --2- 172.21.15.96:0/3563100194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c58104480 0x7f8c5819adc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c00b530 tx=0x7f8c4c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:42.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.245+0000 7f8c54ff9700 1 -- 172.21.15.96:0/3563100194 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c015020 con 0x7f8c58104480 2024-04-25T14:10:42.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.245+0000 7f8c54ff9700 1 -- 172.21.15.96:0/3563100194 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c4c011ce0 con 0x7f8c58104480 2024-04-25T14:10:42.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.245+0000 7f8c5e084700 1 -- 172.21.15.96:0/3563100194 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c5819ff80 con 0x7f8c58104480 2024-04-25T14:10:42.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.245+0000 7f8c54ff9700 1 -- 172.21.15.96:0/3563100194 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c010e40 con 0x7f8c58104480 2024-04-25T14:10:42.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.245+0000 7f8c5e084700 1 -- 172.21.15.96:0/3563100194 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c581a78a0 con 0x7f8c58104480 2024-04-25T14:10:42.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.246+0000 7f8c5e084700 1 -- 172.21.15.96:0/3563100194 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c58051190 con 0x7f8c58104480 2024-04-25T14:10:42.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.248+0000 7f8c54ff9700 1 -- 172.21.15.96:0/3563100194 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f8c4c018020 con 0x7f8c58104480 2024-04-25T14:10:42.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.248+0000 7f8c54ff9700 1 --2- 172.21.15.96:0/3563100194 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f8c400afc00 0x7f8c400b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:42.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.248+0000 7f8c56ffd700 1 --2- 172.21.15.96:0/3563100194 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f8c400afc00 0x7f8c400b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:42.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.249+0000 7f8c54ff9700 1 -- 172.21.15.96:0/3563100194 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f8c4c0de4c0 con 0x7f8c58104480 2024-04-25T14:10:42.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.249+0000 7f8c56ffd700 1 --2- 172.21.15.96:0/3563100194 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f8c400afc00 0x7f8c400b2110 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8c48006010 tx=0x7f8c4800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:42.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.252+0000 7f8c54ff9700 1 -- 172.21.15.96:0/3563100194 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8c4c01f020 con 0x7f8c58104480 2024-04-25T14:10:42.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.449+0000 7f8c5e084700 1 -- 172.21.15.96:0/3563100194 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8c5819c120 con 0x7f8c58104480 2024-04-25T14:10:42.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.450+0000 7f8c54ff9700 1 -- 172.21.15.96:0/3563100194 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+18383 (secure 0 0 0) 0x7f8c4c0a7540 con 0x7f8c58104480 2024-04-25T14:10:42.451 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:10:42.451 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":72,"fsid":"59334182-030c-11ef-bc93-c7b262605968","created":"2024-04-25T14:03:25.680745+0000","modified":"2024-04-25T14:10:37.456185+0000","last_up_change":"2024-04-25T14:10:35.894312+0000","last_in_change":"2024-04-25T14:10:15.583440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:06:47.402927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cf4ba3ac-142e-4e8e-96d4-410d77157eb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6803","nonce":3832506168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6805","nonce":3832506168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832506168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6807","nonce":3832506168}]},"public_addr":"172.21.15.96:6803/3832506168","cluster_addr":"172.21.15.96:6805/3832506168","heartbeat_back_addr":"172.21.15.96:6809/3832506168","heartbeat_front_addr":"172.21.15.96:6807/3832506168","state":["exists","up"]},{"osd":1,"uuid":"866e73b9-4a26-48ba-badf-704fe22f71cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6801","nonce":1083311494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6803","nonce":1083311494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6807","nonce":1083311494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6805","nonce":1083311494}]},"public_addr":"172.21.15.97:6801/1083311494","cluster_addr":"172.21.15.97:6803/1083311494","heartbeat_back_addr":"172.21.15.97:6807/1083311494","heartbeat_front_addr":"172.21.15.97:6805/1083311494","state":["exists","up"]},{"osd":2,"uuid":"45c79ecb-abc4-4dba-bae8-74804e4bc1bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6801","nonce":981902966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6803","nonce":981902966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6807","nonce":981902966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6805","nonce":981902966}]},"public_addr":"172.21.15.151:6801/981902966","cluster_addr":"172.21.15.151:6803/981902966","heartbeat_back_addr":"172.21.15.151:6807/981902966","heartbeat_front_addr":"172.21.15.151:6805/981902966","state":["exists","up"]},{"osd":3,"uuid":"9fc39468-a7ea-4ccf-b365-335ad2ea4c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6811","nonce":2274158748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6813","nonce":2274158748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6817","nonce":2274158748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6815","nonce":2274158748}]},"public_addr":"172.21.15.96:6811/2274158748","cluster_addr":"172.21.15.96:6813/2274158748","heartbeat_back_addr":"172.21.15.96:6817/2274158748","heartbeat_front_addr":"172.21.15.96:6815/2274158748","state":["exists","up"]},{"osd":4,"uuid":"616582f1-12d2-456b-b7b3-e5517800e8a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6809","nonce":3727031962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6811","nonce":3727031962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6815","nonce":3727031962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6813","nonce":3727031962}]},"public_addr":"172.21.15.97:6809/3727031962","cluster_addr":"172.21.15.97:6811/3727031962","heartbeat_back_addr":"172.21.15.97:6815/3727031962","heartbeat_front_addr":"172.21.15.97:6813/3727031962","state":["exists","up"]},{"osd":5,"uuid":"3d277784-ab0f-4c90-81a1-2a4034c52a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6809","nonce":223882546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6811","nonce":223882546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6815","nonce":223882546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6813","nonce":223882546}]},"public_addr":"172.21.15.151:6809/223882546","cluster_addr":"172.21.15.151:6811/223882546","heartbeat_back_addr":"172.21.15.151:6815/223882546","heartbeat_front_addr":"172.21.15.151:6813/223882546","state":["exists","up"]},{"osd":6,"uuid":"9a9ad585-cb1a-4239-9d18-055f59164049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6819","nonce":96197407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6821","nonce":96197407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6825","nonce":96197407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6823","nonce":96197407}]},"public_addr":"172.21.15.96:6819/96197407","cluster_addr":"172.21.15.96:6821/96197407","heartbeat_back_addr":"172.21.15.96:6825/96197407","heartbeat_front_addr":"172.21.15.96:6823/96197407","state":["exists","up"]},{"osd":7,"uuid":"303551c8-6af0-42f2-844f-45bf8db5baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6817","nonce":2247216400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6819","nonce":2247216400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6823","nonce":2247216400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6821","nonce":2247216400}]},"public_addr":"172.21.15.97:6817/2247216400","cluster_addr":"172.21.15.97:6819/2247216400","heartbeat_back_addr":"172.21.15.97:6823/2247216400","heartbeat_front_addr":"172.21.15.97:6821/2247216400","state":["exists","up"]},{"osd":8,"uuid":"81f6cc66-e4ca-430c-b336-b6678d968818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6817","nonce":2549822185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6819","nonce":2549822185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6823","nonce":2549822185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6821","nonce":2549822185}]},"public_addr":"172.21.15.151:6817/2549822185","cluster_addr":"172.21.15.151:6819/2549822185","heartbeat_back_addr":"172.21.15.151:6823/2549822185","heartbeat_front_addr":"172.21.15.151:6821/2549822185","state":["exists","up"]},{"osd":9,"uuid":"29d4da16-feab-4d1c-9d1d-de4f799bd724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6827","nonce":1099399643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6829","nonce":1099399643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6833","nonce":1099399643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6831","nonce":1099399643}]},"public_addr":"172.21.15.96:6827/1099399643","cluster_addr":"172.21.15.96:6829/1099399643","heartbeat_back_addr":"172.21.15.96:6833/1099399643","heartbeat_front_addr":"172.21.15.96:6831/1099399643","state":["exists","up"]},{"osd":10,"uuid":"197fee9b-eee4-4893-be1c-3d57f42250e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6825","nonce":155754520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6827","nonce":155754520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6831","nonce":155754520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6829","nonce":155754520}]},"public_addr":"172.21.15.97:6825/155754520","cluster_addr":"172.21.15.97:6827/155754520","heartbeat_back_addr":"172.21.15.97:6831/155754520","heartbeat_front_addr":"172.21.15.97:6829/155754520","state":["exists","up"]},{"osd":11,"uuid":"9c1d0a83-fc12-41a2-8eb0-80c4abdd2770","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6825","nonce":2027958746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6827","nonce":2027958746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6831","nonce":2027958746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6829","nonce":2027958746}]},"public_addr":"172.21.15.151:6825/2027958746","cluster_addr":"172.21.15.151:6827/2027958746","heartbeat_back_addr":"172.21.15.151:6831/2027958746","heartbeat_front_addr":"172.21.15.151:6829/2027958746","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:05:56.177390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:06:19.917663+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:06:43.608052+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:08.540382+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:33.474072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:58.242306+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:08:24.120442+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:08:49.661064+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:09:15.097271+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:09:41.904505+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:10:08.233252+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:10:33.712296+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/3839807484":"2024-04-26T14:04:21.307574+0000","172.21.15.96:0/1495237551":"2024-04-26T14:04:21.307574+0000","172.21.15.96:0/2793751450":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/129575834":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/2233075139":"2024-04-26T14:04:21.307574+0000","172.21.15.96:6801/2348397481":"2024-04-26T14:03:54.249537+0000","172.21.15.96:6801/433591062":"2024-04-26T14:04:21.307574+0000","172.21.15.96:6800/2348397481":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/3517674924":"2024-04-26T14:03:54.249537+0000","172.21.15.96:6800/433591062":"2024-04-26T14:04:21.307574+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:10:42.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.452+0000 7f8c5e084700 1 -- 172.21.15.96:0/3563100194 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f8c400afc00 msgr2=0x7f8c400b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:42.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.452+0000 7f8c5e084700 1 --2- 172.21.15.96:0/3563100194 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f8c400afc00 0x7f8c400b2110 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8c48006010 tx=0x7f8c4800b540 comp rx=0 tx=0).stop 2024-04-25T14:10:42.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.452+0000 7f8c5e084700 1 -- 172.21.15.96:0/3563100194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c58104480 msgr2=0x7f8c5819adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:42.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.452+0000 7f8c5e084700 1 --2- 172.21.15.96:0/3563100194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c58104480 0x7f8c5819adc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c00b530 tx=0x7f8c4c00b840 comp rx=0 tx=0).stop 2024-04-25T14:10:42.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.453+0000 7f8c5e084700 1 -- 172.21.15.96:0/3563100194 shutdown_connections 2024-04-25T14:10:42.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.453+0000 7f8c5e084700 1 --2- 172.21.15.96:0/3563100194 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f8c400afc00 0x7f8c400b2110 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:42.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.453+0000 7f8c5e084700 1 --2- 172.21.15.96:0/3563100194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c58109700 0x7f8c5819f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:42.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.453+0000 7f8c5e084700 1 --2- 172.21.15.96:0/3563100194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c58104480 0x7f8c5819adc0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:42.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.453+0000 7f8c5e084700 1 --2- 172.21.15.96:0/3563100194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c58108db0 0x7f8c5819b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:42.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.453+0000 7f8c5e084700 1 -- 172.21.15.96:0/3563100194 >> 172.21.15.96:0/3563100194 conn(0x7f8c580ffd70 msgr2=0x7f8c58108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:42.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.453+0000 7f8c5e084700 1 -- 172.21.15.96:0/3563100194 shutdown_connections 2024-04-25T14:10:42.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:42.453+0000 7f8c5e084700 1 -- 172.21.15.96:0/3563100194 wait complete. 2024-04-25T14:10:42.836 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:42 smithi096 ceph-mon[29207]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:10:42.836 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:42 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/3563100194' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:10:42.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:42 smithi097 ceph-mon[32720]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:10:42.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:42 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/3563100194' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:10:42.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:42 smithi151 ceph-mon[32990]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:10:42.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:42 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/3563100194' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:10:43.278 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:06:47.402927+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:10:43.279 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-25T14:10:43.519 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:10:44.733 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:44 smithi096 ceph-mon[29207]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:10:44.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:44 smithi097 ceph-mon[32720]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:10:44.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:44 smithi151 ceph-mon[32990]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:10:45.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.411+0000 7f748d1d1700 1 -- 172.21.15.96:0/1510235879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7488108610 msgr2=0x7f748810ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:45.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.411+0000 7f748d1d1700 1 --2- 172.21.15.96:0/1510235879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7488108610 0x7f748810ff70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f748000b0d0 tx=0x7f748000b3e0 comp rx=0 tx=0).stop 2024-04-25T14:10:45.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.411+0000 7f748d1d1700 1 -- 172.21.15.96:0/1510235879 shutdown_connections 2024-04-25T14:10:45.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.411+0000 7f748d1d1700 1 --2- 172.21.15.96:0/1510235879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7488108610 0x7f748810ff70 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:45.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.411+0000 7f748d1d1700 1 --2- 172.21.15.96:0/1510235879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7488103390 0x7f74881037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:45.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.411+0000 7f748d1d1700 1 --2- 172.21.15.96:0/1510235879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7488107cc0 0x7f74881080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:45.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.411+0000 7f748d1d1700 1 -- 172.21.15.96:0/1510235879 >> 172.21.15.96:0/1510235879 conn(0x7f74880fec90 msgr2=0x7f74881010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:45.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.411+0000 7f748d1d1700 1 -- 172.21.15.96:0/1510235879 shutdown_connections 2024-04-25T14:10:45.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.411+0000 7f748d1d1700 1 -- 172.21.15.96:0/1510235879 wait complete. 2024-04-25T14:10:45.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.412+0000 7f748d1d1700 1 Processor -- start 2024-04-25T14:10:45.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.412+0000 7f748d1d1700 1 -- start start 2024-04-25T14:10:45.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.412+0000 7f748d1d1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7488103390 0x7f748819bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:45.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.412+0000 7f748d1d1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7488107cc0 0x7f748819c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:45.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.412+0000 7f748d1d1700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7488108610 0x7f748819c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:45.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.412+0000 7f748d1d1700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7488113240 con 0x7f7488107cc0 2024-04-25T14:10:45.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.412+0000 7f748d1d1700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f74881130c0 con 0x7f7488103390 2024-04-25T14:10:45.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.412+0000 7f748d1d1700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f74881133c0 con 0x7f7488108610 2024-04-25T14:10:45.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.412+0000 7f748659c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7488107cc0 0x7f748819c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:45.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.412+0000 7f748659c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7488107cc0 0x7f748819c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34148/0 (socket says 172.21.15.96:34148) 2024-04-25T14:10:45.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.412+0000 7f748659c700 1 -- 172.21.15.96:0/3549912804 learned_addr learned my addr 172.21.15.96:0/3549912804 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:10:45.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.412+0000 7f7486d9d700 1 --2- 172.21.15.96:0/3549912804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7488103390 0x7f748819bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:45.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.412+0000 7f748759e700 1 --2- 172.21.15.96:0/3549912804 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7488108610 0x7f748819c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:45.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.413+0000 7f748659c700 1 -- 172.21.15.96:0/3549912804 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7488108610 msgr2=0x7f748819c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:45.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.413+0000 7f748659c700 1 --2- 172.21.15.96:0/3549912804 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7488108610 0x7f748819c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:45.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.413+0000 7f748659c700 1 -- 172.21.15.96:0/3549912804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7488103390 msgr2=0x7f748819bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:45.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.413+0000 7f748659c700 1 --2- 172.21.15.96:0/3549912804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7488103390 0x7f748819bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:45.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.413+0000 7f748659c700 1 -- 172.21.15.96:0/3549912804 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74881a6e30 con 0x7f7488107cc0 2024-04-25T14:10:45.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.413+0000 7f748659c700 1 --2- 172.21.15.96:0/3549912804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7488107cc0 0x7f748819c000 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f747c00d980 tx=0x7f747c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:45.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.413+0000 7f746ffff700 1 -- 172.21.15.96:0/3549912804 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747c017070 con 0x7f7488107cc0 2024-04-25T14:10:45.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.413+0000 7f748d1d1700 1 -- 172.21.15.96:0/3549912804 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7488199910 con 0x7f7488107cc0 2024-04-25T14:10:45.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.413+0000 7f746ffff700 1 -- 172.21.15.96:0/3549912804 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f747c00fd40 con 0x7f7488107cc0 2024-04-25T14:10:45.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.413+0000 7f746ffff700 1 -- 172.21.15.96:0/3549912804 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747c00e3b0 con 0x7f7488107cc0 2024-04-25T14:10:45.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.413+0000 7f748d1d1700 1 -- 172.21.15.96:0/3549912804 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7488199e50 con 0x7f7488107cc0 2024-04-25T14:10:45.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.414+0000 7f748d1d1700 1 -- 172.21.15.96:0/3549912804 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7488051190 con 0x7f7488107cc0 2024-04-25T14:10:45.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.415+0000 7f746ffff700 1 -- 172.21.15.96:0/3549912804 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f747c010610 con 0x7f7488107cc0 2024-04-25T14:10:45.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.416+0000 7f746ffff700 1 --2- 172.21.15.96:0/3549912804 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f74700afb80 0x7f74700b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:45.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.416+0000 7f7486d9d700 1 --2- 172.21.15.96:0/3549912804 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f74700afb80 0x7f74700b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:45.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.416+0000 7f746ffff700 1 -- 172.21.15.96:0/3549912804 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f747c0de190 con 0x7f7488107cc0 2024-04-25T14:10:45.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.416+0000 7f7486d9d700 1 --2- 172.21.15.96:0/3549912804 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f74700afb80 0x7f74700b2090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7478005950 tx=0x7f747800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:45.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.420+0000 7f746ffff700 1 -- 172.21.15.96:0/3549912804 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f747c0a7210 con 0x7f7488107cc0 2024-04-25T14:10:45.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.617+0000 7f748d1d1700 1 -- 172.21.15.96:0/3549912804 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f748802e510 con 0x7f7488107cc0 2024-04-25T14:10:45.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.618+0000 7f746ffff700 1 -- 172.21.15.96:0/3549912804 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) ==== 93+0+10 (secure 0 0 0) 0x7f747c0a6b10 con 0x7f7488107cc0 2024-04-25T14:10:45.618 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-04-25T14:10:45.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.620+0000 7f748d1d1700 1 -- 172.21.15.96:0/3549912804 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f74700afb80 msgr2=0x7f74700b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:45.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.620+0000 7f748d1d1700 1 --2- 172.21.15.96:0/3549912804 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f74700afb80 0x7f74700b2090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7478005950 tx=0x7f747800a300 comp rx=0 tx=0).stop 2024-04-25T14:10:45.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.620+0000 7f748d1d1700 1 -- 172.21.15.96:0/3549912804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7488107cc0 msgr2=0x7f748819c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:45.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.620+0000 7f748d1d1700 1 --2- 172.21.15.96:0/3549912804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7488107cc0 0x7f748819c000 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f747c00d980 tx=0x7f747c00dc90 comp rx=0 tx=0).stop 2024-04-25T14:10:45.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.620+0000 7f748d1d1700 1 -- 172.21.15.96:0/3549912804 shutdown_connections 2024-04-25T14:10:45.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.620+0000 7f748d1d1700 1 --2- 172.21.15.96:0/3549912804 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f74700afb80 0x7f74700b2090 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:45.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.620+0000 7f748d1d1700 1 --2- 172.21.15.96:0/3549912804 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7488108610 0x7f748819c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:45.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.620+0000 7f748d1d1700 1 --2- 172.21.15.96:0/3549912804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7488103390 0x7f748819bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:45.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.620+0000 7f748d1d1700 1 --2- 172.21.15.96:0/3549912804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7488107cc0 0x7f748819c000 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:45.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.620+0000 7f748d1d1700 1 -- 172.21.15.96:0/3549912804 >> 172.21.15.96:0/3549912804 conn(0x7f74880fec90 msgr2=0x7f7488107430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:45.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.620+0000 7f748d1d1700 1 -- 172.21.15.96:0/3549912804 shutdown_connections 2024-04-25T14:10:45.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:45.620+0000 7f748d1d1700 1 -- 172.21.15.96:0/3549912804 wait complete. 2024-04-25T14:10:46.392 INFO:tasks.cephadm:Adding mds.a on smithi096 2024-04-25T14:10:46.392 INFO:tasks.cephadm:Adding mds.d on smithi096 2024-04-25T14:10:46.392 INFO:tasks.cephadm:Adding mds.g on smithi096 2024-04-25T14:10:46.392 INFO:tasks.cephadm:Adding mds.j on smithi096 2024-04-25T14:10:46.393 INFO:tasks.cephadm:Adding mds.b on smithi097 2024-04-25T14:10:46.393 INFO:tasks.cephadm:Adding mds.e on smithi097 2024-04-25T14:10:46.393 INFO:tasks.cephadm:Adding mds.h on smithi097 2024-04-25T14:10:46.393 INFO:tasks.cephadm:Adding mds.k on smithi097 2024-04-25T14:10:46.393 INFO:tasks.cephadm:Adding mds.c on smithi151 2024-04-25T14:10:46.393 INFO:tasks.cephadm:Adding mds.f on smithi151 2024-04-25T14:10:46.393 INFO:tasks.cephadm:Adding mds.i on smithi151 2024-04-25T14:10:46.393 INFO:tasks.cephadm:Adding mds.l on smithi151 2024-04-25T14:10:46.393 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi096=a;smithi096=d;smithi096=g;smithi096=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi151=c;smithi151=f;smithi151=i;smithi151=l' 2024-04-25T14:10:46.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:46 smithi096 ceph-mon[29207]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:10:46.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:46 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/3549912804' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:10:46.648 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:10:46.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:46 smithi151 ceph-mon[32990]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:10:46.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:46 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/3549912804' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:10:46.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:46 smithi097 ceph-mon[32720]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:10:46.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:46 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/3549912804' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:10:48.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.353+0000 7ff4e750a700 1 -- 172.21.15.151:0/4137012108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4e0105cb0 msgr2=0x7ff4e010c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:48.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.353+0000 7ff4e750a700 1 --2- 172.21.15.151:0/4137012108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4e0105cb0 0x7ff4e010c560 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff4d800b0a0 tx=0x7ff4d800b3b0 comp rx=0 tx=0).stop 2024-04-25T14:10:48.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.353+0000 7ff4e750a700 1 -- 172.21.15.151:0/4137012108 shutdown_connections 2024-04-25T14:10:48.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.353+0000 7ff4e750a700 1 --2- 172.21.15.151:0/4137012108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4e0105cb0 0x7ff4e010c560 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:48.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.353+0000 7ff4e750a700 1 --2- 172.21.15.151:0/4137012108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4e01041e0 0x7ff4e01045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:48.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.353+0000 7ff4e750a700 1 --2- 172.21.15.151:0/4137012108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4e0105310 0x7ff4e0105720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:48.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.353+0000 7ff4e750a700 1 -- 172.21.15.151:0/4137012108 >> 172.21.15.151:0/4137012108 conn(0x7ff4e00ff840 msgr2=0x7ff4e0101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:48.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.353+0000 7ff4e750a700 1 -- 172.21.15.151:0/4137012108 shutdown_connections 2024-04-25T14:10:48.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.353+0000 7ff4e750a700 1 -- 172.21.15.151:0/4137012108 wait complete. 2024-04-25T14:10:48.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.353+0000 7ff4e750a700 1 Processor -- start 2024-04-25T14:10:48.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.354+0000 7ff4e750a700 1 -- start start 2024-04-25T14:10:48.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.354+0000 7ff4e750a700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4e01041e0 0x7ff4e019a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:48.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.354+0000 7ff4e750a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4e0105310 0x7ff4e019ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:48.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.354+0000 7ff4e750a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4e0105cb0 0x7ff4e01a1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:48.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.354+0000 7ff4e750a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e010fc40 con 0x7ff4e0105310 2024-04-25T14:10:48.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.354+0000 7ff4e750a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e010fac0 con 0x7ff4e0105cb0 2024-04-25T14:10:48.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.354+0000 7ff4e750a700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e010fdc0 con 0x7ff4e01041e0 2024-04-25T14:10:48.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.354+0000 7ff4e52a6700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4e01041e0 0x7ff4e019a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:48.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.354+0000 7ff4e52a6700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4e01041e0 0x7ff4e019a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44178/0 (socket says 172.21.15.151:44178) 2024-04-25T14:10:48.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.354+0000 7ff4e52a6700 1 -- 172.21.15.151:0/1600414394 learned_addr learned my addr 172.21.15.151:0/1600414394 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-25T14:10:48.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.354+0000 7ff4e4aa5700 1 --2- 172.21.15.151:0/1600414394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4e0105310 0x7ff4e019ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:48.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.354+0000 7ff4e5aa7700 1 --2- 172.21.15.151:0/1600414394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4e0105cb0 0x7ff4e01a1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:48.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.354+0000 7ff4e52a6700 1 -- 172.21.15.151:0/1600414394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4e0105cb0 msgr2=0x7ff4e01a1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:48.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.354+0000 7ff4e52a6700 1 --2- 172.21.15.151:0/1600414394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4e0105cb0 0x7ff4e01a1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:48.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.354+0000 7ff4e52a6700 1 -- 172.21.15.151:0/1600414394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4e0105310 msgr2=0x7ff4e019ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:48.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.354+0000 7ff4e52a6700 1 --2- 172.21.15.151:0/1600414394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4e0105310 0x7ff4e019ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:48.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.354+0000 7ff4e52a6700 1 -- 172.21.15.151:0/1600414394 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e01a21d0 con 0x7ff4e01041e0 2024-04-25T14:10:48.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.354+0000 7ff4e5aa7700 1 --2- 172.21.15.151:0/1600414394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4e0105cb0 0x7ff4e01a1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:10:48.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.354+0000 7ff4e52a6700 1 --2- 172.21.15.151:0/1600414394 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4e01041e0 0x7ff4e019a700 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc00d950 tx=0x7ff4dc00dd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:48.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.354+0000 7ff4d67fc700 1 -- 172.21.15.151:0/1600414394 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc017070 con 0x7ff4e01041e0 2024-04-25T14:10:48.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.355+0000 7ff4e750a700 1 -- 172.21.15.151:0/1600414394 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4e01a24c0 con 0x7ff4e01041e0 2024-04-25T14:10:48.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.355+0000 7ff4d67fc700 1 -- 172.21.15.151:0/1600414394 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4dc00de30 con 0x7ff4e01041e0 2024-04-25T14:10:48.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.355+0000 7ff4d67fc700 1 -- 172.21.15.151:0/1600414394 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc003b10 con 0x7ff4e01041e0 2024-04-25T14:10:48.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.355+0000 7ff4e750a700 1 -- 172.21.15.151:0/1600414394 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4e01a2a50 con 0x7ff4e01041e0 2024-04-25T14:10:48.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.356+0000 7ff4e750a700 1 -- 172.21.15.151:0/1600414394 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4e0051190 con 0x7ff4e01041e0 2024-04-25T14:10:48.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.357+0000 7ff4d67fc700 1 -- 172.21.15.151:0/1600414394 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7ff4dc003cb0 con 0x7ff4e01041e0 2024-04-25T14:10:48.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.358+0000 7ff4d67fc700 1 --2- 172.21.15.151:0/1600414394 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ff4cc0afc00 0x7ff4cc0b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:48.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.358+0000 7ff4d67fc700 1 -- 172.21.15.151:0/1600414394 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7ff4dc0de2d0 con 0x7ff4e01041e0 2024-04-25T14:10:48.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.358+0000 7ff4e4aa5700 1 --2- 172.21.15.151:0/1600414394 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ff4cc0afc00 0x7ff4cc0b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:48.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.358+0000 7ff4e4aa5700 1 --2- 172.21.15.151:0/1600414394 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ff4cc0afc00 0x7ff4cc0b2110 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0005ea0 tx=0x7ff4d0005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:48.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.363+0000 7ff4d67fc700 1 -- 172.21.15.151:0/1600414394 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff4dc020030 con 0x7ff4e01041e0 2024-04-25T14:10:48.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.571+0000 7ff4e750a700 1 -- 172.21.15.151:0/1600414394 --> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi096=a;smithi096=d;smithi096=g;smithi096=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi151=c;smithi151=f;smithi151=i;smithi151=l", "target": ["mon-mgr", ""]}) -- 0x7ff4e0063c90 con 0x7ff4cc0afc00 2024-04-25T14:10:48.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.582+0000 7ff4d67fc700 1 -- 172.21.15.151:0/1600414394 <== mgr.14152 v2:172.21.15.96:6800/536332425 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7ff4e0063c90 con 0x7ff4cc0afc00 2024-04-25T14:10:48.582 INFO:teuthology.orchestra.run.smithi151.stdout:Scheduled mds.all update... 2024-04-25T14:10:48.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.585+0000 7ff4e750a700 1 -- 172.21.15.151:0/1600414394 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ff4cc0afc00 msgr2=0x7ff4cc0b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:48.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.585+0000 7ff4e750a700 1 --2- 172.21.15.151:0/1600414394 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ff4cc0afc00 0x7ff4cc0b2110 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0005ea0 tx=0x7ff4d0005e30 comp rx=0 tx=0).stop 2024-04-25T14:10:48.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.585+0000 7ff4e750a700 1 -- 172.21.15.151:0/1600414394 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4e01041e0 msgr2=0x7ff4e019a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:48.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.585+0000 7ff4e750a700 1 --2- 172.21.15.151:0/1600414394 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4e01041e0 0x7ff4e019a700 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc00d950 tx=0x7ff4dc00dd10 comp rx=0 tx=0).stop 2024-04-25T14:10:48.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.585+0000 7ff4e750a700 1 -- 172.21.15.151:0/1600414394 shutdown_connections 2024-04-25T14:10:48.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.585+0000 7ff4e750a700 1 --2- 172.21.15.151:0/1600414394 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ff4cc0afc00 0x7ff4cc0b2110 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:48.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.585+0000 7ff4e750a700 1 --2- 172.21.15.151:0/1600414394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4e0105cb0 0x7ff4e01a1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:48.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.585+0000 7ff4e750a700 1 --2- 172.21.15.151:0/1600414394 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4e01041e0 0x7ff4e019a700 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:48.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.585+0000 7ff4e750a700 1 --2- 172.21.15.151:0/1600414394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4e0105310 0x7ff4e019ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:48.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.585+0000 7ff4e750a700 1 -- 172.21.15.151:0/1600414394 >> 172.21.15.151:0/1600414394 conn(0x7ff4e00ff840 msgr2=0x7ff4e0108540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:48.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.585+0000 7ff4e750a700 1 -- 172.21.15.151:0/1600414394 shutdown_connections 2024-04-25T14:10:48.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-25T14:10:48.585+0000 7ff4e750a700 1 -- 172.21.15.151:0/1600414394 wait complete. 2024-04-25T14:10:48.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:48 smithi096 ceph-mon[29207]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:10:48.890 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:48 smithi151 ceph-mon[32990]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:10:48.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:48 smithi097 ceph-mon[32720]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:10:49.178 DEBUG:teuthology.orchestra.run.smithi096:mds.a> sudo journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@mds.a.service 2024-04-25T14:10:49.181 DEBUG:teuthology.orchestra.run.smithi096:mds.d> sudo journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@mds.d.service 2024-04-25T14:10:49.224 DEBUG:teuthology.orchestra.run.smithi096:mds.g> sudo journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@mds.g.service 2024-04-25T14:10:49.267 DEBUG:teuthology.orchestra.run.smithi096:mds.j> sudo journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@mds.j.service 2024-04-25T14:10:49.310 DEBUG:teuthology.orchestra.run.smithi097:mds.b> sudo journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@mds.b.service 2024-04-25T14:10:49.313 DEBUG:teuthology.orchestra.run.smithi097:mds.e> sudo journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@mds.e.service 2024-04-25T14:10:49.356 DEBUG:teuthology.orchestra.run.smithi097:mds.h> sudo journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@mds.h.service 2024-04-25T14:10:49.399 DEBUG:teuthology.orchestra.run.smithi097:mds.k> sudo journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@mds.k.service 2024-04-25T14:10:49.442 DEBUG:teuthology.orchestra.run.smithi151:mds.c> sudo journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@mds.c.service 2024-04-25T14:10:49.444 DEBUG:teuthology.orchestra.run.smithi151:mds.f> sudo journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@mds.f.service 2024-04-25T14:10:49.487 DEBUG:teuthology.orchestra.run.smithi151:mds.i> sudo journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@mds.i.service 2024-04-25T14:10:49.530 DEBUG:teuthology.orchestra.run.smithi151:mds.l> sudo journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@mds.l.service 2024-04-25T14:10:49.573 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-25T14:10:49.574 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:10:49.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.690+0000 7f5110e96640 1 -- 172.21.15.96:0/348250965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f510c108940 msgr2=0x7f510c10ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:49.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.690+0000 7f5110e96640 1 --2- 172.21.15.96:0/348250965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f510c108940 0x7f510c10ad60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f50fc009900 tx=0x7f50fc02a410 comp rx=0 tx=0).stop 2024-04-25T14:10:49.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.690+0000 7f5110e96640 1 -- 172.21.15.96:0/348250965 shutdown_connections 2024-04-25T14:10:49.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.690+0000 7f5110e96640 1 --2- 172.21.15.96:0/348250965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f510c108940 0x7f510c10ad60 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:49.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.690+0000 7f5110e96640 1 --2- 172.21.15.96:0/348250965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f510c103c40 0x7f510c104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:49.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.690+0000 7f5110e96640 1 --2- 172.21.15.96:0/348250965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f510c102a00 0x7f510c103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:49.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.690+0000 7f5110e96640 1 -- 172.21.15.96:0/348250965 >> 172.21.15.96:0/348250965 conn(0x7f510c0fdbf0 msgr2=0x7f510c1000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:49.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.690+0000 7f5110e96640 1 -- 172.21.15.96:0/348250965 shutdown_connections 2024-04-25T14:10:49.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.690+0000 7f5110e96640 1 -- 172.21.15.96:0/348250965 wait complete. 2024-04-25T14:10:49.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.691+0000 7f5110e96640 1 Processor -- start 2024-04-25T14:10:49.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.691+0000 7f5110e96640 1 -- start start 2024-04-25T14:10:49.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.691+0000 7f5110e96640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f510c102a00 0x7f510c19a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:49.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.691+0000 7f5110e96640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f510c103c40 0x7f510c19a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:49.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.692+0000 7f5110e96640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f510c108940 0x7f510c1a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:49.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.692+0000 7f5110e96640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f510c10e510 con 0x7f510c103c40 2024-04-25T14:10:49.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.692+0000 7f5110e96640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f510c10e390 con 0x7f510c102a00 2024-04-25T14:10:49.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.692+0000 7f5110e96640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f510c10e690 con 0x7f510c108940 2024-04-25T14:10:49.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.692+0000 7f5109d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f510c103c40 0x7f510c19a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:49.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.692+0000 7f5109d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f510c103c40 0x7f510c19a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34160/0 (socket says 172.21.15.96:34160) 2024-04-25T14:10:49.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.692+0000 7f5109d9b640 1 -- 172.21.15.96:0/1794286221 learned_addr learned my addr 172.21.15.96:0/1794286221 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:10:49.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.692+0000 7f510ad9d640 1 --2- 172.21.15.96:0/1794286221 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f510c108940 0x7f510c1a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:49.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.692+0000 7f510a59c640 1 --2- 172.21.15.96:0/1794286221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f510c102a00 0x7f510c19a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:49.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.692+0000 7f5109d9b640 1 -- 172.21.15.96:0/1794286221 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f510c108940 msgr2=0x7f510c1a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:49.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.692+0000 7f5109d9b640 1 --2- 172.21.15.96:0/1794286221 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f510c108940 0x7f510c1a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:49.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.692+0000 7f5109d9b640 1 -- 172.21.15.96:0/1794286221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f510c102a00 msgr2=0x7f510c19a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:49.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.692+0000 7f5109d9b640 1 --2- 172.21.15.96:0/1794286221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f510c102a00 0x7f510c19a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:49.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.692+0000 7f5109d9b640 1 -- 172.21.15.96:0/1794286221 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f510c1a1cc0 con 0x7f510c103c40 2024-04-25T14:10:49.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.692+0000 7f510a59c640 1 --2- 172.21.15.96:0/1794286221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f510c102a00 0x7f510c19a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:10:49.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.692+0000 7f5109d9b640 1 --2- 172.21.15.96:0/1794286221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f510c103c40 0x7f510c19a6c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f510000d950 tx=0x7f510000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:49.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.692+0000 7f50fb7fe640 1 -- 172.21.15.96:0/1794286221 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5100017070 con 0x7f510c103c40 2024-04-25T14:10:49.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.692+0000 7f50fb7fe640 1 -- 172.21.15.96:0/1794286221 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5100009c30 con 0x7f510c103c40 2024-04-25T14:10:49.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.692+0000 7f5110e96640 1 -- 172.21.15.96:0/1794286221 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f510c1a1fb0 con 0x7f510c103c40 2024-04-25T14:10:49.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.692+0000 7f50fb7fe640 1 -- 172.21.15.96:0/1794286221 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510000f620 con 0x7f510c103c40 2024-04-25T14:10:49.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.692+0000 7f5110e96640 1 -- 172.21.15.96:0/1794286221 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f510c1a2420 con 0x7f510c103c40 2024-04-25T14:10:49.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.693+0000 7f5110e96640 1 -- 172.21.15.96:0/1794286221 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f510c108040 con 0x7f510c103c40 2024-04-25T14:10:49.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.695+0000 7f50fb7fe640 1 -- 172.21.15.96:0/1794286221 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f5100010460 con 0x7f510c103c40 2024-04-25T14:10:49.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.696+0000 7f50fb7fe640 1 --2- 172.21.15.96:0/1794286221 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f50f40afaf0 0x7f50f40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:49.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.696+0000 7f510a59c640 1 --2- 172.21.15.96:0/1794286221 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f50f40afaf0 0x7f50f40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:49.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.696+0000 7f50fb7fe640 1 -- 172.21.15.96:0/1794286221 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f51000de310 con 0x7f510c103c40 2024-04-25T14:10:49.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.696+0000 7f510a59c640 1 --2- 172.21.15.96:0/1794286221 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f50f40afaf0 0x7f50f40b2000 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f510c103600 tx=0x7f5104009530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:49.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.700+0000 7f50fb7fe640 1 -- 172.21.15.96:0/1794286221 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f51000a7390 con 0x7f510c103c40 2024-04-25T14:10:49.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:49 smithi096 ceph-mon[29207]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi096=a;smithi096=d;smithi096=g;smithi096=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi151=c;smithi151=f;smithi151=i;smithi151=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:10:49.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:49 smithi096 ceph-mon[29207]: Saving service mds.all spec with placement smithi096=a;smithi096=d;smithi096=g;smithi096=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi151=c;smithi151=f;smithi151=i;smithi151=l;count:12 2024-04-25T14:10:49.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:49 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:49.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:49 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:10:49.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:49 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:10:49.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:49 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:10:49.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:49 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:49.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:49 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:10:49.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:49 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:10:49.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:49 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:10:49.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:49 smithi096 ceph-mon[29207]: Deploying daemon mds.l on smithi151 2024-04-25T14:10:49.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:49 smithi096 ceph-mon[29207]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:10:49.860 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:49 smithi151 ceph-mon[32990]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi096=a;smithi096=d;smithi096=g;smithi096=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi151=c;smithi151=f;smithi151=i;smithi151=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:10:49.860 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:49 smithi151 ceph-mon[32990]: Saving service mds.all spec with placement smithi096=a;smithi096=d;smithi096=g;smithi096=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi151=c;smithi151=f;smithi151=i;smithi151=l;count:12 2024-04-25T14:10:49.860 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:49 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:49.860 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:49 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:10:49.860 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:49 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:10:49.860 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:49 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:10:49.860 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:49 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:49.860 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:49 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:10:49.861 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:49 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:10:49.861 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:49 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:10:49.861 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:49 smithi151 ceph-mon[32990]: Deploying daemon mds.l on smithi151 2024-04-25T14:10:49.861 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:49 smithi151 ceph-mon[32990]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:10:49.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.880+0000 7f5110e96640 1 -- 172.21.15.96:0/1794286221 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f510c02dda0 con 0x7f510c103c40 2024-04-25T14:10:49.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.882+0000 7f50fb7fe640 1 -- 172.21.15.96:0/1794286221 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+18383 (secure 0 0 0) 0x7f5100009de0 con 0x7f510c103c40 2024-04-25T14:10:49.882 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:10:49.882 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":72,"fsid":"59334182-030c-11ef-bc93-c7b262605968","created":"2024-04-25T14:03:25.680745+0000","modified":"2024-04-25T14:10:37.456185+0000","last_up_change":"2024-04-25T14:10:35.894312+0000","last_in_change":"2024-04-25T14:10:15.583440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:06:47.402927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cf4ba3ac-142e-4e8e-96d4-410d77157eb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6803","nonce":3832506168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6805","nonce":3832506168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832506168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6807","nonce":3832506168}]},"public_addr":"172.21.15.96:6803/3832506168","cluster_addr":"172.21.15.96:6805/3832506168","heartbeat_back_addr":"172.21.15.96:6809/3832506168","heartbeat_front_addr":"172.21.15.96:6807/3832506168","state":["exists","up"]},{"osd":1,"uuid":"866e73b9-4a26-48ba-badf-704fe22f71cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6801","nonce":1083311494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6803","nonce":1083311494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6807","nonce":1083311494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6805","nonce":1083311494}]},"public_addr":"172.21.15.97:6801/1083311494","cluster_addr":"172.21.15.97:6803/1083311494","heartbeat_back_addr":"172.21.15.97:6807/1083311494","heartbeat_front_addr":"172.21.15.97:6805/1083311494","state":["exists","up"]},{"osd":2,"uuid":"45c79ecb-abc4-4dba-bae8-74804e4bc1bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6801","nonce":981902966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6803","nonce":981902966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6807","nonce":981902966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6805","nonce":981902966}]},"public_addr":"172.21.15.151:6801/981902966","cluster_addr":"172.21.15.151:6803/981902966","heartbeat_back_addr":"172.21.15.151:6807/981902966","heartbeat_front_addr":"172.21.15.151:6805/981902966","state":["exists","up"]},{"osd":3,"uuid":"9fc39468-a7ea-4ccf-b365-335ad2ea4c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6811","nonce":2274158748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6813","nonce":2274158748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6817","nonce":2274158748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6815","nonce":2274158748}]},"public_addr":"172.21.15.96:6811/2274158748","cluster_addr":"172.21.15.96:6813/2274158748","heartbeat_back_addr":"172.21.15.96:6817/2274158748","heartbeat_front_addr":"172.21.15.96:6815/2274158748","state":["exists","up"]},{"osd":4,"uuid":"616582f1-12d2-456b-b7b3-e5517800e8a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6809","nonce":3727031962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6811","nonce":3727031962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6815","nonce":3727031962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6813","nonce":3727031962}]},"public_addr":"172.21.15.97:6809/3727031962","cluster_addr":"172.21.15.97:6811/3727031962","heartbeat_back_addr":"172.21.15.97:6815/3727031962","heartbeat_front_addr":"172.21.15.97:6813/3727031962","state":["exists","up"]},{"osd":5,"uuid":"3d277784-ab0f-4c90-81a1-2a4034c52a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6809","nonce":223882546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6811","nonce":223882546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6815","nonce":223882546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6813","nonce":223882546}]},"public_addr":"172.21.15.151:6809/223882546","cluster_addr":"172.21.15.151:6811/223882546","heartbeat_back_addr":"172.21.15.151:6815/223882546","heartbeat_front_addr":"172.21.15.151:6813/223882546","state":["exists","up"]},{"osd":6,"uuid":"9a9ad585-cb1a-4239-9d18-055f59164049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6819","nonce":96197407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6821","nonce":96197407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6825","nonce":96197407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6823","nonce":96197407}]},"public_addr":"172.21.15.96:6819/96197407","cluster_addr":"172.21.15.96:6821/96197407","heartbeat_back_addr":"172.21.15.96:6825/96197407","heartbeat_front_addr":"172.21.15.96:6823/96197407","state":["exists","up"]},{"osd":7,"uuid":"303551c8-6af0-42f2-844f-45bf8db5baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6817","nonce":2247216400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6819","nonce":2247216400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6823","nonce":2247216400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6821","nonce":2247216400}]},"public_addr":"172.21.15.97:6817/2247216400","cluster_addr":"172.21.15.97:6819/2247216400","heartbeat_back_addr":"172.21.15.97:6823/2247216400","heartbeat_front_addr":"172.21.15.97:6821/2247216400","state":["exists","up"]},{"osd":8,"uuid":"81f6cc66-e4ca-430c-b336-b6678d968818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6817","nonce":2549822185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6819","nonce":2549822185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6823","nonce":2549822185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6821","nonce":2549822185}]},"public_addr":"172.21.15.151:6817/2549822185","cluster_addr":"172.21.15.151:6819/2549822185","heartbeat_back_addr":"172.21.15.151:6823/2549822185","heartbeat_front_addr":"172.21.15.151:6821/2549822185","state":["exists","up"]},{"osd":9,"uuid":"29d4da16-feab-4d1c-9d1d-de4f799bd724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6827","nonce":1099399643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6829","nonce":1099399643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6833","nonce":1099399643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6831","nonce":1099399643}]},"public_addr":"172.21.15.96:6827/1099399643","cluster_addr":"172.21.15.96:6829/1099399643","heartbeat_back_addr":"172.21.15.96:6833/1099399643","heartbeat_front_addr":"172.21.15.96:6831/1099399643","state":["exists","up"]},{"osd":10,"uuid":"197fee9b-eee4-4893-be1c-3d57f42250e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6825","nonce":155754520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6827","nonce":155754520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6831","nonce":155754520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6829","nonce":155754520}]},"public_addr":"172.21.15.97:6825/155754520","cluster_addr":"172.21.15.97:6827/155754520","heartbeat_back_addr":"172.21.15.97:6831/155754520","heartbeat_front_addr":"172.21.15.97:6829/155754520","state":["exists","up"]},{"osd":11,"uuid":"9c1d0a83-fc12-41a2-8eb0-80c4abdd2770","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6825","nonce":2027958746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6827","nonce":2027958746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6831","nonce":2027958746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6829","nonce":2027958746}]},"public_addr":"172.21.15.151:6825/2027958746","cluster_addr":"172.21.15.151:6827/2027958746","heartbeat_back_addr":"172.21.15.151:6831/2027958746","heartbeat_front_addr":"172.21.15.151:6829/2027958746","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:05:56.177390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:06:19.917663+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:06:43.608052+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:08.540382+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:33.474072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:58.242306+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:08:24.120442+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:08:49.661064+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:09:15.097271+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:09:41.904505+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:10:08.233252+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:10:33.712296+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/3839807484":"2024-04-26T14:04:21.307574+0000","172.21.15.96:0/1495237551":"2024-04-26T14:04:21.307574+0000","172.21.15.96:0/2793751450":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/129575834":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/2233075139":"2024-04-26T14:04:21.307574+0000","172.21.15.96:6801/2348397481":"2024-04-26T14:03:54.249537+0000","172.21.15.96:6801/433591062":"2024-04-26T14:04:21.307574+0000","172.21.15.96:6800/2348397481":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/3517674924":"2024-04-26T14:03:54.249537+0000","172.21.15.96:6800/433591062":"2024-04-26T14:04:21.307574+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:10:49.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.884+0000 7f5110e96640 1 -- 172.21.15.96:0/1794286221 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f50f40afaf0 msgr2=0x7f50f40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:49.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.884+0000 7f5110e96640 1 --2- 172.21.15.96:0/1794286221 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f50f40afaf0 0x7f50f40b2000 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f510c103600 tx=0x7f5104009530 comp rx=0 tx=0).stop 2024-04-25T14:10:49.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.884+0000 7f5110e96640 1 -- 172.21.15.96:0/1794286221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f510c103c40 msgr2=0x7f510c19a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:49.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.884+0000 7f5110e96640 1 --2- 172.21.15.96:0/1794286221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f510c103c40 0x7f510c19a6c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f510000d950 tx=0x7f510000de10 comp rx=0 tx=0).stop 2024-04-25T14:10:49.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.884+0000 7f5110e96640 1 -- 172.21.15.96:0/1794286221 shutdown_connections 2024-04-25T14:10:49.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.884+0000 7f5110e96640 1 --2- 172.21.15.96:0/1794286221 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f510c108940 0x7f510c1a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:49.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.884+0000 7f5110e96640 1 --2- 172.21.15.96:0/1794286221 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f50f40afaf0 0x7f50f40b2000 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:49.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.884+0000 7f5110e96640 1 --2- 172.21.15.96:0/1794286221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f510c103c40 0x7f510c19a6c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:49.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.884+0000 7f5110e96640 1 --2- 172.21.15.96:0/1794286221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f510c102a00 0x7f510c19a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:49.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.884+0000 7f5110e96640 1 -- 172.21.15.96:0/1794286221 >> 172.21.15.96:0/1794286221 conn(0x7f510c0fdbf0 msgr2=0x7f510c101060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:49.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.884+0000 7f5110e96640 1 -- 172.21.15.96:0/1794286221 shutdown_connections 2024-04-25T14:10:49.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:49.884+0000 7f5110e96640 1 -- 172.21.15.96:0/1794286221 wait complete. 2024-04-25T14:10:49.895 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:06:47.402927+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:10:49.895 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:10:49.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:49 smithi097 ceph-mon[32720]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi096=a;smithi096=d;smithi096=g;smithi096=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi151=c;smithi151=f;smithi151=i;smithi151=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:10:49.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:49 smithi097 ceph-mon[32720]: Saving service mds.all spec with placement smithi096=a;smithi096=d;smithi096=g;smithi096=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi151=c;smithi151=f;smithi151=i;smithi151=l;count:12 2024-04-25T14:10:49.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:49 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:49.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:49 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:10:49.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:49 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:10:49.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:49 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:10:49.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:49 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:49.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:49 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:10:49.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:49 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:10:49.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:49 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:10:49.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:49 smithi097 ceph-mon[32720]: Deploying daemon mds.l on smithi151 2024-04-25T14:10:49.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:49 smithi097 ceph-mon[32720]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:10:50.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.007+0000 7f8e519e3640 1 -- 172.21.15.96:0/1548433916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e4c1029e0 msgr2=0x7f8e4c1036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:50.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.007+0000 7f8e519e3640 1 --2- 172.21.15.96:0/1548433916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e4c1029e0 0x7f8e4c1036e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8e44009900 tx=0x7f8e4402a410 comp rx=0 tx=0).stop 2024-04-25T14:10:50.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.007+0000 7f8e519e3640 1 -- 172.21.15.96:0/1548433916 shutdown_connections 2024-04-25T14:10:50.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.007+0000 7f8e519e3640 1 --2- 172.21.15.96:0/1548433916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e4c108920 0x7f8e4c10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:50.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.007+0000 7f8e519e3640 1 --2- 172.21.15.96:0/1548433916 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e4c103c20 0x7f8e4c104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:50.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.007+0000 7f8e519e3640 1 --2- 172.21.15.96:0/1548433916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e4c1029e0 0x7f8e4c1036e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:50.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.007+0000 7f8e519e3640 1 -- 172.21.15.96:0/1548433916 >> 172.21.15.96:0/1548433916 conn(0x7f8e4c0fdbd0 msgr2=0x7f8e4c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:50.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.007+0000 7f8e519e3640 1 -- 172.21.15.96:0/1548433916 shutdown_connections 2024-04-25T14:10:50.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.007+0000 7f8e519e3640 1 -- 172.21.15.96:0/1548433916 wait complete. 2024-04-25T14:10:50.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.008+0000 7f8e519e3640 1 Processor -- start 2024-04-25T14:10:50.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.008+0000 7f8e519e3640 1 -- start start 2024-04-25T14:10:50.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.008+0000 7f8e519e3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e4c1029e0 0x7f8e4c06bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:50.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.008+0000 7f8e519e3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e4c103c20 0x7f8e4c19f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:50.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.008+0000 7f8e519e3640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e4c108920 0x7f8e4c1a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:50.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.008+0000 7f8e519e3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4c10e4e0 con 0x7f8e4c1029e0 2024-04-25T14:10:50.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.008+0000 7f8e519e3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4c10e360 con 0x7f8e4c103c20 2024-04-25T14:10:50.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.008+0000 7f8e519e3640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4c10e660 con 0x7f8e4c108920 2024-04-25T14:10:50.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.008+0000 7f8e4affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e4c1029e0 0x7f8e4c06bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:50.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.008+0000 7f8e4affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e4c1029e0 0x7f8e4c06bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34178/0 (socket says 172.21.15.96:34178) 2024-04-25T14:10:50.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.008+0000 7f8e4affd640 1 -- 172.21.15.96:0/2457419626 learned_addr learned my addr 172.21.15.96:0/2457419626 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:10:50.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.008+0000 7f8e4b7fe640 1 --2- 172.21.15.96:0/2457419626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e4c108920 0x7f8e4c1a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:50.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.008+0000 7f8e4a7fc640 1 --2- 172.21.15.96:0/2457419626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e4c103c20 0x7f8e4c19f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:50.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.008+0000 7f8e4affd640 1 -- 172.21.15.96:0/2457419626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e4c108920 msgr2=0x7f8e4c1a1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:50.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.008+0000 7f8e4affd640 1 --2- 172.21.15.96:0/2457419626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e4c108920 0x7f8e4c1a1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:50.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.008+0000 7f8e4affd640 1 -- 172.21.15.96:0/2457419626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e4c103c20 msgr2=0x7f8e4c19f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:50.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.008+0000 7f8e4affd640 1 --2- 172.21.15.96:0/2457419626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e4c103c20 0x7f8e4c19f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:50.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.008+0000 7f8e4affd640 1 -- 172.21.15.96:0/2457419626 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e4c1a1d90 con 0x7f8e4c1029e0 2024-04-25T14:10:50.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.009+0000 7f8e4a7fc640 1 --2- 172.21.15.96:0/2457419626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e4c103c20 0x7f8e4c19f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:10:50.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.009+0000 7f8e4affd640 1 --2- 172.21.15.96:0/2457419626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e4c1029e0 0x7f8e4c06bc40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8e44006010 tx=0x7f8e44004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:50.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.009+0000 7f8e509e1640 1 -- 172.21.15.96:0/2457419626 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4403f030 con 0x7f8e4c1029e0 2024-04-25T14:10:50.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.009+0000 7f8e509e1640 1 -- 172.21.15.96:0/2457419626 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e44003aa0 con 0x7f8e4c1029e0 2024-04-25T14:10:50.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.009+0000 7f8e509e1640 1 -- 172.21.15.96:0/2457419626 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e44035070 con 0x7f8e4c1029e0 2024-04-25T14:10:50.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.009+0000 7f8e519e3640 1 -- 172.21.15.96:0/2457419626 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e4c1a2020 con 0x7f8e4c1029e0 2024-04-25T14:10:50.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.009+0000 7f8e519e3640 1 -- 172.21.15.96:0/2457419626 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e4c1a2510 con 0x7f8e4c1029e0 2024-04-25T14:10:50.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.010+0000 7f8e519e3640 1 -- 172.21.15.96:0/2457419626 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e4c107730 con 0x7f8e4c1029e0 2024-04-25T14:10:50.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.011+0000 7f8e509e1640 1 -- 172.21.15.96:0/2457419626 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f8e44042030 con 0x7f8e4c1029e0 2024-04-25T14:10:50.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.012+0000 7f8e509e1640 1 --2- 172.21.15.96:0/2457419626 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f8e340afaf0 0x7f8e340b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:50.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.012+0000 7f8e4a7fc640 1 --2- 172.21.15.96:0/2457419626 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f8e340afaf0 0x7f8e340b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:50.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.012+0000 7f8e509e1640 1 -- 172.21.15.96:0/2457419626 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f8e440ff280 con 0x7f8e4c1029e0 2024-04-25T14:10:50.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.013+0000 7f8e4a7fc640 1 --2- 172.21.15.96:0/2457419626 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f8e340afaf0 0x7f8e340b2000 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c06b5e0 tx=0x7f8e40009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:50.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.016+0000 7f8e509e1640 1 -- 172.21.15.96:0/2457419626 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8e440c8300 con 0x7f8e4c1029e0 2024-04-25T14:10:50.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.193+0000 7f8e519e3640 1 -- 172.21.15.96:0/2457419626 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8e4c02dce0 con 0x7f8e4c1029e0 2024-04-25T14:10:50.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.193+0000 7f8e509e1640 1 -- 172.21.15.96:0/2457419626 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) ==== 93+0+10 (secure 0 0 0) 0x7f8e440c7c00 con 0x7f8e4c1029e0 2024-04-25T14:10:50.193 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-04-25T14:10:50.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.195+0000 7f8e519e3640 1 -- 172.21.15.96:0/2457419626 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f8e340afaf0 msgr2=0x7f8e340b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:50.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.195+0000 7f8e519e3640 1 --2- 172.21.15.96:0/2457419626 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f8e340afaf0 0x7f8e340b2000 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c06b5e0 tx=0x7f8e40009500 comp rx=0 tx=0).stop 2024-04-25T14:10:50.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.195+0000 7f8e519e3640 1 -- 172.21.15.96:0/2457419626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e4c1029e0 msgr2=0x7f8e4c06bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:50.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.195+0000 7f8e519e3640 1 --2- 172.21.15.96:0/2457419626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e4c1029e0 0x7f8e4c06bc40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8e44006010 tx=0x7f8e44004270 comp rx=0 tx=0).stop 2024-04-25T14:10:50.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.195+0000 7f8e519e3640 1 -- 172.21.15.96:0/2457419626 shutdown_connections 2024-04-25T14:10:50.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.195+0000 7f8e519e3640 1 --2- 172.21.15.96:0/2457419626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e4c108920 0x7f8e4c1a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:50.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.195+0000 7f8e519e3640 1 --2- 172.21.15.96:0/2457419626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e4c103c20 0x7f8e4c19f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:50.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.195+0000 7f8e519e3640 1 --2- 172.21.15.96:0/2457419626 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f8e340afaf0 0x7f8e340b2000 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:50.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.195+0000 7f8e519e3640 1 --2- 172.21.15.96:0/2457419626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e4c1029e0 0x7f8e4c06bc40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:50.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.195+0000 7f8e519e3640 1 -- 172.21.15.96:0/2457419626 >> 172.21.15.96:0/2457419626 conn(0x7f8e4c0fdbd0 msgr2=0x7f8e4c104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:50.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.196+0000 7f8e519e3640 1 -- 172.21.15.96:0/2457419626 shutdown_connections 2024-04-25T14:10:50.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.196+0000 7f8e519e3640 1 -- 172.21.15.96:0/2457419626 wait complete. 2024-04-25T14:10:50.207 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:10:50.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.314+0000 7f72ed234640 1 -- 172.21.15.96:0/1107018631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72e8102250 msgr2=0x7f72e810bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:50.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.314+0000 7f72ed234640 1 --2- 172.21.15.96:0/1107018631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72e8102250 0x7f72e810bcb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f72e0009ca0 tx=0x7f72e002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:10:50.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.314+0000 7f72ed234640 1 -- 172.21.15.96:0/1107018631 shutdown_connections 2024-04-25T14:10:50.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.314+0000 7f72ed234640 1 --2- 172.21.15.96:0/1107018631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72e8102250 0x7f72e810bcb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:50.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.314+0000 7f72ed234640 1 --2- 172.21.15.96:0/1107018631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72e8101750 0x7f72e8101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:50.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.314+0000 7f72ed234640 1 --2- 172.21.15.96:0/1107018631 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72e81050a0 0x7f72e81054b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:50.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.314+0000 7f72ed234640 1 -- 172.21.15.96:0/1107018631 >> 172.21.15.96:0/1107018631 conn(0x7f72e80fd370 msgr2=0x7f72e80ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:50.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.314+0000 7f72ed234640 1 -- 172.21.15.96:0/1107018631 shutdown_connections 2024-04-25T14:10:50.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.314+0000 7f72ed234640 1 -- 172.21.15.96:0/1107018631 wait complete. 2024-04-25T14:10:50.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.315+0000 7f72ed234640 1 Processor -- start 2024-04-25T14:10:50.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.315+0000 7f72ed234640 1 -- start start 2024-04-25T14:10:50.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.315+0000 7f72ed234640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72e8101750 0x7f72e819a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:50.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.315+0000 7f72ed234640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72e8102250 0x7f72e819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:50.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.315+0000 7f72ed234640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72e81050a0 0x7f72e81a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:50.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.315+0000 7f72ed234640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f72e810f610 con 0x7f72e8101750 2024-04-25T14:10:50.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.315+0000 7f72ed234640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f72e810f490 con 0x7f72e8102250 2024-04-25T14:10:50.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.315+0000 7f72ed234640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f72e810f790 con 0x7f72e81050a0 2024-04-25T14:10:50.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.315+0000 7f72e6d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72e8101750 0x7f72e819a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:50.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.315+0000 7f72e6d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72e8101750 0x7f72e819a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34208/0 (socket says 172.21.15.96:34208) 2024-04-25T14:10:50.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.315+0000 7f72e6d9d640 1 -- 172.21.15.96:0/88049098 learned_addr learned my addr 172.21.15.96:0/88049098 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:10:50.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.315+0000 7f72e759e640 1 --2- 172.21.15.96:0/88049098 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72e81050a0 0x7f72e81a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:50.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.316+0000 7f72e659c640 1 --2- 172.21.15.96:0/88049098 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72e8102250 0x7f72e819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:50.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.316+0000 7f72e6d9d640 1 -- 172.21.15.96:0/88049098 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72e81050a0 msgr2=0x7f72e81a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:50.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.316+0000 7f72e6d9d640 1 --2- 172.21.15.96:0/88049098 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72e81050a0 0x7f72e81a1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:50.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.316+0000 7f72e6d9d640 1 -- 172.21.15.96:0/88049098 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72e8102250 msgr2=0x7f72e819a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:50.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.316+0000 7f72e6d9d640 1 --2- 172.21.15.96:0/88049098 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72e8102250 0x7f72e819a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:50.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.316+0000 7f72e6d9d640 1 -- 172.21.15.96:0/88049098 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72e81a1c90 con 0x7f72e8101750 2024-04-25T14:10:50.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.316+0000 7f72e659c640 1 --2- 172.21.15.96:0/88049098 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72e8102250 0x7f72e819a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:50.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.316+0000 7f72e6d9d640 1 --2- 172.21.15.96:0/88049098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72e8101750 0x7f72e819a150 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f72d800e9f0 tx=0x7f72d800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:50.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.316+0000 7f72cffff640 1 -- 172.21.15.96:0/88049098 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d800ccc0 con 0x7f72e8101750 2024-04-25T14:10:50.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.316+0000 7f72cffff640 1 -- 172.21.15.96:0/88049098 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72d80035f0 con 0x7f72e8101750 2024-04-25T14:10:50.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.316+0000 7f72ed234640 1 -- 172.21.15.96:0/88049098 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72e81a1f80 con 0x7f72e8101750 2024-04-25T14:10:50.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.316+0000 7f72cffff640 1 -- 172.21.15.96:0/88049098 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d800ce60 con 0x7f72e8101750 2024-04-25T14:10:50.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.316+0000 7f72ed234640 1 -- 172.21.15.96:0/88049098 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72e81a23e0 con 0x7f72e8101750 2024-04-25T14:10:50.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.317+0000 7f72ed234640 1 -- 172.21.15.96:0/88049098 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72e806a1c0 con 0x7f72e8101750 2024-04-25T14:10:50.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.319+0000 7f72cffff640 1 -- 172.21.15.96:0/88049098 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f72d8003aa0 con 0x7f72e8101750 2024-04-25T14:10:50.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.319+0000 7f72cffff640 1 --2- 172.21.15.96:0/88049098 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f72d00afaf0 0x7f72d00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:50.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.319+0000 7f72e659c640 1 --2- 172.21.15.96:0/88049098 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f72d00afaf0 0x7f72d00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:50.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.319+0000 7f72cffff640 1 -- 172.21.15.96:0/88049098 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f72d80debd0 con 0x7f72e8101750 2024-04-25T14:10:50.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.320+0000 7f72e659c640 1 --2- 172.21.15.96:0/88049098 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f72d00afaf0 0x7f72d00b2000 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f72dc005fd0 tx=0x7f72dc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:50.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.323+0000 7f72cffff640 1 -- 172.21.15.96:0/88049098 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f72e806a1c0 con 0x7f72e8101750 2024-04-25T14:10:50.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.537+0000 7f72ed234640 1 -- 172.21.15.96:0/88049098 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72e802dc10 con 0x7f72e8101750 2024-04-25T14:10:50.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.537+0000 7f72cffff640 1 -- 172.21.15.96:0/88049098 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f72d80a7c50 con 0x7f72e8101750 2024-04-25T14:10:50.537 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:10:50.537 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-25T14:10:50.537 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 1 2024-04-25T14:10:50.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.539+0000 7f72ed234640 1 -- 172.21.15.96:0/88049098 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f72d00afaf0 msgr2=0x7f72d00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:50.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.539+0000 7f72ed234640 1 --2- 172.21.15.96:0/88049098 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f72d00afaf0 0x7f72d00b2000 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f72dc005fd0 tx=0x7f72dc005850 comp rx=0 tx=0).stop 2024-04-25T14:10:50.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.539+0000 7f72ed234640 1 -- 172.21.15.96:0/88049098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72e8101750 msgr2=0x7f72e819a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:50.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.539+0000 7f72ed234640 1 --2- 172.21.15.96:0/88049098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72e8101750 0x7f72e819a150 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f72d800e9f0 tx=0x7f72d800eeb0 comp rx=0 tx=0).stop 2024-04-25T14:10:50.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.539+0000 7f72ed234640 1 -- 172.21.15.96:0/88049098 shutdown_connections 2024-04-25T14:10:50.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.539+0000 7f72ed234640 1 --2- 172.21.15.96:0/88049098 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72e81050a0 0x7f72e81a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:50.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.539+0000 7f72ed234640 1 --2- 172.21.15.96:0/88049098 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72e8102250 0x7f72e819a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:50.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.539+0000 7f72ed234640 1 --2- 172.21.15.96:0/88049098 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f72d00afaf0 0x7f72d00b2000 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:50.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.539+0000 7f72ed234640 1 --2- 172.21.15.96:0/88049098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72e8101750 0x7f72e819a150 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:50.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.539+0000 7f72ed234640 1 -- 172.21.15.96:0/88049098 >> 172.21.15.96:0/88049098 conn(0x7f72e80fd370 msgr2=0x7f72e8107c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:50.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.539+0000 7f72ed234640 1 -- 172.21.15.96:0/88049098 shutdown_connections 2024-04-25T14:10:50.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:50.539+0000 7f72ed234640 1 -- 172.21.15.96:0/88049098 wait complete. 2024-04-25T14:10:50.841 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:50 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/1794286221' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:10:50.842 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:50 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/2457419626' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:10:50.842 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:50 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/88049098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:10:50.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:50 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/1794286221' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:10:50.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:50 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/2457419626' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:10:50.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:50 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/88049098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:10:50.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:50 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/1794286221' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:10:50.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:50 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/2457419626' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:10:50.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:50 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/88049098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:10:51.438 INFO:journalctl@ceph.mds.l.smithi151.stdout:Apr 25 14:10:51 smithi151 systemd[1]: Starting Ceph mds.l for 59334182-030c-11ef-bc93-c7b262605968... 2024-04-25T14:10:51.799 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:51 smithi151 ceph-mon[32990]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:10:51.799 INFO:journalctl@ceph.mds.l.smithi151.stdout:Apr 25 14:10:51 smithi151 podman[62911]: 2024-04-25 14:10:51.426306656 +0000 UTC m=+0.017998616 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:10:51.800 INFO:journalctl@ceph.mds.l.smithi151.stdout:Apr 25 14:10:51 smithi151 podman[62911]: 2024-04-25 14:10:51.545787581 +0000 UTC m=+0.137479532 container create 545a1bff6ffd796639bf164e7d7f792db3e24dd31c743ba8197b825dd27df79d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-l, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, ceph=True) 2024-04-25T14:10:51.800 INFO:journalctl@ceph.mds.l.smithi151.stdout:Apr 25 14:10:51 smithi151 podman[62911]: 2024-04-25 14:10:51.765142209 +0000 UTC m=+0.356834161 container init 545a1bff6ffd796639bf164e7d7f792db3e24dd31c743ba8197b825dd27df79d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-l, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-25T14:10:51.800 INFO:journalctl@ceph.mds.l.smithi151.stdout:Apr 25 14:10:51 smithi151 podman[62911]: 2024-04-25 14:10:51.767940417 +0000 UTC m=+0.359632373 container start 545a1bff6ffd796639bf164e7d7f792db3e24dd31c743ba8197b825dd27df79d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-l, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T14:10:51.800 INFO:journalctl@ceph.mds.l.smithi151.stdout:Apr 25 14:10:51 smithi151 ceph-mds[62930]: -- 172.21.15.151:0/536094510 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5635ccfb1a40 con 0x5635ccff9180 2024-04-25T14:10:51.858 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:51 smithi096 ceph-mon[29207]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:10:51.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:51 smithi097 ceph-mon[32720]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:10:52.188 INFO:journalctl@ceph.mds.l.smithi151.stdout:Apr 25 14:10:51 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-l[62926]: starting mds.l at 2024-04-25T14:10:52.188 INFO:journalctl@ceph.mds.l.smithi151.stdout:Apr 25 14:10:51 smithi151 bash[62911]: 545a1bff6ffd796639bf164e7d7f792db3e24dd31c743ba8197b825dd27df79d 2024-04-25T14:10:52.188 INFO:journalctl@ceph.mds.l.smithi151.stdout:Apr 25 14:10:51 smithi151 systemd[1]: Started Ceph mds.l for 59334182-030c-11ef-bc93-c7b262605968. 2024-04-25T14:10:52.552 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:10:52.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.673+0000 7f319bb8f640 1 -- 172.21.15.96:0/3719668175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3194102250 msgr2=0x7f319410bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:52.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.673+0000 7f319bb8f640 1 --2- 172.21.15.96:0/3719668175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3194102250 0x7f319410bcb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f318c009ca0 tx=0x7f318c02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:10:52.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.673+0000 7f319bb8f640 1 -- 172.21.15.96:0/3719668175 shutdown_connections 2024-04-25T14:10:52.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.673+0000 7f319bb8f640 1 --2- 172.21.15.96:0/3719668175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3194102250 0x7f319410bcb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:52.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.673+0000 7f319bb8f640 1 --2- 172.21.15.96:0/3719668175 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3194101750 0x7f3194101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:52.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.673+0000 7f319bb8f640 1 --2- 172.21.15.96:0/3719668175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f31941050a0 0x7f31941054b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:52.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.673+0000 7f319bb8f640 1 -- 172.21.15.96:0/3719668175 >> 172.21.15.96:0/3719668175 conn(0x7f31940fd370 msgr2=0x7f31940ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:52.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.674+0000 7f319bb8f640 1 -- 172.21.15.96:0/3719668175 shutdown_connections 2024-04-25T14:10:52.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.674+0000 7f319bb8f640 1 -- 172.21.15.96:0/3719668175 wait complete. 2024-04-25T14:10:52.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.674+0000 7f319bb8f640 1 Processor -- start 2024-04-25T14:10:52.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.674+0000 7f319bb8f640 1 -- start start 2024-04-25T14:10:52.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.674+0000 7f319bb8f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3194101750 0x7f319419a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:52.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.674+0000 7f319bb8f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3194102250 0x7f319419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:52.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.674+0000 7f319bb8f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31941050a0 0x7f31941a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:52.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.674+0000 7f319bb8f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f319410f630 con 0x7f3194101750 2024-04-25T14:10:52.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.674+0000 7f319bb8f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f319410f4b0 con 0x7f31941050a0 2024-04-25T14:10:52.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.674+0000 7f319bb8f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f319410f7b0 con 0x7f3194102250 2024-04-25T14:10:52.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.675+0000 7f319992b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3194101750 0x7f319419a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:52.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.675+0000 7f319992b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3194101750 0x7f319419a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44472/0 (socket says 172.21.15.96:44472) 2024-04-25T14:10:52.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.675+0000 7f319992b640 1 -- 172.21.15.96:0/1063877156 learned_addr learned my addr 172.21.15.96:0/1063877156 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:10:52.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.675+0000 7f319a12c640 1 --2- 172.21.15.96:0/1063877156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31941050a0 0x7f31941a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:52.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.675+0000 7f319912a640 1 --2- 172.21.15.96:0/1063877156 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3194102250 0x7f319419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:52.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.675+0000 7f319992b640 1 -- 172.21.15.96:0/1063877156 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3194102250 msgr2=0x7f319419a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:52.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.675+0000 7f319992b640 1 --2- 172.21.15.96:0/1063877156 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3194102250 0x7f319419a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:52.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.675+0000 7f319992b640 1 -- 172.21.15.96:0/1063877156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31941050a0 msgr2=0x7f31941a1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:52.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.675+0000 7f319992b640 1 --2- 172.21.15.96:0/1063877156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31941050a0 0x7f31941a1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:52.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.675+0000 7f319992b640 1 -- 172.21.15.96:0/1063877156 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31941a1cb0 con 0x7f3194101750 2024-04-25T14:10:52.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.675+0000 7f319992b640 1 --2- 172.21.15.96:0/1063877156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3194101750 0x7f319419a170 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3190005d00 tx=0x7f319000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:52.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.675+0000 7f318affd640 1 -- 172.21.15.96:0/1063877156 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3190017070 con 0x7f3194101750 2024-04-25T14:10:52.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.675+0000 7f318affd640 1 -- 172.21.15.96:0/1063877156 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3190009c30 con 0x7f3194101750 2024-04-25T14:10:52.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.675+0000 7f319bb8f640 1 -- 172.21.15.96:0/1063877156 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31941a1fa0 con 0x7f3194101750 2024-04-25T14:10:52.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.675+0000 7f318affd640 1 -- 172.21.15.96:0/1063877156 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319000f620 con 0x7f3194101750 2024-04-25T14:10:52.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.675+0000 7f319bb8f640 1 -- 172.21.15.96:0/1063877156 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31941a23b0 con 0x7f3194101750 2024-04-25T14:10:52.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.676+0000 7f319bb8f640 1 -- 172.21.15.96:0/1063877156 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f319406a1c0 con 0x7f3194101750 2024-04-25T14:10:52.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.678+0000 7f318affd640 1 -- 172.21.15.96:0/1063877156 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f319001a070 con 0x7f3194101750 2024-04-25T14:10:52.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.678+0000 7f318affd640 1 --2- 172.21.15.96:0/1063877156 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f31800afaf0 0x7f31800b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:52.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.678+0000 7f319912a640 1 --2- 172.21.15.96:0/1063877156 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f31800afaf0 0x7f31800b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:52.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.678+0000 7f318affd640 1 -- 172.21.15.96:0/1063877156 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f31900de140 con 0x7f3194101750 2024-04-25T14:10:52.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.679+0000 7f319912a640 1 --2- 172.21.15.96:0/1063877156 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f31800afaf0 0x7f31800b2000 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f319419b210 tx=0x7f318400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:52.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.682+0000 7f318affd640 1 -- 172.21.15.96:0/1063877156 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f319406a1c0 con 0x7f3194101750 2024-04-25T14:10:52.850 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:52 smithi097 ceph-mon[32720]: mds.? [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] up:boot 2024-04-25T14:10:52.851 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:52 smithi097 ceph-mon[32720]: fsmap 1 up:standby 2024-04-25T14:10:52.851 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:52 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T14:10:52.851 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:52 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:52.851 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:52 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:52.851 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:52 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:52.851 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:52 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:10:52.851 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:52 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:10:52.851 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:52 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:10:52.851 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:52 smithi097 ceph-mon[32720]: Deploying daemon mds.k on smithi097 2024-04-25T14:10:52.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:52 smithi096 ceph-mon[29207]: mds.? [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] up:boot 2024-04-25T14:10:52.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:52 smithi096 ceph-mon[29207]: fsmap 1 up:standby 2024-04-25T14:10:52.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:52 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T14:10:52.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:52 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:52.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:52 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:52.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:52 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:52.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:52 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:10:52.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:52 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:10:52.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:52 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:10:52.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:52 smithi096 ceph-mon[29207]: Deploying daemon mds.k on smithi097 2024-04-25T14:10:52.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.902+0000 7f319bb8f640 1 -- 172.21.15.96:0/1063877156 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f319402dc10 con 0x7f3194101750 2024-04-25T14:10:52.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.902+0000 7f318affd640 1 -- 172.21.15.96:0/1063877156 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f3190009de0 con 0x7f3194101750 2024-04-25T14:10:52.902 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:10:52.902 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1382721608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1382721608},{"type":"v1","addr":"172.21.15.151:6833","nonce":1382721608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T14:10:52.902 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 2 2024-04-25T14:10:52.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.904+0000 7f319bb8f640 1 -- 172.21.15.96:0/1063877156 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f31800afaf0 msgr2=0x7f31800b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:52.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.904+0000 7f319bb8f640 1 --2- 172.21.15.96:0/1063877156 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f31800afaf0 0x7f31800b2000 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f319419b210 tx=0x7f318400b390 comp rx=0 tx=0).stop 2024-04-25T14:10:52.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.904+0000 7f319bb8f640 1 -- 172.21.15.96:0/1063877156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3194101750 msgr2=0x7f319419a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:52.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.904+0000 7f319bb8f640 1 --2- 172.21.15.96:0/1063877156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3194101750 0x7f319419a170 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3190005d00 tx=0x7f319000dde0 comp rx=0 tx=0).stop 2024-04-25T14:10:52.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.904+0000 7f319bb8f640 1 -- 172.21.15.96:0/1063877156 shutdown_connections 2024-04-25T14:10:52.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.904+0000 7f319bb8f640 1 --2- 172.21.15.96:0/1063877156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31941050a0 0x7f31941a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:52.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.904+0000 7f319bb8f640 1 --2- 172.21.15.96:0/1063877156 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3194102250 0x7f319419a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:52.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.904+0000 7f319bb8f640 1 --2- 172.21.15.96:0/1063877156 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f31800afaf0 0x7f31800b2000 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:52.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.904+0000 7f319bb8f640 1 --2- 172.21.15.96:0/1063877156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3194101750 0x7f319419a170 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:52.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.904+0000 7f319bb8f640 1 -- 172.21.15.96:0/1063877156 >> 172.21.15.96:0/1063877156 conn(0x7f31940fd370 msgr2=0x7f3194107c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:52.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.904+0000 7f319bb8f640 1 -- 172.21.15.96:0/1063877156 shutdown_connections 2024-04-25T14:10:52.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:52.904+0000 7f319bb8f640 1 -- 172.21.15.96:0/1063877156 wait complete. 2024-04-25T14:10:53.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:52 smithi151 ceph-mon[32990]: mds.? [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] up:boot 2024-04-25T14:10:53.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:52 smithi151 ceph-mon[32990]: fsmap 1 up:standby 2024-04-25T14:10:53.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:52 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T14:10:53.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:52 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:53.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:52 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:53.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:52 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:53.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:52 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:10:53.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:52 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:10:53.189 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:52 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:10:53.189 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:52 smithi151 ceph-mon[32990]: Deploying daemon mds.k on smithi097 2024-04-25T14:10:53.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:53 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/1063877156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:10:53.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:53 smithi097 ceph-mon[32720]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:10:54.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:53 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/1063877156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:10:54.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:53 smithi096 ceph-mon[29207]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:10:54.187 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:53 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/1063877156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:10:54.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:53 smithi151 ceph-mon[32990]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:10:54.351 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 14:10:54 smithi097 systemd[1]: Starting Ceph mds.k for 59334182-030c-11ef-bc93-c7b262605968... 2024-04-25T14:10:54.610 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 14:10:54 smithi097 podman[62275]: 2024-04-25 14:10:54.254134732 +0000 UTC m=+0.017959662 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:10:54.610 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 14:10:54 smithi097 podman[62275]: 2024-04-25 14:10:54.367431147 +0000 UTC m=+0.131256052 container create e7a56b736f0f6a390c5347dd97f87272ae8bc80e0e6e9ce608f5f7f2bcd35411 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-k, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-04-25T14:10:54.610 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 14:10:54 smithi097 podman[62275]: 2024-04-25 14:10:54.611616221 +0000 UTC m=+0.375441131 container init e7a56b736f0f6a390c5347dd97f87272ae8bc80e0e6e9ce608f5f7f2bcd35411 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-k, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T14:10:54.917 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:10:54.922 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 14:10:54 smithi097 podman[62275]: 2024-04-25 14:10:54.614317687 +0000 UTC m=+0.378142601 container start e7a56b736f0f6a390c5347dd97f87272ae8bc80e0e6e9ce608f5f7f2bcd35411 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-k, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-25T14:10:54.922 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 14:10:54 smithi097 ceph-mds[62294]: -- 172.21.15.97:0/792007499 <== mon.2 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557567ab9a40 con 0x557567b01600 2024-04-25T14:10:54.922 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 14:10:54 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-k[62290]: starting mds.k at 2024-04-25T14:10:54.923 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 14:10:54 smithi097 bash[62275]: e7a56b736f0f6a390c5347dd97f87272ae8bc80e0e6e9ce608f5f7f2bcd35411 2024-04-25T14:10:54.923 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 14:10:54 smithi097 systemd[1]: Started Ceph mds.k for 59334182-030c-11ef-bc93-c7b262605968. 2024-04-25T14:10:55.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.028+0000 7f0d059b2640 1 -- 172.21.15.96:0/953000113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d00101ef0 msgr2=0x7f0d00106b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:55.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.028+0000 7f0d059b2640 1 --2- 172.21.15.96:0/953000113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d00101ef0 0x7f0d00106b60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4009900 tx=0x7f0cf402a410 comp rx=0 tx=0).stop 2024-04-25T14:10:55.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.028+0000 7f0d059b2640 1 -- 172.21.15.96:0/953000113 shutdown_connections 2024-04-25T14:10:55.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.028+0000 7f0d059b2640 1 --2- 172.21.15.96:0/953000113 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d00109a00 0x7f0d0010be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:55.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.028+0000 7f0d059b2640 1 --2- 172.21.15.96:0/953000113 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d001070a0 0x7f0d001094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:55.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.028+0000 7f0d059b2640 1 --2- 172.21.15.96:0/953000113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d00101ef0 0x7f0d00106b60 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:55.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.028+0000 7f0d059b2640 1 -- 172.21.15.96:0/953000113 >> 172.21.15.96:0/953000113 conn(0x7f0d000fd330 msgr2=0x7f0d000ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:55.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.029+0000 7f0d059b2640 1 -- 172.21.15.96:0/953000113 shutdown_connections 2024-04-25T14:10:55.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.029+0000 7f0d059b2640 1 -- 172.21.15.96:0/953000113 wait complete. 2024-04-25T14:10:55.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.029+0000 7f0d059b2640 1 Processor -- start 2024-04-25T14:10:55.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.029+0000 7f0d059b2640 1 -- start start 2024-04-25T14:10:55.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.030+0000 7f0d059b2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d00101ef0 0x7f0d00104330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:55.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.030+0000 7f0d059b2640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d001070a0 0x7f0d001029d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:55.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.030+0000 7f0d059b2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d00109a00 0x7f0d00102f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:55.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.030+0000 7f0d059b2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0010f650 con 0x7f0d00109a00 2024-04-25T14:10:55.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.030+0000 7f0d059b2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0010f4d0 con 0x7f0d00101ef0 2024-04-25T14:10:55.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.030+0000 7f0d059b2640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0010f7d0 con 0x7f0d001070a0 2024-04-25T14:10:55.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.030+0000 7f0cff7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d00109a00 0x7f0d00102f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:55.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.030+0000 7f0cff7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d00109a00 0x7f0d00102f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44478/0 (socket says 172.21.15.96:44478) 2024-04-25T14:10:55.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.030+0000 7f0cff7fe640 1 -- 172.21.15.96:0/1363265948 learned_addr learned my addr 172.21.15.96:0/1363265948 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:10:55.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.030+0000 7f0cfe7fc640 1 --2- 172.21.15.96:0/1363265948 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d001070a0 0x7f0d001029d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:55.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.030+0000 7f0cfeffd640 1 --2- 172.21.15.96:0/1363265948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d00101ef0 0x7f0d00104330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:55.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.030+0000 7f0cff7fe640 1 -- 172.21.15.96:0/1363265948 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d001070a0 msgr2=0x7f0d001029d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:55.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.030+0000 7f0cff7fe640 1 --2- 172.21.15.96:0/1363265948 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d001070a0 0x7f0d001029d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:55.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.030+0000 7f0cff7fe640 1 -- 172.21.15.96:0/1363265948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d00101ef0 msgr2=0x7f0d00104330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:55.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.030+0000 7f0cff7fe640 1 --2- 172.21.15.96:0/1363265948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d00101ef0 0x7f0d00104330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:55.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.030+0000 7f0cff7fe640 1 -- 172.21.15.96:0/1363265948 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d00103640 con 0x7f0d00109a00 2024-04-25T14:10:55.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.030+0000 7f0cfe7fc640 1 --2- 172.21.15.96:0/1363265948 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d001070a0 0x7f0d001029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:55.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.030+0000 7f0cff7fe640 1 --2- 172.21.15.96:0/1363265948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d00109a00 0x7f0d00102f10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8009fd0 tx=0x7f0cf800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:55.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.030+0000 7f0d049b0640 1 -- 172.21.15.96:0/1363265948 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8017070 con 0x7f0d00109a00 2024-04-25T14:10:55.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.030+0000 7f0d059b2640 1 -- 172.21.15.96:0/1363265948 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d001a4380 con 0x7f0d00109a00 2024-04-25T14:10:55.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.030+0000 7f0d049b0640 1 -- 172.21.15.96:0/1363265948 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cf8003e20 con 0x7f0d00109a00 2024-04-25T14:10:55.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.030+0000 7f0d049b0640 1 -- 172.21.15.96:0/1363265948 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8010460 con 0x7f0d00109a00 2024-04-25T14:10:55.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.030+0000 7f0d059b2640 1 -- 172.21.15.96:0/1363265948 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d001a4840 con 0x7f0d00109a00 2024-04-25T14:10:55.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.031+0000 7f0d059b2640 1 -- 172.21.15.96:0/1363265948 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d00069310 con 0x7f0d00109a00 2024-04-25T14:10:55.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.033+0000 7f0d049b0640 1 -- 172.21.15.96:0/1363265948 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f0cf8003970 con 0x7f0d00109a00 2024-04-25T14:10:55.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.034+0000 7f0d049b0640 1 --2- 172.21.15.96:0/1363265948 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f0ce80afaf0 0x7f0ce80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:55.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.034+0000 7f0cfeffd640 1 --2- 172.21.15.96:0/1363265948 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f0ce80afaf0 0x7f0ce80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:55.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.034+0000 7f0d049b0640 1 -- 172.21.15.96:0/1363265948 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f0cf80de970 con 0x7f0d00109a00 2024-04-25T14:10:55.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.034+0000 7f0cfeffd640 1 --2- 172.21.15.96:0/1363265948 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f0ce80afaf0 0x7f0ce80b2000 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4000c00 tx=0x7f0cf4031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:55.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.042+0000 7f0d049b0640 1 -- 172.21.15.96:0/1363265948 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0cf80a79f0 con 0x7f0d00109a00 2024-04-25T14:10:55.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.267+0000 7f0d059b2640 1 -- 172.21.15.96:0/1363265948 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d0002def0 con 0x7f0d00109a00 2024-04-25T14:10:55.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.267+0000 7f0d049b0640 1 -- 172.21.15.96:0/1363265948 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f0cf80a72f0 con 0x7f0d00109a00 2024-04-25T14:10:55.267 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:10:55.267 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/1995324831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1995324831},{"type":"v1","addr":"172.21.15.97:6833","nonce":1995324831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1382721608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1382721608},{"type":"v1","addr":"172.21.15.151:6833","nonce":1382721608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T14:10:55.268 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 3 2024-04-25T14:10:55.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.270+0000 7f0d059b2640 1 -- 172.21.15.96:0/1363265948 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f0ce80afaf0 msgr2=0x7f0ce80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:55.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.270+0000 7f0d059b2640 1 --2- 172.21.15.96:0/1363265948 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f0ce80afaf0 0x7f0ce80b2000 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4000c00 tx=0x7f0cf4031400 comp rx=0 tx=0).stop 2024-04-25T14:10:55.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.270+0000 7f0d059b2640 1 -- 172.21.15.96:0/1363265948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d00109a00 msgr2=0x7f0d00102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:55.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.270+0000 7f0d059b2640 1 --2- 172.21.15.96:0/1363265948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d00109a00 0x7f0d00102f10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8009fd0 tx=0x7f0cf800b260 comp rx=0 tx=0).stop 2024-04-25T14:10:55.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.271+0000 7f0d059b2640 1 -- 172.21.15.96:0/1363265948 shutdown_connections 2024-04-25T14:10:55.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.271+0000 7f0d059b2640 1 --2- 172.21.15.96:0/1363265948 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f0ce80afaf0 0x7f0ce80b2000 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:55.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.271+0000 7f0d059b2640 1 --2- 172.21.15.96:0/1363265948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d00109a00 0x7f0d00102f10 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:55.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.271+0000 7f0d059b2640 1 --2- 172.21.15.96:0/1363265948 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d001070a0 0x7f0d001029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:55.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.271+0000 7f0d059b2640 1 --2- 172.21.15.96:0/1363265948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d00101ef0 0x7f0d00104330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:55.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.271+0000 7f0d059b2640 1 -- 172.21.15.96:0/1363265948 >> 172.21.15.96:0/1363265948 conn(0x7f0d000fd330 msgr2=0x7f0d00105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:55.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.271+0000 7f0d059b2640 1 -- 172.21.15.96:0/1363265948 shutdown_connections 2024-04-25T14:10:55.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:55.271+0000 7f0d059b2640 1 -- 172.21.15.96:0/1363265948 wait complete. 2024-04-25T14:10:56.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:55 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:56.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:55 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:56.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:55 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:56.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:55 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:10:56.110 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:55 smithi096 ceph-mon[29207]: mds.? [v2:172.21.15.97:6832/1995324831,v1:172.21.15.97:6833/1995324831] up:boot 2024-04-25T14:10:56.110 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:55 smithi096 ceph-mon[29207]: fsmap 2 up:standby 2024-04-25T14:10:56.110 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:55 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T14:10:56.110 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:55 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:10:56.110 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:55 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:10:56.110 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:55 smithi096 ceph-mon[29207]: Deploying daemon mds.j on smithi096 2024-04-25T14:10:56.110 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:55 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/1363265948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:10:56.110 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:55 smithi096 ceph-mon[29207]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:10:56.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:55 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:56.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:55 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:56.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:55 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:56.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:55 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:10:56.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:55 smithi097 ceph-mon[32720]: mds.? [v2:172.21.15.97:6832/1995324831,v1:172.21.15.97:6833/1995324831] up:boot 2024-04-25T14:10:56.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:55 smithi097 ceph-mon[32720]: fsmap 2 up:standby 2024-04-25T14:10:56.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:55 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T14:10:56.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:55 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:10:56.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:55 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:10:56.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:55 smithi097 ceph-mon[32720]: Deploying daemon mds.j on smithi096 2024-04-25T14:10:56.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:55 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/1363265948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:10:56.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:55 smithi097 ceph-mon[32720]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:10:56.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:55 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:56.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:55 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:56.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:55 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:56.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:55 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:10:56.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:55 smithi151 ceph-mon[32990]: mds.? [v2:172.21.15.97:6832/1995324831,v1:172.21.15.97:6833/1995324831] up:boot 2024-04-25T14:10:56.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:55 smithi151 ceph-mon[32990]: fsmap 2 up:standby 2024-04-25T14:10:56.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:55 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T14:10:56.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:55 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:10:56.189 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:55 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:10:56.189 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:55 smithi151 ceph-mon[32990]: Deploying daemon mds.j on smithi096 2024-04-25T14:10:56.189 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:55 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/1363265948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:10:56.189 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:55 smithi151 ceph-mon[32990]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:10:57.109 INFO:journalctl@ceph.mds.j.smithi096.stdout:Apr 25 14:10:57 smithi096 systemd[1]: Starting Ceph mds.j for 59334182-030c-11ef-bc93-c7b262605968... 2024-04-25T14:10:57.284 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:10:57.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.397+0000 7fba4676c640 1 -- 172.21.15.96:0/3960687751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba401097f0 msgr2=0x7fba4010bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:57.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.397+0000 7fba4676c640 1 --2- 172.21.15.96:0/3960687751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba401097f0 0x7fba4010bd00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fba38009ca0 tx=0x7fba3802a8e0 comp rx=0 tx=0).stop 2024-04-25T14:10:57.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.398+0000 7fba4676c640 1 -- 172.21.15.96:0/3960687751 shutdown_connections 2024-04-25T14:10:57.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.398+0000 7fba4676c640 1 --2- 172.21.15.96:0/3960687751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba401097f0 0x7fba4010bd00 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:57.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.398+0000 7fba4676c640 1 --2- 172.21.15.96:0/3960687751 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba40106c80 0x7fba40109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:57.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.398+0000 7fba4676c640 1 --2- 172.21.15.96:0/3960687751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba40101ea0 0x7fba401022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:57.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.398+0000 7fba4676c640 1 -- 172.21.15.96:0/3960687751 >> 172.21.15.96:0/3960687751 conn(0x7fba400fd3b0 msgr2=0x7fba400ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:57.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.398+0000 7fba4676c640 1 -- 172.21.15.96:0/3960687751 shutdown_connections 2024-04-25T14:10:57.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.398+0000 7fba4676c640 1 -- 172.21.15.96:0/3960687751 wait complete. 2024-04-25T14:10:57.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.398+0000 7fba4676c640 1 Processor -- start 2024-04-25T14:10:57.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.398+0000 7fba4676c640 1 -- start start 2024-04-25T14:10:57.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.399+0000 7fba4676c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba40101ea0 0x7fba4019a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:57.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.399+0000 7fba4676c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba40106c80 0x7fba4019a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:57.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.399+0000 7fba4676c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba401097f0 0x7fba401a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:57.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.399+0000 7fba4676c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fba4010f6e0 con 0x7fba40101ea0 2024-04-25T14:10:57.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.399+0000 7fba4676c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fba4010f560 con 0x7fba401097f0 2024-04-25T14:10:57.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.399+0000 7fba4676c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fba4010f860 con 0x7fba40106c80 2024-04-25T14:10:57.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.399+0000 7fba3ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba40101ea0 0x7fba4019a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:57.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.399+0000 7fba3ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba40101ea0 0x7fba4019a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44488/0 (socket says 172.21.15.96:44488) 2024-04-25T14:10:57.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.399+0000 7fba3ffff640 1 -- 172.21.15.96:0/2021771179 learned_addr learned my addr 172.21.15.96:0/2021771179 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:10:57.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.399+0000 7fba3f7fe640 1 --2- 172.21.15.96:0/2021771179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba40106c80 0x7fba4019a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:57.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.399+0000 7fba44d09640 1 --2- 172.21.15.96:0/2021771179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba401097f0 0x7fba401a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:57.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.399+0000 7fba3ffff640 1 -- 172.21.15.96:0/2021771179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba40106c80 msgr2=0x7fba4019a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:57.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.399+0000 7fba3ffff640 1 --2- 172.21.15.96:0/2021771179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba40106c80 0x7fba4019a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:57.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.399+0000 7fba3ffff640 1 -- 172.21.15.96:0/2021771179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba401097f0 msgr2=0x7fba401a1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:57.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.399+0000 7fba3ffff640 1 --2- 172.21.15.96:0/2021771179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba401097f0 0x7fba401a1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:57.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.399+0000 7fba3ffff640 1 -- 172.21.15.96:0/2021771179 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba401a1e00 con 0x7fba40101ea0 2024-04-25T14:10:57.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.399+0000 7fba44d09640 1 --2- 172.21.15.96:0/2021771179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba401097f0 0x7fba401a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:57.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.399+0000 7fba3ffff640 1 --2- 172.21.15.96:0/2021771179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba40101ea0 0x7fba4019a2c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fba3000b470 tx=0x7fba3000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:57.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.399+0000 7fba3d7fa640 1 -- 172.21.15.96:0/2021771179 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba30015020 con 0x7fba40101ea0 2024-04-25T14:10:57.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.399+0000 7fba3d7fa640 1 -- 172.21.15.96:0/2021771179 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba30011ce0 con 0x7fba40101ea0 2024-04-25T14:10:57.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.399+0000 7fba3d7fa640 1 -- 172.21.15.96:0/2021771179 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba30010de0 con 0x7fba40101ea0 2024-04-25T14:10:57.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.399+0000 7fba4676c640 1 -- 172.21.15.96:0/2021771179 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba401a20f0 con 0x7fba40101ea0 2024-04-25T14:10:57.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.399+0000 7fba4676c640 1 -- 172.21.15.96:0/2021771179 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba401a2670 con 0x7fba40101ea0 2024-04-25T14:10:57.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.401+0000 7fba4676c640 1 -- 172.21.15.96:0/2021771179 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba400feed0 con 0x7fba40101ea0 2024-04-25T14:10:57.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.402+0000 7fba3d7fa640 1 -- 172.21.15.96:0/2021771179 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fba30018020 con 0x7fba40101ea0 2024-04-25T14:10:57.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.403+0000 7fba3d7fa640 1 --2- 172.21.15.96:0/2021771179 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fba280afaf0 0x7fba280b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:57.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.403+0000 7fba3f7fe640 1 --2- 172.21.15.96:0/2021771179 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fba280afaf0 0x7fba280b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:57.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.403+0000 7fba3d7fa640 1 -- 172.21.15.96:0/2021771179 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fba300de5f0 con 0x7fba40101ea0 2024-04-25T14:10:57.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.403+0000 7fba3f7fe640 1 --2- 172.21.15.96:0/2021771179 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fba280afaf0 0x7fba280b2000 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fba4019b360 tx=0x7fba34009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:57.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.406+0000 7fba3d7fa640 1 -- 172.21.15.96:0/2021771179 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fba3001f020 con 0x7fba40101ea0 2024-04-25T14:10:57.505 INFO:journalctl@ceph.mds.j.smithi096.stdout:Apr 25 14:10:57 smithi096 podman[69431]: 2024-04-25 14:10:57.132884144 +0000 UTC m=+0.017840463 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:10:57.505 INFO:journalctl@ceph.mds.j.smithi096.stdout:Apr 25 14:10:57 smithi096 podman[69431]: 2024-04-25 14:10:57.26226925 +0000 UTC m=+0.147225614 container create dc649cef1a74dc258a315697e69864118ef09dcb85512ad037bfee8104d683ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-25T14:10:57.505 INFO:journalctl@ceph.mds.j.smithi096.stdout:Apr 25 14:10:57 smithi096 podman[69431]: 2024-04-25 14:10:57.506809665 +0000 UTC m=+0.391765984 container init dc649cef1a74dc258a315697e69864118ef09dcb85512ad037bfee8104d683ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0) 2024-04-25T14:10:57.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.648+0000 7fba4676c640 1 -- 172.21.15.96:0/2021771179 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba4002db50 con 0x7fba40101ea0 2024-04-25T14:10:57.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.649+0000 7fba3d7fa640 1 -- 172.21.15.96:0/2021771179 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7fba300a7670 con 0x7fba40101ea0 2024-04-25T14:10:57.649 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:10:57.649 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/1995324831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1995324831},{"type":"v1","addr":"172.21.15.97:6833","nonce":1995324831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1382721608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1382721608},{"type":"v1","addr":"172.21.15.151:6833","nonce":1382721608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T14:10:57.649 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 3 2024-04-25T14:10:57.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.651+0000 7fba4676c640 1 -- 172.21.15.96:0/2021771179 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fba280afaf0 msgr2=0x7fba280b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:57.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.651+0000 7fba4676c640 1 --2- 172.21.15.96:0/2021771179 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fba280afaf0 0x7fba280b2000 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fba4019b360 tx=0x7fba34009110 comp rx=0 tx=0).stop 2024-04-25T14:10:57.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.651+0000 7fba4676c640 1 -- 172.21.15.96:0/2021771179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba40101ea0 msgr2=0x7fba4019a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:57.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.651+0000 7fba4676c640 1 --2- 172.21.15.96:0/2021771179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba40101ea0 0x7fba4019a2c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fba3000b470 tx=0x7fba3000b870 comp rx=0 tx=0).stop 2024-04-25T14:10:57.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.651+0000 7fba4676c640 1 -- 172.21.15.96:0/2021771179 shutdown_connections 2024-04-25T14:10:57.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.651+0000 7fba4676c640 1 --2- 172.21.15.96:0/2021771179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba401097f0 0x7fba401a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:57.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.651+0000 7fba4676c640 1 --2- 172.21.15.96:0/2021771179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba40106c80 0x7fba4019a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:57.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.651+0000 7fba4676c640 1 --2- 172.21.15.96:0/2021771179 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fba280afaf0 0x7fba280b2000 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:57.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.651+0000 7fba4676c640 1 --2- 172.21.15.96:0/2021771179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba40101ea0 0x7fba4019a2c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:57.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.651+0000 7fba4676c640 1 -- 172.21.15.96:0/2021771179 >> 172.21.15.96:0/2021771179 conn(0x7fba400fd3b0 msgr2=0x7fba401078f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:57.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.651+0000 7fba4676c640 1 -- 172.21.15.96:0/2021771179 shutdown_connections 2024-04-25T14:10:57.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:57.651+0000 7fba4676c640 1 -- 172.21.15.96:0/2021771179 wait complete. 2024-04-25T14:10:57.859 INFO:journalctl@ceph.mds.j.smithi096.stdout:Apr 25 14:10:57 smithi096 podman[69431]: 2024-04-25 14:10:57.509502189 +0000 UTC m=+0.394458508 container start dc649cef1a74dc258a315697e69864118ef09dcb85512ad037bfee8104d683ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-j, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-25T14:10:57.859 INFO:journalctl@ceph.mds.j.smithi096.stdout:Apr 25 14:10:57 smithi096 ceph-mds[69488]: -- 172.21.15.96:0/3870576679 <== mon.0 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c720185a40 con 0x55c7201cda80 2024-04-25T14:10:57.859 INFO:journalctl@ceph.mds.j.smithi096.stdout:Apr 25 14:10:57 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-j[69465]: starting mds.j at 2024-04-25T14:10:57.859 INFO:journalctl@ceph.mds.j.smithi096.stdout:Apr 25 14:10:57 smithi096 bash[69431]: dc649cef1a74dc258a315697e69864118ef09dcb85512ad037bfee8104d683ff 2024-04-25T14:10:57.859 INFO:journalctl@ceph.mds.j.smithi096.stdout:Apr 25 14:10:57 smithi096 systemd[1]: Started Ceph mds.j for 59334182-030c-11ef-bc93-c7b262605968. 2024-04-25T14:10:58.585 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:58 smithi096 ceph-mon[29207]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:10:58.585 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:58 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/2021771179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:10:58.585 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:58 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:58.585 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:58 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:58.586 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:58 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:58.586 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:58 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:10:58.586 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:58 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:10:58.586 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:58 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:10:58.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:58 smithi097 ceph-mon[32720]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:10:58.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:58 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/2021771179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:10:58.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:58 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:58.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:58 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:58.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:58 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:58.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:58 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:10:58.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:58 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:10:58.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:58 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:10:58.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:58 smithi151 ceph-mon[32990]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:10:58.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:58 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/2021771179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:10:58.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:58 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:58.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:58 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:58.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:58 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:10:58.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:58 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:10:58.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:58 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:10:58.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:58 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:10:59.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:59 smithi096 ceph-mon[29207]: Deploying daemon mds.d on smithi096 2024-04-25T14:10:59.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:59 smithi096 ceph-mon[29207]: mds.? [v2:172.21.15.96:6834/606227917,v1:172.21.15.96:6835/606227917] up:boot 2024-04-25T14:10:59.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:59 smithi096 ceph-mon[29207]: fsmap 3 up:standby 2024-04-25T14:10:59.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:10:59 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T14:10:59.663 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:10:59.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:59.792+0000 7fa0e0dd7640 1 -- 172.21.15.96:0/3276492536 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0dc104450 msgr2=0x7fa0dc10ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:59.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:59.792+0000 7fa0e0dd7640 1 --2- 172.21.15.96:0/3276492536 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0dc104450 0x7fa0dc10ab90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc009ca0 tx=0x7fa0cc02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:10:59.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:59.792+0000 7fa0e0dd7640 1 -- 172.21.15.96:0/3276492536 shutdown_connections 2024-04-25T14:10:59.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:59.792+0000 7fa0e0dd7640 1 --2- 172.21.15.96:0/3276492536 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0dc104450 0x7fa0dc10ab90 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:59.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:59.792+0000 7fa0e0dd7640 1 --2- 172.21.15.96:0/3276492536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa0dc103910 0x7fa0dc103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:59.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:59.792+0000 7fa0e0dd7640 1 --2- 172.21.15.96:0/3276492536 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0dc102f30 0x7fa0dc103340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:59.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:59.792+0000 7fa0e0dd7640 1 -- 172.21.15.96:0/3276492536 >> 172.21.15.96:0/3276492536 conn(0x7fa0dc0fd330 msgr2=0x7fa0dc0ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:59.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:59.792+0000 7fa0e0dd7640 1 -- 172.21.15.96:0/3276492536 shutdown_connections 2024-04-25T14:10:59.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:59.792+0000 7fa0e0dd7640 1 -- 172.21.15.96:0/3276492536 wait complete. 2024-04-25T14:10:59.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:59.793+0000 7fa0e0dd7640 1 Processor -- start 2024-04-25T14:10:59.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:59.793+0000 7fa0e0dd7640 1 -- start start 2024-04-25T14:10:59.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:59.793+0000 7fa0e0dd7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0dc102f30 0x7fa0dc19a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:59.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:59.793+0000 7fa0e0dd7640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0dc103910 0x7fa0dc19a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:59.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:59.793+0000 7fa0e0dd7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa0dc104450 0x7fa0dc1a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:59.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:59.793+0000 7fa0e0dd7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc10e520 con 0x7fa0dc102f30 2024-04-25T14:10:59.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:59.793+0000 7fa0e0dd7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc10e3a0 con 0x7fa0dc104450 2024-04-25T14:10:59.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:59.793+0000 7fa0e0dd7640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc10e6a0 con 0x7fa0dc103910 2024-04-25T14:10:59.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:59.793+0000 7fa0da59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0dc102f30 0x7fa0dc19a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:59.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:59.793+0000 7fa0da59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0dc102f30 0x7fa0dc19a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44538/0 (socket says 172.21.15.96:44538) 2024-04-25T14:10:59.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:59.793+0000 7fa0da59c640 1 -- 172.21.15.96:0/668898676 learned_addr learned my addr 172.21.15.96:0/668898676 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:10:59.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:59.794+0000 7fa0d9d9b640 1 --2- 172.21.15.96:0/668898676 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0dc103910 0x7fa0dc19a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:59.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:59.794+0000 7fa0dad9d640 1 --2- 172.21.15.96:0/668898676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa0dc104450 0x7fa0dc1a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:59.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:59.794+0000 7fa0da59c640 1 -- 172.21.15.96:0/668898676 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0dc103910 msgr2=0x7fa0dc19a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:59.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:59.794+0000 7fa0da59c640 1 --2- 172.21.15.96:0/668898676 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0dc103910 0x7fa0dc19a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:59.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:59.794+0000 7fa0da59c640 1 -- 172.21.15.96:0/668898676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa0dc104450 msgr2=0x7fa0dc1a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:59.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:59.794+0000 7fa0da59c640 1 --2- 172.21.15.96:0/668898676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa0dc104450 0x7fa0dc1a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:59.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:59.794+0000 7fa0da59c640 1 -- 172.21.15.96:0/668898676 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0dc1a1d30 con 0x7fa0dc102f30 2024-04-25T14:10:59.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:59.794+0000 7fa0d9d9b640 1 --2- 172.21.15.96:0/668898676 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0dc103910 0x7fa0dc19a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:59.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:59.794+0000 7fa0da59c640 1 --2- 172.21.15.96:0/668898676 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0dc102f30 0x7fa0dc19a1f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa0d400e9c0 tx=0x7fa0d400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:59.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:59.794+0000 7fa0cb7fe640 1 -- 172.21.15.96:0/668898676 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d400ccc0 con 0x7fa0dc102f30 2024-04-25T14:10:59.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:59.794+0000 7fa0cb7fe640 1 -- 172.21.15.96:0/668898676 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0d40054d0 con 0x7fa0dc102f30 2024-04-25T14:10:59.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:59.794+0000 7fa0e0dd7640 1 -- 172.21.15.96:0/668898676 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0dc1a1fc0 con 0x7fa0dc102f30 2024-04-25T14:10:59.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:59.794+0000 7fa0cb7fe640 1 -- 172.21.15.96:0/668898676 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4010430 con 0x7fa0dc102f30 2024-04-25T14:10:59.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:59.794+0000 7fa0e0dd7640 1 -- 172.21.15.96:0/668898676 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0dc1a2430 con 0x7fa0dc102f30 2024-04-25T14:10:59.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:59.795+0000 7fa0e0dd7640 1 -- 172.21.15.96:0/668898676 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0dc1a26f0 con 0x7fa0dc102f30 2024-04-25T14:10:59.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:59.796+0000 7fa0cb7fe640 1 -- 172.21.15.96:0/668898676 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fa0d4005000 con 0x7fa0dc102f30 2024-04-25T14:10:59.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:59.797+0000 7fa0cb7fe640 1 --2- 172.21.15.96:0/668898676 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa0c40afaf0 0x7fa0c40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:59.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:59.797+0000 7fa0d9d9b640 1 --2- 172.21.15.96:0/668898676 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa0c40afaf0 0x7fa0c40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:59.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:59.797+0000 7fa0cb7fe640 1 -- 172.21.15.96:0/668898676 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fa0d40dedc0 con 0x7fa0dc102f30 2024-04-25T14:10:59.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:59.798+0000 7fa0d9d9b640 1 --2- 172.21.15.96:0/668898676 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa0c40afaf0 0x7fa0c40b2000 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0005fd0 tx=0x7fa0d0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:59.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:10:59.801+0000 7fa0cb7fe640 1 -- 172.21.15.96:0/668898676 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa0d4053ff0 con 0x7fa0dc102f30 2024-04-25T14:10:59.909 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:59 smithi151 ceph-mon[32990]: Deploying daemon mds.d on smithi096 2024-04-25T14:10:59.910 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:59 smithi151 ceph-mon[32990]: mds.? [v2:172.21.15.96:6834/606227917,v1:172.21.15.96:6835/606227917] up:boot 2024-04-25T14:10:59.910 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:59 smithi151 ceph-mon[32990]: fsmap 3 up:standby 2024-04-25T14:10:59.910 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:10:59 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T14:10:59.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:59 smithi097 ceph-mon[32720]: Deploying daemon mds.d on smithi096 2024-04-25T14:10:59.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:59 smithi097 ceph-mon[32720]: mds.? [v2:172.21.15.96:6834/606227917,v1:172.21.15.96:6835/606227917] up:boot 2024-04-25T14:10:59.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:59 smithi097 ceph-mon[32720]: fsmap 3 up:standby 2024-04-25T14:10:59.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:10:59 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T14:11:00.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:00.032+0000 7fa0e0dd7640 1 -- 172.21.15.96:0/668898676 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa0dc1a28c0 con 0x7fa0dc102f30 2024-04-25T14:11:00.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:00.033+0000 7fa0cb7fe640 1 -- 172.21.15.96:0/668898676 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3043 (secure 0 0 0) 0x7fa0d40052f0 con 0x7fa0dc102f30 2024-04-25T14:11:00.033 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:11:00.033 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/606227917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":606227917},{"type":"v1","addr":"172.21.15.96:6835","nonce":606227917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/1995324831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1995324831},{"type":"v1","addr":"172.21.15.97:6833","nonce":1995324831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1382721608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1382721608},{"type":"v1","addr":"172.21.15.151:6833","nonce":1382721608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T14:11:00.033 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 4 2024-04-25T14:11:00.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:00.035+0000 7fa0e0dd7640 1 -- 172.21.15.96:0/668898676 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa0c40afaf0 msgr2=0x7fa0c40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:00.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:00.035+0000 7fa0e0dd7640 1 --2- 172.21.15.96:0/668898676 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa0c40afaf0 0x7fa0c40b2000 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0005fd0 tx=0x7fa0d0005850 comp rx=0 tx=0).stop 2024-04-25T14:11:00.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:00.035+0000 7fa0e0dd7640 1 -- 172.21.15.96:0/668898676 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0dc102f30 msgr2=0x7fa0dc19a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:00.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:00.035+0000 7fa0e0dd7640 1 --2- 172.21.15.96:0/668898676 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0dc102f30 0x7fa0dc19a1f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa0d400e9c0 tx=0x7fa0d400ee80 comp rx=0 tx=0).stop 2024-04-25T14:11:00.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:00.035+0000 7fa0e0dd7640 1 -- 172.21.15.96:0/668898676 shutdown_connections 2024-04-25T14:11:00.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:00.035+0000 7fa0e0dd7640 1 --2- 172.21.15.96:0/668898676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa0dc104450 0x7fa0dc1a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:00.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:00.035+0000 7fa0e0dd7640 1 --2- 172.21.15.96:0/668898676 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0dc103910 0x7fa0dc19a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:00.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:00.035+0000 7fa0e0dd7640 1 --2- 172.21.15.96:0/668898676 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa0c40afaf0 0x7fa0c40b2000 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:00.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:00.035+0000 7fa0e0dd7640 1 --2- 172.21.15.96:0/668898676 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0dc102f30 0x7fa0dc19a1f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:00.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:00.035+0000 7fa0e0dd7640 1 -- 172.21.15.96:0/668898676 >> 172.21.15.96:0/668898676 conn(0x7fa0dc0fd330 msgr2=0x7fa0dc0fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:00.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:00.036+0000 7fa0e0dd7640 1 -- 172.21.15.96:0/668898676 shutdown_connections 2024-04-25T14:11:00.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:00.036+0000 7fa0e0dd7640 1 -- 172.21.15.96:0/668898676 wait complete. 2024-04-25T14:11:00.046 INFO:journalctl@ceph.mds.d.smithi096.stdout:Apr 25 14:10:59 smithi096 systemd[1]: Starting Ceph mds.d for 59334182-030c-11ef-bc93-c7b262605968... 2024-04-25T14:11:00.307 INFO:journalctl@ceph.mds.d.smithi096.stdout:Apr 25 14:11:00 smithi096 podman[69818]: 2024-04-25 14:11:00.032933599 +0000 UTC m=+0.018763000 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:11:00.308 INFO:journalctl@ceph.mds.d.smithi096.stdout:Apr 25 14:11:00 smithi096 podman[69818]: 2024-04-25 14:11:00.153323496 +0000 UTC m=+0.139152939 container create d0538fbe5a3c8c5550cf44748fa7934415bd0a17bc6d59b6570d0a2455ecd7ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-d, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-25T14:11:00.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:00 smithi096 ceph-mon[29207]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:00.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:00 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/668898676' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:00.609 INFO:journalctl@ceph.mds.d.smithi096.stdout:Apr 25 14:11:00 smithi096 podman[69818]: 2024-04-25 14:11:00.422890003 +0000 UTC m=+0.408719403 container init d0538fbe5a3c8c5550cf44748fa7934415bd0a17bc6d59b6570d0a2455ecd7ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-d, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T14:11:00.609 INFO:journalctl@ceph.mds.d.smithi096.stdout:Apr 25 14:11:00 smithi096 podman[69818]: 2024-04-25 14:11:00.425693513 +0000 UTC m=+0.411522923 container start d0538fbe5a3c8c5550cf44748fa7934415bd0a17bc6d59b6570d0a2455ecd7ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T14:11:00.609 INFO:journalctl@ceph.mds.d.smithi096.stdout:Apr 25 14:11:00 smithi096 ceph-mds[69840]: -- 172.21.15.96:0/3294962040 <== mon.0 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ca4537fa40 con 0x55ca453c7a80 2024-04-25T14:11:00.609 INFO:journalctl@ceph.mds.d.smithi096.stdout:Apr 25 14:11:00 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-d[69836]: starting mds.d at 2024-04-25T14:11:00.610 INFO:journalctl@ceph.mds.d.smithi096.stdout:Apr 25 14:11:00 smithi096 bash[69818]: d0538fbe5a3c8c5550cf44748fa7934415bd0a17bc6d59b6570d0a2455ecd7ba 2024-04-25T14:11:00.610 INFO:journalctl@ceph.mds.d.smithi096.stdout:Apr 25 14:11:00 smithi096 systemd[1]: Started Ceph mds.d for 59334182-030c-11ef-bc93-c7b262605968. 2024-04-25T14:11:00.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:00 smithi097 ceph-mon[32720]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:00.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:00 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/668898676' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:00.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:00 smithi151 ceph-mon[32990]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:00.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:00 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/668898676' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:01.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:01 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:01.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:01 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:01.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:01 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:01.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:01 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:11:01.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:01 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:11:01.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:01 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:11:01.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:01 smithi096 ceph-mon[29207]: mds.? [v2:172.21.15.96:6836/3265300710,v1:172.21.15.96:6837/3265300710] up:boot 2024-04-25T14:11:01.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:01 smithi096 ceph-mon[29207]: fsmap 4 up:standby 2024-04-25T14:11:01.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:01 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T14:11:01.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:01 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:01.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:01 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:01.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:01 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:01.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:01 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:11:01.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:01 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:11:01.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:01 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:11:01.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:01 smithi097 ceph-mon[32720]: mds.? [v2:172.21.15.96:6836/3265300710,v1:172.21.15.96:6837/3265300710] up:boot 2024-04-25T14:11:01.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:01 smithi097 ceph-mon[32720]: fsmap 4 up:standby 2024-04-25T14:11:01.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:01 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T14:11:01.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:01 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:01.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:01 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:01.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:01 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:01.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:01 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:11:01.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:01 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:11:01.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:01 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:11:01.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:01 smithi151 ceph-mon[32990]: mds.? [v2:172.21.15.96:6836/3265300710,v1:172.21.15.96:6837/3265300710] up:boot 2024-04-25T14:11:01.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:01 smithi151 ceph-mon[32990]: fsmap 4 up:standby 2024-04-25T14:11:01.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:01 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T14:11:02.047 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:11:02.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.171+0000 7feebd1fe640 1 -- 172.21.15.96:0/1535325839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feeb8104d40 msgr2=0x7feeb8105150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:02.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.171+0000 7feebd1fe640 1 --2- 172.21.15.96:0/1535325839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feeb8104d40 0x7feeb8105150 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7feea0009900 tx=0x7feea002a410 comp rx=0 tx=0).stop 2024-04-25T14:11:02.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.171+0000 7feebd1fe640 1 -- 172.21.15.96:0/1535325839 shutdown_connections 2024-04-25T14:11:02.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.171+0000 7feebd1fe640 1 --2- 172.21.15.96:0/1535325839 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feeb8109a40 0x7feeb810be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:02.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.171+0000 7feebd1fe640 1 --2- 172.21.15.96:0/1535325839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feeb8104d40 0x7feeb8105150 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:02.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.171+0000 7feebd1fe640 1 --2- 172.21.15.96:0/1535325839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feeb8103b00 0x7feeb8104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:02.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.171+0000 7feebd1fe640 1 -- 172.21.15.96:0/1535325839 >> 172.21.15.96:0/1535325839 conn(0x7feeb80fed10 msgr2=0x7feeb81011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:02.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.171+0000 7feebd1fe640 1 -- 172.21.15.96:0/1535325839 shutdown_connections 2024-04-25T14:11:02.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.171+0000 7feebd1fe640 1 -- 172.21.15.96:0/1535325839 wait complete. 2024-04-25T14:11:02.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.172+0000 7feebd1fe640 1 Processor -- start 2024-04-25T14:11:02.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.173+0000 7feebd1fe640 1 -- start start 2024-04-25T14:11:02.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.173+0000 7feebd1fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feeb8103b00 0x7feeb819a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:02.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.173+0000 7feebd1fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feeb8104d40 0x7feeb819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:02.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.173+0000 7feebd1fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feeb8109a40 0x7feeb81a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:02.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.173+0000 7feebd1fe640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7feeb810f630 con 0x7feeb8103b00 2024-04-25T14:11:02.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.173+0000 7feebd1fe640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7feeb810f4b0 con 0x7feeb8109a40 2024-04-25T14:11:02.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.173+0000 7feebd1fe640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7feeb810f7b0 con 0x7feeb8104d40 2024-04-25T14:11:02.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.173+0000 7feeb6d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feeb8103b00 0x7feeb819a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:02.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.173+0000 7feeb6d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feeb8103b00 0x7feeb819a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44312/0 (socket says 172.21.15.96:44312) 2024-04-25T14:11:02.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.173+0000 7feeb6d9d640 1 -- 172.21.15.96:0/4118604945 learned_addr learned my addr 172.21.15.96:0/4118604945 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:11:02.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.173+0000 7feeb759e640 1 --2- 172.21.15.96:0/4118604945 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feeb8109a40 0x7feeb81a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:02.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.173+0000 7feeaffff640 1 --2- 172.21.15.96:0/4118604945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feeb8104d40 0x7feeb819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:02.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.173+0000 7feeb6d9d640 1 -- 172.21.15.96:0/4118604945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feeb8104d40 msgr2=0x7feeb819a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:02.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.173+0000 7feeb6d9d640 1 --2- 172.21.15.96:0/4118604945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feeb8104d40 0x7feeb819a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:02.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.173+0000 7feeb6d9d640 1 -- 172.21.15.96:0/4118604945 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feeb8109a40 msgr2=0x7feeb81a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:02.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.173+0000 7feeb6d9d640 1 --2- 172.21.15.96:0/4118604945 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feeb8109a40 0x7feeb81a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:02.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.173+0000 7feeb6d9d640 1 -- 172.21.15.96:0/4118604945 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeb81a1d70 con 0x7feeb8103b00 2024-04-25T14:11:02.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.173+0000 7feeb6d9d640 1 --2- 172.21.15.96:0/4118604945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feeb8103b00 0x7feeb819a230 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7feea800e9f0 tx=0x7feea800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:02.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.173+0000 7feeb4d99640 1 -- 172.21.15.96:0/4118604945 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea800ccc0 con 0x7feeb8103b00 2024-04-25T14:11:02.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.173+0000 7feeb4d99640 1 -- 172.21.15.96:0/4118604945 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feea80035f0 con 0x7feeb8103b00 2024-04-25T14:11:02.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.174+0000 7feeb4d99640 1 -- 172.21.15.96:0/4118604945 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea800ce60 con 0x7feeb8103b00 2024-04-25T14:11:02.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.174+0000 7feebd1fe640 1 -- 172.21.15.96:0/4118604945 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feeb81a2060 con 0x7feeb8103b00 2024-04-25T14:11:02.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.174+0000 7feebd1fe640 1 -- 172.21.15.96:0/4118604945 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feeb81a2600 con 0x7feeb8103b00 2024-04-25T14:11:02.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.175+0000 7feebd1fe640 1 -- 172.21.15.96:0/4118604945 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feeb8108850 con 0x7feeb8103b00 2024-04-25T14:11:02.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.176+0000 7feeb4d99640 1 -- 172.21.15.96:0/4118604945 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7feea8003aa0 con 0x7feeb8103b00 2024-04-25T14:11:02.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.177+0000 7feeb4d99640 1 --2- 172.21.15.96:0/4118604945 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7feea40afb70 0x7feea40b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:02.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.177+0000 7feeaffff640 1 --2- 172.21.15.96:0/4118604945 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7feea40afb70 0x7feea40b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:02.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.177+0000 7feeb4d99640 1 -- 172.21.15.96:0/4118604945 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7feea80deaa0 con 0x7feeb8103b00 2024-04-25T14:11:02.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.177+0000 7feeaffff640 1 --2- 172.21.15.96:0/4118604945 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7feea40afb70 0x7feea40b2080 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7feea0009230 tx=0x7feea0031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:02.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.180+0000 7feeb4d99640 1 -- 172.21.15.96:0/4118604945 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feea80a7b20 con 0x7feeb8103b00 2024-04-25T14:11:02.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.397+0000 7feebd1fe640 1 -- 172.21.15.96:0/4118604945 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feeb802db50 con 0x7feeb8103b00 2024-04-25T14:11:02.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.397+0000 7feeb4d99640 1 -- 172.21.15.96:0/4118604945 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3868 (secure 0 0 0) 0x7feea80a7420 con 0x7feeb8103b00 2024-04-25T14:11:02.397 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:11:02.397 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/606227917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":606227917},{"type":"v1","addr":"172.21.15.96:6835","nonce":606227917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/3265300710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3265300710},{"type":"v1","addr":"172.21.15.96:6837","nonce":3265300710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/1995324831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1995324831},{"type":"v1","addr":"172.21.15.97:6833","nonce":1995324831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1382721608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1382721608},{"type":"v1","addr":"172.21.15.151:6833","nonce":1382721608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T14:11:02.398 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 5 2024-04-25T14:11:02.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.400+0000 7feebd1fe640 1 -- 172.21.15.96:0/4118604945 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7feea40afb70 msgr2=0x7feea40b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:02.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.400+0000 7feebd1fe640 1 --2- 172.21.15.96:0/4118604945 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7feea40afb70 0x7feea40b2080 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7feea0009230 tx=0x7feea0031400 comp rx=0 tx=0).stop 2024-04-25T14:11:02.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.400+0000 7feebd1fe640 1 -- 172.21.15.96:0/4118604945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feeb8103b00 msgr2=0x7feeb819a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:02.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.400+0000 7feebd1fe640 1 --2- 172.21.15.96:0/4118604945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feeb8103b00 0x7feeb819a230 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7feea800e9f0 tx=0x7feea800eeb0 comp rx=0 tx=0).stop 2024-04-25T14:11:02.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.400+0000 7feebd1fe640 1 -- 172.21.15.96:0/4118604945 shutdown_connections 2024-04-25T14:11:02.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.400+0000 7feebd1fe640 1 --2- 172.21.15.96:0/4118604945 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feeb8109a40 0x7feeb81a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:02.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.400+0000 7feebd1fe640 1 --2- 172.21.15.96:0/4118604945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feeb8104d40 0x7feeb819a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:02.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.400+0000 7feebd1fe640 1 --2- 172.21.15.96:0/4118604945 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7feea40afb70 0x7feea40b2080 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:02.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.400+0000 7feebd1fe640 1 --2- 172.21.15.96:0/4118604945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feeb8103b00 0x7feeb819a230 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:02.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.400+0000 7feebd1fe640 1 -- 172.21.15.96:0/4118604945 >> 172.21.15.96:0/4118604945 conn(0x7feeb80fed10 msgr2=0x7feeb8105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:02.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.400+0000 7feebd1fe640 1 -- 172.21.15.96:0/4118604945 shutdown_connections 2024-04-25T14:11:02.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:02.400+0000 7feebd1fe640 1 -- 172.21.15.96:0/4118604945 wait complete. 2024-04-25T14:11:02.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:02 smithi097 ceph-mon[32720]: Deploying daemon mds.e on smithi097 2024-04-25T14:11:02.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:02 smithi097 ceph-mon[32720]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:02.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:02 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/4118604945' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:02.839 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:02 smithi151 ceph-mon[32990]: Deploying daemon mds.e on smithi097 2024-04-25T14:11:02.839 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:02 smithi151 ceph-mon[32990]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:02.839 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:02 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/4118604945' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:02.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:02 smithi096 ceph-mon[29207]: Deploying daemon mds.e on smithi097 2024-04-25T14:11:02.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:02 smithi096 ceph-mon[29207]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:02.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:02 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/4118604945' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:02.922 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 14:11:02 smithi097 systemd[1]: Starting Ceph mds.e for 59334182-030c-11ef-bc93-c7b262605968... 2024-04-25T14:11:03.341 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 14:11:03 smithi097 podman[62592]: 2024-04-25 14:11:02.966940512 +0000 UTC m=+0.017980615 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:11:03.341 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 14:11:03 smithi097 podman[62592]: 2024-04-25 14:11:03.078505545 +0000 UTC m=+0.129545678 container create 5fa8c1d3bf3a8a10c5088878b15e0e6543a21767e8966daf30ae437899525240 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-e, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-25T14:11:03.341 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 14:11:03 smithi097 podman[62592]: 2024-04-25 14:11:03.306360801 +0000 UTC m=+0.357400890 container init 5fa8c1d3bf3a8a10c5088878b15e0e6543a21767e8966daf30ae437899525240 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-e, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T14:11:03.341 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 14:11:03 smithi097 podman[62592]: 2024-04-25 14:11:03.308934838 +0000 UTC m=+0.359974928 container start 5fa8c1d3bf3a8a10c5088878b15e0e6543a21767e8966daf30ae437899525240 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-e, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-25T14:11:03.341 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 14:11:03 smithi097 ceph-mds[62610]: -- 172.21.15.97:0/4176560522 <== mon.2 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a783bcfa40 con 0x55a783c17a80 2024-04-25T14:11:03.672 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 14:11:03 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-e[62606]: starting mds.e at 2024-04-25T14:11:03.672 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 14:11:03 smithi097 bash[62592]: 5fa8c1d3bf3a8a10c5088878b15e0e6543a21767e8966daf30ae437899525240 2024-04-25T14:11:03.673 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 14:11:03 smithi097 systemd[1]: Started Ceph mds.e for 59334182-030c-11ef-bc93-c7b262605968. 2024-04-25T14:11:04.412 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:11:04.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.535+0000 7f55396b6640 1 -- 172.21.15.96:0/1702647278 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5534108900 msgr2=0x7f553410ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:04.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.535+0000 7f55396b6640 1 --2- 172.21.15.96:0/1702647278 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5534108900 0x7f553410ad20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f552c009900 tx=0x7f552c02a410 comp rx=0 tx=0).stop 2024-04-25T14:11:04.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.535+0000 7f55396b6640 1 -- 172.21.15.96:0/1702647278 shutdown_connections 2024-04-25T14:11:04.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.535+0000 7f55396b6640 1 --2- 172.21.15.96:0/1702647278 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5534108900 0x7f553410ad20 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:04.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.535+0000 7f55396b6640 1 --2- 172.21.15.96:0/1702647278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5534103c00 0x7f5534104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:04.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.535+0000 7f55396b6640 1 --2- 172.21.15.96:0/1702647278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55341029c0 0x7f55341036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:04.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.535+0000 7f55396b6640 1 -- 172.21.15.96:0/1702647278 >> 172.21.15.96:0/1702647278 conn(0x7f55340fdbd0 msgr2=0x7f5534100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:04.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.535+0000 7f55396b6640 1 -- 172.21.15.96:0/1702647278 shutdown_connections 2024-04-25T14:11:04.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.535+0000 7f55396b6640 1 -- 172.21.15.96:0/1702647278 wait complete. 2024-04-25T14:11:04.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.535+0000 7f55396b6640 1 Processor -- start 2024-04-25T14:11:04.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.536+0000 7f55396b6640 1 -- start start 2024-04-25T14:11:04.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.536+0000 7f55396b6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55341029c0 0x7f553406a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:04.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.536+0000 7f55396b6640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5534103c00 0x7f553419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:04.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.536+0000 7f55396b6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5534108900 0x7f55341a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:04.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.536+0000 7f55396b6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f553410e4f0 con 0x7f5534108900 2024-04-25T14:11:04.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.536+0000 7f55396b6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f553410e370 con 0x7f55341029c0 2024-04-25T14:11:04.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.536+0000 7f55396b6640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f553410e670 con 0x7f5534103c00 2024-04-25T14:11:04.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.536+0000 7f55337fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5534108900 0x7f55341a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:04.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.536+0000 7f55337fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5534108900 0x7f55341a14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44324/0 (socket says 172.21.15.96:44324) 2024-04-25T14:11:04.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.536+0000 7f55337fe640 1 -- 172.21.15.96:0/3363980878 learned_addr learned my addr 172.21.15.96:0/3363980878 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:11:04.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.536+0000 7f55327fc640 1 --2- 172.21.15.96:0/3363980878 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5534103c00 0x7f553419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:04.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.536+0000 7f5532ffd640 1 --2- 172.21.15.96:0/3363980878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55341029c0 0x7f553406a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:04.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.536+0000 7f55337fe640 1 -- 172.21.15.96:0/3363980878 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5534103c00 msgr2=0x7f553419ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:04.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.536+0000 7f55337fe640 1 --2- 172.21.15.96:0/3363980878 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5534103c00 0x7f553419ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:04.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.536+0000 7f55337fe640 1 -- 172.21.15.96:0/3363980878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55341029c0 msgr2=0x7f553406a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:04.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.536+0000 7f55337fe640 1 --2- 172.21.15.96:0/3363980878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55341029c0 0x7f553406a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:04.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.536+0000 7f55337fe640 1 -- 172.21.15.96:0/3363980878 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55341a1cf0 con 0x7f5534108900 2024-04-25T14:11:04.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.536+0000 7f55337fe640 1 --2- 172.21.15.96:0/3363980878 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5534108900 0x7f55341a14d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f552c009230 tx=0x7f552c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:04.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.537+0000 7f551bfff640 1 -- 172.21.15.96:0/3363980878 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552c03f030 con 0x7f5534108900 2024-04-25T14:11:04.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.537+0000 7f55396b6640 1 -- 172.21.15.96:0/3363980878 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55341a1f80 con 0x7f5534108900 2024-04-25T14:11:04.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.537+0000 7f551bfff640 1 -- 172.21.15.96:0/3363980878 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f552c031910 con 0x7f5534108900 2024-04-25T14:11:04.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.537+0000 7f551bfff640 1 -- 172.21.15.96:0/3363980878 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552c035070 con 0x7f5534108900 2024-04-25T14:11:04.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.537+0000 7f55396b6640 1 -- 172.21.15.96:0/3363980878 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55341a2390 con 0x7f5534108900 2024-04-25T14:11:04.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.538+0000 7f55396b6640 1 -- 172.21.15.96:0/3363980878 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5534107710 con 0x7f5534108900 2024-04-25T14:11:04.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.539+0000 7f551bfff640 1 -- 172.21.15.96:0/3363980878 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f552c042030 con 0x7f5534108900 2024-04-25T14:11:04.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.540+0000 7f551bfff640 1 --2- 172.21.15.96:0/3363980878 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f551c0afaf0 0x7f551c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:04.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.540+0000 7f5532ffd640 1 --2- 172.21.15.96:0/3363980878 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f551c0afaf0 0x7f551c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:04.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.540+0000 7f551bfff640 1 -- 172.21.15.96:0/3363980878 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f552c0ff010 con 0x7f5534108900 2024-04-25T14:11:04.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.540+0000 7f5532ffd640 1 --2- 172.21.15.96:0/3363980878 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f551c0afaf0 0x7f551c0b2000 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f55341035c0 tx=0x7f5524009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:04.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.543+0000 7f551bfff640 1 -- 172.21.15.96:0/3363980878 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f552c0c8090 con 0x7f5534108900 2024-04-25T14:11:04.746 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:04 smithi151 ceph-mon[32990]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:04.746 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:04 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:04.746 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:04 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:04.747 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:04 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:04.747 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:04 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:11:04.747 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:04 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:11:04.747 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:04 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:11:04.747 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:04 smithi151 ceph-mon[32990]: Deploying daemon mds.f on smithi151 2024-04-25T14:11:04.747 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:04 smithi151 ceph-mon[32990]: mds.? [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] up:boot 2024-04-25T14:11:04.747 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:04 smithi151 ceph-mon[32990]: fsmap 5 up:standby 2024-04-25T14:11:04.747 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:04 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T14:11:04.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.760+0000 7f55396b6640 1 -- 172.21.15.96:0/3363980878 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f553402db50 con 0x7f5534108900 2024-04-25T14:11:04.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.761+0000 7f551bfff640 1 -- 172.21.15.96:0/3363980878 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4693 (secure 0 0 0) 0x7f552c0d0110 con 0x7f5534108900 2024-04-25T14:11:04.761 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:11:04.761 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/606227917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":606227917},{"type":"v1","addr":"172.21.15.96:6835","nonce":606227917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/3265300710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3265300710},{"type":"v1","addr":"172.21.15.96:6837","nonce":3265300710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/1995324831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1995324831},{"type":"v1","addr":"172.21.15.97:6833","nonce":1995324831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1382721608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1382721608},{"type":"v1","addr":"172.21.15.151:6833","nonce":1382721608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3872301726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3872301726},{"type":"v1","addr":"172.21.15.97:6835","nonce":3872301726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-25T14:11:04.761 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 6 2024-04-25T14:11:04.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.763+0000 7f55396b6640 1 -- 172.21.15.96:0/3363980878 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f551c0afaf0 msgr2=0x7f551c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:04.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.763+0000 7f55396b6640 1 --2- 172.21.15.96:0/3363980878 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f551c0afaf0 0x7f551c0b2000 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f55341035c0 tx=0x7f5524009870 comp rx=0 tx=0).stop 2024-04-25T14:11:04.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.763+0000 7f55396b6640 1 -- 172.21.15.96:0/3363980878 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5534108900 msgr2=0x7f55341a14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:04.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.763+0000 7f55396b6640 1 --2- 172.21.15.96:0/3363980878 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5534108900 0x7f55341a14d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f552c009230 tx=0x7f552c0042f0 comp rx=0 tx=0).stop 2024-04-25T14:11:04.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.763+0000 7f55396b6640 1 -- 172.21.15.96:0/3363980878 shutdown_connections 2024-04-25T14:11:04.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.763+0000 7f55396b6640 1 --2- 172.21.15.96:0/3363980878 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f551c0afaf0 0x7f551c0b2000 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:04.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.763+0000 7f55396b6640 1 --2- 172.21.15.96:0/3363980878 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5534108900 0x7f55341a14d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:04.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.763+0000 7f55396b6640 1 --2- 172.21.15.96:0/3363980878 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5534103c00 0x7f553419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:04.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.763+0000 7f55396b6640 1 --2- 172.21.15.96:0/3363980878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55341029c0 0x7f553406a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:04.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.763+0000 7f55396b6640 1 -- 172.21.15.96:0/3363980878 >> 172.21.15.96:0/3363980878 conn(0x7f55340fdbd0 msgr2=0x7f55341048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:04.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.763+0000 7f55396b6640 1 -- 172.21.15.96:0/3363980878 shutdown_connections 2024-04-25T14:11:04.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:04.763+0000 7f55396b6640 1 -- 172.21.15.96:0/3363980878 wait complete. 2024-04-25T14:11:04.774 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:04 smithi096 ceph-mon[29207]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:04.774 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:04 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:04.774 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:04 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:04.774 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:04 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:04.775 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:04 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:11:04.775 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:04 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:11:04.775 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:04 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:11:04.775 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:04 smithi096 ceph-mon[29207]: Deploying daemon mds.f on smithi151 2024-04-25T14:11:04.775 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:04 smithi096 ceph-mon[29207]: mds.? [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] up:boot 2024-04-25T14:11:04.775 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:04 smithi096 ceph-mon[29207]: fsmap 5 up:standby 2024-04-25T14:11:04.775 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:04 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T14:11:04.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:04 smithi097 ceph-mon[32720]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:04.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:04 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:04.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:04 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:04.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:04 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:04.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:04 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:11:04.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:04 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:11:04.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:04 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:11:04.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:04 smithi097 ceph-mon[32720]: Deploying daemon mds.f on smithi151 2024-04-25T14:11:04.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:04 smithi097 ceph-mon[32720]: mds.? [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] up:boot 2024-04-25T14:11:04.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:04 smithi097 ceph-mon[32720]: fsmap 5 up:standby 2024-04-25T14:11:04.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:04 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T14:11:05.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:05 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/3363980878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:05.688 INFO:journalctl@ceph.mds.f.smithi151.stdout:Apr 25 14:11:05 smithi151 systemd[1]: Starting Ceph mds.f for 59334182-030c-11ef-bc93-c7b262605968... 2024-04-25T14:11:05.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:05 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/3363980878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:05.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:05 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/3363980878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:06.121 INFO:journalctl@ceph.mds.f.smithi151.stdout:Apr 25 14:11:05 smithi151 podman[63230]: 2024-04-25 14:11:05.74223144 +0000 UTC m=+0.018190879 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:11:06.121 INFO:journalctl@ceph.mds.f.smithi151.stdout:Apr 25 14:11:05 smithi151 podman[63230]: 2024-04-25 14:11:05.852865531 +0000 UTC m=+0.128825001 container create 629f47d1bed267bb8c43de08c81796abfd1570312c2e8172cbc0a6f1bc4cfea7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-f, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True) 2024-04-25T14:11:06.121 INFO:journalctl@ceph.mds.f.smithi151.stdout:Apr 25 14:11:06 smithi151 podman[63230]: 2024-04-25 14:11:06.122077407 +0000 UTC m=+0.398036835 container init 629f47d1bed267bb8c43de08c81796abfd1570312c2e8172cbc0a6f1bc4cfea7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-f, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:11:06.438 INFO:journalctl@ceph.mds.f.smithi151.stdout:Apr 25 14:11:06 smithi151 podman[63230]: 2024-04-25 14:11:06.124980861 +0000 UTC m=+0.400940297 container start 629f47d1bed267bb8c43de08c81796abfd1570312c2e8172cbc0a6f1bc4cfea7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-25T14:11:06.438 INFO:journalctl@ceph.mds.f.smithi151.stdout:Apr 25 14:11:06 smithi151 ceph-mds[63249]: -- 172.21.15.151:0/2015022562 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5597f728da40 con 0x5597f72d5180 2024-04-25T14:11:06.438 INFO:journalctl@ceph.mds.f.smithi151.stdout:Apr 25 14:11:06 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-f[63245]: starting mds.f at 2024-04-25T14:11:06.439 INFO:journalctl@ceph.mds.f.smithi151.stdout:Apr 25 14:11:06 smithi151 bash[63230]: 629f47d1bed267bb8c43de08c81796abfd1570312c2e8172cbc0a6f1bc4cfea7 2024-04-25T14:11:06.439 INFO:journalctl@ceph.mds.f.smithi151.stdout:Apr 25 14:11:06 smithi151 systemd[1]: Started Ceph mds.f for 59334182-030c-11ef-bc93-c7b262605968. 2024-04-25T14:11:06.777 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:11:06.802 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:06 smithi096 ceph-mon[29207]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:06.802 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:06 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:06.802 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:06 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:06.802 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:06 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:06.802 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:06 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:11:06.802 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:06 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:11:06.802 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:06 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:11:06.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:06.897+0000 7f9a7824a640 1 -- 172.21.15.96:0/3743468055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a70103b00 msgr2=0x7f9a70104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:06.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:06.897+0000 7f9a7824a640 1 --2- 172.21.15.96:0/3743468055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a70103b00 0x7f9a70104800 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9a64009900 tx=0x7f9a6402a410 comp rx=0 tx=0).stop 2024-04-25T14:11:06.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:06.897+0000 7f9a7824a640 1 -- 172.21.15.96:0/3743468055 shutdown_connections 2024-04-25T14:11:06.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:06.897+0000 7f9a7824a640 1 --2- 172.21.15.96:0/3743468055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a70109a40 0x7f9a7010be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:06.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:06.897+0000 7f9a7824a640 1 --2- 172.21.15.96:0/3743468055 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a70104d40 0x7f9a70105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:06.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:06.897+0000 7f9a7824a640 1 --2- 172.21.15.96:0/3743468055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a70103b00 0x7f9a70104800 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:06.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:06.897+0000 7f9a7824a640 1 -- 172.21.15.96:0/3743468055 >> 172.21.15.96:0/3743468055 conn(0x7f9a700fed10 msgr2=0x7f9a701011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:06.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:06.897+0000 7f9a7824a640 1 -- 172.21.15.96:0/3743468055 shutdown_connections 2024-04-25T14:11:06.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:06.897+0000 7f9a7824a640 1 -- 172.21.15.96:0/3743468055 wait complete. 2024-04-25T14:11:06.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:06.898+0000 7f9a7824a640 1 Processor -- start 2024-04-25T14:11:06.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:06.898+0000 7f9a7824a640 1 -- start start 2024-04-25T14:11:06.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:06.898+0000 7f9a7824a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a70103b00 0x7f9a7019a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:06.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:06.898+0000 7f9a7824a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a70104d40 0x7f9a7019a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:06.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:06.898+0000 7f9a7824a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a70109a40 0x7f9a701a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:06.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:06.898+0000 7f9a7824a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7010f600 con 0x7f9a70103b00 2024-04-25T14:11:06.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:06.898+0000 7f9a7824a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7010f480 con 0x7f9a70104d40 2024-04-25T14:11:06.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:06.898+0000 7f9a7824a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7010f780 con 0x7f9a70109a40 2024-04-25T14:11:06.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:06.898+0000 7f9a75fe6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a70103b00 0x7f9a7019a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:06.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:06.898+0000 7f9a75fe6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a70103b00 0x7f9a7019a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44350/0 (socket says 172.21.15.96:44350) 2024-04-25T14:11:06.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:06.898+0000 7f9a75fe6640 1 -- 172.21.15.96:0/3201918604 learned_addr learned my addr 172.21.15.96:0/3201918604 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:11:06.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:06.899+0000 7f9a767e7640 1 --2- 172.21.15.96:0/3201918604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a70109a40 0x7f9a701a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:06.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:06.899+0000 7f9a757e5640 1 --2- 172.21.15.96:0/3201918604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a70104d40 0x7f9a7019a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:06.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:06.899+0000 7f9a75fe6640 1 -- 172.21.15.96:0/3201918604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a70109a40 msgr2=0x7f9a701a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:06.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:06.899+0000 7f9a75fe6640 1 --2- 172.21.15.96:0/3201918604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a70109a40 0x7f9a701a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:06.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:06.899+0000 7f9a75fe6640 1 -- 172.21.15.96:0/3201918604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a70104d40 msgr2=0x7f9a7019a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:06.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:06.899+0000 7f9a75fe6640 1 --2- 172.21.15.96:0/3201918604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a70104d40 0x7f9a7019a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:06.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:06.899+0000 7f9a75fe6640 1 -- 172.21.15.96:0/3201918604 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a701a1d60 con 0x7f9a70103b00 2024-04-25T14:11:06.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:06.899+0000 7f9a767e7640 1 --2- 172.21.15.96:0/3201918604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a70109a40 0x7f9a701a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:06.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:06.899+0000 7f9a757e5640 1 --2- 172.21.15.96:0/3201918604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a70104d40 0x7f9a7019a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:06.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:06.899+0000 7f9a75fe6640 1 --2- 172.21.15.96:0/3201918604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a70103b00 0x7f9a7019a220 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9a64006010 tx=0x7f9a64031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:06.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:06.899+0000 7f9a62ffd640 1 -- 172.21.15.96:0/3201918604 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6403f030 con 0x7f9a70103b00 2024-04-25T14:11:06.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:06.899+0000 7f9a62ffd640 1 -- 172.21.15.96:0/3201918604 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a64003aa0 con 0x7f9a70103b00 2024-04-25T14:11:06.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:06.899+0000 7f9a7824a640 1 -- 172.21.15.96:0/3201918604 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a701a1ff0 con 0x7f9a70103b00 2024-04-25T14:11:06.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:06.899+0000 7f9a62ffd640 1 -- 172.21.15.96:0/3201918604 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a64035070 con 0x7f9a70103b00 2024-04-25T14:11:06.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:06.899+0000 7f9a7824a640 1 -- 172.21.15.96:0/3201918604 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a701a2450 con 0x7f9a70103b00 2024-04-25T14:11:06.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:06.900+0000 7f9a7824a640 1 -- 172.21.15.96:0/3201918604 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a70108850 con 0x7f9a70103b00 2024-04-25T14:11:06.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:06.902+0000 7f9a62ffd640 1 -- 172.21.15.96:0/3201918604 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f9a64042030 con 0x7f9a70103b00 2024-04-25T14:11:06.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:06.903+0000 7f9a62ffd640 1 --2- 172.21.15.96:0/3201918604 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f9a5c0afb70 0x7f9a5c0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:06.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:06.903+0000 7f9a757e5640 1 --2- 172.21.15.96:0/3201918604 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f9a5c0afb70 0x7f9a5c0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:06.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:06.903+0000 7f9a62ffd640 1 -- 172.21.15.96:0/3201918604 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f9a640ff100 con 0x7f9a70103b00 2024-04-25T14:11:06.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:06.903+0000 7f9a757e5640 1 --2- 172.21.15.96:0/3201918604 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f9a5c0afb70 0x7f9a5c0b2080 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c005fd0 tx=0x7f9a6c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:06.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:06.907+0000 7f9a62ffd640 1 -- 172.21.15.96:0/3201918604 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9a640c8180 con 0x7f9a70103b00 2024-04-25T14:11:06.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:06 smithi097 ceph-mon[32720]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:06.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:06 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:06.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:06 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:06.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:06 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:06.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:06 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:11:06.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:06 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:11:06.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:06 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:11:06.930 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:06 smithi151 ceph-mon[32990]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:06.930 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:06 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:06.931 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:06 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:06.931 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:06 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:06.931 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:06 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:11:06.931 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:06 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:11:06.931 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:06 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:11:07.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:07.123+0000 7f9a7824a640 1 -- 172.21.15.96:0/3201918604 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a7002dc70 con 0x7f9a70103b00 2024-04-25T14:11:07.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:07.123+0000 7f9a62ffd640 1 -- 172.21.15.96:0/3201918604 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5521 (secure 0 0 0) 0x7f9a640d0020 con 0x7f9a70103b00 2024-04-25T14:11:07.123 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:11:07.123 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/606227917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":606227917},{"type":"v1","addr":"172.21.15.96:6835","nonce":606227917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/3265300710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3265300710},{"type":"v1","addr":"172.21.15.96:6837","nonce":3265300710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/1995324831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1995324831},{"type":"v1","addr":"172.21.15.97:6833","nonce":1995324831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1382721608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1382721608},{"type":"v1","addr":"172.21.15.151:6833","nonce":1382721608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3872301726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3872301726},{"type":"v1","addr":"172.21.15.97:6835","nonce":3872301726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/1313823638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":1313823638},{"type":"v1","addr":"172.21.15.151:6835","nonce":1313823638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-25T14:11:07.123 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 7 2024-04-25T14:11:07.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:07.125+0000 7f9a7824a640 1 -- 172.21.15.96:0/3201918604 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f9a5c0afb70 msgr2=0x7f9a5c0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:07.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:07.125+0000 7f9a7824a640 1 --2- 172.21.15.96:0/3201918604 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f9a5c0afb70 0x7f9a5c0b2080 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c005fd0 tx=0x7f9a6c005850 comp rx=0 tx=0).stop 2024-04-25T14:11:07.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:07.125+0000 7f9a7824a640 1 -- 172.21.15.96:0/3201918604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a70103b00 msgr2=0x7f9a7019a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:07.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:07.125+0000 7f9a7824a640 1 --2- 172.21.15.96:0/3201918604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a70103b00 0x7f9a7019a220 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9a64006010 tx=0x7f9a64031a50 comp rx=0 tx=0).stop 2024-04-25T14:11:07.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:07.125+0000 7f9a7824a640 1 -- 172.21.15.96:0/3201918604 shutdown_connections 2024-04-25T14:11:07.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:07.125+0000 7f9a7824a640 1 --2- 172.21.15.96:0/3201918604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a70109a40 0x7f9a701a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:07.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:07.125+0000 7f9a7824a640 1 --2- 172.21.15.96:0/3201918604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a70104d40 0x7f9a7019a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:07.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:07.125+0000 7f9a7824a640 1 --2- 172.21.15.96:0/3201918604 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f9a5c0afb70 0x7f9a5c0b2080 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:07.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:07.125+0000 7f9a7824a640 1 --2- 172.21.15.96:0/3201918604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a70103b00 0x7f9a7019a220 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:07.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:07.125+0000 7f9a7824a640 1 -- 172.21.15.96:0/3201918604 >> 172.21.15.96:0/3201918604 conn(0x7f9a700fed10 msgr2=0x7f9a70105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:07.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:07.125+0000 7f9a7824a640 1 -- 172.21.15.96:0/3201918604 shutdown_connections 2024-04-25T14:11:07.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:07.125+0000 7f9a7824a640 1 -- 172.21.15.96:0/3201918604 wait complete. 2024-04-25T14:11:07.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:07 smithi096 ceph-mon[29207]: Deploying daemon mds.c on smithi151 2024-04-25T14:11:07.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:07 smithi096 ceph-mon[29207]: mds.? [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] up:boot 2024-04-25T14:11:07.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:07 smithi096 ceph-mon[29207]: fsmap 6 up:standby 2024-04-25T14:11:07.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:07 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T14:11:07.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:07 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/3201918604' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:07.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:07 smithi097 ceph-mon[32720]: Deploying daemon mds.c on smithi151 2024-04-25T14:11:07.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:07 smithi097 ceph-mon[32720]: mds.? [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] up:boot 2024-04-25T14:11:07.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:07 smithi097 ceph-mon[32720]: fsmap 6 up:standby 2024-04-25T14:11:07.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:07 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T14:11:07.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:07 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/3201918604' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:07.933 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:07 smithi151 ceph-mon[32990]: Deploying daemon mds.c on smithi151 2024-04-25T14:11:07.933 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:07 smithi151 ceph-mon[32990]: mds.? [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] up:boot 2024-04-25T14:11:07.933 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:07 smithi151 ceph-mon[32990]: fsmap 6 up:standby 2024-04-25T14:11:07.933 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:07 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T14:11:07.934 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:07 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/3201918604' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:08.667 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:08 smithi151 ceph-mon[32990]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:08.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:08 smithi096 ceph-mon[29207]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:08.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:08 smithi097 ceph-mon[32720]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:09.137 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:11:09.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.267+0000 7f1f7fa27640 1 -- 172.21.15.96:0/1066954563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f78103980 msgr2=0x7f1f78103df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:09.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.267+0000 7f1f7fa27640 1 --2- 172.21.15.96:0/1066954563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f78103980 0x7f1f78103df0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1f68009900 tx=0x7f1f6802a410 comp rx=0 tx=0).stop 2024-04-25T14:11:09.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.267+0000 7f1f7fa27640 1 -- 172.21.15.96:0/1066954563 shutdown_connections 2024-04-25T14:11:09.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.267+0000 7f1f7fa27640 1 --2- 172.21.15.96:0/1066954563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f78104480 0x7f1f7810abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.267+0000 7f1f7fa27640 1 --2- 172.21.15.96:0/1066954563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f78103980 0x7f1f78103df0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.267+0000 7f1f7fa27640 1 --2- 172.21.15.96:0/1066954563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1f78102fa0 0x7f1f781033b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.267+0000 7f1f7fa27640 1 -- 172.21.15.96:0/1066954563 >> 172.21.15.96:0/1066954563 conn(0x7f1f780fd360 msgr2=0x7f1f780ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:09.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.267+0000 7f1f7fa27640 1 -- 172.21.15.96:0/1066954563 shutdown_connections 2024-04-25T14:11:09.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.267+0000 7f1f7fa27640 1 -- 172.21.15.96:0/1066954563 wait complete. 2024-04-25T14:11:09.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.268+0000 7f1f7fa27640 1 Processor -- start 2024-04-25T14:11:09.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.268+0000 7f1f7fa27640 1 -- start start 2024-04-25T14:11:09.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.268+0000 7f1f7fa27640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1f78102fa0 0x7f1f781022a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.268+0000 7f1f7fa27640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f78103980 0x7f1f7819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.268+0000 7f1f7fa27640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f78104480 0x7f1f781a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.268+0000 7f1f7fa27640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7810e540 con 0x7f1f78104480 2024-04-25T14:11:09.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.268+0000 7f1f7fa27640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7810e3c0 con 0x7f1f78103980 2024-04-25T14:11:09.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.268+0000 7f1f7fa27640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7810e6c0 con 0x7f1f78102fa0 2024-04-25T14:11:09.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.268+0000 7f1f7dfc4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f78104480 0x7f1f781a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.268+0000 7f1f7dfc4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f78104480 0x7f1f781a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44368/0 (socket says 172.21.15.96:44368) 2024-04-25T14:11:09.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.268+0000 7f1f7dfc4640 1 -- 172.21.15.96:0/1690118297 learned_addr learned my addr 172.21.15.96:0/1690118297 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:11:09.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.268+0000 7f1f7cfc2640 1 --2- 172.21.15.96:0/1690118297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f78103980 0x7f1f7819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.268+0000 7f1f7d7c3640 1 --2- 172.21.15.96:0/1690118297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1f78102fa0 0x7f1f781022a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.269+0000 7f1f7dfc4640 1 -- 172.21.15.96:0/1690118297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1f78102fa0 msgr2=0x7f1f781022a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:09.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.269+0000 7f1f7dfc4640 1 --2- 172.21.15.96:0/1690118297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1f78102fa0 0x7f1f781022a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.269+0000 7f1f7dfc4640 1 -- 172.21.15.96:0/1690118297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f78103980 msgr2=0x7f1f7819ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:09.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.269+0000 7f1f7dfc4640 1 --2- 172.21.15.96:0/1690118297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f78103980 0x7f1f7819ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.269+0000 7f1f7dfc4640 1 -- 172.21.15.96:0/1690118297 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f781a1d30 con 0x7f1f78104480 2024-04-25T14:11:09.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.269+0000 7f1f7dfc4640 1 --2- 172.21.15.96:0/1690118297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f78104480 0x7f1f781a1510 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1f7000eb10 tx=0x7f1f7000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:09.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.269+0000 7f1f6e7fc640 1 -- 172.21.15.96:0/1690118297 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f70004b20 con 0x7f1f78104480 2024-04-25T14:11:09.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.269+0000 7f1f6e7fc640 1 -- 172.21.15.96:0/1690118297 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f70004cc0 con 0x7f1f78104480 2024-04-25T14:11:09.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.269+0000 7f1f6e7fc640 1 -- 172.21.15.96:0/1690118297 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f70010670 con 0x7f1f78104480 2024-04-25T14:11:09.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.269+0000 7f1f7fa27640 1 -- 172.21.15.96:0/1690118297 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f781a1fc0 con 0x7f1f78104480 2024-04-25T14:11:09.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.269+0000 7f1f7fa27640 1 -- 172.21.15.96:0/1690118297 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f781a2430 con 0x7f1f78104480 2024-04-25T14:11:09.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.270+0000 7f1f7fa27640 1 -- 172.21.15.96:0/1690118297 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f78107940 con 0x7f1f78104480 2024-04-25T14:11:09.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.272+0000 7f1f6e7fc640 1 -- 172.21.15.96:0/1690118297 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f1f70005000 con 0x7f1f78104480 2024-04-25T14:11:09.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.272+0000 7f1f6e7fc640 1 --2- 172.21.15.96:0/1690118297 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1f640afaf0 0x7f1f640b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.272+0000 7f1f7d7c3640 1 --2- 172.21.15.96:0/1690118297 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1f640afaf0 0x7f1f640b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.272+0000 7f1f6e7fc640 1 -- 172.21.15.96:0/1690118297 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f1f700deec0 con 0x7f1f78104480 2024-04-25T14:11:09.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.273+0000 7f1f7d7c3640 1 --2- 172.21.15.96:0/1690118297 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1f640afaf0 0x7f1f640b2000 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1f74005fd0 tx=0x7f1f7400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:09.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.276+0000 7f1f6e7fc640 1 -- 172.21.15.96:0/1690118297 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1f70053ff0 con 0x7f1f78104480 2024-04-25T14:11:09.438 INFO:journalctl@ceph.mds.c.smithi151.stdout:Apr 25 14:11:09 smithi151 systemd[1]: Started Ceph mds.c for 59334182-030c-11ef-bc93-c7b262605968. 2024-04-25T14:11:09.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.495+0000 7f1f7fa27640 1 -- 172.21.15.96:0/1690118297 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f7802db50 con 0x7f1f78104480 2024-04-25T14:11:09.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.495+0000 7f1f6e7fc640 1 -- 172.21.15.96:0/1690118297 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6349 (secure 0 0 0) 0x7f1f700a7f40 con 0x7f1f78104480 2024-04-25T14:11:09.495 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:11:09.495 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/606227917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":606227917},{"type":"v1","addr":"172.21.15.96:6835","nonce":606227917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/3265300710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3265300710},{"type":"v1","addr":"172.21.15.96:6837","nonce":3265300710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/1995324831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1995324831},{"type":"v1","addr":"172.21.15.97:6833","nonce":1995324831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1382721608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1382721608},{"type":"v1","addr":"172.21.15.151:6833","nonce":1382721608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3872301726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3872301726},{"type":"v1","addr":"172.21.15.97:6835","nonce":3872301726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/1313823638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":1313823638},{"type":"v1","addr":"172.21.15.151:6835","nonce":1313823638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/2691348366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2691348366},{"type":"v1","addr":"172.21.15.151:6837","nonce":2691348366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-25T14:11:09.495 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 8 2024-04-25T14:11:09.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.497+0000 7f1f7fa27640 1 -- 172.21.15.96:0/1690118297 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1f640afaf0 msgr2=0x7f1f640b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:09.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.497+0000 7f1f7fa27640 1 --2- 172.21.15.96:0/1690118297 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1f640afaf0 0x7f1f640b2000 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1f74005fd0 tx=0x7f1f7400b390 comp rx=0 tx=0).stop 2024-04-25T14:11:09.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.497+0000 7f1f7fa27640 1 -- 172.21.15.96:0/1690118297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f78104480 msgr2=0x7f1f781a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:09.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.497+0000 7f1f7fa27640 1 --2- 172.21.15.96:0/1690118297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f78104480 0x7f1f781a1510 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1f7000eb10 tx=0x7f1f7000c2d0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.497+0000 7f1f7fa27640 1 -- 172.21.15.96:0/1690118297 shutdown_connections 2024-04-25T14:11:09.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.497+0000 7f1f7fa27640 1 --2- 172.21.15.96:0/1690118297 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1f640afaf0 0x7f1f640b2000 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.497+0000 7f1f7fa27640 1 --2- 172.21.15.96:0/1690118297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f78104480 0x7f1f781a1510 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.497+0000 7f1f7fa27640 1 --2- 172.21.15.96:0/1690118297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f78103980 0x7f1f7819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.497+0000 7f1f7fa27640 1 --2- 172.21.15.96:0/1690118297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1f78102fa0 0x7f1f781022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.497+0000 7f1f7fa27640 1 -- 172.21.15.96:0/1690118297 >> 172.21.15.96:0/1690118297 conn(0x7f1f780fd360 msgr2=0x7f1f78104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:09.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.497+0000 7f1f7fa27640 1 -- 172.21.15.96:0/1690118297 shutdown_connections 2024-04-25T14:11:09.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:09.497+0000 7f1f7fa27640 1 -- 172.21.15.96:0/1690118297 wait complete. 2024-04-25T14:11:10.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:10 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:10.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:10 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:10.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:10 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:10.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:10 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:11:10.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:10 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:11:10.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:10 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:11:10.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:10 smithi096 ceph-mon[29207]: Deploying daemon mds.h on smithi097 2024-04-25T14:11:10.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:10 smithi096 ceph-mon[29207]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:10.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:10 smithi096 ceph-mon[29207]: mds.? [v2:172.21.15.151:6836/2691348366,v1:172.21.15.151:6837/2691348366] up:boot 2024-04-25T14:11:10.360 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:10 smithi096 ceph-mon[29207]: fsmap 7 up:standby 2024-04-25T14:11:10.360 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:10 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T14:11:10.360 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:10 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/1690118297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:10.422 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:10 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:10.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:10 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:10.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:10 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:10.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:10 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:11:10.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:10 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:11:10.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:10 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:11:10.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:10 smithi097 ceph-mon[32720]: Deploying daemon mds.h on smithi097 2024-04-25T14:11:10.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:10 smithi097 ceph-mon[32720]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:10.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:10 smithi097 ceph-mon[32720]: mds.? [v2:172.21.15.151:6836/2691348366,v1:172.21.15.151:6837/2691348366] up:boot 2024-04-25T14:11:10.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:10 smithi097 ceph-mon[32720]: fsmap 7 up:standby 2024-04-25T14:11:10.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:10 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T14:11:10.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:10 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/1690118297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:10.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:10 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:10.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:10 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:10.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:10 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:10.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:10 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:11:10.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:10 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:11:10.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:10 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:11:10.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:10 smithi151 ceph-mon[32990]: Deploying daemon mds.h on smithi097 2024-04-25T14:11:10.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:10 smithi151 ceph-mon[32990]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:10.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:10 smithi151 ceph-mon[32990]: mds.? [v2:172.21.15.151:6836/2691348366,v1:172.21.15.151:6837/2691348366] up:boot 2024-04-25T14:11:10.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:10 smithi151 ceph-mon[32990]: fsmap 7 up:standby 2024-04-25T14:11:10.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:10 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T14:11:10.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:10 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/1690118297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:11.509 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:11:11.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.630+0000 7f45bae30640 1 -- 172.21.15.96:0/780092654 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45b4109750 msgr2=0x7f45b410bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:11.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.630+0000 7f45bae30640 1 --2- 172.21.15.96:0/780092654 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45b4109750 0x7f45b410bc60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f45ac009ca0 tx=0x7f45ac02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.630+0000 7f45bae30640 1 -- 172.21.15.96:0/780092654 shutdown_connections 2024-04-25T14:11:11.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.630+0000 7f45bae30640 1 --2- 172.21.15.96:0/780092654 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45b4109750 0x7f45b410bc60 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.630+0000 7f45bae30640 1 --2- 172.21.15.96:0/780092654 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45b4106be0 0x7f45b41090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.630+0000 7f45bae30640 1 --2- 172.21.15.96:0/780092654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45b41041f0 0x7f45b4106610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.630+0000 7f45bae30640 1 -- 172.21.15.96:0/780092654 >> 172.21.15.96:0/780092654 conn(0x7f45b40fdbd0 msgr2=0x7f45b4100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:11.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.630+0000 7f45bae30640 1 -- 172.21.15.96:0/780092654 shutdown_connections 2024-04-25T14:11:11.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.630+0000 7f45bae30640 1 -- 172.21.15.96:0/780092654 wait complete. 2024-04-25T14:11:11.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.630+0000 7f45bae30640 1 Processor -- start 2024-04-25T14:11:11.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.630+0000 7f45bae30640 1 -- start start 2024-04-25T14:11:11.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.631+0000 7f45bae30640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45b41041f0 0x7f45b419a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:11.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.631+0000 7f45bae30640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45b4106be0 0x7f45b419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:11.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.631+0000 7f45bae30640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45b4109750 0x7f45b41a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:11.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.631+0000 7f45bae30640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f45b410f5c0 con 0x7f45b4106be0 2024-04-25T14:11:11.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.631+0000 7f45bae30640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f45b410f440 con 0x7f45b4109750 2024-04-25T14:11:11.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.631+0000 7f45bae30640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f45b410f740 con 0x7f45b41041f0 2024-04-25T14:11:11.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.631+0000 7f45b3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45b4106be0 0x7f45b419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:11.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.631+0000 7f45b3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45b4106be0 0x7f45b419a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58810/0 (socket says 172.21.15.96:58810) 2024-04-25T14:11:11.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.631+0000 7f45b3fff640 1 -- 172.21.15.96:0/1265241672 learned_addr learned my addr 172.21.15.96:0/1265241672 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:11:11.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.631+0000 7f45b8bcc640 1 --2- 172.21.15.96:0/1265241672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45b41041f0 0x7f45b419a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:11.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.631+0000 7f45b93cd640 1 --2- 172.21.15.96:0/1265241672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45b4109750 0x7f45b41a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:11.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.631+0000 7f45b3fff640 1 -- 172.21.15.96:0/1265241672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45b41041f0 msgr2=0x7f45b419a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:11.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.631+0000 7f45b3fff640 1 --2- 172.21.15.96:0/1265241672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45b41041f0 0x7f45b419a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.631+0000 7f45b3fff640 1 -- 172.21.15.96:0/1265241672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45b4109750 msgr2=0x7f45b41a1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:11.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.631+0000 7f45b3fff640 1 --2- 172.21.15.96:0/1265241672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45b4109750 0x7f45b41a1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.631+0000 7f45b3fff640 1 -- 172.21.15.96:0/1265241672 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45b41a1ca0 con 0x7f45b4106be0 2024-04-25T14:11:11.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.631+0000 7f45b93cd640 1 --2- 172.21.15.96:0/1265241672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45b4109750 0x7f45b41a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:11.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.631+0000 7f45b3fff640 1 --2- 172.21.15.96:0/1265241672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45b4106be0 0x7f45b419a6a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f45a800d770 tx=0x7f45a800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:11.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.631+0000 7f45b1ffb640 1 -- 172.21.15.96:0/1265241672 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a8017070 con 0x7f45b4106be0 2024-04-25T14:11:11.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.631+0000 7f45b1ffb640 1 -- 172.21.15.96:0/1265241672 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45a80054d0 con 0x7f45b4106be0 2024-04-25T14:11:11.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.631+0000 7f45bae30640 1 -- 172.21.15.96:0/1265241672 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45b41a1f90 con 0x7f45b4106be0 2024-04-25T14:11:11.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.631+0000 7f45b1ffb640 1 -- 172.21.15.96:0/1265241672 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a8004c30 con 0x7f45b4106be0 2024-04-25T14:11:11.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.632+0000 7f45bae30640 1 -- 172.21.15.96:0/1265241672 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45b41a2400 con 0x7f45b4106be0 2024-04-25T14:11:11.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.633+0000 7f45bae30640 1 -- 172.21.15.96:0/1265241672 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45b40ff6f0 con 0x7f45b4106be0 2024-04-25T14:11:11.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.634+0000 7f45b1ffb640 1 -- 172.21.15.96:0/1265241672 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f45a8004e10 con 0x7f45b4106be0 2024-04-25T14:11:11.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.634+0000 7f45b1ffb640 1 --2- 172.21.15.96:0/1265241672 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f459c0afaf0 0x7f459c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:11.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.635+0000 7f45b8bcc640 1 --2- 172.21.15.96:0/1265241672 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f459c0afaf0 0x7f459c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:11.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.635+0000 7f45b1ffb640 1 -- 172.21.15.96:0/1265241672 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f45a80de240 con 0x7f45b4106be0 2024-04-25T14:11:11.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.635+0000 7f45b8bcc640 1 --2- 172.21.15.96:0/1265241672 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f459c0afaf0 0x7f459c0b2000 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f45b406a850 tx=0x7f45a4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:11.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.639+0000 7f45b1ffb640 1 -- 172.21.15.96:0/1265241672 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f45a80a72c0 con 0x7f45b4106be0 2024-04-25T14:11:11.647 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 14:11:11 smithi097 systemd[1]: Starting Ceph mds.h for 59334182-030c-11ef-bc93-c7b262605968... 2024-04-25T14:11:11.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.856+0000 7f45bae30640 1 -- 172.21.15.96:0/1265241672 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f45b402dd00 con 0x7f45b4106be0 2024-04-25T14:11:11.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.857+0000 7f45b1ffb640 1 -- 172.21.15.96:0/1265241672 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6349 (secure 0 0 0) 0x7f45a80a6bc0 con 0x7f45b4106be0 2024-04-25T14:11:11.856 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:11:11.856 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/606227917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":606227917},{"type":"v1","addr":"172.21.15.96:6835","nonce":606227917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/3265300710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3265300710},{"type":"v1","addr":"172.21.15.96:6837","nonce":3265300710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/1995324831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1995324831},{"type":"v1","addr":"172.21.15.97:6833","nonce":1995324831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1382721608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1382721608},{"type":"v1","addr":"172.21.15.151:6833","nonce":1382721608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3872301726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3872301726},{"type":"v1","addr":"172.21.15.97:6835","nonce":3872301726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/1313823638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":1313823638},{"type":"v1","addr":"172.21.15.151:6835","nonce":1313823638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/2691348366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2691348366},{"type":"v1","addr":"172.21.15.151:6837","nonce":2691348366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-25T14:11:11.857 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 8 2024-04-25T14:11:11.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.859+0000 7f45bae30640 1 -- 172.21.15.96:0/1265241672 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f459c0afaf0 msgr2=0x7f459c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:11.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.859+0000 7f45bae30640 1 --2- 172.21.15.96:0/1265241672 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f459c0afaf0 0x7f459c0b2000 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f45b406a850 tx=0x7f45a4005f90 comp rx=0 tx=0).stop 2024-04-25T14:11:11.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.859+0000 7f45bae30640 1 -- 172.21.15.96:0/1265241672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45b4106be0 msgr2=0x7f45b419a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:11.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.859+0000 7f45bae30640 1 --2- 172.21.15.96:0/1265241672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45b4106be0 0x7f45b419a6a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f45a800d770 tx=0x7f45a800dc30 comp rx=0 tx=0).stop 2024-04-25T14:11:11.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.859+0000 7f45bae30640 1 -- 172.21.15.96:0/1265241672 shutdown_connections 2024-04-25T14:11:11.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.859+0000 7f45bae30640 1 --2- 172.21.15.96:0/1265241672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45b4109750 0x7f45b41a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.859+0000 7f45bae30640 1 --2- 172.21.15.96:0/1265241672 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f459c0afaf0 0x7f459c0b2000 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.859+0000 7f45bae30640 1 --2- 172.21.15.96:0/1265241672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45b4106be0 0x7f45b419a6a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.859+0000 7f45bae30640 1 --2- 172.21.15.96:0/1265241672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45b41041f0 0x7f45b419a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.859+0000 7f45bae30640 1 -- 172.21.15.96:0/1265241672 >> 172.21.15.96:0/1265241672 conn(0x7f45b40fdbd0 msgr2=0x7f45b4104ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:11.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.859+0000 7f45bae30640 1 -- 172.21.15.96:0/1265241672 shutdown_connections 2024-04-25T14:11:11.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:11.859+0000 7f45bae30640 1 -- 172.21.15.96:0/1265241672 wait complete. 2024-04-25T14:11:11.919 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 14:11:11 smithi097 podman[62900]: 2024-04-25 14:11:11.550732376 +0000 UTC m=+0.017649970 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:11:11.919 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 14:11:11 smithi097 podman[62900]: 2024-04-25 14:11:11.676194007 +0000 UTC m=+0.143111592 container create 84a82aacbfa9e08877fda56721a7dded8656c9062ebeaa51822061a3f6669d84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T14:11:11.919 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 14:11:11 smithi097 podman[62900]: 2024-04-25 14:11:11.920456037 +0000 UTC m=+0.387373626 container init 84a82aacbfa9e08877fda56721a7dded8656c9062ebeaa51822061a3f6669d84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-25T14:11:12.172 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 14:11:11 smithi097 podman[62900]: 2024-04-25 14:11:11.922994506 +0000 UTC m=+0.389912094 container start 84a82aacbfa9e08877fda56721a7dded8656c9062ebeaa51822061a3f6669d84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-h, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-25T14:11:12.173 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 14:11:11 smithi097 ceph-mds[62919]: -- 172.21.15.97:0/969850124 <== mon.2 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55be93e5ba40 con 0x55be93ea3180 2024-04-25T14:11:12.173 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 14:11:11 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-h[62915]: starting mds.h at 2024-04-25T14:11:12.173 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 14:11:12 smithi097 bash[62900]: 84a82aacbfa9e08877fda56721a7dded8656c9062ebeaa51822061a3f6669d84 2024-04-25T14:11:12.173 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 14:11:12 smithi097 systemd[1]: Started Ceph mds.h for 59334182-030c-11ef-bc93-c7b262605968. 2024-04-25T14:11:12.759 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:12 smithi151 ceph-mon[32990]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:12.760 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:12 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/1265241672' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:12.760 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:12 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:12.760 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:12 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:12.760 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:12 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:12.760 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:12 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:11:12.760 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:12 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:11:12.760 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:12 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:11:12.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:12 smithi096 ceph-mon[29207]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:12.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:12 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/1265241672' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:12.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:12 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:12.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:12 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:12.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:12 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:12.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:12 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:11:12.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:12 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:11:12.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:12 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:11:12.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:12 smithi097 ceph-mon[32720]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:12.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:12 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/1265241672' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:12.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:12 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:12.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:12 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:12.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:12 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:12.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:12 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:11:12.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:12 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:11:12.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:12 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:11:13.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:13 smithi097 ceph-mon[32720]: Deploying daemon mds.i on smithi151 2024-04-25T14:11:13.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:13 smithi097 ceph-mon[32720]: mds.? [v2:172.21.15.97:6836/3306780081,v1:172.21.15.97:6837/3306780081] up:boot 2024-04-25T14:11:13.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:13 smithi097 ceph-mon[32720]: fsmap 8 up:standby 2024-04-25T14:11:13.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:13 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T14:11:13.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:13 smithi096 ceph-mon[29207]: Deploying daemon mds.i on smithi151 2024-04-25T14:11:13.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:13 smithi096 ceph-mon[29207]: mds.? [v2:172.21.15.97:6836/3306780081,v1:172.21.15.97:6837/3306780081] up:boot 2024-04-25T14:11:13.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:13 smithi096 ceph-mon[29207]: fsmap 8 up:standby 2024-04-25T14:11:13.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:13 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T14:11:13.867 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:13 smithi151 ceph-mon[32990]: Deploying daemon mds.i on smithi151 2024-04-25T14:11:13.867 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:13 smithi151 ceph-mon[32990]: mds.? [v2:172.21.15.97:6836/3306780081,v1:172.21.15.97:6837/3306780081] up:boot 2024-04-25T14:11:13.867 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:13 smithi151 ceph-mon[32990]: fsmap 8 up:standby 2024-04-25T14:11:13.867 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:13 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T14:11:13.871 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:11:13.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:13.992+0000 7fa246314640 1 -- 172.21.15.96:0/3413946752 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2401029c0 msgr2=0x7fa2401036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:13.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:13.992+0000 7fa246314640 1 --2- 172.21.15.96:0/3413946752 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2401029c0 0x7fa2401036c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa230009900 tx=0x7fa23002a410 comp rx=0 tx=0).stop 2024-04-25T14:11:13.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:13.992+0000 7fa246314640 1 -- 172.21.15.96:0/3413946752 shutdown_connections 2024-04-25T14:11:13.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:13.992+0000 7fa246314640 1 --2- 172.21.15.96:0/3413946752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa240108900 0x7fa24010ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:13.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:13.992+0000 7fa246314640 1 --2- 172.21.15.96:0/3413946752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa240103c00 0x7fa240104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:13.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:13.992+0000 7fa246314640 1 --2- 172.21.15.96:0/3413946752 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2401029c0 0x7fa2401036c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:13.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:13.992+0000 7fa246314640 1 -- 172.21.15.96:0/3413946752 >> 172.21.15.96:0/3413946752 conn(0x7fa2400fdbd0 msgr2=0x7fa240100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:13.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:13.992+0000 7fa246314640 1 -- 172.21.15.96:0/3413946752 shutdown_connections 2024-04-25T14:11:13.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:13.992+0000 7fa246314640 1 -- 172.21.15.96:0/3413946752 wait complete. 2024-04-25T14:11:13.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:13.992+0000 7fa246314640 1 Processor -- start 2024-04-25T14:11:13.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:13.992+0000 7fa246314640 1 -- start start 2024-04-25T14:11:13.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:13.993+0000 7fa246314640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2401029c0 0x7fa240199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:13.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:13.993+0000 7fa246314640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa240103c00 0x7fa240199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:13.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:13.993+0000 7fa246314640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa240108900 0x7fa2401a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:13.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:13.993+0000 7fa246314640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa24010e4c0 con 0x7fa240108900 2024-04-25T14:11:13.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:13.993+0000 7fa246314640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa24010e340 con 0x7fa2401029c0 2024-04-25T14:11:13.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:13.993+0000 7fa246314640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa24010e640 con 0x7fa240103c00 2024-04-25T14:11:13.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:13.993+0000 7fa2448b1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa240108900 0x7fa2401a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:13.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:13.993+0000 7fa2448b1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa240108900 0x7fa2401a0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58832/0 (socket says 172.21.15.96:58832) 2024-04-25T14:11:13.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:13.993+0000 7fa2448b1640 1 -- 172.21.15.96:0/2600628897 learned_addr learned my addr 172.21.15.96:0/2600628897 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:11:13.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:13.993+0000 7fa23f7fe640 1 --2- 172.21.15.96:0/2600628897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa240103c00 0x7fa240199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:13.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:13.993+0000 7fa2448b1640 1 -- 172.21.15.96:0/2600628897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa240103c00 msgr2=0x7fa240199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:13.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:13.993+0000 7fa2448b1640 1 --2- 172.21.15.96:0/2600628897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa240103c00 0x7fa240199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:13.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:13.993+0000 7fa2448b1640 1 -- 172.21.15.96:0/2600628897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2401029c0 msgr2=0x7fa240199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:13.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:13.993+0000 7fa2448b1640 1 --2- 172.21.15.96:0/2600628897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2401029c0 0x7fa240199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:13.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:13.993+0000 7fa2448b1640 1 -- 172.21.15.96:0/2600628897 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2401a0c70 con 0x7fa240108900 2024-04-25T14:11:13.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:13.993+0000 7fa2448b1640 1 --2- 172.21.15.96:0/2600628897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa240108900 0x7fa2401a0600 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa23800eb10 tx=0x7fa23800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:13.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:13.994+0000 7fa23d7fa640 1 -- 172.21.15.96:0/2600628897 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa238004b20 con 0x7fa240108900 2024-04-25T14:11:13.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:13.994+0000 7fa23d7fa640 1 -- 172.21.15.96:0/2600628897 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa238004cc0 con 0x7fa240108900 2024-04-25T14:11:13.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:13.994+0000 7fa246314640 1 -- 172.21.15.96:0/2600628897 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2401a0f60 con 0x7fa240108900 2024-04-25T14:11:13.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:13.994+0000 7fa23d7fa640 1 -- 172.21.15.96:0/2600628897 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa238010670 con 0x7fa240108900 2024-04-25T14:11:13.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:13.994+0000 7fa246314640 1 -- 172.21.15.96:0/2600628897 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2401a1370 con 0x7fa240108900 2024-04-25T14:11:13.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:13.995+0000 7fa246314640 1 -- 172.21.15.96:0/2600628897 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa24006afa0 con 0x7fa240108900 2024-04-25T14:11:13.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:13.996+0000 7fa23d7fa640 1 -- 172.21.15.96:0/2600628897 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fa238005000 con 0x7fa240108900 2024-04-25T14:11:13.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:13.997+0000 7fa23d7fa640 1 --2- 172.21.15.96:0/2600628897 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa2280afaf0 0x7fa2280b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:13.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:13.997+0000 7fa23ffff640 1 --2- 172.21.15.96:0/2600628897 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa2280afaf0 0x7fa2280b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:13.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:13.997+0000 7fa23d7fa640 1 -- 172.21.15.96:0/2600628897 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fa2380def70 con 0x7fa240108900 2024-04-25T14:11:13.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:13.998+0000 7fa23ffff640 1 --2- 172.21.15.96:0/2600628897 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa2280afaf0 0x7fa2280b2000 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa230031500 tx=0x7fa230031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:14.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:14.001+0000 7fa23d7fa640 1 -- 172.21.15.96:0/2600628897 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa238053ff0 con 0x7fa240108900 2024-04-25T14:11:14.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:14.215+0000 7fa246314640 1 -- 172.21.15.96:0/2600628897 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa24002dc10 con 0x7fa240108900 2024-04-25T14:11:14.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:14.216+0000 7fa23d7fa640 1 -- 172.21.15.96:0/2600628897 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7174 (secure 0 0 0) 0x7fa2380a7ff0 con 0x7fa240108900 2024-04-25T14:11:14.216 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:11:14.216 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/606227917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":606227917},{"type":"v1","addr":"172.21.15.96:6835","nonce":606227917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/3265300710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3265300710},{"type":"v1","addr":"172.21.15.96:6837","nonce":3265300710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/1995324831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1995324831},{"type":"v1","addr":"172.21.15.97:6833","nonce":1995324831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1382721608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1382721608},{"type":"v1","addr":"172.21.15.151:6833","nonce":1382721608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3872301726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3872301726},{"type":"v1","addr":"172.21.15.97:6835","nonce":3872301726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/1313823638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":1313823638},{"type":"v1","addr":"172.21.15.151:6835","nonce":1313823638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/2691348366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2691348366},{"type":"v1","addr":"172.21.15.151:6837","nonce":2691348366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24497,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3306780081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3306780081},{"type":"v1","addr":"172.21.15.97:6837","nonce":3306780081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-25T14:11:14.217 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 9 2024-04-25T14:11:14.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:14.218+0000 7fa246314640 1 -- 172.21.15.96:0/2600628897 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa2280afaf0 msgr2=0x7fa2280b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:14.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:14.218+0000 7fa246314640 1 --2- 172.21.15.96:0/2600628897 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa2280afaf0 0x7fa2280b2000 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa230031500 tx=0x7fa230031400 comp rx=0 tx=0).stop 2024-04-25T14:11:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:14.218+0000 7fa246314640 1 -- 172.21.15.96:0/2600628897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa240108900 msgr2=0x7fa2401a0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:14.218+0000 7fa246314640 1 --2- 172.21.15.96:0/2600628897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa240108900 0x7fa2401a0600 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa23800eb10 tx=0x7fa23800c2d0 comp rx=0 tx=0).stop 2024-04-25T14:11:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:14.218+0000 7fa246314640 1 -- 172.21.15.96:0/2600628897 shutdown_connections 2024-04-25T14:11:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:14.218+0000 7fa246314640 1 --2- 172.21.15.96:0/2600628897 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa2280afaf0 0x7fa2280b2000 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:14.218+0000 7fa246314640 1 --2- 172.21.15.96:0/2600628897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa240108900 0x7fa2401a0600 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:14.218+0000 7fa246314640 1 --2- 172.21.15.96:0/2600628897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa240103c00 0x7fa240199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:14.218+0000 7fa246314640 1 --2- 172.21.15.96:0/2600628897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2401029c0 0x7fa240199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:14.218+0000 7fa246314640 1 -- 172.21.15.96:0/2600628897 >> 172.21.15.96:0/2600628897 conn(0x7fa2400fdbd0 msgr2=0x7fa2401048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:14.218+0000 7fa246314640 1 -- 172.21.15.96:0/2600628897 shutdown_connections 2024-04-25T14:11:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:14.218+0000 7fa246314640 1 -- 172.21.15.96:0/2600628897 wait complete. 2024-04-25T14:11:14.438 INFO:journalctl@ceph.mds.i.smithi151.stdout:Apr 25 14:11:14 smithi151 systemd[1]: Starting Ceph mds.i for 59334182-030c-11ef-bc93-c7b262605968... 2024-04-25T14:11:14.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:14 smithi096 ceph-mon[29207]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:14.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:14 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/2600628897' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:14.721 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:14 smithi151 ceph-mon[32990]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:14.721 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:14 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/2600628897' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:14.722 INFO:journalctl@ceph.mds.i.smithi151.stdout:Apr 25 14:11:14 smithi151 podman[63847]: 2024-04-25 14:11:14.468964452 +0000 UTC m=+0.018345194 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:11:14.722 INFO:journalctl@ceph.mds.i.smithi151.stdout:Apr 25 14:11:14 smithi151 podman[63847]: 2024-04-25 14:11:14.587892306 +0000 UTC m=+0.137273033 container create e8f5258ca3f52470872c1addca1198027f1c47ec8e543a26193f9ebc7352414e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-i, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424) 2024-04-25T14:11:14.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:14 smithi097 ceph-mon[32720]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:14.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:14 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/2600628897' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:15.188 INFO:journalctl@ceph.mds.i.smithi151.stdout:Apr 25 14:11:14 smithi151 podman[63847]: 2024-04-25 14:11:14.85759684 +0000 UTC m=+0.406977566 container init e8f5258ca3f52470872c1addca1198027f1c47ec8e543a26193f9ebc7352414e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-i, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-25T14:11:15.188 INFO:journalctl@ceph.mds.i.smithi151.stdout:Apr 25 14:11:14 smithi151 podman[63847]: 2024-04-25 14:11:14.860265307 +0000 UTC m=+0.409646033 container start e8f5258ca3f52470872c1addca1198027f1c47ec8e543a26193f9ebc7352414e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-i, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-25T14:11:15.188 INFO:journalctl@ceph.mds.i.smithi151.stdout:Apr 25 14:11:14 smithi151 ceph-mds[63867]: -- 172.21.15.151:0/87295952 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ee9f509a40 con 0x55ee9f551600 2024-04-25T14:11:15.188 INFO:journalctl@ceph.mds.i.smithi151.stdout:Apr 25 14:11:14 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-i[63862]: starting mds.i at 2024-04-25T14:11:15.188 INFO:journalctl@ceph.mds.i.smithi151.stdout:Apr 25 14:11:14 smithi151 bash[63847]: e8f5258ca3f52470872c1addca1198027f1c47ec8e543a26193f9ebc7352414e 2024-04-25T14:11:15.188 INFO:journalctl@ceph.mds.i.smithi151.stdout:Apr 25 14:11:14 smithi151 systemd[1]: Started Ceph mds.i for 59334182-030c-11ef-bc93-c7b262605968. 2024-04-25T14:11:16.230 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:11:16.253 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:16 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:16.253 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:16 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:16.253 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:16 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:16.253 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:16 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:11:16.253 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:16 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:11:16.254 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:16 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:11:16.254 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:16 smithi096 ceph-mon[29207]: Deploying daemon mds.g on smithi096 2024-04-25T14:11:16.254 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:16 smithi096 ceph-mon[29207]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:16.254 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:16 smithi096 ceph-mon[29207]: mds.? [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] up:boot 2024-04-25T14:11:16.254 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:16 smithi096 ceph-mon[29207]: fsmap 9 up:standby 2024-04-25T14:11:16.254 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:16 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T14:11:16.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.342+0000 7fcc07227640 1 -- 172.21.15.96:0/708849265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc00109a40 msgr2=0x7fcc0010be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:16.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.342+0000 7fcc07227640 1 --2- 172.21.15.96:0/708849265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc00109a40 0x7fcc0010be60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8009ca0 tx=0x7fcbf802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:11:16.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.342+0000 7fcc07227640 1 -- 172.21.15.96:0/708849265 shutdown_connections 2024-04-25T14:11:16.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.342+0000 7fcc07227640 1 --2- 172.21.15.96:0/708849265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc00109a40 0x7fcc0010be60 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:16.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.342+0000 7fcc07227640 1 --2- 172.21.15.96:0/708849265 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcc00104d40 0x7fcc00105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:16.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.342+0000 7fcc07227640 1 --2- 172.21.15.96:0/708849265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc00103b00 0x7fcc00104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:16.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.342+0000 7fcc07227640 1 -- 172.21.15.96:0/708849265 >> 172.21.15.96:0/708849265 conn(0x7fcc000fed10 msgr2=0x7fcc001011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:16.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.342+0000 7fcc07227640 1 -- 172.21.15.96:0/708849265 shutdown_connections 2024-04-25T14:11:16.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.342+0000 7fcc07227640 1 -- 172.21.15.96:0/708849265 wait complete. 2024-04-25T14:11:16.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.343+0000 7fcc07227640 1 Processor -- start 2024-04-25T14:11:16.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.343+0000 7fcc07227640 1 -- start start 2024-04-25T14:11:16.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.343+0000 7fcc07227640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc00103b00 0x7fcc0019a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:16.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.343+0000 7fcc07227640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcc00104d40 0x7fcc0019a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:16.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.343+0000 7fcc07227640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc00109a40 0x7fcc001a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:16.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.343+0000 7fcc057c4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc00109a40 0x7fcc001a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:16.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.343+0000 7fcc057c4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc00109a40 0x7fcc001a3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58862/0 (socket says 172.21.15.96:58862) 2024-04-25T14:11:16.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.343+0000 7fcc057c4640 1 -- 172.21.15.96:0/422958366 learned_addr learned my addr 172.21.15.96:0/422958366 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:11:16.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.343+0000 7fcc07227640 1 -- 172.21.15.96:0/422958366 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0010f610 con 0x7fcc00109a40 2024-04-25T14:11:16.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.343+0000 7fcc07227640 1 -- 172.21.15.96:0/422958366 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0010f490 con 0x7fcc00103b00 2024-04-25T14:11:16.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.343+0000 7fcc07227640 1 -- 172.21.15.96:0/422958366 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0010f790 con 0x7fcc00104d40 2024-04-25T14:11:16.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.344+0000 7fcc04fc3640 1 --2- 172.21.15.96:0/422958366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc00103b00 0x7fcc0019a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:16.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.344+0000 7fcbfffff640 1 --2- 172.21.15.96:0/422958366 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcc00104d40 0x7fcc0019a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:16.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.344+0000 7fcc057c4640 1 -- 172.21.15.96:0/422958366 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcc00104d40 msgr2=0x7fcc0019a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:16.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.344+0000 7fcc057c4640 1 --2- 172.21.15.96:0/422958366 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcc00104d40 0x7fcc0019a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:16.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.344+0000 7fcc057c4640 1 -- 172.21.15.96:0/422958366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc00103b00 msgr2=0x7fcc0019a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:16.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.344+0000 7fcc057c4640 1 --2- 172.21.15.96:0/422958366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc00103b00 0x7fcc0019a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:16.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.344+0000 7fcc057c4640 1 -- 172.21.15.96:0/422958366 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc001a3fa0 con 0x7fcc00109a40 2024-04-25T14:11:16.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.344+0000 7fcc057c4640 1 --2- 172.21.15.96:0/422958366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc00109a40 0x7fcc001a3a60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8006010 tx=0x7fcbf80039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:16.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.344+0000 7fcbfdffb640 1 -- 172.21.15.96:0/422958366 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8003de0 con 0x7fcc00109a40 2024-04-25T14:11:16.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.344+0000 7fcc07227640 1 -- 172.21.15.96:0/422958366 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc001a4170 con 0x7fcc00109a40 2024-04-25T14:11:16.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.344+0000 7fcbfdffb640 1 -- 172.21.15.96:0/422958366 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbf80044f0 con 0x7fcc00109a40 2024-04-25T14:11:16.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.344+0000 7fcbfdffb640 1 -- 172.21.15.96:0/422958366 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf802fce0 con 0x7fcc00109a40 2024-04-25T14:11:16.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.345+0000 7fcc07227640 1 -- 172.21.15.96:0/422958366 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc001a4660 con 0x7fcc00109a40 2024-04-25T14:11:16.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.346+0000 7fcc07227640 1 -- 172.21.15.96:0/422958366 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc00108850 con 0x7fcc00109a40 2024-04-25T14:11:16.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.347+0000 7fcbfdffb640 1 -- 172.21.15.96:0/422958366 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fcbf80046a0 con 0x7fcc00109a40 2024-04-25T14:11:16.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.348+0000 7fcbfdffb640 1 --2- 172.21.15.96:0/422958366 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fcbf00afaf0 0x7fcbf00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:16.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.348+0000 7fcc04fc3640 1 --2- 172.21.15.96:0/422958366 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fcbf00afaf0 0x7fcbf00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:16.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.348+0000 7fcbfdffb640 1 -- 172.21.15.96:0/422958366 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fcbf8100150 con 0x7fcc00109a40 2024-04-25T14:11:16.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.348+0000 7fcc04fc3640 1 --2- 172.21.15.96:0/422958366 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fcbf00afaf0 0x7fcbf00b2000 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcc00104700 tx=0x7fcbf4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:16.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.353+0000 7fcbfdffb640 1 -- 172.21.15.96:0/422958366 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcbf80c91b0 con 0x7fcc00109a40 2024-04-25T14:11:16.422 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:16.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:16.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:16.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:11:16.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:11:16.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:11:16.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:16 smithi097 ceph-mon[32720]: Deploying daemon mds.g on smithi096 2024-04-25T14:11:16.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:16 smithi097 ceph-mon[32720]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:16.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:16 smithi097 ceph-mon[32720]: mds.? [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] up:boot 2024-04-25T14:11:16.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:16 smithi097 ceph-mon[32720]: fsmap 9 up:standby 2024-04-25T14:11:16.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:16 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T14:11:16.437 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:16 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:16.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:16 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:16.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:16 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:16.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:16 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:11:16.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:16 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:11:16.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:16 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:11:16.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:16 smithi151 ceph-mon[32990]: Deploying daemon mds.g on smithi096 2024-04-25T14:11:16.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:16 smithi151 ceph-mon[32990]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:16.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:16 smithi151 ceph-mon[32990]: mds.? [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] up:boot 2024-04-25T14:11:16.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:16 smithi151 ceph-mon[32990]: fsmap 9 up:standby 2024-04-25T14:11:16.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:16 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T14:11:16.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.573+0000 7fcc07227640 1 -- 172.21.15.96:0/422958366 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcc0002db50 con 0x7fcc00109a40 2024-04-25T14:11:16.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.573+0000 7fcbfdffb640 1 -- 172.21.15.96:0/422958366 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8001 (secure 0 0 0) 0x7fcbf80c8ab0 con 0x7fcc00109a40 2024-04-25T14:11:16.572 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:11:16.572 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/606227917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":606227917},{"type":"v1","addr":"172.21.15.96:6835","nonce":606227917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/3265300710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3265300710},{"type":"v1","addr":"172.21.15.96:6837","nonce":3265300710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/1995324831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1995324831},{"type":"v1","addr":"172.21.15.97:6833","nonce":1995324831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1382721608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1382721608},{"type":"v1","addr":"172.21.15.151:6833","nonce":1382721608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3872301726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3872301726},{"type":"v1","addr":"172.21.15.97:6835","nonce":3872301726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/1313823638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":1313823638},{"type":"v1","addr":"172.21.15.151:6835","nonce":1313823638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/2691348366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2691348366},{"type":"v1","addr":"172.21.15.151:6837","nonce":2691348366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24497,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3306780081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3306780081},{"type":"v1","addr":"172.21.15.97:6837","nonce":3306780081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24517,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/245699981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":245699981},{"type":"v1","addr":"172.21.15.151:6839","nonce":245699981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T14:11:16.572 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 10 2024-04-25T14:11:16.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.575+0000 7fcc07227640 1 -- 172.21.15.96:0/422958366 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fcbf00afaf0 msgr2=0x7fcbf00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:16.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.575+0000 7fcc07227640 1 --2- 172.21.15.96:0/422958366 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fcbf00afaf0 0x7fcbf00b2000 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcc00104700 tx=0x7fcbf4005f90 comp rx=0 tx=0).stop 2024-04-25T14:11:16.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.575+0000 7fcc07227640 1 -- 172.21.15.96:0/422958366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc00109a40 msgr2=0x7fcc001a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:16.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.575+0000 7fcc07227640 1 --2- 172.21.15.96:0/422958366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc00109a40 0x7fcc001a3a60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8006010 tx=0x7fcbf80039f0 comp rx=0 tx=0).stop 2024-04-25T14:11:16.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.575+0000 7fcc07227640 1 -- 172.21.15.96:0/422958366 shutdown_connections 2024-04-25T14:11:16.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.575+0000 7fcc07227640 1 --2- 172.21.15.96:0/422958366 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fcbf00afaf0 0x7fcbf00b2000 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:16.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.575+0000 7fcc07227640 1 --2- 172.21.15.96:0/422958366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc00109a40 0x7fcc001a3a60 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:16.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.575+0000 7fcc07227640 1 --2- 172.21.15.96:0/422958366 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcc00104d40 0x7fcc0019a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:16.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.575+0000 7fcc07227640 1 --2- 172.21.15.96:0/422958366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc00103b00 0x7fcc0019a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:16.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.576+0000 7fcc07227640 1 -- 172.21.15.96:0/422958366 >> 172.21.15.96:0/422958366 conn(0x7fcc000fed10 msgr2=0x7fcc00105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:16.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.576+0000 7fcc07227640 1 -- 172.21.15.96:0/422958366 shutdown_connections 2024-04-25T14:11:16.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:16.576+0000 7fcc07227640 1 -- 172.21.15.96:0/422958366 wait complete. 2024-04-25T14:11:17.236 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:17 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/422958366' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:17.422 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:17 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/422958366' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:17.437 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:17 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/422958366' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:17.499 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 14:11:17 smithi096 podman[70431]: 2024-04-25 14:11:17.359639704 +0000 UTC m=+0.017660501 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:11:17.499 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 14:11:17 smithi096 podman[70431]: 2024-04-25 14:11:17.500864329 +0000 UTC m=+0.158885107 container create 64e700cd8c53a8d3a47eac8b74b8023b98714c98fe68eb031bc95d022a043d45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-g, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS) 2024-04-25T14:11:18.080 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 14:11:17 smithi096 podman[70431]: 2024-04-25 14:11:17.753845284 +0000 UTC m=+0.411866067 container init 64e700cd8c53a8d3a47eac8b74b8023b98714c98fe68eb031bc95d022a043d45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-g, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-25T14:11:18.081 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 14:11:17 smithi096 podman[70431]: 2024-04-25 14:11:17.756716593 +0000 UTC m=+0.414737372 container start 64e700cd8c53a8d3a47eac8b74b8023b98714c98fe68eb031bc95d022a043d45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-g, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-25T14:11:18.081 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 14:11:17 smithi096 ceph-mds[70450]: -- 172.21.15.96:0/727180351 <== mon.0 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c48aeeda40 con 0x55c48af35180 2024-04-25T14:11:18.081 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 14:11:17 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-g[70446]: starting mds.g at 2024-04-25T14:11:18.081 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 14:11:17 smithi096 bash[70431]: 64e700cd8c53a8d3a47eac8b74b8023b98714c98fe68eb031bc95d022a043d45 2024-04-25T14:11:18.081 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 14:11:17 smithi096 systemd[1]: Started Ceph mds.g for 59334182-030c-11ef-bc93-c7b262605968. 2024-04-25T14:11:18.081 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:18 smithi096 ceph-mon[29207]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:18.081 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:18 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:18.081 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:18 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:18.081 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:18 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:18.081 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:18 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:11:18.082 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:18 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:11:18.082 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:18 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:11:18.422 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:18 smithi097 ceph-mon[32720]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:18.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:18 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:18.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:18 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:18.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:18 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:18.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:18 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:11:18.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:18 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:11:18.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:18 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:11:18.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:18 smithi151 ceph-mon[32990]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:18.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:18 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:18.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:18 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:18.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:18 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:18.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:18 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:11:18.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:18 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:11:18.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:18 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:11:18.587 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:11:18.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.700+0000 7f0d27d6b640 1 -- 172.21.15.96:0/98345524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d20107910 msgr2=0x7f0d20107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:18.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.700+0000 7f0d27d6b640 1 --2- 172.21.15.96:0/98345524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d20107910 0x7f0d20107d20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0d18009ca0 tx=0x7f0d1802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:11:18.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.700+0000 7f0d27d6b640 1 -- 172.21.15.96:0/98345524 shutdown_connections 2024-04-25T14:11:18.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.700+0000 7f0d27d6b640 1 --2- 172.21.15.96:0/98345524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d20107910 0x7f0d20107d20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:18.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.700+0000 7f0d27d6b640 1 --2- 172.21.15.96:0/98345524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d201027a0 0x7f0d20102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:18.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.700+0000 7f0d27d6b640 1 --2- 172.21.15.96:0/98345524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d20101d60 0x7f0d201021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:18.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.700+0000 7f0d27d6b640 1 -- 172.21.15.96:0/98345524 >> 172.21.15.96:0/98345524 conn(0x7f0d200fd330 msgr2=0x7f0d200ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:18.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.701+0000 7f0d27d6b640 1 -- 172.21.15.96:0/98345524 shutdown_connections 2024-04-25T14:11:18.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.701+0000 7f0d27d6b640 1 -- 172.21.15.96:0/98345524 wait complete. 2024-04-25T14:11:18.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.701+0000 7f0d27d6b640 1 Processor -- start 2024-04-25T14:11:18.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.701+0000 7f0d27d6b640 1 -- start start 2024-04-25T14:11:18.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.701+0000 7f0d27d6b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d20101d60 0x7f0d201a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:18.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.701+0000 7f0d27d6b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d201027a0 0x7f0d201a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:18.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.701+0000 7f0d27d6b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d20107910 0x7f0d201a7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:18.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.701+0000 7f0d27d6b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0d20115be0 con 0x7f0d201027a0 2024-04-25T14:11:18.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.701+0000 7f0d27d6b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0d20115a60 con 0x7f0d20101d60 2024-04-25T14:11:18.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.701+0000 7f0d27d6b640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0d20115d60 con 0x7f0d20107910 2024-04-25T14:11:18.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.701+0000 7f0d25306640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d201027a0 0x7f0d201a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:18.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.701+0000 7f0d25306640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d201027a0 0x7f0d201a0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58902/0 (socket says 172.21.15.96:58902) 2024-04-25T14:11:18.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.702+0000 7f0d25306640 1 -- 172.21.15.96:0/1730141282 learned_addr learned my addr 172.21.15.96:0/1730141282 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:11:18.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.702+0000 7f0d26308640 1 --2- 172.21.15.96:0/1730141282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d20107910 0x7f0d201a7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:18.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.702+0000 7f0d25b07640 1 --2- 172.21.15.96:0/1730141282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d20101d60 0x7f0d201a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:18.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.702+0000 7f0d25306640 1 -- 172.21.15.96:0/1730141282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d20107910 msgr2=0x7f0d201a7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:18.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.702+0000 7f0d25306640 1 --2- 172.21.15.96:0/1730141282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d20107910 0x7f0d201a7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:18.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.702+0000 7f0d25306640 1 -- 172.21.15.96:0/1730141282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d20101d60 msgr2=0x7f0d201a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:18.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.702+0000 7f0d25306640 1 --2- 172.21.15.96:0/1730141282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d20101d60 0x7f0d201a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:18.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.702+0000 7f0d25306640 1 -- 172.21.15.96:0/1730141282 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d201a82d0 con 0x7f0d201027a0 2024-04-25T14:11:18.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.702+0000 7f0d25b07640 1 --2- 172.21.15.96:0/1730141282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d20101d60 0x7f0d201a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:18.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.702+0000 7f0d26308640 1 --2- 172.21.15.96:0/1730141282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d20107910 0x7f0d201a7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:18.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.702+0000 7f0d25306640 1 --2- 172.21.15.96:0/1730141282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d201027a0 0x7f0d201a0cd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0d1000d770 tx=0x7f0d1000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:18.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.702+0000 7f0d16ffd640 1 -- 172.21.15.96:0/1730141282 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d10017070 con 0x7f0d201027a0 2024-04-25T14:11:18.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.702+0000 7f0d16ffd640 1 -- 172.21.15.96:0/1730141282 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d100054d0 con 0x7f0d201027a0 2024-04-25T14:11:18.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.702+0000 7f0d27d6b640 1 -- 172.21.15.96:0/1730141282 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d201a85c0 con 0x7f0d201027a0 2024-04-25T14:11:18.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.702+0000 7f0d16ffd640 1 -- 172.21.15.96:0/1730141282 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d10004c30 con 0x7f0d201027a0 2024-04-25T14:11:18.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.702+0000 7f0d27d6b640 1 -- 172.21.15.96:0/1730141282 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d201a8ac0 con 0x7f0d201027a0 2024-04-25T14:11:18.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.703+0000 7f0d27d6b640 1 -- 172.21.15.96:0/1730141282 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d20106720 con 0x7f0d201027a0 2024-04-25T14:11:18.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.704+0000 7f0d16ffd640 1 -- 172.21.15.96:0/1730141282 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f0d10004dd0 con 0x7f0d201027a0 2024-04-25T14:11:18.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.705+0000 7f0d16ffd640 1 --2- 172.21.15.96:0/1730141282 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f0d0c0afaf0 0x7f0d0c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:18.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.705+0000 7f0d25b07640 1 --2- 172.21.15.96:0/1730141282 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f0d0c0afaf0 0x7f0d0c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:18.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.706+0000 7f0d16ffd640 1 -- 172.21.15.96:0/1730141282 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f0d100de670 con 0x7f0d201027a0 2024-04-25T14:11:18.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.706+0000 7f0d25b07640 1 --2- 172.21.15.96:0/1730141282 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f0d0c0afaf0 0x7f0d0c0b2000 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c005d00 tx=0x7f0d1c005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:18.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.709+0000 7f0d16ffd640 1 -- 172.21.15.96:0/1730141282 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0d100a76d0 con 0x7f0d201027a0 2024-04-25T14:11:18.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.937+0000 7f0d27d6b640 1 -- 172.21.15.96:0/1730141282 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d2002db50 con 0x7f0d201027a0 2024-04-25T14:11:18.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.938+0000 7f0d16ffd640 1 -- 172.21.15.96:0/1730141282 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8827 (secure 0 0 0) 0x7f0d100a6fd0 con 0x7f0d201027a0 2024-04-25T14:11:18.938 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:11:18.938 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/606227917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":606227917},{"type":"v1","addr":"172.21.15.96:6835","nonce":606227917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/3265300710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3265300710},{"type":"v1","addr":"172.21.15.96:6837","nonce":3265300710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/4080308509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":4080308509},{"type":"v1","addr":"172.21.15.96:6839","nonce":4080308509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/1995324831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1995324831},{"type":"v1","addr":"172.21.15.97:6833","nonce":1995324831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1382721608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1382721608},{"type":"v1","addr":"172.21.15.151:6833","nonce":1382721608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3872301726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3872301726},{"type":"v1","addr":"172.21.15.97:6835","nonce":3872301726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/1313823638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":1313823638},{"type":"v1","addr":"172.21.15.151:6835","nonce":1313823638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/2691348366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2691348366},{"type":"v1","addr":"172.21.15.151:6837","nonce":2691348366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24497,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3306780081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3306780081},{"type":"v1","addr":"172.21.15.97:6837","nonce":3306780081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24517,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/245699981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":245699981},{"type":"v1","addr":"172.21.15.151:6839","nonce":245699981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T14:11:18.939 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 11 2024-04-25T14:11:18.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.939+0000 7f0d27d6b640 1 -- 172.21.15.96:0/1730141282 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f0d0c0afaf0 msgr2=0x7f0d0c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:18.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.939+0000 7f0d27d6b640 1 --2- 172.21.15.96:0/1730141282 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f0d0c0afaf0 0x7f0d0c0b2000 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c005d00 tx=0x7f0d1c005c70 comp rx=0 tx=0).stop 2024-04-25T14:11:18.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.940+0000 7f0d27d6b640 1 -- 172.21.15.96:0/1730141282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d201027a0 msgr2=0x7f0d201a0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:18.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.940+0000 7f0d27d6b640 1 --2- 172.21.15.96:0/1730141282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d201027a0 0x7f0d201a0cd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0d1000d770 tx=0x7f0d1000dc30 comp rx=0 tx=0).stop 2024-04-25T14:11:18.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.940+0000 7f0d27d6b640 1 -- 172.21.15.96:0/1730141282 shutdown_connections 2024-04-25T14:11:18.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.940+0000 7f0d27d6b640 1 --2- 172.21.15.96:0/1730141282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d20107910 0x7f0d201a7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:18.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.940+0000 7f0d27d6b640 1 --2- 172.21.15.96:0/1730141282 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f0d0c0afaf0 0x7f0d0c0b2000 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:18.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.940+0000 7f0d27d6b640 1 --2- 172.21.15.96:0/1730141282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d201027a0 0x7f0d201a0cd0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:18.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.940+0000 7f0d27d6b640 1 --2- 172.21.15.96:0/1730141282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d20101d60 0x7f0d201a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:18.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.940+0000 7f0d27d6b640 1 -- 172.21.15.96:0/1730141282 >> 172.21.15.96:0/1730141282 conn(0x7f0d200fd330 msgr2=0x7f0d201038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:18.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.940+0000 7f0d27d6b640 1 -- 172.21.15.96:0/1730141282 shutdown_connections 2024-04-25T14:11:18.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:18.940+0000 7f0d27d6b640 1 -- 172.21.15.96:0/1730141282 wait complete. 2024-04-25T14:11:19.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:19 smithi096 ceph-mon[29207]: Deploying daemon mds.a on smithi096 2024-04-25T14:11:19.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:19 smithi096 ceph-mon[29207]: mds.? [v2:172.21.15.96:6838/4080308509,v1:172.21.15.96:6839/4080308509] up:boot 2024-04-25T14:11:19.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:19 smithi096 ceph-mon[29207]: fsmap 10 up:standby 2024-04-25T14:11:19.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:19 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T14:11:19.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:19 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/1730141282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:19.422 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:19 smithi097 ceph-mon[32720]: Deploying daemon mds.a on smithi096 2024-04-25T14:11:19.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:19 smithi097 ceph-mon[32720]: mds.? [v2:172.21.15.96:6838/4080308509,v1:172.21.15.96:6839/4080308509] up:boot 2024-04-25T14:11:19.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:19 smithi097 ceph-mon[32720]: fsmap 10 up:standby 2024-04-25T14:11:19.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:19 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T14:11:19.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:19 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/1730141282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:19.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:19 smithi151 ceph-mon[32990]: Deploying daemon mds.a on smithi096 2024-04-25T14:11:19.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:19 smithi151 ceph-mon[32990]: mds.? [v2:172.21.15.96:6838/4080308509,v1:172.21.15.96:6839/4080308509] up:boot 2024-04-25T14:11:19.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:19 smithi151 ceph-mon[32990]: fsmap 10 up:standby 2024-04-25T14:11:19.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:19 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T14:11:19.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:19 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/1730141282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:20.246 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:20 smithi096 ceph-mon[29207]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:20.247 INFO:journalctl@ceph.mds.a.smithi096.stdout:Apr 25 14:11:20 smithi096 systemd[1]: Starting Ceph mds.a for 59334182-030c-11ef-bc93-c7b262605968... 2024-04-25T14:11:20.422 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:20 smithi097 ceph-mon[32720]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:20.437 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:20 smithi151 ceph-mon[32990]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:20.596 INFO:journalctl@ceph.mds.a.smithi096.stdout:Apr 25 14:11:20 smithi096 podman[70783]: 2024-04-25 14:11:20.322731458 +0000 UTC m=+0.022446775 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:11:20.596 INFO:journalctl@ceph.mds.a.smithi096.stdout:Apr 25 14:11:20 smithi096 podman[70783]: 2024-04-25 14:11:20.458202399 +0000 UTC m=+0.157917704 container create e1547329698111f23a474a41d29b59bdda6d25df54de770f388d06fa0170009a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T14:11:20.852 INFO:journalctl@ceph.mds.a.smithi096.stdout:Apr 25 14:11:20 smithi096 podman[70783]: 2024-04-25 14:11:20.752802525 +0000 UTC m=+0.452517830 container init e1547329698111f23a474a41d29b59bdda6d25df54de770f388d06fa0170009a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-a, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, GIT_CLEAN=True) 2024-04-25T14:11:20.852 INFO:journalctl@ceph.mds.a.smithi096.stdout:Apr 25 14:11:20 smithi096 podman[70783]: 2024-04-25 14:11:20.755717296 +0000 UTC m=+0.455432605 container start e1547329698111f23a474a41d29b59bdda6d25df54de770f388d06fa0170009a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-a, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-25T14:11:20.852 INFO:journalctl@ceph.mds.a.smithi096.stdout:Apr 25 14:11:20 smithi096 ceph-mds[70813]: -- 172.21.15.96:0/2902486889 <== mon.0 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5583769e1a40 con 0x558376a29180 2024-04-25T14:11:20.852 INFO:journalctl@ceph.mds.a.smithi096.stdout:Apr 25 14:11:20 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-a[70807]: starting mds.a at 2024-04-25T14:11:20.950 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:11:21.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.067+0000 7f009cb14640 1 -- 172.21.15.96:0/3087242489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0098104450 msgr2=0x7f009810aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:21.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.067+0000 7f009cb14640 1 --2- 172.21.15.96:0/3087242489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0098104450 0x7f009810aba0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0090009ca0 tx=0x7f009002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:11:21.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.068+0000 7f009cb14640 1 -- 172.21.15.96:0/3087242489 shutdown_connections 2024-04-25T14:11:21.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.068+0000 7f009cb14640 1 --2- 172.21.15.96:0/3087242489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0098104450 0x7f009810aba0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:21.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.068+0000 7f009cb14640 1 --2- 172.21.15.96:0/3087242489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0098103930 0x7f0098103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:21.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.068+0000 7f009cb14640 1 --2- 172.21.15.96:0/3087242489 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0098102f50 0x7f0098103360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:21.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.068+0000 7f009cb14640 1 -- 172.21.15.96:0/3087242489 >> 172.21.15.96:0/3087242489 conn(0x7f00980fd330 msgr2=0x7f00980ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:21.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.068+0000 7f009cb14640 1 -- 172.21.15.96:0/3087242489 shutdown_connections 2024-04-25T14:11:21.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.068+0000 7f009cb14640 1 -- 172.21.15.96:0/3087242489 wait complete. 2024-04-25T14:11:21.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.068+0000 7f009cb14640 1 Processor -- start 2024-04-25T14:11:21.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.068+0000 7f009cb14640 1 -- start start 2024-04-25T14:11:21.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.069+0000 7f009cb14640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0098102f50 0x7f00981990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:21.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.069+0000 7f009cb14640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0098103930 0x7f0098199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:21.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.069+0000 7f009cb14640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0098104450 0x7f00981a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:21.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.069+0000 7f009cb14640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f009810e570 con 0x7f0098104450 2024-04-25T14:11:21.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.069+0000 7f009cb14640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f009810e3f0 con 0x7f0098103930 2024-04-25T14:11:21.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.069+0000 7f009cb14640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f009810e6f0 con 0x7f0098102f50 2024-04-25T14:11:21.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.069+0000 7f0096d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0098104450 0x7f00981a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:21.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.069+0000 7f0096d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0098104450 0x7f00981a05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58922/0 (socket says 172.21.15.96:58922) 2024-04-25T14:11:21.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.069+0000 7f0096d9d640 1 -- 172.21.15.96:0/608559899 learned_addr learned my addr 172.21.15.96:0/608559899 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:11:21.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.069+0000 7f0095d9b640 1 --2- 172.21.15.96:0/608559899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0098103930 0x7f0098199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:21.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.069+0000 7f009659c640 1 --2- 172.21.15.96:0/608559899 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0098102f50 0x7f00981990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:21.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.069+0000 7f0096d9d640 1 -- 172.21.15.96:0/608559899 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0098102f50 msgr2=0x7f00981990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:21.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.069+0000 7f0096d9d640 1 --2- 172.21.15.96:0/608559899 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0098102f50 0x7f00981990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:21.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.069+0000 7f0096d9d640 1 -- 172.21.15.96:0/608559899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0098103930 msgr2=0x7f0098199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:21.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.069+0000 7f0096d9d640 1 --2- 172.21.15.96:0/608559899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0098103930 0x7f0098199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:21.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.069+0000 7f0096d9d640 1 -- 172.21.15.96:0/608559899 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00981a0c30 con 0x7f0098104450 2024-04-25T14:11:21.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.069+0000 7f0096d9d640 1 --2- 172.21.15.96:0/608559899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0098104450 0x7f00981a05c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f00900097b0 tx=0x7f0090031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:21.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.069+0000 7f00877fe640 1 -- 172.21.15.96:0/608559899 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0090003bf0 con 0x7f0098104450 2024-04-25T14:11:21.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.069+0000 7f00877fe640 1 -- 172.21.15.96:0/608559899 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0090003d90 con 0x7f0098104450 2024-04-25T14:11:21.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.069+0000 7f009cb14640 1 -- 172.21.15.96:0/608559899 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00981a0ec0 con 0x7f0098104450 2024-04-25T14:11:21.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.069+0000 7f00877fe640 1 -- 172.21.15.96:0/608559899 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009002fb70 con 0x7f0098104450 2024-04-25T14:11:21.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.069+0000 7f009cb14640 1 -- 172.21.15.96:0/608559899 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00981a13a0 con 0x7f0098104450 2024-04-25T14:11:21.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.071+0000 7f009cb14640 1 -- 172.21.15.96:0/608559899 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0098069310 con 0x7f0098104450 2024-04-25T14:11:21.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.072+0000 7f00877fe640 1 -- 172.21.15.96:0/608559899 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f009002fd30 con 0x7f0098104450 2024-04-25T14:11:21.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.073+0000 7f00877fe640 1 --2- 172.21.15.96:0/608559899 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f00800afaf0 0x7f00800b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:21.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.073+0000 7f009659c640 1 --2- 172.21.15.96:0/608559899 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f00800afaf0 0x7f00800b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:21.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.073+0000 7f00877fe640 1 -- 172.21.15.96:0/608559899 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f0090100010 con 0x7f0098104450 2024-04-25T14:11:21.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.073+0000 7f009659c640 1 --2- 172.21.15.96:0/608559899 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f00800afaf0 0x7f00800b2000 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0098069bd0 tx=0x7f008c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:21.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.077+0000 7f00877fe640 1 -- 172.21.15.96:0/608559899 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f00900c9090 con 0x7f0098104450 2024-04-25T14:11:21.108 INFO:journalctl@ceph.mds.a.smithi096.stdout:Apr 25 14:11:20 smithi096 bash[70783]: e1547329698111f23a474a41d29b59bdda6d25df54de770f388d06fa0170009a 2024-04-25T14:11:21.109 INFO:journalctl@ceph.mds.a.smithi096.stdout:Apr 25 14:11:20 smithi096 systemd[1]: Started Ceph mds.a for 59334182-030c-11ef-bc93-c7b262605968. 2024-04-25T14:11:21.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.319+0000 7f009cb14640 1 -- 172.21.15.96:0/608559899 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0098109900 con 0x7f0098104450 2024-04-25T14:11:21.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.319+0000 7f00877fe640 1 -- 172.21.15.96:0/608559899 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9653 (secure 0 0 0) 0x7f00900c8990 con 0x7f0098104450 2024-04-25T14:11:21.319 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:11:21.319 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/606227917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":606227917},{"type":"v1","addr":"172.21.15.96:6835","nonce":606227917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/3265300710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3265300710},{"type":"v1","addr":"172.21.15.96:6837","nonce":3265300710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/4080308509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":4080308509},{"type":"v1","addr":"172.21.15.96:6839","nonce":4080308509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6841/1606748971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":1606748971},{"type":"v1","addr":"172.21.15.96:6841","nonce":1606748971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/1995324831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1995324831},{"type":"v1","addr":"172.21.15.97:6833","nonce":1995324831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1382721608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1382721608},{"type":"v1","addr":"172.21.15.151:6833","nonce":1382721608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3872301726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3872301726},{"type":"v1","addr":"172.21.15.97:6835","nonce":3872301726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/1313823638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":1313823638},{"type":"v1","addr":"172.21.15.151:6835","nonce":1313823638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/2691348366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2691348366},{"type":"v1","addr":"172.21.15.151:6837","nonce":2691348366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24497,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3306780081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3306780081},{"type":"v1","addr":"172.21.15.97:6837","nonce":3306780081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24517,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/245699981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":245699981},{"type":"v1","addr":"172.21.15.151:6839","nonce":245699981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T14:11:21.320 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 12 2024-04-25T14:11:21.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.321+0000 7f009cb14640 1 -- 172.21.15.96:0/608559899 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f00800afaf0 msgr2=0x7f00800b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:21.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.321+0000 7f009cb14640 1 --2- 172.21.15.96:0/608559899 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f00800afaf0 0x7f00800b2000 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0098069bd0 tx=0x7f008c005f90 comp rx=0 tx=0).stop 2024-04-25T14:11:21.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.321+0000 7f009cb14640 1 -- 172.21.15.96:0/608559899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0098104450 msgr2=0x7f00981a05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:21.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.321+0000 7f009cb14640 1 --2- 172.21.15.96:0/608559899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0098104450 0x7f00981a05c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f00900097b0 tx=0x7f0090031840 comp rx=0 tx=0).stop 2024-04-25T14:11:21.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.321+0000 7f009cb14640 1 -- 172.21.15.96:0/608559899 shutdown_connections 2024-04-25T14:11:21.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.321+0000 7f009cb14640 1 --2- 172.21.15.96:0/608559899 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f00800afaf0 0x7f00800b2000 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:21.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.321+0000 7f009cb14640 1 --2- 172.21.15.96:0/608559899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0098104450 0x7f00981a05c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:21.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.321+0000 7f009cb14640 1 --2- 172.21.15.96:0/608559899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0098103930 0x7f0098199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:21.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.321+0000 7f009cb14640 1 --2- 172.21.15.96:0/608559899 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0098102f50 0x7f00981990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:21.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.322+0000 7f009cb14640 1 -- 172.21.15.96:0/608559899 >> 172.21.15.96:0/608559899 conn(0x7f00980fd330 msgr2=0x7f0098106ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:21.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.322+0000 7f009cb14640 1 -- 172.21.15.96:0/608559899 shutdown_connections 2024-04-25T14:11:21.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:21.322+0000 7f009cb14640 1 -- 172.21.15.96:0/608559899 wait complete. 2024-04-25T14:11:22.172 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:21 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:22.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:21 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:22.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:21 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:22.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:21 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:11:22.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:21 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:11:22.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:21 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:11:22.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:21 smithi097 ceph-mon[32720]: Deploying daemon mds.b on smithi097 2024-04-25T14:11:22.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:21 smithi097 ceph-mon[32720]: mds.? [v2:172.21.15.96:6840/1606748971,v1:172.21.15.96:6841/1606748971] up:boot 2024-04-25T14:11:22.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:21 smithi097 ceph-mon[32720]: fsmap 11 up:standby 2024-04-25T14:11:22.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:21 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T14:11:22.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:21 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/608559899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:22.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:21 smithi097 ceph-mon[32720]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:22.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:21 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:22.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:21 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:22.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:21 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:22.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:21 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:11:22.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:21 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:11:22.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:21 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:11:22.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:21 smithi151 ceph-mon[32990]: Deploying daemon mds.b on smithi097 2024-04-25T14:11:22.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:21 smithi151 ceph-mon[32990]: mds.? [v2:172.21.15.96:6840/1606748971,v1:172.21.15.96:6841/1606748971] up:boot 2024-04-25T14:11:22.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:21 smithi151 ceph-mon[32990]: fsmap 11 up:standby 2024-04-25T14:11:22.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:21 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T14:11:22.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:21 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/608559899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:22.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:21 smithi151 ceph-mon[32990]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:22.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:21 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:22.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:21 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:22.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:21 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:22.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:21 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:11:22.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:21 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:11:22.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:21 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:11:22.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:21 smithi096 ceph-mon[29207]: Deploying daemon mds.b on smithi097 2024-04-25T14:11:22.360 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:21 smithi096 ceph-mon[29207]: mds.? [v2:172.21.15.96:6840/1606748971,v1:172.21.15.96:6841/1606748971] up:boot 2024-04-25T14:11:22.360 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:21 smithi096 ceph-mon[29207]: fsmap 11 up:standby 2024-04-25T14:11:22.360 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:21 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T14:11:22.360 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:21 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/608559899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:22.360 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:21 smithi096 ceph-mon[29207]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:23.334 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:11:23.423 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 14:11:23 smithi097 systemd[1]: Starting Ceph mds.b for 59334182-030c-11ef-bc93-c7b262605968... 2024-04-25T14:11:23.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.456+0000 7f7b4dd64640 1 -- 172.21.15.96:0/1283754932 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b48108900 msgr2=0x7f7b4810ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:23.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.456+0000 7f7b4dd64640 1 --2- 172.21.15.96:0/1283754932 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b48108900 0x7f7b4810ad20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7b40009ca0 tx=0x7f7b4002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:11:23.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.456+0000 7f7b4dd64640 1 -- 172.21.15.96:0/1283754932 shutdown_connections 2024-04-25T14:11:23.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.456+0000 7f7b4dd64640 1 --2- 172.21.15.96:0/1283754932 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b48108900 0x7f7b4810ad20 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:23.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.456+0000 7f7b4dd64640 1 --2- 172.21.15.96:0/1283754932 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b48103c00 0x7f7b48104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:23.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.456+0000 7f7b4dd64640 1 --2- 172.21.15.96:0/1283754932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b481029c0 0x7f7b481036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:23.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.456+0000 7f7b4dd64640 1 -- 172.21.15.96:0/1283754932 >> 172.21.15.96:0/1283754932 conn(0x7f7b480fdbd0 msgr2=0x7f7b48100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:23.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.456+0000 7f7b4dd64640 1 -- 172.21.15.96:0/1283754932 shutdown_connections 2024-04-25T14:11:23.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.456+0000 7f7b4dd64640 1 -- 172.21.15.96:0/1283754932 wait complete. 2024-04-25T14:11:23.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.457+0000 7f7b4dd64640 1 Processor -- start 2024-04-25T14:11:23.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.457+0000 7f7b4dd64640 1 -- start start 2024-04-25T14:11:23.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.457+0000 7f7b4dd64640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b481029c0 0x7f7b4806a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:23.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.457+0000 7f7b4dd64640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b48103c00 0x7f7b4819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:23.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.457+0000 7f7b4dd64640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b48108900 0x7f7b481a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:23.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.457+0000 7f7b4dd64640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4810e540 con 0x7f7b48103c00 2024-04-25T14:11:23.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.457+0000 7f7b4dd64640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4810e3c0 con 0x7f7b481029c0 2024-04-25T14:11:23.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.457+0000 7f7b4dd64640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4810e6c0 con 0x7f7b48108900 2024-04-25T14:11:23.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.458+0000 7f7b46ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b48103c00 0x7f7b4819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:23.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.458+0000 7f7b46ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b48103c00 0x7f7b4819efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56330/0 (socket says 172.21.15.96:56330) 2024-04-25T14:11:23.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.458+0000 7f7b46ffd640 1 -- 172.21.15.96:0/1506812889 learned_addr learned my addr 172.21.15.96:0/1506812889 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:11:23.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.458+0000 7f7b477fe640 1 --2- 172.21.15.96:0/1506812889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b481029c0 0x7f7b4806a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:23.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.458+0000 7f7b47fff640 1 --2- 172.21.15.96:0/1506812889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b48108900 0x7f7b481a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:23.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.458+0000 7f7b46ffd640 1 -- 172.21.15.96:0/1506812889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b48108900 msgr2=0x7f7b481a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:23.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.458+0000 7f7b46ffd640 1 --2- 172.21.15.96:0/1506812889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b48108900 0x7f7b481a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:23.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.458+0000 7f7b46ffd640 1 -- 172.21.15.96:0/1506812889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b481029c0 msgr2=0x7f7b4806a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:23.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.458+0000 7f7b46ffd640 1 --2- 172.21.15.96:0/1506812889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b481029c0 0x7f7b4806a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:23.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.458+0000 7f7b46ffd640 1 -- 172.21.15.96:0/1506812889 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b481a1d30 con 0x7f7b48103c00 2024-04-25T14:11:23.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.458+0000 7f7b477fe640 1 --2- 172.21.15.96:0/1506812889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b481029c0 0x7f7b4806a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:11:23.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.458+0000 7f7b46ffd640 1 --2- 172.21.15.96:0/1506812889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b48103c00 0x7f7b4819efa0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c00b4a0 tx=0x7f7b3c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:23.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.458+0000 7f7b44ff9640 1 -- 172.21.15.96:0/1506812889 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c014070 con 0x7f7b48103c00 2024-04-25T14:11:23.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.458+0000 7f7b44ff9640 1 -- 172.21.15.96:0/1506812889 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b3c010ac0 con 0x7f7b48103c00 2024-04-25T14:11:23.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.458+0000 7f7b4dd64640 1 -- 172.21.15.96:0/1506812889 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b481a1fc0 con 0x7f7b48103c00 2024-04-25T14:11:23.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.458+0000 7f7b44ff9640 1 -- 172.21.15.96:0/1506812889 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c017c30 con 0x7f7b48103c00 2024-04-25T14:11:23.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.458+0000 7f7b4dd64640 1 -- 172.21.15.96:0/1506812889 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b481a24e0 con 0x7f7b48103c00 2024-04-25T14:11:23.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.459+0000 7f7b4dd64640 1 -- 172.21.15.96:0/1506812889 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b48107710 con 0x7f7b48103c00 2024-04-25T14:11:23.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.461+0000 7f7b44ff9640 1 -- 172.21.15.96:0/1506812889 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f7b3c017070 con 0x7f7b48103c00 2024-04-25T14:11:23.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.461+0000 7f7b44ff9640 1 --2- 172.21.15.96:0/1506812889 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f7b300afaf0 0x7f7b300b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:23.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.462+0000 7f7b477fe640 1 --2- 172.21.15.96:0/1506812889 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f7b300afaf0 0x7f7b300b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:23.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.462+0000 7f7b44ff9640 1 -- 172.21.15.96:0/1506812889 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f7b3c0ddb40 con 0x7f7b48103c00 2024-04-25T14:11:23.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.462+0000 7f7b477fe640 1 --2- 172.21.15.96:0/1506812889 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f7b300afaf0 0x7f7b300b2000 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7b481035c0 tx=0x7f7b38005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:23.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.467+0000 7f7b44ff9640 1 -- 172.21.15.96:0/1506812889 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7b3c0a6bc0 con 0x7f7b48103c00 2024-04-25T14:11:23.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.685+0000 7f7b4dd64640 1 -- 172.21.15.96:0/1506812889 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b4802db50 con 0x7f7b48103c00 2024-04-25T14:11:23.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.686+0000 7f7b44ff9640 1 -- 172.21.15.96:0/1506812889 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9653 (secure 0 0 0) 0x7f7b3c0a64c0 con 0x7f7b48103c00 2024-04-25T14:11:23.686 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:11:23.686 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/606227917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":606227917},{"type":"v1","addr":"172.21.15.96:6835","nonce":606227917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/3265300710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3265300710},{"type":"v1","addr":"172.21.15.96:6837","nonce":3265300710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/4080308509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":4080308509},{"type":"v1","addr":"172.21.15.96:6839","nonce":4080308509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6841/1606748971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":1606748971},{"type":"v1","addr":"172.21.15.96:6841","nonce":1606748971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/1995324831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1995324831},{"type":"v1","addr":"172.21.15.97:6833","nonce":1995324831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1382721608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1382721608},{"type":"v1","addr":"172.21.15.151:6833","nonce":1382721608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3872301726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3872301726},{"type":"v1","addr":"172.21.15.97:6835","nonce":3872301726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/1313823638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":1313823638},{"type":"v1","addr":"172.21.15.151:6835","nonce":1313823638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/2691348366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2691348366},{"type":"v1","addr":"172.21.15.151:6837","nonce":2691348366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24497,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3306780081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3306780081},{"type":"v1","addr":"172.21.15.97:6837","nonce":3306780081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24517,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/245699981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":245699981},{"type":"v1","addr":"172.21.15.151:6839","nonce":245699981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T14:11:23.686 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 12 2024-04-25T14:11:23.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.688+0000 7f7b4dd64640 1 -- 172.21.15.96:0/1506812889 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f7b300afaf0 msgr2=0x7f7b300b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:23.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.688+0000 7f7b4dd64640 1 --2- 172.21.15.96:0/1506812889 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f7b300afaf0 0x7f7b300b2000 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7b481035c0 tx=0x7f7b38005f90 comp rx=0 tx=0).stop 2024-04-25T14:11:23.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.688+0000 7f7b4dd64640 1 -- 172.21.15.96:0/1506812889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b48103c00 msgr2=0x7f7b4819efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:23.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.688+0000 7f7b4dd64640 1 --2- 172.21.15.96:0/1506812889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b48103c00 0x7f7b4819efa0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c00b4a0 tx=0x7f7b3c00b960 comp rx=0 tx=0).stop 2024-04-25T14:11:23.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.688+0000 7f7b4dd64640 1 -- 172.21.15.96:0/1506812889 shutdown_connections 2024-04-25T14:11:23.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.688+0000 7f7b4dd64640 1 --2- 172.21.15.96:0/1506812889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b48108900 0x7f7b481a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:23.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.688+0000 7f7b4dd64640 1 --2- 172.21.15.96:0/1506812889 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f7b300afaf0 0x7f7b300b2000 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:23.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.688+0000 7f7b4dd64640 1 --2- 172.21.15.96:0/1506812889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b48103c00 0x7f7b4819efa0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:23.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.688+0000 7f7b4dd64640 1 --2- 172.21.15.96:0/1506812889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b481029c0 0x7f7b4806a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:23.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.688+0000 7f7b4dd64640 1 -- 172.21.15.96:0/1506812889 >> 172.21.15.96:0/1506812889 conn(0x7f7b480fdbd0 msgr2=0x7f7b481048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:23.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.688+0000 7f7b4dd64640 1 -- 172.21.15.96:0/1506812889 shutdown_connections 2024-04-25T14:11:23.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:23.688+0000 7f7b4dd64640 1 -- 172.21.15.96:0/1506812889 wait complete. 2024-04-25T14:11:23.766 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 14:11:23 smithi097 podman[63210]: 2024-04-25 14:11:23.38838698 +0000 UTC m=+0.017775922 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:11:23.766 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 14:11:23 smithi097 podman[63210]: 2024-04-25 14:11:23.514911358 +0000 UTC m=+0.144300289 container create 7bfcbedc3a91b98f1c10f9f05c54b7ef27a35b358f7e486349d1d95a14157689 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-b, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-25T14:11:23.766 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 14:11:23 smithi097 podman[63210]: 2024-04-25 14:11:23.76791957 +0000 UTC m=+0.397308502 container init 7bfcbedc3a91b98f1c10f9f05c54b7ef27a35b358f7e486349d1d95a14157689 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424) 2024-04-25T14:11:24.172 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 14:11:23 smithi097 podman[63210]: 2024-04-25 14:11:23.770732569 +0000 UTC m=+0.400121509 container start 7bfcbedc3a91b98f1c10f9f05c54b7ef27a35b358f7e486349d1d95a14157689 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T14:11:24.172 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 14:11:23 smithi097 ceph-mds[63229]: -- 172.21.15.97:0/4062315814 <== mon.2 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56170d7c1a40 con 0x56170d809180 2024-04-25T14:11:24.173 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 14:11:23 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-b[63225]: starting mds.b at 2024-04-25T14:11:24.173 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 14:11:23 smithi097 bash[63210]: 7bfcbedc3a91b98f1c10f9f05c54b7ef27a35b358f7e486349d1d95a14157689 2024-04-25T14:11:24.173 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 14:11:23 smithi097 systemd[1]: Started Ceph mds.b for 59334182-030c-11ef-bc93-c7b262605968. 2024-04-25T14:11:24.829 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:24 smithi097 ceph-mon[32720]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:24.829 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:24 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/1506812889' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:24.830 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:24 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:24.830 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:24 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:24.830 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:24 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:24.830 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:24 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:24.830 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:24 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:24.830 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:24 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:11:24.852 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:24 smithi151 ceph-mon[32990]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:24.852 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:24 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/1506812889' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:24.852 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:24 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:24.853 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:24 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:24.853 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:24 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:24.853 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:24 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:24.853 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:24 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:24.853 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:24 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:11:24.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:24 smithi096 ceph-mon[29207]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:24.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:24 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/1506812889' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:24.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:24 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:24.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:24 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:24.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:24 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:24.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:24 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:24.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:24 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:24.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:24 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:11:25.700 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:11:25.722 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:25 smithi096 ceph-mon[29207]: mds.? [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] up:boot 2024-04-25T14:11:25.722 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:25 smithi096 ceph-mon[29207]: fsmap 12 up:standby 2024-04-25T14:11:25.722 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:25 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T14:11:25.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:25.818+0000 7f2bf5d72640 1 -- 172.21.15.96:0/2377207165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bf0103c00 msgr2=0x7f2bf0104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:25.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:25.818+0000 7f2bf5d72640 1 --2- 172.21.15.96:0/2377207165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bf0103c00 0x7f2bf0104010 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8009ca0 tx=0x7f2bd802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:11:25.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:25.818+0000 7f2bf5d72640 1 -- 172.21.15.96:0/2377207165 shutdown_connections 2024-04-25T14:11:25.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:25.818+0000 7f2bf5d72640 1 --2- 172.21.15.96:0/2377207165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bf0108900 0x7f2bf010ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:25.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:25.818+0000 7f2bf5d72640 1 --2- 172.21.15.96:0/2377207165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bf0103c00 0x7f2bf0104010 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:25.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:25.818+0000 7f2bf5d72640 1 --2- 172.21.15.96:0/2377207165 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2bf01029c0 0x7f2bf01036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:25.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:25.818+0000 7f2bf5d72640 1 -- 172.21.15.96:0/2377207165 >> 172.21.15.96:0/2377207165 conn(0x7f2bf00fdbd0 msgr2=0x7f2bf0100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:25.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:25.818+0000 7f2bf5d72640 1 -- 172.21.15.96:0/2377207165 shutdown_connections 2024-04-25T14:11:25.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:25.818+0000 7f2bf5d72640 1 -- 172.21.15.96:0/2377207165 wait complete. 2024-04-25T14:11:25.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:25.819+0000 7f2bf5d72640 1 Processor -- start 2024-04-25T14:11:25.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:25.819+0000 7f2bf5d72640 1 -- start start 2024-04-25T14:11:25.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:25.819+0000 7f2bf5d72640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bf01029c0 0x7f2bf006a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:25.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:25.819+0000 7f2bf5d72640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bf0103c00 0x7f2bf019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:25.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:25.819+0000 7f2bf5d72640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2bf0108900 0x7f2bf01a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:25.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:25.819+0000 7f2bf5d72640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf010e4e0 con 0x7f2bf01029c0 2024-04-25T14:11:25.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:25.819+0000 7f2bf5d72640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf010e360 con 0x7f2bf0103c00 2024-04-25T14:11:25.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:25.819+0000 7f2bf5d72640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf010e660 con 0x7f2bf0108900 2024-04-25T14:11:25.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:25.819+0000 7f2bef7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bf01029c0 0x7f2bf006a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:25.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:25.819+0000 7f2bef7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bf01029c0 0x7f2bf006a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56334/0 (socket says 172.21.15.96:56334) 2024-04-25T14:11:25.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:25.819+0000 7f2bef7fe640 1 -- 172.21.15.96:0/693542465 learned_addr learned my addr 172.21.15.96:0/693542465 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:11:25.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:25.819+0000 7f2beeffd640 1 --2- 172.21.15.96:0/693542465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bf0103c00 0x7f2bf019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:25.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:25.819+0000 7f2beffff640 1 --2- 172.21.15.96:0/693542465 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2bf0108900 0x7f2bf01a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:25.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:25.820+0000 7f2bef7fe640 1 -- 172.21.15.96:0/693542465 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2bf0108900 msgr2=0x7f2bf01a1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:25.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:25.820+0000 7f2bef7fe640 1 --2- 172.21.15.96:0/693542465 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2bf0108900 0x7f2bf01a1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:25.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:25.820+0000 7f2bef7fe640 1 -- 172.21.15.96:0/693542465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bf0103c00 msgr2=0x7f2bf019efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:25.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:25.820+0000 7f2bef7fe640 1 --2- 172.21.15.96:0/693542465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bf0103c00 0x7f2bf019efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:25.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:25.820+0000 7f2bef7fe640 1 -- 172.21.15.96:0/693542465 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bf01a1d40 con 0x7f2bf01029c0 2024-04-25T14:11:25.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:25.820+0000 7f2beeffd640 1 --2- 172.21.15.96:0/693542465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bf0103c00 0x7f2bf019efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:11:25.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:25.820+0000 7f2bef7fe640 1 --2- 172.21.15.96:0/693542465 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bf01029c0 0x7f2bf006a920 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2be000e9f0 tx=0x7f2be000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:25.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:25.820+0000 7f2becff9640 1 -- 172.21.15.96:0/693542465 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be000ccc0 con 0x7f2bf01029c0 2024-04-25T14:11:25.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:25.820+0000 7f2bf5d72640 1 -- 172.21.15.96:0/693542465 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bf01a1fd0 con 0x7f2bf01029c0 2024-04-25T14:11:25.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:25.820+0000 7f2becff9640 1 -- 172.21.15.96:0/693542465 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2be00035f0 con 0x7f2bf01029c0 2024-04-25T14:11:25.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:25.820+0000 7f2becff9640 1 -- 172.21.15.96:0/693542465 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be000ce60 con 0x7f2bf01029c0 2024-04-25T14:11:25.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:25.820+0000 7f2bf5d72640 1 -- 172.21.15.96:0/693542465 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bf01a2440 con 0x7f2bf01029c0 2024-04-25T14:11:25.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:25.821+0000 7f2bf5d72640 1 -- 172.21.15.96:0/693542465 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bf0107710 con 0x7f2bf01029c0 2024-04-25T14:11:25.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:25.823+0000 7f2becff9640 1 -- 172.21.15.96:0/693542465 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f2be0003aa0 con 0x7f2bf01029c0 2024-04-25T14:11:25.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:25.824+0000 7f2becff9640 1 --2- 172.21.15.96:0/693542465 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f2bdc0afaf0 0x7f2bdc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:25.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:25.824+0000 7f2beeffd640 1 --2- 172.21.15.96:0/693542465 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f2bdc0afaf0 0x7f2bdc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:25.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:25.824+0000 7f2becff9640 1 -- 172.21.15.96:0/693542465 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f2be00deb30 con 0x7f2bf01029c0 2024-04-25T14:11:25.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:25.825+0000 7f2beeffd640 1 --2- 172.21.15.96:0/693542465 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f2bdc0afaf0 0x7f2bdc0b2000 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2bd80094e0 tx=0x7f2bd8004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:25.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:25.828+0000 7f2becff9640 1 -- 172.21.15.96:0/693542465 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2be00a7bb0 con 0x7f2bf01029c0 2024-04-25T14:11:25.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:25 smithi097 ceph-mon[32720]: mds.? [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] up:boot 2024-04-25T14:11:25.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:25 smithi097 ceph-mon[32720]: fsmap 12 up:standby 2024-04-25T14:11:25.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:25 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T14:11:25.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:25 smithi151 ceph-mon[32990]: mds.? [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] up:boot 2024-04-25T14:11:25.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:25 smithi151 ceph-mon[32990]: fsmap 12 up:standby 2024-04-25T14:11:25.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:25 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T14:11:26.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.052+0000 7f2bf5d72640 1 -- 172.21.15.96:0/693542465 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2bf002db50 con 0x7f2bf01029c0 2024-04-25T14:11:26.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.053+0000 7f2becff9640 1 -- 172.21.15.96:0/693542465 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10479 (secure 0 0 0) 0x7f2be00a74b0 con 0x7f2bf01029c0 2024-04-25T14:11:26.053 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:11:26.053 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/606227917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":606227917},{"type":"v1","addr":"172.21.15.96:6835","nonce":606227917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/3265300710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3265300710},{"type":"v1","addr":"172.21.15.96:6837","nonce":3265300710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/4080308509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":4080308509},{"type":"v1","addr":"172.21.15.96:6839","nonce":4080308509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6841/1606748971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":1606748971},{"type":"v1","addr":"172.21.15.96:6841","nonce":1606748971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/1995324831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1995324831},{"type":"v1","addr":"172.21.15.97:6833","nonce":1995324831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1382721608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1382721608},{"type":"v1","addr":"172.21.15.151:6833","nonce":1382721608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3872301726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3872301726},{"type":"v1","addr":"172.21.15.97:6835","nonce":3872301726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/1313823638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":1313823638},{"type":"v1","addr":"172.21.15.151:6835","nonce":1313823638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/2691348366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2691348366},{"type":"v1","addr":"172.21.15.151:6837","nonce":2691348366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24497,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3306780081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3306780081},{"type":"v1","addr":"172.21.15.97:6837","nonce":3306780081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24517,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/245699981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":245699981},{"type":"v1","addr":"172.21.15.151:6839","nonce":245699981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24536,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/2850442269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":2850442269},{"type":"v1","addr":"172.21.15.97:6839","nonce":2850442269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-25T14:11:26.053 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 13 2024-04-25T14:11:26.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.055+0000 7f2bf5d72640 1 -- 172.21.15.96:0/693542465 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f2bdc0afaf0 msgr2=0x7f2bdc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:26.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.055+0000 7f2bf5d72640 1 --2- 172.21.15.96:0/693542465 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f2bdc0afaf0 0x7f2bdc0b2000 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2bd80094e0 tx=0x7f2bd8004a40 comp rx=0 tx=0).stop 2024-04-25T14:11:26.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.055+0000 7f2bf5d72640 1 -- 172.21.15.96:0/693542465 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bf01029c0 msgr2=0x7f2bf006a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:26.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.055+0000 7f2bf5d72640 1 --2- 172.21.15.96:0/693542465 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bf01029c0 0x7f2bf006a920 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2be000e9f0 tx=0x7f2be000eeb0 comp rx=0 tx=0).stop 2024-04-25T14:11:26.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.055+0000 7f2bf5d72640 1 -- 172.21.15.96:0/693542465 shutdown_connections 2024-04-25T14:11:26.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.055+0000 7f2bf5d72640 1 --2- 172.21.15.96:0/693542465 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2bf0108900 0x7f2bf01a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:26.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.055+0000 7f2bf5d72640 1 --2- 172.21.15.96:0/693542465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bf0103c00 0x7f2bf019efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:26.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.055+0000 7f2bf5d72640 1 --2- 172.21.15.96:0/693542465 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f2bdc0afaf0 0x7f2bdc0b2000 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:26.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.055+0000 7f2bf5d72640 1 --2- 172.21.15.96:0/693542465 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bf01029c0 0x7f2bf006a920 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:26.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.055+0000 7f2bf5d72640 1 -- 172.21.15.96:0/693542465 >> 172.21.15.96:0/693542465 conn(0x7f2bf00fdbd0 msgr2=0x7f2bf01048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:26.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.056+0000 7f2bf5d72640 1 -- 172.21.15.96:0/693542465 shutdown_connections 2024-04-25T14:11:26.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.056+0000 7f2bf5d72640 1 -- 172.21.15.96:0/693542465 wait complete. 2024-04-25T14:11:26.071 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:11:26.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.221+0000 7f7d8cdab640 1 -- 172.21.15.96:0/3003979531 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d881029e0 msgr2=0x7f7d88102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:26.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.221+0000 7f7d8cdab640 1 --2- 172.21.15.96:0/3003979531 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d881029e0 0x7f7d88102eb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c009900 tx=0x7f7d7c02a400 comp rx=0 tx=0).stop 2024-04-25T14:11:26.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.221+0000 7f7d8cdab640 1 -- 172.21.15.96:0/3003979531 shutdown_connections 2024-04-25T14:11:26.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.221+0000 7f7d8cdab640 1 --2- 172.21.15.96:0/3003979531 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d88103540 0x7f7d88103a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:26.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.221+0000 7f7d8cdab640 1 --2- 172.21.15.96:0/3003979531 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d881029e0 0x7f7d88102eb0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:26.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.221+0000 7f7d8cdab640 1 --2- 172.21.15.96:0/3003979531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d88102000 0x7f7d88102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:26.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.221+0000 7f7d8cdab640 1 -- 172.21.15.96:0/3003979531 >> 172.21.15.96:0/3003979531 conn(0x7f7d880fdbd0 msgr2=0x7f7d88100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:26.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.221+0000 7f7d8cdab640 1 -- 172.21.15.96:0/3003979531 shutdown_connections 2024-04-25T14:11:26.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.221+0000 7f7d8cdab640 1 -- 172.21.15.96:0/3003979531 wait complete. 2024-04-25T14:11:26.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.222+0000 7f7d8cdab640 1 Processor -- start 2024-04-25T14:11:26.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.222+0000 7f7d8cdab640 1 -- start start 2024-04-25T14:11:26.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.222+0000 7f7d8cdab640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d88102000 0x7f7d8819c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:26.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.222+0000 7f7d8cdab640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d881029e0 0x7f7d8819c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:26.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.222+0000 7f7d8cdab640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d88103540 0x7f7d881a3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:26.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.222+0000 7f7d8cdab640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7d881117c0 con 0x7f7d88103540 2024-04-25T14:11:26.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.222+0000 7f7d8cdab640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7d88111640 con 0x7f7d881029e0 2024-04-25T14:11:26.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.222+0000 7f7d8cdab640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7d88111940 con 0x7f7d88102000 2024-04-25T14:11:26.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.222+0000 7f7d86d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d88103540 0x7f7d881a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:26.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.222+0000 7f7d86d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d88103540 0x7f7d881a3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56352/0 (socket says 172.21.15.96:56352) 2024-04-25T14:11:26.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.222+0000 7f7d86d9d640 1 -- 172.21.15.96:0/2264435559 learned_addr learned my addr 172.21.15.96:0/2264435559 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:11:26.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.222+0000 7f7d8659c640 1 --2- 172.21.15.96:0/2264435559 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d88102000 0x7f7d8819c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:26.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.222+0000 7f7d85d9b640 1 --2- 172.21.15.96:0/2264435559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d881029e0 0x7f7d8819c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:26.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.223+0000 7f7d86d9d640 1 -- 172.21.15.96:0/2264435559 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d88102000 msgr2=0x7f7d8819c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:26.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.223+0000 7f7d86d9d640 1 --2- 172.21.15.96:0/2264435559 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d88102000 0x7f7d8819c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:26.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.223+0000 7f7d86d9d640 1 -- 172.21.15.96:0/2264435559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d881029e0 msgr2=0x7f7d8819c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:26.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.223+0000 7f7d86d9d640 1 --2- 172.21.15.96:0/2264435559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d881029e0 0x7f7d8819c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:26.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.223+0000 7f7d86d9d640 1 -- 172.21.15.96:0/2264435559 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d881a3e40 con 0x7f7d88103540 2024-04-25T14:11:26.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.223+0000 7f7d86d9d640 1 --2- 172.21.15.96:0/2264435559 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d88103540 0x7f7d881a3880 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7d8000eaa0 tx=0x7f7d8000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:26.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.223+0000 7f7d777fe640 1 -- 172.21.15.96:0/2264435559 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d80004d20 con 0x7f7d88103540 2024-04-25T14:11:26.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.223+0000 7f7d777fe640 1 -- 172.21.15.96:0/2264435559 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d80005020 con 0x7f7d88103540 2024-04-25T14:11:26.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.223+0000 7f7d8cdab640 1 -- 172.21.15.96:0/2264435559 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d881a4130 con 0x7f7d88103540 2024-04-25T14:11:26.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.223+0000 7f7d777fe640 1 -- 172.21.15.96:0/2264435559 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d800116b0 con 0x7f7d88103540 2024-04-25T14:11:26.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.223+0000 7f7d8cdab640 1 -- 172.21.15.96:0/2264435559 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d881a46d0 con 0x7f7d88103540 2024-04-25T14:11:26.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.224+0000 7f7d8cdab640 1 -- 172.21.15.96:0/2264435559 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d8810d460 con 0x7f7d88103540 2024-04-25T14:11:26.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.227+0000 7f7d777fe640 1 -- 172.21.15.96:0/2264435559 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f7d800051d0 con 0x7f7d88103540 2024-04-25T14:11:26.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.228+0000 7f7d777fe640 1 --2- 172.21.15.96:0/2264435559 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f7d700afb70 0x7f7d700b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:26.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.228+0000 7f7d8659c640 1 --2- 172.21.15.96:0/2264435559 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f7d700afb70 0x7f7d700b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:26.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.228+0000 7f7d777fe640 1 -- 172.21.15.96:0/2264435559 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f7d800dec20 con 0x7f7d88103540 2024-04-25T14:11:26.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.228+0000 7f7d8659c640 1 --2- 172.21.15.96:0/2264435559 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f7d700afb70 0x7f7d700b2080 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7d7800abe0 tx=0x7f7d7800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:26.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.231+0000 7f7d777fe640 1 -- 172.21.15.96:0/2264435559 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7d80016030 con 0x7f7d88103540 2024-04-25T14:11:26.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.438+0000 7f7d8cdab640 1 -- 172.21.15.96:0/2264435559 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7d880637a0 con 0x7f7d88103540 2024-04-25T14:11:26.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.439+0000 7f7d777fe640 1 -- 172.21.15.96:0/2264435559 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+18383 (secure 0 0 0) 0x7f7d800a7d20 con 0x7f7d88103540 2024-04-25T14:11:26.439 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:11:26.439 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":72,"fsid":"59334182-030c-11ef-bc93-c7b262605968","created":"2024-04-25T14:03:25.680745+0000","modified":"2024-04-25T14:10:37.456185+0000","last_up_change":"2024-04-25T14:10:35.894312+0000","last_in_change":"2024-04-25T14:10:15.583440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:06:47.402927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cf4ba3ac-142e-4e8e-96d4-410d77157eb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6803","nonce":3832506168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6805","nonce":3832506168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832506168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6807","nonce":3832506168}]},"public_addr":"172.21.15.96:6803/3832506168","cluster_addr":"172.21.15.96:6805/3832506168","heartbeat_back_addr":"172.21.15.96:6809/3832506168","heartbeat_front_addr":"172.21.15.96:6807/3832506168","state":["exists","up"]},{"osd":1,"uuid":"866e73b9-4a26-48ba-badf-704fe22f71cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6801","nonce":1083311494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6803","nonce":1083311494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6807","nonce":1083311494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6805","nonce":1083311494}]},"public_addr":"172.21.15.97:6801/1083311494","cluster_addr":"172.21.15.97:6803/1083311494","heartbeat_back_addr":"172.21.15.97:6807/1083311494","heartbeat_front_addr":"172.21.15.97:6805/1083311494","state":["exists","up"]},{"osd":2,"uuid":"45c79ecb-abc4-4dba-bae8-74804e4bc1bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6801","nonce":981902966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6803","nonce":981902966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6807","nonce":981902966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6805","nonce":981902966}]},"public_addr":"172.21.15.151:6801/981902966","cluster_addr":"172.21.15.151:6803/981902966","heartbeat_back_addr":"172.21.15.151:6807/981902966","heartbeat_front_addr":"172.21.15.151:6805/981902966","state":["exists","up"]},{"osd":3,"uuid":"9fc39468-a7ea-4ccf-b365-335ad2ea4c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6811","nonce":2274158748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6813","nonce":2274158748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6817","nonce":2274158748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6815","nonce":2274158748}]},"public_addr":"172.21.15.96:6811/2274158748","cluster_addr":"172.21.15.96:6813/2274158748","heartbeat_back_addr":"172.21.15.96:6817/2274158748","heartbeat_front_addr":"172.21.15.96:6815/2274158748","state":["exists","up"]},{"osd":4,"uuid":"616582f1-12d2-456b-b7b3-e5517800e8a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6809","nonce":3727031962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6811","nonce":3727031962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6815","nonce":3727031962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6813","nonce":3727031962}]},"public_addr":"172.21.15.97:6809/3727031962","cluster_addr":"172.21.15.97:6811/3727031962","heartbeat_back_addr":"172.21.15.97:6815/3727031962","heartbeat_front_addr":"172.21.15.97:6813/3727031962","state":["exists","up"]},{"osd":5,"uuid":"3d277784-ab0f-4c90-81a1-2a4034c52a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6809","nonce":223882546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6811","nonce":223882546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6815","nonce":223882546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6813","nonce":223882546}]},"public_addr":"172.21.15.151:6809/223882546","cluster_addr":"172.21.15.151:6811/223882546","heartbeat_back_addr":"172.21.15.151:6815/223882546","heartbeat_front_addr":"172.21.15.151:6813/223882546","state":["exists","up"]},{"osd":6,"uuid":"9a9ad585-cb1a-4239-9d18-055f59164049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6819","nonce":96197407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6821","nonce":96197407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6825","nonce":96197407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6823","nonce":96197407}]},"public_addr":"172.21.15.96:6819/96197407","cluster_addr":"172.21.15.96:6821/96197407","heartbeat_back_addr":"172.21.15.96:6825/96197407","heartbeat_front_addr":"172.21.15.96:6823/96197407","state":["exists","up"]},{"osd":7,"uuid":"303551c8-6af0-42f2-844f-45bf8db5baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6817","nonce":2247216400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6819","nonce":2247216400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6823","nonce":2247216400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6821","nonce":2247216400}]},"public_addr":"172.21.15.97:6817/2247216400","cluster_addr":"172.21.15.97:6819/2247216400","heartbeat_back_addr":"172.21.15.97:6823/2247216400","heartbeat_front_addr":"172.21.15.97:6821/2247216400","state":["exists","up"]},{"osd":8,"uuid":"81f6cc66-e4ca-430c-b336-b6678d968818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6817","nonce":2549822185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6819","nonce":2549822185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6823","nonce":2549822185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6821","nonce":2549822185}]},"public_addr":"172.21.15.151:6817/2549822185","cluster_addr":"172.21.15.151:6819/2549822185","heartbeat_back_addr":"172.21.15.151:6823/2549822185","heartbeat_front_addr":"172.21.15.151:6821/2549822185","state":["exists","up"]},{"osd":9,"uuid":"29d4da16-feab-4d1c-9d1d-de4f799bd724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6827","nonce":1099399643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6829","nonce":1099399643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6833","nonce":1099399643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6831","nonce":1099399643}]},"public_addr":"172.21.15.96:6827/1099399643","cluster_addr":"172.21.15.96:6829/1099399643","heartbeat_back_addr":"172.21.15.96:6833/1099399643","heartbeat_front_addr":"172.21.15.96:6831/1099399643","state":["exists","up"]},{"osd":10,"uuid":"197fee9b-eee4-4893-be1c-3d57f42250e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6825","nonce":155754520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6827","nonce":155754520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6831","nonce":155754520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6829","nonce":155754520}]},"public_addr":"172.21.15.97:6825/155754520","cluster_addr":"172.21.15.97:6827/155754520","heartbeat_back_addr":"172.21.15.97:6831/155754520","heartbeat_front_addr":"172.21.15.97:6829/155754520","state":["exists","up"]},{"osd":11,"uuid":"9c1d0a83-fc12-41a2-8eb0-80c4abdd2770","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6825","nonce":2027958746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6827","nonce":2027958746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6831","nonce":2027958746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6829","nonce":2027958746}]},"public_addr":"172.21.15.151:6825/2027958746","cluster_addr":"172.21.15.151:6827/2027958746","heartbeat_back_addr":"172.21.15.151:6831/2027958746","heartbeat_front_addr":"172.21.15.151:6829/2027958746","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:05:56.177390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:06:19.917663+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:06:43.608052+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:08.540382+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:33.474072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:58.242306+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:08:24.120442+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:08:49.661064+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:09:15.097271+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:09:41.904505+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:10:08.233252+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:10:33.712296+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/3839807484":"2024-04-26T14:04:21.307574+0000","172.21.15.96:0/1495237551":"2024-04-26T14:04:21.307574+0000","172.21.15.96:0/2793751450":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/129575834":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/2233075139":"2024-04-26T14:04:21.307574+0000","172.21.15.96:6801/2348397481":"2024-04-26T14:03:54.249537+0000","172.21.15.96:6801/433591062":"2024-04-26T14:04:21.307574+0000","172.21.15.96:6800/2348397481":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/3517674924":"2024-04-26T14:03:54.249537+0000","172.21.15.96:6800/433591062":"2024-04-26T14:04:21.307574+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:11:26.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.442+0000 7f7d8cdab640 1 -- 172.21.15.96:0/2264435559 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f7d700afb70 msgr2=0x7f7d700b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:26.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.442+0000 7f7d8cdab640 1 --2- 172.21.15.96:0/2264435559 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f7d700afb70 0x7f7d700b2080 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7d7800abe0 tx=0x7f7d7800a2d0 comp rx=0 tx=0).stop 2024-04-25T14:11:26.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.442+0000 7f7d8cdab640 1 -- 172.21.15.96:0/2264435559 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d88103540 msgr2=0x7f7d881a3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:26.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.442+0000 7f7d8cdab640 1 --2- 172.21.15.96:0/2264435559 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d88103540 0x7f7d881a3880 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7d8000eaa0 tx=0x7f7d8000eea0 comp rx=0 tx=0).stop 2024-04-25T14:11:26.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.442+0000 7f7d8cdab640 1 -- 172.21.15.96:0/2264435559 shutdown_connections 2024-04-25T14:11:26.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.442+0000 7f7d8cdab640 1 --2- 172.21.15.96:0/2264435559 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f7d700afb70 0x7f7d700b2080 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:26.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.442+0000 7f7d8cdab640 1 --2- 172.21.15.96:0/2264435559 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d88103540 0x7f7d881a3880 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:26.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.442+0000 7f7d8cdab640 1 --2- 172.21.15.96:0/2264435559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d881029e0 0x7f7d8819c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:26.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.442+0000 7f7d8cdab640 1 --2- 172.21.15.96:0/2264435559 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d88102000 0x7f7d8819c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:26.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.442+0000 7f7d8cdab640 1 -- 172.21.15.96:0/2264435559 >> 172.21.15.96:0/2264435559 conn(0x7f7d880fdbd0 msgr2=0x7f7d8810a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:26.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.442+0000 7f7d8cdab640 1 -- 172.21.15.96:0/2264435559 shutdown_connections 2024-04-25T14:11:26.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.442+0000 7f7d8cdab640 1 -- 172.21.15.96:0/2264435559 wait complete. 2024-04-25T14:11:26.454 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:06:47.402927+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:11:26.454 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:11:26.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.567+0000 7f2195a67640 1 -- 172.21.15.96:0/1429899020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21901046a0 msgr2=0x7f219010adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:26.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.567+0000 7f2195a67640 1 --2- 172.21.15.96:0/1429899020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21901046a0 0x7f219010adf0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2188009ca0 tx=0x7f218802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:11:26.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.567+0000 7f2195a67640 1 -- 172.21.15.96:0/1429899020 shutdown_connections 2024-04-25T14:11:26.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.567+0000 7f2195a67640 1 --2- 172.21.15.96:0/1429899020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21901046a0 0x7f219010adf0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:26.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.567+0000 7f2195a67640 1 --2- 172.21.15.96:0/1429899020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2190103b40 0x7f2190104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:26.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.567+0000 7f2195a67640 1 --2- 172.21.15.96:0/1429899020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2190102fe0 0x7f21901033f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:26.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.567+0000 7f2195a67640 1 -- 172.21.15.96:0/1429899020 >> 172.21.15.96:0/1429899020 conn(0x7f21900fd380 msgr2=0x7f21900ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:26.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.567+0000 7f2195a67640 1 -- 172.21.15.96:0/1429899020 shutdown_connections 2024-04-25T14:11:26.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.567+0000 7f2195a67640 1 -- 172.21.15.96:0/1429899020 wait complete. 2024-04-25T14:11:26.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.568+0000 7f2195a67640 1 Processor -- start 2024-04-25T14:11:26.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.568+0000 7f2195a67640 1 -- start start 2024-04-25T14:11:26.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.568+0000 7f2195a67640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2190102fe0 0x7f2190102280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:26.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.568+0000 7f2195a67640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2190103b40 0x7f219019ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:26.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.568+0000 7f2195a67640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f219019f350 0x7f21901a1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:26.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.568+0000 7f2195a67640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f219010e810 con 0x7f2190102fe0 2024-04-25T14:11:26.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.568+0000 7f2195a67640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f219010e690 con 0x7f2190103b40 2024-04-25T14:11:26.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.568+0000 7f2195a67640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f219010e990 con 0x7f219019f350 2024-04-25T14:11:26.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.568+0000 7f218f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2190102fe0 0x7f2190102280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:26.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.568+0000 7f218f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2190102fe0 0x7f2190102280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56360/0 (socket says 172.21.15.96:56360) 2024-04-25T14:11:26.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.568+0000 7f218f7fe640 1 -- 172.21.15.96:0/289883994 learned_addr learned my addr 172.21.15.96:0/289883994 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:11:26.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.568+0000 7f218f7fe640 1 -- 172.21.15.96:0/289883994 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f219019f350 msgr2=0x7f21901a1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:26.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.568+0000 7f218effd640 1 --2- 172.21.15.96:0/289883994 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2190103b40 0x7f219019ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:26.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.568+0000 7f218f7fe640 1 --2- 172.21.15.96:0/289883994 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f219019f350 0x7f21901a1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:26.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.568+0000 7f218f7fe640 1 -- 172.21.15.96:0/289883994 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2190103b40 msgr2=0x7f219019ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:26.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.568+0000 7f218f7fe640 1 --2- 172.21.15.96:0/289883994 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2190103b40 0x7f219019ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:26.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.568+0000 7f218f7fe640 1 -- 172.21.15.96:0/289883994 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21901a1f90 con 0x7f2190102fe0 2024-04-25T14:11:26.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.568+0000 7f218effd640 1 --2- 172.21.15.96:0/289883994 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2190103b40 0x7f219019ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:26.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.569+0000 7f218f7fe640 1 --2- 172.21.15.96:0/289883994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2190102fe0 0x7f2190102280 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f218000b450 tx=0x7f218000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:26.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.569+0000 7f218cff9640 1 -- 172.21.15.96:0/289883994 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2180015020 con 0x7f2190102fe0 2024-04-25T14:11:26.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.569+0000 7f218cff9640 1 -- 172.21.15.96:0/289883994 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2180011d40 con 0x7f2190102fe0 2024-04-25T14:11:26.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.569+0000 7f2195a67640 1 -- 172.21.15.96:0/289883994 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21901a2280 con 0x7f2190102fe0 2024-04-25T14:11:26.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.569+0000 7f218cff9640 1 -- 172.21.15.96:0/289883994 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218000f340 con 0x7f2190102fe0 2024-04-25T14:11:26.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.569+0000 7f2195a67640 1 -- 172.21.15.96:0/289883994 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21901a28a0 con 0x7f2190102fe0 2024-04-25T14:11:26.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.570+0000 7f2195a67640 1 -- 172.21.15.96:0/289883994 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f219006b350 con 0x7f2190102fe0 2024-04-25T14:11:26.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.571+0000 7f218cff9640 1 -- 172.21.15.96:0/289883994 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f2180018020 con 0x7f2190102fe0 2024-04-25T14:11:26.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.572+0000 7f218cff9640 1 --2- 172.21.15.96:0/289883994 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f21780afb70 0x7f21780b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:26.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.572+0000 7f218effd640 1 --2- 172.21.15.96:0/289883994 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f21780afb70 0x7f21780b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:26.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.572+0000 7f218cff9640 1 -- 172.21.15.96:0/289883994 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f21800de230 con 0x7f2190102fe0 2024-04-25T14:11:26.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.572+0000 7f218effd640 1 --2- 172.21.15.96:0/289883994 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f21780afb70 0x7f21780b2080 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f21840058c0 tx=0x7f2184005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:26.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.576+0000 7f218cff9640 1 -- 172.21.15.96:0/289883994 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f218001f020 con 0x7f2190102fe0 2024-04-25T14:11:26.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.763+0000 7f2195a67640 1 -- 172.21.15.96:0/289883994 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f219002de70 con 0x7f2190102fe0 2024-04-25T14:11:26.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.763+0000 7f218cff9640 1 -- 172.21.15.96:0/289883994 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) ==== 93+0+10 (secure 0 0 0) 0x7f218000d040 con 0x7f2190102fe0 2024-04-25T14:11:26.763 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-04-25T14:11:26.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.765+0000 7f2195a67640 1 -- 172.21.15.96:0/289883994 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f21780afb70 msgr2=0x7f21780b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:26.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.765+0000 7f2195a67640 1 --2- 172.21.15.96:0/289883994 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f21780afb70 0x7f21780b2080 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f21840058c0 tx=0x7f2184005d50 comp rx=0 tx=0).stop 2024-04-25T14:11:26.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.765+0000 7f2195a67640 1 -- 172.21.15.96:0/289883994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2190102fe0 msgr2=0x7f2190102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:26.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.765+0000 7f2195a67640 1 --2- 172.21.15.96:0/289883994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2190102fe0 0x7f2190102280 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f218000b450 tx=0x7f218000b910 comp rx=0 tx=0).stop 2024-04-25T14:11:26.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.765+0000 7f2195a67640 1 -- 172.21.15.96:0/289883994 shutdown_connections 2024-04-25T14:11:26.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.765+0000 7f2195a67640 1 --2- 172.21.15.96:0/289883994 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f219019f350 0x7f21901a1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:26.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.765+0000 7f2195a67640 1 --2- 172.21.15.96:0/289883994 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2190103b40 0x7f219019ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:26.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.765+0000 7f2195a67640 1 --2- 172.21.15.96:0/289883994 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f21780afb70 0x7f21780b2080 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:26.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.765+0000 7f2195a67640 1 --2- 172.21.15.96:0/289883994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2190102fe0 0x7f2190102280 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:26.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.765+0000 7f2195a67640 1 -- 172.21.15.96:0/289883994 >> 172.21.15.96:0/289883994 conn(0x7f21900fd380 msgr2=0x7f2190106ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:26.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.766+0000 7f2195a67640 1 -- 172.21.15.96:0/289883994 shutdown_connections 2024-04-25T14:11:26.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.766+0000 7f2195a67640 1 -- 172.21.15.96:0/289883994 wait complete. 2024-04-25T14:11:26.776 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:26 smithi096 ceph-mon[29207]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:26.776 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:26 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:26.776 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:26 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:26.776 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:26 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:26.776 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:26 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:26.776 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:26 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:26.777 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:26 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:26.777 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:26 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/693542465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:26.777 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:26 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:26.777 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:26 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/2264435559' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:11:26.778 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T14:11:26.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.888+0000 7fc9b8005640 1 -- 172.21.15.96:0/1326654655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9b0102f30 msgr2=0x7fc9b0103340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:26.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.888+0000 7fc9b8005640 1 --2- 172.21.15.96:0/1326654655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9b0102f30 0x7fc9b0103340 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0009900 tx=0x7fc9a002a410 comp rx=0 tx=0).stop 2024-04-25T14:11:26.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.888+0000 7fc9b8005640 1 -- 172.21.15.96:0/1326654655 shutdown_connections 2024-04-25T14:11:26.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.888+0000 7fc9b8005640 1 --2- 172.21.15.96:0/1326654655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc9b0104450 0x7fc9b010ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:26.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.888+0000 7fc9b8005640 1 --2- 172.21.15.96:0/1326654655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9b0103910 0x7fc9b0103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:26.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.888+0000 7fc9b8005640 1 --2- 172.21.15.96:0/1326654655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9b0102f30 0x7fc9b0103340 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:26.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.888+0000 7fc9b8005640 1 -- 172.21.15.96:0/1326654655 >> 172.21.15.96:0/1326654655 conn(0x7fc9b00fd330 msgr2=0x7fc9b00ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:26.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.888+0000 7fc9b8005640 1 -- 172.21.15.96:0/1326654655 shutdown_connections 2024-04-25T14:11:26.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.888+0000 7fc9b8005640 1 -- 172.21.15.96:0/1326654655 wait complete. 2024-04-25T14:11:26.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.888+0000 7fc9b8005640 1 Processor -- start 2024-04-25T14:11:26.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.888+0000 7fc9b8005640 1 -- start start 2024-04-25T14:11:26.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.889+0000 7fc9b8005640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc9b0102f30 0x7fc9b019a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:26.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.889+0000 7fc9b8005640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9b0103910 0x7fc9b019a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:26.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.889+0000 7fc9b8005640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9b0104450 0x7fc9b01a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:26.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.889+0000 7fc9b8005640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b010e4c0 con 0x7fc9b0103910 2024-04-25T14:11:26.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.889+0000 7fc9b8005640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b010e340 con 0x7fc9b0104450 2024-04-25T14:11:26.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.889+0000 7fc9b8005640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b010e640 con 0x7fc9b0102f30 2024-04-25T14:11:26.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.889+0000 7fc9b55a0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9b0103910 0x7fc9b019a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:26.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.889+0000 7fc9b55a0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9b0103910 0x7fc9b019a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56378/0 (socket says 172.21.15.96:56378) 2024-04-25T14:11:26.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.889+0000 7fc9b55a0640 1 -- 172.21.15.96:0/183831323 learned_addr learned my addr 172.21.15.96:0/183831323 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:11:26.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.889+0000 7fc9b5da1640 1 --2- 172.21.15.96:0/183831323 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc9b0102f30 0x7fc9b019a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:26.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.889+0000 7fc9b65a2640 1 --2- 172.21.15.96:0/183831323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9b0104450 0x7fc9b01a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:26.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.889+0000 7fc9b55a0640 1 -- 172.21.15.96:0/183831323 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc9b0102f30 msgr2=0x7fc9b019a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:26.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.889+0000 7fc9b55a0640 1 --2- 172.21.15.96:0/183831323 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc9b0102f30 0x7fc9b019a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:26.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.889+0000 7fc9b55a0640 1 -- 172.21.15.96:0/183831323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9b0104450 msgr2=0x7fc9b01a17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:26.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.889+0000 7fc9b55a0640 1 --2- 172.21.15.96:0/183831323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9b0104450 0x7fc9b01a17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:26.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.889+0000 7fc9b55a0640 1 -- 172.21.15.96:0/183831323 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9b01a1e40 con 0x7fc9b0103910 2024-04-25T14:11:26.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.889+0000 7fc9b55a0640 1 --2- 172.21.15.96:0/183831323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9b0103910 0x7fc9b019a840 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac00e9f0 tx=0x7fc9ac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:26.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.889+0000 7fc9a6ffd640 1 -- 172.21.15.96:0/183831323 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac00cd80 con 0x7fc9b0103910 2024-04-25T14:11:26.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.889+0000 7fc9a6ffd640 1 -- 172.21.15.96:0/183831323 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9ac0108e0 con 0x7fc9b0103910 2024-04-25T14:11:26.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.889+0000 7fc9b8005640 1 -- 172.21.15.96:0/183831323 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9b01a2130 con 0x7fc9b0103910 2024-04-25T14:11:26.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.889+0000 7fc9a6ffd640 1 -- 172.21.15.96:0/183831323 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac018980 con 0x7fc9b0103910 2024-04-25T14:11:26.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.889+0000 7fc9b8005640 1 -- 172.21.15.96:0/183831323 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9b01a2680 con 0x7fc9b0103910 2024-04-25T14:11:26.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.890+0000 7fc9b8005640 1 -- 172.21.15.96:0/183831323 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9b01a28e0 con 0x7fc9b0103910 2024-04-25T14:11:26.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.892+0000 7fc9a6ffd640 1 -- 172.21.15.96:0/183831323 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fc9ac010430 con 0x7fc9b0103910 2024-04-25T14:11:26.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.892+0000 7fc9a6ffd640 1 --2- 172.21.15.96:0/183831323 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fc99c0afaf0 0x7fc99c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:26.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.893+0000 7fc9b5da1640 1 --2- 172.21.15.96:0/183831323 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fc99c0afaf0 0x7fc99c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:26.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.893+0000 7fc9a6ffd640 1 -- 172.21.15.96:0/183831323 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fc9ac0def30 con 0x7fc9b0103910 2024-04-25T14:11:26.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.893+0000 7fc9b5da1640 1 --2- 172.21.15.96:0/183831323 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fc99c0afaf0 0x7fc99c0b2000 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0005ab0 tx=0x7fc9a0031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:26.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:26.901+0000 7fc9a6ffd640 1 -- 172.21.15.96:0/183831323 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc9ac0a7960 con 0x7fc9b0103910 2024-04-25T14:11:26.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:26 smithi097 ceph-mon[32720]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:26.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:26 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:26.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:26 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:26.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:26 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:26.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:26 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:26.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:26 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:26.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:26 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:26.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:26 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/693542465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:26.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:26 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:26.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:26 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/2264435559' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:11:26.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:26 smithi151 ceph-mon[32990]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:26.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:26 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:26.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:26 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:26.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:26 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:26.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:26 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:26.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:26 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:26.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:26 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:26.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:26 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/693542465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:26.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:26 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:26.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:26 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/2264435559' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:11:27.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.087+0000 7fc9b8005640 1 -- 172.21.15.96:0/183831323 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fc9b01a2ab0 con 0x7fc9b0103910 2024-04-25T14:11:27.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.088+0000 7fc9a6ffd640 1 -- 172.21.15.96:0/183831323 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v72) ==== 84+0+65 (secure 0 0 0) 0x7fc9ac0abc60 con 0x7fc9b0103910 2024-04-25T14:11:27.087 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:11:27.088 INFO:teuthology.orchestra.run.smithi096.stdout:[ 2024-04-25T14:11:27.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:11:27.088 INFO:teuthology.orchestra.run.smithi096.stdout: "poolnum": 1, 2024-04-25T14:11:27.088 INFO:teuthology.orchestra.run.smithi096.stdout: "poolname": ".mgr" 2024-04-25T14:11:27.088 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:11:27.088 INFO:teuthology.orchestra.run.smithi096.stdout:] 2024-04-25T14:11:27.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.090+0000 7fc9b8005640 1 -- 172.21.15.96:0/183831323 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fc99c0afaf0 msgr2=0x7fc99c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:27.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.090+0000 7fc9b8005640 1 --2- 172.21.15.96:0/183831323 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fc99c0afaf0 0x7fc99c0b2000 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0005ab0 tx=0x7fc9a0031400 comp rx=0 tx=0).stop 2024-04-25T14:11:27.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.090+0000 7fc9b8005640 1 -- 172.21.15.96:0/183831323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9b0103910 msgr2=0x7fc9b019a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:27.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.090+0000 7fc9b8005640 1 --2- 172.21.15.96:0/183831323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9b0103910 0x7fc9b019a840 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac00e9f0 tx=0x7fc9ac00eeb0 comp rx=0 tx=0).stop 2024-04-25T14:11:27.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.090+0000 7fc9b8005640 1 -- 172.21.15.96:0/183831323 shutdown_connections 2024-04-25T14:11:27.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.090+0000 7fc9b8005640 1 --2- 172.21.15.96:0/183831323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9b0104450 0x7fc9b01a17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:27.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.090+0000 7fc9b8005640 1 --2- 172.21.15.96:0/183831323 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fc99c0afaf0 0x7fc99c0b2000 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:27.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.090+0000 7fc9b8005640 1 --2- 172.21.15.96:0/183831323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9b0103910 0x7fc9b019a840 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:27.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.090+0000 7fc9b8005640 1 --2- 172.21.15.96:0/183831323 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc9b0102f30 0x7fc9b019a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:27.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.090+0000 7fc9b8005640 1 -- 172.21.15.96:0/183831323 >> 172.21.15.96:0/183831323 conn(0x7fc9b00fd330 msgr2=0x7fc9b0109d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:27.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.090+0000 7fc9b8005640 1 -- 172.21.15.96:0/183831323 shutdown_connections 2024-04-25T14:11:27.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.090+0000 7fc9b8005640 1 -- 172.21.15.96:0/183831323 wait complete. 2024-04-25T14:11:27.101 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T14:11:27.101 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T14:11:27.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.228+0000 7f6015a13640 1 -- 172.21.15.96:0/1564930369 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6010102170 msgr2=0x7f6010102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:27.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.228+0000 7f6015a13640 1 --2- 172.21.15.96:0/1564930369 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6010102170 0x7f6010102e70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6004009900 tx=0x7f600402a410 comp rx=0 tx=0).stop 2024-04-25T14:11:27.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.229+0000 7f6015a13640 1 -- 172.21.15.96:0/1564930369 shutdown_connections 2024-04-25T14:11:27.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.229+0000 7f6015a13640 1 --2- 172.21.15.96:0/1564930369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60101080b0 0x7f601010a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:27.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.229+0000 7f6015a13640 1 --2- 172.21.15.96:0/1564930369 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60101033b0 0x7f60101037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:27.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.229+0000 7f6015a13640 1 --2- 172.21.15.96:0/1564930369 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6010102170 0x7f6010102e70 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:27.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.229+0000 7f6015a13640 1 -- 172.21.15.96:0/1564930369 >> 172.21.15.96:0/1564930369 conn(0x7f60100fd340 msgr2=0x7f60100ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:27.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.229+0000 7f6015a13640 1 -- 172.21.15.96:0/1564930369 shutdown_connections 2024-04-25T14:11:27.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.229+0000 7f6015a13640 1 -- 172.21.15.96:0/1564930369 wait complete. 2024-04-25T14:11:27.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.229+0000 7f6015a13640 1 Processor -- start 2024-04-25T14:11:27.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.229+0000 7f6015a13640 1 -- start start 2024-04-25T14:11:27.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.230+0000 7f6015a13640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6010102170 0x7f6010199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:27.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.230+0000 7f6015a13640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60101033b0 0x7f6010199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:27.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.230+0000 7f6015a13640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60101080b0 0x7f60101a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:27.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.230+0000 7f6015a13640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f601010e4f0 con 0x7f60101033b0 2024-04-25T14:11:27.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.230+0000 7f6015a13640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f601010e370 con 0x7f6010102170 2024-04-25T14:11:27.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.230+0000 7f6015a13640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f601010e670 con 0x7f60101080b0 2024-04-25T14:11:27.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.230+0000 7f600e7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60101033b0 0x7f6010199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:27.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.230+0000 7f600e7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60101033b0 0x7f6010199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56392/0 (socket says 172.21.15.96:56392) 2024-04-25T14:11:27.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.230+0000 7f600e7fc640 1 -- 172.21.15.96:0/364699839 learned_addr learned my addr 172.21.15.96:0/364699839 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:11:27.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.230+0000 7f600effd640 1 --2- 172.21.15.96:0/364699839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6010102170 0x7f6010199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:27.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.230+0000 7f600f7fe640 1 --2- 172.21.15.96:0/364699839 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60101080b0 0x7f60101a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:27.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.230+0000 7f600e7fc640 1 -- 172.21.15.96:0/364699839 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60101080b0 msgr2=0x7f60101a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:27.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.230+0000 7f600e7fc640 1 --2- 172.21.15.96:0/364699839 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60101080b0 0x7f60101a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:27.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.230+0000 7f600e7fc640 1 -- 172.21.15.96:0/364699839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6010102170 msgr2=0x7f6010199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:27.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.230+0000 7f600e7fc640 1 --2- 172.21.15.96:0/364699839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6010102170 0x7f6010199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:27.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.230+0000 7f600e7fc640 1 -- 172.21.15.96:0/364699839 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60101a0c90 con 0x7f60101033b0 2024-04-25T14:11:27.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.230+0000 7f600e7fc640 1 --2- 172.21.15.96:0/364699839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60101033b0 0x7f6010199690 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f600000e9f0 tx=0x7f600000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:27.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.230+0000 7f6014a11640 1 -- 172.21.15.96:0/364699839 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600000cd80 con 0x7f60101033b0 2024-04-25T14:11:27.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.230+0000 7f6014a11640 1 -- 172.21.15.96:0/364699839 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60000054d0 con 0x7f60101033b0 2024-04-25T14:11:27.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.230+0000 7f6015a13640 1 -- 172.21.15.96:0/364699839 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60101a0f80 con 0x7f60101033b0 2024-04-25T14:11:27.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.230+0000 7f6014a11640 1 -- 172.21.15.96:0/364699839 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6000010690 con 0x7f60101033b0 2024-04-25T14:11:27.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.231+0000 7f6015a13640 1 -- 172.21.15.96:0/364699839 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60101a1390 con 0x7f60101033b0 2024-04-25T14:11:27.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.232+0000 7f6015a13640 1 -- 172.21.15.96:0/364699839 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6010106ec0 con 0x7f60101033b0 2024-04-25T14:11:27.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.233+0000 7f6014a11640 1 -- 172.21.15.96:0/364699839 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f6000004e30 con 0x7f60101033b0 2024-04-25T14:11:27.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.234+0000 7f6014a11640 1 --2- 172.21.15.96:0/364699839 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f5ff80afaf0 0x7f5ff80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:27.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.234+0000 7f600effd640 1 --2- 172.21.15.96:0/364699839 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f5ff80afaf0 0x7f5ff80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:27.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.234+0000 7f6014a11640 1 -- 172.21.15.96:0/364699839 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f60000ded30 con 0x7f60101033b0 2024-04-25T14:11:27.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.234+0000 7f600effd640 1 --2- 172.21.15.96:0/364699839 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f5ff80afaf0 0x7f5ff80b2000 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6004005ab0 tx=0x7f6004031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:27.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.238+0000 7f6014a11640 1 -- 172.21.15.96:0/364699839 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6000053ff0 con 0x7f60101033b0 2024-04-25T14:11:27.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.429+0000 7f6015a13640 1 -- 172.21.15.96:0/364699839 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f601002db50 con 0x7f60101033b0 2024-04-25T14:11:27.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.521+0000 7f6014a11640 1 -- 172.21.15.96:0/364699839 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v73) ==== 138+0+0 (secure 0 0 0) 0x7f60000a7db0 con 0x7f60101033b0 2024-04-25T14:11:27.521 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_metadata' created 2024-04-25T14:11:27.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.523+0000 7f6015a13640 1 -- 172.21.15.96:0/364699839 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f5ff80afaf0 msgr2=0x7f5ff80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:27.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.523+0000 7f6015a13640 1 --2- 172.21.15.96:0/364699839 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f5ff80afaf0 0x7f5ff80b2000 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6004005ab0 tx=0x7f6004031400 comp rx=0 tx=0).stop 2024-04-25T14:11:27.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.523+0000 7f6015a13640 1 -- 172.21.15.96:0/364699839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60101033b0 msgr2=0x7f6010199690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:27.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.523+0000 7f6015a13640 1 --2- 172.21.15.96:0/364699839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60101033b0 0x7f6010199690 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f600000e9f0 tx=0x7f600000edf0 comp rx=0 tx=0).stop 2024-04-25T14:11:27.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.523+0000 7f6015a13640 1 -- 172.21.15.96:0/364699839 shutdown_connections 2024-04-25T14:11:27.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.523+0000 7f6015a13640 1 --2- 172.21.15.96:0/364699839 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60101080b0 0x7f60101a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:27.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.523+0000 7f6015a13640 1 --2- 172.21.15.96:0/364699839 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f5ff80afaf0 0x7f5ff80b2000 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:27.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.523+0000 7f6015a13640 1 --2- 172.21.15.96:0/364699839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60101033b0 0x7f6010199690 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:27.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.523+0000 7f6015a13640 1 --2- 172.21.15.96:0/364699839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6010102170 0x7f6010199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:27.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.523+0000 7f6015a13640 1 -- 172.21.15.96:0/364699839 >> 172.21.15.96:0/364699839 conn(0x7f60100fd340 msgr2=0x7f6010104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:27.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.523+0000 7f6015a13640 1 -- 172.21.15.96:0/364699839 shutdown_connections 2024-04-25T14:11:27.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.523+0000 7f6015a13640 1 -- 172.21.15.96:0/364699839 wait complete. 2024-04-25T14:11:27.538 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T14:11:27.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.657+0000 7f972de0c640 1 -- 172.21.15.96:0/1825618374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9728108900 msgr2=0x7f972810ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:27.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.657+0000 7f972de0c640 1 --2- 172.21.15.96:0/1825618374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9728108900 0x7f972810ad20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9720009ca0 tx=0x7f972002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:11:27.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.657+0000 7f972de0c640 1 -- 172.21.15.96:0/1825618374 shutdown_connections 2024-04-25T14:11:27.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.657+0000 7f972de0c640 1 --2- 172.21.15.96:0/1825618374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9728108900 0x7f972810ad20 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:27.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.657+0000 7f972de0c640 1 --2- 172.21.15.96:0/1825618374 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9728103c00 0x7f9728104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:27.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.657+0000 7f972de0c640 1 --2- 172.21.15.96:0/1825618374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97281029c0 0x7f97281036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:27.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.657+0000 7f972de0c640 1 -- 172.21.15.96:0/1825618374 >> 172.21.15.96:0/1825618374 conn(0x7f97280fdbd0 msgr2=0x7f9728100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:27.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.657+0000 7f972de0c640 1 -- 172.21.15.96:0/1825618374 shutdown_connections 2024-04-25T14:11:27.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.657+0000 7f972de0c640 1 -- 172.21.15.96:0/1825618374 wait complete. 2024-04-25T14:11:27.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.658+0000 7f972de0c640 1 Processor -- start 2024-04-25T14:11:27.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.658+0000 7f972de0c640 1 -- start start 2024-04-25T14:11:27.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.658+0000 7f972de0c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97281029c0 0x7f9728199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:27.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.658+0000 7f972de0c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9728103c00 0x7f97281995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:27.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.658+0000 7f972de0c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9728108900 0x7f97281a0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:27.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.658+0000 7f972de0c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f972810e4d0 con 0x7f9728103c00 2024-04-25T14:11:27.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.658+0000 7f972de0c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f972810e350 con 0x7f9728108900 2024-04-25T14:11:27.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.658+0000 7f972de0c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f972810e650 con 0x7f97281029c0 2024-04-25T14:11:27.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.658+0000 7f9726ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9728103c00 0x7f97281995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:27.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.658+0000 7f9726ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9728103c00 0x7f97281995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56410/0 (socket says 172.21.15.96:56410) 2024-04-25T14:11:27.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.658+0000 7f9726ffd640 1 -- 172.21.15.96:0/911878914 learned_addr learned my addr 172.21.15.96:0/911878914 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:11:27.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.658+0000 7f9727fff640 1 --2- 172.21.15.96:0/911878914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9728108900 0x7f97281a0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:27.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.658+0000 7f97277fe640 1 --2- 172.21.15.96:0/911878914 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97281029c0 0x7f9728199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:27.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.658+0000 7f9726ffd640 1 -- 172.21.15.96:0/911878914 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97281029c0 msgr2=0x7f9728199070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:27.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.658+0000 7f9726ffd640 1 --2- 172.21.15.96:0/911878914 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97281029c0 0x7f9728199070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:27.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.658+0000 7f9726ffd640 1 -- 172.21.15.96:0/911878914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9728108900 msgr2=0x7f97281a0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:27.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.658+0000 7f9726ffd640 1 --2- 172.21.15.96:0/911878914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9728108900 0x7f97281a0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:27.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.658+0000 7f9726ffd640 1 -- 172.21.15.96:0/911878914 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97281a0b00 con 0x7f9728103c00 2024-04-25T14:11:27.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.658+0000 7f97277fe640 1 --2- 172.21.15.96:0/911878914 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97281029c0 0x7f9728199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:27.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.658+0000 7f9727fff640 1 --2- 172.21.15.96:0/911878914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9728108900 0x7f97281a0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:11:27.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.659+0000 7f9726ffd640 1 --2- 172.21.15.96:0/911878914 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9728103c00 0x7f97281995b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f971800c920 tx=0x7f971800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:27.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.659+0000 7f9724ff9640 1 -- 172.21.15.96:0/911878914 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9718007c70 con 0x7f9728103c00 2024-04-25T14:11:27.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.659+0000 7f972de0c640 1 -- 172.21.15.96:0/911878914 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97281a0df0 con 0x7f9728103c00 2024-04-25T14:11:27.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.659+0000 7f9724ff9640 1 -- 172.21.15.96:0/911878914 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9718007e10 con 0x7f9728103c00 2024-04-25T14:11:27.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.659+0000 7f9724ff9640 1 -- 172.21.15.96:0/911878914 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9718018820 con 0x7f9728103c00 2024-04-25T14:11:27.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.659+0000 7f972de0c640 1 -- 172.21.15.96:0/911878914 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97281a1260 con 0x7f9728103c00 2024-04-25T14:11:27.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.660+0000 7f972de0c640 1 -- 172.21.15.96:0/911878914 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9728107710 con 0x7f9728103c00 2024-04-25T14:11:27.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.661+0000 7f9724ff9640 1 -- 172.21.15.96:0/911878914 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f971800f450 con 0x7f9728103c00 2024-04-25T14:11:27.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.662+0000 7f9724ff9640 1 --2- 172.21.15.96:0/911878914 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f97100afaf0 0x7f97100b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:27.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.663+0000 7f97277fe640 1 --2- 172.21.15.96:0/911878914 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f97100afaf0 0x7f97100b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:27.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.663+0000 7f9724ff9640 1 -- 172.21.15.96:0/911878914 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7497+0+0 (secure 0 0 0) 0x7f97180ded70 con 0x7f9728103c00 2024-04-25T14:11:27.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.663+0000 7f97277fe640 1 --2- 172.21.15.96:0/911878914 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f97100afaf0 0x7f97100b2000 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f97281035c0 tx=0x7f971c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:27.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.666+0000 7f9724ff9640 1 -- 172.21.15.96:0/911878914 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9718014030 con 0x7f9728103c00 2024-04-25T14:11:27.786 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:27 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:11:27.786 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:27 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:11:27.786 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:27 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:27.786 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:27 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:11:27.786 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:27 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/289883994' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:11:27.787 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:27 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/183831323' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T14:11:27.787 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:27 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/364699839' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T14:11:27.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:27.849+0000 7f972de0c640 1 -- 172.21.15.96:0/911878914 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f972802db50 con 0x7f9728103c00 2024-04-25T14:11:27.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:27 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:11:27.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:27 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:11:27.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:27 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:27.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:27 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:11:27.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:27 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/289883994' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:11:27.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:27 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/183831323' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T14:11:27.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:27 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/364699839' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T14:11:27.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:27 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:11:27.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:27 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:11:27.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:27 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:27.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:27 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:11:27.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:27 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/289883994' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:11:27.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:27 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/183831323' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T14:11:27.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:27 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/364699839' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T14:11:28.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.523+0000 7f9724ff9640 1 -- 172.21.15.96:0/911878914 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v74) ==== 170+0+0 (secure 0 0 0) 0x7f97180a7c70 con 0x7f9728103c00 2024-04-25T14:11:28.523 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_data' created 2024-04-25T14:11:28.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.525+0000 7f972de0c640 1 -- 172.21.15.96:0/911878914 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f97100afaf0 msgr2=0x7f97100b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:28.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.525+0000 7f972de0c640 1 --2- 172.21.15.96:0/911878914 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f97100afaf0 0x7f97100b2000 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f97281035c0 tx=0x7f971c005f90 comp rx=0 tx=0).stop 2024-04-25T14:11:28.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.525+0000 7f972de0c640 1 -- 172.21.15.96:0/911878914 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9728103c00 msgr2=0x7f97281995b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:28.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.525+0000 7f972de0c640 1 --2- 172.21.15.96:0/911878914 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9728103c00 0x7f97281995b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f971800c920 tx=0x7f971800cde0 comp rx=0 tx=0).stop 2024-04-25T14:11:28.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.525+0000 7f972de0c640 1 -- 172.21.15.96:0/911878914 shutdown_connections 2024-04-25T14:11:28.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.525+0000 7f972de0c640 1 --2- 172.21.15.96:0/911878914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9728108900 0x7f97281a0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:28.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.525+0000 7f972de0c640 1 --2- 172.21.15.96:0/911878914 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f97100afaf0 0x7f97100b2000 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:28.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.525+0000 7f972de0c640 1 --2- 172.21.15.96:0/911878914 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9728103c00 0x7f97281995b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:28.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.525+0000 7f972de0c640 1 --2- 172.21.15.96:0/911878914 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97281029c0 0x7f9728199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:28.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.525+0000 7f972de0c640 1 -- 172.21.15.96:0/911878914 >> 172.21.15.96:0/911878914 conn(0x7f97280fdbd0 msgr2=0x7f97281048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:28.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.525+0000 7f972de0c640 1 -- 172.21.15.96:0/911878914 shutdown_connections 2024-04-25T14:11:28.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.525+0000 7f972de0c640 1 -- 172.21.15.96:0/911878914 wait complete. 2024-04-25T14:11:28.544 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T14:11:28.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.660+0000 7f48852f0640 1 -- 172.21.15.96:0/3572088387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4880103c90 msgr2=0x7f48801040a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:28.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.660+0000 7f48852f0640 1 --2- 172.21.15.96:0/3572088387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4880103c90 0x7f48801040a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4874009900 tx=0x7f487402a410 comp rx=0 tx=0).stop 2024-04-25T14:11:28.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.660+0000 7f48852f0640 1 -- 172.21.15.96:0/3572088387 shutdown_connections 2024-04-25T14:11:28.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.660+0000 7f48852f0640 1 --2- 172.21.15.96:0/3572088387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4880108990 0x7f488010adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:28.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.660+0000 7f48852f0640 1 --2- 172.21.15.96:0/3572088387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4880103c90 0x7f48801040a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:28.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.660+0000 7f48852f0640 1 --2- 172.21.15.96:0/3572088387 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4880102a50 0x7f4880103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:28.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.660+0000 7f48852f0640 1 -- 172.21.15.96:0/3572088387 >> 172.21.15.96:0/3572088387 conn(0x7f48800fdc40 msgr2=0x7f48801000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:28.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.660+0000 7f48852f0640 1 -- 172.21.15.96:0/3572088387 shutdown_connections 2024-04-25T14:11:28.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.660+0000 7f48852f0640 1 -- 172.21.15.96:0/3572088387 wait complete. 2024-04-25T14:11:28.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.660+0000 7f48852f0640 1 Processor -- start 2024-04-25T14:11:28.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.660+0000 7f48852f0640 1 -- start start 2024-04-25T14:11:28.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.661+0000 7f48852f0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4880102a50 0x7f4880075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:28.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.661+0000 7f48852f0640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4880103c90 0x7f488019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:28.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.661+0000 7f48852f0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4880108990 0x7f48801a1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:28.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.661+0000 7f48852f0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f488010e500 con 0x7f4880108990 2024-04-25T14:11:28.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.661+0000 7f48852f0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f488010e380 con 0x7f4880102a50 2024-04-25T14:11:28.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.661+0000 7f48852f0640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f488010e680 con 0x7f4880103c90 2024-04-25T14:11:28.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.661+0000 7f487f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4880108990 0x7f48801a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:28.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.661+0000 7f487f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4880108990 0x7f48801a1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56426/0 (socket says 172.21.15.96:56426) 2024-04-25T14:11:28.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.661+0000 7f487f7fe640 1 -- 172.21.15.96:0/1876131909 learned_addr learned my addr 172.21.15.96:0/1876131909 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:11:28.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.661+0000 7f487effd640 1 --2- 172.21.15.96:0/1876131909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4880102a50 0x7f4880075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:28.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.661+0000 7f487e7fc640 1 --2- 172.21.15.96:0/1876131909 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4880103c90 0x7f488019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:28.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.661+0000 7f487f7fe640 1 -- 172.21.15.96:0/1876131909 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4880103c90 msgr2=0x7f488019efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:28.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.661+0000 7f487f7fe640 1 --2- 172.21.15.96:0/1876131909 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4880103c90 0x7f488019efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:28.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.661+0000 7f487f7fe640 1 -- 172.21.15.96:0/1876131909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4880102a50 msgr2=0x7f4880075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:28.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.661+0000 7f487f7fe640 1 --2- 172.21.15.96:0/1876131909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4880102a50 0x7f4880075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:28.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.661+0000 7f487f7fe640 1 -- 172.21.15.96:0/1876131909 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48801a1d50 con 0x7f4880108990 2024-04-25T14:11:28.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.661+0000 7f487effd640 1 --2- 172.21.15.96:0/1876131909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4880102a50 0x7f4880075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:28.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.661+0000 7f487f7fe640 1 --2- 172.21.15.96:0/1876131909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4880108990 0x7f48801a1530 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f487800e970 tx=0x7f487800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:28.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.662+0000 7f4867fff640 1 -- 172.21.15.96:0/1876131909 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487800ccc0 con 0x7f4880108990 2024-04-25T14:11:28.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.662+0000 7f4867fff640 1 -- 172.21.15.96:0/1876131909 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f487800ce60 con 0x7f4880108990 2024-04-25T14:11:28.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.662+0000 7f48852f0640 1 -- 172.21.15.96:0/1876131909 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48801a1fe0 con 0x7f4880108990 2024-04-25T14:11:28.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.662+0000 7f4867fff640 1 -- 172.21.15.96:0/1876131909 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4878010430 con 0x7f4880108990 2024-04-25T14:11:28.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.662+0000 7f48852f0640 1 -- 172.21.15.96:0/1876131909 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48801a2450 con 0x7f4880108990 2024-04-25T14:11:28.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.663+0000 7f48852f0640 1 -- 172.21.15.96:0/1876131909 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48801077a0 con 0x7f4880108990 2024-04-25T14:11:28.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.664+0000 7f4867fff640 1 -- 172.21.15.96:0/1876131909 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f4878004fb0 con 0x7f4880108990 2024-04-25T14:11:28.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.665+0000 7f4867fff640 1 --2- 172.21.15.96:0/1876131909 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f48680afaf0 0x7f48680b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:28.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.665+0000 7f487effd640 1 --2- 172.21.15.96:0/1876131909 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f48680afaf0 0x7f48680b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:28.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.665+0000 7f4867fff640 1 -- 172.21.15.96:0/1876131909 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7879+0+0 (secure 0 0 0) 0x7f48780df160 con 0x7f4880108990 2024-04-25T14:11:28.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.665+0000 7f487effd640 1 --2- 172.21.15.96:0/1876131909 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f48680afaf0 0x7f48680b2000 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4880103650 tx=0x7f4870009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:28.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.669+0000 7f4867fff640 1 -- 172.21.15.96:0/1876131909 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4878053ff0 con 0x7f4880108990 2024-04-25T14:11:28.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:28 smithi096 ceph-mon[29207]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:28.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:28 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/364699839' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T14:11:28.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:28 smithi096 ceph-mon[29207]: osdmap e73: 12 total, 12 up, 12 in 2024-04-25T14:11:28.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:28 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/911878914' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T14:11:28.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:28 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:28.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:28 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:28.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:28 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:28.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:28 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:28.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:28 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:28.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:28 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:28.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:28 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:11:28.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:28 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:11:28.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:28 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:28.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:28.885+0000 7f48852f0640 1 -- 172.21.15.96:0/1876131909 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f488002db50 con 0x7f4880108990 2024-04-25T14:11:28.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:28 smithi097 ceph-mon[32720]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:28.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:28 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/364699839' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T14:11:28.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:28 smithi097 ceph-mon[32720]: osdmap e73: 12 total, 12 up, 12 in 2024-04-25T14:11:28.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:28 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/911878914' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T14:11:28.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:28 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:28.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:28 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:28.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:28 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:28.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:28 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:28.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:28 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:28.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:28 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:28.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:28 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:11:28.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:28 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:11:28.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:28 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:28.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:28 smithi151 ceph-mon[32990]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:28.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:28 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/364699839' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T14:11:28.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:28 smithi151 ceph-mon[32990]: osdmap e73: 12 total, 12 up, 12 in 2024-04-25T14:11:28.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:28 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/911878914' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T14:11:28.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:28 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:28.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:28 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:28.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:28 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:28.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:28 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:28.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:28 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:28.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:28 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:28.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:28 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:11:28.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:28 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:11:28.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:28 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:11:29.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.541+0000 7f4867fff640 1 -- 172.21.15.96:0/1876131909 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f48780a7ef0 con 0x7f4880108990 2024-04-25T14:11:29.541 INFO:teuthology.orchestra.run.smithi096.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T14:11:29.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.544+0000 7f48852f0640 1 -- 172.21.15.96:0/1876131909 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f48680afaf0 msgr2=0x7f48680b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:29.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.544+0000 7f48852f0640 1 --2- 172.21.15.96:0/1876131909 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f48680afaf0 0x7f48680b2000 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4880103650 tx=0x7f4870009500 comp rx=0 tx=0).stop 2024-04-25T14:11:29.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.544+0000 7f48852f0640 1 -- 172.21.15.96:0/1876131909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4880108990 msgr2=0x7f48801a1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:29.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.544+0000 7f48852f0640 1 --2- 172.21.15.96:0/1876131909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4880108990 0x7f48801a1530 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f487800e970 tx=0x7f487800ed70 comp rx=0 tx=0).stop 2024-04-25T14:11:29.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.544+0000 7f48852f0640 1 -- 172.21.15.96:0/1876131909 shutdown_connections 2024-04-25T14:11:29.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.544+0000 7f48852f0640 1 --2- 172.21.15.96:0/1876131909 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f48680afaf0 0x7f48680b2000 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:29.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.544+0000 7f48852f0640 1 --2- 172.21.15.96:0/1876131909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4880108990 0x7f48801a1530 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:29.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.544+0000 7f48852f0640 1 --2- 172.21.15.96:0/1876131909 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4880103c90 0x7f488019efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:29.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.544+0000 7f48852f0640 1 --2- 172.21.15.96:0/1876131909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4880102a50 0x7f4880075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:29.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.544+0000 7f48852f0640 1 -- 172.21.15.96:0/1876131909 >> 172.21.15.96:0/1876131909 conn(0x7f48800fdc40 msgr2=0x7f4880104970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:29.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.545+0000 7f48852f0640 1 -- 172.21.15.96:0/1876131909 shutdown_connections 2024-04-25T14:11:29.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.545+0000 7f48852f0640 1 -- 172.21.15.96:0/1876131909 wait complete. 2024-04-25T14:11:29.569 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:11:29.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.684+0000 7f078f73f640 1 -- 172.21.15.96:0/3007779728 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0788102230 msgr2=0x7f078810bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:29.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.684+0000 7f078f73f640 1 --2- 172.21.15.96:0/3007779728 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0788102230 0x7f078810bc90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0784009900 tx=0x7f078402a410 comp rx=0 tx=0).stop 2024-04-25T14:11:29.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.684+0000 7f078f73f640 1 -- 172.21.15.96:0/3007779728 shutdown_connections 2024-04-25T14:11:29.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.684+0000 7f078f73f640 1 --2- 172.21.15.96:0/3007779728 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0788102230 0x7f078810bc90 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:29.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.684+0000 7f078f73f640 1 --2- 172.21.15.96:0/3007779728 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0788101730 0x7f0788101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:29.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.684+0000 7f078f73f640 1 --2- 172.21.15.96:0/3007779728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0788105080 0x7f0788105490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:29.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.684+0000 7f078f73f640 1 -- 172.21.15.96:0/3007779728 >> 172.21.15.96:0/3007779728 conn(0x7f07880fd330 msgr2=0x7f07880ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:29.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.684+0000 7f078f73f640 1 -- 172.21.15.96:0/3007779728 shutdown_connections 2024-04-25T14:11:29.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.684+0000 7f078f73f640 1 -- 172.21.15.96:0/3007779728 wait complete. 2024-04-25T14:11:29.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.684+0000 7f078f73f640 1 Processor -- start 2024-04-25T14:11:29.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.684+0000 7f078f73f640 1 -- start start 2024-04-25T14:11:29.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.684+0000 7f078f73f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0788101730 0x7f078819a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:29.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.684+0000 7f078f73f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0788102230 0x7f078819a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:29.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.684+0000 7f078f73f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0788105080 0x7f07881a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:29.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.684+0000 7f078f73f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f078810f660 con 0x7f0788102230 2024-04-25T14:11:29.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.685+0000 7f078f73f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f078810f4e0 con 0x7f0788105080 2024-04-25T14:11:29.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.685+0000 7f078f73f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f078810f7e0 con 0x7f0788101730 2024-04-25T14:11:29.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.685+0000 7f078ccda640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0788102230 0x7f078819a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:29.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.685+0000 7f078ccda640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0788102230 0x7f078819a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56436/0 (socket says 172.21.15.96:56436) 2024-04-25T14:11:29.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.685+0000 7f078ccda640 1 -- 172.21.15.96:0/1956521206 learned_addr learned my addr 172.21.15.96:0/1956521206 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:11:29.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.685+0000 7f078d4db640 1 --2- 172.21.15.96:0/1956521206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0788101730 0x7f078819a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:29.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.685+0000 7f078ccda640 1 -- 172.21.15.96:0/1956521206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0788101730 msgr2=0x7f078819a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:29.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.685+0000 7f078ccda640 1 --2- 172.21.15.96:0/1956521206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0788101730 0x7f078819a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:29.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.685+0000 7f078ccda640 1 -- 172.21.15.96:0/1956521206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0788105080 msgr2=0x7f07881a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:29.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.685+0000 7f078dcdc640 1 --2- 172.21.15.96:0/1956521206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0788105080 0x7f07881a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:29.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.685+0000 7f078ccda640 1 --2- 172.21.15.96:0/1956521206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0788105080 0x7f07881a16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:29.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.685+0000 7f078ccda640 1 -- 172.21.15.96:0/1956521206 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07881a1d60 con 0x7f0788102230 2024-04-25T14:11:29.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.685+0000 7f078ccda640 1 --2- 172.21.15.96:0/1956521206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0788102230 0x7f078819a760 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f077800b4a0 tx=0x7f077800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:29.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.685+0000 7f077e7fc640 1 -- 172.21.15.96:0/1956521206 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0778014070 con 0x7f0788102230 2024-04-25T14:11:29.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.685+0000 7f077e7fc640 1 -- 172.21.15.96:0/1956521206 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0778010ac0 con 0x7f0788102230 2024-04-25T14:11:29.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.685+0000 7f078f73f640 1 -- 172.21.15.96:0/1956521206 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07881a1ff0 con 0x7f0788102230 2024-04-25T14:11:29.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.685+0000 7f077e7fc640 1 -- 172.21.15.96:0/1956521206 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0778017c30 con 0x7f0788102230 2024-04-25T14:11:29.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.685+0000 7f078f73f640 1 -- 172.21.15.96:0/1956521206 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07881a2530 con 0x7f0788102230 2024-04-25T14:11:29.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.686+0000 7f078f73f640 1 -- 172.21.15.96:0/1956521206 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f078810aa20 con 0x7f0788102230 2024-04-25T14:11:29.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.688+0000 7f077e7fc640 1 -- 172.21.15.96:0/1956521206 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f0778017070 con 0x7f0788102230 2024-04-25T14:11:29.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.689+0000 7f077e7fc640 1 --2- 172.21.15.96:0/1956521206 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f07740afaf0 0x7f07740b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:29.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.689+0000 7f078d4db640 1 --2- 172.21.15.96:0/1956521206 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f07740afaf0 0x7f07740b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:29.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.689+0000 7f077e7fc640 1 -- 172.21.15.96:0/1956521206 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f07780de320 con 0x7f0788102230 2024-04-25T14:11:29.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.689+0000 7f078d4db640 1 --2- 172.21.15.96:0/1956521206 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f07740afaf0 0x7f07740b2000 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0788069bf0 tx=0x7f0780008230 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:29.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.692+0000 7f077e7fc640 1 -- 172.21.15.96:0/1956521206 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f07780a7050 con 0x7f0788102230 2024-04-25T14:11:29.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:29 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/911878914' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T14:11:29.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:29 smithi096 ceph-mon[29207]: osdmap e74: 12 total, 12 up, 12 in 2024-04-25T14:11:29.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:29 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/1876131909' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T14:11:29.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:29 smithi096 ceph-mon[29207]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T14:11:29.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.874+0000 7f078f73f640 1 -- 172.21.15.96:0/1956521206 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f078802dd20 con 0x7f0788102230 2024-04-25T14:11:29.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.875+0000 7f077e7fc640 1 -- 172.21.15.96:0/1956521206 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+22159 (secure 0 0 0) 0x7f07780af0d0 con 0x7f0788102230 2024-04-25T14:11:29.876 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:11:29.876 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":75,"fsid":"59334182-030c-11ef-bc93-c7b262605968","created":"2024-04-25T14:03:25.680745+0000","modified":"2024-04-25T14:11:28.887003+0000","last_up_change":"2024-04-25T14:10:35.894312+0000","last_in_change":"2024-04-25T14:10:15.583440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:06:47.402927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:11:27.434331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:11:27.854004+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cf4ba3ac-142e-4e8e-96d4-410d77157eb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6803","nonce":3832506168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6805","nonce":3832506168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832506168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6807","nonce":3832506168}]},"public_addr":"172.21.15.96:6803/3832506168","cluster_addr":"172.21.15.96:6805/3832506168","heartbeat_back_addr":"172.21.15.96:6809/3832506168","heartbeat_front_addr":"172.21.15.96:6807/3832506168","state":["exists","up"]},{"osd":1,"uuid":"866e73b9-4a26-48ba-badf-704fe22f71cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6801","nonce":1083311494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6803","nonce":1083311494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6807","nonce":1083311494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6805","nonce":1083311494}]},"public_addr":"172.21.15.97:6801/1083311494","cluster_addr":"172.21.15.97:6803/1083311494","heartbeat_back_addr":"172.21.15.97:6807/1083311494","heartbeat_front_addr":"172.21.15.97:6805/1083311494","state":["exists","up"]},{"osd":2,"uuid":"45c79ecb-abc4-4dba-bae8-74804e4bc1bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6801","nonce":981902966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6803","nonce":981902966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6807","nonce":981902966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6805","nonce":981902966}]},"public_addr":"172.21.15.151:6801/981902966","cluster_addr":"172.21.15.151:6803/981902966","heartbeat_back_addr":"172.21.15.151:6807/981902966","heartbeat_front_addr":"172.21.15.151:6805/981902966","state":["exists","up"]},{"osd":3,"uuid":"9fc39468-a7ea-4ccf-b365-335ad2ea4c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6811","nonce":2274158748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6813","nonce":2274158748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6817","nonce":2274158748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6815","nonce":2274158748}]},"public_addr":"172.21.15.96:6811/2274158748","cluster_addr":"172.21.15.96:6813/2274158748","heartbeat_back_addr":"172.21.15.96:6817/2274158748","heartbeat_front_addr":"172.21.15.96:6815/2274158748","state":["exists","up"]},{"osd":4,"uuid":"616582f1-12d2-456b-b7b3-e5517800e8a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6809","nonce":3727031962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6811","nonce":3727031962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6815","nonce":3727031962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6813","nonce":3727031962}]},"public_addr":"172.21.15.97:6809/3727031962","cluster_addr":"172.21.15.97:6811/3727031962","heartbeat_back_addr":"172.21.15.97:6815/3727031962","heartbeat_front_addr":"172.21.15.97:6813/3727031962","state":["exists","up"]},{"osd":5,"uuid":"3d277784-ab0f-4c90-81a1-2a4034c52a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6809","nonce":223882546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6811","nonce":223882546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6815","nonce":223882546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6813","nonce":223882546}]},"public_addr":"172.21.15.151:6809/223882546","cluster_addr":"172.21.15.151:6811/223882546","heartbeat_back_addr":"172.21.15.151:6815/223882546","heartbeat_front_addr":"172.21.15.151:6813/223882546","state":["exists","up"]},{"osd":6,"uuid":"9a9ad585-cb1a-4239-9d18-055f59164049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6819","nonce":96197407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6821","nonce":96197407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6825","nonce":96197407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6823","nonce":96197407}]},"public_addr":"172.21.15.96:6819/96197407","cluster_addr":"172.21.15.96:6821/96197407","heartbeat_back_addr":"172.21.15.96:6825/96197407","heartbeat_front_addr":"172.21.15.96:6823/96197407","state":["exists","up"]},{"osd":7,"uuid":"303551c8-6af0-42f2-844f-45bf8db5baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6817","nonce":2247216400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6819","nonce":2247216400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6823","nonce":2247216400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6821","nonce":2247216400}]},"public_addr":"172.21.15.97:6817/2247216400","cluster_addr":"172.21.15.97:6819/2247216400","heartbeat_back_addr":"172.21.15.97:6823/2247216400","heartbeat_front_addr":"172.21.15.97:6821/2247216400","state":["exists","up"]},{"osd":8,"uuid":"81f6cc66-e4ca-430c-b336-b6678d968818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6817","nonce":2549822185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6819","nonce":2549822185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6823","nonce":2549822185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6821","nonce":2549822185}]},"public_addr":"172.21.15.151:6817/2549822185","cluster_addr":"172.21.15.151:6819/2549822185","heartbeat_back_addr":"172.21.15.151:6823/2549822185","heartbeat_front_addr":"172.21.15.151:6821/2549822185","state":["exists","up"]},{"osd":9,"uuid":"29d4da16-feab-4d1c-9d1d-de4f799bd724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6827","nonce":1099399643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6829","nonce":1099399643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6833","nonce":1099399643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6831","nonce":1099399643}]},"public_addr":"172.21.15.96:6827/1099399643","cluster_addr":"172.21.15.96:6829/1099399643","heartbeat_back_addr":"172.21.15.96:6833/1099399643","heartbeat_front_addr":"172.21.15.96:6831/1099399643","state":["exists","up"]},{"osd":10,"uuid":"197fee9b-eee4-4893-be1c-3d57f42250e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6825","nonce":155754520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6827","nonce":155754520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6831","nonce":155754520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6829","nonce":155754520}]},"public_addr":"172.21.15.97:6825/155754520","cluster_addr":"172.21.15.97:6827/155754520","heartbeat_back_addr":"172.21.15.97:6831/155754520","heartbeat_front_addr":"172.21.15.97:6829/155754520","state":["exists","up"]},{"osd":11,"uuid":"9c1d0a83-fc12-41a2-8eb0-80c4abdd2770","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6825","nonce":2027958746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6827","nonce":2027958746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6831","nonce":2027958746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6829","nonce":2027958746}]},"public_addr":"172.21.15.151:6825/2027958746","cluster_addr":"172.21.15.151:6827/2027958746","heartbeat_back_addr":"172.21.15.151:6831/2027958746","heartbeat_front_addr":"172.21.15.151:6829/2027958746","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:05:56.177390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:06:19.917663+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:06:43.608052+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:08.540382+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:33.474072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:58.242306+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:08:24.120442+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:08:49.661064+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:09:15.097271+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:09:41.904505+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:10:08.233252+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:10:33.712296+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/3839807484":"2024-04-26T14:04:21.307574+0000","172.21.15.96:0/1495237551":"2024-04-26T14:04:21.307574+0000","172.21.15.96:0/2793751450":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/129575834":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/2233075139":"2024-04-26T14:04:21.307574+0000","172.21.15.96:6801/2348397481":"2024-04-26T14:03:54.249537+0000","172.21.15.96:6801/433591062":"2024-04-26T14:04:21.307574+0000","172.21.15.96:6800/2348397481":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/3517674924":"2024-04-26T14:03:54.249537+0000","172.21.15.96:6800/433591062":"2024-04-26T14:04:21.307574+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:11:29.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.877+0000 7f078f73f640 1 -- 172.21.15.96:0/1956521206 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f07740afaf0 msgr2=0x7f07740b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:29.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.877+0000 7f078f73f640 1 --2- 172.21.15.96:0/1956521206 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f07740afaf0 0x7f07740b2000 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0788069bf0 tx=0x7f0780008230 comp rx=0 tx=0).stop 2024-04-25T14:11:29.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.877+0000 7f078f73f640 1 -- 172.21.15.96:0/1956521206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0788102230 msgr2=0x7f078819a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:29.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.877+0000 7f078f73f640 1 --2- 172.21.15.96:0/1956521206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0788102230 0x7f078819a760 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f077800b4a0 tx=0x7f077800b960 comp rx=0 tx=0).stop 2024-04-25T14:11:29.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.878+0000 7f078f73f640 1 -- 172.21.15.96:0/1956521206 shutdown_connections 2024-04-25T14:11:29.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.878+0000 7f078f73f640 1 --2- 172.21.15.96:0/1956521206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0788105080 0x7f07881a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:29.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.878+0000 7f078f73f640 1 --2- 172.21.15.96:0/1956521206 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f07740afaf0 0x7f07740b2000 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:29.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.878+0000 7f078f73f640 1 --2- 172.21.15.96:0/1956521206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0788102230 0x7f078819a760 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:29.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.878+0000 7f078f73f640 1 --2- 172.21.15.96:0/1956521206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0788101730 0x7f078819a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:29.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.878+0000 7f078f73f640 1 -- 172.21.15.96:0/1956521206 >> 172.21.15.96:0/1956521206 conn(0x7f07880fd330 msgr2=0x7f0788107bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:29.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.878+0000 7f078f73f640 1 -- 172.21.15.96:0/1956521206 shutdown_connections 2024-04-25T14:11:29.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.878+0000 7f078f73f640 1 -- 172.21.15.96:0/1956521206 wait complete. 2024-04-25T14:11:29.889 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:11:29.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:29 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/911878914' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T14:11:29.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:29 smithi097 ceph-mon[32720]: osdmap e74: 12 total, 12 up, 12 in 2024-04-25T14:11:29.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:29 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/1876131909' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T14:11:29.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:29 smithi097 ceph-mon[32720]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T14:11:29.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:29 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/911878914' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T14:11:29.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:29 smithi151 ceph-mon[32990]: osdmap e74: 12 total, 12 up, 12 in 2024-04-25T14:11:29.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:29 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/1876131909' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T14:11:29.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:29 smithi151 ceph-mon[32990]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T14:11:29.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.996+0000 7f0349734640 1 -- 172.21.15.96:0/1009848185 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0344106f20 msgr2=0x7f0344109340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:29.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.996+0000 7f0349734640 1 --2- 172.21.15.96:0/1009848185 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0344106f20 0x7f0344109340 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f032c009900 tx=0x7f032c02a410 comp rx=0 tx=0).stop 2024-04-25T14:11:29.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.996+0000 7f0349734640 1 -- 172.21.15.96:0/1009848185 shutdown_connections 2024-04-25T14:11:29.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.996+0000 7f0349734640 1 --2- 172.21.15.96:0/1009848185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0344109880 0x7f034410bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:29.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.996+0000 7f0349734640 1 --2- 172.21.15.96:0/1009848185 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0344106f20 0x7f0344109340 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:29.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.996+0000 7f0349734640 1 --2- 172.21.15.96:0/1009848185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03441045c0 0x7f03441069e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:29.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.996+0000 7f0349734640 1 -- 172.21.15.96:0/1009848185 >> 172.21.15.96:0/1009848185 conn(0x7f03440fdba0 msgr2=0x7f0344100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:29.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.996+0000 7f0349734640 1 -- 172.21.15.96:0/1009848185 shutdown_connections 2024-04-25T14:11:29.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.996+0000 7f0349734640 1 -- 172.21.15.96:0/1009848185 wait complete. 2024-04-25T14:11:29.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.996+0000 7f0349734640 1 Processor -- start 2024-04-25T14:11:29.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.997+0000 7f0349734640 1 -- start start 2024-04-25T14:11:29.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.997+0000 7f0349734640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03441045c0 0x7f034419a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:29.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.997+0000 7f0349734640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0344106f20 0x7f034419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:29.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.997+0000 7f0349734640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0344109880 0x7f03441a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:29.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.997+0000 7f0349734640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f034410f600 con 0x7f0344106f20 2024-04-25T14:11:29.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.997+0000 7f0349734640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f034410f480 con 0x7f0344109880 2024-04-25T14:11:30.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.997+0000 7f0349734640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f034410f780 con 0x7f03441045c0 2024-04-25T14:11:30.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.997+0000 7f03427fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0344106f20 0x7f034419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:30.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.997+0000 7f03427fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0344106f20 0x7f034419a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56442/0 (socket says 172.21.15.96:56442) 2024-04-25T14:11:30.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.997+0000 7f03427fc640 1 -- 172.21.15.96:0/3824087721 learned_addr learned my addr 172.21.15.96:0/3824087721 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:11:30.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.997+0000 7f0342ffd640 1 --2- 172.21.15.96:0/3824087721 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03441045c0 0x7f034419a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:30.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.997+0000 7f03437fe640 1 --2- 172.21.15.96:0/3824087721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0344109880 0x7f03441a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:30.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.997+0000 7f03427fc640 1 -- 172.21.15.96:0/3824087721 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03441045c0 msgr2=0x7f034419a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:30.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.997+0000 7f03427fc640 1 --2- 172.21.15.96:0/3824087721 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03441045c0 0x7f034419a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:30.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.997+0000 7f03427fc640 1 -- 172.21.15.96:0/3824087721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0344109880 msgr2=0x7f03441a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:30.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.997+0000 7f03427fc640 1 --2- 172.21.15.96:0/3824087721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0344109880 0x7f03441a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:30.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.997+0000 7f03427fc640 1 -- 172.21.15.96:0/3824087721 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03441a1ca0 con 0x7f0344106f20 2024-04-25T14:11:30.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.997+0000 7f03437fe640 1 --2- 172.21.15.96:0/3824087721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0344109880 0x7f03441a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:30.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.997+0000 7f03427fc640 1 --2- 172.21.15.96:0/3824087721 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0344106f20 0x7f034419a750 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f032c006010 tx=0x7f032c004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:30.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.998+0000 7f033bfff640 1 -- 172.21.15.96:0/3824087721 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032c03f030 con 0x7f0344106f20 2024-04-25T14:11:30.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.998+0000 7f0349734640 1 -- 172.21.15.96:0/3824087721 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03441a1f30 con 0x7f0344106f20 2024-04-25T14:11:30.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.998+0000 7f033bfff640 1 -- 172.21.15.96:0/3824087721 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f032c003cb0 con 0x7f0344106f20 2024-04-25T14:11:30.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.998+0000 7f033bfff640 1 -- 172.21.15.96:0/3824087721 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032c035070 con 0x7f0344106f20 2024-04-25T14:11:30.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.998+0000 7f0349734640 1 -- 172.21.15.96:0/3824087721 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03441a2390 con 0x7f0344106f20 2024-04-25T14:11:30.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:29.999+0000 7f0349734640 1 -- 172.21.15.96:0/3824087721 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03440ff6c0 con 0x7f0344106f20 2024-04-25T14:11:30.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.000+0000 7f033bfff640 1 -- 172.21.15.96:0/3824087721 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f032c042030 con 0x7f0344106f20 2024-04-25T14:11:30.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.000+0000 7f033bfff640 1 --2- 172.21.15.96:0/3824087721 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f03300afaf0 0x7f03300b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:30.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.001+0000 7f0342ffd640 1 --2- 172.21.15.96:0/3824087721 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f03300afaf0 0x7f03300b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:30.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.001+0000 7f033bfff640 1 -- 172.21.15.96:0/3824087721 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f032c0ff8b0 con 0x7f0344106f20 2024-04-25T14:11:30.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.001+0000 7f0342ffd640 1 --2- 172.21.15.96:0/3824087721 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f03300afaf0 0x7f03300b2000 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f034406bcf0 tx=0x7f0334009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:30.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.004+0000 7f033bfff640 1 -- 172.21.15.96:0/3824087721 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f032c0c85c0 con 0x7f0344106f20 2024-04-25T14:11:30.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.182+0000 7f0349734640 1 -- 172.21.15.96:0/3824087721 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f03440637a0 con 0x7f0344106f20 2024-04-25T14:11:30.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.184+0000 7f033bfff640 1 -- 172.21.15.96:0/3824087721 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+22159 (secure 0 0 0) 0x7f032c003e60 con 0x7f0344106f20 2024-04-25T14:11:30.184 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:11:30.184 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":75,"fsid":"59334182-030c-11ef-bc93-c7b262605968","created":"2024-04-25T14:03:25.680745+0000","modified":"2024-04-25T14:11:28.887003+0000","last_up_change":"2024-04-25T14:10:35.894312+0000","last_in_change":"2024-04-25T14:10:15.583440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:06:47.402927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:11:27.434331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:11:27.854004+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cf4ba3ac-142e-4e8e-96d4-410d77157eb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6803","nonce":3832506168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6805","nonce":3832506168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832506168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6807","nonce":3832506168}]},"public_addr":"172.21.15.96:6803/3832506168","cluster_addr":"172.21.15.96:6805/3832506168","heartbeat_back_addr":"172.21.15.96:6809/3832506168","heartbeat_front_addr":"172.21.15.96:6807/3832506168","state":["exists","up"]},{"osd":1,"uuid":"866e73b9-4a26-48ba-badf-704fe22f71cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6801","nonce":1083311494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6803","nonce":1083311494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6807","nonce":1083311494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6805","nonce":1083311494}]},"public_addr":"172.21.15.97:6801/1083311494","cluster_addr":"172.21.15.97:6803/1083311494","heartbeat_back_addr":"172.21.15.97:6807/1083311494","heartbeat_front_addr":"172.21.15.97:6805/1083311494","state":["exists","up"]},{"osd":2,"uuid":"45c79ecb-abc4-4dba-bae8-74804e4bc1bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6801","nonce":981902966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6803","nonce":981902966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6807","nonce":981902966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6805","nonce":981902966}]},"public_addr":"172.21.15.151:6801/981902966","cluster_addr":"172.21.15.151:6803/981902966","heartbeat_back_addr":"172.21.15.151:6807/981902966","heartbeat_front_addr":"172.21.15.151:6805/981902966","state":["exists","up"]},{"osd":3,"uuid":"9fc39468-a7ea-4ccf-b365-335ad2ea4c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6811","nonce":2274158748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6813","nonce":2274158748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6817","nonce":2274158748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6815","nonce":2274158748}]},"public_addr":"172.21.15.96:6811/2274158748","cluster_addr":"172.21.15.96:6813/2274158748","heartbeat_back_addr":"172.21.15.96:6817/2274158748","heartbeat_front_addr":"172.21.15.96:6815/2274158748","state":["exists","up"]},{"osd":4,"uuid":"616582f1-12d2-456b-b7b3-e5517800e8a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6809","nonce":3727031962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6811","nonce":3727031962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6815","nonce":3727031962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6813","nonce":3727031962}]},"public_addr":"172.21.15.97:6809/3727031962","cluster_addr":"172.21.15.97:6811/3727031962","heartbeat_back_addr":"172.21.15.97:6815/3727031962","heartbeat_front_addr":"172.21.15.97:6813/3727031962","state":["exists","up"]},{"osd":5,"uuid":"3d277784-ab0f-4c90-81a1-2a4034c52a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6809","nonce":223882546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6811","nonce":223882546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6815","nonce":223882546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6813","nonce":223882546}]},"public_addr":"172.21.15.151:6809/223882546","cluster_addr":"172.21.15.151:6811/223882546","heartbeat_back_addr":"172.21.15.151:6815/223882546","heartbeat_front_addr":"172.21.15.151:6813/223882546","state":["exists","up"]},{"osd":6,"uuid":"9a9ad585-cb1a-4239-9d18-055f59164049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6819","nonce":96197407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6821","nonce":96197407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6825","nonce":96197407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6823","nonce":96197407}]},"public_addr":"172.21.15.96:6819/96197407","cluster_addr":"172.21.15.96:6821/96197407","heartbeat_back_addr":"172.21.15.96:6825/96197407","heartbeat_front_addr":"172.21.15.96:6823/96197407","state":["exists","up"]},{"osd":7,"uuid":"303551c8-6af0-42f2-844f-45bf8db5baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6817","nonce":2247216400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6819","nonce":2247216400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6823","nonce":2247216400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6821","nonce":2247216400}]},"public_addr":"172.21.15.97:6817/2247216400","cluster_addr":"172.21.15.97:6819/2247216400","heartbeat_back_addr":"172.21.15.97:6823/2247216400","heartbeat_front_addr":"172.21.15.97:6821/2247216400","state":["exists","up"]},{"osd":8,"uuid":"81f6cc66-e4ca-430c-b336-b6678d968818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6817","nonce":2549822185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6819","nonce":2549822185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6823","nonce":2549822185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6821","nonce":2549822185}]},"public_addr":"172.21.15.151:6817/2549822185","cluster_addr":"172.21.15.151:6819/2549822185","heartbeat_back_addr":"172.21.15.151:6823/2549822185","heartbeat_front_addr":"172.21.15.151:6821/2549822185","state":["exists","up"]},{"osd":9,"uuid":"29d4da16-feab-4d1c-9d1d-de4f799bd724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6827","nonce":1099399643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6829","nonce":1099399643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6833","nonce":1099399643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6831","nonce":1099399643}]},"public_addr":"172.21.15.96:6827/1099399643","cluster_addr":"172.21.15.96:6829/1099399643","heartbeat_back_addr":"172.21.15.96:6833/1099399643","heartbeat_front_addr":"172.21.15.96:6831/1099399643","state":["exists","up"]},{"osd":10,"uuid":"197fee9b-eee4-4893-be1c-3d57f42250e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6825","nonce":155754520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6827","nonce":155754520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6831","nonce":155754520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6829","nonce":155754520}]},"public_addr":"172.21.15.97:6825/155754520","cluster_addr":"172.21.15.97:6827/155754520","heartbeat_back_addr":"172.21.15.97:6831/155754520","heartbeat_front_addr":"172.21.15.97:6829/155754520","state":["exists","up"]},{"osd":11,"uuid":"9c1d0a83-fc12-41a2-8eb0-80c4abdd2770","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6825","nonce":2027958746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6827","nonce":2027958746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6831","nonce":2027958746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6829","nonce":2027958746}]},"public_addr":"172.21.15.151:6825/2027958746","cluster_addr":"172.21.15.151:6827/2027958746","heartbeat_back_addr":"172.21.15.151:6831/2027958746","heartbeat_front_addr":"172.21.15.151:6829/2027958746","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:05:56.177390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:06:19.917663+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:06:43.608052+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:08.540382+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:33.474072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:58.242306+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:08:24.120442+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:08:49.661064+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:09:15.097271+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:09:41.904505+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:10:08.233252+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:10:33.712296+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/3839807484":"2024-04-26T14:04:21.307574+0000","172.21.15.96:0/1495237551":"2024-04-26T14:04:21.307574+0000","172.21.15.96:0/2793751450":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/129575834":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/2233075139":"2024-04-26T14:04:21.307574+0000","172.21.15.96:6801/2348397481":"2024-04-26T14:03:54.249537+0000","172.21.15.96:6801/433591062":"2024-04-26T14:04:21.307574+0000","172.21.15.96:6800/2348397481":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/3517674924":"2024-04-26T14:03:54.249537+0000","172.21.15.96:6800/433591062":"2024-04-26T14:04:21.307574+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:11:30.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.186+0000 7f0349734640 1 -- 172.21.15.96:0/3824087721 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f03300afaf0 msgr2=0x7f03300b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:30.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.186+0000 7f0349734640 1 --2- 172.21.15.96:0/3824087721 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f03300afaf0 0x7f03300b2000 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f034406bcf0 tx=0x7f0334009870 comp rx=0 tx=0).stop 2024-04-25T14:11:30.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.186+0000 7f0349734640 1 -- 172.21.15.96:0/3824087721 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0344106f20 msgr2=0x7f034419a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:30.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.186+0000 7f0349734640 1 --2- 172.21.15.96:0/3824087721 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0344106f20 0x7f034419a750 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f032c006010 tx=0x7f032c004340 comp rx=0 tx=0).stop 2024-04-25T14:11:30.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.186+0000 7f0349734640 1 -- 172.21.15.96:0/3824087721 shutdown_connections 2024-04-25T14:11:30.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.186+0000 7f0349734640 1 --2- 172.21.15.96:0/3824087721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0344109880 0x7f03441a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:30.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.186+0000 7f0349734640 1 --2- 172.21.15.96:0/3824087721 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f03300afaf0 0x7f03300b2000 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:30.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.186+0000 7f0349734640 1 --2- 172.21.15.96:0/3824087721 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0344106f20 0x7f034419a750 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:30.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.186+0000 7f0349734640 1 --2- 172.21.15.96:0/3824087721 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03441045c0 0x7f034419a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:30.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.186+0000 7f0349734640 1 -- 172.21.15.96:0/3824087721 >> 172.21.15.96:0/3824087721 conn(0x7f03440fdba0 msgr2=0x7f0344105250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:30.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.186+0000 7f0349734640 1 -- 172.21.15.96:0/3824087721 shutdown_connections 2024-04-25T14:11:30.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.186+0000 7f0349734640 1 -- 172.21.15.96:0/3824087721 wait complete. 2024-04-25T14:11:30.197 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T14:11:30.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.307+0000 7f456ce51640 1 -- 172.21.15.96:0/4086512237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4568103af0 msgr2=0x7f45681047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:30.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.307+0000 7f456ce51640 1 --2- 172.21.15.96:0/4086512237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4568103af0 0x7f45681047f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f455c009900 tx=0x7f455c02a410 comp rx=0 tx=0).stop 2024-04-25T14:11:30.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.307+0000 7f456ce51640 1 -- 172.21.15.96:0/4086512237 shutdown_connections 2024-04-25T14:11:30.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.307+0000 7f456ce51640 1 --2- 172.21.15.96:0/4086512237 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4568109a30 0x7f456810be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:30.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.307+0000 7f456ce51640 1 --2- 172.21.15.96:0/4086512237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4568104d30 0x7f4568105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:30.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.307+0000 7f456ce51640 1 --2- 172.21.15.96:0/4086512237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4568103af0 0x7f45681047f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:30.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.307+0000 7f456ce51640 1 -- 172.21.15.96:0/4086512237 >> 172.21.15.96:0/4086512237 conn(0x7f45680fece0 msgr2=0x7f4568101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:30.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.307+0000 7f456ce51640 1 -- 172.21.15.96:0/4086512237 shutdown_connections 2024-04-25T14:11:30.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.307+0000 7f456ce51640 1 -- 172.21.15.96:0/4086512237 wait complete. 2024-04-25T14:11:30.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.308+0000 7f456ce51640 1 Processor -- start 2024-04-25T14:11:30.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.308+0000 7f456ce51640 1 -- start start 2024-04-25T14:11:30.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.308+0000 7f456ce51640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4568103af0 0x7f456819a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:30.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.308+0000 7f456ce51640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4568104d30 0x7f456819a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:30.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.308+0000 7f456ce51640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4568109a30 0x7f45681a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:30.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.308+0000 7f456ce51640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f456810f5f0 con 0x7f4568104d30 2024-04-25T14:11:30.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.308+0000 7f456ce51640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f456810f470 con 0x7f4568109a30 2024-04-25T14:11:30.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.308+0000 7f456ce51640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f456810f770 con 0x7f4568103af0 2024-04-25T14:11:30.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.308+0000 7f4565d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4568104d30 0x7f456819a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:30.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.308+0000 7f4565d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4568104d30 0x7f456819a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56446/0 (socket says 172.21.15.96:56446) 2024-04-25T14:11:30.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.308+0000 7f4565d9b640 1 -- 172.21.15.96:0/2487325412 learned_addr learned my addr 172.21.15.96:0/2487325412 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:11:30.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.308+0000 7f4566d9d640 1 --2- 172.21.15.96:0/2487325412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4568109a30 0x7f45681a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:30.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.308+0000 7f4565d9b640 1 -- 172.21.15.96:0/2487325412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4568103af0 msgr2=0x7f456819a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:30.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.308+0000 7f456659c640 1 --2- 172.21.15.96:0/2487325412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4568103af0 0x7f456819a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:30.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.308+0000 7f4565d9b640 1 --2- 172.21.15.96:0/2487325412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4568103af0 0x7f456819a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:30.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.308+0000 7f4565d9b640 1 -- 172.21.15.96:0/2487325412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4568109a30 msgr2=0x7f45681a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:30.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.308+0000 7f4565d9b640 1 --2- 172.21.15.96:0/2487325412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4568109a30 0x7f45681a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:30.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.309+0000 7f4565d9b640 1 -- 172.21.15.96:0/2487325412 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45681a1d90 con 0x7f4568104d30 2024-04-25T14:11:30.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.309+0000 7f4566d9d640 1 --2- 172.21.15.96:0/2487325412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4568109a30 0x7f45681a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:30.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.309+0000 7f4565d9b640 1 --2- 172.21.15.96:0/2487325412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4568104d30 0x7f456819a790 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f455800d950 tx=0x7f455800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:30.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.309+0000 7f45577fe640 1 -- 172.21.15.96:0/2487325412 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4558017070 con 0x7f4568104d30 2024-04-25T14:11:30.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.309+0000 7f45577fe640 1 -- 172.21.15.96:0/2487325412 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4558009c30 con 0x7f4568104d30 2024-04-25T14:11:30.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.309+0000 7f456ce51640 1 -- 172.21.15.96:0/2487325412 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45681a2080 con 0x7f4568104d30 2024-04-25T14:11:30.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.309+0000 7f45577fe640 1 -- 172.21.15.96:0/2487325412 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455800f620 con 0x7f4568104d30 2024-04-25T14:11:30.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.309+0000 7f456ce51640 1 -- 172.21.15.96:0/2487325412 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45681a25d0 con 0x7f4568104d30 2024-04-25T14:11:30.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.310+0000 7f456ce51640 1 -- 172.21.15.96:0/2487325412 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4568108840 con 0x7f4568104d30 2024-04-25T14:11:30.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.311+0000 7f45577fe640 1 -- 172.21.15.96:0/2487325412 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f4558010460 con 0x7f4568104d30 2024-04-25T14:11:30.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.312+0000 7f45577fe640 1 --2- 172.21.15.96:0/2487325412 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f45500afaf0 0x7f45500b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:30.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.312+0000 7f456659c640 1 --2- 172.21.15.96:0/2487325412 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f45500afaf0 0x7f45500b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:30.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.312+0000 7f45577fe640 1 -- 172.21.15.96:0/2487325412 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f45580de760 con 0x7f4568104d30 2024-04-25T14:11:30.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.312+0000 7f456659c640 1 --2- 172.21.15.96:0/2487325412 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f45500afaf0 0x7f45500b2000 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f455c02a930 tx=0x7f455c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:30.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.316+0000 7f45577fe640 1 -- 172.21.15.96:0/2487325412 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f45580a7490 con 0x7f4568104d30 2024-04-25T14:11:30.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.532+0000 7f456ce51640 1 -- 172.21.15.96:0/2487325412 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f456802db50 con 0x7f4568104d30 2024-04-25T14:11:30.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.552+0000 7f45577fe640 1 -- 172.21.15.96:0/2487325412 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7f4558009de0 con 0x7f4568104d30 2024-04-25T14:11:30.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.554+0000 7f456ce51640 1 -- 172.21.15.96:0/2487325412 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f45500afaf0 msgr2=0x7f45500b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:30.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.554+0000 7f456ce51640 1 --2- 172.21.15.96:0/2487325412 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f45500afaf0 0x7f45500b2000 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f455c02a930 tx=0x7f455c005fd0 comp rx=0 tx=0).stop 2024-04-25T14:11:30.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.554+0000 7f456ce51640 1 -- 172.21.15.96:0/2487325412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4568104d30 msgr2=0x7f456819a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:30.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.554+0000 7f456ce51640 1 --2- 172.21.15.96:0/2487325412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4568104d30 0x7f456819a790 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f455800d950 tx=0x7f455800de10 comp rx=0 tx=0).stop 2024-04-25T14:11:30.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.554+0000 7f456ce51640 1 -- 172.21.15.96:0/2487325412 shutdown_connections 2024-04-25T14:11:30.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.554+0000 7f456ce51640 1 --2- 172.21.15.96:0/2487325412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4568109a30 0x7f45681a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:30.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.554+0000 7f456ce51640 1 --2- 172.21.15.96:0/2487325412 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f45500afaf0 0x7f45500b2000 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:30.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.554+0000 7f456ce51640 1 --2- 172.21.15.96:0/2487325412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4568104d30 0x7f456819a790 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:30.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.554+0000 7f456ce51640 1 --2- 172.21.15.96:0/2487325412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4568103af0 0x7f456819a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:30.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.554+0000 7f456ce51640 1 -- 172.21.15.96:0/2487325412 >> 172.21.15.96:0/2487325412 conn(0x7f45680fece0 msgr2=0x7f4568105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:30.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.554+0000 7f456ce51640 1 -- 172.21.15.96:0/2487325412 shutdown_connections 2024-04-25T14:11:30.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.554+0000 7f456ce51640 1 -- 172.21.15.96:0/2487325412 wait complete. 2024-04-25T14:11:30.567 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-25T14:11:30.567 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-04-25T14:11:30.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.681+0000 7fb5815ce640 1 -- 172.21.15.96:0/3583561220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb57c101740 msgr2=0x7fb57c101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:30.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.681+0000 7fb5815ce640 1 --2- 172.21.15.96:0/3583561220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb57c101740 0x7fb57c101bb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb570009900 tx=0x7fb57002a410 comp rx=0 tx=0).stop 2024-04-25T14:11:30.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.681+0000 7fb5815ce640 1 -- 172.21.15.96:0/3583561220 shutdown_connections 2024-04-25T14:11:30.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.681+0000 7fb5815ce640 1 --2- 172.21.15.96:0/3583561220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb57c102240 0x7fb57c10bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:30.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.681+0000 7fb5815ce640 1 --2- 172.21.15.96:0/3583561220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb57c101740 0x7fb57c101bb0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:30.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.681+0000 7fb5815ce640 1 --2- 172.21.15.96:0/3583561220 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb57c105090 0x7fb57c1054a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:30.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.681+0000 7fb5815ce640 1 -- 172.21.15.96:0/3583561220 >> 172.21.15.96:0/3583561220 conn(0x7fb57c0fd340 msgr2=0x7fb57c0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:30.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.681+0000 7fb5815ce640 1 -- 172.21.15.96:0/3583561220 shutdown_connections 2024-04-25T14:11:30.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.682+0000 7fb5815ce640 1 -- 172.21.15.96:0/3583561220 wait complete. 2024-04-25T14:11:30.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.682+0000 7fb5815ce640 1 Processor -- start 2024-04-25T14:11:30.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.682+0000 7fb5815ce640 1 -- start start 2024-04-25T14:11:30.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.682+0000 7fb5815ce640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb57c101740 0x7fb57c19a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:30.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.682+0000 7fb5815ce640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb57c102240 0x7fb57c19a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:30.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.682+0000 7fb5815ce640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb57c105090 0x7fb57c1a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:30.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.683+0000 7fb5815ce640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c10f670 con 0x7fb57c102240 2024-04-25T14:11:30.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.683+0000 7fb5815ce640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c10f4f0 con 0x7fb57c101740 2024-04-25T14:11:30.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.683+0000 7fb5815ce640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c10f7f0 con 0x7fb57c105090 2024-04-25T14:11:30.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.683+0000 7fb57a7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb57c102240 0x7fb57c19a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:30.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.683+0000 7fb57a7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb57c102240 0x7fb57c19a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56470/0 (socket says 172.21.15.96:56470) 2024-04-25T14:11:30.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.683+0000 7fb57a7fc640 1 -- 172.21.15.96:0/3576101096 learned_addr learned my addr 172.21.15.96:0/3576101096 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:11:30.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.683+0000 7fb57affd640 1 --2- 172.21.15.96:0/3576101096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb57c101740 0x7fb57c19a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:30.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.683+0000 7fb57b7fe640 1 --2- 172.21.15.96:0/3576101096 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb57c105090 0x7fb57c1a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:30.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.683+0000 7fb57a7fc640 1 -- 172.21.15.96:0/3576101096 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb57c105090 msgr2=0x7fb57c1a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:30.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.683+0000 7fb57a7fc640 1 --2- 172.21.15.96:0/3576101096 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb57c105090 0x7fb57c1a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:30.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.683+0000 7fb57a7fc640 1 -- 172.21.15.96:0/3576101096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb57c101740 msgr2=0x7fb57c19a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:30.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.683+0000 7fb57a7fc640 1 --2- 172.21.15.96:0/3576101096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb57c101740 0x7fb57c19a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:30.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.683+0000 7fb57a7fc640 1 -- 172.21.15.96:0/3576101096 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb57c1a1df0 con 0x7fb57c102240 2024-04-25T14:11:30.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.683+0000 7fb57a7fc640 1 --2- 172.21.15.96:0/3576101096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb57c102240 0x7fb57c19a7f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb570006010 tx=0x7fb570004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:30.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.683+0000 7fb563fff640 1 -- 172.21.15.96:0/3576101096 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57003f030 con 0x7fb57c102240 2024-04-25T14:11:30.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.683+0000 7fb563fff640 1 -- 172.21.15.96:0/3576101096 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb570003cb0 con 0x7fb57c102240 2024-04-25T14:11:30.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.683+0000 7fb5815ce640 1 -- 172.21.15.96:0/3576101096 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb57c1a2080 con 0x7fb57c102240 2024-04-25T14:11:30.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.683+0000 7fb563fff640 1 -- 172.21.15.96:0/3576101096 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb570035070 con 0x7fb57c102240 2024-04-25T14:11:30.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.683+0000 7fb5815ce640 1 -- 172.21.15.96:0/3576101096 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb57c1a25f0 con 0x7fb57c102240 2024-04-25T14:11:30.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.685+0000 7fb5815ce640 1 -- 172.21.15.96:0/3576101096 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb57c108a20 con 0x7fb57c102240 2024-04-25T14:11:30.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.686+0000 7fb563fff640 1 -- 172.21.15.96:0/3576101096 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fb570042030 con 0x7fb57c102240 2024-04-25T14:11:30.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.686+0000 7fb563fff640 1 --2- 172.21.15.96:0/3576101096 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb5640afaf0 0x7fb5640b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:30.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.687+0000 7fb57affd640 1 --2- 172.21.15.96:0/3576101096 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb5640afaf0 0x7fb5640b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:30.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.687+0000 7fb563fff640 1 -- 172.21.15.96:0/3576101096 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7fb5700ff080 con 0x7fb57c102240 2024-04-25T14:11:30.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.687+0000 7fb57affd640 1 --2- 172.21.15.96:0/3576101096 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb5640afaf0 0x7fb5640b2000 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb57c069bf0 tx=0x7fb56c009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:30.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.691+0000 7fb563fff640 1 -- 172.21.15.96:0/3576101096 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb5700c7db0 con 0x7fb57c102240 2024-04-25T14:11:30.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:30 smithi096 ceph-mon[29207]: pgmap v278: 97 pgs: 15 creating+peering, 1 active+clean, 81 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:30.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:30 smithi096 ceph-mon[29207]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T14:11:30.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:30 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/1876131909' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T14:11:30.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:30 smithi096 ceph-mon[29207]: osdmap e75: 12 total, 12 up, 12 in 2024-04-25T14:11:30.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:30 smithi096 ceph-mon[29207]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T14:11:30.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:30 smithi096 ceph-mon[29207]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T14:11:30.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:30 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/1956521206' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:11:30.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:30 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/3824087721' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:11:30.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:30 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/2487325412' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T14:11:30.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:30.914+0000 7fb5815ce640 1 -- 172.21.15.96:0/3576101096 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7fb57c0637a0 con 0x7fb57c102240 2024-04-25T14:11:30.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:30 smithi097 ceph-mon[32720]: pgmap v278: 97 pgs: 15 creating+peering, 1 active+clean, 81 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:30.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:30 smithi097 ceph-mon[32720]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T14:11:30.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:30 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/1876131909' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T14:11:30.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:30 smithi097 ceph-mon[32720]: osdmap e75: 12 total, 12 up, 12 in 2024-04-25T14:11:30.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:30 smithi097 ceph-mon[32720]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T14:11:30.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:30 smithi097 ceph-mon[32720]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T14:11:30.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:30 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/1956521206' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:11:30.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:30 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/3824087721' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:11:30.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:30 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/2487325412' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T14:11:30.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:30 smithi151 ceph-mon[32990]: pgmap v278: 97 pgs: 15 creating+peering, 1 active+clean, 81 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:11:30.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:30 smithi151 ceph-mon[32990]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T14:11:30.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:30 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/1876131909' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T14:11:30.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:30 smithi151 ceph-mon[32990]: osdmap e75: 12 total, 12 up, 12 in 2024-04-25T14:11:30.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:30 smithi151 ceph-mon[32990]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T14:11:30.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:30 smithi151 ceph-mon[32990]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T14:11:30.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:30 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/1956521206' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:11:30.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:30 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/3824087721' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:11:30.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:30 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/2487325412' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T14:11:31.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.565+0000 7fb563fff640 1 -- 172.21.15.96:0/3576101096 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v16) ==== 105+0+0 (secure 0 0 0) 0x7fb5700c76b0 con 0x7fb57c102240 2024-04-25T14:11:31.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.567+0000 7fb5815ce640 1 -- 172.21.15.96:0/3576101096 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb5640afaf0 msgr2=0x7fb5640b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:31.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.567+0000 7fb5815ce640 1 --2- 172.21.15.96:0/3576101096 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb5640afaf0 0x7fb5640b2000 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb57c069bf0 tx=0x7fb56c009110 comp rx=0 tx=0).stop 2024-04-25T14:11:31.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.568+0000 7fb5815ce640 1 -- 172.21.15.96:0/3576101096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb57c102240 msgr2=0x7fb57c19a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:31.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.568+0000 7fb5815ce640 1 --2- 172.21.15.96:0/3576101096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb57c102240 0x7fb57c19a7f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb570006010 tx=0x7fb570004340 comp rx=0 tx=0).stop 2024-04-25T14:11:31.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.568+0000 7fb5815ce640 1 -- 172.21.15.96:0/3576101096 shutdown_connections 2024-04-25T14:11:31.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.568+0000 7fb5815ce640 1 --2- 172.21.15.96:0/3576101096 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb57c105090 0x7fb57c1a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:31.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.568+0000 7fb5815ce640 1 --2- 172.21.15.96:0/3576101096 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb5640afaf0 0x7fb5640b2000 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:31.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.568+0000 7fb5815ce640 1 --2- 172.21.15.96:0/3576101096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb57c102240 0x7fb57c19a7f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:31.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.568+0000 7fb5815ce640 1 --2- 172.21.15.96:0/3576101096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb57c101740 0x7fb57c19a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:31.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.568+0000 7fb5815ce640 1 -- 172.21.15.96:0/3576101096 >> 172.21.15.96:0/3576101096 conn(0x7fb57c0fd340 msgr2=0x7fb57c105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:31.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.568+0000 7fb5815ce640 1 -- 172.21.15.96:0/3576101096 shutdown_connections 2024-04-25T14:11:31.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.568+0000 7fb5815ce640 1 -- 172.21.15.96:0/3576101096 wait complete. 2024-04-25T14:11:31.582 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-25T14:11:31.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.693+0000 7f5cb2dee640 1 -- 172.21.15.96:0/921344847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5cac103af0 msgr2=0x7f5cac1047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:31.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.693+0000 7f5cb2dee640 1 --2- 172.21.15.96:0/921344847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5cac103af0 0x7f5cac1047f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c009900 tx=0x7f5c9c02a410 comp rx=0 tx=0).stop 2024-04-25T14:11:31.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.693+0000 7f5cb2dee640 1 -- 172.21.15.96:0/921344847 shutdown_connections 2024-04-25T14:11:31.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.693+0000 7f5cb2dee640 1 --2- 172.21.15.96:0/921344847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5cac109a30 0x7f5cac10be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:31.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.693+0000 7f5cb2dee640 1 --2- 172.21.15.96:0/921344847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cac104d30 0x7f5cac105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:31.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.693+0000 7f5cb2dee640 1 --2- 172.21.15.96:0/921344847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5cac103af0 0x7f5cac1047f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:31.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.693+0000 7f5cb2dee640 1 -- 172.21.15.96:0/921344847 >> 172.21.15.96:0/921344847 conn(0x7f5cac0fece0 msgr2=0x7f5cac101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:31.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.693+0000 7f5cb2dee640 1 -- 172.21.15.96:0/921344847 shutdown_connections 2024-04-25T14:11:31.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.694+0000 7f5cb2dee640 1 -- 172.21.15.96:0/921344847 wait complete. 2024-04-25T14:11:31.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.694+0000 7f5cb2dee640 1 Processor -- start 2024-04-25T14:11:31.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.694+0000 7f5cb2dee640 1 -- start start 2024-04-25T14:11:31.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.694+0000 7f5cb2dee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5cac103af0 0x7f5cac19a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:31.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.694+0000 7f5cb2dee640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5cac104d30 0x7f5cac19a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:31.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.694+0000 7f5cb2dee640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cac109a30 0x7f5cac1a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:31.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.694+0000 7f5cb2dee640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5cac10f5d0 con 0x7f5cac103af0 2024-04-25T14:11:31.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.694+0000 7f5cb2dee640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5cac10f450 con 0x7f5cac109a30 2024-04-25T14:11:31.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.694+0000 7f5cb2dee640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5cac10f750 con 0x7f5cac104d30 2024-04-25T14:11:31.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.694+0000 7f5cb0b8a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5cac103af0 0x7f5cac19a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:31.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.694+0000 7f5cb0b8a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5cac103af0 0x7f5cac19a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35228/0 (socket says 172.21.15.96:35228) 2024-04-25T14:11:31.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.694+0000 7f5cb0b8a640 1 -- 172.21.15.96:0/1381957929 learned_addr learned my addr 172.21.15.96:0/1381957929 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:11:31.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.695+0000 7f5cabfff640 1 --2- 172.21.15.96:0/1381957929 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5cac104d30 0x7f5cac19a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:31.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.695+0000 7f5cb138b640 1 --2- 172.21.15.96:0/1381957929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cac109a30 0x7f5cac1a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:31.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.695+0000 7f5cb0b8a640 1 -- 172.21.15.96:0/1381957929 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5cac104d30 msgr2=0x7f5cac19a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:31.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.695+0000 7f5cb0b8a640 1 --2- 172.21.15.96:0/1381957929 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5cac104d30 0x7f5cac19a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:31.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.695+0000 7f5cb0b8a640 1 -- 172.21.15.96:0/1381957929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cac109a30 msgr2=0x7f5cac1a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:31.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.695+0000 7f5cb0b8a640 1 --2- 172.21.15.96:0/1381957929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cac109a30 0x7f5cac1a1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:31.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.695+0000 7f5cb0b8a640 1 -- 172.21.15.96:0/1381957929 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cac1a1d80 con 0x7f5cac103af0 2024-04-25T14:11:31.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.695+0000 7f5cb0b8a640 1 --2- 172.21.15.96:0/1381957929 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5cac103af0 0x7f5cac19a290 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c006010 tx=0x7f5c9c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:31.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.695+0000 7f5ca9ffb640 1 -- 172.21.15.96:0/1381957929 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c03f030 con 0x7f5cac103af0 2024-04-25T14:11:31.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.695+0000 7f5ca9ffb640 1 -- 172.21.15.96:0/1381957929 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c9c003aa0 con 0x7f5cac103af0 2024-04-25T14:11:31.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.695+0000 7f5ca9ffb640 1 -- 172.21.15.96:0/1381957929 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c035070 con 0x7f5cac103af0 2024-04-25T14:11:31.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.695+0000 7f5cb2dee640 1 -- 172.21.15.96:0/1381957929 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cac1a2010 con 0x7f5cac103af0 2024-04-25T14:11:31.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.695+0000 7f5cb2dee640 1 -- 172.21.15.96:0/1381957929 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cac1a2500 con 0x7f5cac103af0 2024-04-25T14:11:31.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.696+0000 7f5cb2dee640 1 -- 172.21.15.96:0/1381957929 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cac108840 con 0x7f5cac103af0 2024-04-25T14:11:31.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.698+0000 7f5ca9ffb640 1 -- 172.21.15.96:0/1381957929 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f5c9c042030 con 0x7f5cac103af0 2024-04-25T14:11:31.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.698+0000 7f5ca9ffb640 1 --2- 172.21.15.96:0/1381957929 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f5c940afaf0 0x7f5c940b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:31.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.698+0000 7f5cabfff640 1 --2- 172.21.15.96:0/1381957929 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f5c940afaf0 0x7f5c940b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:31.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.698+0000 7f5ca9ffb640 1 -- 172.21.15.96:0/1381957929 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7f5c9c0ff630 con 0x7f5cac103af0 2024-04-25T14:11:31.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.699+0000 7f5cabfff640 1 --2- 172.21.15.96:0/1381957929 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f5c940afaf0 0x7f5c940b2000 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5ca0005fd0 tx=0x7f5ca0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:31.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.702+0000 7f5ca9ffb640 1 -- 172.21.15.96:0/1381957929 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5c9c0c82c0 con 0x7f5cac103af0 2024-04-25T14:11:31.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:31 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/2487325412' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T14:11:31.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:31 smithi096 ceph-mon[29207]: mds.? [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] up:active 2024-04-25T14:11:31.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:31 smithi096 ceph-mon[29207]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T14:11:31.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:31 smithi096 ceph-mon[29207]: osdmap e76: 12 total, 12 up, 12 in 2024-04-25T14:11:31.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:31 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/3576101096' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-25T14:11:31.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:31.920+0000 7f5cb2dee640 1 -- 172.21.15.96:0/1381957929 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f5cac02db50 con 0x7f5cac103af0 2024-04-25T14:11:31.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:31 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/2487325412' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T14:11:31.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:31 smithi097 ceph-mon[32720]: mds.? [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] up:active 2024-04-25T14:11:31.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:31 smithi097 ceph-mon[32720]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T14:11:31.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:31 smithi097 ceph-mon[32720]: osdmap e76: 12 total, 12 up, 12 in 2024-04-25T14:11:31.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:31 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/3576101096' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-25T14:11:31.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:31 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/2487325412' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T14:11:31.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:31 smithi151 ceph-mon[32990]: mds.? [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] up:active 2024-04-25T14:11:31.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:31 smithi151 ceph-mon[32990]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T14:11:31.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:31 smithi151 ceph-mon[32990]: osdmap e76: 12 total, 12 up, 12 in 2024-04-25T14:11:31.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:31 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/3576101096' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-25T14:11:32.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.505+0000 7f5ca9ffb640 1 -- 172.21.15.96:0/1381957929 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v18) ==== 121+0+0 (secure 0 0 0) 0x7f5c9c0d0340 con 0x7f5cac103af0 2024-04-25T14:11:32.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.507+0000 7f5cb2dee640 1 -- 172.21.15.96:0/1381957929 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f5c940afaf0 msgr2=0x7f5c940b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:32.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.507+0000 7f5cb2dee640 1 --2- 172.21.15.96:0/1381957929 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f5c940afaf0 0x7f5c940b2000 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5ca0005fd0 tx=0x7f5ca0005850 comp rx=0 tx=0).stop 2024-04-25T14:11:32.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.507+0000 7f5cb2dee640 1 -- 172.21.15.96:0/1381957929 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5cac103af0 msgr2=0x7f5cac19a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:32.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.507+0000 7f5cb2dee640 1 --2- 172.21.15.96:0/1381957929 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5cac103af0 0x7f5cac19a290 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c006010 tx=0x7f5c9c031a50 comp rx=0 tx=0).stop 2024-04-25T14:11:32.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.507+0000 7f5cb2dee640 1 -- 172.21.15.96:0/1381957929 shutdown_connections 2024-04-25T14:11:32.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.507+0000 7f5cb2dee640 1 --2- 172.21.15.96:0/1381957929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cac109a30 0x7f5cac1a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:32.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.507+0000 7f5cb2dee640 1 --2- 172.21.15.96:0/1381957929 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5cac104d30 0x7f5cac19a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:32.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.507+0000 7f5cb2dee640 1 --2- 172.21.15.96:0/1381957929 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f5c940afaf0 0x7f5c940b2000 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:32.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.507+0000 7f5cb2dee640 1 --2- 172.21.15.96:0/1381957929 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5cac103af0 0x7f5cac19a290 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:32.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.507+0000 7f5cb2dee640 1 -- 172.21.15.96:0/1381957929 >> 172.21.15.96:0/1381957929 conn(0x7f5cac0fece0 msgr2=0x7f5cac105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:32.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.508+0000 7f5cb2dee640 1 -- 172.21.15.96:0/1381957929 shutdown_connections 2024-04-25T14:11:32.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.508+0000 7f5cb2dee640 1 -- 172.21.15.96:0/1381957929 wait complete. 2024-04-25T14:11:32.525 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-25T14:11:32.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.639+0000 7fd9cef34640 1 -- 172.21.15.96:0/1999398087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9c8109a40 msgr2=0x7fd9c810be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:32.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.639+0000 7fd9cef34640 1 --2- 172.21.15.96:0/1999398087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9c8109a40 0x7fd9c810be60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0009ca0 tx=0x7fd9c002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:11:32.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.639+0000 7fd9cef34640 1 -- 172.21.15.96:0/1999398087 shutdown_connections 2024-04-25T14:11:32.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.639+0000 7fd9cef34640 1 --2- 172.21.15.96:0/1999398087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9c8109a40 0x7fd9c810be60 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:32.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.639+0000 7fd9cef34640 1 --2- 172.21.15.96:0/1999398087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9c8104d40 0x7fd9c8105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:32.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.639+0000 7fd9cef34640 1 --2- 172.21.15.96:0/1999398087 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9c8103b00 0x7fd9c8104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:32.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.639+0000 7fd9cef34640 1 -- 172.21.15.96:0/1999398087 >> 172.21.15.96:0/1999398087 conn(0x7fd9c80fecf0 msgr2=0x7fd9c81011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:32.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.639+0000 7fd9cef34640 1 -- 172.21.15.96:0/1999398087 shutdown_connections 2024-04-25T14:11:32.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.639+0000 7fd9cef34640 1 -- 172.21.15.96:0/1999398087 wait complete. 2024-04-25T14:11:32.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.639+0000 7fd9cef34640 1 Processor -- start 2024-04-25T14:11:32.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.639+0000 7fd9cef34640 1 -- start start 2024-04-25T14:11:32.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.640+0000 7fd9cef34640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9c8103b00 0x7fd9c819a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:32.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.640+0000 7fd9cef34640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9c8104d40 0x7fd9c819a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:32.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.640+0000 7fd9cef34640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9c8109a40 0x7fd9c81a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:32.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.640+0000 7fd9cef34640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c810f630 con 0x7fd9c8104d40 2024-04-25T14:11:32.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.640+0000 7fd9cef34640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c810f4b0 con 0x7fd9c8109a40 2024-04-25T14:11:32.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.640+0000 7fd9cef34640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c810f7b0 con 0x7fd9c8103b00 2024-04-25T14:11:32.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.640+0000 7fd9c7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9c8104d40 0x7fd9c819a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:32.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.640+0000 7fd9c7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9c8104d40 0x7fd9c819a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35250/0 (socket says 172.21.15.96:35250) 2024-04-25T14:11:32.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.640+0000 7fd9c7fff640 1 -- 172.21.15.96:0/836849141 learned_addr learned my addr 172.21.15.96:0/836849141 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:11:32.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.640+0000 7fd9cd4d1640 1 --2- 172.21.15.96:0/836849141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9c8109a40 0x7fd9c81a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:32.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.640+0000 7fd9cccd0640 1 --2- 172.21.15.96:0/836849141 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9c8103b00 0x7fd9c819a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:32.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.640+0000 7fd9c7fff640 1 -- 172.21.15.96:0/836849141 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9c8103b00 msgr2=0x7fd9c819a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:32.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.640+0000 7fd9c7fff640 1 --2- 172.21.15.96:0/836849141 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9c8103b00 0x7fd9c819a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:32.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.640+0000 7fd9c7fff640 1 -- 172.21.15.96:0/836849141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9c8109a40 msgr2=0x7fd9c81a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:32.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.640+0000 7fd9c7fff640 1 --2- 172.21.15.96:0/836849141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9c8109a40 0x7fd9c81a16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:32.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.640+0000 7fd9c7fff640 1 -- 172.21.15.96:0/836849141 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9c81a1d10 con 0x7fd9c8104d40 2024-04-25T14:11:32.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.640+0000 7fd9cd4d1640 1 --2- 172.21.15.96:0/836849141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9c8109a40 0x7fd9c81a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:32.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.640+0000 7fd9cccd0640 1 --2- 172.21.15.96:0/836849141 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9c8103b00 0x7fd9c819a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:32.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.640+0000 7fd9c7fff640 1 --2- 172.21.15.96:0/836849141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9c8104d40 0x7fd9c819a710 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd9b400e9f0 tx=0x7fd9b400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:32.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.640+0000 7fd9c5ffb640 1 -- 172.21.15.96:0/836849141 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b400cd80 con 0x7fd9c8104d40 2024-04-25T14:11:32.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.640+0000 7fd9c5ffb640 1 -- 172.21.15.96:0/836849141 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9b40108e0 con 0x7fd9c8104d40 2024-04-25T14:11:32.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.640+0000 7fd9c5ffb640 1 -- 172.21.15.96:0/836849141 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4018980 con 0x7fd9c8104d40 2024-04-25T14:11:32.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.640+0000 7fd9cef34640 1 -- 172.21.15.96:0/836849141 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9c81a1fa0 con 0x7fd9c8104d40 2024-04-25T14:11:32.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.641+0000 7fd9cef34640 1 -- 172.21.15.96:0/836849141 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9c81a2490 con 0x7fd9c8104d40 2024-04-25T14:11:32.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.641+0000 7fd9cef34640 1 -- 172.21.15.96:0/836849141 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9c8108850 con 0x7fd9c8104d40 2024-04-25T14:11:32.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.643+0000 7fd9c5ffb640 1 -- 172.21.15.96:0/836849141 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fd9b4010430 con 0x7fd9c8104d40 2024-04-25T14:11:32.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.643+0000 7fd9c5ffb640 1 --2- 172.21.15.96:0/836849141 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd9b80afaf0 0x7fd9b80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:32.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.644+0000 7fd9cccd0640 1 --2- 172.21.15.96:0/836849141 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd9b80afaf0 0x7fd9b80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:32.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.644+0000 7fd9c5ffb640 1 -- 172.21.15.96:0/836849141 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7fd9b40df3e0 con 0x7fd9c8104d40 2024-04-25T14:11:32.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.644+0000 7fd9cccd0640 1 --2- 172.21.15.96:0/836849141 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd9b80afaf0 0x7fd9b80b2000 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc0058c0 tx=0x7fd9bc005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:32.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.647+0000 7fd9c5ffb640 1 -- 172.21.15.96:0/836849141 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd9b40a80f0 con 0x7fd9c8104d40 2024-04-25T14:11:32.824 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:32 smithi097 ceph-mon[32720]: pgmap v281: 97 pgs: 17 creating+peering, 43 active+clean, 37 unknown; 578 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 4 op/s 2024-04-25T14:11:32.824 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:32 smithi097 ceph-mon[32720]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T14:11:32.824 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:32 smithi097 ceph-mon[32720]: Cluster is now healthy 2024-04-25T14:11:32.824 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:32 smithi097 ceph-mon[32720]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T14:11:32.824 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:32 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/3576101096' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-25T14:11:32.824 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:32 smithi097 ceph-mon[32720]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T14:11:32.824 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:32 smithi097 ceph-mon[32720]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T14:11:32.825 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:32 smithi097 ceph-mon[32720]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-25T14:11:32.825 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:32 smithi097 ceph-mon[32720]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-25T14:11:32.825 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:32 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/1381957929' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T14:11:32.825 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:32 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/1381957929' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T14:11:32.825 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:32 smithi097 ceph-mon[32720]: mds.? [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] up:active 2024-04-25T14:11:32.825 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:32 smithi097 ceph-mon[32720]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T14:11:32.825 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:32 smithi097 ceph-mon[32720]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2024-04-25T14:11:32.825 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:32 smithi097 ceph-mon[32720]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:creating} 2 up:standby-replay 7 up:standby 2024-04-25T14:11:32.825 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:32 smithi097 ceph-mon[32720]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-04-25T14:11:32.839 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:32 smithi151 ceph-mon[32990]: pgmap v281: 97 pgs: 17 creating+peering, 43 active+clean, 37 unknown; 578 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 4 op/s 2024-04-25T14:11:32.839 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:32 smithi151 ceph-mon[32990]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T14:11:32.840 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:32 smithi151 ceph-mon[32990]: Cluster is now healthy 2024-04-25T14:11:32.840 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:32 smithi151 ceph-mon[32990]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T14:11:32.840 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:32 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/3576101096' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-25T14:11:32.840 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:32 smithi151 ceph-mon[32990]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T14:11:32.840 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:32 smithi151 ceph-mon[32990]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T14:11:32.840 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:32 smithi151 ceph-mon[32990]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-25T14:11:32.840 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:32 smithi151 ceph-mon[32990]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-25T14:11:32.840 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:32 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/1381957929' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T14:11:32.840 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:32 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/1381957929' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T14:11:32.840 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:32 smithi151 ceph-mon[32990]: mds.? [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] up:active 2024-04-25T14:11:32.840 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:32 smithi151 ceph-mon[32990]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T14:11:32.840 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:32 smithi151 ceph-mon[32990]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2024-04-25T14:11:32.840 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:32 smithi151 ceph-mon[32990]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:creating} 2 up:standby-replay 7 up:standby 2024-04-25T14:11:32.840 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:32 smithi151 ceph-mon[32990]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-04-25T14:11:32.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:32 smithi096 ceph-mon[29207]: pgmap v281: 97 pgs: 17 creating+peering, 43 active+clean, 37 unknown; 578 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 4 op/s 2024-04-25T14:11:32.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:32 smithi096 ceph-mon[29207]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T14:11:32.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:32 smithi096 ceph-mon[29207]: Cluster is now healthy 2024-04-25T14:11:32.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:32 smithi096 ceph-mon[29207]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T14:11:32.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:32 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/3576101096' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-25T14:11:32.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:32 smithi096 ceph-mon[29207]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T14:11:32.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:32 smithi096 ceph-mon[29207]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T14:11:32.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:32 smithi096 ceph-mon[29207]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-25T14:11:32.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:32 smithi096 ceph-mon[29207]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-25T14:11:32.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:32 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/1381957929' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T14:11:32.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:32 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/1381957929' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T14:11:32.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:32 smithi096 ceph-mon[29207]: mds.? [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] up:active 2024-04-25T14:11:32.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:32 smithi096 ceph-mon[29207]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T14:11:32.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:32 smithi096 ceph-mon[29207]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2024-04-25T14:11:32.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:32 smithi096 ceph-mon[29207]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:creating} 2 up:standby-replay 7 up:standby 2024-04-25T14:11:32.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:32 smithi096 ceph-mon[29207]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-04-25T14:11:32.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:32.864+0000 7fd9cef34640 1 -- 172.21.15.96:0/836849141 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fd9c802db50 con 0x7fd9c8104d40 2024-04-25T14:11:33.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.512+0000 7fd9c5ffb640 1 -- 172.21.15.96:0/836849141 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v20) ==== 115+0+0 (secure 0 0 0) 0x7fd9b40a79f0 con 0x7fd9c8104d40 2024-04-25T14:11:33.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.515+0000 7fd9cef34640 1 -- 172.21.15.96:0/836849141 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd9b80afaf0 msgr2=0x7fd9b80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:33.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.515+0000 7fd9cef34640 1 --2- 172.21.15.96:0/836849141 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd9b80afaf0 0x7fd9b80b2000 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc0058c0 tx=0x7fd9bc005d50 comp rx=0 tx=0).stop 2024-04-25T14:11:33.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.515+0000 7fd9cef34640 1 -- 172.21.15.96:0/836849141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9c8104d40 msgr2=0x7fd9c819a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:33.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.515+0000 7fd9cef34640 1 --2- 172.21.15.96:0/836849141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9c8104d40 0x7fd9c819a710 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd9b400e9f0 tx=0x7fd9b400eeb0 comp rx=0 tx=0).stop 2024-04-25T14:11:33.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.515+0000 7fd9cef34640 1 -- 172.21.15.96:0/836849141 shutdown_connections 2024-04-25T14:11:33.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.515+0000 7fd9cef34640 1 --2- 172.21.15.96:0/836849141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9c8109a40 0x7fd9c81a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:33.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.515+0000 7fd9cef34640 1 --2- 172.21.15.96:0/836849141 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd9b80afaf0 0x7fd9b80b2000 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:33.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.515+0000 7fd9cef34640 1 --2- 172.21.15.96:0/836849141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9c8104d40 0x7fd9c819a710 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:33.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.515+0000 7fd9cef34640 1 --2- 172.21.15.96:0/836849141 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9c8103b00 0x7fd9c819a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:33.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.515+0000 7fd9cef34640 1 -- 172.21.15.96:0/836849141 >> 172.21.15.96:0/836849141 conn(0x7fd9c80fecf0 msgr2=0x7fd9c8105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:33.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.515+0000 7fd9cef34640 1 -- 172.21.15.96:0/836849141 shutdown_connections 2024-04-25T14:11:33.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.515+0000 7fd9cef34640 1 -- 172.21.15.96:0/836849141 wait complete. 2024-04-25T14:11:33.532 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-04-25T14:11:33.532 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-25T14:11:33.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.652+0000 7f6958914640 1 -- 172.21.15.96:0/1227222737 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6954102170 msgr2=0x7f6954102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:33.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.652+0000 7f6958914640 1 --2- 172.21.15.96:0/1227222737 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6954102170 0x7f6954102e70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6944009900 tx=0x7f694402a410 comp rx=0 tx=0).stop 2024-04-25T14:11:33.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.652+0000 7f6958914640 1 -- 172.21.15.96:0/1227222737 shutdown_connections 2024-04-25T14:11:33.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.652+0000 7f6958914640 1 --2- 172.21.15.96:0/1227222737 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69541080b0 0x7f695410a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:33.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.652+0000 7f6958914640 1 --2- 172.21.15.96:0/1227222737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69541033b0 0x7f69541037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:33.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.652+0000 7f6958914640 1 --2- 172.21.15.96:0/1227222737 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6954102170 0x7f6954102e70 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:33.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.652+0000 7f6958914640 1 -- 172.21.15.96:0/1227222737 >> 172.21.15.96:0/1227222737 conn(0x7f69540fd380 msgr2=0x7f69540ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:33.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.652+0000 7f6958914640 1 -- 172.21.15.96:0/1227222737 shutdown_connections 2024-04-25T14:11:33.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.652+0000 7f6958914640 1 -- 172.21.15.96:0/1227222737 wait complete. 2024-04-25T14:11:33.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.653+0000 7f6958914640 1 Processor -- start 2024-04-25T14:11:33.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.653+0000 7f6958914640 1 -- start start 2024-04-25T14:11:33.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.653+0000 7f6958914640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6954102170 0x7f69541991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:33.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.653+0000 7f6958914640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69541033b0 0x7f69541996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:33.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.653+0000 7f6958914640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69541080b0 0x7f69541a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:33.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.653+0000 7f6958914640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f695410dce0 con 0x7f6954102170 2024-04-25T14:11:33.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.653+0000 7f6958914640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f695410db60 con 0x7f69541080b0 2024-04-25T14:11:33.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.653+0000 7f6958914640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f695410de60 con 0x7f69541033b0 2024-04-25T14:11:33.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.653+0000 7f695259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6954102170 0x7f69541991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:33.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.653+0000 7f695259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6954102170 0x7f69541991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35258/0 (socket says 172.21.15.96:35258) 2024-04-25T14:11:33.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.653+0000 7f695259c640 1 -- 172.21.15.96:0/3903972626 learned_addr learned my addr 172.21.15.96:0/3903972626 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:11:33.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.653+0000 7f6952d9d640 1 --2- 172.21.15.96:0/3903972626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69541080b0 0x7f69541a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:33.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.653+0000 7f6951d9b640 1 --2- 172.21.15.96:0/3903972626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69541033b0 0x7f69541996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:33.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.654+0000 7f695259c640 1 -- 172.21.15.96:0/3903972626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69541033b0 msgr2=0x7f69541996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:33.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.654+0000 7f695259c640 1 --2- 172.21.15.96:0/3903972626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69541033b0 0x7f69541996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:33.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.654+0000 7f695259c640 1 -- 172.21.15.96:0/3903972626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69541080b0 msgr2=0x7f69541a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:33.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.654+0000 7f695259c640 1 --2- 172.21.15.96:0/3903972626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69541080b0 0x7f69541a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:33.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.654+0000 7f695259c640 1 -- 172.21.15.96:0/3903972626 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69541a0ce0 con 0x7f6954102170 2024-04-25T14:11:33.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.654+0000 7f695259c640 1 --2- 172.21.15.96:0/3903972626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6954102170 0x7f69541991a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6944031440 tx=0x7f6944003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:33.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.654+0000 7f69437fe640 1 -- 172.21.15.96:0/3903972626 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694403f070 con 0x7f6954102170 2024-04-25T14:11:33.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.654+0000 7f69437fe640 1 -- 172.21.15.96:0/3903972626 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6944031c50 con 0x7f6954102170 2024-04-25T14:11:33.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.654+0000 7f6958914640 1 -- 172.21.15.96:0/3903972626 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69541a0f70 con 0x7f6954102170 2024-04-25T14:11:33.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.654+0000 7f69437fe640 1 -- 172.21.15.96:0/3903972626 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6944035030 con 0x7f6954102170 2024-04-25T14:11:33.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.654+0000 7f6958914640 1 -- 172.21.15.96:0/3903972626 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69541a1370 con 0x7f6954102170 2024-04-25T14:11:33.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.655+0000 7f6958914640 1 -- 172.21.15.96:0/3903972626 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69541077b0 con 0x7f6954102170 2024-04-25T14:11:33.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.656+0000 7f69437fe640 1 -- 172.21.15.96:0/3903972626 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f6944042070 con 0x7f6954102170 2024-04-25T14:11:33.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.657+0000 7f69437fe640 1 --2- 172.21.15.96:0/3903972626 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f693c0afaf0 0x7f693c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:33.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.657+0000 7f6951d9b640 1 --2- 172.21.15.96:0/3903972626 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f693c0afaf0 0x7f693c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:33.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.657+0000 7f69437fe640 1 -- 172.21.15.96:0/3903972626 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7f69440ff920 con 0x7f6954102170 2024-04-25T14:11:33.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.657+0000 7f6951d9b640 1 --2- 172.21.15.96:0/3903972626 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f693c0afaf0 0x7f693c0b2000 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f695419a240 tx=0x7f6948005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:33.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.661+0000 7f69437fe640 1 -- 172.21.15.96:0/3903972626 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f69440c8000 con 0x7f6954102170 2024-04-25T14:11:33.839 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:33 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/836849141' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T14:11:33.840 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:33 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/836849141' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T14:11:33.840 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:33 smithi097 ceph-mon[32720]: mds.? [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] up:active 2024-04-25T14:11:33.840 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:33 smithi097 ceph-mon[32720]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-25T14:11:33.840 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:33 smithi097 ceph-mon[32720]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T14:11:33.840 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:33 smithi097 ceph-mon[32720]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=f=up:active,3=e=up:creating} 2 up:standby-replay 6 up:standby 2024-04-25T14:11:33.840 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:33 smithi097 ceph-mon[32720]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-04-25T14:11:33.858 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:33 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/836849141' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T14:11:33.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:33 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/836849141' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T14:11:33.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:33 smithi096 ceph-mon[29207]: mds.? [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] up:active 2024-04-25T14:11:33.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:33 smithi096 ceph-mon[29207]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-25T14:11:33.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:33 smithi096 ceph-mon[29207]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T14:11:33.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:33 smithi096 ceph-mon[29207]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=f=up:active,3=e=up:creating} 2 up:standby-replay 6 up:standby 2024-04-25T14:11:33.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:33 smithi096 ceph-mon[29207]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-04-25T14:11:33.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.864+0000 7f6958914640 1 -- 172.21.15.96:0/3903972626 --> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f6954062fe0 con 0x7f693c0afaf0 2024-04-25T14:11:33.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.894+0000 7f69437fe640 1 -- 172.21.15.96:0/3903972626 <== mgr.14152 v2:172.21.15.96:6800/536332425 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6954062fe0 con 0x7f693c0afaf0 2024-04-25T14:11:33.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.896+0000 7f6958914640 1 -- 172.21.15.96:0/3903972626 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f693c0afaf0 msgr2=0x7f693c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:33.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.896+0000 7f6958914640 1 --2- 172.21.15.96:0/3903972626 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f693c0afaf0 0x7f693c0b2000 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f695419a240 tx=0x7f6948005f90 comp rx=0 tx=0).stop 2024-04-25T14:11:33.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.896+0000 7f6958914640 1 -- 172.21.15.96:0/3903972626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6954102170 msgr2=0x7f69541991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:33.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.896+0000 7f6958914640 1 --2- 172.21.15.96:0/3903972626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6954102170 0x7f69541991a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6944031440 tx=0x7f6944003c10 comp rx=0 tx=0).stop 2024-04-25T14:11:33.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.896+0000 7f6958914640 1 -- 172.21.15.96:0/3903972626 shutdown_connections 2024-04-25T14:11:33.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.896+0000 7f6958914640 1 --2- 172.21.15.96:0/3903972626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69541080b0 0x7f69541a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:33.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.896+0000 7f6958914640 1 --2- 172.21.15.96:0/3903972626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69541033b0 0x7f69541996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:33.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.896+0000 7f6958914640 1 --2- 172.21.15.96:0/3903972626 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f693c0afaf0 0x7f693c0b2000 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:33.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.896+0000 7f6958914640 1 --2- 172.21.15.96:0/3903972626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6954102170 0x7f69541991a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:33.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.896+0000 7f6958914640 1 -- 172.21.15.96:0/3903972626 >> 172.21.15.96:0/3903972626 conn(0x7f69540fd380 msgr2=0x7f69540fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:33.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.896+0000 7f6958914640 1 -- 172.21.15.96:0/3903972626 shutdown_connections 2024-04-25T14:11:33.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:33.896+0000 7f6958914640 1 -- 172.21.15.96:0/3903972626 wait complete. 2024-04-25T14:11:33.908 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-25T14:11:33.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:33 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/836849141' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T14:11:33.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:33 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/836849141' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T14:11:33.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:33 smithi151 ceph-mon[32990]: mds.? [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] up:active 2024-04-25T14:11:33.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:33 smithi151 ceph-mon[32990]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-25T14:11:33.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:33 smithi151 ceph-mon[32990]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T14:11:33.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:33 smithi151 ceph-mon[32990]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=f=up:active,3=e=up:creating} 2 up:standby-replay 6 up:standby 2024-04-25T14:11:33.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:33 smithi151 ceph-mon[32990]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-04-25T14:11:34.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.021+0000 7f73d0d8c640 1 -- 172.21.15.96:0/2472607951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73cc108920 msgr2=0x7f73cc10ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:34.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.021+0000 7f73d0d8c640 1 --2- 172.21.15.96:0/2472607951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73cc108920 0x7f73cc10ad40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f73c4009ca0 tx=0x7f73c402a8c0 comp rx=0 tx=0).stop 2024-04-25T14:11:34.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.021+0000 7f73d0d8c640 1 -- 172.21.15.96:0/2472607951 shutdown_connections 2024-04-25T14:11:34.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.021+0000 7f73d0d8c640 1 --2- 172.21.15.96:0/2472607951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73cc108920 0x7f73cc10ad40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:34.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.021+0000 7f73d0d8c640 1 --2- 172.21.15.96:0/2472607951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73cc103c20 0x7f73cc104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:34.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.021+0000 7f73d0d8c640 1 --2- 172.21.15.96:0/2472607951 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73cc1029e0 0x7f73cc1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:34.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.021+0000 7f73d0d8c640 1 -- 172.21.15.96:0/2472607951 >> 172.21.15.96:0/2472607951 conn(0x7f73cc0fdbd0 msgr2=0x7f73cc100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:34.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.021+0000 7f73d0d8c640 1 -- 172.21.15.96:0/2472607951 shutdown_connections 2024-04-25T14:11:34.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.021+0000 7f73d0d8c640 1 -- 172.21.15.96:0/2472607951 wait complete. 2024-04-25T14:11:34.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.022+0000 7f73d0d8c640 1 Processor -- start 2024-04-25T14:11:34.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.022+0000 7f73d0d8c640 1 -- start start 2024-04-25T14:11:34.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.022+0000 7f73d0d8c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73cc1029e0 0x7f73cc06a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:34.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.022+0000 7f73d0d8c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73cc103c20 0x7f73cc19f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:34.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.022+0000 7f73d0d8c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73cc108920 0x7f73cc1a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:34.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.022+0000 7f73d0d8c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc10e510 con 0x7f73cc1029e0 2024-04-25T14:11:34.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.022+0000 7f73d0d8c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc10e390 con 0x7f73cc103c20 2024-04-25T14:11:34.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.022+0000 7f73d0d8c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc10e690 con 0x7f73cc108920 2024-04-25T14:11:34.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.022+0000 7f73ca59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73cc1029e0 0x7f73cc06a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:34.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.022+0000 7f73ca59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73cc1029e0 0x7f73cc06a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35304/0 (socket says 172.21.15.96:35304) 2024-04-25T14:11:34.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.022+0000 7f73ca59c640 1 -- 172.21.15.96:0/1134437489 learned_addr learned my addr 172.21.15.96:0/1134437489 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:11:34.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.022+0000 7f73c3fff640 1 --2- 172.21.15.96:0/1134437489 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73cc103c20 0x7f73cc19f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:34.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.022+0000 7f73cad9d640 1 --2- 172.21.15.96:0/1134437489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73cc108920 0x7f73cc1a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:34.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.022+0000 7f73ca59c640 1 -- 172.21.15.96:0/1134437489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73cc108920 msgr2=0x7f73cc1a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:34.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.022+0000 7f73ca59c640 1 --2- 172.21.15.96:0/1134437489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73cc108920 0x7f73cc1a1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:34.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.022+0000 7f73ca59c640 1 -- 172.21.15.96:0/1134437489 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73cc103c20 msgr2=0x7f73cc19f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:34.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.022+0000 7f73ca59c640 1 --2- 172.21.15.96:0/1134437489 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73cc103c20 0x7f73cc19f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:34.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.022+0000 7f73ca59c640 1 -- 172.21.15.96:0/1134437489 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73cc1a2030 con 0x7f73cc1029e0 2024-04-25T14:11:34.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.023+0000 7f73ca59c640 1 --2- 172.21.15.96:0/1134437489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73cc1029e0 0x7f73cc06a900 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f73bc005d00 tx=0x7f73bc00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:34.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.023+0000 7f73c37fe640 1 -- 172.21.15.96:0/1134437489 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73bc017070 con 0x7f73cc1029e0 2024-04-25T14:11:34.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.023+0000 7f73c37fe640 1 -- 172.21.15.96:0/1134437489 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73bc009c30 con 0x7f73cc1029e0 2024-04-25T14:11:34.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.023+0000 7f73d0d8c640 1 -- 172.21.15.96:0/1134437489 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73cc1a2320 con 0x7f73cc1029e0 2024-04-25T14:11:34.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.023+0000 7f73c37fe640 1 -- 172.21.15.96:0/1134437489 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73bc00f620 con 0x7f73cc1029e0 2024-04-25T14:11:34.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.023+0000 7f73d0d8c640 1 -- 172.21.15.96:0/1134437489 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73cc1a2820 con 0x7f73cc1029e0 2024-04-25T14:11:34.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.024+0000 7f73d0d8c640 1 -- 172.21.15.96:0/1134437489 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73cc107730 con 0x7f73cc1029e0 2024-04-25T14:11:34.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.025+0000 7f73c37fe640 1 -- 172.21.15.96:0/1134437489 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f73bc01a070 con 0x7f73cc1029e0 2024-04-25T14:11:34.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.026+0000 7f73c37fe640 1 --2- 172.21.15.96:0/1134437489 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f73b80afaf0 0x7f73b80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:34.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.026+0000 7f73c3fff640 1 --2- 172.21.15.96:0/1134437489 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f73b80afaf0 0x7f73b80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:34.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.026+0000 7f73c37fe640 1 -- 172.21.15.96:0/1134437489 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7f73bc0de480 con 0x7f73cc1029e0 2024-04-25T14:11:34.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.026+0000 7f73c3fff640 1 --2- 172.21.15.96:0/1134437489 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f73b80afaf0 0x7f73b80b2000 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f73cc06a2a0 tx=0x7f73b400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:34.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.029+0000 7f73c37fe640 1 -- 172.21.15.96:0/1134437489 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f73bc009de0 con 0x7f73cc1029e0 2024-04-25T14:11:34.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.235+0000 7f73d0d8c640 1 -- 172.21.15.96:0/1134437489 --> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f73cc02dcd0 con 0x7f73b80afaf0 2024-04-25T14:11:34.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.263+0000 7f73c37fe640 1 -- 172.21.15.96:0/1134437489 <== mgr.14152 v2:172.21.15.96:6800/536332425 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f73cc02dcd0 con 0x7f73b80afaf0 2024-04-25T14:11:34.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.265+0000 7f73d0d8c640 1 -- 172.21.15.96:0/1134437489 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f73b80afaf0 msgr2=0x7f73b80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:34.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.265+0000 7f73d0d8c640 1 --2- 172.21.15.96:0/1134437489 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f73b80afaf0 0x7f73b80b2000 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f73cc06a2a0 tx=0x7f73b400b410 comp rx=0 tx=0).stop 2024-04-25T14:11:34.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.265+0000 7f73d0d8c640 1 -- 172.21.15.96:0/1134437489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73cc1029e0 msgr2=0x7f73cc06a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:34.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.265+0000 7f73d0d8c640 1 --2- 172.21.15.96:0/1134437489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73cc1029e0 0x7f73cc06a900 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f73bc005d00 tx=0x7f73bc00dde0 comp rx=0 tx=0).stop 2024-04-25T14:11:34.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.265+0000 7f73d0d8c640 1 -- 172.21.15.96:0/1134437489 shutdown_connections 2024-04-25T14:11:34.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.265+0000 7f73d0d8c640 1 --2- 172.21.15.96:0/1134437489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73cc108920 0x7f73cc1a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:34.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.265+0000 7f73d0d8c640 1 --2- 172.21.15.96:0/1134437489 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73cc103c20 0x7f73cc19f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:34.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.265+0000 7f73d0d8c640 1 --2- 172.21.15.96:0/1134437489 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f73b80afaf0 0x7f73b80b2000 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:34.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.265+0000 7f73d0d8c640 1 --2- 172.21.15.96:0/1134437489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73cc1029e0 0x7f73cc06a900 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:34.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.265+0000 7f73d0d8c640 1 -- 172.21.15.96:0/1134437489 >> 172.21.15.96:0/1134437489 conn(0x7f73cc0fdbd0 msgr2=0x7f73cc104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:34.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.265+0000 7f73d0d8c640 1 -- 172.21.15.96:0/1134437489 shutdown_connections 2024-04-25T14:11:34.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.265+0000 7f73d0d8c640 1 -- 172.21.15.96:0/1134437489 wait complete. 2024-04-25T14:11:34.275 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:11:33 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:11:33.869+0000 7fed5d43f700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:11:34.275 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:11:33 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:11:33.869+0000 7fed5d43f700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:11:34.275 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:11:33 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:11:33.869+0000 7fed5d43f700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:11:34.275 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:11:33 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:11:33.869+0000 7fed5d43f700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:11:34.276 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:11:33 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:11:33.869+0000 7fed5d43f700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:11:34.276 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-25T14:11:34.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.393+0000 7ff66392e640 1 -- 172.21.15.96:0/741642374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff65c102fc0 msgr2=0x7ff65c1033d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:34.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.393+0000 7ff66392e640 1 --2- 172.21.15.96:0/741642374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff65c102fc0 0x7ff65c1033d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff654009ca0 tx=0x7ff65402a8c0 comp rx=0 tx=0).stop 2024-04-25T14:11:34.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.393+0000 7ff66392e640 1 -- 172.21.15.96:0/741642374 shutdown_connections 2024-04-25T14:11:34.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.393+0000 7ff66392e640 1 --2- 172.21.15.96:0/741642374 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff65c104680 0x7ff65c10add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:34.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.393+0000 7ff66392e640 1 --2- 172.21.15.96:0/741642374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff65c103b20 0x7ff65c103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:34.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.393+0000 7ff66392e640 1 --2- 172.21.15.96:0/741642374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff65c102fc0 0x7ff65c1033d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:34.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.393+0000 7ff66392e640 1 -- 172.21.15.96:0/741642374 >> 172.21.15.96:0/741642374 conn(0x7ff65c0fd340 msgr2=0x7ff65c0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:34.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.393+0000 7ff66392e640 1 -- 172.21.15.96:0/741642374 shutdown_connections 2024-04-25T14:11:34.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.393+0000 7ff66392e640 1 -- 172.21.15.96:0/741642374 wait complete. 2024-04-25T14:11:34.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.394+0000 7ff66392e640 1 Processor -- start 2024-04-25T14:11:34.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.394+0000 7ff66392e640 1 -- start start 2024-04-25T14:11:34.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.394+0000 7ff66392e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff65c102fc0 0x7ff65c198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:34.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.394+0000 7ff66392e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff65c103b20 0x7ff65c1994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:34.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.394+0000 7ff66392e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff65c19e410 0x7ff65c1a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:34.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.394+0000 7ff66392e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff65c10e720 con 0x7ff65c103b20 2024-04-25T14:11:34.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.394+0000 7ff66392e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff65c10e5a0 con 0x7ff65c102fc0 2024-04-25T14:11:34.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.394+0000 7ff66392e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff65c10e8a0 con 0x7ff65c19e410 2024-04-25T14:11:34.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.394+0000 7ff660ec9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff65c103b20 0x7ff65c1994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:34.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.394+0000 7ff660ec9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff65c103b20 0x7ff65c1994e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35324/0 (socket says 172.21.15.96:35324) 2024-04-25T14:11:34.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.394+0000 7ff660ec9640 1 -- 172.21.15.96:0/438417403 learned_addr learned my addr 172.21.15.96:0/438417403 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:11:34.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.394+0000 7ff6616ca640 1 --2- 172.21.15.96:0/438417403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff65c102fc0 0x7ff65c198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:34.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.394+0000 7ff661ecb640 1 --2- 172.21.15.96:0/438417403 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff65c19e410 0x7ff65c1a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:34.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.394+0000 7ff660ec9640 1 -- 172.21.15.96:0/438417403 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff65c19e410 msgr2=0x7ff65c1a0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:34.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.394+0000 7ff660ec9640 1 --2- 172.21.15.96:0/438417403 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff65c19e410 0x7ff65c1a0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:34.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.394+0000 7ff660ec9640 1 -- 172.21.15.96:0/438417403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff65c102fc0 msgr2=0x7ff65c198fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:34.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.394+0000 7ff660ec9640 1 --2- 172.21.15.96:0/438417403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff65c102fc0 0x7ff65c198fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:34.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.394+0000 7ff660ec9640 1 -- 172.21.15.96:0/438417403 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff65c1a0ea0 con 0x7ff65c103b20 2024-04-25T14:11:34.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.394+0000 7ff6616ca640 1 --2- 172.21.15.96:0/438417403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff65c102fc0 0x7ff65c198fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:11:34.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.394+0000 7ff660ec9640 1 --2- 172.21.15.96:0/438417403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff65c103b20 0x7ff65c1994e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff65000e9f0 tx=0x7ff65000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:34.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.395+0000 7ff64e7fc640 1 -- 172.21.15.96:0/438417403 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65000cd80 con 0x7ff65c103b20 2024-04-25T14:11:34.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.395+0000 7ff64e7fc640 1 -- 172.21.15.96:0/438417403 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6500108e0 con 0x7ff65c103b20 2024-04-25T14:11:34.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.395+0000 7ff66392e640 1 -- 172.21.15.96:0/438417403 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff65c1a1190 con 0x7ff65c103b20 2024-04-25T14:11:34.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.395+0000 7ff64e7fc640 1 -- 172.21.15.96:0/438417403 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff650018980 con 0x7ff65c103b20 2024-04-25T14:11:34.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.395+0000 7ff66392e640 1 -- 172.21.15.96:0/438417403 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff65c1a16e0 con 0x7ff65c103b20 2024-04-25T14:11:34.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.396+0000 7ff66392e640 1 -- 172.21.15.96:0/438417403 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff65c069c70 con 0x7ff65c103b20 2024-04-25T14:11:34.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.397+0000 7ff64e7fc640 1 -- 172.21.15.96:0/438417403 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7ff650010430 con 0x7ff65c103b20 2024-04-25T14:11:34.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.398+0000 7ff64e7fc640 1 --2- 172.21.15.96:0/438417403 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ff6480afaf0 0x7ff6480b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:34.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.398+0000 7ff6616ca640 1 --2- 172.21.15.96:0/438417403 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ff6480afaf0 0x7ff6480b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:34.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.398+0000 7ff64e7fc640 1 -- 172.21.15.96:0/438417403 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7ff6500dedd0 con 0x7ff65c103b20 2024-04-25T14:11:34.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.398+0000 7ff6616ca640 1 --2- 172.21.15.96:0/438417403 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ff6480afaf0 0x7ff6480b2000 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff6540094e0 tx=0x7ff65402f370 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:34.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.403+0000 7ff64e7fc640 1 -- 172.21.15.96:0/438417403 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff6500a74b0 con 0x7ff65c103b20 2024-04-25T14:11:34.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.624+0000 7ff66392e640 1 -- 172.21.15.96:0/438417403 --> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7ff65c02de30 con 0x7ff6480afaf0 2024-04-25T14:11:34.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.631+0000 7ff64e7fc640 1 -- 172.21.15.96:0/438417403 <== mgr.14152 v2:172.21.15.96:6800/536332425 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7ff65c02de30 con 0x7ff6480afaf0 2024-04-25T14:11:34.631 INFO:teuthology.orchestra.run.smithi096.stdout:/volumes/qa/sv_0/4ded6220-a07c-4017-996d-ae56efe56342 2024-04-25T14:11:34.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.633+0000 7ff66392e640 1 -- 172.21.15.96:0/438417403 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ff6480afaf0 msgr2=0x7ff6480b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:34.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.633+0000 7ff66392e640 1 --2- 172.21.15.96:0/438417403 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ff6480afaf0 0x7ff6480b2000 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff6540094e0 tx=0x7ff65402f370 comp rx=0 tx=0).stop 2024-04-25T14:11:34.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.633+0000 7ff66392e640 1 -- 172.21.15.96:0/438417403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff65c103b20 msgr2=0x7ff65c1994e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:34.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.633+0000 7ff66392e640 1 --2- 172.21.15.96:0/438417403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff65c103b20 0x7ff65c1994e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff65000e9f0 tx=0x7ff65000eeb0 comp rx=0 tx=0).stop 2024-04-25T14:11:34.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.633+0000 7ff66392e640 1 -- 172.21.15.96:0/438417403 shutdown_connections 2024-04-25T14:11:34.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.633+0000 7ff66392e640 1 --2- 172.21.15.96:0/438417403 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff65c19e410 0x7ff65c1a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:34.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.633+0000 7ff66392e640 1 --2- 172.21.15.96:0/438417403 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ff6480afaf0 0x7ff6480b2000 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:34.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.633+0000 7ff66392e640 1 --2- 172.21.15.96:0/438417403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff65c103b20 0x7ff65c1994e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:34.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.633+0000 7ff66392e640 1 --2- 172.21.15.96:0/438417403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff65c102fc0 0x7ff65c198fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:34.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.633+0000 7ff66392e640 1 -- 172.21.15.96:0/438417403 >> 172.21.15.96:0/438417403 conn(0x7ff65c0fd340 msgr2=0x7ff65c0ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:34.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.633+0000 7ff66392e640 1 -- 172.21.15.96:0/438417403 shutdown_connections 2024-04-25T14:11:34.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.633+0000 7ff66392e640 1 -- 172.21.15.96:0/438417403 wait complete. 2024-04-25T14:11:34.646 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-25T14:11:34.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.764+0000 7f8533b64640 1 -- 172.21.15.96:0/4056395847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f852c1029e0 msgr2=0x7f852c1036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:34.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.764+0000 7f8533b64640 1 --2- 172.21.15.96:0/4056395847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f852c1029e0 0x7f852c1036e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8528009900 tx=0x7f852802a410 comp rx=0 tx=0).stop 2024-04-25T14:11:34.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.764+0000 7f8533b64640 1 -- 172.21.15.96:0/4056395847 shutdown_connections 2024-04-25T14:11:34.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.764+0000 7f8533b64640 1 --2- 172.21.15.96:0/4056395847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f852c108920 0x7f852c10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:34.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.764+0000 7f8533b64640 1 --2- 172.21.15.96:0/4056395847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f852c103c20 0x7f852c104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:34.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.764+0000 7f8533b64640 1 --2- 172.21.15.96:0/4056395847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f852c1029e0 0x7f852c1036e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:34.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.764+0000 7f8533b64640 1 -- 172.21.15.96:0/4056395847 >> 172.21.15.96:0/4056395847 conn(0x7f852c0fdbd0 msgr2=0x7f852c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:34.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.764+0000 7f8533b64640 1 -- 172.21.15.96:0/4056395847 shutdown_connections 2024-04-25T14:11:34.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.764+0000 7f8533b64640 1 -- 172.21.15.96:0/4056395847 wait complete. 2024-04-25T14:11:34.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.765+0000 7f8533b64640 1 Processor -- start 2024-04-25T14:11:34.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.765+0000 7f8533b64640 1 -- start start 2024-04-25T14:11:34.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.765+0000 7f8533b64640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f852c1029e0 0x7f852c0ffa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:34.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.765+0000 7f8533b64640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f852c103c20 0x7f852c0fe100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:34.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.765+0000 7f8533b64640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f852c108920 0x7f852c0fe640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:34.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.765+0000 7f8533b64640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f852c10e4e0 con 0x7f852c108920 2024-04-25T14:11:34.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.765+0000 7f8533b64640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f852c10e360 con 0x7f852c103c20 2024-04-25T14:11:34.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.765+0000 7f8533b64640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f852c10e660 con 0x7f852c1029e0 2024-04-25T14:11:34.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.765+0000 7f8532101640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f852c108920 0x7f852c0fe640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:34.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.765+0000 7f8532101640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f852c108920 0x7f852c0fe640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35344/0 (socket says 172.21.15.96:35344) 2024-04-25T14:11:34.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.765+0000 7f8532101640 1 -- 172.21.15.96:0/3508542579 learned_addr learned my addr 172.21.15.96:0/3508542579 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:11:34.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.765+0000 7f85310ff640 1 --2- 172.21.15.96:0/3508542579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f852c103c20 0x7f852c0fe100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:34.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.765+0000 7f8531900640 1 --2- 172.21.15.96:0/3508542579 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f852c1029e0 0x7f852c0ffa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:34.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.766+0000 7f8532101640 1 -- 172.21.15.96:0/3508542579 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f852c1029e0 msgr2=0x7f852c0ffa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:34.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.766+0000 7f8532101640 1 --2- 172.21.15.96:0/3508542579 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f852c1029e0 0x7f852c0ffa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:34.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.766+0000 7f8532101640 1 -- 172.21.15.96:0/3508542579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f852c103c20 msgr2=0x7f852c0fe100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:34.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.766+0000 7f8532101640 1 --2- 172.21.15.96:0/3508542579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f852c103c20 0x7f852c0fe100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:34.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.766+0000 7f8532101640 1 -- 172.21.15.96:0/3508542579 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f852c0fed70 con 0x7f852c108920 2024-04-25T14:11:34.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.766+0000 7f8532101640 1 --2- 172.21.15.96:0/3508542579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f852c108920 0x7f852c0fe640 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f852400cad0 tx=0x7f852400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:34.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.766+0000 7f8522ffd640 1 -- 172.21.15.96:0/3508542579 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852400ae20 con 0x7f852c108920 2024-04-25T14:11:34.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.766+0000 7f8533b64640 1 -- 172.21.15.96:0/3508542579 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f852c1a4640 con 0x7f852c108920 2024-04-25T14:11:34.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.766+0000 7f8522ffd640 1 -- 172.21.15.96:0/3508542579 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8524005020 con 0x7f852c108920 2024-04-25T14:11:34.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.766+0000 7f8522ffd640 1 -- 172.21.15.96:0/3508542579 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852400f6b0 con 0x7f852c108920 2024-04-25T14:11:34.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.766+0000 7f8533b64640 1 -- 172.21.15.96:0/3508542579 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f852c1a4b00 con 0x7f852c108920 2024-04-25T14:11:34.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.767+0000 7f8533b64640 1 -- 172.21.15.96:0/3508542579 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f852c06c2a0 con 0x7f852c108920 2024-04-25T14:11:34.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.768+0000 7f8522ffd640 1 -- 172.21.15.96:0/3508542579 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f85240051d0 con 0x7f852c108920 2024-04-25T14:11:34.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.769+0000 7f8522ffd640 1 --2- 172.21.15.96:0/3508542579 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f85180afaf0 0x7f85180b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:34.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.769+0000 7f8531900640 1 --2- 172.21.15.96:0/3508542579 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f85180afaf0 0x7f85180b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:34.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.769+0000 7f8522ffd640 1 -- 172.21.15.96:0/3508542579 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7f85240dea20 con 0x7f852c108920 2024-04-25T14:11:34.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.770+0000 7f8531900640 1 --2- 172.21.15.96:0/3508542579 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f85180afaf0 0x7f85180b2000 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8528005ab0 tx=0x7f8528031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:34.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:34.774+0000 7f8522ffd640 1 -- 172.21.15.96:0/3508542579 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8524014030 con 0x7f852c108920 2024-04-25T14:11:34.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:34 smithi096 ceph-mon[29207]: pgmap v282: 97 pgs: 10 creating+peering, 87 active+clean; 578 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 697 B/s rd, 3.7 KiB/s wr, 10 op/s 2024-04-25T14:11:34.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:34 smithi096 ceph-mon[29207]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:11:34.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:34 smithi096 ceph-mon[29207]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:11:34.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:34 smithi096 ceph-mon[29207]: mds.? [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] up:active 2024-04-25T14:11:34.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:34 smithi096 ceph-mon[29207]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-25T14:11:34.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:34 smithi096 ceph-mon[29207]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T14:11:34.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:34 smithi096 ceph-mon[29207]: Cluster is now healthy 2024-04-25T14:11:34.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:34 smithi096 ceph-mon[29207]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=f=up:active,3=e=up:active} 2 up:standby-replay 6 up:standby 2024-04-25T14:11:34.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:34 smithi096 ceph-mon[29207]: fsmap cephfs:5 2 up:standby-replay 1 up:creating 5 up:standby 4 up:active 2024-04-25T14:11:34.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:34 smithi096 ceph-mon[29207]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-04-25T14:11:34.922 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 14:11:34 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-b[63225]: 2024-04-25T14:11:34.458+0000 7f387f969700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T14:11:34.923 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 14:11:34 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-b[63225]: 2024-04-25T14:11:34.458+0000 7f387f969700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-25T14:11:34.923 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 14:11:34 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-b[63225]: 2024-04-25T14:11:34.458+0000 7f387f969700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-25T14:11:34.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:34 smithi097 ceph-mon[32720]: pgmap v282: 97 pgs: 10 creating+peering, 87 active+clean; 578 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 697 B/s rd, 3.7 KiB/s wr, 10 op/s 2024-04-25T14:11:34.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:34 smithi097 ceph-mon[32720]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:11:34.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:34 smithi097 ceph-mon[32720]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:11:34.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:34 smithi097 ceph-mon[32720]: mds.? [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] up:active 2024-04-25T14:11:34.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:34 smithi097 ceph-mon[32720]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-25T14:11:34.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:34 smithi097 ceph-mon[32720]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T14:11:34.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:34 smithi097 ceph-mon[32720]: Cluster is now healthy 2024-04-25T14:11:34.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:34 smithi097 ceph-mon[32720]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=f=up:active,3=e=up:active} 2 up:standby-replay 6 up:standby 2024-04-25T14:11:34.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:34 smithi097 ceph-mon[32720]: fsmap cephfs:5 2 up:standby-replay 1 up:creating 5 up:standby 4 up:active 2024-04-25T14:11:34.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:34 smithi097 ceph-mon[32720]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-04-25T14:11:34.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:34 smithi151 ceph-mon[32990]: pgmap v282: 97 pgs: 10 creating+peering, 87 active+clean; 578 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 697 B/s rd, 3.7 KiB/s wr, 10 op/s 2024-04-25T14:11:34.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:34 smithi151 ceph-mon[32990]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:11:34.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:34 smithi151 ceph-mon[32990]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:11:34.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:34 smithi151 ceph-mon[32990]: mds.? [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] up:active 2024-04-25T14:11:34.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:34 smithi151 ceph-mon[32990]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-25T14:11:34.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:34 smithi151 ceph-mon[32990]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T14:11:34.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:34 smithi151 ceph-mon[32990]: Cluster is now healthy 2024-04-25T14:11:34.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:34 smithi151 ceph-mon[32990]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=f=up:active,3=e=up:active} 2 up:standby-replay 6 up:standby 2024-04-25T14:11:34.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:34 smithi151 ceph-mon[32990]: fsmap cephfs:5 2 up:standby-replay 1 up:creating 5 up:standby 4 up:active 2024-04-25T14:11:34.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:34 smithi151 ceph-mon[32990]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-04-25T14:11:35.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.009+0000 7f8533b64640 1 -- 172.21.15.96:0/3508542579 --> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f852c10a5e0 con 0x7f85180afaf0 2024-04-25T14:11:35.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.035+0000 7f8522ffd640 1 -- 172.21.15.96:0/3508542579 <== mgr.14152 v2:172.21.15.96:6800/536332425 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f852c10a5e0 con 0x7f85180afaf0 2024-04-25T14:11:35.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.038+0000 7f8533b64640 1 -- 172.21.15.96:0/3508542579 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f85180afaf0 msgr2=0x7f85180b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:35.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.038+0000 7f8533b64640 1 --2- 172.21.15.96:0/3508542579 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f85180afaf0 0x7f85180b2000 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8528005ab0 tx=0x7f8528031400 comp rx=0 tx=0).stop 2024-04-25T14:11:35.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.038+0000 7f8533b64640 1 -- 172.21.15.96:0/3508542579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f852c108920 msgr2=0x7f852c0fe640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:35.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.038+0000 7f8533b64640 1 --2- 172.21.15.96:0/3508542579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f852c108920 0x7f852c0fe640 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f852400cad0 tx=0x7f852400cf90 comp rx=0 tx=0).stop 2024-04-25T14:11:35.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.038+0000 7f8533b64640 1 -- 172.21.15.96:0/3508542579 shutdown_connections 2024-04-25T14:11:35.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.038+0000 7f8533b64640 1 --2- 172.21.15.96:0/3508542579 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f85180afaf0 0x7f85180b2000 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:35.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.038+0000 7f8533b64640 1 --2- 172.21.15.96:0/3508542579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f852c108920 0x7f852c0fe640 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:35.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.038+0000 7f8533b64640 1 --2- 172.21.15.96:0/3508542579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f852c103c20 0x7f852c0fe100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:35.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.038+0000 7f8533b64640 1 --2- 172.21.15.96:0/3508542579 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f852c1029e0 0x7f852c0ffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:35.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.038+0000 7f8533b64640 1 -- 172.21.15.96:0/3508542579 >> 172.21.15.96:0/3508542579 conn(0x7f852c0fdbd0 msgr2=0x7f852c109ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:35.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.038+0000 7f8533b64640 1 -- 172.21.15.96:0/3508542579 shutdown_connections 2024-04-25T14:11:35.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.038+0000 7f8533b64640 1 -- 172.21.15.96:0/3508542579 wait complete. 2024-04-25T14:11:35.050 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-25T14:11:35.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.165+0000 7f6154c15640 1 -- 172.21.15.96:0/1249346912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6150105090 msgr2=0x7f61501054a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:35.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.165+0000 7f6154c15640 1 --2- 172.21.15.96:0/1249346912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6150105090 0x7f61501054a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6144009900 tx=0x7f614402a410 comp rx=0 tx=0).stop 2024-04-25T14:11:35.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.165+0000 7f6154c15640 1 -- 172.21.15.96:0/1249346912 shutdown_connections 2024-04-25T14:11:35.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.165+0000 7f6154c15640 1 --2- 172.21.15.96:0/1249346912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6150102240 0x7f615010bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:35.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.165+0000 7f6154c15640 1 --2- 172.21.15.96:0/1249346912 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6150101740 0x7f6150101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:35.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.165+0000 7f6154c15640 1 --2- 172.21.15.96:0/1249346912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6150105090 0x7f61501054a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:35.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.165+0000 7f6154c15640 1 -- 172.21.15.96:0/1249346912 >> 172.21.15.96:0/1249346912 conn(0x7f61500fd340 msgr2=0x7f61500ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:35.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.165+0000 7f6154c15640 1 -- 172.21.15.96:0/1249346912 shutdown_connections 2024-04-25T14:11:35.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.165+0000 7f6154c15640 1 -- 172.21.15.96:0/1249346912 wait complete. 2024-04-25T14:11:35.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.166+0000 7f6154c15640 1 Processor -- start 2024-04-25T14:11:35.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.166+0000 7f6154c15640 1 -- start start 2024-04-25T14:11:35.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.166+0000 7f6154c15640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6150101740 0x7f6150104240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:35.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.166+0000 7f6154c15640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6150102240 0x7f61501028e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:35.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.166+0000 7f6154c15640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6150105090 0x7f6150102e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:35.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.166+0000 7f6154c15640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f615010f600 con 0x7f6150101740 2024-04-25T14:11:35.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.166+0000 7f6154c15640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f615010f480 con 0x7f6150105090 2024-04-25T14:11:35.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.166+0000 7f6154c15640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f615010f780 con 0x7f6150102240 2024-04-25T14:11:35.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.166+0000 7f614e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6150101740 0x7f6150104240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:35.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.166+0000 7f614e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6150101740 0x7f6150104240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35366/0 (socket says 172.21.15.96:35366) 2024-04-25T14:11:35.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.166+0000 7f614e59c640 1 -- 172.21.15.96:0/3095593152 learned_addr learned my addr 172.21.15.96:0/3095593152 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:11:35.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.166+0000 7f614ed9d640 1 --2- 172.21.15.96:0/3095593152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6150105090 0x7f6150102e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:35.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.166+0000 7f614dd9b640 1 --2- 172.21.15.96:0/3095593152 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6150102240 0x7f61501028e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:35.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.166+0000 7f614e59c640 1 -- 172.21.15.96:0/3095593152 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6150102240 msgr2=0x7f61501028e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:35.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.166+0000 7f614e59c640 1 --2- 172.21.15.96:0/3095593152 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6150102240 0x7f61501028e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:35.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.166+0000 7f614e59c640 1 -- 172.21.15.96:0/3095593152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6150105090 msgr2=0x7f6150102e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:35.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.166+0000 7f614e59c640 1 --2- 172.21.15.96:0/3095593152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6150105090 0x7f6150102e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:35.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.166+0000 7f614e59c640 1 -- 172.21.15.96:0/3095593152 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6150103550 con 0x7f6150101740 2024-04-25T14:11:35.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.166+0000 7f614dd9b640 1 --2- 172.21.15.96:0/3095593152 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6150102240 0x7f61501028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:35.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.166+0000 7f614e59c640 1 --2- 172.21.15.96:0/3095593152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6150101740 0x7f6150104240 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6144006010 tx=0x7f6144004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:35.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.167+0000 7f613f7fe640 1 -- 172.21.15.96:0/3095593152 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614403f030 con 0x7f6150101740 2024-04-25T14:11:35.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.167+0000 7f6154c15640 1 -- 172.21.15.96:0/3095593152 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61501a4320 con 0x7f6150101740 2024-04-25T14:11:35.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.167+0000 7f613f7fe640 1 -- 172.21.15.96:0/3095593152 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6144003aa0 con 0x7f6150101740 2024-04-25T14:11:35.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.167+0000 7f613f7fe640 1 -- 172.21.15.96:0/3095593152 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6144035070 con 0x7f6150101740 2024-04-25T14:11:35.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.167+0000 7f6154c15640 1 -- 172.21.15.96:0/3095593152 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61501a4780 con 0x7f6150101740 2024-04-25T14:11:35.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.168+0000 7f6154c15640 1 -- 172.21.15.96:0/3095593152 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6150108a20 con 0x7f6150101740 2024-04-25T14:11:35.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.169+0000 7f613f7fe640 1 -- 172.21.15.96:0/3095593152 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f6144042030 con 0x7f6150101740 2024-04-25T14:11:35.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.170+0000 7f613f7fe640 1 --2- 172.21.15.96:0/3095593152 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f61380afaf0 0x7f61380b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:35.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.170+0000 7f614dd9b640 1 --2- 172.21.15.96:0/3095593152 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f61380afaf0 0x7f61380b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:35.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.170+0000 7f613f7fe640 1 -- 172.21.15.96:0/3095593152 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7f61440ff580 con 0x7f6150101740 2024-04-25T14:11:35.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.170+0000 7f614dd9b640 1 --2- 172.21.15.96:0/3095593152 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f61380afaf0 0x7f61380b2000 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6150103be0 tx=0x7f6140005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:35.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.174+0000 7f613f7fe640 1 -- 172.21.15.96:0/3095593152 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f61440c82b0 con 0x7f6150101740 2024-04-25T14:11:35.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.389+0000 7f6154c15640 1 -- 172.21.15.96:0/3095593152 --> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f6150062fe0 con 0x7f61380afaf0 2024-04-25T14:11:35.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.395+0000 7f613f7fe640 1 -- 172.21.15.96:0/3095593152 <== mgr.14152 v2:172.21.15.96:6800/536332425 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f6150062fe0 con 0x7f61380afaf0 2024-04-25T14:11:35.395 INFO:teuthology.orchestra.run.smithi096.stdout:/volumes/qa/sv_1/358c30dc-ec4f-409a-a4ed-b22ff1971e3f 2024-04-25T14:11:35.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.397+0000 7f6154c15640 1 -- 172.21.15.96:0/3095593152 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f61380afaf0 msgr2=0x7f61380b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:35.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.397+0000 7f6154c15640 1 --2- 172.21.15.96:0/3095593152 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f61380afaf0 0x7f61380b2000 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6150103be0 tx=0x7f6140005e70 comp rx=0 tx=0).stop 2024-04-25T14:11:35.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.397+0000 7f6154c15640 1 -- 172.21.15.96:0/3095593152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6150101740 msgr2=0x7f6150104240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:35.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.397+0000 7f6154c15640 1 --2- 172.21.15.96:0/3095593152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6150101740 0x7f6150104240 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6144006010 tx=0x7f6144004270 comp rx=0 tx=0).stop 2024-04-25T14:11:35.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.397+0000 7f6154c15640 1 -- 172.21.15.96:0/3095593152 shutdown_connections 2024-04-25T14:11:35.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.397+0000 7f6154c15640 1 --2- 172.21.15.96:0/3095593152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6150105090 0x7f6150102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:35.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.397+0000 7f6154c15640 1 --2- 172.21.15.96:0/3095593152 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6150102240 0x7f61501028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:35.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.397+0000 7f6154c15640 1 --2- 172.21.15.96:0/3095593152 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f61380afaf0 0x7f61380b2000 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:35.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.397+0000 7f6154c15640 1 --2- 172.21.15.96:0/3095593152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6150101740 0x7f6150104240 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:35.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.397+0000 7f6154c15640 1 -- 172.21.15.96:0/3095593152 >> 172.21.15.96:0/3095593152 conn(0x7f61500fd340 msgr2=0x7f6150105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:35.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.397+0000 7f6154c15640 1 -- 172.21.15.96:0/3095593152 shutdown_connections 2024-04-25T14:11:35.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.397+0000 7f6154c15640 1 -- 172.21.15.96:0/3095593152 wait complete. 2024-04-25T14:11:35.408 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:11:35.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.518+0000 7f285bdb2640 1 -- 172.21.15.96:0/3798270020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2854101ec0 msgr2=0x7f2854106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:35.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.518+0000 7f285bdb2640 1 --2- 172.21.15.96:0/3798270020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2854101ec0 0x7f2854106b80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2850009900 tx=0x7f285002a410 comp rx=0 tx=0).stop 2024-04-25T14:11:35.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.518+0000 7f285bdb2640 1 -- 172.21.15.96:0/3798270020 shutdown_connections 2024-04-25T14:11:35.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.518+0000 7f285bdb2640 1 --2- 172.21.15.96:0/3798270020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28541097e0 0x7f285410bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:35.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.518+0000 7f285bdb2640 1 --2- 172.21.15.96:0/3798270020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28541023c0 0x7f2854109150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:35.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.518+0000 7f285bdb2640 1 --2- 172.21.15.96:0/3798270020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2854101ec0 0x7f2854106b80 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:35.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.518+0000 7f285bdb2640 1 -- 172.21.15.96:0/3798270020 >> 172.21.15.96:0/3798270020 conn(0x7f28540fd3b0 msgr2=0x7f28540ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:35.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.518+0000 7f285bdb2640 1 -- 172.21.15.96:0/3798270020 shutdown_connections 2024-04-25T14:11:35.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.518+0000 7f285bdb2640 1 -- 172.21.15.96:0/3798270020 wait complete. 2024-04-25T14:11:35.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.519+0000 7f285bdb2640 1 Processor -- start 2024-04-25T14:11:35.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.519+0000 7f285bdb2640 1 -- start start 2024-04-25T14:11:35.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.519+0000 7f285bdb2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2854101ec0 0x7f285419a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:35.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.519+0000 7f285bdb2640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28541023c0 0x7f285419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:35.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.519+0000 7f285bdb2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28541097e0 0x7f28541a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:35.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.519+0000 7f285bdb2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f285410f5d0 con 0x7f2854101ec0 2024-04-25T14:11:35.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.519+0000 7f285bdb2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f285410f450 con 0x7f28541097e0 2024-04-25T14:11:35.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.519+0000 7f285bdb2640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f285410f750 con 0x7f28541023c0 2024-04-25T14:11:35.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.519+0000 7f2859b4e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2854101ec0 0x7f285419a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:35.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.519+0000 7f2859b4e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2854101ec0 0x7f285419a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35388/0 (socket says 172.21.15.96:35388) 2024-04-25T14:11:35.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.519+0000 7f2859b4e640 1 -- 172.21.15.96:0/4249911663 learned_addr learned my addr 172.21.15.96:0/4249911663 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:11:35.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.519+0000 7f285a34f640 1 --2- 172.21.15.96:0/4249911663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28541097e0 0x7f28541a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:35.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.519+0000 7f285934d640 1 --2- 172.21.15.96:0/4249911663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28541023c0 0x7f285419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:35.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.520+0000 7f2859b4e640 1 -- 172.21.15.96:0/4249911663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28541023c0 msgr2=0x7f285419a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:35.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.520+0000 7f2859b4e640 1 --2- 172.21.15.96:0/4249911663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28541023c0 0x7f285419a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:35.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.520+0000 7f2859b4e640 1 -- 172.21.15.96:0/4249911663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28541097e0 msgr2=0x7f28541a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:35.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.520+0000 7f2859b4e640 1 --2- 172.21.15.96:0/4249911663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28541097e0 0x7f28541a16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:35.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.520+0000 7f2859b4e640 1 -- 172.21.15.96:0/4249911663 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28541a1d10 con 0x7f2854101ec0 2024-04-25T14:11:35.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.520+0000 7f285934d640 1 --2- 172.21.15.96:0/4249911663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28541023c0 0x7f285419a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:35.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.520+0000 7f2859b4e640 1 --2- 172.21.15.96:0/4249911663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2854101ec0 0x7f285419a1d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2850006010 tx=0x7f28500042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:35.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.520+0000 7f284affd640 1 -- 172.21.15.96:0/4249911663 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285003f030 con 0x7f2854101ec0 2024-04-25T14:11:35.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.520+0000 7f284affd640 1 -- 172.21.15.96:0/4249911663 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2850031910 con 0x7f2854101ec0 2024-04-25T14:11:35.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.520+0000 7f285bdb2640 1 -- 172.21.15.96:0/4249911663 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28541a1fa0 con 0x7f2854101ec0 2024-04-25T14:11:35.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.520+0000 7f284affd640 1 -- 172.21.15.96:0/4249911663 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2850035070 con 0x7f2854101ec0 2024-04-25T14:11:35.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.520+0000 7f285bdb2640 1 -- 172.21.15.96:0/4249911663 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28541a23b0 con 0x7f2854101ec0 2024-04-25T14:11:35.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.521+0000 7f285bdb2640 1 -- 172.21.15.96:0/4249911663 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28540feed0 con 0x7f2854101ec0 2024-04-25T14:11:35.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.522+0000 7f284affd640 1 -- 172.21.15.96:0/4249911663 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f2850042030 con 0x7f2854101ec0 2024-04-25T14:11:35.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.523+0000 7f284affd640 1 --2- 172.21.15.96:0/4249911663 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f28400afaf0 0x7f28400b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:35.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.523+0000 7f285934d640 1 --2- 172.21.15.96:0/4249911663 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f28400afaf0 0x7f28400b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:35.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.523+0000 7f284affd640 1 -- 172.21.15.96:0/4249911663 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7f28500ff740 con 0x7f2854101ec0 2024-04-25T14:11:35.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.523+0000 7f285934d640 1 --2- 172.21.15.96:0/4249911663 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f28400afaf0 0x7f28400b2000 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f285419b270 tx=0x7f2844005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:35.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.527+0000 7f284affd640 1 -- 172.21.15.96:0/4249911663 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f28500c8470 con 0x7f2854101ec0 2024-04-25T14:11:35.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.594+0000 7f284affd640 1 -- 172.21.15.96:0/4249911663 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f28500c4000 con 0x7f2854101ec0 2024-04-25T14:11:35.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.757+0000 7f285bdb2640 1 -- 172.21.15.96:0/4249911663 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f285402dc70 con 0x7f2854101ec0 2024-04-25T14:11:35.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.758+0000 7f284affd640 1 -- 172.21.15.96:0/4249911663 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11927 (secure 0 0 0) 0x7f285004d3e0 con 0x7f2854101ec0 2024-04-25T14:11:35.758 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:11:35.758 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/606227917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":606227917},{"type":"v1","addr":"172.21.15.96:6835","nonce":606227917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/3265300710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3265300710},{"type":"v1","addr":"172.21.15.96:6837","nonce":3265300710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T14:11:28.886961+0000","modified":"2024-04-25T14:11:35.532775+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24517,"mds_2":24478,"mds_3":24455,"mds_4":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/4080308509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":4080308509},{"type":"v1","addr":"172.21.15.96:6839","nonce":4080308509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/1606748971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":1606748971},{"type":"v1","addr":"172.21.15.96:6841","nonce":1606748971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6833/1995324831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1995324831},{"type":"v1","addr":"172.21.15.97:6833","nonce":1995324831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":4,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.151:6833/1382721608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1382721608},{"type":"v1","addr":"172.21.15.151:6833","nonce":1382721608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6835/3872301726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3872301726},{"type":"v1","addr":"172.21.15.97:6835","nonce":3872301726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.151:6835/1313823638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":1313823638},{"type":"v1","addr":"172.21.15.151:6835","nonce":1313823638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/2691348366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2691348366},{"type":"v1","addr":"172.21.15.151:6837","nonce":2691348366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3306780081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3306780081},{"type":"v1","addr":"172.21.15.97:6837","nonce":3306780081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.151:6839/245699981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":245699981},{"type":"v1","addr":"172.21.15.151:6839","nonce":245699981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6839/2850442269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":2850442269},{"type":"v1","addr":"172.21.15.97:6839","nonce":2850442269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24517,24536,24478,24455,24427]},"id":1}]} 2024-04-25T14:11:35.758 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 25 2024-04-25T14:11:35.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.760+0000 7f285bdb2640 1 -- 172.21.15.96:0/4249911663 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f28400afaf0 msgr2=0x7f28400b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:35.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.760+0000 7f285bdb2640 1 --2- 172.21.15.96:0/4249911663 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f28400afaf0 0x7f28400b2000 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f285419b270 tx=0x7f2844005f90 comp rx=0 tx=0).stop 2024-04-25T14:11:35.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.760+0000 7f285bdb2640 1 -- 172.21.15.96:0/4249911663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2854101ec0 msgr2=0x7f285419a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:35.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.760+0000 7f285bdb2640 1 --2- 172.21.15.96:0/4249911663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2854101ec0 0x7f285419a1d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2850006010 tx=0x7f28500042f0 comp rx=0 tx=0).stop 2024-04-25T14:11:35.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.760+0000 7f285bdb2640 1 -- 172.21.15.96:0/4249911663 shutdown_connections 2024-04-25T14:11:35.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.760+0000 7f285bdb2640 1 --2- 172.21.15.96:0/4249911663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28541097e0 0x7f28541a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:35.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.760+0000 7f285bdb2640 1 --2- 172.21.15.96:0/4249911663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28541023c0 0x7f285419a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:35.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.760+0000 7f285bdb2640 1 --2- 172.21.15.96:0/4249911663 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f28400afaf0 0x7f28400b2000 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:35.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.760+0000 7f285bdb2640 1 --2- 172.21.15.96:0/4249911663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2854101ec0 0x7f285419a1d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:35.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.760+0000 7f285bdb2640 1 -- 172.21.15.96:0/4249911663 >> 172.21.15.96:0/4249911663 conn(0x7f28540fd3b0 msgr2=0x7f2854105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:35.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.760+0000 7f285bdb2640 1 -- 172.21.15.96:0/4249911663 shutdown_connections 2024-04-25T14:11:35.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.760+0000 7f285bdb2640 1 -- 172.21.15.96:0/4249911663 wait complete. 2024-04-25T14:11:35.771 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:11:35.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.878+0000 7f89360d9640 1 -- 172.21.15.96:0/3146609474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8930109a40 msgr2=0x7f893010be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:35.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.878+0000 7f89360d9640 1 --2- 172.21.15.96:0/3146609474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8930109a40 0x7f893010be60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8928009ca0 tx=0x7f892802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:11:35.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.878+0000 7f89360d9640 1 -- 172.21.15.96:0/3146609474 shutdown_connections 2024-04-25T14:11:35.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.878+0000 7f89360d9640 1 --2- 172.21.15.96:0/3146609474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8930109a40 0x7f893010be60 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:35.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.878+0000 7f89360d9640 1 --2- 172.21.15.96:0/3146609474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8930104d40 0x7f8930105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:35.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.878+0000 7f89360d9640 1 --2- 172.21.15.96:0/3146609474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8930103b00 0x7f8930104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:35.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.878+0000 7f89360d9640 1 -- 172.21.15.96:0/3146609474 >> 172.21.15.96:0/3146609474 conn(0x7f89300fed30 msgr2=0x7f89301011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:35.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.878+0000 7f89360d9640 1 -- 172.21.15.96:0/3146609474 shutdown_connections 2024-04-25T14:11:35.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.878+0000 7f89360d9640 1 -- 172.21.15.96:0/3146609474 wait complete. 2024-04-25T14:11:35.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.878+0000 7f89360d9640 1 Processor -- start 2024-04-25T14:11:35.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.879+0000 7f89360d9640 1 -- start start 2024-04-25T14:11:35.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.879+0000 7f89360d9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8930103b00 0x7f893019a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:35.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.879+0000 7f89360d9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8930104d40 0x7f893019a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:35.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.879+0000 7f89360d9640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8930109a40 0x7f89301a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:35.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.879+0000 7f89360d9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f893010f630 con 0x7f8930104d40 2024-04-25T14:11:35.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.879+0000 7f89360d9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f893010f4b0 con 0x7f8930103b00 2024-04-25T14:11:35.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.879+0000 7f89360d9640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f893010f7b0 con 0x7f8930109a40 2024-04-25T14:11:35.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.879+0000 7f892effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8930104d40 0x7f893019a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:35.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.879+0000 7f892effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8930104d40 0x7f893019a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35412/0 (socket says 172.21.15.96:35412) 2024-04-25T14:11:35.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.879+0000 7f892effd640 1 -- 172.21.15.96:0/3462612178 learned_addr learned my addr 172.21.15.96:0/3462612178 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:11:35.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.879+0000 7f892f7fe640 1 --2- 172.21.15.96:0/3462612178 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8930103b00 0x7f893019a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:35.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.879+0000 7f892ffff640 1 --2- 172.21.15.96:0/3462612178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8930109a40 0x7f89301a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:35.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.879+0000 7f892effd640 1 -- 172.21.15.96:0/3462612178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8930109a40 msgr2=0x7f89301a1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:35.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.879+0000 7f892effd640 1 --2- 172.21.15.96:0/3462612178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8930109a40 0x7f89301a1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:35.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.879+0000 7f892effd640 1 -- 172.21.15.96:0/3462612178 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8930103b00 msgr2=0x7f893019a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:35.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.879+0000 7f892effd640 1 --2- 172.21.15.96:0/3462612178 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8930103b00 0x7f893019a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:35.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.879+0000 7f892effd640 1 -- 172.21.15.96:0/3462612178 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89301a1c80 con 0x7f8930104d40 2024-04-25T14:11:35.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.879+0000 7f892effd640 1 --2- 172.21.15.96:0/3462612178 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8930104d40 0x7f893019a680 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f892400c970 tx=0x7f892400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:35.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.879+0000 7f892cff9640 1 -- 172.21.15.96:0/3462612178 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8924007bb0 con 0x7f8930104d40 2024-04-25T14:11:35.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.879+0000 7f892cff9640 1 -- 172.21.15.96:0/3462612178 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8924007d50 con 0x7f8930104d40 2024-04-25T14:11:35.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.879+0000 7f89360d9640 1 -- 172.21.15.96:0/3462612178 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89301a1f70 con 0x7f8930104d40 2024-04-25T14:11:35.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.879+0000 7f892cff9640 1 -- 172.21.15.96:0/3462612178 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892400f450 con 0x7f8930104d40 2024-04-25T14:11:35.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.879+0000 7f89360d9640 1 -- 172.21.15.96:0/3462612178 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89301a23d0 con 0x7f8930104d40 2024-04-25T14:11:35.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.880+0000 7f89360d9640 1 -- 172.21.15.96:0/3462612178 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8930108850 con 0x7f8930104d40 2024-04-25T14:11:35.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.882+0000 7f892cff9640 1 -- 172.21.15.96:0/3462612178 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f8924004f00 con 0x7f8930104d40 2024-04-25T14:11:35.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.882+0000 7f892cff9640 1 --2- 172.21.15.96:0/3462612178 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f89180afb80 0x7f89180b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:35.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.882+0000 7f892f7fe640 1 --2- 172.21.15.96:0/3462612178 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f89180afb80 0x7f89180b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:35.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.882+0000 7f892cff9640 1 -- 172.21.15.96:0/3462612178 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7f89240ded70 con 0x7f8930104d40 2024-04-25T14:11:35.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.883+0000 7f892f7fe640 1 --2- 172.21.15.96:0/3462612178 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f89180afb80 0x7f89180b2090 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8930104700 tx=0x7f8920005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:35.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:35.886+0000 7f892cff9640 1 -- 172.21.15.96:0/3462612178 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8924014030 con 0x7f8930104d40 2024-04-25T14:11:36.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.063+0000 7f89360d9640 1 -- 172.21.15.96:0/3462612178 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f89300637d0 con 0x7f8930104d40 2024-04-25T14:11:36.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.065+0000 7f892cff9640 1 -- 172.21.15.96:0/3462612178 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+22146 (secure 0 0 0) 0x7f89240a79a0 con 0x7f8930104d40 2024-04-25T14:11:36.065 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:11:36.065 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":76,"fsid":"59334182-030c-11ef-bc93-c7b262605968","created":"2024-04-25T14:03:25.680745+0000","modified":"2024-04-25T14:11:30.552550+0000","last_up_change":"2024-04-25T14:10:35.894312+0000","last_in_change":"2024-04-25T14:10:15.583440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:06:47.402927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:11:27.434331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:11:27.854004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cf4ba3ac-142e-4e8e-96d4-410d77157eb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6803","nonce":3832506168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6805","nonce":3832506168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832506168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6807","nonce":3832506168}]},"public_addr":"172.21.15.96:6803/3832506168","cluster_addr":"172.21.15.96:6805/3832506168","heartbeat_back_addr":"172.21.15.96:6809/3832506168","heartbeat_front_addr":"172.21.15.96:6807/3832506168","state":["exists","up"]},{"osd":1,"uuid":"866e73b9-4a26-48ba-badf-704fe22f71cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6801","nonce":1083311494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6803","nonce":1083311494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6807","nonce":1083311494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6805","nonce":1083311494}]},"public_addr":"172.21.15.97:6801/1083311494","cluster_addr":"172.21.15.97:6803/1083311494","heartbeat_back_addr":"172.21.15.97:6807/1083311494","heartbeat_front_addr":"172.21.15.97:6805/1083311494","state":["exists","up"]},{"osd":2,"uuid":"45c79ecb-abc4-4dba-bae8-74804e4bc1bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6801","nonce":981902966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6803","nonce":981902966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6807","nonce":981902966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6805","nonce":981902966}]},"public_addr":"172.21.15.151:6801/981902966","cluster_addr":"172.21.15.151:6803/981902966","heartbeat_back_addr":"172.21.15.151:6807/981902966","heartbeat_front_addr":"172.21.15.151:6805/981902966","state":["exists","up"]},{"osd":3,"uuid":"9fc39468-a7ea-4ccf-b365-335ad2ea4c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6811","nonce":2274158748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6813","nonce":2274158748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6817","nonce":2274158748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6815","nonce":2274158748}]},"public_addr":"172.21.15.96:6811/2274158748","cluster_addr":"172.21.15.96:6813/2274158748","heartbeat_back_addr":"172.21.15.96:6817/2274158748","heartbeat_front_addr":"172.21.15.96:6815/2274158748","state":["exists","up"]},{"osd":4,"uuid":"616582f1-12d2-456b-b7b3-e5517800e8a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6809","nonce":3727031962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6811","nonce":3727031962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6815","nonce":3727031962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6813","nonce":3727031962}]},"public_addr":"172.21.15.97:6809/3727031962","cluster_addr":"172.21.15.97:6811/3727031962","heartbeat_back_addr":"172.21.15.97:6815/3727031962","heartbeat_front_addr":"172.21.15.97:6813/3727031962","state":["exists","up"]},{"osd":5,"uuid":"3d277784-ab0f-4c90-81a1-2a4034c52a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6809","nonce":223882546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6811","nonce":223882546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6815","nonce":223882546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6813","nonce":223882546}]},"public_addr":"172.21.15.151:6809/223882546","cluster_addr":"172.21.15.151:6811/223882546","heartbeat_back_addr":"172.21.15.151:6815/223882546","heartbeat_front_addr":"172.21.15.151:6813/223882546","state":["exists","up"]},{"osd":6,"uuid":"9a9ad585-cb1a-4239-9d18-055f59164049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6819","nonce":96197407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6821","nonce":96197407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6825","nonce":96197407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6823","nonce":96197407}]},"public_addr":"172.21.15.96:6819/96197407","cluster_addr":"172.21.15.96:6821/96197407","heartbeat_back_addr":"172.21.15.96:6825/96197407","heartbeat_front_addr":"172.21.15.96:6823/96197407","state":["exists","up"]},{"osd":7,"uuid":"303551c8-6af0-42f2-844f-45bf8db5baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6817","nonce":2247216400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6819","nonce":2247216400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6823","nonce":2247216400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6821","nonce":2247216400}]},"public_addr":"172.21.15.97:6817/2247216400","cluster_addr":"172.21.15.97:6819/2247216400","heartbeat_back_addr":"172.21.15.97:6823/2247216400","heartbeat_front_addr":"172.21.15.97:6821/2247216400","state":["exists","up"]},{"osd":8,"uuid":"81f6cc66-e4ca-430c-b336-b6678d968818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6817","nonce":2549822185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6819","nonce":2549822185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6823","nonce":2549822185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6821","nonce":2549822185}]},"public_addr":"172.21.15.151:6817/2549822185","cluster_addr":"172.21.15.151:6819/2549822185","heartbeat_back_addr":"172.21.15.151:6823/2549822185","heartbeat_front_addr":"172.21.15.151:6821/2549822185","state":["exists","up"]},{"osd":9,"uuid":"29d4da16-feab-4d1c-9d1d-de4f799bd724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6827","nonce":1099399643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6829","nonce":1099399643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6833","nonce":1099399643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6831","nonce":1099399643}]},"public_addr":"172.21.15.96:6827/1099399643","cluster_addr":"172.21.15.96:6829/1099399643","heartbeat_back_addr":"172.21.15.96:6833/1099399643","heartbeat_front_addr":"172.21.15.96:6831/1099399643","state":["exists","up"]},{"osd":10,"uuid":"197fee9b-eee4-4893-be1c-3d57f42250e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6825","nonce":155754520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6827","nonce":155754520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6831","nonce":155754520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6829","nonce":155754520}]},"public_addr":"172.21.15.97:6825/155754520","cluster_addr":"172.21.15.97:6827/155754520","heartbeat_back_addr":"172.21.15.97:6831/155754520","heartbeat_front_addr":"172.21.15.97:6829/155754520","state":["exists","up"]},{"osd":11,"uuid":"9c1d0a83-fc12-41a2-8eb0-80c4abdd2770","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6825","nonce":2027958746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6827","nonce":2027958746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6831","nonce":2027958746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6829","nonce":2027958746}]},"public_addr":"172.21.15.151:6825/2027958746","cluster_addr":"172.21.15.151:6827/2027958746","heartbeat_back_addr":"172.21.15.151:6831/2027958746","heartbeat_front_addr":"172.21.15.151:6829/2027958746","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:05:56.177390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:06:19.917663+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:06:43.608052+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:08.540382+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:33.474072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:58.242306+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:08:24.120442+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:08:49.661064+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:09:15.097271+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:09:41.904505+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:10:08.233252+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:10:33.712296+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/3839807484":"2024-04-26T14:04:21.307574+0000","172.21.15.96:0/1495237551":"2024-04-26T14:04:21.307574+0000","172.21.15.96:0/2793751450":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/129575834":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/2233075139":"2024-04-26T14:04:21.307574+0000","172.21.15.96:6801/2348397481":"2024-04-26T14:03:54.249537+0000","172.21.15.96:6801/433591062":"2024-04-26T14:04:21.307574+0000","172.21.15.96:6800/2348397481":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/3517674924":"2024-04-26T14:03:54.249537+0000","172.21.15.96:6800/433591062":"2024-04-26T14:04:21.307574+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:11:36.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.067+0000 7f89360d9640 1 -- 172.21.15.96:0/3462612178 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f89180afb80 msgr2=0x7f89180b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:36.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.067+0000 7f89360d9640 1 --2- 172.21.15.96:0/3462612178 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f89180afb80 0x7f89180b2090 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8930104700 tx=0x7f8920005f90 comp rx=0 tx=0).stop 2024-04-25T14:11:36.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.067+0000 7f89360d9640 1 -- 172.21.15.96:0/3462612178 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8930104d40 msgr2=0x7f893019a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:36.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.067+0000 7f89360d9640 1 --2- 172.21.15.96:0/3462612178 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8930104d40 0x7f893019a680 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f892400c970 tx=0x7f892400ce30 comp rx=0 tx=0).stop 2024-04-25T14:11:36.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.067+0000 7f89360d9640 1 -- 172.21.15.96:0/3462612178 shutdown_connections 2024-04-25T14:11:36.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.067+0000 7f89360d9640 1 --2- 172.21.15.96:0/3462612178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8930109a40 0x7f89301a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:36.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.067+0000 7f89360d9640 1 --2- 172.21.15.96:0/3462612178 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f89180afb80 0x7f89180b2090 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:36.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.067+0000 7f89360d9640 1 --2- 172.21.15.96:0/3462612178 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8930104d40 0x7f893019a680 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:36.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.067+0000 7f89360d9640 1 --2- 172.21.15.96:0/3462612178 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8930103b00 0x7f893019a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:36.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.067+0000 7f89360d9640 1 -- 172.21.15.96:0/3462612178 >> 172.21.15.96:0/3462612178 conn(0x7f89300fed30 msgr2=0x7f8930105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:36.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.067+0000 7f89360d9640 1 -- 172.21.15.96:0/3462612178 shutdown_connections 2024-04-25T14:11:36.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.067+0000 7f89360d9640 1 -- 172.21.15.96:0/3462612178 wait complete. 2024-04-25T14:11:36.079 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T14:11:36.079 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:11:36.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.188+0000 7fc5d398f640 1 -- 172.21.15.96:0/1278372661 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5cc109a40 msgr2=0x7fc5cc10be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:36.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.188+0000 7fc5d398f640 1 --2- 172.21.15.96:0/1278372661 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5cc109a40 0x7fc5cc10be60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4009ca0 tx=0x7fc5c402a8c0 comp rx=0 tx=0).stop 2024-04-25T14:11:36.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.188+0000 7fc5d398f640 1 -- 172.21.15.96:0/1278372661 shutdown_connections 2024-04-25T14:11:36.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.188+0000 7fc5d398f640 1 --2- 172.21.15.96:0/1278372661 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5cc109a40 0x7fc5cc10be60 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:36.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.188+0000 7fc5d398f640 1 --2- 172.21.15.96:0/1278372661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5cc104d40 0x7fc5cc105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:36.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.188+0000 7fc5d398f640 1 --2- 172.21.15.96:0/1278372661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5cc103b00 0x7fc5cc104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:36.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.188+0000 7fc5d398f640 1 -- 172.21.15.96:0/1278372661 >> 172.21.15.96:0/1278372661 conn(0x7fc5cc0fed10 msgr2=0x7fc5cc1011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:36.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.188+0000 7fc5d398f640 1 -- 172.21.15.96:0/1278372661 shutdown_connections 2024-04-25T14:11:36.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.188+0000 7fc5d398f640 1 -- 172.21.15.96:0/1278372661 wait complete. 2024-04-25T14:11:36.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.189+0000 7fc5d398f640 1 Processor -- start 2024-04-25T14:11:36.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.189+0000 7fc5d398f640 1 -- start start 2024-04-25T14:11:36.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.189+0000 7fc5d398f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5cc103b00 0x7fc5cc19a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:36.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.189+0000 7fc5d398f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5cc104d40 0x7fc5cc19a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:36.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.189+0000 7fc5d398f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5cc109a40 0x7fc5cc1a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:36.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.189+0000 7fc5d398f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc5cc10f680 con 0x7fc5cc103b00 2024-04-25T14:11:36.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.189+0000 7fc5d398f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc5cc10f500 con 0x7fc5cc104d40 2024-04-25T14:11:36.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.189+0000 7fc5d398f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc5cc10f800 con 0x7fc5cc109a40 2024-04-25T14:11:36.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.189+0000 7fc5d172b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5cc103b00 0x7fc5cc19a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:36.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.189+0000 7fc5d172b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5cc103b00 0x7fc5cc19a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35426/0 (socket says 172.21.15.96:35426) 2024-04-25T14:11:36.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.189+0000 7fc5d172b640 1 -- 172.21.15.96:0/190199286 learned_addr learned my addr 172.21.15.96:0/190199286 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:11:36.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.189+0000 7fc5d0f2a640 1 --2- 172.21.15.96:0/190199286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5cc104d40 0x7fc5cc19a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:36.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.189+0000 7fc5d1f2c640 1 --2- 172.21.15.96:0/190199286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5cc109a40 0x7fc5cc1a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:36.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.189+0000 7fc5d172b640 1 -- 172.21.15.96:0/190199286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5cc109a40 msgr2=0x7fc5cc1a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:36.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.189+0000 7fc5d172b640 1 --2- 172.21.15.96:0/190199286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5cc109a40 0x7fc5cc1a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:36.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.189+0000 7fc5d172b640 1 -- 172.21.15.96:0/190199286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5cc104d40 msgr2=0x7fc5cc19a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:36.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.189+0000 7fc5d172b640 1 --2- 172.21.15.96:0/190199286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5cc104d40 0x7fc5cc19a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:36.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.189+0000 7fc5d172b640 1 -- 172.21.15.96:0/190199286 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5cc1a1d80 con 0x7fc5cc103b00 2024-04-25T14:11:36.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.189+0000 7fc5d1f2c640 1 --2- 172.21.15.96:0/190199286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5cc109a40 0x7fc5cc1a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:36.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.189+0000 7fc5d172b640 1 --2- 172.21.15.96:0/190199286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5cc103b00 0x7fc5cc19a240 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800e980 tx=0x7fc5c800ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:36.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.190+0000 7fc5c27fc640 1 -- 172.21.15.96:0/190199286 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c800ccc0 con 0x7fc5cc103b00 2024-04-25T14:11:36.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.190+0000 7fc5d398f640 1 -- 172.21.15.96:0/190199286 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5cc1a2070 con 0x7fc5cc103b00 2024-04-25T14:11:36.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.190+0000 7fc5c27fc640 1 -- 172.21.15.96:0/190199286 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5c80108e0 con 0x7fc5cc103b00 2024-04-25T14:11:36.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.190+0000 7fc5c27fc640 1 -- 172.21.15.96:0/190199286 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c80187d0 con 0x7fc5cc103b00 2024-04-25T14:11:36.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.190+0000 7fc5d398f640 1 -- 172.21.15.96:0/190199286 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5cc1a2690 con 0x7fc5cc103b00 2024-04-25T14:11:36.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.191+0000 7fc5d398f640 1 -- 172.21.15.96:0/190199286 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5cc06a190 con 0x7fc5cc103b00 2024-04-25T14:11:36.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.192+0000 7fc5c27fc640 1 -- 172.21.15.96:0/190199286 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fc5c801f020 con 0x7fc5cc103b00 2024-04-25T14:11:36.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.192+0000 7fc5c27fc640 1 --2- 172.21.15.96:0/190199286 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fc5b80afb80 0x7fc5b80b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:36.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.193+0000 7fc5d0f2a640 1 --2- 172.21.15.96:0/190199286 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fc5b80afb80 0x7fc5b80b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:36.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.193+0000 7fc5c27fc640 1 -- 172.21.15.96:0/190199286 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7fc5c80df320 con 0x7fc5cc103b00 2024-04-25T14:11:36.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.193+0000 7fc5d0f2a640 1 --2- 172.21.15.96:0/190199286 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fc5b80afb80 0x7fc5b80b2090 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc19b2e0 tx=0x7fc5bc009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:36.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.196+0000 7fc5c27fc640 1 -- 172.21.15.96:0/190199286 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc5c80a7f50 con 0x7fc5cc103b00 2024-04-25T14:11:36.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.369+0000 7fc5d398f640 1 -- 172.21.15.96:0/190199286 --> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc5cc062fe0 con 0x7fc5b80afb80 2024-04-25T14:11:36.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.377+0000 7fc5c27fc640 1 -- 172.21.15.96:0/190199286 <== mgr.14152 v2:172.21.15.96:6800/536332425 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258903 (secure 0 0 0) 0x7fc5cc062fe0 con 0x7fc5b80afb80 2024-04-25T14:11:36.380 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:11:36.382 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-04-25T14:11:36.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.380+0000 7fc5d398f640 1 -- 172.21.15.96:0/190199286 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fc5b80afb80 msgr2=0x7fc5b80b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:36.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.380+0000 7fc5d398f640 1 --2- 172.21.15.96:0/190199286 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fc5b80afb80 0x7fc5b80b2090 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc19b2e0 tx=0x7fc5bc009110 comp rx=0 tx=0).stop 2024-04-25T14:11:36.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.380+0000 7fc5d398f640 1 -- 172.21.15.96:0/190199286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5cc103b00 msgr2=0x7fc5cc19a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:36.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.380+0000 7fc5d398f640 1 --2- 172.21.15.96:0/190199286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5cc103b00 0x7fc5cc19a240 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800e980 tx=0x7fc5c800ed80 comp rx=0 tx=0).stop 2024-04-25T14:11:36.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.380+0000 7fc5d398f640 1 -- 172.21.15.96:0/190199286 shutdown_connections 2024-04-25T14:11:36.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.380+0000 7fc5d398f640 1 --2- 172.21.15.96:0/190199286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5cc109a40 0x7fc5cc1a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:36.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.380+0000 7fc5d398f640 1 --2- 172.21.15.96:0/190199286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5cc104d40 0x7fc5cc19a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:36.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.380+0000 7fc5d398f640 1 --2- 172.21.15.96:0/190199286 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fc5b80afb80 0x7fc5b80b2090 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:36.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.380+0000 7fc5d398f640 1 --2- 172.21.15.96:0/190199286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5cc103b00 0x7fc5cc19a240 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:36.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.380+0000 7fc5d398f640 1 -- 172.21.15.96:0/190199286 >> 172.21.15.96:0/190199286 conn(0x7fc5cc0fed10 msgr2=0x7fc5cc105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:36.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.380+0000 7fc5d398f640 1 -- 172.21.15.96:0/190199286 shutdown_connections 2024-04-25T14:11:36.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.380+0000 7fc5d398f640 1 -- 172.21.15.96:0/190199286 wait complete. 2024-04-25T14:11:36.391 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":283,"stamp":"2024-04-25T14:11:35.399091+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592912,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":203,"num_write_kb":2173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":221,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":358912,"kb_used_data":9692,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124582912,"statfs":{"total":1151940427776,"available":1151572901888,"internally_reserved":0,"allocated":9924608,"data_stored":6379088,"data_compressed":107307,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2544,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":80,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.872230"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.565759+0000","last_change":"2024-04-25T14:11:29.569615+0000","last_active":"2024-04-25T14:11:30.565759+0000","last_peered":"2024-04-25T14:11:30.565759+0000","last_clean":"2024-04-25T14:11:30.565759+0000","last_became_active":"2024-04-25T14:11:29.569237+0000","last_became_peered":"2024-04-25T14:11:29.569237+0000","last_unstale":"2024-04-25T14:11:30.565759+0000","last_undegraded":"2024-04-25T14:11:30.565759+0000","last_fullsized":"2024-04-25T14:11:30.565759+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:24.367533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.637236+0000","last_change":"2024-04-25T14:11:29.571820+0000","last_active":"2024-04-25T14:11:30.637236+0000","last_peered":"2024-04-25T14:11:30.637236+0000","last_clean":"2024-04-25T14:11:30.637236+0000","last_became_active":"2024-04-25T14:11:29.571456+0000","last_became_peered":"2024-04-25T14:11:29.571456+0000","last_unstale":"2024-04-25T14:11:30.637236+0000","last_undegraded":"2024-04-25T14:11:30.637236+0000","last_fullsized":"2024-04-25T14:11:30.637236+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:33.442301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.637007+0000","last_change":"2024-04-25T14:11:29.579302+0000","last_active":"2024-04-25T14:11:30.637007+0000","last_peered":"2024-04-25T14:11:30.637007+0000","last_clean":"2024-04-25T14:11:30.637007+0000","last_became_active":"2024-04-25T14:11:29.577734+0000","last_became_peered":"2024-04-25T14:11:29.577734+0000","last_unstale":"2024-04-25T14:11:30.637007+0000","last_undegraded":"2024-04-25T14:11:30.637007+0000","last_fullsized":"2024-04-25T14:11:30.637007+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:59.147554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.637447+0000","last_change":"2024-04-25T14:11:29.574942+0000","last_active":"2024-04-25T14:11:30.637447+0000","last_peered":"2024-04-25T14:11:30.637447+0000","last_clean":"2024-04-25T14:11:30.637447+0000","last_became_active":"2024-04-25T14:11:29.573774+0000","last_became_peered":"2024-04-25T14:11:29.573774+0000","last_unstale":"2024-04-25T14:11:30.637447+0000","last_undegraded":"2024-04-25T14:11:30.637447+0000","last_fullsized":"2024-04-25T14:11:30.637447+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:10.626265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.637063+0000","last_change":"2024-04-25T14:11:29.557199+0000","last_active":"2024-04-25T14:11:30.637063+0000","last_peered":"2024-04-25T14:11:30.637063+0000","last_clean":"2024-04-25T14:11:30.637063+0000","last_became_active":"2024-04-25T14:11:29.556836+0000","last_became_peered":"2024-04-25T14:11:29.556836+0000","last_unstale":"2024-04-25T14:11:30.637063+0000","last_undegraded":"2024-04-25T14:11:30.637063+0000","last_fullsized":"2024-04-25T14:11:30.637063+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:04:45.882935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.563277+0000","last_change":"2024-04-25T14:11:29.557988+0000","last_active":"2024-04-25T14:11:30.563277+0000","last_peered":"2024-04-25T14:11:30.563277+0000","last_clean":"2024-04-25T14:11:30.563277+0000","last_became_active":"2024-04-25T14:11:29.557622+0000","last_became_peered":"2024-04-25T14:11:29.557622+0000","last_unstale":"2024-04-25T14:11:30.563277+0000","last_undegraded":"2024-04-25T14:11:30.563277+0000","last_fullsized":"2024-04-25T14:11:30.563277+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:28:37.578368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.785804+0000","last_change":"2024-04-25T14:11:29.565049+0000","last_active":"2024-04-25T14:11:30.785804+0000","last_peered":"2024-04-25T14:11:30.785804+0000","last_clean":"2024-04-25T14:11:30.785804+0000","last_became_active":"2024-04-25T14:11:29.564688+0000","last_became_peered":"2024-04-25T14:11:29.564688+0000","last_unstale":"2024-04-25T14:11:30.785804+0000","last_undegraded":"2024-04-25T14:11:30.785804+0000","last_fullsized":"2024-04-25T14:11:30.785804+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:48.128778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.562584+0000","last_change":"2024-04-25T14:11:29.565879+0000","last_active":"2024-04-25T14:11:30.562584+0000","last_peered":"2024-04-25T14:11:30.562584+0000","last_clean":"2024-04-25T14:11:30.562584+0000","last_became_active":"2024-04-25T14:11:29.565525+0000","last_became_peered":"2024-04-25T14:11:29.565525+0000","last_unstale":"2024-04-25T14:11:30.562584+0000","last_undegraded":"2024-04-25T14:11:30.562584+0000","last_fullsized":"2024-04-25T14:11:30.562584+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:06.956334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.637103+0000","last_change":"2024-04-25T14:11:29.567280+0000","last_active":"2024-04-25T14:11:30.637103+0000","last_peered":"2024-04-25T14:11:30.637103+0000","last_clean":"2024-04-25T14:11:30.637103+0000","last_became_active":"2024-04-25T14:11:29.566741+0000","last_became_peered":"2024-04-25T14:11:29.566741+0000","last_unstale":"2024-04-25T14:11:30.637103+0000","last_undegraded":"2024-04-25T14:11:30.637103+0000","last_fullsized":"2024-04-25T14:11:30.637103+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:32:51.809191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.563723+0000","last_change":"2024-04-25T14:11:29.561841+0000","last_active":"2024-04-25T14:11:30.563723+0000","last_peered":"2024-04-25T14:11:30.563723+0000","last_clean":"2024-04-25T14:11:30.563723+0000","last_became_active":"2024-04-25T14:11:29.560145+0000","last_became_peered":"2024-04-25T14:11:29.560145+0000","last_unstale":"2024-04-25T14:11:30.563723+0000","last_undegraded":"2024-04-25T14:11:30.563723+0000","last_fullsized":"2024-04-25T14:11:30.563723+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:11.654692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.786084+0000","last_change":"2024-04-25T14:11:29.570954+0000","last_active":"2024-04-25T14:11:30.786084+0000","last_peered":"2024-04-25T14:11:30.786084+0000","last_clean":"2024-04-25T14:11:30.786084+0000","last_became_active":"2024-04-25T14:11:29.570368+0000","last_became_peered":"2024-04-25T14:11:29.570368+0000","last_unstale":"2024-04-25T14:11:30.786084+0000","last_undegraded":"2024-04-25T14:11:30.786084+0000","last_fullsized":"2024-04-25T14:11:30.786084+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:50.153695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.637447+0000","last_change":"2024-04-25T14:11:29.571911+0000","last_active":"2024-04-25T14:11:30.637447+0000","last_peered":"2024-04-25T14:11:30.637447+0000","last_clean":"2024-04-25T14:11:30.637447+0000","last_became_active":"2024-04-25T14:11:29.568557+0000","last_became_peered":"2024-04-25T14:11:29.568557+0000","last_unstale":"2024-04-25T14:11:30.637447+0000","last_undegraded":"2024-04-25T14:11:30.637447+0000","last_fullsized":"2024-04-25T14:11:30.637447+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:24.221527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.636682+0000","last_change":"2024-04-25T14:11:29.566826+0000","last_active":"2024-04-25T14:11:30.636682+0000","last_peered":"2024-04-25T14:11:30.636682+0000","last_clean":"2024-04-25T14:11:30.636682+0000","last_became_active":"2024-04-25T14:11:29.565825+0000","last_became_peered":"2024-04-25T14:11:29.565825+0000","last_unstale":"2024-04-25T14:11:30.636682+0000","last_undegraded":"2024-04-25T14:11:30.636682+0000","last_fullsized":"2024-04-25T14:11:30.636682+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:46.711809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.636428+0000","last_change":"2024-04-25T14:11:29.577359+0000","last_active":"2024-04-25T14:11:30.636428+0000","last_peered":"2024-04-25T14:11:30.636428+0000","last_clean":"2024-04-25T14:11:30.636428+0000","last_became_active":"2024-04-25T14:11:29.574643+0000","last_became_peered":"2024-04-25T14:11:29.574643+0000","last_unstale":"2024-04-25T14:11:30.636428+0000","last_undegraded":"2024-04-25T14:11:30.636428+0000","last_fullsized":"2024-04-25T14:11:30.636428+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:07.582154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.636838+0000","last_change":"2024-04-25T14:11:29.583606+0000","last_active":"2024-04-25T14:11:30.636838+0000","last_peered":"2024-04-25T14:11:30.636838+0000","last_clean":"2024-04-25T14:11:30.636838+0000","last_became_active":"2024-04-25T14:11:29.582091+0000","last_became_peered":"2024-04-25T14:11:29.582091+0000","last_unstale":"2024-04-25T14:11:30.636838+0000","last_undegraded":"2024-04-25T14:11:30.636838+0000","last_fullsized":"2024-04-25T14:11:30.636838+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:58.711591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.636279+0000","last_change":"2024-04-25T14:11:29.571223+0000","last_active":"2024-04-25T14:11:30.636279+0000","last_peered":"2024-04-25T14:11:30.636279+0000","last_clean":"2024-04-25T14:11:30.636279+0000","last_became_active":"2024-04-25T14:11:29.570856+0000","last_became_peered":"2024-04-25T14:11:29.570856+0000","last_unstale":"2024-04-25T14:11:30.636279+0000","last_undegraded":"2024-04-25T14:11:30.636279+0000","last_fullsized":"2024-04-25T14:11:30.636279+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:53.058088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.565824+0000","last_change":"2024-04-25T14:11:29.574467+0000","last_active":"2024-04-25T14:11:30.565824+0000","last_peered":"2024-04-25T14:11:30.565824+0000","last_clean":"2024-04-25T14:11:30.565824+0000","last_became_active":"2024-04-25T14:11:29.573878+0000","last_became_peered":"2024-04-25T14:11:29.573878+0000","last_unstale":"2024-04-25T14:11:30.565824+0000","last_undegraded":"2024-04-25T14:11:30.565824+0000","last_fullsized":"2024-04-25T14:11:30.565824+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:22:02.087606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.636916+0000","last_change":"2024-04-25T14:11:29.561612+0000","last_active":"2024-04-25T14:11:30.636916+0000","last_peered":"2024-04-25T14:11:30.636916+0000","last_clean":"2024-04-25T14:11:30.636916+0000","last_became_active":"2024-04-25T14:11:29.561211+0000","last_became_peered":"2024-04-25T14:11:29.561211+0000","last_unstale":"2024-04-25T14:11:30.636916+0000","last_undegraded":"2024-04-25T14:11:30.636916+0000","last_fullsized":"2024-04-25T14:11:30.636916+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:29.195093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T14:11:29.577388+0000","last_change":"2024-04-25T14:11:29.575231+0000","last_active":"2024-04-25T14:11:29.577388+0000","last_peered":"2024-04-25T14:11:29.577388+0000","last_clean":"2024-04-25T14:11:29.577388+0000","last_became_active":"2024-04-25T14:11:29.573928+0000","last_became_peered":"2024-04-25T14:11:29.573928+0000","last_unstale":"2024-04-25T14:11:29.577388+0000","last_undegraded":"2024-04-25T14:11:29.577388+0000","last_fullsized":"2024-04-25T14:11:29.577388+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:34.209117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"76'11","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:33.528849+0000","last_change":"2024-04-25T14:11:28.538883+0000","last_active":"2024-04-25T14:11:33.528849+0000","last_peered":"2024-04-25T14:11:33.528849+0000","last_clean":"2024-04-25T14:11:33.528849+0000","last_became_active":"2024-04-25T14:11:28.538569+0000","last_became_peered":"2024-04-25T14:11:28.538569+0000","last_unstale":"2024-04-25T14:11:33.528849+0000","last_undegraded":"2024-04-25T14:11:33.528849+0000","last_fullsized":"2024-04-25T14:11:33.528849+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:05:51.265162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.562476+0000","last_change":"2024-04-25T14:11:29.576855+0000","last_active":"2024-04-25T14:11:30.562476+0000","last_peered":"2024-04-25T14:11:30.562476+0000","last_clean":"2024-04-25T14:11:30.562476+0000","last_became_active":"2024-04-25T14:11:29.574566+0000","last_became_peered":"2024-04-25T14:11:29.574566+0000","last_unstale":"2024-04-25T14:11:30.562476+0000","last_undegraded":"2024-04-25T14:11:30.562476+0000","last_fullsized":"2024-04-25T14:11:30.562476+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:31.325583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"75'1","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.562620+0000","last_change":"2024-04-25T14:11:28.552763+0000","last_active":"2024-04-25T14:11:30.562620+0000","last_peered":"2024-04-25T14:11:30.562620+0000","last_clean":"2024-04-25T14:11:30.562620+0000","last_became_active":"2024-04-25T14:11:28.552403+0000","last_became_peered":"2024-04-25T14:11:28.552403+0000","last_unstale":"2024-04-25T14:11:30.562620+0000","last_undegraded":"2024-04-25T14:11:30.562620+0000","last_fullsized":"2024-04-25T14:11:30.562620+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:31.473779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.636509+0000","last_change":"2024-04-25T14:11:29.571019+0000","last_active":"2024-04-25T14:11:30.636509+0000","last_peered":"2024-04-25T14:11:30.636509+0000","last_clean":"2024-04-25T14:11:30.636509+0000","last_became_active":"2024-04-25T14:11:29.570633+0000","last_became_peered":"2024-04-25T14:11:29.570633+0000","last_unstale":"2024-04-25T14:11:30.636509+0000","last_undegraded":"2024-04-25T14:11:30.636509+0000","last_fullsized":"2024-04-25T14:11:30.636509+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:38:32.664318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"76'4","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.524336+0000","last_change":"2024-04-25T14:11:28.545189+0000","last_active":"2024-04-25T14:11:32.524336+0000","last_peered":"2024-04-25T14:11:32.524336+0000","last_clean":"2024-04-25T14:11:32.524336+0000","last_became_active":"2024-04-25T14:11:28.544857+0000","last_became_peered":"2024-04-25T14:11:28.544857+0000","last_unstale":"2024-04-25T14:11:32.524336+0000","last_undegraded":"2024-04-25T14:11:32.524336+0000","last_fullsized":"2024-04-25T14:11:32.524336+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:58.464226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.636385+0000","last_change":"2024-04-25T14:11:29.576696+0000","last_active":"2024-04-25T14:11:30.636385+0000","last_peered":"2024-04-25T14:11:30.636385+0000","last_clean":"2024-04-25T14:11:30.636385+0000","last_became_active":"2024-04-25T14:11:29.576337+0000","last_became_peered":"2024-04-25T14:11:29.576337+0000","last_unstale":"2024-04-25T14:11:30.636385+0000","last_undegraded":"2024-04-25T14:11:30.636385+0000","last_fullsized":"2024-04-25T14:11:30.636385+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:31:30.410434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.637344+0000","last_change":"2024-04-25T14:11:28.542466+0000","last_active":"2024-04-25T14:11:30.637344+0000","last_peered":"2024-04-25T14:11:30.637344+0000","last_clean":"2024-04-25T14:11:30.637344+0000","last_became_active":"2024-04-25T14:11:28.542092+0000","last_became_peered":"2024-04-25T14:11:28.542092+0000","last_unstale":"2024-04-25T14:11:30.637344+0000","last_undegraded":"2024-04-25T14:11:30.637344+0000","last_fullsized":"2024-04-25T14:11:30.637344+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:33.810111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.637957+0000","last_change":"2024-04-25T14:11:29.567842+0000","last_active":"2024-04-25T14:11:30.637957+0000","last_peered":"2024-04-25T14:11:30.637957+0000","last_clean":"2024-04-25T14:11:30.637957+0000","last_became_active":"2024-04-25T14:11:29.566269+0000","last_became_peered":"2024-04-25T14:11:29.566269+0000","last_unstale":"2024-04-25T14:11:30.637957+0000","last_undegraded":"2024-04-25T14:11:30.637957+0000","last_fullsized":"2024-04-25T14:11:30.637957+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:31.009318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"75'1","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.638358+0000","last_change":"2024-04-25T14:11:28.546215+0000","last_active":"2024-04-25T14:11:30.638358+0000","last_peered":"2024-04-25T14:11:30.638358+0000","last_clean":"2024-04-25T14:11:30.638358+0000","last_became_active":"2024-04-25T14:11:28.545828+0000","last_became_peered":"2024-04-25T14:11:28.545828+0000","last_unstale":"2024-04-25T14:11:30.638358+0000","last_undegraded":"2024-04-25T14:11:30.638358+0000","last_fullsized":"2024-04-25T14:11:30.638358+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:27.152582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.637164+0000","last_change":"2024-04-25T14:11:29.559566+0000","last_active":"2024-04-25T14:11:30.637164+0000","last_peered":"2024-04-25T14:11:30.637164+0000","last_clean":"2024-04-25T14:11:30.637164+0000","last_became_active":"2024-04-25T14:11:29.558560+0000","last_became_peered":"2024-04-25T14:11:29.558560+0000","last_unstale":"2024-04-25T14:11:30.637164+0000","last_undegraded":"2024-04-25T14:11:30.637164+0000","last_fullsized":"2024-04-25T14:11:30.637164+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:23.677099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"75'1","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.563150+0000","last_change":"2024-04-25T14:11:28.544020+0000","last_active":"2024-04-25T14:11:30.563150+0000","last_peered":"2024-04-25T14:11:30.563150+0000","last_clean":"2024-04-25T14:11:30.563150+0000","last_became_active":"2024-04-25T14:11:28.543598+0000","last_became_peered":"2024-04-25T14:11:28.543598+0000","last_unstale":"2024-04-25T14:11:30.563150+0000","last_undegraded":"2024-04-25T14:11:30.563150+0000","last_fullsized":"2024-04-25T14:11:30.563150+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:49:06.543876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.638609+0000","last_change":"2024-04-25T14:11:29.576122+0000","last_active":"2024-04-25T14:11:30.638609+0000","last_peered":"2024-04-25T14:11:30.638609+0000","last_clean":"2024-04-25T14:11:30.638609+0000","last_became_active":"2024-04-25T14:11:29.573668+0000","last_became_peered":"2024-04-25T14:11:29.573668+0000","last_unstale":"2024-04-25T14:11:30.638609+0000","last_undegraded":"2024-04-25T14:11:30.638609+0000","last_fullsized":"2024-04-25T14:11:30.638609+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:06.998876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.637203+0000","last_change":"2024-04-25T14:11:28.541554+0000","last_active":"2024-04-25T14:11:30.637203+0000","last_peered":"2024-04-25T14:11:30.637203+0000","last_clean":"2024-04-25T14:11:30.637203+0000","last_became_active":"2024-04-25T14:11:28.541062+0000","last_became_peered":"2024-04-25T14:11:28.541062+0000","last_unstale":"2024-04-25T14:11:30.637203+0000","last_undegraded":"2024-04-25T14:11:30.637203+0000","last_fullsized":"2024-04-25T14:11:30.637203+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:16.719820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.562351+0000","last_change":"2024-04-25T14:11:29.564550+0000","last_active":"2024-04-25T14:11:30.562351+0000","last_peered":"2024-04-25T14:11:30.562351+0000","last_clean":"2024-04-25T14:11:30.562351+0000","last_became_active":"2024-04-25T14:11:29.564193+0000","last_became_peered":"2024-04-25T14:11:29.564193+0000","last_unstale":"2024-04-25T14:11:30.562351+0000","last_undegraded":"2024-04-25T14:11:30.562351+0000","last_fullsized":"2024-04-25T14:11:30.562351+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:59.357327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.636807+0000","last_change":"2024-04-25T14:11:28.540504+0000","last_active":"2024-04-25T14:11:30.636807+0000","last_peered":"2024-04-25T14:11:30.636807+0000","last_clean":"2024-04-25T14:11:30.636807+0000","last_became_active":"2024-04-25T14:11:28.540150+0000","last_became_peered":"2024-04-25T14:11:28.540150+0000","last_unstale":"2024-04-25T14:11:30.636807+0000","last_undegraded":"2024-04-25T14:11:30.636807+0000","last_fullsized":"2024-04-25T14:11:30.636807+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:35.331489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.636684+0000","last_change":"2024-04-25T14:11:29.561226+0000","last_active":"2024-04-25T14:11:30.636684+0000","last_peered":"2024-04-25T14:11:30.636684+0000","last_clean":"2024-04-25T14:11:30.636684+0000","last_became_active":"2024-04-25T14:11:29.560847+0000","last_became_peered":"2024-04-25T14:11:29.560847+0000","last_unstale":"2024-04-25T14:11:30.636684+0000","last_undegraded":"2024-04-25T14:11:30.636684+0000","last_fullsized":"2024-04-25T14:11:30.636684+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:05.299388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.563561+0000","last_change":"2024-04-25T14:11:28.548580+0000","last_active":"2024-04-25T14:11:30.563561+0000","last_peered":"2024-04-25T14:11:30.563561+0000","last_clean":"2024-04-25T14:11:30.563561+0000","last_became_active":"2024-04-25T14:11:28.548250+0000","last_became_peered":"2024-04-25T14:11:28.548250+0000","last_unstale":"2024-04-25T14:11:30.563561+0000","last_undegraded":"2024-04-25T14:11:30.563561+0000","last_fullsized":"2024-04-25T14:11:30.563561+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:01.166285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.636865+0000","last_change":"2024-04-25T14:11:29.575983+0000","last_active":"2024-04-25T14:11:30.636865+0000","last_peered":"2024-04-25T14:11:30.636865+0000","last_clean":"2024-04-25T14:11:30.636865+0000","last_became_active":"2024-04-25T14:11:29.575147+0000","last_became_peered":"2024-04-25T14:11:29.575147+0000","last_unstale":"2024-04-25T14:11:30.636865+0000","last_undegraded":"2024-04-25T14:11:30.636865+0000","last_fullsized":"2024-04-25T14:11:30.636865+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:40:45.366736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.785881+0000","last_change":"2024-04-25T14:11:28.542398+0000","last_active":"2024-04-25T14:11:30.785881+0000","last_peered":"2024-04-25T14:11:30.785881+0000","last_clean":"2024-04-25T14:11:30.785881+0000","last_became_active":"2024-04-25T14:11:28.539982+0000","last_became_peered":"2024-04-25T14:11:28.539982+0000","last_unstale":"2024-04-25T14:11:30.785881+0000","last_undegraded":"2024-04-25T14:11:30.785881+0000","last_fullsized":"2024-04-25T14:11:30.785881+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:24.144635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.636688+0000","last_change":"2024-04-25T14:11:29.581424+0000","last_active":"2024-04-25T14:11:30.636688+0000","last_peered":"2024-04-25T14:11:30.636688+0000","last_clean":"2024-04-25T14:11:30.636688+0000","last_became_active":"2024-04-25T14:11:29.581086+0000","last_became_peered":"2024-04-25T14:11:29.581086+0000","last_unstale":"2024-04-25T14:11:30.636688+0000","last_undegraded":"2024-04-25T14:11:30.636688+0000","last_fullsized":"2024-04-25T14:11:30.636688+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:30.468728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'3","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.638088+0000","last_change":"2024-04-25T14:11:28.540771+0000","last_active":"2024-04-25T14:11:30.638088+0000","last_peered":"2024-04-25T14:11:30.638088+0000","last_clean":"2024-04-25T14:11:30.638088+0000","last_became_active":"2024-04-25T14:11:28.540442+0000","last_became_peered":"2024-04-25T14:11:28.540442+0000","last_unstale":"2024-04-25T14:11:30.638088+0000","last_undegraded":"2024-04-25T14:11:30.638088+0000","last_fullsized":"2024-04-25T14:11:30.638088+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:52.479423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.636535+0000","last_change":"2024-04-25T14:11:29.579382+0000","last_active":"2024-04-25T14:11:30.636535+0000","last_peered":"2024-04-25T14:11:30.636535+0000","last_clean":"2024-04-25T14:11:30.636535+0000","last_became_active":"2024-04-25T14:11:29.578975+0000","last_became_peered":"2024-04-25T14:11:29.578975+0000","last_unstale":"2024-04-25T14:11:30.636535+0000","last_undegraded":"2024-04-25T14:11:30.636535+0000","last_fullsized":"2024-04-25T14:11:30.636535+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:31:04.202332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"75'6","reported_seq":21,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.563528+0000","last_change":"2024-04-25T14:11:28.543093+0000","last_active":"2024-04-25T14:11:30.563528+0000","last_peered":"2024-04-25T14:11:30.563528+0000","last_clean":"2024-04-25T14:11:30.563528+0000","last_became_active":"2024-04-25T14:11:28.542407+0000","last_became_peered":"2024-04-25T14:11:28.542407+0000","last_unstale":"2024-04-25T14:11:30.563528+0000","last_undegraded":"2024-04-25T14:11:30.563528+0000","last_fullsized":"2024-04-25T14:11:30.563528+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:43.767375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.637791+0000","last_change":"2024-04-25T14:11:29.571896+0000","last_active":"2024-04-25T14:11:30.637791+0000","last_peered":"2024-04-25T14:11:30.637791+0000","last_clean":"2024-04-25T14:11:30.637791+0000","last_became_active":"2024-04-25T14:11:29.571537+0000","last_became_peered":"2024-04-25T14:11:29.571537+0000","last_unstale":"2024-04-25T14:11:30.637791+0000","last_undegraded":"2024-04-25T14:11:30.637791+0000","last_fullsized":"2024-04-25T14:11:30.637791+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:32:07.914841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'2","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T14:11:29.582109+0000","last_change":"2024-04-25T14:11:28.543293+0000","last_active":"2024-04-25T14:11:29.582109+0000","last_peered":"2024-04-25T14:11:29.582109+0000","last_clean":"2024-04-25T14:11:29.582109+0000","last_became_active":"2024-04-25T14:11:28.542967+0000","last_became_peered":"2024-04-25T14:11:28.542967+0000","last_unstale":"2024-04-25T14:11:29.582109+0000","last_undegraded":"2024-04-25T14:11:29.582109+0000","last_fullsized":"2024-04-25T14:11:29.582109+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:29.452792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.636928+0000","last_change":"2024-04-25T14:11:29.563877+0000","last_active":"2024-04-25T14:11:30.636928+0000","last_peered":"2024-04-25T14:11:30.636928+0000","last_clean":"2024-04-25T14:11:30.636928+0000","last_became_active":"2024-04-25T14:11:29.563535+0000","last_became_peered":"2024-04-25T14:11:29.563535+0000","last_unstale":"2024-04-25T14:11:30.636928+0000","last_undegraded":"2024-04-25T14:11:30.636928+0000","last_fullsized":"2024-04-25T14:11:30.636928+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:07:38.805863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.637739+0000","last_change":"2024-04-25T14:11:28.543787+0000","last_active":"2024-04-25T14:11:30.637739+0000","last_peered":"2024-04-25T14:11:30.637739+0000","last_clean":"2024-04-25T14:11:30.637739+0000","last_became_active":"2024-04-25T14:11:28.540816+0000","last_became_peered":"2024-04-25T14:11:28.540816+0000","last_unstale":"2024-04-25T14:11:30.637739+0000","last_undegraded":"2024-04-25T14:11:30.637739+0000","last_fullsized":"2024-04-25T14:11:30.637739+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:18.776280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.786057+0000","last_change":"2024-04-25T14:11:29.574785+0000","last_active":"2024-04-25T14:11:30.786057+0000","last_peered":"2024-04-25T14:11:30.786057+0000","last_clean":"2024-04-25T14:11:30.786057+0000","last_became_active":"2024-04-25T14:11:29.574422+0000","last_became_peered":"2024-04-25T14:11:29.574422+0000","last_unstale":"2024-04-25T14:11:30.786057+0000","last_undegraded":"2024-04-25T14:11:30.786057+0000","last_fullsized":"2024-04-25T14:11:30.786057+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:45:57.256214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.636964+0000","last_change":"2024-04-25T14:11:28.537450+0000","last_active":"2024-04-25T14:11:30.636964+0000","last_peered":"2024-04-25T14:11:30.636964+0000","last_clean":"2024-04-25T14:11:30.636964+0000","last_became_active":"2024-04-25T14:11:28.537088+0000","last_became_peered":"2024-04-25T14:11:28.537088+0000","last_unstale":"2024-04-25T14:11:30.636964+0000","last_undegraded":"2024-04-25T14:11:30.636964+0000","last_fullsized":"2024-04-25T14:11:30.636964+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:05.779706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.785530+0000","last_change":"2024-04-25T14:11:29.572789+0000","last_active":"2024-04-25T14:11:30.785530+0000","last_peered":"2024-04-25T14:11:30.785530+0000","last_clean":"2024-04-25T14:11:30.785530+0000","last_became_active":"2024-04-25T14:11:29.572201+0000","last_became_peered":"2024-04-25T14:11:29.572201+0000","last_unstale":"2024-04-25T14:11:30.785530+0000","last_undegraded":"2024-04-25T14:11:30.785530+0000","last_fullsized":"2024-04-25T14:11:30.785530+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:39:58.237070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"76'15","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.526981+0000","last_change":"2024-04-25T14:11:28.540577+0000","last_active":"2024-04-25T14:11:32.526981+0000","last_peered":"2024-04-25T14:11:32.526981+0000","last_clean":"2024-04-25T14:11:32.526981+0000","last_became_active":"2024-04-25T14:11:28.540212+0000","last_became_peered":"2024-04-25T14:11:28.540212+0000","last_unstale":"2024-04-25T14:11:32.526981+0000","last_undegraded":"2024-04-25T14:11:32.526981+0000","last_fullsized":"2024-04-25T14:11:32.526981+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:31.422332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.638097+0000","last_change":"2024-04-25T14:11:29.574952+0000","last_active":"2024-04-25T14:11:30.638097+0000","last_peered":"2024-04-25T14:11:30.638097+0000","last_clean":"2024-04-25T14:11:30.638097+0000","last_became_active":"2024-04-25T14:11:29.573677+0000","last_became_peered":"2024-04-25T14:11:29.573677+0000","last_unstale":"2024-04-25T14:11:30.638097+0000","last_undegraded":"2024-04-25T14:11:30.638097+0000","last_fullsized":"2024-04-25T14:11:30.638097+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:35.896103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'1","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.637501+0000","last_change":"2024-04-25T14:11:28.541356+0000","last_active":"2024-04-25T14:11:30.637501+0000","last_peered":"2024-04-25T14:11:30.637501+0000","last_clean":"2024-04-25T14:11:30.637501+0000","last_became_active":"2024-04-25T14:11:28.540976+0000","last_became_peered":"2024-04-25T14:11:28.540976+0000","last_unstale":"2024-04-25T14:11:30.637501+0000","last_undegraded":"2024-04-25T14:11:30.637501+0000","last_fullsized":"2024-04-25T14:11:30.637501+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:32:50.599900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.565549+0000","last_change":"2024-04-25T14:11:29.554317+0000","last_active":"2024-04-25T14:11:30.565549+0000","last_peered":"2024-04-25T14:11:30.565549+0000","last_clean":"2024-04-25T14:11:30.565549+0000","last_became_active":"2024-04-25T14:11:29.553935+0000","last_became_peered":"2024-04-25T14:11:29.553935+0000","last_unstale":"2024-04-25T14:11:30.565549+0000","last_undegraded":"2024-04-25T14:11:30.565549+0000","last_fullsized":"2024-04-25T14:11:30.565549+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:01.405964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"76'1","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.636638+0000","last_change":"2024-04-25T14:11:28.546892+0000","last_active":"2024-04-25T14:11:30.636638+0000","last_peered":"2024-04-25T14:11:30.636638+0000","last_clean":"2024-04-25T14:11:30.636638+0000","last_became_active":"2024-04-25T14:11:28.546566+0000","last_became_peered":"2024-04-25T14:11:28.546566+0000","last_unstale":"2024-04-25T14:11:30.636638+0000","last_undegraded":"2024-04-25T14:11:30.636638+0000","last_fullsized":"2024-04-25T14:11:30.636638+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:05.239635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.563584+0000","last_change":"2024-04-25T14:11:29.565083+0000","last_active":"2024-04-25T14:11:30.563584+0000","last_peered":"2024-04-25T14:11:30.563584+0000","last_clean":"2024-04-25T14:11:30.563584+0000","last_became_active":"2024-04-25T14:11:29.564062+0000","last_became_peered":"2024-04-25T14:11:29.564062+0000","last_unstale":"2024-04-25T14:11:30.563584+0000","last_undegraded":"2024-04-25T14:11:30.563584+0000","last_fullsized":"2024-04-25T14:11:30.563584+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:07:10.664113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.785729+0000","last_change":"2024-04-25T14:11:28.538649+0000","last_active":"2024-04-25T14:11:30.785729+0000","last_peered":"2024-04-25T14:11:30.785729+0000","last_clean":"2024-04-25T14:11:30.785729+0000","last_became_active":"2024-04-25T14:11:28.538353+0000","last_became_peered":"2024-04-25T14:11:28.538353+0000","last_unstale":"2024-04-25T14:11:30.785729+0000","last_undegraded":"2024-04-25T14:11:30.785729+0000","last_fullsized":"2024-04-25T14:11:30.785729+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:06.513039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.563535+0000","last_change":"2024-04-25T14:11:29.576263+0000","last_active":"2024-04-25T14:11:30.563535+0000","last_peered":"2024-04-25T14:11:30.563535+0000","last_clean":"2024-04-25T14:11:30.563535+0000","last_became_active":"2024-04-25T14:11:29.575877+0000","last_became_peered":"2024-04-25T14:11:29.575877+0000","last_unstale":"2024-04-25T14:11:30.563535+0000","last_undegraded":"2024-04-25T14:11:30.563535+0000","last_fullsized":"2024-04-25T14:11:30.563535+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:33:37.639537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.637390+0000","last_change":"2024-04-25T14:11:29.568220+0000","last_active":"2024-04-25T14:11:30.637390+0000","last_peered":"2024-04-25T14:11:30.637390+0000","last_clean":"2024-04-25T14:11:30.637390+0000","last_became_active":"2024-04-25T14:11:29.567741+0000","last_became_peered":"2024-04-25T14:11:29.567741+0000","last_unstale":"2024-04-25T14:11:30.637390+0000","last_undegraded":"2024-04-25T14:11:30.637390+0000","last_fullsized":"2024-04-25T14:11:30.637390+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:05.134466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"76'1","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:31.585278+0000","last_change":"2024-04-25T14:11:28.537503+0000","last_active":"2024-04-25T14:11:31.585278+0000","last_peered":"2024-04-25T14:11:31.585278+0000","last_clean":"2024-04-25T14:11:31.585278+0000","last_became_active":"2024-04-25T14:11:28.537169+0000","last_became_peered":"2024-04-25T14:11:28.537169+0000","last_unstale":"2024-04-25T14:11:31.585278+0000","last_undegraded":"2024-04-25T14:11:31.585278+0000","last_fullsized":"2024-04-25T14:11:31.585278+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:09.752988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.562696+0000","last_change":"2024-04-25T14:11:29.569487+0000","last_active":"2024-04-25T14:11:30.562696+0000","last_peered":"2024-04-25T14:11:30.562696+0000","last_clean":"2024-04-25T14:11:30.562696+0000","last_became_active":"2024-04-25T14:11:29.569148+0000","last_became_peered":"2024-04-25T14:11:29.569148+0000","last_unstale":"2024-04-25T14:11:30.562696+0000","last_undegraded":"2024-04-25T14:11:30.562696+0000","last_fullsized":"2024-04-25T14:11:30.562696+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:03.395620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.637257+0000","last_change":"2024-04-25T14:11:28.538216+0000","last_active":"2024-04-25T14:11:30.637257+0000","last_peered":"2024-04-25T14:11:30.637257+0000","last_clean":"2024-04-25T14:11:30.637257+0000","last_became_active":"2024-04-25T14:11:28.537772+0000","last_became_peered":"2024-04-25T14:11:28.537772+0000","last_unstale":"2024-04-25T14:11:30.637257+0000","last_undegraded":"2024-04-25T14:11:30.637257+0000","last_fullsized":"2024-04-25T14:11:30.637257+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:55.460966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":89,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.785401+0000","last_change":"2024-04-25T14:10:12.194216+0000","last_active":"2024-04-25T14:11:30.785401+0000","last_peered":"2024-04-25T14:11:30.785401+0000","last_clean":"2024-04-25T14:11:30.785401+0000","last_became_active":"2024-04-25T14:10:12.189616+0000","last_became_peered":"2024-04-25T14:10:12.189616+0000","last_unstale":"2024-04-25T14:11:30.785401+0000","last_undegraded":"2024-04-25T14:11:30.785401+0000","last_fullsized":"2024-04-25T14:11:30.785401+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:47.553264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:47.553264+0000","last_clean_scrub_stamp":"2024-04-25T14:06:47.553264+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:47.576058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.637008+0000","last_change":"2024-04-25T14:11:29.574985+0000","last_active":"2024-04-25T14:11:30.637008+0000","last_peered":"2024-04-25T14:11:30.637008+0000","last_clean":"2024-04-25T14:11:30.637008+0000","last_became_active":"2024-04-25T14:11:29.574629+0000","last_became_peered":"2024-04-25T14:11:29.574629+0000","last_unstale":"2024-04-25T14:11:30.637008+0000","last_undegraded":"2024-04-25T14:11:30.637008+0000","last_fullsized":"2024-04-25T14:11:30.637008+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:01.899889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.785674+0000","last_change":"2024-04-25T14:11:28.543251+0000","last_active":"2024-04-25T14:11:30.785674+0000","last_peered":"2024-04-25T14:11:30.785674+0000","last_clean":"2024-04-25T14:11:30.785674+0000","last_became_active":"2024-04-25T14:11:28.542439+0000","last_became_peered":"2024-04-25T14:11:28.542439+0000","last_unstale":"2024-04-25T14:11:30.785674+0000","last_undegraded":"2024-04-25T14:11:30.785674+0000","last_fullsized":"2024-04-25T14:11:30.785674+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:30:06.939009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"76'1","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:31.588391+0000","last_change":"2024-04-25T14:11:28.544269+0000","last_active":"2024-04-25T14:11:31.588391+0000","last_peered":"2024-04-25T14:11:31.588391+0000","last_clean":"2024-04-25T14:11:31.588391+0000","last_became_active":"2024-04-25T14:11:28.543897+0000","last_became_peered":"2024-04-25T14:11:28.543897+0000","last_unstale":"2024-04-25T14:11:31.588391+0000","last_undegraded":"2024-04-25T14:11:31.588391+0000","last_fullsized":"2024-04-25T14:11:31.588391+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:37.668187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.637384+0000","last_change":"2024-04-25T14:11:29.566428+0000","last_active":"2024-04-25T14:11:30.637384+0000","last_peered":"2024-04-25T14:11:30.637384+0000","last_clean":"2024-04-25T14:11:30.637384+0000","last_became_active":"2024-04-25T14:11:29.562598+0000","last_became_peered":"2024-04-25T14:11:29.562598+0000","last_unstale":"2024-04-25T14:11:30.637384+0000","last_undegraded":"2024-04-25T14:11:30.637384+0000","last_fullsized":"2024-04-25T14:11:30.637384+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:46.611359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"76'7","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:33.530432+0000","last_change":"2024-04-25T14:11:28.542571+0000","last_active":"2024-04-25T14:11:33.530432+0000","last_peered":"2024-04-25T14:11:33.530432+0000","last_clean":"2024-04-25T14:11:33.530432+0000","last_became_active":"2024-04-25T14:11:28.542272+0000","last_became_peered":"2024-04-25T14:11:28.542272+0000","last_unstale":"2024-04-25T14:11:33.530432+0000","last_undegraded":"2024-04-25T14:11:33.530432+0000","last_fullsized":"2024-04-25T14:11:33.530432+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:41:05.128068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.563023+0000","last_change":"2024-04-25T14:11:29.578565+0000","last_active":"2024-04-25T14:11:30.563023+0000","last_peered":"2024-04-25T14:11:30.563023+0000","last_clean":"2024-04-25T14:11:30.563023+0000","last_became_active":"2024-04-25T14:11:29.576782+0000","last_became_peered":"2024-04-25T14:11:29.576782+0000","last_unstale":"2024-04-25T14:11:30.563023+0000","last_undegraded":"2024-04-25T14:11:30.563023+0000","last_fullsized":"2024-04-25T14:11:30.563023+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:31.634602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'2","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.637163+0000","last_change":"2024-04-25T14:11:28.543838+0000","last_active":"2024-04-25T14:11:30.637163+0000","last_peered":"2024-04-25T14:11:30.637163+0000","last_clean":"2024-04-25T14:11:30.637163+0000","last_became_active":"2024-04-25T14:11:28.543511+0000","last_became_peered":"2024-04-25T14:11:28.543511+0000","last_unstale":"2024-04-25T14:11:30.637163+0000","last_undegraded":"2024-04-25T14:11:30.637163+0000","last_fullsized":"2024-04-25T14:11:30.637163+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:34.440234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.637420+0000","last_change":"2024-04-25T14:11:29.578686+0000","last_active":"2024-04-25T14:11:30.637420+0000","last_peered":"2024-04-25T14:11:30.637420+0000","last_clean":"2024-04-25T14:11:30.637420+0000","last_became_active":"2024-04-25T14:11:29.578324+0000","last_became_peered":"2024-04-25T14:11:29.578324+0000","last_unstale":"2024-04-25T14:11:30.637420+0000","last_undegraded":"2024-04-25T14:11:30.637420+0000","last_fullsized":"2024-04-25T14:11:30.637420+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:51:56.244792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"76'2","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:31.587097+0000","last_change":"2024-04-25T14:11:28.539443+0000","last_active":"2024-04-25T14:11:31.587097+0000","last_peered":"2024-04-25T14:11:31.587097+0000","last_clean":"2024-04-25T14:11:31.587097+0000","last_became_active":"2024-04-25T14:11:28.539113+0000","last_became_peered":"2024-04-25T14:11:28.539113+0000","last_unstale":"2024-04-25T14:11:31.587097+0000","last_undegraded":"2024-04-25T14:11:31.587097+0000","last_fullsized":"2024-04-25T14:11:31.587097+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:36.664972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.636879+0000","last_change":"2024-04-25T14:11:29.575451+0000","last_active":"2024-04-25T14:11:30.636879+0000","last_peered":"2024-04-25T14:11:30.636879+0000","last_clean":"2024-04-25T14:11:30.636879+0000","last_became_active":"2024-04-25T14:11:29.574983+0000","last_became_peered":"2024-04-25T14:11:29.574983+0000","last_unstale":"2024-04-25T14:11:30.636879+0000","last_undegraded":"2024-04-25T14:11:30.636879+0000","last_fullsized":"2024-04-25T14:11:30.636879+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:53.317608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.562283+0000","last_change":"2024-04-25T14:11:28.546394+0000","last_active":"2024-04-25T14:11:30.562283+0000","last_peered":"2024-04-25T14:11:30.562283+0000","last_clean":"2024-04-25T14:11:30.562283+0000","last_became_active":"2024-04-25T14:11:28.546064+0000","last_became_peered":"2024-04-25T14:11:28.546064+0000","last_unstale":"2024-04-25T14:11:30.562283+0000","last_undegraded":"2024-04-25T14:11:30.562283+0000","last_fullsized":"2024-04-25T14:11:30.562283+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:15.301492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.565402+0000","last_change":"2024-04-25T14:11:29.569584+0000","last_active":"2024-04-25T14:11:30.565402+0000","last_peered":"2024-04-25T14:11:30.565402+0000","last_clean":"2024-04-25T14:11:30.565402+0000","last_became_active":"2024-04-25T14:11:29.569217+0000","last_became_peered":"2024-04-25T14:11:29.569217+0000","last_unstale":"2024-04-25T14:11:30.565402+0000","last_undegraded":"2024-04-25T14:11:30.565402+0000","last_fullsized":"2024-04-25T14:11:30.565402+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:05.708277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T14:11:29.553096+0000","last_change":"2024-04-25T14:11:28.544399+0000","last_active":"2024-04-25T14:11:29.553096+0000","last_peered":"2024-04-25T14:11:29.553096+0000","last_clean":"2024-04-25T14:11:29.553096+0000","last_became_active":"2024-04-25T14:11:28.544098+0000","last_became_peered":"2024-04-25T14:11:28.544098+0000","last_unstale":"2024-04-25T14:11:29.553096+0000","last_undegraded":"2024-04-25T14:11:29.553096+0000","last_fullsized":"2024-04-25T14:11:29.553096+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:05.148386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.562679+0000","last_change":"2024-04-25T14:11:29.573081+0000","last_active":"2024-04-25T14:11:30.562679+0000","last_peered":"2024-04-25T14:11:30.562679+0000","last_clean":"2024-04-25T14:11:30.562679+0000","last_became_active":"2024-04-25T14:11:29.572654+0000","last_became_peered":"2024-04-25T14:11:29.572654+0000","last_unstale":"2024-04-25T14:11:30.562679+0000","last_undegraded":"2024-04-25T14:11:30.562679+0000","last_fullsized":"2024-04-25T14:11:30.562679+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:04.076576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"76'4","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:31.587584+0000","last_change":"2024-04-25T14:11:28.542488+0000","last_active":"2024-04-25T14:11:31.587584+0000","last_peered":"2024-04-25T14:11:31.587584+0000","last_clean":"2024-04-25T14:11:31.587584+0000","last_became_active":"2024-04-25T14:11:28.542171+0000","last_became_peered":"2024-04-25T14:11:28.542171+0000","last_unstale":"2024-04-25T14:11:31.587584+0000","last_undegraded":"2024-04-25T14:11:31.587584+0000","last_fullsized":"2024-04-25T14:11:31.587584+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:48.201762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.786093+0000","last_change":"2024-04-25T14:11:29.573805+0000","last_active":"2024-04-25T14:11:30.786093+0000","last_peered":"2024-04-25T14:11:30.786093+0000","last_clean":"2024-04-25T14:11:30.786093+0000","last_became_active":"2024-04-25T14:11:29.573420+0000","last_became_peered":"2024-04-25T14:11:29.573420+0000","last_unstale":"2024-04-25T14:11:30.786093+0000","last_undegraded":"2024-04-25T14:11:30.786093+0000","last_fullsized":"2024-04-25T14:11:30.786093+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:18.721977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T14:11:29.554250+0000","last_change":"2024-04-25T14:11:28.552863+0000","last_active":"2024-04-25T14:11:29.554250+0000","last_peered":"2024-04-25T14:11:29.554250+0000","last_clean":"2024-04-25T14:11:29.554250+0000","last_became_active":"2024-04-25T14:11:28.552461+0000","last_became_peered":"2024-04-25T14:11:28.552461+0000","last_unstale":"2024-04-25T14:11:29.554250+0000","last_undegraded":"2024-04-25T14:11:29.554250+0000","last_fullsized":"2024-04-25T14:11:29.554250+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:46.264473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.786062+0000","last_change":"2024-04-25T14:11:29.571268+0000","last_active":"2024-04-25T14:11:30.786062+0000","last_peered":"2024-04-25T14:11:30.786062+0000","last_clean":"2024-04-25T14:11:30.786062+0000","last_became_active":"2024-04-25T14:11:29.570876+0000","last_became_peered":"2024-04-25T14:11:29.570876+0000","last_unstale":"2024-04-25T14:11:30.786062+0000","last_undegraded":"2024-04-25T14:11:30.786062+0000","last_fullsized":"2024-04-25T14:11:30.786062+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:15.093635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.637751+0000","last_change":"2024-04-25T14:11:28.543343+0000","last_active":"2024-04-25T14:11:30.637751+0000","last_peered":"2024-04-25T14:11:30.637751+0000","last_clean":"2024-04-25T14:11:30.637751+0000","last_became_active":"2024-04-25T14:11:28.540790+0000","last_became_peered":"2024-04-25T14:11:28.540790+0000","last_unstale":"2024-04-25T14:11:30.637751+0000","last_undegraded":"2024-04-25T14:11:30.637751+0000","last_fullsized":"2024-04-25T14:11:30.637751+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:20.498016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.785925+0000","last_change":"2024-04-25T14:11:29.566161+0000","last_active":"2024-04-25T14:11:30.785925+0000","last_peered":"2024-04-25T14:11:30.785925+0000","last_clean":"2024-04-25T14:11:30.785925+0000","last_became_active":"2024-04-25T14:11:29.564197+0000","last_became_peered":"2024-04-25T14:11:29.564197+0000","last_unstale":"2024-04-25T14:11:30.785925+0000","last_undegraded":"2024-04-25T14:11:30.785925+0000","last_fullsized":"2024-04-25T14:11:30.785925+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:35.291546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.636425+0000","last_change":"2024-04-25T14:11:28.543479+0000","last_active":"2024-04-25T14:11:30.636425+0000","last_peered":"2024-04-25T14:11:30.636425+0000","last_clean":"2024-04-25T14:11:30.636425+0000","last_became_active":"2024-04-25T14:11:28.543167+0000","last_became_peered":"2024-04-25T14:11:28.543167+0000","last_unstale":"2024-04-25T14:11:30.636425+0000","last_undegraded":"2024-04-25T14:11:30.636425+0000","last_fullsized":"2024-04-25T14:11:30.636425+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:54:53.572809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.785969+0000","last_change":"2024-04-25T14:11:29.572548+0000","last_active":"2024-04-25T14:11:30.785969+0000","last_peered":"2024-04-25T14:11:30.785969+0000","last_clean":"2024-04-25T14:11:30.785969+0000","last_became_active":"2024-04-25T14:11:29.572167+0000","last_became_peered":"2024-04-25T14:11:29.572167+0000","last_unstale":"2024-04-25T14:11:30.785969+0000","last_undegraded":"2024-04-25T14:11:30.785969+0000","last_fullsized":"2024-04-25T14:11:30.785969+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:38.423331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.565601+0000","last_change":"2024-04-25T14:11:29.560956+0000","last_active":"2024-04-25T14:11:30.565601+0000","last_peered":"2024-04-25T14:11:30.565601+0000","last_clean":"2024-04-25T14:11:30.565601+0000","last_became_active":"2024-04-25T14:11:29.560519+0000","last_became_peered":"2024-04-25T14:11:29.560519+0000","last_unstale":"2024-04-25T14:11:30.565601+0000","last_undegraded":"2024-04-25T14:11:30.565601+0000","last_fullsized":"2024-04-25T14:11:30.565601+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:53:47.677923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T14:11:29.578220+0000","last_change":"2024-04-25T14:11:29.574372+0000","last_active":"2024-04-25T14:11:29.578220+0000","last_peered":"2024-04-25T14:11:29.578220+0000","last_clean":"2024-04-25T14:11:29.578220+0000","last_became_active":"2024-04-25T14:11:29.573995+0000","last_became_peered":"2024-04-25T14:11:29.573995+0000","last_unstale":"2024-04-25T14:11:29.578220+0000","last_undegraded":"2024-04-25T14:11:29.578220+0000","last_fullsized":"2024-04-25T14:11:29.578220+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:40.130748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T14:11:29.579258+0000","last_change":"2024-04-25T14:11:29.578324+0000","last_active":"2024-04-25T14:11:29.579258+0000","last_peered":"2024-04-25T14:11:29.579258+0000","last_clean":"2024-04-25T14:11:29.579258+0000","last_became_active":"2024-04-25T14:11:29.577917+0000","last_became_peered":"2024-04-25T14:11:29.577917+0000","last_unstale":"2024-04-25T14:11:29.579258+0000","last_undegraded":"2024-04-25T14:11:29.579258+0000","last_fullsized":"2024-04-25T14:11:29.579258+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:47.358075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.637153+0000","last_change":"2024-04-25T14:11:29.578879+0000","last_active":"2024-04-25T14:11:30.637153+0000","last_peered":"2024-04-25T14:11:30.637153+0000","last_clean":"2024-04-25T14:11:30.637153+0000","last_became_active":"2024-04-25T14:11:29.578551+0000","last_became_peered":"2024-04-25T14:11:29.578551+0000","last_unstale":"2024-04-25T14:11:30.637153+0000","last_undegraded":"2024-04-25T14:11:30.637153+0000","last_fullsized":"2024-04-25T14:11:30.637153+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:41.083794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.637481+0000","last_change":"2024-04-25T14:11:29.574054+0000","last_active":"2024-04-25T14:11:30.637481+0000","last_peered":"2024-04-25T14:11:30.637481+0000","last_clean":"2024-04-25T14:11:30.637481+0000","last_became_active":"2024-04-25T14:11:29.573700+0000","last_became_peered":"2024-04-25T14:11:29.573700+0000","last_unstale":"2024-04-25T14:11:30.637481+0000","last_undegraded":"2024-04-25T14:11:30.637481+0000","last_fullsized":"2024-04-25T14:11:30.637481+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:23.304580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.565807+0000","last_change":"2024-04-25T14:11:29.553768+0000","last_active":"2024-04-25T14:11:30.565807+0000","last_peered":"2024-04-25T14:11:30.565807+0000","last_clean":"2024-04-25T14:11:30.565807+0000","last_became_active":"2024-04-25T14:11:29.553376+0000","last_became_peered":"2024-04-25T14:11:29.553376+0000","last_unstale":"2024-04-25T14:11:30.565807+0000","last_undegraded":"2024-04-25T14:11:30.565807+0000","last_fullsized":"2024-04-25T14:11:30.565807+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:46.688368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T14:11:29.563685+0000","last_change":"2024-04-25T14:11:29.563122+0000","last_active":"2024-04-25T14:11:29.563685+0000","last_peered":"2024-04-25T14:11:29.563685+0000","last_clean":"2024-04-25T14:11:29.563685+0000","last_became_active":"2024-04-25T14:11:29.562717+0000","last_became_peered":"2024-04-25T14:11:29.562717+0000","last_unstale":"2024-04-25T14:11:29.563685+0000","last_undegraded":"2024-04-25T14:11:29.563685+0000","last_fullsized":"2024-04-25T14:11:29.563685+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:44.829953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.562938+0000","last_change":"2024-04-25T14:11:29.576487+0000","last_active":"2024-04-25T14:11:30.562938+0000","last_peered":"2024-04-25T14:11:30.562938+0000","last_clean":"2024-04-25T14:11:30.562938+0000","last_became_active":"2024-04-25T14:11:29.575070+0000","last_became_peered":"2024-04-25T14:11:29.575070+0000","last_unstale":"2024-04-25T14:11:30.562938+0000","last_undegraded":"2024-04-25T14:11:30.562938+0000","last_fullsized":"2024-04-25T14:11:30.562938+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:13.803505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.637011+0000","last_change":"2024-04-25T14:11:29.568733+0000","last_active":"2024-04-25T14:11:30.637011+0000","last_peered":"2024-04-25T14:11:30.637011+0000","last_clean":"2024-04-25T14:11:30.637011+0000","last_became_active":"2024-04-25T14:11:29.567665+0000","last_became_peered":"2024-04-25T14:11:29.567665+0000","last_unstale":"2024-04-25T14:11:30.637011+0000","last_undegraded":"2024-04-25T14:11:30.637011+0000","last_fullsized":"2024-04-25T14:11:30.637011+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:57.956199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.562318+0000","last_change":"2024-04-25T14:11:29.574648+0000","last_active":"2024-04-25T14:11:30.562318+0000","last_peered":"2024-04-25T14:11:30.562318+0000","last_clean":"2024-04-25T14:11:30.562318+0000","last_became_active":"2024-04-25T14:11:29.574247+0000","last_became_peered":"2024-04-25T14:11:29.574247+0000","last_unstale":"2024-04-25T14:11:30.562318+0000","last_undegraded":"2024-04-25T14:11:30.562318+0000","last_fullsized":"2024-04-25T14:11:30.562318+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:57.110017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.636493+0000","last_change":"2024-04-25T14:11:29.573633+0000","last_active":"2024-04-25T14:11:30.636493+0000","last_peered":"2024-04-25T14:11:30.636493+0000","last_clean":"2024-04-25T14:11:30.636493+0000","last_became_active":"2024-04-25T14:11:29.573303+0000","last_became_peered":"2024-04-25T14:11:29.573303+0000","last_unstale":"2024-04-25T14:11:30.636493+0000","last_undegraded":"2024-04-25T14:11:30.636493+0000","last_fullsized":"2024-04-25T14:11:30.636493+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:11.458117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.636835+0000","last_change":"2024-04-25T14:11:29.569136+0000","last_active":"2024-04-25T14:11:30.636835+0000","last_peered":"2024-04-25T14:11:30.636835+0000","last_clean":"2024-04-25T14:11:30.636835+0000","last_became_active":"2024-04-25T14:11:29.568801+0000","last_became_peered":"2024-04-25T14:11:29.568801+0000","last_unstale":"2024-04-25T14:11:30.636835+0000","last_undegraded":"2024-04-25T14:11:30.636835+0000","last_fullsized":"2024-04-25T14:11:30.636835+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:30.969081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.565404+0000","last_change":"2024-04-25T14:11:29.566754+0000","last_active":"2024-04-25T14:11:30.565404+0000","last_peered":"2024-04-25T14:11:30.565404+0000","last_clean":"2024-04-25T14:11:30.565404+0000","last_became_active":"2024-04-25T14:11:29.566368+0000","last_became_peered":"2024-04-25T14:11:29.566368+0000","last_unstale":"2024-04-25T14:11:30.565404+0000","last_undegraded":"2024-04-25T14:11:30.565404+0000","last_fullsized":"2024-04-25T14:11:30.565404+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:37.057718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281277,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2544,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":80,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":159744,"data_stored":9666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678030,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31972,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713180,"statfs":{"total":95995035648,"available":95962296320,"internally_reserved":0,"allocated":806912,"data_stored":386813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874259,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":868352,"data_stored":970417,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037784,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27828,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717324,"statfs":{"total":95995035648,"available":95966539776,"internally_reserved":0,"allocated":831488,"data_stored":387721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234014,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31984,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713168,"statfs":{"total":95995035648,"available":95962284032,"internally_reserved":0,"allocated":827392,"data_stored":387551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430242,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32020,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713132,"statfs":{"total":95995035648,"available":95962247168,"internally_reserved":0,"allocated":860160,"data_stored":977763,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593768,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31972,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713180,"statfs":{"total":95995035648,"available":95962296320,"internally_reserved":0,"allocated":806912,"data_stored":386813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":819200,"data_stored":388425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31992,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713160,"statfs":{"total":95995035648,"available":95962275840,"internally_reserved":0,"allocated":835584,"data_stored":388291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717340,"statfs":{"total":95995035648,"available":95966556160,"internally_reserved":0,"allocated":811008,"data_stored":379889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149751,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717348,"statfs":{"total":95995035648,"available":95966564352,"internally_reserved":0,"allocated":802816,"data_stored":378347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953521,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27808,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717344,"statfs":{"total":95995035648,"available":95966560256,"internally_reserved":0,"allocated":798720,"data_stored":378265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":37,"seq":158913789997,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":856064,"data_stored":968793,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:11:36.395 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:11:36.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.539+0000 7fdacf972640 1 -- 172.21.15.96:0/3895733348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdac81045b0 msgr2=0x7fdac81069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:36.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.539+0000 7fdacf972640 1 --2- 172.21.15.96:0/3895733348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdac81045b0 0x7fdac81069d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdac4009860 tx=0x7fdac402a340 comp rx=0 tx=0).stop 2024-04-25T14:11:36.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.539+0000 7fdacf972640 1 -- 172.21.15.96:0/3895733348 shutdown_connections 2024-04-25T14:11:36.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.539+0000 7fdacf972640 1 --2- 172.21.15.96:0/3895733348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdac8109870 0x7fdac810bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:36.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.539+0000 7fdacf972640 1 --2- 172.21.15.96:0/3895733348 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdac8106f10 0x7fdac8109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:36.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.539+0000 7fdacf972640 1 --2- 172.21.15.96:0/3895733348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdac81045b0 0x7fdac81069d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:36.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.539+0000 7fdacf972640 1 -- 172.21.15.96:0/3895733348 >> 172.21.15.96:0/3895733348 conn(0x7fdac80fdbd0 msgr2=0x7fdac8100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:36.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.539+0000 7fdacf972640 1 -- 172.21.15.96:0/3895733348 shutdown_connections 2024-04-25T14:11:36.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.539+0000 7fdacf972640 1 -- 172.21.15.96:0/3895733348 wait complete. 2024-04-25T14:11:36.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.540+0000 7fdacf972640 1 Processor -- start 2024-04-25T14:11:36.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.540+0000 7fdacf972640 1 -- start start 2024-04-25T14:11:36.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.540+0000 7fdacf972640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdac81045b0 0x7fdac819a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:36.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.540+0000 7fdacf972640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdac8106f10 0x7fdac819a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:36.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.540+0000 7fdacf972640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdac8109870 0x7fdac81a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:36.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.540+0000 7fdacf972640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdac810f5c0 con 0x7fdac81045b0 2024-04-25T14:11:36.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.540+0000 7fdacf972640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdac810f440 con 0x7fdac8109870 2024-04-25T14:11:36.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.540+0000 7fdacf972640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdac810f740 con 0x7fdac8106f10 2024-04-25T14:11:36.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.540+0000 7fdacd70e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdac81045b0 0x7fdac819a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:36.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.540+0000 7fdacd70e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdac81045b0 0x7fdac819a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35442/0 (socket says 172.21.15.96:35442) 2024-04-25T14:11:36.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.540+0000 7fdacd70e640 1 -- 172.21.15.96:0/810357052 learned_addr learned my addr 172.21.15.96:0/810357052 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:11:36.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.540+0000 7fdacdf0f640 1 --2- 172.21.15.96:0/810357052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdac8109870 0x7fdac81a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:36.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.540+0000 7fdaccf0d640 1 --2- 172.21.15.96:0/810357052 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdac8106f10 0x7fdac819a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:36.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.540+0000 7fdacd70e640 1 -- 172.21.15.96:0/810357052 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdac8106f10 msgr2=0x7fdac819a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:36.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.540+0000 7fdacd70e640 1 --2- 172.21.15.96:0/810357052 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdac8106f10 0x7fdac819a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:36.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.540+0000 7fdacd70e640 1 -- 172.21.15.96:0/810357052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdac8109870 msgr2=0x7fdac81a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:36.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.540+0000 7fdacd70e640 1 --2- 172.21.15.96:0/810357052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdac8109870 0x7fdac81a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:36.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.540+0000 7fdacd70e640 1 -- 172.21.15.96:0/810357052 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdac81a1d60 con 0x7fdac81045b0 2024-04-25T14:11:36.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.540+0000 7fdaccf0d640 1 --2- 172.21.15.96:0/810357052 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdac8106f10 0x7fdac819a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:36.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.540+0000 7fdacd70e640 1 --2- 172.21.15.96:0/810357052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdac81045b0 0x7fdac819a220 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdac4000c00 tx=0x7fdac4032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:36.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.541+0000 7fdabe7fc640 1 -- 172.21.15.96:0/810357052 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac4040020 con 0x7fdac81045b0 2024-04-25T14:11:36.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.541+0000 7fdacf972640 1 -- 172.21.15.96:0/810357052 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdac81a1ff0 con 0x7fdac81045b0 2024-04-25T14:11:36.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.541+0000 7fdabe7fc640 1 -- 172.21.15.96:0/810357052 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdac4004040 con 0x7fdac81045b0 2024-04-25T14:11:36.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.541+0000 7fdacf972640 1 -- 172.21.15.96:0/810357052 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdac81a2450 con 0x7fdac81045b0 2024-04-25T14:11:36.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.541+0000 7fdabe7fc640 1 -- 172.21.15.96:0/810357052 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac4037040 con 0x7fdac81045b0 2024-04-25T14:11:36.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.542+0000 7fdacf972640 1 -- 172.21.15.96:0/810357052 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdac80ff6f0 con 0x7fdac81045b0 2024-04-25T14:11:36.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.544+0000 7fdabe7fc640 1 -- 172.21.15.96:0/810357052 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fdac4043020 con 0x7fdac81045b0 2024-04-25T14:11:36.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.545+0000 7fdabe7fc640 1 --2- 172.21.15.96:0/810357052 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fdab40afb80 0x7fdab40b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:36.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.545+0000 7fdaccf0d640 1 --2- 172.21.15.96:0/810357052 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fdab40afb80 0x7fdab40b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:36.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.545+0000 7fdabe7fc640 1 -- 172.21.15.96:0/810357052 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7fdac40ff100 con 0x7fdac81045b0 2024-04-25T14:11:36.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.546+0000 7fdaccf0d640 1 --2- 172.21.15.96:0/810357052 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fdab40afb80 0x7fdab40b2090 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdab8005fd0 tx=0x7fdab8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:36.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.549+0000 7fdabe7fc640 1 -- 172.21.15.96:0/810357052 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdac404a020 con 0x7fdac81045b0 2024-04-25T14:11:36.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.725+0000 7fdacf972640 1 -- 172.21.15.96:0/810357052 --> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdac8062fe0 con 0x7fdab40afb80 2024-04-25T14:11:36.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.732+0000 7fdabe7fc640 1 -- 172.21.15.96:0/810357052 <== mgr.14152 v2:172.21.15.96:6800/536332425 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258903 (secure 0 0 0) 0x7fdac8062fe0 con 0x7fdab40afb80 2024-04-25T14:11:36.734 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:11:36.736 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-04-25T14:11:36.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.735+0000 7fdacf972640 1 -- 172.21.15.96:0/810357052 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fdab40afb80 msgr2=0x7fdab40b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:36.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.735+0000 7fdacf972640 1 --2- 172.21.15.96:0/810357052 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fdab40afb80 0x7fdab40b2090 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdab8005fd0 tx=0x7fdab8005850 comp rx=0 tx=0).stop 2024-04-25T14:11:36.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.735+0000 7fdacf972640 1 -- 172.21.15.96:0/810357052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdac81045b0 msgr2=0x7fdac819a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:36.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.735+0000 7fdacf972640 1 --2- 172.21.15.96:0/810357052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdac81045b0 0x7fdac819a220 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdac4000c00 tx=0x7fdac4032d50 comp rx=0 tx=0).stop 2024-04-25T14:11:36.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.735+0000 7fdacf972640 1 -- 172.21.15.96:0/810357052 shutdown_connections 2024-04-25T14:11:36.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.735+0000 7fdacf972640 1 --2- 172.21.15.96:0/810357052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdac8109870 0x7fdac81a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:36.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.735+0000 7fdacf972640 1 --2- 172.21.15.96:0/810357052 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdac8106f10 0x7fdac819a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:36.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.735+0000 7fdacf972640 1 --2- 172.21.15.96:0/810357052 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fdab40afb80 0x7fdab40b2090 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:36.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.735+0000 7fdacf972640 1 --2- 172.21.15.96:0/810357052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdac81045b0 0x7fdac819a220 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:36.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.735+0000 7fdacf972640 1 -- 172.21.15.96:0/810357052 >> 172.21.15.96:0/810357052 conn(0x7fdac80fdbd0 msgr2=0x7fdac8105200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:36.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.735+0000 7fdacf972640 1 -- 172.21.15.96:0/810357052 shutdown_connections 2024-04-25T14:11:36.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:36.735+0000 7fdacf972640 1 -- 172.21.15.96:0/810357052 wait complete. 2024-04-25T14:11:36.744 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:36 smithi096 ceph-mon[29207]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:11:36.745 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:36 smithi096 ceph-mon[29207]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:11:36.745 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:36 smithi096 ceph-mon[29207]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:11:36.745 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:36 smithi096 ceph-mon[29207]: pgmap v283: 97 pgs: 97 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 745 B/s rd, 3.3 KiB/s wr, 12 op/s 2024-04-25T14:11:36.745 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:36 smithi096 ceph-mon[29207]: mds.? [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] up:active 2024-04-25T14:11:36.745 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:36 smithi096 ceph-mon[29207]: fsmap cephfs:5 2 up:standby-replay 5 up:standby 5 up:active 2024-04-25T14:11:36.745 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:36 smithi096 ceph-mon[29207]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T14:11:36.745 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:36 smithi096 ceph-mon[29207]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T14:11:36.745 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:36 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/4249911663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:36.745 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:36 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/3462612178' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:11:36.746 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":283,"stamp":"2024-04-25T14:11:35.399091+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592912,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":203,"num_write_kb":2173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":221,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":358912,"kb_used_data":9692,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124582912,"statfs":{"total":1151940427776,"available":1151572901888,"internally_reserved":0,"allocated":9924608,"data_stored":6379088,"data_compressed":107307,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2544,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":80,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.872230"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.565759+0000","last_change":"2024-04-25T14:11:29.569615+0000","last_active":"2024-04-25T14:11:30.565759+0000","last_peered":"2024-04-25T14:11:30.565759+0000","last_clean":"2024-04-25T14:11:30.565759+0000","last_became_active":"2024-04-25T14:11:29.569237+0000","last_became_peered":"2024-04-25T14:11:29.569237+0000","last_unstale":"2024-04-25T14:11:30.565759+0000","last_undegraded":"2024-04-25T14:11:30.565759+0000","last_fullsized":"2024-04-25T14:11:30.565759+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:24.367533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.637236+0000","last_change":"2024-04-25T14:11:29.571820+0000","last_active":"2024-04-25T14:11:30.637236+0000","last_peered":"2024-04-25T14:11:30.637236+0000","last_clean":"2024-04-25T14:11:30.637236+0000","last_became_active":"2024-04-25T14:11:29.571456+0000","last_became_peered":"2024-04-25T14:11:29.571456+0000","last_unstale":"2024-04-25T14:11:30.637236+0000","last_undegraded":"2024-04-25T14:11:30.637236+0000","last_fullsized":"2024-04-25T14:11:30.637236+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:33.442301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.637007+0000","last_change":"2024-04-25T14:11:29.579302+0000","last_active":"2024-04-25T14:11:30.637007+0000","last_peered":"2024-04-25T14:11:30.637007+0000","last_clean":"2024-04-25T14:11:30.637007+0000","last_became_active":"2024-04-25T14:11:29.577734+0000","last_became_peered":"2024-04-25T14:11:29.577734+0000","last_unstale":"2024-04-25T14:11:30.637007+0000","last_undegraded":"2024-04-25T14:11:30.637007+0000","last_fullsized":"2024-04-25T14:11:30.637007+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:59.147554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.637447+0000","last_change":"2024-04-25T14:11:29.574942+0000","last_active":"2024-04-25T14:11:30.637447+0000","last_peered":"2024-04-25T14:11:30.637447+0000","last_clean":"2024-04-25T14:11:30.637447+0000","last_became_active":"2024-04-25T14:11:29.573774+0000","last_became_peered":"2024-04-25T14:11:29.573774+0000","last_unstale":"2024-04-25T14:11:30.637447+0000","last_undegraded":"2024-04-25T14:11:30.637447+0000","last_fullsized":"2024-04-25T14:11:30.637447+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:10.626265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.637063+0000","last_change":"2024-04-25T14:11:29.557199+0000","last_active":"2024-04-25T14:11:30.637063+0000","last_peered":"2024-04-25T14:11:30.637063+0000","last_clean":"2024-04-25T14:11:30.637063+0000","last_became_active":"2024-04-25T14:11:29.556836+0000","last_became_peered":"2024-04-25T14:11:29.556836+0000","last_unstale":"2024-04-25T14:11:30.637063+0000","last_undegraded":"2024-04-25T14:11:30.637063+0000","last_fullsized":"2024-04-25T14:11:30.637063+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:04:45.882935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.563277+0000","last_change":"2024-04-25T14:11:29.557988+0000","last_active":"2024-04-25T14:11:30.563277+0000","last_peered":"2024-04-25T14:11:30.563277+0000","last_clean":"2024-04-25T14:11:30.563277+0000","last_became_active":"2024-04-25T14:11:29.557622+0000","last_became_peered":"2024-04-25T14:11:29.557622+0000","last_unstale":"2024-04-25T14:11:30.563277+0000","last_undegraded":"2024-04-25T14:11:30.563277+0000","last_fullsized":"2024-04-25T14:11:30.563277+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:28:37.578368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.785804+0000","last_change":"2024-04-25T14:11:29.565049+0000","last_active":"2024-04-25T14:11:30.785804+0000","last_peered":"2024-04-25T14:11:30.785804+0000","last_clean":"2024-04-25T14:11:30.785804+0000","last_became_active":"2024-04-25T14:11:29.564688+0000","last_became_peered":"2024-04-25T14:11:29.564688+0000","last_unstale":"2024-04-25T14:11:30.785804+0000","last_undegraded":"2024-04-25T14:11:30.785804+0000","last_fullsized":"2024-04-25T14:11:30.785804+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:48.128778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.562584+0000","last_change":"2024-04-25T14:11:29.565879+0000","last_active":"2024-04-25T14:11:30.562584+0000","last_peered":"2024-04-25T14:11:30.562584+0000","last_clean":"2024-04-25T14:11:30.562584+0000","last_became_active":"2024-04-25T14:11:29.565525+0000","last_became_peered":"2024-04-25T14:11:29.565525+0000","last_unstale":"2024-04-25T14:11:30.562584+0000","last_undegraded":"2024-04-25T14:11:30.562584+0000","last_fullsized":"2024-04-25T14:11:30.562584+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:06.956334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.637103+0000","last_change":"2024-04-25T14:11:29.567280+0000","last_active":"2024-04-25T14:11:30.637103+0000","last_peered":"2024-04-25T14:11:30.637103+0000","last_clean":"2024-04-25T14:11:30.637103+0000","last_became_active":"2024-04-25T14:11:29.566741+0000","last_became_peered":"2024-04-25T14:11:29.566741+0000","last_unstale":"2024-04-25T14:11:30.637103+0000","last_undegraded":"2024-04-25T14:11:30.637103+0000","last_fullsized":"2024-04-25T14:11:30.637103+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:32:51.809191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.563723+0000","last_change":"2024-04-25T14:11:29.561841+0000","last_active":"2024-04-25T14:11:30.563723+0000","last_peered":"2024-04-25T14:11:30.563723+0000","last_clean":"2024-04-25T14:11:30.563723+0000","last_became_active":"2024-04-25T14:11:29.560145+0000","last_became_peered":"2024-04-25T14:11:29.560145+0000","last_unstale":"2024-04-25T14:11:30.563723+0000","last_undegraded":"2024-04-25T14:11:30.563723+0000","last_fullsized":"2024-04-25T14:11:30.563723+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:11.654692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.786084+0000","last_change":"2024-04-25T14:11:29.570954+0000","last_active":"2024-04-25T14:11:30.786084+0000","last_peered":"2024-04-25T14:11:30.786084+0000","last_clean":"2024-04-25T14:11:30.786084+0000","last_became_active":"2024-04-25T14:11:29.570368+0000","last_became_peered":"2024-04-25T14:11:29.570368+0000","last_unstale":"2024-04-25T14:11:30.786084+0000","last_undegraded":"2024-04-25T14:11:30.786084+0000","last_fullsized":"2024-04-25T14:11:30.786084+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:50.153695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.637447+0000","last_change":"2024-04-25T14:11:29.571911+0000","last_active":"2024-04-25T14:11:30.637447+0000","last_peered":"2024-04-25T14:11:30.637447+0000","last_clean":"2024-04-25T14:11:30.637447+0000","last_became_active":"2024-04-25T14:11:29.568557+0000","last_became_peered":"2024-04-25T14:11:29.568557+0000","last_unstale":"2024-04-25T14:11:30.637447+0000","last_undegraded":"2024-04-25T14:11:30.637447+0000","last_fullsized":"2024-04-25T14:11:30.637447+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:24.221527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.636682+0000","last_change":"2024-04-25T14:11:29.566826+0000","last_active":"2024-04-25T14:11:30.636682+0000","last_peered":"2024-04-25T14:11:30.636682+0000","last_clean":"2024-04-25T14:11:30.636682+0000","last_became_active":"2024-04-25T14:11:29.565825+0000","last_became_peered":"2024-04-25T14:11:29.565825+0000","last_unstale":"2024-04-25T14:11:30.636682+0000","last_undegraded":"2024-04-25T14:11:30.636682+0000","last_fullsized":"2024-04-25T14:11:30.636682+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:46.711809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.636428+0000","last_change":"2024-04-25T14:11:29.577359+0000","last_active":"2024-04-25T14:11:30.636428+0000","last_peered":"2024-04-25T14:11:30.636428+0000","last_clean":"2024-04-25T14:11:30.636428+0000","last_became_active":"2024-04-25T14:11:29.574643+0000","last_became_peered":"2024-04-25T14:11:29.574643+0000","last_unstale":"2024-04-25T14:11:30.636428+0000","last_undegraded":"2024-04-25T14:11:30.636428+0000","last_fullsized":"2024-04-25T14:11:30.636428+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:07.582154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.636838+0000","last_change":"2024-04-25T14:11:29.583606+0000","last_active":"2024-04-25T14:11:30.636838+0000","last_peered":"2024-04-25T14:11:30.636838+0000","last_clean":"2024-04-25T14:11:30.636838+0000","last_became_active":"2024-04-25T14:11:29.582091+0000","last_became_peered":"2024-04-25T14:11:29.582091+0000","last_unstale":"2024-04-25T14:11:30.636838+0000","last_undegraded":"2024-04-25T14:11:30.636838+0000","last_fullsized":"2024-04-25T14:11:30.636838+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:58.711591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.636279+0000","last_change":"2024-04-25T14:11:29.571223+0000","last_active":"2024-04-25T14:11:30.636279+0000","last_peered":"2024-04-25T14:11:30.636279+0000","last_clean":"2024-04-25T14:11:30.636279+0000","last_became_active":"2024-04-25T14:11:29.570856+0000","last_became_peered":"2024-04-25T14:11:29.570856+0000","last_unstale":"2024-04-25T14:11:30.636279+0000","last_undegraded":"2024-04-25T14:11:30.636279+0000","last_fullsized":"2024-04-25T14:11:30.636279+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:53.058088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.565824+0000","last_change":"2024-04-25T14:11:29.574467+0000","last_active":"2024-04-25T14:11:30.565824+0000","last_peered":"2024-04-25T14:11:30.565824+0000","last_clean":"2024-04-25T14:11:30.565824+0000","last_became_active":"2024-04-25T14:11:29.573878+0000","last_became_peered":"2024-04-25T14:11:29.573878+0000","last_unstale":"2024-04-25T14:11:30.565824+0000","last_undegraded":"2024-04-25T14:11:30.565824+0000","last_fullsized":"2024-04-25T14:11:30.565824+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:22:02.087606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.636916+0000","last_change":"2024-04-25T14:11:29.561612+0000","last_active":"2024-04-25T14:11:30.636916+0000","last_peered":"2024-04-25T14:11:30.636916+0000","last_clean":"2024-04-25T14:11:30.636916+0000","last_became_active":"2024-04-25T14:11:29.561211+0000","last_became_peered":"2024-04-25T14:11:29.561211+0000","last_unstale":"2024-04-25T14:11:30.636916+0000","last_undegraded":"2024-04-25T14:11:30.636916+0000","last_fullsized":"2024-04-25T14:11:30.636916+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:29.195093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T14:11:29.577388+0000","last_change":"2024-04-25T14:11:29.575231+0000","last_active":"2024-04-25T14:11:29.577388+0000","last_peered":"2024-04-25T14:11:29.577388+0000","last_clean":"2024-04-25T14:11:29.577388+0000","last_became_active":"2024-04-25T14:11:29.573928+0000","last_became_peered":"2024-04-25T14:11:29.573928+0000","last_unstale":"2024-04-25T14:11:29.577388+0000","last_undegraded":"2024-04-25T14:11:29.577388+0000","last_fullsized":"2024-04-25T14:11:29.577388+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:34.209117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"76'11","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:33.528849+0000","last_change":"2024-04-25T14:11:28.538883+0000","last_active":"2024-04-25T14:11:33.528849+0000","last_peered":"2024-04-25T14:11:33.528849+0000","last_clean":"2024-04-25T14:11:33.528849+0000","last_became_active":"2024-04-25T14:11:28.538569+0000","last_became_peered":"2024-04-25T14:11:28.538569+0000","last_unstale":"2024-04-25T14:11:33.528849+0000","last_undegraded":"2024-04-25T14:11:33.528849+0000","last_fullsized":"2024-04-25T14:11:33.528849+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:05:51.265162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.562476+0000","last_change":"2024-04-25T14:11:29.576855+0000","last_active":"2024-04-25T14:11:30.562476+0000","last_peered":"2024-04-25T14:11:30.562476+0000","last_clean":"2024-04-25T14:11:30.562476+0000","last_became_active":"2024-04-25T14:11:29.574566+0000","last_became_peered":"2024-04-25T14:11:29.574566+0000","last_unstale":"2024-04-25T14:11:30.562476+0000","last_undegraded":"2024-04-25T14:11:30.562476+0000","last_fullsized":"2024-04-25T14:11:30.562476+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:31.325583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"75'1","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.562620+0000","last_change":"2024-04-25T14:11:28.552763+0000","last_active":"2024-04-25T14:11:30.562620+0000","last_peered":"2024-04-25T14:11:30.562620+0000","last_clean":"2024-04-25T14:11:30.562620+0000","last_became_active":"2024-04-25T14:11:28.552403+0000","last_became_peered":"2024-04-25T14:11:28.552403+0000","last_unstale":"2024-04-25T14:11:30.562620+0000","last_undegraded":"2024-04-25T14:11:30.562620+0000","last_fullsized":"2024-04-25T14:11:30.562620+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:31.473779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.636509+0000","last_change":"2024-04-25T14:11:29.571019+0000","last_active":"2024-04-25T14:11:30.636509+0000","last_peered":"2024-04-25T14:11:30.636509+0000","last_clean":"2024-04-25T14:11:30.636509+0000","last_became_active":"2024-04-25T14:11:29.570633+0000","last_became_peered":"2024-04-25T14:11:29.570633+0000","last_unstale":"2024-04-25T14:11:30.636509+0000","last_undegraded":"2024-04-25T14:11:30.636509+0000","last_fullsized":"2024-04-25T14:11:30.636509+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:38:32.664318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"76'4","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.524336+0000","last_change":"2024-04-25T14:11:28.545189+0000","last_active":"2024-04-25T14:11:32.524336+0000","last_peered":"2024-04-25T14:11:32.524336+0000","last_clean":"2024-04-25T14:11:32.524336+0000","last_became_active":"2024-04-25T14:11:28.544857+0000","last_became_peered":"2024-04-25T14:11:28.544857+0000","last_unstale":"2024-04-25T14:11:32.524336+0000","last_undegraded":"2024-04-25T14:11:32.524336+0000","last_fullsized":"2024-04-25T14:11:32.524336+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:58.464226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.636385+0000","last_change":"2024-04-25T14:11:29.576696+0000","last_active":"2024-04-25T14:11:30.636385+0000","last_peered":"2024-04-25T14:11:30.636385+0000","last_clean":"2024-04-25T14:11:30.636385+0000","last_became_active":"2024-04-25T14:11:29.576337+0000","last_became_peered":"2024-04-25T14:11:29.576337+0000","last_unstale":"2024-04-25T14:11:30.636385+0000","last_undegraded":"2024-04-25T14:11:30.636385+0000","last_fullsized":"2024-04-25T14:11:30.636385+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:31:30.410434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.637344+0000","last_change":"2024-04-25T14:11:28.542466+0000","last_active":"2024-04-25T14:11:30.637344+0000","last_peered":"2024-04-25T14:11:30.637344+0000","last_clean":"2024-04-25T14:11:30.637344+0000","last_became_active":"2024-04-25T14:11:28.542092+0000","last_became_peered":"2024-04-25T14:11:28.542092+0000","last_unstale":"2024-04-25T14:11:30.637344+0000","last_undegraded":"2024-04-25T14:11:30.637344+0000","last_fullsized":"2024-04-25T14:11:30.637344+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:33.810111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.637957+0000","last_change":"2024-04-25T14:11:29.567842+0000","last_active":"2024-04-25T14:11:30.637957+0000","last_peered":"2024-04-25T14:11:30.637957+0000","last_clean":"2024-04-25T14:11:30.637957+0000","last_became_active":"2024-04-25T14:11:29.566269+0000","last_became_peered":"2024-04-25T14:11:29.566269+0000","last_unstale":"2024-04-25T14:11:30.637957+0000","last_undegraded":"2024-04-25T14:11:30.637957+0000","last_fullsized":"2024-04-25T14:11:30.637957+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:31.009318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"75'1","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.638358+0000","last_change":"2024-04-25T14:11:28.546215+0000","last_active":"2024-04-25T14:11:30.638358+0000","last_peered":"2024-04-25T14:11:30.638358+0000","last_clean":"2024-04-25T14:11:30.638358+0000","last_became_active":"2024-04-25T14:11:28.545828+0000","last_became_peered":"2024-04-25T14:11:28.545828+0000","last_unstale":"2024-04-25T14:11:30.638358+0000","last_undegraded":"2024-04-25T14:11:30.638358+0000","last_fullsized":"2024-04-25T14:11:30.638358+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:27.152582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.637164+0000","last_change":"2024-04-25T14:11:29.559566+0000","last_active":"2024-04-25T14:11:30.637164+0000","last_peered":"2024-04-25T14:11:30.637164+0000","last_clean":"2024-04-25T14:11:30.637164+0000","last_became_active":"2024-04-25T14:11:29.558560+0000","last_became_peered":"2024-04-25T14:11:29.558560+0000","last_unstale":"2024-04-25T14:11:30.637164+0000","last_undegraded":"2024-04-25T14:11:30.637164+0000","last_fullsized":"2024-04-25T14:11:30.637164+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:23.677099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"75'1","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.563150+0000","last_change":"2024-04-25T14:11:28.544020+0000","last_active":"2024-04-25T14:11:30.563150+0000","last_peered":"2024-04-25T14:11:30.563150+0000","last_clean":"2024-04-25T14:11:30.563150+0000","last_became_active":"2024-04-25T14:11:28.543598+0000","last_became_peered":"2024-04-25T14:11:28.543598+0000","last_unstale":"2024-04-25T14:11:30.563150+0000","last_undegraded":"2024-04-25T14:11:30.563150+0000","last_fullsized":"2024-04-25T14:11:30.563150+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:49:06.543876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.638609+0000","last_change":"2024-04-25T14:11:29.576122+0000","last_active":"2024-04-25T14:11:30.638609+0000","last_peered":"2024-04-25T14:11:30.638609+0000","last_clean":"2024-04-25T14:11:30.638609+0000","last_became_active":"2024-04-25T14:11:29.573668+0000","last_became_peered":"2024-04-25T14:11:29.573668+0000","last_unstale":"2024-04-25T14:11:30.638609+0000","last_undegraded":"2024-04-25T14:11:30.638609+0000","last_fullsized":"2024-04-25T14:11:30.638609+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:06.998876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.637203+0000","last_change":"2024-04-25T14:11:28.541554+0000","last_active":"2024-04-25T14:11:30.637203+0000","last_peered":"2024-04-25T14:11:30.637203+0000","last_clean":"2024-04-25T14:11:30.637203+0000","last_became_active":"2024-04-25T14:11:28.541062+0000","last_became_peered":"2024-04-25T14:11:28.541062+0000","last_unstale":"2024-04-25T14:11:30.637203+0000","last_undegraded":"2024-04-25T14:11:30.637203+0000","last_fullsized":"2024-04-25T14:11:30.637203+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:16.719820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.562351+0000","last_change":"2024-04-25T14:11:29.564550+0000","last_active":"2024-04-25T14:11:30.562351+0000","last_peered":"2024-04-25T14:11:30.562351+0000","last_clean":"2024-04-25T14:11:30.562351+0000","last_became_active":"2024-04-25T14:11:29.564193+0000","last_became_peered":"2024-04-25T14:11:29.564193+0000","last_unstale":"2024-04-25T14:11:30.562351+0000","last_undegraded":"2024-04-25T14:11:30.562351+0000","last_fullsized":"2024-04-25T14:11:30.562351+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:59.357327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.636807+0000","last_change":"2024-04-25T14:11:28.540504+0000","last_active":"2024-04-25T14:11:30.636807+0000","last_peered":"2024-04-25T14:11:30.636807+0000","last_clean":"2024-04-25T14:11:30.636807+0000","last_became_active":"2024-04-25T14:11:28.540150+0000","last_became_peered":"2024-04-25T14:11:28.540150+0000","last_unstale":"2024-04-25T14:11:30.636807+0000","last_undegraded":"2024-04-25T14:11:30.636807+0000","last_fullsized":"2024-04-25T14:11:30.636807+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:35.331489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.636684+0000","last_change":"2024-04-25T14:11:29.561226+0000","last_active":"2024-04-25T14:11:30.636684+0000","last_peered":"2024-04-25T14:11:30.636684+0000","last_clean":"2024-04-25T14:11:30.636684+0000","last_became_active":"2024-04-25T14:11:29.560847+0000","last_became_peered":"2024-04-25T14:11:29.560847+0000","last_unstale":"2024-04-25T14:11:30.636684+0000","last_undegraded":"2024-04-25T14:11:30.636684+0000","last_fullsized":"2024-04-25T14:11:30.636684+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:05.299388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.563561+0000","last_change":"2024-04-25T14:11:28.548580+0000","last_active":"2024-04-25T14:11:30.563561+0000","last_peered":"2024-04-25T14:11:30.563561+0000","last_clean":"2024-04-25T14:11:30.563561+0000","last_became_active":"2024-04-25T14:11:28.548250+0000","last_became_peered":"2024-04-25T14:11:28.548250+0000","last_unstale":"2024-04-25T14:11:30.563561+0000","last_undegraded":"2024-04-25T14:11:30.563561+0000","last_fullsized":"2024-04-25T14:11:30.563561+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:01.166285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.636865+0000","last_change":"2024-04-25T14:11:29.575983+0000","last_active":"2024-04-25T14:11:30.636865+0000","last_peered":"2024-04-25T14:11:30.636865+0000","last_clean":"2024-04-25T14:11:30.636865+0000","last_became_active":"2024-04-25T14:11:29.575147+0000","last_became_peered":"2024-04-25T14:11:29.575147+0000","last_unstale":"2024-04-25T14:11:30.636865+0000","last_undegraded":"2024-04-25T14:11:30.636865+0000","last_fullsized":"2024-04-25T14:11:30.636865+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:40:45.366736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.785881+0000","last_change":"2024-04-25T14:11:28.542398+0000","last_active":"2024-04-25T14:11:30.785881+0000","last_peered":"2024-04-25T14:11:30.785881+0000","last_clean":"2024-04-25T14:11:30.785881+0000","last_became_active":"2024-04-25T14:11:28.539982+0000","last_became_peered":"2024-04-25T14:11:28.539982+0000","last_unstale":"2024-04-25T14:11:30.785881+0000","last_undegraded":"2024-04-25T14:11:30.785881+0000","last_fullsized":"2024-04-25T14:11:30.785881+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:24.144635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.636688+0000","last_change":"2024-04-25T14:11:29.581424+0000","last_active":"2024-04-25T14:11:30.636688+0000","last_peered":"2024-04-25T14:11:30.636688+0000","last_clean":"2024-04-25T14:11:30.636688+0000","last_became_active":"2024-04-25T14:11:29.581086+0000","last_became_peered":"2024-04-25T14:11:29.581086+0000","last_unstale":"2024-04-25T14:11:30.636688+0000","last_undegraded":"2024-04-25T14:11:30.636688+0000","last_fullsized":"2024-04-25T14:11:30.636688+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:30.468728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'3","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.638088+0000","last_change":"2024-04-25T14:11:28.540771+0000","last_active":"2024-04-25T14:11:30.638088+0000","last_peered":"2024-04-25T14:11:30.638088+0000","last_clean":"2024-04-25T14:11:30.638088+0000","last_became_active":"2024-04-25T14:11:28.540442+0000","last_became_peered":"2024-04-25T14:11:28.540442+0000","last_unstale":"2024-04-25T14:11:30.638088+0000","last_undegraded":"2024-04-25T14:11:30.638088+0000","last_fullsized":"2024-04-25T14:11:30.638088+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:52.479423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.636535+0000","last_change":"2024-04-25T14:11:29.579382+0000","last_active":"2024-04-25T14:11:30.636535+0000","last_peered":"2024-04-25T14:11:30.636535+0000","last_clean":"2024-04-25T14:11:30.636535+0000","last_became_active":"2024-04-25T14:11:29.578975+0000","last_became_peered":"2024-04-25T14:11:29.578975+0000","last_unstale":"2024-04-25T14:11:30.636535+0000","last_undegraded":"2024-04-25T14:11:30.636535+0000","last_fullsized":"2024-04-25T14:11:30.636535+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:31:04.202332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"75'6","reported_seq":21,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.563528+0000","last_change":"2024-04-25T14:11:28.543093+0000","last_active":"2024-04-25T14:11:30.563528+0000","last_peered":"2024-04-25T14:11:30.563528+0000","last_clean":"2024-04-25T14:11:30.563528+0000","last_became_active":"2024-04-25T14:11:28.542407+0000","last_became_peered":"2024-04-25T14:11:28.542407+0000","last_unstale":"2024-04-25T14:11:30.563528+0000","last_undegraded":"2024-04-25T14:11:30.563528+0000","last_fullsized":"2024-04-25T14:11:30.563528+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:43.767375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.637791+0000","last_change":"2024-04-25T14:11:29.571896+0000","last_active":"2024-04-25T14:11:30.637791+0000","last_peered":"2024-04-25T14:11:30.637791+0000","last_clean":"2024-04-25T14:11:30.637791+0000","last_became_active":"2024-04-25T14:11:29.571537+0000","last_became_peered":"2024-04-25T14:11:29.571537+0000","last_unstale":"2024-04-25T14:11:30.637791+0000","last_undegraded":"2024-04-25T14:11:30.637791+0000","last_fullsized":"2024-04-25T14:11:30.637791+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:32:07.914841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'2","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T14:11:29.582109+0000","last_change":"2024-04-25T14:11:28.543293+0000","last_active":"2024-04-25T14:11:29.582109+0000","last_peered":"2024-04-25T14:11:29.582109+0000","last_clean":"2024-04-25T14:11:29.582109+0000","last_became_active":"2024-04-25T14:11:28.542967+0000","last_became_peered":"2024-04-25T14:11:28.542967+0000","last_unstale":"2024-04-25T14:11:29.582109+0000","last_undegraded":"2024-04-25T14:11:29.582109+0000","last_fullsized":"2024-04-25T14:11:29.582109+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:29.452792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.636928+0000","last_change":"2024-04-25T14:11:29.563877+0000","last_active":"2024-04-25T14:11:30.636928+0000","last_peered":"2024-04-25T14:11:30.636928+0000","last_clean":"2024-04-25T14:11:30.636928+0000","last_became_active":"2024-04-25T14:11:29.563535+0000","last_became_peered":"2024-04-25T14:11:29.563535+0000","last_unstale":"2024-04-25T14:11:30.636928+0000","last_undegraded":"2024-04-25T14:11:30.636928+0000","last_fullsized":"2024-04-25T14:11:30.636928+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:07:38.805863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.637739+0000","last_change":"2024-04-25T14:11:28.543787+0000","last_active":"2024-04-25T14:11:30.637739+0000","last_peered":"2024-04-25T14:11:30.637739+0000","last_clean":"2024-04-25T14:11:30.637739+0000","last_became_active":"2024-04-25T14:11:28.540816+0000","last_became_peered":"2024-04-25T14:11:28.540816+0000","last_unstale":"2024-04-25T14:11:30.637739+0000","last_undegraded":"2024-04-25T14:11:30.637739+0000","last_fullsized":"2024-04-25T14:11:30.637739+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:18.776280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.786057+0000","last_change":"2024-04-25T14:11:29.574785+0000","last_active":"2024-04-25T14:11:30.786057+0000","last_peered":"2024-04-25T14:11:30.786057+0000","last_clean":"2024-04-25T14:11:30.786057+0000","last_became_active":"2024-04-25T14:11:29.574422+0000","last_became_peered":"2024-04-25T14:11:29.574422+0000","last_unstale":"2024-04-25T14:11:30.786057+0000","last_undegraded":"2024-04-25T14:11:30.786057+0000","last_fullsized":"2024-04-25T14:11:30.786057+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:45:57.256214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.636964+0000","last_change":"2024-04-25T14:11:28.537450+0000","last_active":"2024-04-25T14:11:30.636964+0000","last_peered":"2024-04-25T14:11:30.636964+0000","last_clean":"2024-04-25T14:11:30.636964+0000","last_became_active":"2024-04-25T14:11:28.537088+0000","last_became_peered":"2024-04-25T14:11:28.537088+0000","last_unstale":"2024-04-25T14:11:30.636964+0000","last_undegraded":"2024-04-25T14:11:30.636964+0000","last_fullsized":"2024-04-25T14:11:30.636964+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:05.779706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.785530+0000","last_change":"2024-04-25T14:11:29.572789+0000","last_active":"2024-04-25T14:11:30.785530+0000","last_peered":"2024-04-25T14:11:30.785530+0000","last_clean":"2024-04-25T14:11:30.785530+0000","last_became_active":"2024-04-25T14:11:29.572201+0000","last_became_peered":"2024-04-25T14:11:29.572201+0000","last_unstale":"2024-04-25T14:11:30.785530+0000","last_undegraded":"2024-04-25T14:11:30.785530+0000","last_fullsized":"2024-04-25T14:11:30.785530+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:39:58.237070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"76'15","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.526981+0000","last_change":"2024-04-25T14:11:28.540577+0000","last_active":"2024-04-25T14:11:32.526981+0000","last_peered":"2024-04-25T14:11:32.526981+0000","last_clean":"2024-04-25T14:11:32.526981+0000","last_became_active":"2024-04-25T14:11:28.540212+0000","last_became_peered":"2024-04-25T14:11:28.540212+0000","last_unstale":"2024-04-25T14:11:32.526981+0000","last_undegraded":"2024-04-25T14:11:32.526981+0000","last_fullsized":"2024-04-25T14:11:32.526981+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:31.422332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.638097+0000","last_change":"2024-04-25T14:11:29.574952+0000","last_active":"2024-04-25T14:11:30.638097+0000","last_peered":"2024-04-25T14:11:30.638097+0000","last_clean":"2024-04-25T14:11:30.638097+0000","last_became_active":"2024-04-25T14:11:29.573677+0000","last_became_peered":"2024-04-25T14:11:29.573677+0000","last_unstale":"2024-04-25T14:11:30.638097+0000","last_undegraded":"2024-04-25T14:11:30.638097+0000","last_fullsized":"2024-04-25T14:11:30.638097+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:35.896103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'1","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.637501+0000","last_change":"2024-04-25T14:11:28.541356+0000","last_active":"2024-04-25T14:11:30.637501+0000","last_peered":"2024-04-25T14:11:30.637501+0000","last_clean":"2024-04-25T14:11:30.637501+0000","last_became_active":"2024-04-25T14:11:28.540976+0000","last_became_peered":"2024-04-25T14:11:28.540976+0000","last_unstale":"2024-04-25T14:11:30.637501+0000","last_undegraded":"2024-04-25T14:11:30.637501+0000","last_fullsized":"2024-04-25T14:11:30.637501+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:32:50.599900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.565549+0000","last_change":"2024-04-25T14:11:29.554317+0000","last_active":"2024-04-25T14:11:30.565549+0000","last_peered":"2024-04-25T14:11:30.565549+0000","last_clean":"2024-04-25T14:11:30.565549+0000","last_became_active":"2024-04-25T14:11:29.553935+0000","last_became_peered":"2024-04-25T14:11:29.553935+0000","last_unstale":"2024-04-25T14:11:30.565549+0000","last_undegraded":"2024-04-25T14:11:30.565549+0000","last_fullsized":"2024-04-25T14:11:30.565549+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:01.405964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"76'1","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.636638+0000","last_change":"2024-04-25T14:11:28.546892+0000","last_active":"2024-04-25T14:11:30.636638+0000","last_peered":"2024-04-25T14:11:30.636638+0000","last_clean":"2024-04-25T14:11:30.636638+0000","last_became_active":"2024-04-25T14:11:28.546566+0000","last_became_peered":"2024-04-25T14:11:28.546566+0000","last_unstale":"2024-04-25T14:11:30.636638+0000","last_undegraded":"2024-04-25T14:11:30.636638+0000","last_fullsized":"2024-04-25T14:11:30.636638+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:05.239635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.563584+0000","last_change":"2024-04-25T14:11:29.565083+0000","last_active":"2024-04-25T14:11:30.563584+0000","last_peered":"2024-04-25T14:11:30.563584+0000","last_clean":"2024-04-25T14:11:30.563584+0000","last_became_active":"2024-04-25T14:11:29.564062+0000","last_became_peered":"2024-04-25T14:11:29.564062+0000","last_unstale":"2024-04-25T14:11:30.563584+0000","last_undegraded":"2024-04-25T14:11:30.563584+0000","last_fullsized":"2024-04-25T14:11:30.563584+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:07:10.664113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.785729+0000","last_change":"2024-04-25T14:11:28.538649+0000","last_active":"2024-04-25T14:11:30.785729+0000","last_peered":"2024-04-25T14:11:30.785729+0000","last_clean":"2024-04-25T14:11:30.785729+0000","last_became_active":"2024-04-25T14:11:28.538353+0000","last_became_peered":"2024-04-25T14:11:28.538353+0000","last_unstale":"2024-04-25T14:11:30.785729+0000","last_undegraded":"2024-04-25T14:11:30.785729+0000","last_fullsized":"2024-04-25T14:11:30.785729+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:06.513039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.563535+0000","last_change":"2024-04-25T14:11:29.576263+0000","last_active":"2024-04-25T14:11:30.563535+0000","last_peered":"2024-04-25T14:11:30.563535+0000","last_clean":"2024-04-25T14:11:30.563535+0000","last_became_active":"2024-04-25T14:11:29.575877+0000","last_became_peered":"2024-04-25T14:11:29.575877+0000","last_unstale":"2024-04-25T14:11:30.563535+0000","last_undegraded":"2024-04-25T14:11:30.563535+0000","last_fullsized":"2024-04-25T14:11:30.563535+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:33:37.639537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.637390+0000","last_change":"2024-04-25T14:11:29.568220+0000","last_active":"2024-04-25T14:11:30.637390+0000","last_peered":"2024-04-25T14:11:30.637390+0000","last_clean":"2024-04-25T14:11:30.637390+0000","last_became_active":"2024-04-25T14:11:29.567741+0000","last_became_peered":"2024-04-25T14:11:29.567741+0000","last_unstale":"2024-04-25T14:11:30.637390+0000","last_undegraded":"2024-04-25T14:11:30.637390+0000","last_fullsized":"2024-04-25T14:11:30.637390+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:05.134466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"76'1","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:31.585278+0000","last_change":"2024-04-25T14:11:28.537503+0000","last_active":"2024-04-25T14:11:31.585278+0000","last_peered":"2024-04-25T14:11:31.585278+0000","last_clean":"2024-04-25T14:11:31.585278+0000","last_became_active":"2024-04-25T14:11:28.537169+0000","last_became_peered":"2024-04-25T14:11:28.537169+0000","last_unstale":"2024-04-25T14:11:31.585278+0000","last_undegraded":"2024-04-25T14:11:31.585278+0000","last_fullsized":"2024-04-25T14:11:31.585278+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:09.752988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.562696+0000","last_change":"2024-04-25T14:11:29.569487+0000","last_active":"2024-04-25T14:11:30.562696+0000","last_peered":"2024-04-25T14:11:30.562696+0000","last_clean":"2024-04-25T14:11:30.562696+0000","last_became_active":"2024-04-25T14:11:29.569148+0000","last_became_peered":"2024-04-25T14:11:29.569148+0000","last_unstale":"2024-04-25T14:11:30.562696+0000","last_undegraded":"2024-04-25T14:11:30.562696+0000","last_fullsized":"2024-04-25T14:11:30.562696+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:03.395620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.637257+0000","last_change":"2024-04-25T14:11:28.538216+0000","last_active":"2024-04-25T14:11:30.637257+0000","last_peered":"2024-04-25T14:11:30.637257+0000","last_clean":"2024-04-25T14:11:30.637257+0000","last_became_active":"2024-04-25T14:11:28.537772+0000","last_became_peered":"2024-04-25T14:11:28.537772+0000","last_unstale":"2024-04-25T14:11:30.637257+0000","last_undegraded":"2024-04-25T14:11:30.637257+0000","last_fullsized":"2024-04-25T14:11:30.637257+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:55.460966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":89,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.785401+0000","last_change":"2024-04-25T14:10:12.194216+0000","last_active":"2024-04-25T14:11:30.785401+0000","last_peered":"2024-04-25T14:11:30.785401+0000","last_clean":"2024-04-25T14:11:30.785401+0000","last_became_active":"2024-04-25T14:10:12.189616+0000","last_became_peered":"2024-04-25T14:10:12.189616+0000","last_unstale":"2024-04-25T14:11:30.785401+0000","last_undegraded":"2024-04-25T14:11:30.785401+0000","last_fullsized":"2024-04-25T14:11:30.785401+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:47.553264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:47.553264+0000","last_clean_scrub_stamp":"2024-04-25T14:06:47.553264+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:47.576058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.637008+0000","last_change":"2024-04-25T14:11:29.574985+0000","last_active":"2024-04-25T14:11:30.637008+0000","last_peered":"2024-04-25T14:11:30.637008+0000","last_clean":"2024-04-25T14:11:30.637008+0000","last_became_active":"2024-04-25T14:11:29.574629+0000","last_became_peered":"2024-04-25T14:11:29.574629+0000","last_unstale":"2024-04-25T14:11:30.637008+0000","last_undegraded":"2024-04-25T14:11:30.637008+0000","last_fullsized":"2024-04-25T14:11:30.637008+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:01.899889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.785674+0000","last_change":"2024-04-25T14:11:28.543251+0000","last_active":"2024-04-25T14:11:30.785674+0000","last_peered":"2024-04-25T14:11:30.785674+0000","last_clean":"2024-04-25T14:11:30.785674+0000","last_became_active":"2024-04-25T14:11:28.542439+0000","last_became_peered":"2024-04-25T14:11:28.542439+0000","last_unstale":"2024-04-25T14:11:30.785674+0000","last_undegraded":"2024-04-25T14:11:30.785674+0000","last_fullsized":"2024-04-25T14:11:30.785674+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:30:06.939009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"76'1","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:31.588391+0000","last_change":"2024-04-25T14:11:28.544269+0000","last_active":"2024-04-25T14:11:31.588391+0000","last_peered":"2024-04-25T14:11:31.588391+0000","last_clean":"2024-04-25T14:11:31.588391+0000","last_became_active":"2024-04-25T14:11:28.543897+0000","last_became_peered":"2024-04-25T14:11:28.543897+0000","last_unstale":"2024-04-25T14:11:31.588391+0000","last_undegraded":"2024-04-25T14:11:31.588391+0000","last_fullsized":"2024-04-25T14:11:31.588391+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:37.668187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.637384+0000","last_change":"2024-04-25T14:11:29.566428+0000","last_active":"2024-04-25T14:11:30.637384+0000","last_peered":"2024-04-25T14:11:30.637384+0000","last_clean":"2024-04-25T14:11:30.637384+0000","last_became_active":"2024-04-25T14:11:29.562598+0000","last_became_peered":"2024-04-25T14:11:29.562598+0000","last_unstale":"2024-04-25T14:11:30.637384+0000","last_undegraded":"2024-04-25T14:11:30.637384+0000","last_fullsized":"2024-04-25T14:11:30.637384+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:46.611359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"76'7","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:33.530432+0000","last_change":"2024-04-25T14:11:28.542571+0000","last_active":"2024-04-25T14:11:33.530432+0000","last_peered":"2024-04-25T14:11:33.530432+0000","last_clean":"2024-04-25T14:11:33.530432+0000","last_became_active":"2024-04-25T14:11:28.542272+0000","last_became_peered":"2024-04-25T14:11:28.542272+0000","last_unstale":"2024-04-25T14:11:33.530432+0000","last_undegraded":"2024-04-25T14:11:33.530432+0000","last_fullsized":"2024-04-25T14:11:33.530432+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:41:05.128068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.563023+0000","last_change":"2024-04-25T14:11:29.578565+0000","last_active":"2024-04-25T14:11:30.563023+0000","last_peered":"2024-04-25T14:11:30.563023+0000","last_clean":"2024-04-25T14:11:30.563023+0000","last_became_active":"2024-04-25T14:11:29.576782+0000","last_became_peered":"2024-04-25T14:11:29.576782+0000","last_unstale":"2024-04-25T14:11:30.563023+0000","last_undegraded":"2024-04-25T14:11:30.563023+0000","last_fullsized":"2024-04-25T14:11:30.563023+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:31.634602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'2","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.637163+0000","last_change":"2024-04-25T14:11:28.543838+0000","last_active":"2024-04-25T14:11:30.637163+0000","last_peered":"2024-04-25T14:11:30.637163+0000","last_clean":"2024-04-25T14:11:30.637163+0000","last_became_active":"2024-04-25T14:11:28.543511+0000","last_became_peered":"2024-04-25T14:11:28.543511+0000","last_unstale":"2024-04-25T14:11:30.637163+0000","last_undegraded":"2024-04-25T14:11:30.637163+0000","last_fullsized":"2024-04-25T14:11:30.637163+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:34.440234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.637420+0000","last_change":"2024-04-25T14:11:29.578686+0000","last_active":"2024-04-25T14:11:30.637420+0000","last_peered":"2024-04-25T14:11:30.637420+0000","last_clean":"2024-04-25T14:11:30.637420+0000","last_became_active":"2024-04-25T14:11:29.578324+0000","last_became_peered":"2024-04-25T14:11:29.578324+0000","last_unstale":"2024-04-25T14:11:30.637420+0000","last_undegraded":"2024-04-25T14:11:30.637420+0000","last_fullsized":"2024-04-25T14:11:30.637420+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:51:56.244792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"76'2","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:31.587097+0000","last_change":"2024-04-25T14:11:28.539443+0000","last_active":"2024-04-25T14:11:31.587097+0000","last_peered":"2024-04-25T14:11:31.587097+0000","last_clean":"2024-04-25T14:11:31.587097+0000","last_became_active":"2024-04-25T14:11:28.539113+0000","last_became_peered":"2024-04-25T14:11:28.539113+0000","last_unstale":"2024-04-25T14:11:31.587097+0000","last_undegraded":"2024-04-25T14:11:31.587097+0000","last_fullsized":"2024-04-25T14:11:31.587097+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:36.664972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.636879+0000","last_change":"2024-04-25T14:11:29.575451+0000","last_active":"2024-04-25T14:11:30.636879+0000","last_peered":"2024-04-25T14:11:30.636879+0000","last_clean":"2024-04-25T14:11:30.636879+0000","last_became_active":"2024-04-25T14:11:29.574983+0000","last_became_peered":"2024-04-25T14:11:29.574983+0000","last_unstale":"2024-04-25T14:11:30.636879+0000","last_undegraded":"2024-04-25T14:11:30.636879+0000","last_fullsized":"2024-04-25T14:11:30.636879+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:53.317608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.562283+0000","last_change":"2024-04-25T14:11:28.546394+0000","last_active":"2024-04-25T14:11:30.562283+0000","last_peered":"2024-04-25T14:11:30.562283+0000","last_clean":"2024-04-25T14:11:30.562283+0000","last_became_active":"2024-04-25T14:11:28.546064+0000","last_became_peered":"2024-04-25T14:11:28.546064+0000","last_unstale":"2024-04-25T14:11:30.562283+0000","last_undegraded":"2024-04-25T14:11:30.562283+0000","last_fullsized":"2024-04-25T14:11:30.562283+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:15.301492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.565402+0000","last_change":"2024-04-25T14:11:29.569584+0000","last_active":"2024-04-25T14:11:30.565402+0000","last_peered":"2024-04-25T14:11:30.565402+0000","last_clean":"2024-04-25T14:11:30.565402+0000","last_became_active":"2024-04-25T14:11:29.569217+0000","last_became_peered":"2024-04-25T14:11:29.569217+0000","last_unstale":"2024-04-25T14:11:30.565402+0000","last_undegraded":"2024-04-25T14:11:30.565402+0000","last_fullsized":"2024-04-25T14:11:30.565402+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:05.708277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T14:11:29.553096+0000","last_change":"2024-04-25T14:11:28.544399+0000","last_active":"2024-04-25T14:11:29.553096+0000","last_peered":"2024-04-25T14:11:29.553096+0000","last_clean":"2024-04-25T14:11:29.553096+0000","last_became_active":"2024-04-25T14:11:28.544098+0000","last_became_peered":"2024-04-25T14:11:28.544098+0000","last_unstale":"2024-04-25T14:11:29.553096+0000","last_undegraded":"2024-04-25T14:11:29.553096+0000","last_fullsized":"2024-04-25T14:11:29.553096+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:05.148386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.562679+0000","last_change":"2024-04-25T14:11:29.573081+0000","last_active":"2024-04-25T14:11:30.562679+0000","last_peered":"2024-04-25T14:11:30.562679+0000","last_clean":"2024-04-25T14:11:30.562679+0000","last_became_active":"2024-04-25T14:11:29.572654+0000","last_became_peered":"2024-04-25T14:11:29.572654+0000","last_unstale":"2024-04-25T14:11:30.562679+0000","last_undegraded":"2024-04-25T14:11:30.562679+0000","last_fullsized":"2024-04-25T14:11:30.562679+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:04.076576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"76'4","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:31.587584+0000","last_change":"2024-04-25T14:11:28.542488+0000","last_active":"2024-04-25T14:11:31.587584+0000","last_peered":"2024-04-25T14:11:31.587584+0000","last_clean":"2024-04-25T14:11:31.587584+0000","last_became_active":"2024-04-25T14:11:28.542171+0000","last_became_peered":"2024-04-25T14:11:28.542171+0000","last_unstale":"2024-04-25T14:11:31.587584+0000","last_undegraded":"2024-04-25T14:11:31.587584+0000","last_fullsized":"2024-04-25T14:11:31.587584+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:48.201762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.786093+0000","last_change":"2024-04-25T14:11:29.573805+0000","last_active":"2024-04-25T14:11:30.786093+0000","last_peered":"2024-04-25T14:11:30.786093+0000","last_clean":"2024-04-25T14:11:30.786093+0000","last_became_active":"2024-04-25T14:11:29.573420+0000","last_became_peered":"2024-04-25T14:11:29.573420+0000","last_unstale":"2024-04-25T14:11:30.786093+0000","last_undegraded":"2024-04-25T14:11:30.786093+0000","last_fullsized":"2024-04-25T14:11:30.786093+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:18.721977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T14:11:29.554250+0000","last_change":"2024-04-25T14:11:28.552863+0000","last_active":"2024-04-25T14:11:29.554250+0000","last_peered":"2024-04-25T14:11:29.554250+0000","last_clean":"2024-04-25T14:11:29.554250+0000","last_became_active":"2024-04-25T14:11:28.552461+0000","last_became_peered":"2024-04-25T14:11:28.552461+0000","last_unstale":"2024-04-25T14:11:29.554250+0000","last_undegraded":"2024-04-25T14:11:29.554250+0000","last_fullsized":"2024-04-25T14:11:29.554250+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:46.264473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.786062+0000","last_change":"2024-04-25T14:11:29.571268+0000","last_active":"2024-04-25T14:11:30.786062+0000","last_peered":"2024-04-25T14:11:30.786062+0000","last_clean":"2024-04-25T14:11:30.786062+0000","last_became_active":"2024-04-25T14:11:29.570876+0000","last_became_peered":"2024-04-25T14:11:29.570876+0000","last_unstale":"2024-04-25T14:11:30.786062+0000","last_undegraded":"2024-04-25T14:11:30.786062+0000","last_fullsized":"2024-04-25T14:11:30.786062+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:15.093635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.637751+0000","last_change":"2024-04-25T14:11:28.543343+0000","last_active":"2024-04-25T14:11:30.637751+0000","last_peered":"2024-04-25T14:11:30.637751+0000","last_clean":"2024-04-25T14:11:30.637751+0000","last_became_active":"2024-04-25T14:11:28.540790+0000","last_became_peered":"2024-04-25T14:11:28.540790+0000","last_unstale":"2024-04-25T14:11:30.637751+0000","last_undegraded":"2024-04-25T14:11:30.637751+0000","last_fullsized":"2024-04-25T14:11:30.637751+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:20.498016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.785925+0000","last_change":"2024-04-25T14:11:29.566161+0000","last_active":"2024-04-25T14:11:30.785925+0000","last_peered":"2024-04-25T14:11:30.785925+0000","last_clean":"2024-04-25T14:11:30.785925+0000","last_became_active":"2024-04-25T14:11:29.564197+0000","last_became_peered":"2024-04-25T14:11:29.564197+0000","last_unstale":"2024-04-25T14:11:30.785925+0000","last_undegraded":"2024-04-25T14:11:30.785925+0000","last_fullsized":"2024-04-25T14:11:30.785925+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:35.291546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.636425+0000","last_change":"2024-04-25T14:11:28.543479+0000","last_active":"2024-04-25T14:11:30.636425+0000","last_peered":"2024-04-25T14:11:30.636425+0000","last_clean":"2024-04-25T14:11:30.636425+0000","last_became_active":"2024-04-25T14:11:28.543167+0000","last_became_peered":"2024-04-25T14:11:28.543167+0000","last_unstale":"2024-04-25T14:11:30.636425+0000","last_undegraded":"2024-04-25T14:11:30.636425+0000","last_fullsized":"2024-04-25T14:11:30.636425+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:54:53.572809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.785969+0000","last_change":"2024-04-25T14:11:29.572548+0000","last_active":"2024-04-25T14:11:30.785969+0000","last_peered":"2024-04-25T14:11:30.785969+0000","last_clean":"2024-04-25T14:11:30.785969+0000","last_became_active":"2024-04-25T14:11:29.572167+0000","last_became_peered":"2024-04-25T14:11:29.572167+0000","last_unstale":"2024-04-25T14:11:30.785969+0000","last_undegraded":"2024-04-25T14:11:30.785969+0000","last_fullsized":"2024-04-25T14:11:30.785969+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:38.423331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.565601+0000","last_change":"2024-04-25T14:11:29.560956+0000","last_active":"2024-04-25T14:11:30.565601+0000","last_peered":"2024-04-25T14:11:30.565601+0000","last_clean":"2024-04-25T14:11:30.565601+0000","last_became_active":"2024-04-25T14:11:29.560519+0000","last_became_peered":"2024-04-25T14:11:29.560519+0000","last_unstale":"2024-04-25T14:11:30.565601+0000","last_undegraded":"2024-04-25T14:11:30.565601+0000","last_fullsized":"2024-04-25T14:11:30.565601+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:53:47.677923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T14:11:29.578220+0000","last_change":"2024-04-25T14:11:29.574372+0000","last_active":"2024-04-25T14:11:29.578220+0000","last_peered":"2024-04-25T14:11:29.578220+0000","last_clean":"2024-04-25T14:11:29.578220+0000","last_became_active":"2024-04-25T14:11:29.573995+0000","last_became_peered":"2024-04-25T14:11:29.573995+0000","last_unstale":"2024-04-25T14:11:29.578220+0000","last_undegraded":"2024-04-25T14:11:29.578220+0000","last_fullsized":"2024-04-25T14:11:29.578220+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:40.130748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T14:11:29.579258+0000","last_change":"2024-04-25T14:11:29.578324+0000","last_active":"2024-04-25T14:11:29.579258+0000","last_peered":"2024-04-25T14:11:29.579258+0000","last_clean":"2024-04-25T14:11:29.579258+0000","last_became_active":"2024-04-25T14:11:29.577917+0000","last_became_peered":"2024-04-25T14:11:29.577917+0000","last_unstale":"2024-04-25T14:11:29.579258+0000","last_undegraded":"2024-04-25T14:11:29.579258+0000","last_fullsized":"2024-04-25T14:11:29.579258+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:47.358075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.637153+0000","last_change":"2024-04-25T14:11:29.578879+0000","last_active":"2024-04-25T14:11:30.637153+0000","last_peered":"2024-04-25T14:11:30.637153+0000","last_clean":"2024-04-25T14:11:30.637153+0000","last_became_active":"2024-04-25T14:11:29.578551+0000","last_became_peered":"2024-04-25T14:11:29.578551+0000","last_unstale":"2024-04-25T14:11:30.637153+0000","last_undegraded":"2024-04-25T14:11:30.637153+0000","last_fullsized":"2024-04-25T14:11:30.637153+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:41.083794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.637481+0000","last_change":"2024-04-25T14:11:29.574054+0000","last_active":"2024-04-25T14:11:30.637481+0000","last_peered":"2024-04-25T14:11:30.637481+0000","last_clean":"2024-04-25T14:11:30.637481+0000","last_became_active":"2024-04-25T14:11:29.573700+0000","last_became_peered":"2024-04-25T14:11:29.573700+0000","last_unstale":"2024-04-25T14:11:30.637481+0000","last_undegraded":"2024-04-25T14:11:30.637481+0000","last_fullsized":"2024-04-25T14:11:30.637481+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:23.304580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.565807+0000","last_change":"2024-04-25T14:11:29.553768+0000","last_active":"2024-04-25T14:11:30.565807+0000","last_peered":"2024-04-25T14:11:30.565807+0000","last_clean":"2024-04-25T14:11:30.565807+0000","last_became_active":"2024-04-25T14:11:29.553376+0000","last_became_peered":"2024-04-25T14:11:29.553376+0000","last_unstale":"2024-04-25T14:11:30.565807+0000","last_undegraded":"2024-04-25T14:11:30.565807+0000","last_fullsized":"2024-04-25T14:11:30.565807+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:46.688368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T14:11:29.563685+0000","last_change":"2024-04-25T14:11:29.563122+0000","last_active":"2024-04-25T14:11:29.563685+0000","last_peered":"2024-04-25T14:11:29.563685+0000","last_clean":"2024-04-25T14:11:29.563685+0000","last_became_active":"2024-04-25T14:11:29.562717+0000","last_became_peered":"2024-04-25T14:11:29.562717+0000","last_unstale":"2024-04-25T14:11:29.563685+0000","last_undegraded":"2024-04-25T14:11:29.563685+0000","last_fullsized":"2024-04-25T14:11:29.563685+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:44.829953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.562938+0000","last_change":"2024-04-25T14:11:29.576487+0000","last_active":"2024-04-25T14:11:30.562938+0000","last_peered":"2024-04-25T14:11:30.562938+0000","last_clean":"2024-04-25T14:11:30.562938+0000","last_became_active":"2024-04-25T14:11:29.575070+0000","last_became_peered":"2024-04-25T14:11:29.575070+0000","last_unstale":"2024-04-25T14:11:30.562938+0000","last_undegraded":"2024-04-25T14:11:30.562938+0000","last_fullsized":"2024-04-25T14:11:30.562938+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:13.803505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.637011+0000","last_change":"2024-04-25T14:11:29.568733+0000","last_active":"2024-04-25T14:11:30.637011+0000","last_peered":"2024-04-25T14:11:30.637011+0000","last_clean":"2024-04-25T14:11:30.637011+0000","last_became_active":"2024-04-25T14:11:29.567665+0000","last_became_peered":"2024-04-25T14:11:29.567665+0000","last_unstale":"2024-04-25T14:11:30.637011+0000","last_undegraded":"2024-04-25T14:11:30.637011+0000","last_fullsized":"2024-04-25T14:11:30.637011+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:57.956199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.562318+0000","last_change":"2024-04-25T14:11:29.574648+0000","last_active":"2024-04-25T14:11:30.562318+0000","last_peered":"2024-04-25T14:11:30.562318+0000","last_clean":"2024-04-25T14:11:30.562318+0000","last_became_active":"2024-04-25T14:11:29.574247+0000","last_became_peered":"2024-04-25T14:11:29.574247+0000","last_unstale":"2024-04-25T14:11:30.562318+0000","last_undegraded":"2024-04-25T14:11:30.562318+0000","last_fullsized":"2024-04-25T14:11:30.562318+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:57.110017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.636493+0000","last_change":"2024-04-25T14:11:29.573633+0000","last_active":"2024-04-25T14:11:30.636493+0000","last_peered":"2024-04-25T14:11:30.636493+0000","last_clean":"2024-04-25T14:11:30.636493+0000","last_became_active":"2024-04-25T14:11:29.573303+0000","last_became_peered":"2024-04-25T14:11:29.573303+0000","last_unstale":"2024-04-25T14:11:30.636493+0000","last_undegraded":"2024-04-25T14:11:30.636493+0000","last_fullsized":"2024-04-25T14:11:30.636493+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:11.458117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.636835+0000","last_change":"2024-04-25T14:11:29.569136+0000","last_active":"2024-04-25T14:11:30.636835+0000","last_peered":"2024-04-25T14:11:30.636835+0000","last_clean":"2024-04-25T14:11:30.636835+0000","last_became_active":"2024-04-25T14:11:29.568801+0000","last_became_peered":"2024-04-25T14:11:29.568801+0000","last_unstale":"2024-04-25T14:11:30.636835+0000","last_undegraded":"2024-04-25T14:11:30.636835+0000","last_fullsized":"2024-04-25T14:11:30.636835+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:30.969081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T14:11:30.565404+0000","last_change":"2024-04-25T14:11:29.566754+0000","last_active":"2024-04-25T14:11:30.565404+0000","last_peered":"2024-04-25T14:11:30.565404+0000","last_clean":"2024-04-25T14:11:30.565404+0000","last_became_active":"2024-04-25T14:11:29.566368+0000","last_became_peered":"2024-04-25T14:11:29.566368+0000","last_unstale":"2024-04-25T14:11:30.565404+0000","last_undegraded":"2024-04-25T14:11:30.565404+0000","last_fullsized":"2024-04-25T14:11:30.565404+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:37.057718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281277,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2544,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":80,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":159744,"data_stored":9666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678030,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31972,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713180,"statfs":{"total":95995035648,"available":95962296320,"internally_reserved":0,"allocated":806912,"data_stored":386813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874259,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":868352,"data_stored":970417,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037784,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27828,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717324,"statfs":{"total":95995035648,"available":95966539776,"internally_reserved":0,"allocated":831488,"data_stored":387721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234014,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31984,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713168,"statfs":{"total":95995035648,"available":95962284032,"internally_reserved":0,"allocated":827392,"data_stored":387551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430242,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32020,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713132,"statfs":{"total":95995035648,"available":95962247168,"internally_reserved":0,"allocated":860160,"data_stored":977763,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593768,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31972,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713180,"statfs":{"total":95995035648,"available":95962296320,"internally_reserved":0,"allocated":806912,"data_stored":386813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":819200,"data_stored":388425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31992,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713160,"statfs":{"total":95995035648,"available":95962275840,"internally_reserved":0,"allocated":835584,"data_stored":388291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717340,"statfs":{"total":95995035648,"available":95966556160,"internally_reserved":0,"allocated":811008,"data_stored":379889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149751,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717348,"statfs":{"total":95995035648,"available":95966564352,"internally_reserved":0,"allocated":802816,"data_stored":378347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953521,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27808,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717344,"statfs":{"total":95995035648,"available":95966560256,"internally_reserved":0,"allocated":798720,"data_stored":378265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":37,"seq":158913789997,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":856064,"data_stored":968793,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:11:36.750 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T14:11:36.750 INFO:tasks.cephadm:Setting up client nodes... 2024-04-25T14:11:36.751 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-25T14:11:36.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:36 smithi097 ceph-mon[32720]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:11:36.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:36 smithi097 ceph-mon[32720]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:11:36.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:36 smithi097 ceph-mon[32720]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:11:36.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:36 smithi097 ceph-mon[32720]: pgmap v283: 97 pgs: 97 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 745 B/s rd, 3.3 KiB/s wr, 12 op/s 2024-04-25T14:11:36.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:36 smithi097 ceph-mon[32720]: mds.? [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] up:active 2024-04-25T14:11:36.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:36 smithi097 ceph-mon[32720]: fsmap cephfs:5 2 up:standby-replay 5 up:standby 5 up:active 2024-04-25T14:11:36.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:36 smithi097 ceph-mon[32720]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T14:11:36.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:36 smithi097 ceph-mon[32720]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T14:11:36.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:36 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/4249911663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:36.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:36 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/3462612178' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:11:36.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:36 smithi151 ceph-mon[32990]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:11:36.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:36 smithi151 ceph-mon[32990]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:11:36.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:36 smithi151 ceph-mon[32990]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:11:36.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:36 smithi151 ceph-mon[32990]: pgmap v283: 97 pgs: 97 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 745 B/s rd, 3.3 KiB/s wr, 12 op/s 2024-04-25T14:11:36.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:36 smithi151 ceph-mon[32990]: mds.? [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] up:active 2024-04-25T14:11:36.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:36 smithi151 ceph-mon[32990]: fsmap cephfs:5 2 up:standby-replay 5 up:standby 5 up:active 2024-04-25T14:11:36.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:36 smithi151 ceph-mon[32990]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T14:11:36.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:36 smithi151 ceph-mon[32990]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T14:11:36.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:36 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/4249911663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:36.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:36 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/3462612178' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:11:37.025 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:11:37.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:37 smithi096 ceph-mon[29207]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:11:37.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:37 smithi097 ceph-mon[32720]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:11:37.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:37 smithi151 ceph-mon[32990]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:11:38.793 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:38 smithi096 ceph-mon[29207]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:11:38.793 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:38 smithi096 ceph-mon[29207]: pgmap v284: 97 pgs: 97 active+clean; 620 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 13 KiB/s wr, 31 op/s 2024-04-25T14:11:38.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:38.913+0000 7fb1228cc700 1 -- 172.21.15.96:0/2017595551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb11c105340 msgr2=0x7fb11c105750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:38.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:38.913+0000 7fb1228cc700 1 --2- 172.21.15.96:0/2017595551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb11c105340 0x7fb11c105750 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb110009990 tx=0x7fb110009ca0 comp rx=0 tx=0).stop 2024-04-25T14:11:38.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:38.913+0000 7fb1228cc700 1 -- 172.21.15.96:0/2017595551 shutdown_connections 2024-04-25T14:11:38.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:38.913+0000 7fb1228cc700 1 --2- 172.21.15.96:0/2017595551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb11c105ce0 0x7fb11c10c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:38.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:38.913+0000 7fb1228cc700 1 --2- 172.21.15.96:0/2017595551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb11c104210 0x7fb11c104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:38.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:38.913+0000 7fb1228cc700 1 --2- 172.21.15.96:0/2017595551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb11c105340 0x7fb11c105750 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:38.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:38.913+0000 7fb1228cc700 1 -- 172.21.15.96:0/2017595551 >> 172.21.15.96:0/2017595551 conn(0x7fb11c0ff870 msgr2=0x7fb11c101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:38.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:38.913+0000 7fb1228cc700 1 -- 172.21.15.96:0/2017595551 shutdown_connections 2024-04-25T14:11:38.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:38.913+0000 7fb1228cc700 1 -- 172.21.15.96:0/2017595551 wait complete. 2024-04-25T14:11:38.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:38.914+0000 7fb1228cc700 1 Processor -- start 2024-04-25T14:11:38.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:38.914+0000 7fb1228cc700 1 -- start start 2024-04-25T14:11:38.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:38.914+0000 7fb1228cc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb11c104210 0x7fb11c078440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:38.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:38.914+0000 7fb1228cc700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb11c105340 0x7fb11c076af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:38.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:38.914+0000 7fb1228cc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb11c105ce0 0x7fb11c077030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:38.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:38.914+0000 7fb1228cc700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb11c10fc70 con 0x7fb11c104210 2024-04-25T14:11:38.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:38.914+0000 7fb1228cc700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb11c10faf0 con 0x7fb11c105ce0 2024-04-25T14:11:38.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:38.914+0000 7fb1228cc700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb11c10fdf0 con 0x7fb11c105340 2024-04-25T14:11:38.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:38.914+0000 7fb11bfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb11c104210 0x7fb11c078440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:38.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:38.914+0000 7fb11bfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb11c104210 0x7fb11c078440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35466/0 (socket says 172.21.15.96:35466) 2024-04-25T14:11:38.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:38.914+0000 7fb11bfff700 1 -- 172.21.15.96:0/993451817 learned_addr learned my addr 172.21.15.96:0/993451817 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:11:38.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:38.914+0000 7fb11b7fe700 1 --2- 172.21.15.96:0/993451817 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb11c105340 0x7fb11c076af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:38.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:38.914+0000 7fb120e69700 1 --2- 172.21.15.96:0/993451817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb11c105ce0 0x7fb11c077030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:38.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:38.915+0000 7fb11bfff700 1 -- 172.21.15.96:0/993451817 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb11c105340 msgr2=0x7fb11c076af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:38.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:38.915+0000 7fb11bfff700 1 --2- 172.21.15.96:0/993451817 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb11c105340 0x7fb11c076af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:38.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:38.915+0000 7fb11bfff700 1 -- 172.21.15.96:0/993451817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb11c105ce0 msgr2=0x7fb11c077030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:38.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:38.915+0000 7fb11bfff700 1 --2- 172.21.15.96:0/993451817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb11c105ce0 0x7fb11c077030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:38.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:38.915+0000 7fb11bfff700 1 -- 172.21.15.96:0/993451817 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb11c077790 con 0x7fb11c104210 2024-04-25T14:11:38.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:38.915+0000 7fb11bfff700 1 --2- 172.21.15.96:0/993451817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb11c104210 0x7fb11c078440 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb10c00d950 tx=0x7fb10c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:38.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:38.915+0000 7fb1197fa700 1 -- 172.21.15.96:0/993451817 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10c017070 con 0x7fb11c104210 2024-04-25T14:11:38.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:38.915+0000 7fb1197fa700 1 -- 172.21.15.96:0/993451817 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb10c00f690 con 0x7fb11c104210 2024-04-25T14:11:38.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:38.915+0000 7fb1228cc700 1 -- 172.21.15.96:0/993451817 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb11c1a4880 con 0x7fb11c104210 2024-04-25T14:11:38.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:38.915+0000 7fb1197fa700 1 -- 172.21.15.96:0/993451817 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10c003aa0 con 0x7fb11c104210 2024-04-25T14:11:38.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:38.915+0000 7fb1228cc700 1 -- 172.21.15.96:0/993451817 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb11c1a4d60 con 0x7fb11c104210 2024-04-25T14:11:38.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:38.916+0000 7fb1228cc700 1 -- 172.21.15.96:0/993451817 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb11c051190 con 0x7fb11c104210 2024-04-25T14:11:38.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:38.917+0000 7fb1197fa700 1 -- 172.21.15.96:0/993451817 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fb10c00f840 con 0x7fb11c104210 2024-04-25T14:11:38.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:38.918+0000 7fb1197fa700 1 --2- 172.21.15.96:0/993451817 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb1040afc10 0x7fb1040b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:38.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:38.918+0000 7fb11b7fe700 1 --2- 172.21.15.96:0/993451817 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb1040afc10 0x7fb1040b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:38.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:38.918+0000 7fb1197fa700 1 -- 172.21.15.96:0/993451817 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7fb10c0dee80 con 0x7fb11c104210 2024-04-25T14:11:38.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:38.919+0000 7fb11b7fe700 1 --2- 172.21.15.96:0/993451817 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb1040afc10 0x7fb1040b2120 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb11000b5c0 tx=0x7fb11001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:38.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:38.922+0000 7fb1197fa700 1 -- 172.21.15.96:0/993451817 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb10c020030 con 0x7fb11c104210 2024-04-25T14:11:38.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:38 smithi097 ceph-mon[32720]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:11:38.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:38 smithi097 ceph-mon[32720]: pgmap v284: 97 pgs: 97 active+clean; 620 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 13 KiB/s wr, 31 op/s 2024-04-25T14:11:38.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:38 smithi151 ceph-mon[32990]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:11:38.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:38 smithi151 ceph-mon[32990]: pgmap v284: 97 pgs: 97 active+clean; 620 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 13 KiB/s wr, 31 op/s 2024-04-25T14:11:39.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:39.200+0000 7fb1228cc700 1 -- 172.21.15.96:0/993451817 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fb11c02e510 con 0x7fb11c104210 2024-04-25T14:11:39.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:39.203+0000 7fb1197fa700 1 -- 172.21.15.96:0/993451817 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fb10c0a7ab0 con 0x7fb11c104210 2024-04-25T14:11:39.203 INFO:teuthology.orchestra.run.smithi096.stdout:[client.0] 2024-04-25T14:11:39.203 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQCbZCpmQygODBAAivTJMATdGOK1IK1Al798vA== 2024-04-25T14:11:39.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:39.205+0000 7fb1228cc700 1 -- 172.21.15.96:0/993451817 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb1040afc10 msgr2=0x7fb1040b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:39.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:39.205+0000 7fb1228cc700 1 --2- 172.21.15.96:0/993451817 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb1040afc10 0x7fb1040b2120 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb11000b5c0 tx=0x7fb11001a040 comp rx=0 tx=0).stop 2024-04-25T14:11:39.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:39.205+0000 7fb1228cc700 1 -- 172.21.15.96:0/993451817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb11c104210 msgr2=0x7fb11c078440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:39.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:39.205+0000 7fb1228cc700 1 --2- 172.21.15.96:0/993451817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb11c104210 0x7fb11c078440 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb10c00d950 tx=0x7fb10c00dc60 comp rx=0 tx=0).stop 2024-04-25T14:11:39.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:39.205+0000 7fb1228cc700 1 -- 172.21.15.96:0/993451817 shutdown_connections 2024-04-25T14:11:39.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:39.205+0000 7fb1228cc700 1 --2- 172.21.15.96:0/993451817 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb1040afc10 0x7fb1040b2120 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:39.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:39.205+0000 7fb1228cc700 1 --2- 172.21.15.96:0/993451817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb11c105ce0 0x7fb11c077030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:39.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:39.205+0000 7fb1228cc700 1 --2- 172.21.15.96:0/993451817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb11c104210 0x7fb11c078440 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:39.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:39.205+0000 7fb1228cc700 1 --2- 172.21.15.96:0/993451817 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb11c105340 0x7fb11c076af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:39.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:39.205+0000 7fb1228cc700 1 -- 172.21.15.96:0/993451817 >> 172.21.15.96:0/993451817 conn(0x7fb11c0ff870 msgr2=0x7fb11c108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:39.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:39.205+0000 7fb1228cc700 1 -- 172.21.15.96:0/993451817 shutdown_connections 2024-04-25T14:11:39.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:39.205+0000 7fb1228cc700 1 -- 172.21.15.96:0/993451817 wait complete. 2024-04-25T14:11:39.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:39 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/993451817' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T14:11:39.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:39 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/993451817' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T14:11:39.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:39 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/993451817' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T14:11:39.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:39 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/993451817' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T14:11:39.932 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T14:11:39.932 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-25T14:11:39.933 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T14:11:39.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:39 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/993451817' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T14:11:39.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:39 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/993451817' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T14:11:39.967 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T14:11:39.968 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-25T14:11:39.968 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-25T14:11:40.245 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:11:40.643 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:40 smithi096 ceph-mon[29207]: pgmap v285: 97 pgs: 97 active+clean; 621 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 10 KiB/s wr, 28 op/s 2024-04-25T14:11:40.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:40 smithi097 ceph-mon[32720]: pgmap v285: 97 pgs: 97 active+clean; 621 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 10 KiB/s wr, 28 op/s 2024-04-25T14:11:40.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:40 smithi151 ceph-mon[32990]: pgmap v285: 97 pgs: 97 active+clean; 621 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 10 KiB/s wr, 28 op/s 2024-04-25T14:11:41.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:41.997+0000 7f7de3d2e700 1 -- 172.21.15.96:0/981853211 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ddc107c90 msgr2=0x7f7ddc1080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:41.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:41.997+0000 7f7de3d2e700 1 --2- 172.21.15.96:0/981853211 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ddc107c90 0x7f7ddc1080a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc0099c0 tx=0x7f7dcc009cd0 comp rx=0 tx=0).stop 2024-04-25T14:11:41.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:41.998+0000 7f7de3d2e700 1 -- 172.21.15.96:0/981853211 shutdown_connections 2024-04-25T14:11:42.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:41.998+0000 7f7de3d2e700 1 --2- 172.21.15.96:0/981853211 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ddc1085e0 0x7f7ddc10ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:42.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:41.998+0000 7f7de3d2e700 1 --2- 172.21.15.96:0/981853211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ddc103360 0x7f7ddc103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:42.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:41.998+0000 7f7de3d2e700 1 --2- 172.21.15.96:0/981853211 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ddc107c90 0x7f7ddc1080a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:42.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:41.998+0000 7f7de3d2e700 1 -- 172.21.15.96:0/981853211 >> 172.21.15.96:0/981853211 conn(0x7f7ddc0fec90 msgr2=0x7f7ddc1010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:42.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:41.998+0000 7f7de3d2e700 1 -- 172.21.15.96:0/981853211 shutdown_connections 2024-04-25T14:11:42.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:41.998+0000 7f7de3d2e700 1 -- 172.21.15.96:0/981853211 wait complete. 2024-04-25T14:11:42.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:41.998+0000 7f7de3d2e700 1 Processor -- start 2024-04-25T14:11:42.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:41.998+0000 7f7de3d2e700 1 -- start start 2024-04-25T14:11:42.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:41.999+0000 7f7de3d2e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ddc103360 0x7f7ddc19baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:42.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:41.999+0000 7f7de3d2e700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ddc107c90 0x7f7ddc19c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:42.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:41.999+0000 7f7de3d2e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ddc1085e0 0x7f7ddc19c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:42.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:41.999+0000 7f7de3d2e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7ddc113210 con 0x7f7ddc1085e0 2024-04-25T14:11:42.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:41.999+0000 7f7de3d2e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7ddc113090 con 0x7f7ddc103360 2024-04-25T14:11:42.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:41.999+0000 7f7de3d2e700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7ddc113390 con 0x7f7ddc107c90 2024-04-25T14:11:42.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:41.999+0000 7f7de22cb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ddc1085e0 0x7f7ddc19c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:42.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:41.999+0000 7f7de22cb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ddc1085e0 0x7f7ddc19c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57954/0 (socket says 172.21.15.96:57954) 2024-04-25T14:11:42.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:41.999+0000 7f7de22cb700 1 -- 172.21.15.96:0/1243536871 learned_addr learned my addr 172.21.15.96:0/1243536871 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:11:42.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:41.999+0000 7f7de1aca700 1 --2- 172.21.15.96:0/1243536871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ddc103360 0x7f7ddc19baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:42.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:41.999+0000 7f7de12c9700 1 --2- 172.21.15.96:0/1243536871 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ddc107c90 0x7f7ddc19c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:42.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:41.999+0000 7f7de22cb700 1 -- 172.21.15.96:0/1243536871 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ddc107c90 msgr2=0x7f7ddc19c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:42.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:41.999+0000 7f7de22cb700 1 --2- 172.21.15.96:0/1243536871 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ddc107c90 0x7f7ddc19c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:42.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:41.999+0000 7f7de22cb700 1 -- 172.21.15.96:0/1243536871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ddc103360 msgr2=0x7f7ddc19baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:42.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:41.999+0000 7f7de22cb700 1 --2- 172.21.15.96:0/1243536871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ddc103360 0x7f7ddc19baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:42.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:41.999+0000 7f7de22cb700 1 -- 172.21.15.96:0/1243536871 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ddc1a6e40 con 0x7f7ddc1085e0 2024-04-25T14:11:42.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:41.999+0000 7f7de1aca700 1 --2- 172.21.15.96:0/1243536871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ddc103360 0x7f7ddc19baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:11:42.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:41.999+0000 7f7de22cb700 1 --2- 172.21.15.96:0/1243536871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ddc1085e0 0x7f7ddc19c590 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4007ee0 tx=0x7f7dd400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:42.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:42.000+0000 7f7dd2ffd700 1 -- 172.21.15.96:0/1243536871 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd400dbd0 con 0x7f7ddc1085e0 2024-04-25T14:11:42.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:42.000+0000 7f7de3d2e700 1 -- 172.21.15.96:0/1243536871 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ddc1a7130 con 0x7f7ddc1085e0 2024-04-25T14:11:42.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:42.000+0000 7f7dd2ffd700 1 -- 172.21.15.96:0/1243536871 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7dd400dd70 con 0x7f7ddc1085e0 2024-04-25T14:11:42.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:42.000+0000 7f7dd2ffd700 1 -- 172.21.15.96:0/1243536871 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd4005990 con 0x7f7ddc1085e0 2024-04-25T14:11:42.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:42.000+0000 7f7de3d2e700 1 -- 172.21.15.96:0/1243536871 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ddc1a7640 con 0x7f7ddc1085e0 2024-04-25T14:11:42.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:42.000+0000 7f7de3d2e700 1 -- 172.21.15.96:0/1243536871 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ddc051190 con 0x7f7ddc1085e0 2024-04-25T14:11:42.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:42.002+0000 7f7dd2ffd700 1 -- 172.21.15.96:0/1243536871 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f7dd401a020 con 0x7f7ddc1085e0 2024-04-25T14:11:42.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:42.003+0000 7f7dd2ffd700 1 --2- 172.21.15.96:0/1243536871 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f7dc80afc10 0x7f7dc80b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:42.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:42.003+0000 7f7de1aca700 1 --2- 172.21.15.96:0/1243536871 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f7dc80afc10 0x7f7dc80b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:42.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:42.003+0000 7f7dd2ffd700 1 -- 172.21.15.96:0/1243536871 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7f7dd40de9a0 con 0x7f7ddc1085e0 2024-04-25T14:11:42.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:42.003+0000 7f7de1aca700 1 --2- 172.21.15.96:0/1243536871 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f7dc80afc10 0x7f7dc80b2120 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7dd8007fd0 tx=0x7f7dd80073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:42.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:42.008+0000 7f7dd2ffd700 1 -- 172.21.15.96:0/1243536871 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7dd4010070 con 0x7f7ddc1085e0 2024-04-25T14:11:42.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:42.263+0000 7f7de3d2e700 1 -- 172.21.15.96:0/1243536871 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f7ddc02e6d0 con 0x7f7ddc1085e0 2024-04-25T14:11:42.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:42.269+0000 7f7dd2ffd700 1 -- 172.21.15.96:0/1243536871 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+284475 (secure 0 0 0) 0x7f7dd40a75d0 con 0x7f7ddc1085e0 2024-04-25T14:11:42.272 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:11:42.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:42.271+0000 7f7de3d2e700 1 -- 172.21.15.96:0/1243536871 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f7dc80afc10 msgr2=0x7f7dc80b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:42.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:42.271+0000 7f7de3d2e700 1 --2- 172.21.15.96:0/1243536871 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f7dc80afc10 0x7f7dc80b2120 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7dd8007fd0 tx=0x7f7dd80073d0 comp rx=0 tx=0).stop 2024-04-25T14:11:42.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:42.272+0000 7f7de3d2e700 1 -- 172.21.15.96:0/1243536871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ddc1085e0 msgr2=0x7f7ddc19c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:42.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:42.272+0000 7f7de3d2e700 1 --2- 172.21.15.96:0/1243536871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ddc1085e0 0x7f7ddc19c590 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4007ee0 tx=0x7f7dd400d3b0 comp rx=0 tx=0).stop 2024-04-25T14:11:42.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:42.272+0000 7f7de3d2e700 1 -- 172.21.15.96:0/1243536871 shutdown_connections 2024-04-25T14:11:42.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:42.272+0000 7f7de3d2e700 1 --2- 172.21.15.96:0/1243536871 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f7dc80afc10 0x7f7dc80b2120 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:42.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:42.272+0000 7f7de3d2e700 1 --2- 172.21.15.96:0/1243536871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ddc1085e0 0x7f7ddc19c590 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:42.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:42.272+0000 7f7de3d2e700 1 --2- 172.21.15.96:0/1243536871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ddc103360 0x7f7ddc19baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:42.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:42.272+0000 7f7de3d2e700 1 --2- 172.21.15.96:0/1243536871 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ddc107c90 0x7f7ddc19c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:42.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:42.272+0000 7f7de3d2e700 1 -- 172.21.15.96:0/1243536871 >> 172.21.15.96:0/1243536871 conn(0x7f7ddc0fec90 msgr2=0x7f7ddc109540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:42.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:42.272+0000 7f7de3d2e700 1 -- 172.21.15.96:0/1243536871 shutdown_connections 2024-04-25T14:11:42.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:42.272+0000 7f7de3d2e700 1 -- 172.21.15.96:0/1243536871 wait complete. 2024-04-25T14:11:42.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:42 smithi096 ceph-mon[29207]: pgmap v286: 97 pgs: 97 active+clean; 626 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 10 KiB/s wr, 29 op/s 2024-04-25T14:11:42.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:42 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/1243536871' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T14:11:42.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:42 smithi097 ceph-mon[32720]: pgmap v286: 97 pgs: 97 active+clean; 626 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 10 KiB/s wr, 29 op/s 2024-04-25T14:11:42.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:42 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/1243536871' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T14:11:42.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:42 smithi151 ceph-mon[32990]: pgmap v286: 97 pgs: 97 active+clean; 626 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 10 KiB/s wr, 29 op/s 2024-04-25T14:11:42.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:42 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/1243536871' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T14:11:43.043 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":536332425},{"type":"v1","addr":"172.21.15.96:6801","nonce":536332425}]},"active_addr":"172.21.15.96:6801/536332425","active_change":"2024-04-25T14:04:21.307702+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24119,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.96:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.96:0","nonce":2939571162}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.96:0","nonce":344482924}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.96:0","nonce":3955641945}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.96:0","nonce":588225129}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.96:0","nonce":4202714178}]}]} 2024-04-25T14:11:43.050 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-25T14:11:43.050 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-25T14:11:43.050 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T14:11:43.285 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:11:44.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:44 smithi096 ceph-mon[29207]: pgmap v287: 97 pgs: 97 active+clean; 626 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.3 KiB/s wr, 26 op/s 2024-04-25T14:11:44.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:44 smithi097 ceph-mon[32720]: pgmap v287: 97 pgs: 97 active+clean; 626 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.3 KiB/s wr, 26 op/s 2024-04-25T14:11:44.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:44 smithi151 ceph-mon[32990]: pgmap v287: 97 pgs: 97 active+clean; 626 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.3 KiB/s wr, 26 op/s 2024-04-25T14:11:45.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.159+0000 7f58a3cfe700 1 -- 172.21.15.96:0/162588859 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f589c1085e0 msgr2=0x7f589c10ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:45.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.159+0000 7f58a3cfe700 1 --2- 172.21.15.96:0/162588859 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f589c1085e0 0x7f589c10ff40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f589400b0d0 tx=0x7f589400b3e0 comp rx=0 tx=0).stop 2024-04-25T14:11:45.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.159+0000 7f58a3cfe700 1 -- 172.21.15.96:0/162588859 shutdown_connections 2024-04-25T14:11:45.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.159+0000 7f58a3cfe700 1 --2- 172.21.15.96:0/162588859 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f589c1085e0 0x7f589c10ff40 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:45.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.159+0000 7f58a3cfe700 1 --2- 172.21.15.96:0/162588859 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f589c103360 0x7f589c103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:45.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.159+0000 7f58a3cfe700 1 --2- 172.21.15.96:0/162588859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f589c107c90 0x7f589c1080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:45.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.159+0000 7f58a3cfe700 1 -- 172.21.15.96:0/162588859 >> 172.21.15.96:0/162588859 conn(0x7f589c0fec90 msgr2=0x7f589c1010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:45.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.159+0000 7f58a3cfe700 1 -- 172.21.15.96:0/162588859 shutdown_connections 2024-04-25T14:11:45.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.160+0000 7f58a3cfe700 1 -- 172.21.15.96:0/162588859 wait complete. 2024-04-25T14:11:45.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.160+0000 7f58a3cfe700 1 Processor -- start 2024-04-25T14:11:45.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.160+0000 7f58a3cfe700 1 -- start start 2024-04-25T14:11:45.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.161+0000 7f58a3cfe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f589c103360 0x7f589c100d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:45.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.161+0000 7f58a3cfe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f589c107c90 0x7f589c0ff170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:45.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.161+0000 7f58a3cfe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f589c1085e0 0x7f589c0ff750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:45.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.161+0000 7f58a3cfe700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f589c113210 con 0x7f589c107c90 2024-04-25T14:11:45.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.161+0000 7f58a3cfe700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f589c113090 con 0x7f589c103360 2024-04-25T14:11:45.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.161+0000 7f58a3cfe700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f589c113390 con 0x7f589c1085e0 2024-04-25T14:11:45.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.161+0000 7f58a1299700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f589c107c90 0x7f589c0ff170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:45.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.161+0000 7f58a1299700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f589c107c90 0x7f589c0ff170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57970/0 (socket says 172.21.15.96:57970) 2024-04-25T14:11:45.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.161+0000 7f58a1299700 1 -- 172.21.15.96:0/836595060 learned_addr learned my addr 172.21.15.96:0/836595060 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:11:45.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.161+0000 7f58a1a9a700 1 --2- 172.21.15.96:0/836595060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f589c103360 0x7f589c100d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:45.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.161+0000 7f58a229b700 1 --2- 172.21.15.96:0/836595060 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f589c1085e0 0x7f589c0ff750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:45.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.161+0000 7f58a1299700 1 -- 172.21.15.96:0/836595060 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f589c1085e0 msgr2=0x7f589c0ff750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:45.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.161+0000 7f58a1299700 1 --2- 172.21.15.96:0/836595060 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f589c1085e0 0x7f589c0ff750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:45.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.161+0000 7f58a1299700 1 -- 172.21.15.96:0/836595060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f589c103360 msgr2=0x7f589c100d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:45.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.161+0000 7f58a1299700 1 --2- 172.21.15.96:0/836595060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f589c103360 0x7f589c100d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:45.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.161+0000 7f58a1299700 1 -- 172.21.15.96:0/836595060 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f589c0ffef0 con 0x7f589c107c90 2024-04-25T14:11:45.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.161+0000 7f58a229b700 1 --2- 172.21.15.96:0/836595060 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f589c1085e0 0x7f589c0ff750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:45.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.161+0000 7f58a1299700 1 --2- 172.21.15.96:0/836595060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f589c107c90 0x7f589c0ff170 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f588c00c9e0 tx=0x7f588c00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:45.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.161+0000 7f5892ffd700 1 -- 172.21.15.96:0/836595060 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c007aa0 con 0x7f589c107c90 2024-04-25T14:11:45.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.161+0000 7f58a3cfe700 1 -- 172.21.15.96:0/836595060 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f589c1ad5e0 con 0x7f589c107c90 2024-04-25T14:11:45.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.161+0000 7f5892ffd700 1 -- 172.21.15.96:0/836595060 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f588c007c40 con 0x7f589c107c90 2024-04-25T14:11:45.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.161+0000 7f5892ffd700 1 -- 172.21.15.96:0/836595060 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c018660 con 0x7f589c107c90 2024-04-25T14:11:45.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.162+0000 7f58a3cfe700 1 -- 172.21.15.96:0/836595060 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f589c1adb70 con 0x7f589c107c90 2024-04-25T14:11:45.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.163+0000 7f58a3cfe700 1 -- 172.21.15.96:0/836595060 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f589c104810 con 0x7f589c107c90 2024-04-25T14:11:45.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.164+0000 7f5892ffd700 1 -- 172.21.15.96:0/836595060 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f588c018800 con 0x7f589c107c90 2024-04-25T14:11:45.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.165+0000 7f5892ffd700 1 --2- 172.21.15.96:0/836595060 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f58880afc10 0x7f58880b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:45.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.165+0000 7f58a1a9a700 1 --2- 172.21.15.96:0/836595060 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f58880afc10 0x7f58880b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:45.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.165+0000 7f5892ffd700 1 -- 172.21.15.96:0/836595060 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7f588c0dede0 con 0x7f589c107c90 2024-04-25T14:11:45.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.165+0000 7f58a1a9a700 1 --2- 172.21.15.96:0/836595060 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f58880afc10 0x7f58880b2120 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f589800ac50 tx=0x7f589800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:45.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.168+0000 7f5892ffd700 1 -- 172.21.15.96:0/836595060 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f588c014030 con 0x7f589c107c90 2024-04-25T14:11:45.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.366+0000 7f58a3cfe700 1 -- 172.21.15.96:0/836595060 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f589c02e510 con 0x7f589c107c90 2024-04-25T14:11:45.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.368+0000 7f5892ffd700 1 -- 172.21.15.96:0/836595060 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+22146 (secure 0 0 0) 0x7f588c0a7aa0 con 0x7f589c107c90 2024-04-25T14:11:45.368 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:11:45.368 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":76,"fsid":"59334182-030c-11ef-bc93-c7b262605968","created":"2024-04-25T14:03:25.680745+0000","modified":"2024-04-25T14:11:30.552550+0000","last_up_change":"2024-04-25T14:10:35.894312+0000","last_in_change":"2024-04-25T14:10:15.583440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:06:47.402927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:11:27.434331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:11:27.854004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cf4ba3ac-142e-4e8e-96d4-410d77157eb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6803","nonce":3832506168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6805","nonce":3832506168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832506168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6807","nonce":3832506168}]},"public_addr":"172.21.15.96:6803/3832506168","cluster_addr":"172.21.15.96:6805/3832506168","heartbeat_back_addr":"172.21.15.96:6809/3832506168","heartbeat_front_addr":"172.21.15.96:6807/3832506168","state":["exists","up"]},{"osd":1,"uuid":"866e73b9-4a26-48ba-badf-704fe22f71cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6801","nonce":1083311494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6803","nonce":1083311494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6807","nonce":1083311494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6805","nonce":1083311494}]},"public_addr":"172.21.15.97:6801/1083311494","cluster_addr":"172.21.15.97:6803/1083311494","heartbeat_back_addr":"172.21.15.97:6807/1083311494","heartbeat_front_addr":"172.21.15.97:6805/1083311494","state":["exists","up"]},{"osd":2,"uuid":"45c79ecb-abc4-4dba-bae8-74804e4bc1bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6801","nonce":981902966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6803","nonce":981902966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6807","nonce":981902966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6805","nonce":981902966}]},"public_addr":"172.21.15.151:6801/981902966","cluster_addr":"172.21.15.151:6803/981902966","heartbeat_back_addr":"172.21.15.151:6807/981902966","heartbeat_front_addr":"172.21.15.151:6805/981902966","state":["exists","up"]},{"osd":3,"uuid":"9fc39468-a7ea-4ccf-b365-335ad2ea4c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6811","nonce":2274158748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6813","nonce":2274158748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6817","nonce":2274158748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6815","nonce":2274158748}]},"public_addr":"172.21.15.96:6811/2274158748","cluster_addr":"172.21.15.96:6813/2274158748","heartbeat_back_addr":"172.21.15.96:6817/2274158748","heartbeat_front_addr":"172.21.15.96:6815/2274158748","state":["exists","up"]},{"osd":4,"uuid":"616582f1-12d2-456b-b7b3-e5517800e8a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6809","nonce":3727031962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6811","nonce":3727031962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6815","nonce":3727031962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6813","nonce":3727031962}]},"public_addr":"172.21.15.97:6809/3727031962","cluster_addr":"172.21.15.97:6811/3727031962","heartbeat_back_addr":"172.21.15.97:6815/3727031962","heartbeat_front_addr":"172.21.15.97:6813/3727031962","state":["exists","up"]},{"osd":5,"uuid":"3d277784-ab0f-4c90-81a1-2a4034c52a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6809","nonce":223882546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6811","nonce":223882546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6815","nonce":223882546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6813","nonce":223882546}]},"public_addr":"172.21.15.151:6809/223882546","cluster_addr":"172.21.15.151:6811/223882546","heartbeat_back_addr":"172.21.15.151:6815/223882546","heartbeat_front_addr":"172.21.15.151:6813/223882546","state":["exists","up"]},{"osd":6,"uuid":"9a9ad585-cb1a-4239-9d18-055f59164049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6819","nonce":96197407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6821","nonce":96197407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6825","nonce":96197407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6823","nonce":96197407}]},"public_addr":"172.21.15.96:6819/96197407","cluster_addr":"172.21.15.96:6821/96197407","heartbeat_back_addr":"172.21.15.96:6825/96197407","heartbeat_front_addr":"172.21.15.96:6823/96197407","state":["exists","up"]},{"osd":7,"uuid":"303551c8-6af0-42f2-844f-45bf8db5baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6817","nonce":2247216400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6819","nonce":2247216400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6823","nonce":2247216400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6821","nonce":2247216400}]},"public_addr":"172.21.15.97:6817/2247216400","cluster_addr":"172.21.15.97:6819/2247216400","heartbeat_back_addr":"172.21.15.97:6823/2247216400","heartbeat_front_addr":"172.21.15.97:6821/2247216400","state":["exists","up"]},{"osd":8,"uuid":"81f6cc66-e4ca-430c-b336-b6678d968818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6817","nonce":2549822185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6819","nonce":2549822185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6823","nonce":2549822185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6821","nonce":2549822185}]},"public_addr":"172.21.15.151:6817/2549822185","cluster_addr":"172.21.15.151:6819/2549822185","heartbeat_back_addr":"172.21.15.151:6823/2549822185","heartbeat_front_addr":"172.21.15.151:6821/2549822185","state":["exists","up"]},{"osd":9,"uuid":"29d4da16-feab-4d1c-9d1d-de4f799bd724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6827","nonce":1099399643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6829","nonce":1099399643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6833","nonce":1099399643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6831","nonce":1099399643}]},"public_addr":"172.21.15.96:6827/1099399643","cluster_addr":"172.21.15.96:6829/1099399643","heartbeat_back_addr":"172.21.15.96:6833/1099399643","heartbeat_front_addr":"172.21.15.96:6831/1099399643","state":["exists","up"]},{"osd":10,"uuid":"197fee9b-eee4-4893-be1c-3d57f42250e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6825","nonce":155754520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6827","nonce":155754520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6831","nonce":155754520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6829","nonce":155754520}]},"public_addr":"172.21.15.97:6825/155754520","cluster_addr":"172.21.15.97:6827/155754520","heartbeat_back_addr":"172.21.15.97:6831/155754520","heartbeat_front_addr":"172.21.15.97:6829/155754520","state":["exists","up"]},{"osd":11,"uuid":"9c1d0a83-fc12-41a2-8eb0-80c4abdd2770","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6825","nonce":2027958746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6827","nonce":2027958746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6831","nonce":2027958746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6829","nonce":2027958746}]},"public_addr":"172.21.15.151:6825/2027958746","cluster_addr":"172.21.15.151:6827/2027958746","heartbeat_back_addr":"172.21.15.151:6831/2027958746","heartbeat_front_addr":"172.21.15.151:6829/2027958746","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:05:56.177390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:06:19.917663+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:06:43.608052+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:08.540382+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:33.474072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:58.242306+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:08:24.120442+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:08:49.661064+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:09:15.097271+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:09:41.904505+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:10:08.233252+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:10:33.712296+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/3839807484":"2024-04-26T14:04:21.307574+0000","172.21.15.96:0/1495237551":"2024-04-26T14:04:21.307574+0000","172.21.15.96:0/2793751450":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/129575834":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/2233075139":"2024-04-26T14:04:21.307574+0000","172.21.15.96:6801/2348397481":"2024-04-26T14:03:54.249537+0000","172.21.15.96:6801/433591062":"2024-04-26T14:04:21.307574+0000","172.21.15.96:6800/2348397481":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/3517674924":"2024-04-26T14:03:54.249537+0000","172.21.15.96:6800/433591062":"2024-04-26T14:04:21.307574+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:11:45.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.370+0000 7f58a3cfe700 1 -- 172.21.15.96:0/836595060 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f58880afc10 msgr2=0x7f58880b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:45.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.370+0000 7f58a3cfe700 1 --2- 172.21.15.96:0/836595060 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f58880afc10 0x7f58880b2120 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f589800ac50 tx=0x7f589800a380 comp rx=0 tx=0).stop 2024-04-25T14:11:45.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.370+0000 7f58a3cfe700 1 -- 172.21.15.96:0/836595060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f589c107c90 msgr2=0x7f589c0ff170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:45.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.370+0000 7f58a3cfe700 1 --2- 172.21.15.96:0/836595060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f589c107c90 0x7f589c0ff170 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f588c00c9e0 tx=0x7f588c00ccf0 comp rx=0 tx=0).stop 2024-04-25T14:11:45.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.370+0000 7f58a3cfe700 1 -- 172.21.15.96:0/836595060 shutdown_connections 2024-04-25T14:11:45.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.370+0000 7f58a3cfe700 1 --2- 172.21.15.96:0/836595060 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f58880afc10 0x7f58880b2120 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:45.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.370+0000 7f58a3cfe700 1 --2- 172.21.15.96:0/836595060 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f589c1085e0 0x7f589c0ff750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:45.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.370+0000 7f58a3cfe700 1 --2- 172.21.15.96:0/836595060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f589c103360 0x7f589c100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:45.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.370+0000 7f58a3cfe700 1 --2- 172.21.15.96:0/836595060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f589c107c90 0x7f589c0ff170 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:45.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.370+0000 7f58a3cfe700 1 -- 172.21.15.96:0/836595060 >> 172.21.15.96:0/836595060 conn(0x7f589c0fec90 msgr2=0x7f589c06b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:45.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.370+0000 7f58a3cfe700 1 -- 172.21.15.96:0/836595060 shutdown_connections 2024-04-25T14:11:45.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:45.370+0000 7f58a3cfe700 1 -- 172.21.15.96:0/836595060 wait complete. 2024-04-25T14:11:45.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:45 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/836595060' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:11:45.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:45 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/836595060' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:11:45.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:45 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/836595060' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:11:46.081 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-25T14:11:46.081 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T14:11:46.327 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:11:46.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:46 smithi096 ceph-mon[29207]: pgmap v288: 97 pgs: 97 active+clean; 626 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.6 KiB/s wr, 23 op/s 2024-04-25T14:11:46.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:46 smithi097 ceph-mon[32720]: pgmap v288: 97 pgs: 97 active+clean; 626 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.6 KiB/s wr, 23 op/s 2024-04-25T14:11:46.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:46 smithi151 ceph-mon[32990]: pgmap v288: 97 pgs: 97 active+clean; 626 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.6 KiB/s wr, 23 op/s 2024-04-25T14:11:48.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.201+0000 7f826f5fa700 1 -- 172.21.15.96:0/1184905158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8268102890 msgr2=0x7f8268102ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:48.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.201+0000 7f826f5fa700 1 --2- 172.21.15.96:0/1184905158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8268102890 0x7f8268102ca0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f825800aaa0 tx=0x7f825800adb0 comp rx=0 tx=0).stop 2024-04-25T14:11:48.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.201+0000 7f826f5fa700 1 -- 172.21.15.96:0/1184905158 shutdown_connections 2024-04-25T14:11:48.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.201+0000 7f826f5fa700 1 --2- 172.21.15.96:0/1184905158 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82681031e0 0x7f826810f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:48.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.201+0000 7f826f5fa700 1 --2- 172.21.15.96:0/1184905158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82681088c0 0x7f8268108cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:48.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.201+0000 7f826f5fa700 1 --2- 172.21.15.96:0/1184905158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8268102890 0x7f8268102ca0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:48.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.201+0000 7f826f5fa700 1 -- 172.21.15.96:0/1184905158 >> 172.21.15.96:0/1184905158 conn(0x7f82680fe5c0 msgr2=0x7f82681009e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:48.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.202+0000 7f826f5fa700 1 -- 172.21.15.96:0/1184905158 shutdown_connections 2024-04-25T14:11:48.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.202+0000 7f826f5fa700 1 -- 172.21.15.96:0/1184905158 wait complete. 2024-04-25T14:11:48.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.202+0000 7f826f5fa700 1 Processor -- start 2024-04-25T14:11:48.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.202+0000 7f826f5fa700 1 -- start start 2024-04-25T14:11:48.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.203+0000 7f826f5fa700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8268102890 0x7f82681989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:48.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.203+0000 7f826f5fa700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82681031e0 0x7f8268198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:48.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.203+0000 7f826f5fa700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82681088c0 0x7f8268199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:48.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.203+0000 7f826f5fa700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f82681121b0 con 0x7f8268102890 2024-04-25T14:11:48.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.203+0000 7f826f5fa700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8268112030 con 0x7f82681031e0 2024-04-25T14:11:48.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.203+0000 7f826f5fa700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8268112330 con 0x7f82681088c0 2024-04-25T14:11:48.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.203+0000 7f826d396700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8268102890 0x7f82681989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:48.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.203+0000 7f826cb95700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82681031e0 0x7f8268198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:48.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.203+0000 7f826cb95700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82681031e0 0x7f8268198f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:42990/0 (socket says 172.21.15.96:42990) 2024-04-25T14:11:48.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.203+0000 7f826cb95700 1 -- 172.21.15.96:0/364935028 learned_addr learned my addr 172.21.15.96:0/364935028 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:11:48.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.203+0000 7f826d396700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8268102890 0x7f82681989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57982/0 (socket says 172.21.15.96:57982) 2024-04-25T14:11:48.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.203+0000 7f826db97700 1 --2- 172.21.15.96:0/364935028 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82681088c0 0x7f8268199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:48.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.204+0000 7f826d396700 1 -- 172.21.15.96:0/364935028 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82681088c0 msgr2=0x7f8268199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:48.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.204+0000 7f826d396700 1 --2- 172.21.15.96:0/364935028 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82681088c0 0x7f8268199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:48.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.204+0000 7f826d396700 1 -- 172.21.15.96:0/364935028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82681031e0 msgr2=0x7f8268198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:48.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.204+0000 7f826d396700 1 --2- 172.21.15.96:0/364935028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82681031e0 0x7f8268198f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:48.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.204+0000 7f826d396700 1 -- 172.21.15.96:0/364935028 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82681a9150 con 0x7f8268102890 2024-04-25T14:11:48.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.204+0000 7f826cb95700 1 --2- 172.21.15.96:0/364935028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82681031e0 0x7f8268198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:11:48.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.204+0000 7f826d396700 1 --2- 172.21.15.96:0/364935028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8268102890 0x7f82681989d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f826400e9c0 tx=0x7f826400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:48.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.204+0000 7f825e7fc700 1 -- 172.21.15.96:0/364935028 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826400c9c0 con 0x7f8268102890 2024-04-25T14:11:48.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.204+0000 7f825e7fc700 1 -- 172.21.15.96:0/364935028 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f826400cb60 con 0x7f8268102890 2024-04-25T14:11:48.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.204+0000 7f826f5fa700 1 -- 172.21.15.96:0/364935028 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82681a9440 con 0x7f8268102890 2024-04-25T14:11:48.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.204+0000 7f825e7fc700 1 -- 172.21.15.96:0/364935028 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8264018680 con 0x7f8268102890 2024-04-25T14:11:48.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.204+0000 7f826f5fa700 1 -- 172.21.15.96:0/364935028 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82681a99d0 con 0x7f8268102890 2024-04-25T14:11:48.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.205+0000 7f826f5fa700 1 -- 172.21.15.96:0/364935028 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8268051190 con 0x7f8268102890 2024-04-25T14:11:48.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.207+0000 7f825e7fc700 1 -- 172.21.15.96:0/364935028 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f8264010430 con 0x7f8268102890 2024-04-25T14:11:48.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.208+0000 7f825e7fc700 1 --2- 172.21.15.96:0/364935028 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f82540afc90 0x7f82540b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:48.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.208+0000 7f826cb95700 1 --2- 172.21.15.96:0/364935028 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f82540afc90 0x7f82540b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:48.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.208+0000 7f825e7fc700 1 -- 172.21.15.96:0/364935028 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7f82640dec90 con 0x7f8268102890 2024-04-25T14:11:48.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.208+0000 7f826cb95700 1 --2- 172.21.15.96:0/364935028 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f82540afc90 0x7f82540b21a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8268105f70 tx=0x7f825800c650 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:48.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.213+0000 7f825e7fc700 1 -- 172.21.15.96:0/364935028 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f82640a78c0 con 0x7f8268102890 2024-04-25T14:11:48.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.418+0000 7f826f5fa700 1 -- 172.21.15.96:0/364935028 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f826802e510 con 0x7f8268102890 2024-04-25T14:11:48.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.420+0000 7f825e7fc700 1 -- 172.21.15.96:0/364935028 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+22146 (secure 0 0 0) 0x7f82640a71c0 con 0x7f8268102890 2024-04-25T14:11:48.420 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:11:48.420 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":76,"fsid":"59334182-030c-11ef-bc93-c7b262605968","created":"2024-04-25T14:03:25.680745+0000","modified":"2024-04-25T14:11:30.552550+0000","last_up_change":"2024-04-25T14:10:35.894312+0000","last_in_change":"2024-04-25T14:10:15.583440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:06:47.402927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:11:27.434331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:11:27.854004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cf4ba3ac-142e-4e8e-96d4-410d77157eb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6803","nonce":3832506168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6805","nonce":3832506168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832506168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6807","nonce":3832506168}]},"public_addr":"172.21.15.96:6803/3832506168","cluster_addr":"172.21.15.96:6805/3832506168","heartbeat_back_addr":"172.21.15.96:6809/3832506168","heartbeat_front_addr":"172.21.15.96:6807/3832506168","state":["exists","up"]},{"osd":1,"uuid":"866e73b9-4a26-48ba-badf-704fe22f71cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6801","nonce":1083311494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6803","nonce":1083311494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6807","nonce":1083311494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6805","nonce":1083311494}]},"public_addr":"172.21.15.97:6801/1083311494","cluster_addr":"172.21.15.97:6803/1083311494","heartbeat_back_addr":"172.21.15.97:6807/1083311494","heartbeat_front_addr":"172.21.15.97:6805/1083311494","state":["exists","up"]},{"osd":2,"uuid":"45c79ecb-abc4-4dba-bae8-74804e4bc1bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6801","nonce":981902966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6803","nonce":981902966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6807","nonce":981902966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6805","nonce":981902966}]},"public_addr":"172.21.15.151:6801/981902966","cluster_addr":"172.21.15.151:6803/981902966","heartbeat_back_addr":"172.21.15.151:6807/981902966","heartbeat_front_addr":"172.21.15.151:6805/981902966","state":["exists","up"]},{"osd":3,"uuid":"9fc39468-a7ea-4ccf-b365-335ad2ea4c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6811","nonce":2274158748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6813","nonce":2274158748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6817","nonce":2274158748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6815","nonce":2274158748}]},"public_addr":"172.21.15.96:6811/2274158748","cluster_addr":"172.21.15.96:6813/2274158748","heartbeat_back_addr":"172.21.15.96:6817/2274158748","heartbeat_front_addr":"172.21.15.96:6815/2274158748","state":["exists","up"]},{"osd":4,"uuid":"616582f1-12d2-456b-b7b3-e5517800e8a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6809","nonce":3727031962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6811","nonce":3727031962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6815","nonce":3727031962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6813","nonce":3727031962}]},"public_addr":"172.21.15.97:6809/3727031962","cluster_addr":"172.21.15.97:6811/3727031962","heartbeat_back_addr":"172.21.15.97:6815/3727031962","heartbeat_front_addr":"172.21.15.97:6813/3727031962","state":["exists","up"]},{"osd":5,"uuid":"3d277784-ab0f-4c90-81a1-2a4034c52a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6809","nonce":223882546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6811","nonce":223882546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6815","nonce":223882546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6813","nonce":223882546}]},"public_addr":"172.21.15.151:6809/223882546","cluster_addr":"172.21.15.151:6811/223882546","heartbeat_back_addr":"172.21.15.151:6815/223882546","heartbeat_front_addr":"172.21.15.151:6813/223882546","state":["exists","up"]},{"osd":6,"uuid":"9a9ad585-cb1a-4239-9d18-055f59164049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6819","nonce":96197407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6821","nonce":96197407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6825","nonce":96197407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6823","nonce":96197407}]},"public_addr":"172.21.15.96:6819/96197407","cluster_addr":"172.21.15.96:6821/96197407","heartbeat_back_addr":"172.21.15.96:6825/96197407","heartbeat_front_addr":"172.21.15.96:6823/96197407","state":["exists","up"]},{"osd":7,"uuid":"303551c8-6af0-42f2-844f-45bf8db5baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6817","nonce":2247216400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6819","nonce":2247216400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6823","nonce":2247216400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6821","nonce":2247216400}]},"public_addr":"172.21.15.97:6817/2247216400","cluster_addr":"172.21.15.97:6819/2247216400","heartbeat_back_addr":"172.21.15.97:6823/2247216400","heartbeat_front_addr":"172.21.15.97:6821/2247216400","state":["exists","up"]},{"osd":8,"uuid":"81f6cc66-e4ca-430c-b336-b6678d968818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6817","nonce":2549822185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6819","nonce":2549822185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6823","nonce":2549822185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6821","nonce":2549822185}]},"public_addr":"172.21.15.151:6817/2549822185","cluster_addr":"172.21.15.151:6819/2549822185","heartbeat_back_addr":"172.21.15.151:6823/2549822185","heartbeat_front_addr":"172.21.15.151:6821/2549822185","state":["exists","up"]},{"osd":9,"uuid":"29d4da16-feab-4d1c-9d1d-de4f799bd724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6827","nonce":1099399643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6829","nonce":1099399643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6833","nonce":1099399643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6831","nonce":1099399643}]},"public_addr":"172.21.15.96:6827/1099399643","cluster_addr":"172.21.15.96:6829/1099399643","heartbeat_back_addr":"172.21.15.96:6833/1099399643","heartbeat_front_addr":"172.21.15.96:6831/1099399643","state":["exists","up"]},{"osd":10,"uuid":"197fee9b-eee4-4893-be1c-3d57f42250e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6825","nonce":155754520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6827","nonce":155754520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6831","nonce":155754520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6829","nonce":155754520}]},"public_addr":"172.21.15.97:6825/155754520","cluster_addr":"172.21.15.97:6827/155754520","heartbeat_back_addr":"172.21.15.97:6831/155754520","heartbeat_front_addr":"172.21.15.97:6829/155754520","state":["exists","up"]},{"osd":11,"uuid":"9c1d0a83-fc12-41a2-8eb0-80c4abdd2770","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6825","nonce":2027958746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6827","nonce":2027958746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6831","nonce":2027958746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6829","nonce":2027958746}]},"public_addr":"172.21.15.151:6825/2027958746","cluster_addr":"172.21.15.151:6827/2027958746","heartbeat_back_addr":"172.21.15.151:6831/2027958746","heartbeat_front_addr":"172.21.15.151:6829/2027958746","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:05:56.177390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:06:19.917663+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:06:43.608052+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:08.540382+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:33.474072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:58.242306+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:08:24.120442+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:08:49.661064+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:09:15.097271+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:09:41.904505+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:10:08.233252+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:10:33.712296+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/3839807484":"2024-04-26T14:04:21.307574+0000","172.21.15.96:0/1495237551":"2024-04-26T14:04:21.307574+0000","172.21.15.96:0/2793751450":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/129575834":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/2233075139":"2024-04-26T14:04:21.307574+0000","172.21.15.96:6801/2348397481":"2024-04-26T14:03:54.249537+0000","172.21.15.96:6801/433591062":"2024-04-26T14:04:21.307574+0000","172.21.15.96:6800/2348397481":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/3517674924":"2024-04-26T14:03:54.249537+0000","172.21.15.96:6800/433591062":"2024-04-26T14:04:21.307574+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:11:48.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.422+0000 7f826f5fa700 1 -- 172.21.15.96:0/364935028 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f82540afc90 msgr2=0x7f82540b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:48.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.422+0000 7f826f5fa700 1 --2- 172.21.15.96:0/364935028 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f82540afc90 0x7f82540b21a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8268105f70 tx=0x7f825800c650 comp rx=0 tx=0).stop 2024-04-25T14:11:48.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.422+0000 7f826f5fa700 1 -- 172.21.15.96:0/364935028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8268102890 msgr2=0x7f82681989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:48.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.422+0000 7f826f5fa700 1 --2- 172.21.15.96:0/364935028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8268102890 0x7f82681989d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f826400e9c0 tx=0x7f826400ecd0 comp rx=0 tx=0).stop 2024-04-25T14:11:48.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.422+0000 7f826f5fa700 1 -- 172.21.15.96:0/364935028 shutdown_connections 2024-04-25T14:11:48.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.422+0000 7f826f5fa700 1 --2- 172.21.15.96:0/364935028 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f82540afc90 0x7f82540b21a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:48.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.422+0000 7f826f5fa700 1 --2- 172.21.15.96:0/364935028 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82681088c0 0x7f8268199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:48.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.422+0000 7f826f5fa700 1 --2- 172.21.15.96:0/364935028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8268102890 0x7f82681989d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:48.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.422+0000 7f826f5fa700 1 --2- 172.21.15.96:0/364935028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82681031e0 0x7f8268198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:48.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.422+0000 7f826f5fa700 1 -- 172.21.15.96:0/364935028 >> 172.21.15.96:0/364935028 conn(0x7f82680fe5c0 msgr2=0x7f8268107b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:48.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.422+0000 7f826f5fa700 1 -- 172.21.15.96:0/364935028 shutdown_connections 2024-04-25T14:11:48.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:48.422+0000 7f826f5fa700 1 -- 172.21.15.96:0/364935028 wait complete. 2024-04-25T14:11:48.770 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:48 smithi096 ceph-mon[29207]: pgmap v289: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 25 op/s 2024-04-25T14:11:48.770 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:48 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/364935028' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:11:48.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:48 smithi097 ceph-mon[32720]: pgmap v289: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 25 op/s 2024-04-25T14:11:48.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:48 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/364935028' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:11:48.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:48 smithi151 ceph-mon[32990]: pgmap v289: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 25 op/s 2024-04-25T14:11:48.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:48 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/364935028' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:11:49.226 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-25T14:11:49.227 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-25T14:11:49.227 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-25T14:11:49.228 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-25T14:11:49.228 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-25T14:11:49.229 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-25T14:11:49.229 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-25T14:11:49.229 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-25T14:11:49.230 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-25T14:11:49.230 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-25T14:11:49.231 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-25T14:11:49.231 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-25T14:11:49.908 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:11:49.910 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:11:49.921 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:11:49.921 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:11:50.265 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:11:50.266 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:11:50.267 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:11:50.268 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:11:50.269 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:11:50.746 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:50 smithi096 ceph-mon[29207]: pgmap v290: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-04-25T14:11:50.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:50 smithi097 ceph-mon[32720]: pgmap v290: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-04-25T14:11:50.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:50 smithi151 ceph-mon[32990]: pgmap v290: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-04-25T14:11:52.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:52 smithi096 ceph-mon[29207]: pgmap v291: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-04-25T14:11:52.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:52 smithi097 ceph-mon[32720]: pgmap v291: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-04-25T14:11:52.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:52 smithi151 ceph-mon[32990]: pgmap v291: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-04-25T14:11:53.203 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:11:54.584 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:11:54.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:54 smithi096 ceph-mon[29207]: pgmap v292: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-04-25T14:11:54.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:54 smithi097 ceph-mon[32720]: pgmap v292: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-04-25T14:11:54.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:54 smithi151 ceph-mon[32990]: pgmap v292: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-04-25T14:11:56.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:56 smithi096 ceph-mon[29207]: pgmap v293: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-04-25T14:11:56.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:56 smithi097 ceph-mon[32720]: pgmap v293: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-04-25T14:11:56.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:56 smithi151 ceph-mon[32990]: pgmap v293: 97 pgs: 97 active+clean; 638 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-04-25T14:11:57.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.398+0000 7f143b2e1700 1 -- 172.21.15.96:0/1763219486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1434108df0 msgr2=0x7f1434109200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:57.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.398+0000 7f143b2e1700 1 --2- 172.21.15.96:0/1763219486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1434108df0 0x7f1434109200 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f14240099c0 tx=0x7f1424009cd0 comp rx=0 tx=0).stop 2024-04-25T14:11:57.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.398+0000 7f143b2e1700 1 -- 172.21.15.96:0/1763219486 shutdown_connections 2024-04-25T14:11:57.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.398+0000 7f143b2e1700 1 --2- 172.21.15.96:0/1763219486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1434109740 0x7f14341110a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:57.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.398+0000 7f143b2e1700 1 --2- 172.21.15.96:0/1763219486 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14341044c0 0x7f14341048d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:57.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.398+0000 7f143b2e1700 1 --2- 172.21.15.96:0/1763219486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1434108df0 0x7f1434109200 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:57.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.398+0000 7f143b2e1700 1 -- 172.21.15.96:0/1763219486 >> 172.21.15.96:0/1763219486 conn(0x7f14340fff10 msgr2=0x7f1434102330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:57.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.398+0000 7f143b2e1700 1 -- 172.21.15.96:0/1763219486 shutdown_connections 2024-04-25T14:11:57.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.398+0000 7f143b2e1700 1 -- 172.21.15.96:0/1763219486 wait complete. 2024-04-25T14:11:57.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.399+0000 7f143b2e1700 1 Processor -- start 2024-04-25T14:11:57.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.399+0000 7f143b2e1700 1 -- start start 2024-04-25T14:11:57.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.399+0000 7f143b2e1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14341044c0 0x7f143419ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:57.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.399+0000 7f143b2e1700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1434108df0 0x7f143419b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:57.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.399+0000 7f143b2e1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1434109740 0x7f143419f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:57.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.399+0000 7f143b2e1700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1434114370 con 0x7f14341044c0 2024-04-25T14:11:57.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.399+0000 7f143b2e1700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f14341141f0 con 0x7f1434109740 2024-04-25T14:11:57.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.399+0000 7f143b2e1700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f14341144f0 con 0x7f1434108df0 2024-04-25T14:11:57.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.399+0000 7f143907d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14341044c0 0x7f143419ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:57.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.399+0000 7f143907d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14341044c0 0x7f143419ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42068/0 (socket says 172.21.15.96:42068) 2024-04-25T14:11:57.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.399+0000 7f143907d700 1 -- 172.21.15.96:0/2809725164 learned_addr learned my addr 172.21.15.96:0/2809725164 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:11:57.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.399+0000 7f143887c700 1 --2- 172.21.15.96:0/2809725164 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1434108df0 0x7f143419b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:57.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.400+0000 7f143987e700 1 --2- 172.21.15.96:0/2809725164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1434109740 0x7f143419f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:57.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.400+0000 7f143907d700 1 -- 172.21.15.96:0/2809725164 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1434108df0 msgr2=0x7f143419b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:57.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.400+0000 7f143907d700 1 --2- 172.21.15.96:0/2809725164 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1434108df0 0x7f143419b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:57.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.400+0000 7f143907d700 1 -- 172.21.15.96:0/2809725164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1434109740 msgr2=0x7f143419f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:57.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.400+0000 7f143907d700 1 --2- 172.21.15.96:0/2809725164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1434109740 0x7f143419f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:57.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.400+0000 7f143907d700 1 -- 172.21.15.96:0/2809725164 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f143419fc10 con 0x7f14341044c0 2024-04-25T14:11:57.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.400+0000 7f143987e700 1 --2- 172.21.15.96:0/2809725164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1434109740 0x7f143419f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:57.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.400+0000 7f143907d700 1 --2- 172.21.15.96:0/2809725164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14341044c0 0x7f143419ad40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f143000e9c0 tx=0x7f143000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:57.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.400+0000 7f142a7fc700 1 -- 172.21.15.96:0/2809725164 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143000cbd0 con 0x7f14341044c0 2024-04-25T14:11:57.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.400+0000 7f143b2e1700 1 -- 172.21.15.96:0/2809725164 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f143419ff00 con 0x7f14341044c0 2024-04-25T14:11:57.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.400+0000 7f142a7fc700 1 -- 172.21.15.96:0/2809725164 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f143000cd70 con 0x7f14341044c0 2024-04-25T14:11:57.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.400+0000 7f142a7fc700 1 -- 172.21.15.96:0/2809725164 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1430005610 con 0x7f14341044c0 2024-04-25T14:11:57.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.400+0000 7f143b2e1700 1 -- 172.21.15.96:0/2809725164 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14341a7820 con 0x7f14341044c0 2024-04-25T14:11:57.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.401+0000 7f143b2e1700 1 -- 172.21.15.96:0/2809725164 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1434063d60 con 0x7f14341044c0 2024-04-25T14:11:57.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.402+0000 7f142a7fc700 1 -- 172.21.15.96:0/2809725164 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f1430004750 con 0x7f14341044c0 2024-04-25T14:11:57.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.403+0000 7f142a7fc700 1 --2- 172.21.15.96:0/2809725164 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f14200afc10 0x7f14200b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:57.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.403+0000 7f143887c700 1 --2- 172.21.15.96:0/2809725164 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f14200afc10 0x7f14200b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:57.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.403+0000 7f142a7fc700 1 -- 172.21.15.96:0/2809725164 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7f14300df350 con 0x7f14341044c0 2024-04-25T14:11:57.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.403+0000 7f142a7fc700 1 --2- 172.21.15.96:0/2809725164 >> [v2:172.21.15.151:6800/981902966,v1:172.21.15.151:6801/981902966] conn(0x7f14200b94a0 0x7f14200bb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:57.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.403+0000 7f142a7fc700 1 -- 172.21.15.96:0/2809725164 --> [v2:172.21.15.151:6800/981902966,v1:172.21.15.151:6801/981902966] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1430025ea0 con 0x7f14200b94a0 2024-04-25T14:11:57.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.403+0000 7f142a7fc700 1 -- 172.21.15.96:0/2809725164 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f14300df830 con 0x7f14341044c0 2024-04-25T14:11:57.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.403+0000 7f143887c700 1 --2- 172.21.15.96:0/2809725164 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f14200afc10 0x7f14200b2120 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1424000c00 tx=0x7f142401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:57.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.404+0000 7f143987e700 1 --2- 172.21.15.96:0/2809725164 >> [v2:172.21.15.151:6800/981902966,v1:172.21.15.151:6801/981902966] conn(0x7f14200b94a0 0x7f14200bb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:57.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.404+0000 7f143987e700 1 --2- 172.21.15.96:0/2809725164 >> [v2:172.21.15.151:6800/981902966,v1:172.21.15.151:6801/981902966] conn(0x7f14200b94a0 0x7f14200bb8f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:57.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.405+0000 7f142a7fc700 1 -- 172.21.15.96:0/2809725164 <== osd.2 v2:172.21.15.151:6800/981902966 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1430025ea0 con 0x7f14200b94a0 2024-04-25T14:11:57.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.421+0000 7f143b2e1700 1 -- 172.21.15.96:0/2809725164 --> [v2:172.21.15.151:6800/981902966,v1:172.21.15.151:6801/981902966] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f14340641e0 con 0x7f14200b94a0 2024-04-25T14:11:57.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.421+0000 7f142a7fc700 1 -- 172.21.15.96:0/2809725164 <== osd.2 v2:172.21.15.151:6800/981902966 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f14340641e0 con 0x7f14200b94a0 2024-04-25T14:11:57.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.422+0000 7f143b2e1700 1 -- 172.21.15.96:0/2809725164 >> [v2:172.21.15.151:6800/981902966,v1:172.21.15.151:6801/981902966] conn(0x7f14200b94a0 msgr2=0x7f14200bb8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:57.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.422+0000 7f143b2e1700 1 --2- 172.21.15.96:0/2809725164 >> [v2:172.21.15.151:6800/981902966,v1:172.21.15.151:6801/981902966] conn(0x7f14200b94a0 0x7f14200bb8f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:57.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.422+0000 7f143b2e1700 1 -- 172.21.15.96:0/2809725164 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f14200afc10 msgr2=0x7f14200b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:57.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.422+0000 7f143b2e1700 1 --2- 172.21.15.96:0/2809725164 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f14200afc10 0x7f14200b2120 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1424000c00 tx=0x7f142401a040 comp rx=0 tx=0).stop 2024-04-25T14:11:57.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.422+0000 7f143b2e1700 1 -- 172.21.15.96:0/2809725164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14341044c0 msgr2=0x7f143419ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:57.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.422+0000 7f143b2e1700 1 --2- 172.21.15.96:0/2809725164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14341044c0 0x7f143419ad40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f143000e9c0 tx=0x7f143000ecd0 comp rx=0 tx=0).stop 2024-04-25T14:11:57.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.422+0000 7f143987e700 1 -- 172.21.15.96:0/2809725164 reap_dead start 2024-04-25T14:11:57.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.422+0000 7f143b2e1700 1 -- 172.21.15.96:0/2809725164 shutdown_connections 2024-04-25T14:11:57.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.422+0000 7f143b2e1700 1 -- 172.21.15.96:0/2809725164 >> 172.21.15.96:0/2809725164 conn(0x7f14340fff10 msgr2=0x7f143410a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:57.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.422+0000 7f143b2e1700 1 -- 172.21.15.96:0/2809725164 shutdown_connections 2024-04-25T14:11:57.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:11:57.422+0000 7f143b2e1700 1 -- 172.21.15.96:0/2809725164 wait complete. 2024-04-25T14:11:58.728 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:11:58 smithi096 ceph-mon[29207]: pgmap v294: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-25T14:11:58.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:11:58 smithi097 ceph-mon[32720]: pgmap v294: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-25T14:11:58.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:11:58 smithi151 ceph-mon[32990]: pgmap v294: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-25T14:12:00.454 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:12:00.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:00 smithi096 ceph-mon[29207]: pgmap v295: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-25T14:12:00.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:00 smithi097 ceph-mon[32720]: pgmap v295: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-25T14:12:00.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:00 smithi151 ceph-mon[32990]: pgmap v295: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-25T14:12:02.501 INFO:teuthology.orchestra.run.smithi096.stdout:77309411393 2024-04-25T14:12:02.501 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-25T14:12:02.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.716+0000 7f354220b700 1 -- 172.21.15.96:0/838613047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f353c104480 msgr2=0x7f353c104890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:02.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.716+0000 7f354220b700 1 --2- 172.21.15.96:0/838613047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f353c104480 0x7f353c104890 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f352c0099c0 tx=0x7f352c009cd0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.716+0000 7f354220b700 1 -- 172.21.15.96:0/838613047 shutdown_connections 2024-04-25T14:12:02.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.716+0000 7f354220b700 1 --2- 172.21.15.96:0/838613047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f353c109700 0x7f353c111060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.716+0000 7f354220b700 1 --2- 172.21.15.96:0/838613047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f353c104480 0x7f353c104890 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.716+0000 7f354220b700 1 --2- 172.21.15.96:0/838613047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f353c108db0 0x7f353c1091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.716+0000 7f354220b700 1 -- 172.21.15.96:0/838613047 >> 172.21.15.96:0/838613047 conn(0x7f353c0ffd70 msgr2=0x7f353c1021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:02.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.716+0000 7f354220b700 1 -- 172.21.15.96:0/838613047 shutdown_connections 2024-04-25T14:12:02.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.716+0000 7f354220b700 1 -- 172.21.15.96:0/838613047 wait complete. 2024-04-25T14:12:02.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.717+0000 7f354220b700 1 Processor -- start 2024-04-25T14:12:02.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.717+0000 7f354220b700 1 -- start start 2024-04-25T14:12:02.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.717+0000 7f354220b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f353c104480 0x7f353c1a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:02.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.717+0000 7f354220b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f353c108db0 0x7f353c1a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:02.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.717+0000 7f354220b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f353c109700 0x7f353c19aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:02.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.717+0000 7f354220b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f353c114330 con 0x7f353c104480 2024-04-25T14:12:02.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.717+0000 7f354220b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f353c1141b0 con 0x7f353c109700 2024-04-25T14:12:02.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.717+0000 7f354220b700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f353c1144b0 con 0x7f353c108db0 2024-04-25T14:12:02.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.717+0000 7f353b7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f353c104480 0x7f353c1a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:02.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.717+0000 7f353b7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f353c104480 0x7f353c1a08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53012/0 (socket says 172.21.15.96:53012) 2024-04-25T14:12:02.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.717+0000 7f353b7fe700 1 -- 172.21.15.96:0/1616033748 learned_addr learned my addr 172.21.15.96:0/1616033748 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:12:02.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.717+0000 7f353affd700 1 --2- 172.21.15.96:0/1616033748 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f353c108db0 0x7f353c1a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:02.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.718+0000 7f353bfff700 1 --2- 172.21.15.96:0/1616033748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f353c109700 0x7f353c19aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:02.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.718+0000 7f353b7fe700 1 -- 172.21.15.96:0/1616033748 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f353c108db0 msgr2=0x7f353c1a0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:02.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.718+0000 7f353b7fe700 1 --2- 172.21.15.96:0/1616033748 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f353c108db0 0x7f353c1a0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.718+0000 7f353b7fe700 1 -- 172.21.15.96:0/1616033748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f353c109700 msgr2=0x7f353c19aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:02.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.718+0000 7f353b7fe700 1 --2- 172.21.15.96:0/1616033748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f353c109700 0x7f353c19aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.718+0000 7f353b7fe700 1 -- 172.21.15.96:0/1616033748 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f353c19b160 con 0x7f353c104480 2024-04-25T14:12:02.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.718+0000 7f353b7fe700 1 --2- 172.21.15.96:0/1616033748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f353c104480 0x7f353c1a08b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f352c00b580 tx=0x7f352c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:02.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.718+0000 7f3538ff9700 1 -- 172.21.15.96:0/1616033748 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c027070 con 0x7f353c104480 2024-04-25T14:12:02.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.718+0000 7f354220b700 1 -- 172.21.15.96:0/1616033748 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f353c19b3f0 con 0x7f353c104480 2024-04-25T14:12:02.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.718+0000 7f3538ff9700 1 -- 172.21.15.96:0/1616033748 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f352c004020 con 0x7f353c104480 2024-04-25T14:12:02.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.718+0000 7f3538ff9700 1 -- 172.21.15.96:0/1616033748 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c022e10 con 0x7f353c104480 2024-04-25T14:12:02.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.718+0000 7f354220b700 1 -- 172.21.15.96:0/1616033748 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f353c19b920 con 0x7f353c104480 2024-04-25T14:12:02.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.719+0000 7f354220b700 1 -- 172.21.15.96:0/1616033748 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f353c063e20 con 0x7f353c104480 2024-04-25T14:12:02.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.720+0000 7f3538ff9700 1 -- 172.21.15.96:0/1616033748 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f352c02a070 con 0x7f353c104480 2024-04-25T14:12:02.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.721+0000 7f3538ff9700 1 --2- 172.21.15.96:0/1616033748 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f35240b4020 0x7f35240b6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:02.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.721+0000 7f353affd700 1 --2- 172.21.15.96:0/1616033748 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f35240b4020 0x7f35240b6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:02.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.721+0000 7f3538ff9700 1 -- 172.21.15.96:0/1616033748 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7f352c0e0330 con 0x7f353c104480 2024-04-25T14:12:02.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.721+0000 7f3538ff9700 1 --2- 172.21.15.96:0/1616033748 >> [v2:172.21.15.96:6802/3832506168,v1:172.21.15.96:6803/3832506168] conn(0x7f35240bd850 0x7f35240bfca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:02.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.721+0000 7f3538ff9700 1 -- 172.21.15.96:0/1616033748 --> [v2:172.21.15.96:6802/3832506168,v1:172.21.15.96:6803/3832506168] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f352c00eea0 con 0x7f35240bd850 2024-04-25T14:12:02.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.721+0000 7f3538ff9700 1 -- 172.21.15.96:0/1616033748 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f352c0e0810 con 0x7f353c104480 2024-04-25T14:12:02.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.722+0000 7f353affd700 1 --2- 172.21.15.96:0/1616033748 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f35240b4020 0x7f35240b6530 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f353c103310 tx=0x7f353000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:02.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.722+0000 7f353bfff700 1 --2- 172.21.15.96:0/1616033748 >> [v2:172.21.15.96:6802/3832506168,v1:172.21.15.96:6803/3832506168] conn(0x7f35240bd850 0x7f35240bfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:02.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.722+0000 7f353bfff700 1 --2- 172.21.15.96:0/1616033748 >> [v2:172.21.15.96:6802/3832506168,v1:172.21.15.96:6803/3832506168] conn(0x7f35240bd850 0x7f35240bfca0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:02.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.723+0000 7f3538ff9700 1 -- 172.21.15.96:0/1616033748 <== osd.0 v2:172.21.15.96:6802/3832506168 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f352c00eea0 con 0x7f35240bd850 2024-04-25T14:12:02.733 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:02 smithi096 ceph-mon[29207]: pgmap v296: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-25T14:12:02.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.744+0000 7f354220b700 1 -- 172.21.15.96:0/1616033748 --> [v2:172.21.15.96:6802/3832506168,v1:172.21.15.96:6803/3832506168] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f353c064220 con 0x7f35240bd850 2024-04-25T14:12:02.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.744+0000 7f3538ff9700 1 -- 172.21.15.96:0/1616033748 <== osd.0 v2:172.21.15.96:6802/3832506168 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f353c064220 con 0x7f35240bd850 2024-04-25T14:12:02.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.745+0000 7f354220b700 1 -- 172.21.15.96:0/1616033748 >> [v2:172.21.15.96:6802/3832506168,v1:172.21.15.96:6803/3832506168] conn(0x7f35240bd850 msgr2=0x7f35240bfca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:02.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.745+0000 7f354220b700 1 --2- 172.21.15.96:0/1616033748 >> [v2:172.21.15.96:6802/3832506168,v1:172.21.15.96:6803/3832506168] conn(0x7f35240bd850 0x7f35240bfca0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.745+0000 7f354220b700 1 -- 172.21.15.96:0/1616033748 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f35240b4020 msgr2=0x7f35240b6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:02.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.745+0000 7f354220b700 1 --2- 172.21.15.96:0/1616033748 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f35240b4020 0x7f35240b6530 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f353c103310 tx=0x7f353000a380 comp rx=0 tx=0).stop 2024-04-25T14:12:02.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.745+0000 7f354220b700 1 -- 172.21.15.96:0/1616033748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f353c104480 msgr2=0x7f353c1a08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:02.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.745+0000 7f354220b700 1 --2- 172.21.15.96:0/1616033748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f353c104480 0x7f353c1a08b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f352c00b580 tx=0x7f352c0047b0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.745+0000 7f353bfff700 1 -- 172.21.15.96:0/1616033748 reap_dead start 2024-04-25T14:12:02.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.745+0000 7f354220b700 1 -- 172.21.15.96:0/1616033748 shutdown_connections 2024-04-25T14:12:02.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.745+0000 7f354220b700 1 -- 172.21.15.96:0/1616033748 >> 172.21.15.96:0/1616033748 conn(0x7f353c0ffd70 msgr2=0x7f353c1085b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:02.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.745+0000 7f354220b700 1 -- 172.21.15.96:0/1616033748 shutdown_connections 2024-04-25T14:12:02.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:02.745+0000 7f354220b700 1 -- 172.21.15.96:0/1616033748 wait complete. 2024-04-25T14:12:02.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:02 smithi097 ceph-mon[32720]: pgmap v296: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-25T14:12:02.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:02 smithi151 ceph-mon[32990]: pgmap v296: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-25T14:12:03.919 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:03 smithi096 ceph-mon[29207]: pgmap v297: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-25T14:12:03.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:03 smithi097 ceph-mon[32720]: pgmap v297: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-25T14:12:03.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.934+0000 7f0bedef3700 1 -- 172.21.15.96:0/2175643996 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0be81088c0 msgr2=0x7f0be8108cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:03.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.934+0000 7f0bedef3700 1 --2- 172.21.15.96:0/2175643996 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0be81088c0 0x7f0be8108cd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8009990 tx=0x7f0bd8009ca0 comp rx=0 tx=0).stop 2024-04-25T14:12:03.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.934+0000 7f0bedef3700 1 -- 172.21.15.96:0/2175643996 shutdown_connections 2024-04-25T14:12:03.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.934+0000 7f0bedef3700 1 --2- 172.21.15.96:0/2175643996 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0be81031e0 0x7f0be810f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:03.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.934+0000 7f0bedef3700 1 --2- 172.21.15.96:0/2175643996 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0be81088c0 0x7f0be8108cd0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:03.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.934+0000 7f0bedef3700 1 --2- 172.21.15.96:0/2175643996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0be8102890 0x7f0be8102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:03.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.934+0000 7f0bedef3700 1 -- 172.21.15.96:0/2175643996 >> 172.21.15.96:0/2175643996 conn(0x7f0be80fe5c0 msgr2=0x7f0be81009e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:03.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.934+0000 7f0bedef3700 1 -- 172.21.15.96:0/2175643996 shutdown_connections 2024-04-25T14:12:03.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.934+0000 7f0bedef3700 1 -- 172.21.15.96:0/2175643996 wait complete. 2024-04-25T14:12:03.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.935+0000 7f0bedef3700 1 Processor -- start 2024-04-25T14:12:03.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.935+0000 7f0bedef3700 1 -- start start 2024-04-25T14:12:03.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.935+0000 7f0bedef3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0be8102890 0x7f0be819bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:03.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.935+0000 7f0bedef3700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0be81031e0 0x7f0be819c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:03.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.935+0000 7f0bedef3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0be81088c0 0x7f0be819c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:03.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.935+0000 7f0bedef3700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0be81129c0 con 0x7f0be8102890 2024-04-25T14:12:03.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.935+0000 7f0bedef3700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0be8112840 con 0x7f0be81088c0 2024-04-25T14:12:03.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.935+0000 7f0bedef3700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0be8112b40 con 0x7f0be81031e0 2024-04-25T14:12:03.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.935+0000 7f0be77fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0be8102890 0x7f0be819bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:03.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.935+0000 7f0be77fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0be8102890 0x7f0be819bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53028/0 (socket says 172.21.15.96:53028) 2024-04-25T14:12:03.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.935+0000 7f0be77fe700 1 -- 172.21.15.96:0/574690342 learned_addr learned my addr 172.21.15.96:0/574690342 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:12:03.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.936+0000 7f0bdffff700 1 --2- 172.21.15.96:0/574690342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0be81031e0 0x7f0be819c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:03.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.936+0000 7f0be7fff700 1 --2- 172.21.15.96:0/574690342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0be81088c0 0x7f0be819c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:03.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.936+0000 7f0be77fe700 1 -- 172.21.15.96:0/574690342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0be81031e0 msgr2=0x7f0be819c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:03.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.936+0000 7f0be77fe700 1 --2- 172.21.15.96:0/574690342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0be81031e0 0x7f0be819c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:03.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.936+0000 7f0be77fe700 1 -- 172.21.15.96:0/574690342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0be81088c0 msgr2=0x7f0be819c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:03.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.936+0000 7f0be77fe700 1 --2- 172.21.15.96:0/574690342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0be81088c0 0x7f0be819c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:03.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.936+0000 7f0be77fe700 1 -- 172.21.15.96:0/574690342 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be81a6e00 con 0x7f0be8102890 2024-04-25T14:12:03.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.936+0000 7f0be77fe700 1 --2- 172.21.15.96:0/574690342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0be8102890 0x7f0be819bb40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8000c00 tx=0x7f0bd80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:03.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.936+0000 7f0be57fa700 1 -- 172.21.15.96:0/574690342 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd8027070 con 0x7f0be8102890 2024-04-25T14:12:03.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.936+0000 7f0bedef3700 1 -- 172.21.15.96:0/574690342 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0be81a7090 con 0x7f0be8102890 2024-04-25T14:12:03.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.936+0000 7f0be57fa700 1 -- 172.21.15.96:0/574690342 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bd80041d0 con 0x7f0be8102890 2024-04-25T14:12:03.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.936+0000 7f0bedef3700 1 -- 172.21.15.96:0/574690342 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0be81a7540 con 0x7f0be8102890 2024-04-25T14:12:03.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.936+0000 7f0be57fa700 1 -- 172.21.15.96:0/574690342 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd800f460 con 0x7f0be8102890 2024-04-25T14:12:03.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.937+0000 7f0bedef3700 1 -- 172.21.15.96:0/574690342 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0be81077b0 con 0x7f0be8102890 2024-04-25T14:12:03.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:03 smithi151 ceph-mon[32990]: pgmap v297: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-25T14:12:03.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.939+0000 7f0be57fa700 1 -- 172.21.15.96:0/574690342 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f0bd802a070 con 0x7f0be8102890 2024-04-25T14:12:03.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.939+0000 7f0be57fa700 1 --2- 172.21.15.96:0/574690342 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f0bd40afc10 0x7f0bd40b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:03.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.939+0000 7f0bdffff700 1 --2- 172.21.15.96:0/574690342 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f0bd40afc10 0x7f0bd40b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:03.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.939+0000 7f0be57fa700 1 -- 172.21.15.96:0/574690342 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7f0bd80e0450 con 0x7f0be8102890 2024-04-25T14:12:03.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.940+0000 7f0be57fa700 1 --2- 172.21.15.96:0/574690342 >> [v2:172.21.15.97:6808/3727031962,v1:172.21.15.97:6809/3727031962] conn(0x7f0bd40b9440 0x7f0bd40bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:03.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.940+0000 7f0be57fa700 1 -- 172.21.15.96:0/574690342 --> [v2:172.21.15.97:6808/3727031962,v1:172.21.15.97:6809/3727031962] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0bd80043d0 con 0x7f0bd40b9440 2024-04-25T14:12:03.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.940+0000 7f0be57fa700 1 -- 172.21.15.96:0/574690342 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f0bd80e0930 con 0x7f0be8102890 2024-04-25T14:12:03.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.940+0000 7f0bdffff700 1 --2- 172.21.15.96:0/574690342 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f0bd40afc10 0x7f0bd40b2120 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0005ea0 tx=0x7f0bd0005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:03.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.940+0000 7f0be7fff700 1 --2- 172.21.15.96:0/574690342 >> [v2:172.21.15.97:6808/3727031962,v1:172.21.15.97:6809/3727031962] conn(0x7f0bd40b9440 0x7f0bd40bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:03.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.940+0000 7f0be7fff700 1 --2- 172.21.15.96:0/574690342 >> [v2:172.21.15.97:6808/3727031962,v1:172.21.15.97:6809/3727031962] conn(0x7f0bd40b9440 0x7f0bd40bb890 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:03.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.941+0000 7f0be57fa700 1 -- 172.21.15.96:0/574690342 <== osd.4 v2:172.21.15.97:6808/3727031962 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0bd80043d0 con 0x7f0bd40b9440 2024-04-25T14:12:03.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.957+0000 7f0bedef3700 1 -- 172.21.15.96:0/574690342 --> [v2:172.21.15.97:6808/3727031962,v1:172.21.15.97:6809/3727031962] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0be8103fd0 con 0x7f0bd40b9440 2024-04-25T14:12:03.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.958+0000 7f0be57fa700 1 -- 172.21.15.96:0/574690342 <== osd.4 v2:172.21.15.97:6808/3727031962 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0be8103fd0 con 0x7f0bd40b9440 2024-04-25T14:12:03.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.958+0000 7f0bedef3700 1 -- 172.21.15.96:0/574690342 >> [v2:172.21.15.97:6808/3727031962,v1:172.21.15.97:6809/3727031962] conn(0x7f0bd40b9440 msgr2=0x7f0bd40bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:03.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.958+0000 7f0bedef3700 1 --2- 172.21.15.96:0/574690342 >> [v2:172.21.15.97:6808/3727031962,v1:172.21.15.97:6809/3727031962] conn(0x7f0bd40b9440 0x7f0bd40bb890 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:03.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.958+0000 7f0bedef3700 1 -- 172.21.15.96:0/574690342 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f0bd40afc10 msgr2=0x7f0bd40b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:03.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.958+0000 7f0bedef3700 1 --2- 172.21.15.96:0/574690342 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f0bd40afc10 0x7f0bd40b2120 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0005ea0 tx=0x7f0bd0005e30 comp rx=0 tx=0).stop 2024-04-25T14:12:03.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.958+0000 7f0bedef3700 1 -- 172.21.15.96:0/574690342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0be8102890 msgr2=0x7f0be819bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:03.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.958+0000 7f0bedef3700 1 --2- 172.21.15.96:0/574690342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0be8102890 0x7f0be819bb40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8000c00 tx=0x7f0bd80047b0 comp rx=0 tx=0).stop 2024-04-25T14:12:03.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.958+0000 7f0be7fff700 1 -- 172.21.15.96:0/574690342 reap_dead start 2024-04-25T14:12:03.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.958+0000 7f0bedef3700 1 -- 172.21.15.96:0/574690342 shutdown_connections 2024-04-25T14:12:03.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.958+0000 7f0bedef3700 1 -- 172.21.15.96:0/574690342 >> 172.21.15.96:0/574690342 conn(0x7f0be80fe5c0 msgr2=0x7f0be8106dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:03.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.958+0000 7f0bedef3700 1 -- 172.21.15.96:0/574690342 shutdown_connections 2024-04-25T14:12:03.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:03.958+0000 7f0bedef3700 1 -- 172.21.15.96:0/574690342 wait complete. 2024-04-25T14:12:05.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.823+0000 7f094d99d700 1 -- 172.21.15.96:0/2692109025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0948105390 msgr2=0x7f09481057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:05.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.823+0000 7f094d99d700 1 --2- 172.21.15.96:0/2692109025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0948105390 0x7f09481057a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0938009990 tx=0x7f0938009ca0 comp rx=0 tx=0).stop 2024-04-25T14:12:05.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.823+0000 7f094d99d700 1 -- 172.21.15.96:0/2692109025 shutdown_connections 2024-04-25T14:12:05.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.823+0000 7f094d99d700 1 --2- 172.21.15.96:0/2692109025 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f094810e7c0 0x7f0948110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:05.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.823+0000 7f094d99d700 1 --2- 172.21.15.96:0/2692109025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0948105390 0x7f09481057a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:05.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.823+0000 7f094d99d700 1 --2- 172.21.15.96:0/2692109025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0948105ce0 0x7f094810e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:05.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.823+0000 7f094d99d700 1 -- 172.21.15.96:0/2692109025 >> 172.21.15.96:0/2692109025 conn(0x7f09480fec90 msgr2=0x7f09481010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:05.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.823+0000 7f094d99d700 1 -- 172.21.15.96:0/2692109025 shutdown_connections 2024-04-25T14:12:05.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.823+0000 7f094d99d700 1 -- 172.21.15.96:0/2692109025 wait complete. 2024-04-25T14:12:05.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.824+0000 7f094d99d700 1 Processor -- start 2024-04-25T14:12:05.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.824+0000 7f094d99d700 1 -- start start 2024-04-25T14:12:05.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.824+0000 7f094d99d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0948105390 0x7f0948103080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:05.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.824+0000 7f094d99d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0948105ce0 0x7f09481035c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:05.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.824+0000 7f094d99d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f094810e7c0 0x7f0948103b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:05.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.824+0000 7f094d99d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0948114390 con 0x7f0948105ce0 2024-04-25T14:12:05.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.824+0000 7f094d99d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0948114210 con 0x7f094810e7c0 2024-04-25T14:12:05.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.824+0000 7f094d99d700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0948114510 con 0x7f0948105390 2024-04-25T14:12:05.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.824+0000 7f09467fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0948105ce0 0x7f09481035c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:05.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.824+0000 7f09467fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0948105ce0 0x7f09481035c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53036/0 (socket says 172.21.15.96:53036) 2024-04-25T14:12:05.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.824+0000 7f09467fc700 1 -- 172.21.15.96:0/1376123094 learned_addr learned my addr 172.21.15.96:0/1376123094 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:12:05.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.824+0000 7f09477fe700 1 --2- 172.21.15.96:0/1376123094 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f094810e7c0 0x7f0948103b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:05.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.825+0000 7f0946ffd700 1 --2- 172.21.15.96:0/1376123094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0948105390 0x7f0948103080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:05.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.825+0000 7f09467fc700 1 -- 172.21.15.96:0/1376123094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0948105390 msgr2=0x7f0948103080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:05.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.825+0000 7f09467fc700 1 --2- 172.21.15.96:0/1376123094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0948105390 0x7f0948103080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:05.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.825+0000 7f09467fc700 1 -- 172.21.15.96:0/1376123094 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f094810e7c0 msgr2=0x7f0948103b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:05.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.825+0000 7f09467fc700 1 --2- 172.21.15.96:0/1376123094 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f094810e7c0 0x7f0948103b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:05.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.825+0000 7f09467fc700 1 -- 172.21.15.96:0/1376123094 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09481a90a0 con 0x7f0948105ce0 2024-04-25T14:12:05.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.825+0000 7f09467fc700 1 --2- 172.21.15.96:0/1376123094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0948105ce0 0x7f09481035c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f093c00d950 tx=0x7f093c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:05.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.825+0000 7f094c99b700 1 -- 172.21.15.96:0/1376123094 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093c016070 con 0x7f0948105ce0 2024-04-25T14:12:05.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.825+0000 7f094c99b700 1 -- 172.21.15.96:0/1376123094 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f093c009df0 con 0x7f0948105ce0 2024-04-25T14:12:05.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.825+0000 7f094d99d700 1 -- 172.21.15.96:0/1376123094 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09481a9390 con 0x7f0948105ce0 2024-04-25T14:12:05.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.825+0000 7f094c99b700 1 -- 172.21.15.96:0/1376123094 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093c00f3c0 con 0x7f0948105ce0 2024-04-25T14:12:05.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.825+0000 7f094d99d700 1 -- 172.21.15.96:0/1376123094 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09481a98d0 con 0x7f0948105ce0 2024-04-25T14:12:05.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.826+0000 7f094d99d700 1 -- 172.21.15.96:0/1376123094 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0948063da0 con 0x7f0948105ce0 2024-04-25T14:12:05.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.828+0000 7f094c99b700 1 -- 172.21.15.96:0/1376123094 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f093c019070 con 0x7f0948105ce0 2024-04-25T14:12:05.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.829+0000 7f094c99b700 1 --2- 172.21.15.96:0/1376123094 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f09300afc10 0x7f09300b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:05.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.829+0000 7f0946ffd700 1 --2- 172.21.15.96:0/1376123094 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f09300afc10 0x7f09300b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:05.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.829+0000 7f094c99b700 1 -- 172.21.15.96:0/1376123094 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7f093c0dede0 con 0x7f0948105ce0 2024-04-25T14:12:05.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.829+0000 7f094c99b700 1 --2- 172.21.15.96:0/1376123094 >> [v2:172.21.15.97:6816/2247216400,v1:172.21.15.97:6817/2247216400] conn(0x7f09300b9440 0x7f09300bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:05.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.829+0000 7f094c99b700 1 -- 172.21.15.96:0/1376123094 --> [v2:172.21.15.97:6816/2247216400,v1:172.21.15.97:6817/2247216400] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f093c01de50 con 0x7f09300b9440 2024-04-25T14:12:05.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.829+0000 7f094c99b700 1 -- 172.21.15.96:0/1376123094 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f093c0df2c0 con 0x7f0948105ce0 2024-04-25T14:12:05.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.830+0000 7f0946ffd700 1 --2- 172.21.15.96:0/1376123094 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f09300afc10 0x7f09300b2120 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0938003800 tx=0x7f093801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:05.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.830+0000 7f09477fe700 1 --2- 172.21.15.96:0/1376123094 >> [v2:172.21.15.97:6816/2247216400,v1:172.21.15.97:6817/2247216400] conn(0x7f09300b9440 0x7f09300bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:05.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.830+0000 7f09477fe700 1 --2- 172.21.15.96:0/1376123094 >> [v2:172.21.15.97:6816/2247216400,v1:172.21.15.97:6817/2247216400] conn(0x7f09300b9440 0x7f09300bb890 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:05.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.831+0000 7f094c99b700 1 -- 172.21.15.96:0/1376123094 <== osd.7 v2:172.21.15.97:6816/2247216400 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f093c01de50 con 0x7f09300b9440 2024-04-25T14:12:05.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.847+0000 7f094d99d700 1 -- 172.21.15.96:0/1376123094 --> [v2:172.21.15.97:6816/2247216400,v1:172.21.15.97:6817/2247216400] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f09480641a0 con 0x7f09300b9440 2024-04-25T14:12:05.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.848+0000 7f094c99b700 1 -- 172.21.15.96:0/1376123094 <== osd.7 v2:172.21.15.97:6816/2247216400 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f09480641a0 con 0x7f09300b9440 2024-04-25T14:12:05.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.848+0000 7f094d99d700 1 -- 172.21.15.96:0/1376123094 >> [v2:172.21.15.97:6816/2247216400,v1:172.21.15.97:6817/2247216400] conn(0x7f09300b9440 msgr2=0x7f09300bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:05.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.848+0000 7f094d99d700 1 --2- 172.21.15.96:0/1376123094 >> [v2:172.21.15.97:6816/2247216400,v1:172.21.15.97:6817/2247216400] conn(0x7f09300b9440 0x7f09300bb890 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:05.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.848+0000 7f094d99d700 1 -- 172.21.15.96:0/1376123094 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f09300afc10 msgr2=0x7f09300b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:05.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.848+0000 7f094d99d700 1 --2- 172.21.15.96:0/1376123094 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f09300afc10 0x7f09300b2120 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0938003800 tx=0x7f093801a040 comp rx=0 tx=0).stop 2024-04-25T14:12:05.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.848+0000 7f094d99d700 1 -- 172.21.15.96:0/1376123094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0948105ce0 msgr2=0x7f09481035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:05.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.848+0000 7f094d99d700 1 --2- 172.21.15.96:0/1376123094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0948105ce0 0x7f09481035c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f093c00d950 tx=0x7f093c00dc60 comp rx=0 tx=0).stop 2024-04-25T14:12:05.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.848+0000 7f09477fe700 1 -- 172.21.15.96:0/1376123094 reap_dead start 2024-04-25T14:12:05.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.848+0000 7f094d99d700 1 -- 172.21.15.96:0/1376123094 shutdown_connections 2024-04-25T14:12:05.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.848+0000 7f094d99d700 1 -- 172.21.15.96:0/1376123094 >> 172.21.15.96:0/1376123094 conn(0x7f09480fec90 msgr2=0x7f09480ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:05.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.848+0000 7f094d99d700 1 -- 172.21.15.96:0/1376123094 shutdown_connections 2024-04-25T14:12:05.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:05.848+0000 7f094d99d700 1 -- 172.21.15.96:0/1376123094 wait complete. 2024-04-25T14:12:06.685 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:06 smithi096 ceph-mon[29207]: pgmap v298: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-25T14:12:06.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:06 smithi097 ceph-mon[32720]: pgmap v298: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-25T14:12:06.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:06 smithi151 ceph-mon[32990]: pgmap v298: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-25T14:12:07.734 INFO:teuthology.orchestra.run.smithi096.stdout:34359738444 2024-04-25T14:12:07.734 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-25T14:12:08.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:08 smithi096 ceph-mon[29207]: pgmap v299: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 12 op/s 2024-04-25T14:12:08.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:08 smithi097 ceph-mon[32720]: pgmap v299: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 12 op/s 2024-04-25T14:12:08.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:08 smithi151 ceph-mon[32990]: pgmap v299: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 12 op/s 2024-04-25T14:12:08.948 INFO:teuthology.orchestra.run.smithi096.stdout:206158430249 2024-04-25T14:12:08.948 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-25T14:12:09.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.002+0000 7f9cd649b700 1 -- 172.21.15.96:0/3000763045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9cd01044a0 msgr2=0x7f9cd01048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:09.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.002+0000 7f9cd649b700 1 --2- 172.21.15.96:0/3000763045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9cd01044a0 0x7f9cd01048b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00099c0 tx=0x7f9cc0009cd0 comp rx=0 tx=0).stop 2024-04-25T14:12:09.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.002+0000 7f9cd649b700 1 -- 172.21.15.96:0/3000763045 shutdown_connections 2024-04-25T14:12:09.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.002+0000 7f9cd649b700 1 --2- 172.21.15.96:0/3000763045 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cd0109720 0x7f9cd0111080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:09.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.002+0000 7f9cd649b700 1 --2- 172.21.15.96:0/3000763045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9cd01044a0 0x7f9cd01048b0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:09.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.002+0000 7f9cd649b700 1 --2- 172.21.15.96:0/3000763045 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9cd0108dd0 0x7f9cd01091e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:09.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.002+0000 7f9cd649b700 1 -- 172.21.15.96:0/3000763045 >> 172.21.15.96:0/3000763045 conn(0x7f9cd00ffd70 msgr2=0x7f9cd01021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:09.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.002+0000 7f9cd649b700 1 -- 172.21.15.96:0/3000763045 shutdown_connections 2024-04-25T14:12:09.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.002+0000 7f9cd649b700 1 -- 172.21.15.96:0/3000763045 wait complete. 2024-04-25T14:12:09.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.003+0000 7f9cd649b700 1 Processor -- start 2024-04-25T14:12:09.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.003+0000 7f9cd649b700 1 -- start start 2024-04-25T14:12:09.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.003+0000 7f9cd649b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9cd01044a0 0x7f9cd0076aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:09.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.003+0000 7f9cd649b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9cd0108dd0 0x7f9cd0076fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:09.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.003+0000 7f9cd649b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cd0109720 0x7f9cd0077520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:09.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.003+0000 7f9cd649b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd01143c0 con 0x7f9cd01044a0 2024-04-25T14:12:09.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.003+0000 7f9cd649b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd0114240 con 0x7f9cd0109720 2024-04-25T14:12:09.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.003+0000 7f9cd649b700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd0114540 con 0x7f9cd0108dd0 2024-04-25T14:12:09.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.003+0000 7f9ccffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9cd01044a0 0x7f9cd0076aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:09.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.003+0000 7f9ccffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9cd01044a0 0x7f9cd0076aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53058/0 (socket says 172.21.15.96:53058) 2024-04-25T14:12:09.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.003+0000 7f9ccffff700 1 -- 172.21.15.96:0/4159597119 learned_addr learned my addr 172.21.15.96:0/4159597119 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:12:09.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.003+0000 7f9cd4a38700 1 --2- 172.21.15.96:0/4159597119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cd0109720 0x7f9cd0077520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:09.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.003+0000 7f9ccf7fe700 1 --2- 172.21.15.96:0/4159597119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9cd0108dd0 0x7f9cd0076fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:09.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.003+0000 7f9ccffff700 1 -- 172.21.15.96:0/4159597119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9cd0108dd0 msgr2=0x7f9cd0076fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:09.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.003+0000 7f9ccffff700 1 --2- 172.21.15.96:0/4159597119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9cd0108dd0 0x7f9cd0076fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:09.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.003+0000 7f9ccffff700 1 -- 172.21.15.96:0/4159597119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cd0109720 msgr2=0x7f9cd0077520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:09.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.003+0000 7f9ccffff700 1 --2- 172.21.15.96:0/4159597119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cd0109720 0x7f9cd0077520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:09.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.003+0000 7f9ccffff700 1 -- 172.21.15.96:0/4159597119 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cd01a90f0 con 0x7f9cd01044a0 2024-04-25T14:12:09.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.004+0000 7f9ccf7fe700 1 --2- 172.21.15.96:0/4159597119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9cd0108dd0 0x7f9cd0076fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:12:09.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.004+0000 7f9cd4a38700 1 --2- 172.21.15.96:0/4159597119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cd0109720 0x7f9cd0077520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:12:09.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.004+0000 7f9ccffff700 1 --2- 172.21.15.96:0/4159597119 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9cd01044a0 0x7f9cd0076aa0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00098e0 tx=0x7f9cc0003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:09.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.004+0000 7f9ccd7fa700 1 -- 172.21.15.96:0/4159597119 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc0027070 con 0x7f9cd01044a0 2024-04-25T14:12:09.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.004+0000 7f9ccd7fa700 1 -- 172.21.15.96:0/4159597119 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cc00040a0 con 0x7f9cd01044a0 2024-04-25T14:12:09.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.004+0000 7f9cd649b700 1 -- 172.21.15.96:0/4159597119 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cd01a9380 con 0x7f9cd01044a0 2024-04-25T14:12:09.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.004+0000 7f9ccd7fa700 1 -- 172.21.15.96:0/4159597119 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc000f460 con 0x7f9cd01044a0 2024-04-25T14:12:09.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.004+0000 7f9cd649b700 1 -- 172.21.15.96:0/4159597119 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cd01a97d0 con 0x7f9cd01044a0 2024-04-25T14:12:09.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.005+0000 7f9cd649b700 1 -- 172.21.15.96:0/4159597119 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9cd0063da0 con 0x7f9cd01044a0 2024-04-25T14:12:09.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.006+0000 7f9ccd7fa700 1 -- 172.21.15.96:0/4159597119 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f9cc002a070 con 0x7f9cd01044a0 2024-04-25T14:12:09.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.007+0000 7f9ccd7fa700 1 --2- 172.21.15.96:0/4159597119 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f9cb80afc10 0x7f9cb80b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:09.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.007+0000 7f9ccf7fe700 1 --2- 172.21.15.96:0/4159597119 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f9cb80afc10 0x7f9cb80b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:09.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.007+0000 7f9ccd7fa700 1 -- 172.21.15.96:0/4159597119 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7f9cc00e09e0 con 0x7f9cd01044a0 2024-04-25T14:12:09.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.007+0000 7f9ccd7fa700 1 --2- 172.21.15.96:0/4159597119 >> [v2:172.21.15.97:6800/1083311494,v1:172.21.15.97:6801/1083311494] conn(0x7f9cb80b9440 0x7f9cb80bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:09.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.007+0000 7f9ccd7fa700 1 -- 172.21.15.96:0/4159597119 --> [v2:172.21.15.97:6800/1083311494,v1:172.21.15.97:6801/1083311494] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9cb80bbf60 con 0x7f9cb80b9440 2024-04-25T14:12:09.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.007+0000 7f9ccd7fa700 1 -- 172.21.15.96:0/4159597119 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f9cc00e0ec0 con 0x7f9cd01044a0 2024-04-25T14:12:09.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.007+0000 7f9ccf7fe700 1 --2- 172.21.15.96:0/4159597119 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f9cb80afc10 0x7f9cb80b2120 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0107530 tx=0x7f9cc4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:09.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.007+0000 7f9cd4a38700 1 --2- 172.21.15.96:0/4159597119 >> [v2:172.21.15.97:6800/1083311494,v1:172.21.15.97:6801/1083311494] conn(0x7f9cb80b9440 0x7f9cb80bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:09.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.008+0000 7f9cd4a38700 1 --2- 172.21.15.96:0/4159597119 >> [v2:172.21.15.97:6800/1083311494,v1:172.21.15.97:6801/1083311494] conn(0x7f9cb80b9440 0x7f9cb80bb890 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:09.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.009+0000 7f9ccd7fa700 1 -- 172.21.15.96:0/4159597119 <== osd.1 v2:172.21.15.97:6800/1083311494 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9cb80bbf60 con 0x7f9cb80b9440 2024-04-25T14:12:09.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.025+0000 7f9cd649b700 1 -- 172.21.15.96:0/4159597119 --> [v2:172.21.15.97:6800/1083311494,v1:172.21.15.97:6801/1083311494] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9cd00641a0 con 0x7f9cb80b9440 2024-04-25T14:12:09.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.025+0000 7f9ccd7fa700 1 -- 172.21.15.96:0/4159597119 <== osd.1 v2:172.21.15.97:6800/1083311494 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9cd00641a0 con 0x7f9cb80b9440 2024-04-25T14:12:09.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.025+0000 7f9cd649b700 1 -- 172.21.15.96:0/4159597119 >> [v2:172.21.15.97:6800/1083311494,v1:172.21.15.97:6801/1083311494] conn(0x7f9cb80b9440 msgr2=0x7f9cb80bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:09.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.025+0000 7f9cd649b700 1 --2- 172.21.15.96:0/4159597119 >> [v2:172.21.15.97:6800/1083311494,v1:172.21.15.97:6801/1083311494] conn(0x7f9cb80b9440 0x7f9cb80bb890 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:09.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.026+0000 7f9cd649b700 1 -- 172.21.15.96:0/4159597119 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f9cb80afc10 msgr2=0x7f9cb80b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:09.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.026+0000 7f9cd649b700 1 --2- 172.21.15.96:0/4159597119 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f9cb80afc10 0x7f9cb80b2120 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0107530 tx=0x7f9cc4008040 comp rx=0 tx=0).stop 2024-04-25T14:12:09.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.026+0000 7f9cd649b700 1 -- 172.21.15.96:0/4159597119 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9cd01044a0 msgr2=0x7f9cd0076aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:09.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.026+0000 7f9cd649b700 1 --2- 172.21.15.96:0/4159597119 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9cd01044a0 0x7f9cd0076aa0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00098e0 tx=0x7f9cc0003680 comp rx=0 tx=0).stop 2024-04-25T14:12:09.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.026+0000 7f9cd4a38700 1 -- 172.21.15.96:0/4159597119 reap_dead start 2024-04-25T14:12:09.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.026+0000 7f9cd649b700 1 -- 172.21.15.96:0/4159597119 shutdown_connections 2024-04-25T14:12:09.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.026+0000 7f9cd649b700 1 -- 172.21.15.96:0/4159597119 >> 172.21.15.96:0/4159597119 conn(0x7f9cd00ffd70 msgr2=0x7f9cd0102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:09.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.026+0000 7f9cd649b700 1 -- 172.21.15.96:0/4159597119 shutdown_connections 2024-04-25T14:12:09.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:08.026+0000 7f9cd649b700 1 -- 172.21.15.96:0/4159597119 wait complete. 2024-04-25T14:12:09.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.461+0000 7f0ef5236700 1 -- 172.21.15.96:0/455184694 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ef0103360 msgr2=0x7f0ef0103770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:09.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.461+0000 7f0ef5236700 1 --2- 172.21.15.96:0/455184694 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ef0103360 0x7f0ef0103770 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0ee00099c0 tx=0x7f0ee0009cd0 comp rx=0 tx=0).stop 2024-04-25T14:12:09.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.461+0000 7f0ef5236700 1 -- 172.21.15.96:0/455184694 shutdown_connections 2024-04-25T14:12:09.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.461+0000 7f0ef5236700 1 --2- 172.21.15.96:0/455184694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ef01085e0 0x7f0ef010ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:09.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.461+0000 7f0ef5236700 1 --2- 172.21.15.96:0/455184694 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ef0103360 0x7f0ef0103770 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:09.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.461+0000 7f0ef5236700 1 --2- 172.21.15.96:0/455184694 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ef0107c90 0x7f0ef01080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:09.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.461+0000 7f0ef5236700 1 -- 172.21.15.96:0/455184694 >> 172.21.15.96:0/455184694 conn(0x7f0ef00fec90 msgr2=0x7f0ef01010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:09.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.461+0000 7f0ef5236700 1 -- 172.21.15.96:0/455184694 shutdown_connections 2024-04-25T14:12:09.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.461+0000 7f0ef5236700 1 -- 172.21.15.96:0/455184694 wait complete. 2024-04-25T14:12:09.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.462+0000 7f0ef5236700 1 Processor -- start 2024-04-25T14:12:09.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.462+0000 7f0ef5236700 1 -- start start 2024-04-25T14:12:09.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.462+0000 7f0ef5236700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ef0103360 0x7f0ef019bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:09.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.462+0000 7f0ef5236700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ef0107c90 0x7f0ef019c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:09.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.462+0000 7f0ef5236700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ef01085e0 0x7f0ef019c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:09.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.462+0000 7f0ef5236700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef0113300 con 0x7f0ef0103360 2024-04-25T14:12:09.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.462+0000 7f0ef5236700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef0113180 con 0x7f0ef01085e0 2024-04-25T14:12:09.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.462+0000 7f0ef5236700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef0113480 con 0x7f0ef0107c90 2024-04-25T14:12:09.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.462+0000 7f0eeed9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ef0103360 0x7f0ef019bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:09.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.462+0000 7f0eeed9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ef0103360 0x7f0ef019bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53072/0 (socket says 172.21.15.96:53072) 2024-04-25T14:12:09.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.463+0000 7f0eeed9d700 1 -- 172.21.15.96:0/121244930 learned_addr learned my addr 172.21.15.96:0/121244930 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:12:09.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.463+0000 7f0eee59c700 1 --2- 172.21.15.96:0/121244930 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ef0107c90 0x7f0ef019c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:09.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.463+0000 7f0eef59e700 1 --2- 172.21.15.96:0/121244930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ef01085e0 0x7f0ef019c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:09.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.463+0000 7f0eeed9d700 1 -- 172.21.15.96:0/121244930 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ef0107c90 msgr2=0x7f0ef019c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:09.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.463+0000 7f0eeed9d700 1 --2- 172.21.15.96:0/121244930 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ef0107c90 0x7f0ef019c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:09.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.463+0000 7f0eeed9d700 1 -- 172.21.15.96:0/121244930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ef01085e0 msgr2=0x7f0ef019c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:09.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.463+0000 7f0eeed9d700 1 --2- 172.21.15.96:0/121244930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ef01085e0 0x7f0ef019c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:09.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.463+0000 7f0eeed9d700 1 -- 172.21.15.96:0/121244930 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef01a6f50 con 0x7f0ef0103360 2024-04-25T14:12:09.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.463+0000 7f0eef59e700 1 --2- 172.21.15.96:0/121244930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ef01085e0 0x7f0ef019c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:12:09.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.463+0000 7f0eeed9d700 1 --2- 172.21.15.96:0/121244930 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ef0103360 0x7f0ef019bbe0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0ee000b580 tx=0x7f0ee00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:09.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.463+0000 7f0ed7fff700 1 -- 172.21.15.96:0/121244930 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee0027070 con 0x7f0ef0103360 2024-04-25T14:12:09.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.463+0000 7f0ef5236700 1 -- 172.21.15.96:0/121244930 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ef01a71e0 con 0x7f0ef0103360 2024-04-25T14:12:09.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.463+0000 7f0ed7fff700 1 -- 172.21.15.96:0/121244930 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ee00040a0 con 0x7f0ef0103360 2024-04-25T14:12:09.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.463+0000 7f0ed7fff700 1 -- 172.21.15.96:0/121244930 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee000f460 con 0x7f0ef0103360 2024-04-25T14:12:09.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.463+0000 7f0ef5236700 1 -- 172.21.15.96:0/121244930 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ef01a7630 con 0x7f0ef0103360 2024-04-25T14:12:09.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.464+0000 7f0ef5236700 1 -- 172.21.15.96:0/121244930 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0ef0063da0 con 0x7f0ef0103360 2024-04-25T14:12:09.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.466+0000 7f0ed7fff700 1 -- 172.21.15.96:0/121244930 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f0ee002a070 con 0x7f0ef0103360 2024-04-25T14:12:09.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.467+0000 7f0ed7fff700 1 --2- 172.21.15.96:0/121244930 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f0ed80afc10 0x7f0ed80b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:09.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.467+0000 7f0eee59c700 1 --2- 172.21.15.96:0/121244930 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f0ed80afc10 0x7f0ed80b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:09.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.467+0000 7f0ed7fff700 1 -- 172.21.15.96:0/121244930 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7f0ee00e0370 con 0x7f0ef0103360 2024-04-25T14:12:09.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.467+0000 7f0ed7fff700 1 --2- 172.21.15.96:0/121244930 >> [v2:172.21.15.96:6818/96197407,v1:172.21.15.96:6819/96197407] conn(0x7f0ed80b9440 0x7f0ed80bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:09.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.467+0000 7f0ed7fff700 1 -- 172.21.15.96:0/121244930 --> [v2:172.21.15.96:6818/96197407,v1:172.21.15.96:6819/96197407] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0ed80bbf60 con 0x7f0ed80b9440 2024-04-25T14:12:09.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.467+0000 7f0ed7fff700 1 -- 172.21.15.96:0/121244930 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f0ee00e0850 con 0x7f0ef0103360 2024-04-25T14:12:09.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.467+0000 7f0eef59e700 1 --2- 172.21.15.96:0/121244930 >> [v2:172.21.15.96:6818/96197407,v1:172.21.15.96:6819/96197407] conn(0x7f0ed80b9440 0x7f0ed80bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:09.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.467+0000 7f0eee59c700 1 --2- 172.21.15.96:0/121244930 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f0ed80afc10 0x7f0ed80b2120 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0ef01064d0 tx=0x7f0ee4009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:09.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.467+0000 7f0eef59e700 1 --2- 172.21.15.96:0/121244930 >> [v2:172.21.15.96:6818/96197407,v1:172.21.15.96:6819/96197407] conn(0x7f0ed80b9440 0x7f0ed80bb890 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:09.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.468+0000 7f0ed7fff700 1 -- 172.21.15.96:0/121244930 <== osd.6 v2:172.21.15.96:6818/96197407 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0ed80bbf60 con 0x7f0ed80b9440 2024-04-25T14:12:09.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.488+0000 7f0ef5236700 1 -- 172.21.15.96:0/121244930 --> [v2:172.21.15.96:6818/96197407,v1:172.21.15.96:6819/96197407] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0ef00641a0 con 0x7f0ed80b9440 2024-04-25T14:12:09.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.488+0000 7f0ed7fff700 1 -- 172.21.15.96:0/121244930 <== osd.6 v2:172.21.15.96:6818/96197407 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0ef00641a0 con 0x7f0ed80b9440 2024-04-25T14:12:09.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.488+0000 7f0ef5236700 1 -- 172.21.15.96:0/121244930 >> [v2:172.21.15.96:6818/96197407,v1:172.21.15.96:6819/96197407] conn(0x7f0ed80b9440 msgr2=0x7f0ed80bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:09.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.488+0000 7f0ef5236700 1 --2- 172.21.15.96:0/121244930 >> [v2:172.21.15.96:6818/96197407,v1:172.21.15.96:6819/96197407] conn(0x7f0ed80b9440 0x7f0ed80bb890 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:09.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.489+0000 7f0ef5236700 1 -- 172.21.15.96:0/121244930 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f0ed80afc10 msgr2=0x7f0ed80b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:09.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.489+0000 7f0ef5236700 1 --2- 172.21.15.96:0/121244930 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f0ed80afc10 0x7f0ed80b2120 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0ef01064d0 tx=0x7f0ee4009190 comp rx=0 tx=0).stop 2024-04-25T14:12:09.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.489+0000 7f0ef5236700 1 -- 172.21.15.96:0/121244930 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ef0103360 msgr2=0x7f0ef019bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:09.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.489+0000 7f0ef5236700 1 --2- 172.21.15.96:0/121244930 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ef0103360 0x7f0ef019bbe0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0ee000b580 tx=0x7f0ee00047b0 comp rx=0 tx=0).stop 2024-04-25T14:12:09.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.489+0000 7f0eef59e700 1 -- 172.21.15.96:0/121244930 reap_dead start 2024-04-25T14:12:09.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.489+0000 7f0ef5236700 1 -- 172.21.15.96:0/121244930 shutdown_connections 2024-04-25T14:12:09.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.489+0000 7f0ef5236700 1 -- 172.21.15.96:0/121244930 >> 172.21.15.96:0/121244930 conn(0x7f0ef00fec90 msgr2=0x7f0ef0101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:09.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.489+0000 7f0ef5236700 1 -- 172.21.15.96:0/121244930 shutdown_connections 2024-04-25T14:12:09.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:09.489+0000 7f0ef5236700 1 -- 172.21.15.96:0/121244930 wait complete. 2024-04-25T14:12:10.466 INFO:teuthology.orchestra.run.smithi096.stdout:55834574921 2024-04-25T14:12:10.466 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-25T14:12:10.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:10 smithi096 ceph-mon[29207]: pgmap v300: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T14:12:10.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.788+0000 7fb519c2b700 1 -- 172.21.15.96:0/1698804408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb514108680 msgr2=0x7fb51410ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:10.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.788+0000 7fb519c2b700 1 --2- 172.21.15.96:0/1698804408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb514108680 0x7fb51410ffc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb50c00b0a0 tx=0x7fb50c00b3b0 comp rx=0 tx=0).stop 2024-04-25T14:12:10.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.788+0000 7fb519c2b700 1 -- 172.21.15.96:0/1698804408 shutdown_connections 2024-04-25T14:12:10.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.788+0000 7fb519c2b700 1 --2- 172.21.15.96:0/1698804408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb514108680 0x7fb51410ffc0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:10.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.788+0000 7fb519c2b700 1 --2- 172.21.15.96:0/1698804408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb514103a80 0x7fb514103e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:10.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.788+0000 7fb519c2b700 1 --2- 172.21.15.96:0/1698804408 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb514107d30 0x7fb514108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:10.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.788+0000 7fb519c2b700 1 -- 172.21.15.96:0/1698804408 >> 172.21.15.96:0/1698804408 conn(0x7fb5140fe3f0 msgr2=0x7fb514100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:10.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.788+0000 7fb519c2b700 1 -- 172.21.15.96:0/1698804408 shutdown_connections 2024-04-25T14:12:10.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.788+0000 7fb519c2b700 1 -- 172.21.15.96:0/1698804408 wait complete. 2024-04-25T14:12:10.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.789+0000 7fb519c2b700 1 Processor -- start 2024-04-25T14:12:10.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.789+0000 7fb519c2b700 1 -- start start 2024-04-25T14:12:10.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.789+0000 7fb519c2b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb514103a80 0x7fb51419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:10.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.789+0000 7fb519c2b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb514107d30 0x7fb51419fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:10.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.789+0000 7fb519c2b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb514108680 0x7fb514199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:10.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.789+0000 7fb519c2b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb514113280 con 0x7fb514107d30 2024-04-25T14:12:10.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.789+0000 7fb519c2b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb514113100 con 0x7fb514103a80 2024-04-25T14:12:10.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.789+0000 7fb519c2b700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb514113400 con 0x7fb514108680 2024-04-25T14:12:10.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.789+0000 7fb50bfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb514107d30 0x7fb51419fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:10.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.789+0000 7fb50bfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb514107d30 0x7fb51419fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53110/0 (socket says 172.21.15.96:53110) 2024-04-25T14:12:10.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.789+0000 7fb50bfff700 1 -- 172.21.15.96:0/3215061040 learned_addr learned my addr 172.21.15.96:0/3215061040 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:12:10.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.789+0000 7fb5137fe700 1 --2- 172.21.15.96:0/3215061040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb514103a80 0x7fb51419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:10.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.789+0000 7fb50bfff700 1 -- 172.21.15.96:0/3215061040 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb514108680 msgr2=0x7fb514199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:10.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.789+0000 7fb50bfff700 1 --2- 172.21.15.96:0/3215061040 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb514108680 0x7fb514199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:10.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.789+0000 7fb50bfff700 1 -- 172.21.15.96:0/3215061040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb514103a80 msgr2=0x7fb51419f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:10.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.789+0000 7fb50bfff700 1 --2- 172.21.15.96:0/3215061040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb514103a80 0x7fb51419f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:10.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.789+0000 7fb50bfff700 1 -- 172.21.15.96:0/3215061040 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb514199ff0 con 0x7fb514107d30 2024-04-25T14:12:10.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.789+0000 7fb5137fe700 1 --2- 172.21.15.96:0/3215061040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb514103a80 0x7fb51419f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:12:10.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.790+0000 7fb50bfff700 1 --2- 172.21.15.96:0/3215061040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb514107d30 0x7fb51419fc60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc00d950 tx=0x7fb4fc00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:10.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.790+0000 7fb5117fa700 1 -- 172.21.15.96:0/3215061040 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc017070 con 0x7fb514107d30 2024-04-25T14:12:10.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.790+0000 7fb5117fa700 1 -- 172.21.15.96:0/3215061040 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4fc00de30 con 0x7fb514107d30 2024-04-25T14:12:10.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.790+0000 7fb519c2b700 1 -- 172.21.15.96:0/3215061040 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb51419a2e0 con 0x7fb514107d30 2024-04-25T14:12:10.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.790+0000 7fb5117fa700 1 -- 172.21.15.96:0/3215061040 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc00f3c0 con 0x7fb514107d30 2024-04-25T14:12:10.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.790+0000 7fb519c2b700 1 -- 172.21.15.96:0/3215061040 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5141a65b0 con 0x7fb514107d30 2024-04-25T14:12:10.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.791+0000 7fb519c2b700 1 -- 172.21.15.96:0/3215061040 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb514063e20 con 0x7fb514107d30 2024-04-25T14:12:10.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.792+0000 7fb5117fa700 1 -- 172.21.15.96:0/3215061040 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fb4fc01a070 con 0x7fb514107d30 2024-04-25T14:12:10.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.793+0000 7fb5117fa700 1 --2- 172.21.15.96:0/3215061040 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb5000afc90 0x7fb5000b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:10.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.793+0000 7fb5137fe700 1 --2- 172.21.15.96:0/3215061040 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb5000afc90 0x7fb5000b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:10.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.793+0000 7fb5117fa700 1 -- 172.21.15.96:0/3215061040 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7fb4fc0de960 con 0x7fb514107d30 2024-04-25T14:12:10.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.793+0000 7fb5117fa700 1 --2- 172.21.15.96:0/3215061040 >> [v2:172.21.15.151:6816/2549822185,v1:172.21.15.151:6817/2549822185] conn(0x7fb5000b94c0 0x7fb5000bb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:10.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.793+0000 7fb5117fa700 1 -- 172.21.15.96:0/3215061040 --> [v2:172.21.15.151:6816/2549822185,v1:172.21.15.151:6817/2549822185] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb4fc001d30 con 0x7fb5000b94c0 2024-04-25T14:12:10.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.793+0000 7fb5117fa700 1 -- 172.21.15.96:0/3215061040 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fb4fc0dee40 con 0x7fb514107d30 2024-04-25T14:12:10.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.794+0000 7fb5137fe700 1 --2- 172.21.15.96:0/3215061040 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb5000afc90 0x7fb5000b21a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb504005950 tx=0x7fb504005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:10.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.794+0000 7fb513fff700 1 --2- 172.21.15.96:0/3215061040 >> [v2:172.21.15.151:6816/2549822185,v1:172.21.15.151:6817/2549822185] conn(0x7fb5000b94c0 0x7fb5000bb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:10.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.794+0000 7fb513fff700 1 --2- 172.21.15.96:0/3215061040 >> [v2:172.21.15.151:6816/2549822185,v1:172.21.15.151:6817/2549822185] conn(0x7fb5000b94c0 0x7fb5000bb910 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:10.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.795+0000 7fb5117fa700 1 -- 172.21.15.96:0/3215061040 <== osd.8 v2:172.21.15.151:6816/2549822185 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb4fc001d30 con 0x7fb5000b94c0 2024-04-25T14:12:10.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.811+0000 7fb519c2b700 1 -- 172.21.15.96:0/3215061040 --> [v2:172.21.15.151:6816/2549822185,v1:172.21.15.151:6817/2549822185] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb514064220 con 0x7fb5000b94c0 2024-04-25T14:12:10.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.812+0000 7fb5117fa700 1 -- 172.21.15.96:0/3215061040 <== osd.8 v2:172.21.15.151:6816/2549822185 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb514064220 con 0x7fb5000b94c0 2024-04-25T14:12:10.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.812+0000 7fb519c2b700 1 -- 172.21.15.96:0/3215061040 >> [v2:172.21.15.151:6816/2549822185,v1:172.21.15.151:6817/2549822185] conn(0x7fb5000b94c0 msgr2=0x7fb5000bb910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:10.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.812+0000 7fb519c2b700 1 --2- 172.21.15.96:0/3215061040 >> [v2:172.21.15.151:6816/2549822185,v1:172.21.15.151:6817/2549822185] conn(0x7fb5000b94c0 0x7fb5000bb910 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:10.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.812+0000 7fb519c2b700 1 -- 172.21.15.96:0/3215061040 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb5000afc90 msgr2=0x7fb5000b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:10.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.812+0000 7fb519c2b700 1 --2- 172.21.15.96:0/3215061040 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb5000afc90 0x7fb5000b21a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb504005950 tx=0x7fb504005f50 comp rx=0 tx=0).stop 2024-04-25T14:12:10.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.812+0000 7fb519c2b700 1 -- 172.21.15.96:0/3215061040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb514107d30 msgr2=0x7fb51419fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:10.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.812+0000 7fb519c2b700 1 --2- 172.21.15.96:0/3215061040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb514107d30 0x7fb51419fc60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc00d950 tx=0x7fb4fc00dd10 comp rx=0 tx=0).stop 2024-04-25T14:12:10.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.812+0000 7fb513fff700 1 -- 172.21.15.96:0/3215061040 reap_dead start 2024-04-25T14:12:10.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.813+0000 7fb519c2b700 1 -- 172.21.15.96:0/3215061040 shutdown_connections 2024-04-25T14:12:10.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.813+0000 7fb519c2b700 1 -- 172.21.15.96:0/3215061040 >> 172.21.15.96:0/3215061040 conn(0x7fb5140fe3f0 msgr2=0x7fb514105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:10.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.813+0000 7fb519c2b700 1 -- 172.21.15.96:0/3215061040 shutdown_connections 2024-04-25T14:12:10.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.813+0000 7fb519c2b700 1 -- 172.21.15.96:0/3215061040 wait complete. 2024-04-25T14:12:10.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.681+0000 7fe9cc0dc700 1 -- 172.21.15.96:0/2511368524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9c41044b0 msgr2=0x7fe9c41048c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:10.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.681+0000 7fe9cc0dc700 1 --2- 172.21.15.96:0/2511368524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9c41044b0 0x7fe9c41048c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe9c00099c0 tx=0x7fe9c0009cd0 comp rx=0 tx=0).stop 2024-04-25T14:12:10.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.681+0000 7fe9cc0dc700 1 -- 172.21.15.96:0/2511368524 shutdown_connections 2024-04-25T14:12:10.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.681+0000 7fe9cc0dc700 1 --2- 172.21.15.96:0/2511368524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9c4109730 0x7fe9c4111090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:10.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.681+0000 7fe9cc0dc700 1 --2- 172.21.15.96:0/2511368524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9c41044b0 0x7fe9c41048c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:10.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.681+0000 7fe9cc0dc700 1 --2- 172.21.15.96:0/2511368524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9c4108de0 0x7fe9c41091f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:10.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.681+0000 7fe9cc0dc700 1 -- 172.21.15.96:0/2511368524 >> 172.21.15.96:0/2511368524 conn(0x7fe9c40ffdc0 msgr2=0x7fe9c4102210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:10.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.681+0000 7fe9cc0dc700 1 -- 172.21.15.96:0/2511368524 shutdown_connections 2024-04-25T14:12:10.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.681+0000 7fe9cc0dc700 1 -- 172.21.15.96:0/2511368524 wait complete. 2024-04-25T14:12:10.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.682+0000 7fe9cc0dc700 1 Processor -- start 2024-04-25T14:12:10.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.682+0000 7fe9cc0dc700 1 -- start start 2024-04-25T14:12:10.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.682+0000 7fe9cc0dc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9c41044b0 0x7fe9c41a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:10.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.682+0000 7fe9cc0dc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9c4108de0 0x7fe9c41a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:10.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.682+0000 7fe9cc0dc700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9c4109730 0x7fe9c419aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:10.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.682+0000 7fe9cc0dc700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c4114360 con 0x7fe9c41044b0 2024-04-25T14:12:10.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.682+0000 7fe9cc0dc700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c41141e0 con 0x7fe9c4108de0 2024-04-25T14:12:10.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.682+0000 7fe9cc0dc700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c41144e0 con 0x7fe9c4109730 2024-04-25T14:12:10.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.682+0000 7fe9c9e78700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9c41044b0 0x7fe9c41a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:10.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.682+0000 7fe9c9e78700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9c41044b0 0x7fe9c41a08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53092/0 (socket says 172.21.15.96:53092) 2024-04-25T14:12:10.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.682+0000 7fe9c9e78700 1 -- 172.21.15.96:0/892980596 learned_addr learned my addr 172.21.15.96:0/892980596 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:12:10.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.682+0000 7fe9c9677700 1 --2- 172.21.15.96:0/892980596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9c4108de0 0x7fe9c41a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:10.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.682+0000 7fe9ca679700 1 --2- 172.21.15.96:0/892980596 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9c4109730 0x7fe9c419aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:10.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.683+0000 7fe9c9e78700 1 -- 172.21.15.96:0/892980596 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9c4109730 msgr2=0x7fe9c419aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:10.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.683+0000 7fe9c9e78700 1 --2- 172.21.15.96:0/892980596 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9c4109730 0x7fe9c419aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:10.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.683+0000 7fe9c9e78700 1 -- 172.21.15.96:0/892980596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9c4108de0 msgr2=0x7fe9c41a0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:10.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.683+0000 7fe9c9e78700 1 --2- 172.21.15.96:0/892980596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9c4108de0 0x7fe9c41a0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:10.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.683+0000 7fe9c9e78700 1 -- 172.21.15.96:0/892980596 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9c419b170 con 0x7fe9c41044b0 2024-04-25T14:12:10.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.683+0000 7fe9c9e78700 1 --2- 172.21.15.96:0/892980596 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9c41044b0 0x7fe9c41a08a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe9c000b580 tx=0x7fe9c00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:10.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.683+0000 7fe9baffd700 1 -- 172.21.15.96:0/892980596 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0027070 con 0x7fe9c41044b0 2024-04-25T14:12:10.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.683+0000 7fe9cc0dc700 1 -- 172.21.15.96:0/892980596 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9c419b400 con 0x7fe9c41044b0 2024-04-25T14:12:10.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.683+0000 7fe9baffd700 1 -- 172.21.15.96:0/892980596 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9c0003f20 con 0x7fe9c41044b0 2024-04-25T14:12:10.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.683+0000 7fe9baffd700 1 -- 172.21.15.96:0/892980596 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0022d10 con 0x7fe9c41044b0 2024-04-25T14:12:10.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.683+0000 7fe9cc0dc700 1 -- 172.21.15.96:0/892980596 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9c41a7730 con 0x7fe9c41044b0 2024-04-25T14:12:10.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.684+0000 7fe9cc0dc700 1 -- 172.21.15.96:0/892980596 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe9c4063e20 con 0x7fe9c41044b0 2024-04-25T14:12:10.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.686+0000 7fe9baffd700 1 -- 172.21.15.96:0/892980596 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fe9c0022470 con 0x7fe9c41044b0 2024-04-25T14:12:10.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.687+0000 7fe9baffd700 1 --2- 172.21.15.96:0/892980596 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fe9b00afc10 0x7fe9b00b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:10.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.687+0000 7fe9c9677700 1 --2- 172.21.15.96:0/892980596 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fe9b00afc10 0x7fe9b00b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:10.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.687+0000 7fe9baffd700 1 -- 172.21.15.96:0/892980596 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7fe9c00dfed0 con 0x7fe9c41044b0 2024-04-25T14:12:10.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.687+0000 7fe9baffd700 1 --2- 172.21.15.96:0/892980596 >> [v2:172.21.15.96:6826/1099399643,v1:172.21.15.96:6827/1099399643] conn(0x7fe9b00b9440 0x7fe9b00bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:10.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.687+0000 7fe9baffd700 1 -- 172.21.15.96:0/892980596 --> [v2:172.21.15.96:6826/1099399643,v1:172.21.15.96:6827/1099399643] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe9c00043d0 con 0x7fe9b00b9440 2024-04-25T14:12:10.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.687+0000 7fe9baffd700 1 -- 172.21.15.96:0/892980596 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fe9c00e0280 con 0x7fe9c41044b0 2024-04-25T14:12:10.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.687+0000 7fe9ca679700 1 --2- 172.21.15.96:0/892980596 >> [v2:172.21.15.96:6826/1099399643,v1:172.21.15.96:6827/1099399643] conn(0x7fe9b00b9440 0x7fe9b00bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:10.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.687+0000 7fe9c9677700 1 --2- 172.21.15.96:0/892980596 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fe9b00afc10 0x7fe9b00b2120 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4107520 tx=0x7fe9b400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:10.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.688+0000 7fe9ca679700 1 --2- 172.21.15.96:0/892980596 >> [v2:172.21.15.96:6826/1099399643,v1:172.21.15.96:6827/1099399643] conn(0x7fe9b00b9440 0x7fe9b00bb890 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:10.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.688+0000 7fe9baffd700 1 -- 172.21.15.96:0/892980596 <== osd.9 v2:172.21.15.96:6826/1099399643 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe9c00043d0 con 0x7fe9b00b9440 2024-04-25T14:12:10.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.704+0000 7fe9cc0dc700 1 -- 172.21.15.96:0/892980596 --> [v2:172.21.15.96:6826/1099399643,v1:172.21.15.96:6827/1099399643] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe9c4064250 con 0x7fe9b00b9440 2024-04-25T14:12:10.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.705+0000 7fe9baffd700 1 -- 172.21.15.96:0/892980596 <== osd.9 v2:172.21.15.96:6826/1099399643 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe9c4064250 con 0x7fe9b00b9440 2024-04-25T14:12:10.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.705+0000 7fe9cc0dc700 1 -- 172.21.15.96:0/892980596 >> [v2:172.21.15.96:6826/1099399643,v1:172.21.15.96:6827/1099399643] conn(0x7fe9b00b9440 msgr2=0x7fe9b00bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:10.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.705+0000 7fe9cc0dc700 1 --2- 172.21.15.96:0/892980596 >> [v2:172.21.15.96:6826/1099399643,v1:172.21.15.96:6827/1099399643] conn(0x7fe9b00b9440 0x7fe9b00bb890 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:10.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.705+0000 7fe9cc0dc700 1 -- 172.21.15.96:0/892980596 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fe9b00afc10 msgr2=0x7fe9b00b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:10.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.705+0000 7fe9cc0dc700 1 --2- 172.21.15.96:0/892980596 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fe9b00afc10 0x7fe9b00b2120 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4107520 tx=0x7fe9b400b410 comp rx=0 tx=0).stop 2024-04-25T14:12:10.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.705+0000 7fe9cc0dc700 1 -- 172.21.15.96:0/892980596 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9c41044b0 msgr2=0x7fe9c41a08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:10.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.705+0000 7fe9cc0dc700 1 --2- 172.21.15.96:0/892980596 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9c41044b0 0x7fe9c41a08a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe9c000b580 tx=0x7fe9c00047b0 comp rx=0 tx=0).stop 2024-04-25T14:12:10.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.705+0000 7fe9ca679700 1 -- 172.21.15.96:0/892980596 reap_dead start 2024-04-25T14:12:10.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.705+0000 7fe9cc0dc700 1 -- 172.21.15.96:0/892980596 shutdown_connections 2024-04-25T14:12:10.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.705+0000 7fe9cc0dc700 1 -- 172.21.15.96:0/892980596 >> 172.21.15.96:0/892980596 conn(0x7fe9c40ffdc0 msgr2=0x7fe9c41080c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:10.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.705+0000 7fe9cc0dc700 1 -- 172.21.15.96:0/892980596 shutdown_connections 2024-04-25T14:12:10.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:10.705+0000 7fe9cc0dc700 1 -- 172.21.15.96:0/892980596 wait complete. 2024-04-25T14:12:10.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:10 smithi097 ceph-mon[32720]: pgmap v300: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T14:12:10.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:10 smithi151 ceph-mon[32990]: pgmap v300: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T14:12:11.594 INFO:teuthology.orchestra.run.smithi096.stdout:184683593776 2024-04-25T14:12:11.594 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-25T14:12:12.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.483+0000 7fafcd59a700 1 -- 172.21.15.96:0/3552072920 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc8009480 con 0x7fafd01031b0 2024-04-25T14:12:12.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.483+0000 7fafd4d40700 1 -- 172.21.15.96:0/3552072920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafd01031b0 msgr2=0x7fafd010f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:12.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.483+0000 7fafd4d40700 1 --2- 172.21.15.96:0/3552072920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafd01031b0 0x7fafd010f730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fafc800b0a0 tx=0x7fafc800b3b0 comp rx=0 tx=0).stop 2024-04-25T14:12:12.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.484+0000 7fafd4d40700 1 -- 172.21.15.96:0/3552072920 shutdown_connections 2024-04-25T14:12:12.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.484+0000 7fafd4d40700 1 --2- 172.21.15.96:0/3552072920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafd01031b0 0x7fafd010f730 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:12.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.484+0000 7fafd4d40700 1 --2- 172.21.15.96:0/3552072920 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fafd0108890 0x7fafd0108ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:12.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.484+0000 7fafd4d40700 1 --2- 172.21.15.96:0/3552072920 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fafd0102860 0x7fafd0102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:12.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.484+0000 7fafd4d40700 1 -- 172.21.15.96:0/3552072920 >> 172.21.15.96:0/3552072920 conn(0x7fafd00fe3f0 msgr2=0x7fafd0100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:12.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.484+0000 7fafd4d40700 1 -- 172.21.15.96:0/3552072920 shutdown_connections 2024-04-25T14:12:12.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.484+0000 7fafd4d40700 1 -- 172.21.15.96:0/3552072920 wait complete. 2024-04-25T14:12:12.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.485+0000 7fafd4d40700 1 Processor -- start 2024-04-25T14:12:12.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.485+0000 7fafd4d40700 1 -- start start 2024-04-25T14:12:12.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.485+0000 7fafd4d40700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafd0102860 0x7fafd01a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:12.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.485+0000 7fafd4d40700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fafd01031b0 0x7fafd01a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:12.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.485+0000 7fafd4d40700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fafd0108890 0x7fafd019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:12.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.485+0000 7fafd4d40700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fafd0112230 con 0x7fafd0108890 2024-04-25T14:12:12.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.485+0000 7fafd4d40700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fafd01120b0 con 0x7fafd0102860 2024-04-25T14:12:12.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.485+0000 7fafd4d40700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fafd01123b0 con 0x7fafd01031b0 2024-04-25T14:12:12.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.485+0000 7fafced9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fafd0108890 0x7fafd019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:12.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.485+0000 7fafced9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fafd0108890 0x7fafd019aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34894/0 (socket says 172.21.15.96:34894) 2024-04-25T14:12:12.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.485+0000 7fafced9d700 1 -- 172.21.15.96:0/4101772200 learned_addr learned my addr 172.21.15.96:0/4101772200 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:12:12.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.485+0000 7fafce59c700 1 --2- 172.21.15.96:0/4101772200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafd0102860 0x7fafd01a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:12.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.485+0000 7fafcdd9b700 1 --2- 172.21.15.96:0/4101772200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fafd01031b0 0x7fafd01a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:12.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.485+0000 7fafced9d700 1 -- 172.21.15.96:0/4101772200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fafd01031b0 msgr2=0x7fafd01a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:12.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.485+0000 7fafced9d700 1 --2- 172.21.15.96:0/4101772200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fafd01031b0 0x7fafd01a0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:12.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.485+0000 7fafced9d700 1 -- 172.21.15.96:0/4101772200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafd0102860 msgr2=0x7fafd01a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:12.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.485+0000 7fafced9d700 1 --2- 172.21.15.96:0/4101772200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafd0102860 0x7fafd01a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:12.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.485+0000 7fafced9d700 1 -- 172.21.15.96:0/4101772200 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd019b140 con 0x7fafd0108890 2024-04-25T14:12:12.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.486+0000 7fafce59c700 1 --2- 172.21.15.96:0/4101772200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafd0102860 0x7fafd01a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:12:12.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.486+0000 7fafced9d700 1 --2- 172.21.15.96:0/4101772200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fafd0108890 0x7fafd019aad0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fafc80062a0 tx=0x7fafc8004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:12.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.486+0000 7fafbf7fe700 1 -- 172.21.15.96:0/4101772200 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc8021070 con 0x7fafd0108890 2024-04-25T14:12:12.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.486+0000 7fafd4d40700 1 -- 172.21.15.96:0/4101772200 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafd019b3d0 con 0x7fafd0108890 2024-04-25T14:12:12.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.486+0000 7fafbf7fe700 1 -- 172.21.15.96:0/4101772200 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafc8003c00 con 0x7fafd0108890 2024-04-25T14:12:12.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.486+0000 7fafd4d40700 1 -- 172.21.15.96:0/4101772200 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafd01a7740 con 0x7fafd0108890 2024-04-25T14:12:12.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.487+0000 7fafbf7fe700 1 -- 172.21.15.96:0/4101772200 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc8004830 con 0x7fafd0108890 2024-04-25T14:12:12.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.487+0000 7fafd4d40700 1 -- 172.21.15.96:0/4101772200 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fafb0001030 con 0x7fafd0108890 2024-04-25T14:12:12.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.489+0000 7fafbf7fe700 1 -- 172.21.15.96:0/4101772200 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fafc8007a00 con 0x7fafd0108890 2024-04-25T14:12:12.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.489+0000 7fafbf7fe700 1 --2- 172.21.15.96:0/4101772200 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fafb80afc10 0x7fafb80b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:12.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.489+0000 7fafce59c700 1 --2- 172.21.15.96:0/4101772200 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fafb80afc10 0x7fafb80b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:12.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.489+0000 7fafbf7fe700 1 -- 172.21.15.96:0/4101772200 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7fafc800f020 con 0x7fafd0108890 2024-04-25T14:12:12.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.490+0000 7fafbf7fe700 1 --2- 172.21.15.96:0/4101772200 >> [v2:172.21.15.96:6810/2274158748,v1:172.21.15.96:6811/2274158748] conn(0x7fafb80b9440 0x7fafb80bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:12.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.490+0000 7fafbf7fe700 1 -- 172.21.15.96:0/4101772200 --> [v2:172.21.15.96:6810/2274158748,v1:172.21.15.96:6811/2274158748] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fafc8030e50 con 0x7fafb80b9440 2024-04-25T14:12:12.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.490+0000 7fafbf7fe700 1 -- 172.21.15.96:0/4101772200 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fafc80e0a60 con 0x7fafd0108890 2024-04-25T14:12:12.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.490+0000 7fafcdd9b700 1 --2- 172.21.15.96:0/4101772200 >> [v2:172.21.15.96:6810/2274158748,v1:172.21.15.96:6811/2274158748] conn(0x7fafb80b9440 0x7fafb80bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:12.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.490+0000 7fafce59c700 1 --2- 172.21.15.96:0/4101772200 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fafb80afc10 0x7fafb80b2120 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fafc0005ea0 tx=0x7fafc0005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:12.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.490+0000 7fafcdd9b700 1 --2- 172.21.15.96:0/4101772200 >> [v2:172.21.15.96:6810/2274158748,v1:172.21.15.96:6811/2274158748] conn(0x7fafb80b9440 0x7fafb80bb890 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:12.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.491+0000 7fafbf7fe700 1 -- 172.21.15.96:0/4101772200 <== osd.3 v2:172.21.15.96:6810/2274158748 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fafc8030e50 con 0x7fafb80b9440 2024-04-25T14:12:12.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.508+0000 7fafd4d40700 1 -- 172.21.15.96:0/4101772200 --> [v2:172.21.15.96:6810/2274158748,v1:172.21.15.96:6811/2274158748] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fafb0002e60 con 0x7fafb80b9440 2024-04-25T14:12:12.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.509+0000 7fafbf7fe700 1 -- 172.21.15.96:0/4101772200 <== osd.3 v2:172.21.15.96:6810/2274158748 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fafb0002e60 con 0x7fafb80b9440 2024-04-25T14:12:12.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.509+0000 7fafd4d40700 1 -- 172.21.15.96:0/4101772200 >> [v2:172.21.15.96:6810/2274158748,v1:172.21.15.96:6811/2274158748] conn(0x7fafb80b9440 msgr2=0x7fafb80bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:12.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.509+0000 7fafd4d40700 1 --2- 172.21.15.96:0/4101772200 >> [v2:172.21.15.96:6810/2274158748,v1:172.21.15.96:6811/2274158748] conn(0x7fafb80b9440 0x7fafb80bb890 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:12.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.509+0000 7fafd4d40700 1 -- 172.21.15.96:0/4101772200 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fafb80afc10 msgr2=0x7fafb80b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:12.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.509+0000 7fafd4d40700 1 --2- 172.21.15.96:0/4101772200 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fafb80afc10 0x7fafb80b2120 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fafc0005ea0 tx=0x7fafc0005e30 comp rx=0 tx=0).stop 2024-04-25T14:12:12.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.509+0000 7fafd4d40700 1 -- 172.21.15.96:0/4101772200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fafd0108890 msgr2=0x7fafd019aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:12.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.509+0000 7fafd4d40700 1 --2- 172.21.15.96:0/4101772200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fafd0108890 0x7fafd019aad0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fafc80062a0 tx=0x7fafc8004d30 comp rx=0 tx=0).stop 2024-04-25T14:12:12.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.509+0000 7fafced9d700 1 -- 172.21.15.96:0/4101772200 reap_dead start 2024-04-25T14:12:12.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.509+0000 7fafd4d40700 1 -- 172.21.15.96:0/4101772200 shutdown_connections 2024-04-25T14:12:12.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.509+0000 7fafd4d40700 1 -- 172.21.15.96:0/4101772200 >> 172.21.15.96:0/4101772200 conn(0x7fafd00fe3f0 msgr2=0x7fafd0106ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:12.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.509+0000 7fafd4d40700 1 -- 172.21.15.96:0/4101772200 shutdown_connections 2024-04-25T14:12:12.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.509+0000 7fafd4d40700 1 -- 172.21.15.96:0/4101772200 wait complete. 2024-04-25T14:12:12.690 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:12 smithi096 ceph-mon[29207]: pgmap v301: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T14:12:12.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.704+0000 7fa5efb7b700 1 -- 172.21.15.96:0/1357720712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5e81088c0 msgr2=0x7fa5e8108cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:12.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.704+0000 7fa5efb7b700 1 --2- 172.21.15.96:0/1357720712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5e81088c0 0x7fa5e8108cd0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4009990 tx=0x7fa5e4009ca0 comp rx=0 tx=0).stop 2024-04-25T14:12:12.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.704+0000 7fa5efb7b700 1 -- 172.21.15.96:0/1357720712 shutdown_connections 2024-04-25T14:12:12.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.704+0000 7fa5efb7b700 1 --2- 172.21.15.96:0/1357720712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5e81031e0 0x7fa5e810f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:12.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.704+0000 7fa5efb7b700 1 --2- 172.21.15.96:0/1357720712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5e81088c0 0x7fa5e8108cd0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:12.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.704+0000 7fa5efb7b700 1 --2- 172.21.15.96:0/1357720712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5e8102890 0x7fa5e8102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:12.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.704+0000 7fa5efb7b700 1 -- 172.21.15.96:0/1357720712 >> 172.21.15.96:0/1357720712 conn(0x7fa5e80fe5c0 msgr2=0x7fa5e81009e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:12.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.704+0000 7fa5efb7b700 1 -- 172.21.15.96:0/1357720712 shutdown_connections 2024-04-25T14:12:12.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.704+0000 7fa5efb7b700 1 -- 172.21.15.96:0/1357720712 wait complete. 2024-04-25T14:12:12.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.705+0000 7fa5efb7b700 1 Processor -- start 2024-04-25T14:12:12.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.705+0000 7fa5efb7b700 1 -- start start 2024-04-25T14:12:12.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.705+0000 7fa5efb7b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5e8102890 0x7fa5e819bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:12.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.705+0000 7fa5efb7b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5e81031e0 0x7fa5e819c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:12.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.705+0000 7fa5efb7b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5e81088c0 0x7fa5e819c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:12.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.705+0000 7fa5efb7b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e81129c0 con 0x7fa5e81031e0 2024-04-25T14:12:12.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.705+0000 7fa5efb7b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e8112840 con 0x7fa5e81088c0 2024-04-25T14:12:12.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.705+0000 7fa5efb7b700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e8112b40 con 0x7fa5e8102890 2024-04-25T14:12:12.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.705+0000 7fa5ed116700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5e81031e0 0x7fa5e819c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:12.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.705+0000 7fa5ed116700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5e81031e0 0x7fa5e819c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34922/0 (socket says 172.21.15.96:34922) 2024-04-25T14:12:12.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.705+0000 7fa5ed116700 1 -- 172.21.15.96:0/1567601585 learned_addr learned my addr 172.21.15.96:0/1567601585 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:12:12.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.705+0000 7fa5ed917700 1 --2- 172.21.15.96:0/1567601585 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5e8102890 0x7fa5e819bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:12.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.705+0000 7fa5ee118700 1 --2- 172.21.15.96:0/1567601585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5e81088c0 0x7fa5e819c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:12.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.705+0000 7fa5ed116700 1 -- 172.21.15.96:0/1567601585 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5e8102890 msgr2=0x7fa5e819bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:12.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.705+0000 7fa5ed116700 1 --2- 172.21.15.96:0/1567601585 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5e8102890 0x7fa5e819bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:12.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.705+0000 7fa5ed116700 1 -- 172.21.15.96:0/1567601585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5e81088c0 msgr2=0x7fa5e819c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:12.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.705+0000 7fa5ed116700 1 --2- 172.21.15.96:0/1567601585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5e81088c0 0x7fa5e819c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:12.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.705+0000 7fa5ed116700 1 -- 172.21.15.96:0/1567601585 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5e81999e0 con 0x7fa5e81031e0 2024-04-25T14:12:12.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.706+0000 7fa5ed116700 1 --2- 172.21.15.96:0/1567601585 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5e81031e0 0x7fa5e819c130 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa5d800d980 tx=0x7fa5d800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:12.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.706+0000 7fa5deffd700 1 -- 172.21.15.96:0/1567601585 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d8017070 con 0x7fa5e81031e0 2024-04-25T14:12:12.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.706+0000 7fa5deffd700 1 -- 172.21.15.96:0/1567601585 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5d800fd40 con 0x7fa5e81031e0 2024-04-25T14:12:12.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.706+0000 7fa5efb7b700 1 -- 172.21.15.96:0/1567601585 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5e8199cd0 con 0x7fa5e81031e0 2024-04-25T14:12:12.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.706+0000 7fa5deffd700 1 -- 172.21.15.96:0/1567601585 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d800e3b0 con 0x7fa5e81031e0 2024-04-25T14:12:12.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.706+0000 7fa5efb7b700 1 -- 172.21.15.96:0/1567601585 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5e819a210 con 0x7fa5e81031e0 2024-04-25T14:12:12.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.707+0000 7fa5efb7b700 1 -- 172.21.15.96:0/1567601585 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa5e8107d90 con 0x7fa5e81031e0 2024-04-25T14:12:12.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.709+0000 7fa5deffd700 1 -- 172.21.15.96:0/1567601585 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fa5d8010610 con 0x7fa5e81031e0 2024-04-25T14:12:12.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.709+0000 7fa5deffd700 1 --2- 172.21.15.96:0/1567601585 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa5d40afc10 0x7fa5d40b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:12.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.710+0000 7fa5ed917700 1 --2- 172.21.15.96:0/1567601585 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa5d40afc10 0x7fa5d40b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:12.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.710+0000 7fa5deffd700 1 -- 172.21.15.96:0/1567601585 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7fa5d80dead0 con 0x7fa5e81031e0 2024-04-25T14:12:12.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.710+0000 7fa5deffd700 1 --2- 172.21.15.96:0/1567601585 >> [v2:172.21.15.97:6824/155754520,v1:172.21.15.97:6825/155754520] conn(0x7fa5d40b9440 0x7fa5d40bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:12.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.710+0000 7fa5deffd700 1 -- 172.21.15.96:0/1567601585 --> [v2:172.21.15.97:6824/155754520,v1:172.21.15.97:6825/155754520] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa5d8016ec0 con 0x7fa5d40b9440 2024-04-25T14:12:12.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.710+0000 7fa5deffd700 1 -- 172.21.15.96:0/1567601585 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fa5d80defb0 con 0x7fa5e81031e0 2024-04-25T14:12:12.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.710+0000 7fa5ed917700 1 --2- 172.21.15.96:0/1567601585 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa5d40afc10 0x7fa5d40b2120 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4006010 tx=0x7fa5e4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:12.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.710+0000 7fa5ee118700 1 --2- 172.21.15.96:0/1567601585 >> [v2:172.21.15.97:6824/155754520,v1:172.21.15.97:6825/155754520] conn(0x7fa5d40b9440 0x7fa5d40bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:12.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.711+0000 7fa5ee118700 1 --2- 172.21.15.96:0/1567601585 >> [v2:172.21.15.97:6824/155754520,v1:172.21.15.97:6825/155754520] conn(0x7fa5d40b9440 0x7fa5d40bb890 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:12.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.711+0000 7fa5deffd700 1 -- 172.21.15.96:0/1567601585 <== osd.10 v2:172.21.15.97:6824/155754520 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa5d8016ec0 con 0x7fa5d40b9440 2024-04-25T14:12:12.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.729+0000 7fa5efb7b700 1 -- 172.21.15.96:0/1567601585 --> [v2:172.21.15.97:6824/155754520,v1:172.21.15.97:6825/155754520] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa5e8103fd0 con 0x7fa5d40b9440 2024-04-25T14:12:12.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.729+0000 7fa5deffd700 1 -- 172.21.15.96:0/1567601585 <== osd.10 v2:172.21.15.97:6824/155754520 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa5e8103fd0 con 0x7fa5d40b9440 2024-04-25T14:12:12.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.730+0000 7fa5efb7b700 1 -- 172.21.15.96:0/1567601585 >> [v2:172.21.15.97:6824/155754520,v1:172.21.15.97:6825/155754520] conn(0x7fa5d40b9440 msgr2=0x7fa5d40bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:12.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.730+0000 7fa5efb7b700 1 --2- 172.21.15.96:0/1567601585 >> [v2:172.21.15.97:6824/155754520,v1:172.21.15.97:6825/155754520] conn(0x7fa5d40b9440 0x7fa5d40bb890 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:12.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.730+0000 7fa5efb7b700 1 -- 172.21.15.96:0/1567601585 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa5d40afc10 msgr2=0x7fa5d40b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:12.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.730+0000 7fa5efb7b700 1 --2- 172.21.15.96:0/1567601585 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa5d40afc10 0x7fa5d40b2120 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4006010 tx=0x7fa5e4005960 comp rx=0 tx=0).stop 2024-04-25T14:12:12.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.730+0000 7fa5efb7b700 1 -- 172.21.15.96:0/1567601585 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5e81031e0 msgr2=0x7fa5e819c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:12.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.730+0000 7fa5efb7b700 1 --2- 172.21.15.96:0/1567601585 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5e81031e0 0x7fa5e819c130 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa5d800d980 tx=0x7fa5d800dc90 comp rx=0 tx=0).stop 2024-04-25T14:12:12.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.730+0000 7fa5ee118700 1 -- 172.21.15.96:0/1567601585 reap_dead start 2024-04-25T14:12:12.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.730+0000 7fa5efb7b700 1 -- 172.21.15.96:0/1567601585 shutdown_connections 2024-04-25T14:12:12.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.730+0000 7fa5efb7b700 1 -- 172.21.15.96:0/1567601585 >> 172.21.15.96:0/1567601585 conn(0x7fa5e80fe5c0 msgr2=0x7fa5e81063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:12.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.730+0000 7fa5efb7b700 1 -- 172.21.15.96:0/1567601585 shutdown_connections 2024-04-25T14:12:12.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:12.730+0000 7fa5efb7b700 1 -- 172.21.15.96:0/1567601585 wait complete. 2024-04-25T14:12:12.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:12 smithi097 ceph-mon[32720]: pgmap v301: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T14:12:12.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:12 smithi151 ceph-mon[32990]: pgmap v301: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T14:12:13.180 INFO:teuthology.orchestra.run.smithi096.stdout:137438953528 2024-04-25T14:12:13.180 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-25T14:12:13.858 INFO:teuthology.orchestra.run.smithi096.stdout:279172874268 2024-04-25T14:12:13.859 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-25T14:12:14.755 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:14 smithi096 ceph-mon[29207]: pgmap v302: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-25T14:12:14.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:14 smithi097 ceph-mon[32720]: pgmap v302: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-25T14:12:14.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:14 smithi151 ceph-mon[32990]: pgmap v302: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-25T14:12:15.529 INFO:teuthology.orchestra.run.smithi096.stdout:257698037792 2024-04-25T14:12:15.530 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-25T14:12:15.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.732+0000 7fe9fd35f700 1 -- 172.21.15.96:0/1662565462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9f8105c40 msgr2=0x7fe9f810e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:15.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.732+0000 7fe9fd35f700 1 --2- 172.21.15.96:0/1662565462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9f8105c40 0x7fe9f810e190 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec009990 tx=0x7fe9ec009ca0 comp rx=0 tx=0).stop 2024-04-25T14:12:15.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.732+0000 7fe9fd35f700 1 -- 172.21.15.96:0/1662565462 shutdown_connections 2024-04-25T14:12:15.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.732+0000 7fe9fd35f700 1 --2- 172.21.15.96:0/1662565462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9f810e760 0x7fe9f8110c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:15.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.732+0000 7fe9fd35f700 1 --2- 172.21.15.96:0/1662565462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9f81052f0 0x7fe9f8105700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:15.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.732+0000 7fe9fd35f700 1 --2- 172.21.15.96:0/1662565462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9f8105c40 0x7fe9f810e190 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:15.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.732+0000 7fe9fd35f700 1 -- 172.21.15.96:0/1662565462 >> 172.21.15.96:0/1662565462 conn(0x7fe9f80fec30 msgr2=0x7fe9f8101050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:15.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.732+0000 7fe9fd35f700 1 -- 172.21.15.96:0/1662565462 shutdown_connections 2024-04-25T14:12:15.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.732+0000 7fe9fd35f700 1 -- 172.21.15.96:0/1662565462 wait complete. 2024-04-25T14:12:15.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.733+0000 7fe9fd35f700 1 Processor -- start 2024-04-25T14:12:15.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.733+0000 7fe9fd35f700 1 -- start start 2024-04-25T14:12:15.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.733+0000 7fe9fd35f700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9f81052f0 0x7fe9f8102fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:15.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.733+0000 7fe9fd35f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9f8105c40 0x7fe9f8103520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:15.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.733+0000 7fe9fd35f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9f810e760 0x7fe9f8103a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:15.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.733+0000 7fe9fd35f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f8114330 con 0x7fe9f8105c40 2024-04-25T14:12:15.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.733+0000 7fe9fd35f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f81141b0 con 0x7fe9f810e760 2024-04-25T14:12:15.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.733+0000 7fe9fd35f700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f81144b0 con 0x7fe9f81052f0 2024-04-25T14:12:15.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.733+0000 7fe9f67fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9f8105c40 0x7fe9f8103520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:15.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.733+0000 7fe9f67fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9f8105c40 0x7fe9f8103520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34928/0 (socket says 172.21.15.96:34928) 2024-04-25T14:12:15.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.733+0000 7fe9f67fc700 1 -- 172.21.15.96:0/2782294883 learned_addr learned my addr 172.21.15.96:0/2782294883 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:12:15.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.733+0000 7fe9f77fe700 1 --2- 172.21.15.96:0/2782294883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9f810e760 0x7fe9f8103a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:15.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.733+0000 7fe9f6ffd700 1 --2- 172.21.15.96:0/2782294883 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9f81052f0 0x7fe9f8102fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:15.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.734+0000 7fe9f67fc700 1 -- 172.21.15.96:0/2782294883 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9f81052f0 msgr2=0x7fe9f8102fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:15.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.734+0000 7fe9f67fc700 1 --2- 172.21.15.96:0/2782294883 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9f81052f0 0x7fe9f8102fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:15.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.734+0000 7fe9f67fc700 1 -- 172.21.15.96:0/2782294883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9f810e760 msgr2=0x7fe9f8103a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:15.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.734+0000 7fe9f67fc700 1 --2- 172.21.15.96:0/2782294883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9f810e760 0x7fe9f8103a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:15.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.734+0000 7fe9f67fc700 1 -- 172.21.15.96:0/2782294883 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9f81a8ff0 con 0x7fe9f8105c40 2024-04-25T14:12:15.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.734+0000 7fe9f67fc700 1 --2- 172.21.15.96:0/2782294883 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9f8105c40 0x7fe9f8103520 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec000c00 tx=0x7fe9ec0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:15.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.734+0000 7fe9dffff700 1 -- 172.21.15.96:0/2782294883 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec027070 con 0x7fe9f8105c40 2024-04-25T14:12:15.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.734+0000 7fe9dffff700 1 -- 172.21.15.96:0/2782294883 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9ec0040e0 con 0x7fe9f8105c40 2024-04-25T14:12:15.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.734+0000 7fe9dffff700 1 -- 172.21.15.96:0/2782294883 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec00f460 con 0x7fe9f8105c40 2024-04-25T14:12:15.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.734+0000 7fe9fd35f700 1 -- 172.21.15.96:0/2782294883 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9f81a9280 con 0x7fe9f8105c40 2024-04-25T14:12:15.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.734+0000 7fe9fd35f700 1 -- 172.21.15.96:0/2782294883 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9f81a9680 con 0x7fe9f8105c40 2024-04-25T14:12:15.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.735+0000 7fe9fd35f700 1 -- 172.21.15.96:0/2782294883 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe9f8063da0 con 0x7fe9f8105c40 2024-04-25T14:12:15.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.738+0000 7fe9dffff700 1 -- 172.21.15.96:0/2782294883 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fe9ec02a070 con 0x7fe9f8105c40 2024-04-25T14:12:15.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.738+0000 7fe9dffff700 1 --2- 172.21.15.96:0/2782294883 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fe9e00afc90 0x7fe9e00b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:15.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.738+0000 7fe9f6ffd700 1 --2- 172.21.15.96:0/2782294883 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fe9e00afc90 0x7fe9e00b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:15.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.738+0000 7fe9dffff700 1 -- 172.21.15.96:0/2782294883 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7fe9ec0e0890 con 0x7fe9f8105c40 2024-04-25T14:12:15.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.739+0000 7fe9dffff700 1 --2- 172.21.15.96:0/2782294883 >> [v2:172.21.15.151:6808/223882546,v1:172.21.15.151:6809/223882546] conn(0x7fe9e00b94c0 0x7fe9e00bb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:15.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.739+0000 7fe9dffff700 1 -- 172.21.15.96:0/2782294883 --> [v2:172.21.15.151:6808/223882546,v1:172.21.15.151:6809/223882546] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe9ec01de50 con 0x7fe9e00b94c0 2024-04-25T14:12:15.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.739+0000 7fe9dffff700 1 -- 172.21.15.96:0/2782294883 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fe9ec0e0d70 con 0x7fe9f8105c40 2024-04-25T14:12:15.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.739+0000 7fe9f77fe700 1 --2- 172.21.15.96:0/2782294883 >> [v2:172.21.15.151:6808/223882546,v1:172.21.15.151:6809/223882546] conn(0x7fe9e00b94c0 0x7fe9e00bb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:15.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.739+0000 7fe9f6ffd700 1 --2- 172.21.15.96:0/2782294883 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fe9e00afc90 0x7fe9e00b21a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8006fd0 tx=0x7fe9e8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:15.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.740+0000 7fe9f77fe700 1 --2- 172.21.15.96:0/2782294883 >> [v2:172.21.15.151:6808/223882546,v1:172.21.15.151:6809/223882546] conn(0x7fe9e00b94c0 0x7fe9e00bb910 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:15.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.740+0000 7fe9dffff700 1 -- 172.21.15.96:0/2782294883 <== osd.5 v2:172.21.15.151:6808/223882546 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe9ec01de50 con 0x7fe9e00b94c0 2024-04-25T14:12:15.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.766+0000 7fe9fd35f700 1 -- 172.21.15.96:0/2782294883 --> [v2:172.21.15.151:6808/223882546,v1:172.21.15.151:6809/223882546] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe9f8064270 con 0x7fe9e00b94c0 2024-04-25T14:12:15.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.767+0000 7fe9dffff700 1 -- 172.21.15.96:0/2782294883 <== osd.5 v2:172.21.15.151:6808/223882546 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe9f8064270 con 0x7fe9e00b94c0 2024-04-25T14:12:15.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.768+0000 7fe9fd35f700 1 -- 172.21.15.96:0/2782294883 >> [v2:172.21.15.151:6808/223882546,v1:172.21.15.151:6809/223882546] conn(0x7fe9e00b94c0 msgr2=0x7fe9e00bb910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:15.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.768+0000 7fe9fd35f700 1 --2- 172.21.15.96:0/2782294883 >> [v2:172.21.15.151:6808/223882546,v1:172.21.15.151:6809/223882546] conn(0x7fe9e00b94c0 0x7fe9e00bb910 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:15.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.768+0000 7fe9fd35f700 1 -- 172.21.15.96:0/2782294883 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fe9e00afc90 msgr2=0x7fe9e00b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:15.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.768+0000 7fe9fd35f700 1 --2- 172.21.15.96:0/2782294883 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fe9e00afc90 0x7fe9e00b21a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8006fd0 tx=0x7fe9e8008040 comp rx=0 tx=0).stop 2024-04-25T14:12:15.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.768+0000 7fe9fd35f700 1 -- 172.21.15.96:0/2782294883 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9f8105c40 msgr2=0x7fe9f8103520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:15.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.768+0000 7fe9fd35f700 1 --2- 172.21.15.96:0/2782294883 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9f8105c40 0x7fe9f8103520 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec000c00 tx=0x7fe9ec0047b0 comp rx=0 tx=0).stop 2024-04-25T14:12:15.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.768+0000 7fe9f77fe700 1 -- 172.21.15.96:0/2782294883 reap_dead start 2024-04-25T14:12:15.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.768+0000 7fe9fd35f700 1 -- 172.21.15.96:0/2782294883 shutdown_connections 2024-04-25T14:12:15.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.768+0000 7fe9fd35f700 1 -- 172.21.15.96:0/2782294883 >> 172.21.15.96:0/2782294883 conn(0x7fe9f80fec30 msgr2=0x7fe9f81009c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:15.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.768+0000 7fe9fd35f700 1 -- 172.21.15.96:0/2782294883 shutdown_connections 2024-04-25T14:12:15.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:15.768+0000 7fe9fd35f700 1 -- 172.21.15.96:0/2782294883 wait complete. 2024-04-25T14:12:16.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.431+0000 7f0422a10700 1 -- 172.21.15.96:0/269822553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f041c103360 msgr2=0x7f041c103770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:16.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.431+0000 7f0422a10700 1 --2- 172.21.15.96:0/269822553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f041c103360 0x7f041c103770 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f040c0099c0 tx=0x7f040c009cd0 comp rx=0 tx=0).stop 2024-04-25T14:12:16.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.431+0000 7f0422a10700 1 -- 172.21.15.96:0/269822553 shutdown_connections 2024-04-25T14:12:16.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.431+0000 7f0422a10700 1 --2- 172.21.15.96:0/269822553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f041c1085e0 0x7f041c10ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:16.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.431+0000 7f0422a10700 1 --2- 172.21.15.96:0/269822553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f041c103360 0x7f041c103770 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:16.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.431+0000 7f0422a10700 1 --2- 172.21.15.96:0/269822553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f041c107c90 0x7f041c1080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:16.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.431+0000 7f0422a10700 1 -- 172.21.15.96:0/269822553 >> 172.21.15.96:0/269822553 conn(0x7f041c0fecb0 msgr2=0x7f041c1010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:16.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.431+0000 7f0422a10700 1 -- 172.21.15.96:0/269822553 shutdown_connections 2024-04-25T14:12:16.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.431+0000 7f0422a10700 1 -- 172.21.15.96:0/269822553 wait complete. 2024-04-25T14:12:16.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.432+0000 7f0422a10700 1 Processor -- start 2024-04-25T14:12:16.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.432+0000 7f0422a10700 1 -- start start 2024-04-25T14:12:16.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.433+0000 7f0422a10700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f041c103360 0x7f041c19bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:16.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.433+0000 7f0422a10700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f041c107c90 0x7f041c19c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:16.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.433+0000 7f0422a10700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f041c1085e0 0x7f041c19c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:16.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.433+0000 7f0422a10700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f041c113210 con 0x7f041c1085e0 2024-04-25T14:12:16.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.433+0000 7f0422a10700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f041c113090 con 0x7f041c103360 2024-04-25T14:12:16.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.433+0000 7f0422a10700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f041c113390 con 0x7f041c107c90 2024-04-25T14:12:16.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.433+0000 7f0420fad700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f041c1085e0 0x7f041c19c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:16.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.433+0000 7f0420fad700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f041c1085e0 0x7f041c19c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34954/0 (socket says 172.21.15.96:34954) 2024-04-25T14:12:16.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.433+0000 7f0420fad700 1 -- 172.21.15.96:0/1109340499 learned_addr learned my addr 172.21.15.96:0/1109340499 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:12:16.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.433+0000 7f041bfff700 1 --2- 172.21.15.96:0/1109340499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f041c103360 0x7f041c19bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:16.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.433+0000 7f041b7fe700 1 --2- 172.21.15.96:0/1109340499 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f041c107c90 0x7f041c19c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:16.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.433+0000 7f0420fad700 1 -- 172.21.15.96:0/1109340499 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f041c107c90 msgr2=0x7f041c19c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:16.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.433+0000 7f0420fad700 1 --2- 172.21.15.96:0/1109340499 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f041c107c90 0x7f041c19c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:16.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.433+0000 7f0420fad700 1 -- 172.21.15.96:0/1109340499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f041c103360 msgr2=0x7f041c19bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:16.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.433+0000 7f0420fad700 1 --2- 172.21.15.96:0/1109340499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f041c103360 0x7f041c19bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:16.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.433+0000 7f0420fad700 1 -- 172.21.15.96:0/1109340499 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f041c1a6e40 con 0x7f041c1085e0 2024-04-25T14:12:16.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.434+0000 7f0420fad700 1 --2- 172.21.15.96:0/1109340499 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f041c1085e0 0x7f041c19c590 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f041400ada0 tx=0x7f041400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:16.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.434+0000 7f04197fa700 1 -- 172.21.15.96:0/1109340499 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0414017020 con 0x7f041c1085e0 2024-04-25T14:12:16.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.434+0000 7f0422a10700 1 -- 172.21.15.96:0/1109340499 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f041c1a7130 con 0x7f041c1085e0 2024-04-25T14:12:16.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.434+0000 7f04197fa700 1 -- 172.21.15.96:0/1109340499 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f041400cd80 con 0x7f041c1085e0 2024-04-25T14:12:16.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.434+0000 7f04197fa700 1 -- 172.21.15.96:0/1109340499 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0414005260 con 0x7f041c1085e0 2024-04-25T14:12:16.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.434+0000 7f0422a10700 1 -- 172.21.15.96:0/1109340499 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f041c1a7620 con 0x7f041c1085e0 2024-04-25T14:12:16.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.435+0000 7f0422a10700 1 -- 172.21.15.96:0/1109340499 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f041c063da0 con 0x7f041c1085e0 2024-04-25T14:12:16.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.437+0000 7f04197fa700 1 -- 172.21.15.96:0/1109340499 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f041400a080 con 0x7f041c1085e0 2024-04-25T14:12:16.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.437+0000 7f04197fa700 1 --2- 172.21.15.96:0/1109340499 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f04040afc10 0x7f04040b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:16.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.437+0000 7f041bfff700 1 --2- 172.21.15.96:0/1109340499 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f04040afc10 0x7f04040b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:16.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.437+0000 7f04197fa700 1 -- 172.21.15.96:0/1109340499 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7f04140dec70 con 0x7f041c1085e0 2024-04-25T14:12:16.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.437+0000 7f04197fa700 1 --2- 172.21.15.96:0/1109340499 >> [v2:172.21.15.151:6824/2027958746,v1:172.21.15.151:6825/2027958746] conn(0x7f04040b9440 0x7f04040bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:16.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.438+0000 7f04197fa700 1 -- 172.21.15.96:0/1109340499 --> [v2:172.21.15.151:6824/2027958746,v1:172.21.15.151:6825/2027958746] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0414022e50 con 0x7f04040b9440 2024-04-25T14:12:16.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.438+0000 7f04197fa700 1 -- 172.21.15.96:0/1109340499 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f04140df150 con 0x7f041c1085e0 2024-04-25T14:12:16.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.438+0000 7f041bfff700 1 --2- 172.21.15.96:0/1109340499 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f04040afc10 0x7f04040b2120 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f040c009990 tx=0x7f040c0036e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:16.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.438+0000 7f041b7fe700 1 --2- 172.21.15.96:0/1109340499 >> [v2:172.21.15.151:6824/2027958746,v1:172.21.15.151:6825/2027958746] conn(0x7f04040b9440 0x7f04040bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:16.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.438+0000 7f041b7fe700 1 --2- 172.21.15.96:0/1109340499 >> [v2:172.21.15.151:6824/2027958746,v1:172.21.15.151:6825/2027958746] conn(0x7f04040b9440 0x7f04040bb890 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:16.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.439+0000 7f04197fa700 1 -- 172.21.15.96:0/1109340499 <== osd.11 v2:172.21.15.151:6824/2027958746 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0414022e50 con 0x7f04040b9440 2024-04-25T14:12:16.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.456+0000 7f0422a10700 1 -- 172.21.15.96:0/1109340499 --> [v2:172.21.15.151:6824/2027958746,v1:172.21.15.151:6825/2027958746] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f041c0641a0 con 0x7f04040b9440 2024-04-25T14:12:16.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.456+0000 7f04197fa700 1 -- 172.21.15.96:0/1109340499 <== osd.11 v2:172.21.15.151:6824/2027958746 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f041c0641a0 con 0x7f04040b9440 2024-04-25T14:12:16.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.457+0000 7f0422a10700 1 -- 172.21.15.96:0/1109340499 >> [v2:172.21.15.151:6824/2027958746,v1:172.21.15.151:6825/2027958746] conn(0x7f04040b9440 msgr2=0x7f04040bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:16.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.457+0000 7f0422a10700 1 --2- 172.21.15.96:0/1109340499 >> [v2:172.21.15.151:6824/2027958746,v1:172.21.15.151:6825/2027958746] conn(0x7f04040b9440 0x7f04040bb890 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:16.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.457+0000 7f0422a10700 1 -- 172.21.15.96:0/1109340499 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f04040afc10 msgr2=0x7f04040b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:16.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.457+0000 7f0422a10700 1 --2- 172.21.15.96:0/1109340499 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f04040afc10 0x7f04040b2120 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f040c009990 tx=0x7f040c0036e0 comp rx=0 tx=0).stop 2024-04-25T14:12:16.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.457+0000 7f0422a10700 1 -- 172.21.15.96:0/1109340499 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f041c1085e0 msgr2=0x7f041c19c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:16.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.457+0000 7f0422a10700 1 --2- 172.21.15.96:0/1109340499 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f041c1085e0 0x7f041c19c590 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f041400ada0 tx=0x7f041400c330 comp rx=0 tx=0).stop 2024-04-25T14:12:16.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.457+0000 7f0420fad700 1 -- 172.21.15.96:0/1109340499 reap_dead start 2024-04-25T14:12:16.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.457+0000 7f0422a10700 1 -- 172.21.15.96:0/1109340499 shutdown_connections 2024-04-25T14:12:16.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.457+0000 7f0422a10700 1 -- 172.21.15.96:0/1109340499 >> 172.21.15.96:0/1109340499 conn(0x7f041c0fecb0 msgr2=0x7f041c107010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:16.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.457+0000 7f0422a10700 1 -- 172.21.15.96:0/1109340499 shutdown_connections 2024-04-25T14:12:16.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:16.457+0000 7f0422a10700 1 -- 172.21.15.96:0/1109340499 wait complete. 2024-04-25T14:12:16.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:16 smithi096 ceph-mon[29207]: pgmap v303: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-04-25T14:12:16.899 INFO:teuthology.orchestra.run.smithi096.stdout:158913790006 2024-04-25T14:12:16.899 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-25T14:12:16.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:16 smithi097 ceph-mon[32720]: pgmap v303: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-04-25T14:12:16.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:16 smithi151 ceph-mon[32990]: pgmap v303: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-04-25T14:12:17.811 INFO:teuthology.orchestra.run.smithi096.stdout:231928234022 2024-04-25T14:12:17.812 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-25T14:12:18.233 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:12:18.272 INFO:teuthology.orchestra.run.smithi096.stdout:111669149760 2024-04-25T14:12:18.272 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-25T14:12:18.769 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:18 smithi096 ceph-mon[29207]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T14:12:18.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:18 smithi097 ceph-mon[32720]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T14:12:18.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:18 smithi151 ceph-mon[32990]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T14:12:19.011 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:12:19.032 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:12:19.032 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:12:19.033 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:12:19.034 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:12:19.035 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:12:19.036 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:12:19.040 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:12:19.493 INFO:teuthology.orchestra.run.smithi096.stdout:304942678039 2024-04-25T14:12:19.494 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-25T14:12:20.608 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:20 smithi096 ceph-mon[29207]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T14:12:20.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:20 smithi097 ceph-mon[32720]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T14:12:20.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:20 smithi151 ceph-mon[32990]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T14:12:21.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:21 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-04-25T14:12:21.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:21 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-04-25T14:12:21.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:21 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-04-25T14:12:21.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:21 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]: dispatch 2024-04-25T14:12:21.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:21 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-04-25T14:12:21.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:21 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-04-25T14:12:21.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:21 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]: dispatch 2024-04-25T14:12:21.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:21 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-04-25T14:12:21.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:21 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]: dispatch 2024-04-25T14:12:21.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:21 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]}]: dispatch 2024-04-25T14:12:21.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:21 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-04-25T14:12:21.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:21 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-04-25T14:12:21.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:21 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-04-25T14:12:21.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:21 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]: dispatch 2024-04-25T14:12:21.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:21 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-04-25T14:12:21.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:21 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-04-25T14:12:21.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:21 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]: dispatch 2024-04-25T14:12:21.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:21 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-04-25T14:12:21.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:21 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]: dispatch 2024-04-25T14:12:21.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:21 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]}]: dispatch 2024-04-25T14:12:21.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:21 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-04-25T14:12:21.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:21 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-04-25T14:12:21.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:21 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-04-25T14:12:21.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:21 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]: dispatch 2024-04-25T14:12:21.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:21 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-04-25T14:12:21.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:21 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-04-25T14:12:21.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:21 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]: dispatch 2024-04-25T14:12:21.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:21 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-04-25T14:12:21.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:21 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]: dispatch 2024-04-25T14:12:21.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:21 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]}]: dispatch 2024-04-25T14:12:22.817 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:22 smithi096 ceph-mon[29207]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T14:12:22.818 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:22 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-04-25T14:12:22.818 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:22 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-04-25T14:12:22.818 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:22 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-04-25T14:12:22.818 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:22 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]': finished 2024-04-25T14:12:22.818 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:22 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-04-25T14:12:22.818 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:22 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-04-25T14:12:22.818 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:22 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]': finished 2024-04-25T14:12:22.818 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:22 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-04-25T14:12:22.818 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:22 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]': finished 2024-04-25T14:12:22.818 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:22 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]}]': finished 2024-04-25T14:12:22.818 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:22 smithi096 ceph-mon[29207]: osdmap e77: 12 total, 12 up, 12 in 2024-04-25T14:12:22.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:22 smithi097 ceph-mon[32720]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T14:12:22.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:22 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-04-25T14:12:22.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:22 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-04-25T14:12:22.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:22 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-04-25T14:12:22.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:22 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]': finished 2024-04-25T14:12:22.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:22 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-04-25T14:12:22.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:22 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-04-25T14:12:22.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:22 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]': finished 2024-04-25T14:12:22.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:22 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-04-25T14:12:22.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:22 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]': finished 2024-04-25T14:12:22.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:22 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]}]': finished 2024-04-25T14:12:22.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:22 smithi097 ceph-mon[32720]: osdmap e77: 12 total, 12 up, 12 in 2024-04-25T14:12:22.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:22 smithi151 ceph-mon[32990]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T14:12:22.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:22 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-04-25T14:12:22.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:22 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-04-25T14:12:22.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:22 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-04-25T14:12:22.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:22 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]': finished 2024-04-25T14:12:22.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:22 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-04-25T14:12:22.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:22 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-04-25T14:12:22.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:22 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]': finished 2024-04-25T14:12:22.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:22 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-04-25T14:12:22.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:22 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]': finished 2024-04-25T14:12:22.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:22 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]}]': finished 2024-04-25T14:12:22.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:22 smithi151 ceph-mon[32990]: osdmap e77: 12 total, 12 up, 12 in 2024-04-25T14:12:23.621 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:23 smithi096 ceph-mon[29207]: osdmap e78: 12 total, 12 up, 12 in 2024-04-25T14:12:23.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:23 smithi097 ceph-mon[32720]: osdmap e78: 12 total, 12 up, 12 in 2024-04-25T14:12:23.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:23 smithi151 ceph-mon[32990]: osdmap e78: 12 total, 12 up, 12 in 2024-04-25T14:12:24.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.437+0000 7f583d0e4700 1 -- 172.21.15.96:0/2178741058 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58381033c0 msgr2=0x7f58381037d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:24.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.437+0000 7f583d0e4700 1 --2- 172.21.15.96:0/2178741058 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58381033c0 0x7f58381037d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f582c0099c0 tx=0x7f582c009cd0 comp rx=0 tx=0).stop 2024-04-25T14:12:24.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.437+0000 7f583d0e4700 1 -- 172.21.15.96:0/2178741058 shutdown_connections 2024-04-25T14:12:24.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.437+0000 7f583d0e4700 1 --2- 172.21.15.96:0/2178741058 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5838108640 0x7f583810ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:24.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.437+0000 7f583d0e4700 1 --2- 172.21.15.96:0/2178741058 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58381033c0 0x7f58381037d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:24.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.437+0000 7f583d0e4700 1 --2- 172.21.15.96:0/2178741058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5838107cf0 0x7f5838108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:24.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.437+0000 7f583d0e4700 1 -- 172.21.15.96:0/2178741058 >> 172.21.15.96:0/2178741058 conn(0x7f58380fedb0 msgr2=0x7f58381011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:24.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.437+0000 7f583d0e4700 1 -- 172.21.15.96:0/2178741058 shutdown_connections 2024-04-25T14:12:24.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.437+0000 7f583d0e4700 1 -- 172.21.15.96:0/2178741058 wait complete. 2024-04-25T14:12:24.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.438+0000 7f583d0e4700 1 Processor -- start 2024-04-25T14:12:24.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.438+0000 7f583d0e4700 1 -- start start 2024-04-25T14:12:24.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.438+0000 7f583d0e4700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58381033c0 0x7f583819f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:24.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.439+0000 7f583d0e4700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5838107cf0 0x7f583819fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:24.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.439+0000 7f583d0e4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5838108640 0x7f5838199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:24.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.439+0000 7f583d0e4700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5838113270 con 0x7f5838107cf0 2024-04-25T14:12:24.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.439+0000 7f583d0e4700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f58381130f0 con 0x7f5838108640 2024-04-25T14:12:24.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.439+0000 7f583d0e4700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f58381133f0 con 0x7f58381033c0 2024-04-25T14:12:24.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.439+0000 7f583659c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5838107cf0 0x7f583819fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:24.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.439+0000 7f583659c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5838107cf0 0x7f583819fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34558/0 (socket says 172.21.15.96:34558) 2024-04-25T14:12:24.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.439+0000 7f583659c700 1 -- 172.21.15.96:0/3812981221 learned_addr learned my addr 172.21.15.96:0/3812981221 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:12:24.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.439+0000 7f583759e700 1 --2- 172.21.15.96:0/3812981221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5838108640 0x7f5838199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:24.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.439+0000 7f5836d9d700 1 --2- 172.21.15.96:0/3812981221 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58381033c0 0x7f583819f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:24.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.439+0000 7f583659c700 1 -- 172.21.15.96:0/3812981221 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58381033c0 msgr2=0x7f583819f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:24.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.439+0000 7f583659c700 1 --2- 172.21.15.96:0/3812981221 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58381033c0 0x7f583819f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:24.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.439+0000 7f583659c700 1 -- 172.21.15.96:0/3812981221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5838108640 msgr2=0x7f5838199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:24.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.439+0000 7f583659c700 1 --2- 172.21.15.96:0/3812981221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5838108640 0x7f5838199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:24.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.439+0000 7f583659c700 1 -- 172.21.15.96:0/3812981221 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f583819a050 con 0x7f5838107cf0 2024-04-25T14:12:24.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.439+0000 7f583759e700 1 --2- 172.21.15.96:0/3812981221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5838108640 0x7f5838199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:12:24.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.439+0000 7f583659c700 1 --2- 172.21.15.96:0/3812981221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5838107cf0 0x7f583819fd30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f582800d980 tx=0x7f582800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:24.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.439+0000 7f581ffff700 1 -- 172.21.15.96:0/3812981221 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5828017070 con 0x7f5838107cf0 2024-04-25T14:12:24.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.439+0000 7f583d0e4700 1 -- 172.21.15.96:0/3812981221 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f583819a340 con 0x7f5838107cf0 2024-04-25T14:12:24.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.439+0000 7f581ffff700 1 -- 172.21.15.96:0/3812981221 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5828010460 con 0x7f5838107cf0 2024-04-25T14:12:24.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.439+0000 7f581ffff700 1 -- 172.21.15.96:0/3812981221 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582800de60 con 0x7f5838107cf0 2024-04-25T14:12:24.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.439+0000 7f583d0e4700 1 -- 172.21.15.96:0/3812981221 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58381a6680 con 0x7f5838107cf0 2024-04-25T14:12:24.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.441+0000 7f583d0e4700 1 -- 172.21.15.96:0/3812981221 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5838051190 con 0x7f5838107cf0 2024-04-25T14:12:24.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.442+0000 7f581ffff700 1 -- 172.21.15.96:0/3812981221 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f5828010610 con 0x7f5838107cf0 2024-04-25T14:12:24.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.443+0000 7f581ffff700 1 --2- 172.21.15.96:0/3812981221 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f58200afc10 0x7f58200b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:24.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.443+0000 7f5836d9d700 1 --2- 172.21.15.96:0/3812981221 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f58200afc10 0x7f58200b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:24.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.443+0000 7f581ffff700 1 -- 172.21.15.96:0/3812981221 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f58280de980 con 0x7f5838107cf0 2024-04-25T14:12:24.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.443+0000 7f5836d9d700 1 --2- 172.21.15.96:0/3812981221 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f58200afc10 0x7f58200b2120 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f582c000c00 tx=0x7f582c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:24.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.446+0000 7f581ffff700 1 -- 172.21.15.96:0/3812981221 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f58280a73e0 con 0x7f5838107cf0 2024-04-25T14:12:24.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:24 smithi096 ceph-mon[29207]: pgmap v309: 97 pgs: 2 peering, 95 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 10 op/s 2024-04-25T14:12:24.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.653+0000 7f583d0e4700 1 -- 172.21.15.96:0/3812981221 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f583802e510 con 0x7f5838107cf0 2024-04-25T14:12:24.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.654+0000 7f581ffff700 1 -- 172.21.15.96:0/3812981221 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f58280a6ce0 con 0x7f5838107cf0 2024-04-25T14:12:24.654 INFO:teuthology.orchestra.run.smithi096.stdout:77309411398 2024-04-25T14:12:24.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.656+0000 7f583d0e4700 1 -- 172.21.15.96:0/3812981221 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f58200afc10 msgr2=0x7f58200b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:24.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.656+0000 7f583d0e4700 1 --2- 172.21.15.96:0/3812981221 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f58200afc10 0x7f58200b2120 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f582c000c00 tx=0x7f582c01a040 comp rx=0 tx=0).stop 2024-04-25T14:12:24.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.656+0000 7f583d0e4700 1 -- 172.21.15.96:0/3812981221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5838107cf0 msgr2=0x7f583819fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:24.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.656+0000 7f583d0e4700 1 --2- 172.21.15.96:0/3812981221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5838107cf0 0x7f583819fd30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f582800d980 tx=0x7f582800dd40 comp rx=0 tx=0).stop 2024-04-25T14:12:24.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.656+0000 7f583d0e4700 1 -- 172.21.15.96:0/3812981221 shutdown_connections 2024-04-25T14:12:24.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.656+0000 7f583d0e4700 1 --2- 172.21.15.96:0/3812981221 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f58200afc10 0x7f58200b2120 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:24.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.656+0000 7f583d0e4700 1 --2- 172.21.15.96:0/3812981221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5838108640 0x7f5838199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:24.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.656+0000 7f583d0e4700 1 --2- 172.21.15.96:0/3812981221 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58381033c0 0x7f583819f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:24.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.656+0000 7f583d0e4700 1 --2- 172.21.15.96:0/3812981221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5838107cf0 0x7f583819fd30 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:24.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.656+0000 7f583d0e4700 1 -- 172.21.15.96:0/3812981221 >> 172.21.15.96:0/3812981221 conn(0x7f58380fedb0 msgr2=0x7f5838107460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:24.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.656+0000 7f583d0e4700 1 -- 172.21.15.96:0/3812981221 shutdown_connections 2024-04-25T14:12:24.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:24.656+0000 7f583d0e4700 1 -- 172.21.15.96:0/3812981221 wait complete. 2024-04-25T14:12:24.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:24 smithi097 ceph-mon[32720]: pgmap v309: 97 pgs: 2 peering, 95 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 10 op/s 2024-04-25T14:12:24.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:24 smithi151 ceph-mon[32990]: pgmap v309: 97 pgs: 2 peering, 95 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 10 op/s 2024-04-25T14:12:25.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:25 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/3812981221' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T14:12:25.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:25 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/3812981221' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T14:12:25.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:25 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/3812981221' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T14:12:26.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:26 smithi096 ceph-mon[29207]: pgmap v310: 97 pgs: 2 peering, 95 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 83 B/s, 0 objects/s recovering 2024-04-25T14:12:26.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:26 smithi097 ceph-mon[32720]: pgmap v310: 97 pgs: 2 peering, 95 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 83 B/s, 0 objects/s recovering 2024-04-25T14:12:26.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:26 smithi151 ceph-mon[32990]: pgmap v310: 97 pgs: 2 peering, 95 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 83 B/s, 0 objects/s recovering 2024-04-25T14:12:28.862 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:28 smithi096 ceph-mon[29207]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s; 92 B/s, 1 keys/s, 1 objects/s recovering 2024-04-25T14:12:28.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:28 smithi097 ceph-mon[32720]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s; 92 B/s, 1 keys/s, 1 objects/s recovering 2024-04-25T14:12:28.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:28 smithi151 ceph-mon[32990]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s; 92 B/s, 1 keys/s, 1 objects/s recovering 2024-04-25T14:12:29.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:29 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:12:29.698 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:29 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:12:29.755 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:29 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:12:29.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:29.958+0000 7f2ee3eda700 1 -- 172.21.15.96:0/2460512433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2edc1033c0 msgr2=0x7f2edc1037d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:29.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:29.958+0000 7f2ee3eda700 1 --2- 172.21.15.96:0/2460512433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2edc1033c0 0x7f2edc1037d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2ed80099c0 tx=0x7f2ed8009cd0 comp rx=0 tx=0).stop 2024-04-25T14:12:29.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:29.958+0000 7f2ee3eda700 1 -- 172.21.15.96:0/2460512433 shutdown_connections 2024-04-25T14:12:29.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:29.958+0000 7f2ee3eda700 1 --2- 172.21.15.96:0/2460512433 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2edc108640 0x7f2edc10ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:29.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:29.958+0000 7f2ee3eda700 1 --2- 172.21.15.96:0/2460512433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2edc1033c0 0x7f2edc1037d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:29.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:29.958+0000 7f2ee3eda700 1 --2- 172.21.15.96:0/2460512433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2edc107cf0 0x7f2edc108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:29.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:29.958+0000 7f2ee3eda700 1 -- 172.21.15.96:0/2460512433 >> 172.21.15.96:0/2460512433 conn(0x7f2edc0fedb0 msgr2=0x7f2edc1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:29.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:29.959+0000 7f2ee3eda700 1 -- 172.21.15.96:0/2460512433 shutdown_connections 2024-04-25T14:12:29.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:29.959+0000 7f2ee3eda700 1 -- 172.21.15.96:0/2460512433 wait complete. 2024-04-25T14:12:29.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:29.959+0000 7f2ee3eda700 1 Processor -- start 2024-04-25T14:12:29.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:29.959+0000 7f2ee3eda700 1 -- start start 2024-04-25T14:12:29.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:29.959+0000 7f2ee3eda700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2edc1033c0 0x7f2edc19bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:29.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:29.960+0000 7f2ee3eda700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2edc107cf0 0x7f2edc19c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:29.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:29.960+0000 7f2ee3eda700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2edc108640 0x7f2edc19c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:29.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:29.960+0000 7f2ee3eda700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc1132f0 con 0x7f2edc108640 2024-04-25T14:12:29.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:29.960+0000 7f2ee3eda700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc113170 con 0x7f2edc1033c0 2024-04-25T14:12:29.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:29.960+0000 7f2ee3eda700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc113470 con 0x7f2edc107cf0 2024-04-25T14:12:29.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:29.960+0000 7f2ee2477700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2edc108640 0x7f2edc19c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:29.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:29.960+0000 7f2ee2477700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2edc108640 0x7f2edc19c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34574/0 (socket says 172.21.15.96:34574) 2024-04-25T14:12:29.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:29.960+0000 7f2ee1c76700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2edc1033c0 0x7f2edc19bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:29.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:29.960+0000 7f2ee2477700 1 -- 172.21.15.96:0/2901468788 learned_addr learned my addr 172.21.15.96:0/2901468788 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:12:29.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:29.960+0000 7f2ee1475700 1 --2- 172.21.15.96:0/2901468788 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2edc107cf0 0x7f2edc19c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:29.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:29.960+0000 7f2ee2477700 1 -- 172.21.15.96:0/2901468788 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2edc107cf0 msgr2=0x7f2edc19c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:29.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:29.960+0000 7f2ee2477700 1 --2- 172.21.15.96:0/2901468788 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2edc107cf0 0x7f2edc19c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:29.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:29.960+0000 7f2ee2477700 1 -- 172.21.15.96:0/2901468788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2edc1033c0 msgr2=0x7f2edc19bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:29.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:29.960+0000 7f2ee2477700 1 --2- 172.21.15.96:0/2901468788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2edc1033c0 0x7f2edc19bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:29.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:29.960+0000 7f2ee2477700 1 -- 172.21.15.96:0/2901468788 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2edc1a6f00 con 0x7f2edc108640 2024-04-25T14:12:29.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:29.960+0000 7f2ee2477700 1 --2- 172.21.15.96:0/2901468788 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2edc108640 0x7f2edc19c650 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2ed400b970 tx=0x7f2ed400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:29.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:29.960+0000 7f2ed2ffd700 1 -- 172.21.15.96:0/2901468788 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed400d860 con 0x7f2edc108640 2024-04-25T14:12:29.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:29.960+0000 7f2ed2ffd700 1 -- 172.21.15.96:0/2901468788 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ed400e430 con 0x7f2edc108640 2024-04-25T14:12:29.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:29.960+0000 7f2ee3eda700 1 -- 172.21.15.96:0/2901468788 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2edc1999e0 con 0x7f2edc108640 2024-04-25T14:12:29.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:29.960+0000 7f2ed2ffd700 1 -- 172.21.15.96:0/2901468788 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4017660 con 0x7f2edc108640 2024-04-25T14:12:29.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:29.960+0000 7f2ee3eda700 1 -- 172.21.15.96:0/2901468788 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2edc199ed0 con 0x7f2edc108640 2024-04-25T14:12:29.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:29.962+0000 7f2ee3eda700 1 -- 172.21.15.96:0/2901468788 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2edc051190 con 0x7f2edc108640 2024-04-25T14:12:29.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:29.963+0000 7f2ed2ffd700 1 -- 172.21.15.96:0/2901468788 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f2ed400da00 con 0x7f2edc108640 2024-04-25T14:12:29.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:29.964+0000 7f2ed2ffd700 1 --2- 172.21.15.96:0/2901468788 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f2ec80afc10 0x7f2ec80b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:29.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:29.964+0000 7f2ee1c76700 1 --2- 172.21.15.96:0/2901468788 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f2ec80afc10 0x7f2ec80b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:29.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:29.964+0000 7f2ed2ffd700 1 -- 172.21.15.96:0/2901468788 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f2ed40debf0 con 0x7f2edc108640 2024-04-25T14:12:29.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:29.964+0000 7f2ee1c76700 1 --2- 172.21.15.96:0/2901468788 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f2ec80afc10 0x7f2ec80b2120 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8005b40 tx=0x7f2ed8005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:29.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:29.967+0000 7f2ed2ffd700 1 -- 172.21.15.96:0/2901468788 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2ed4052ff0 con 0x7f2edc108640 2024-04-25T14:12:30.076 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:12:30.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:30.183+0000 7f2ee3eda700 1 -- 172.21.15.96:0/2901468788 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f2edc02e6d0 con 0x7f2edc108640 2024-04-25T14:12:30.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:30.183+0000 7f2ed2ffd700 1 -- 172.21.15.96:0/2901468788 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f2ed4013030 con 0x7f2edc108640 2024-04-25T14:12:30.183 INFO:teuthology.orchestra.run.smithi096.stdout:184683593780 2024-04-25T14:12:30.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:30.186+0000 7f2ee3eda700 1 -- 172.21.15.96:0/2901468788 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f2ec80afc10 msgr2=0x7f2ec80b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:30.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:30.186+0000 7f2ee3eda700 1 --2- 172.21.15.96:0/2901468788 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f2ec80afc10 0x7f2ec80b2120 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8005b40 tx=0x7f2ed8005ab0 comp rx=0 tx=0).stop 2024-04-25T14:12:30.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:30.186+0000 7f2ee3eda700 1 -- 172.21.15.96:0/2901468788 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2edc108640 msgr2=0x7f2edc19c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:30.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:30.186+0000 7f2ee3eda700 1 --2- 172.21.15.96:0/2901468788 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2edc108640 0x7f2edc19c650 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2ed400b970 tx=0x7f2ed400bc80 comp rx=0 tx=0).stop 2024-04-25T14:12:30.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:30.186+0000 7f2ee3eda700 1 -- 172.21.15.96:0/2901468788 shutdown_connections 2024-04-25T14:12:30.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:30.186+0000 7f2ee3eda700 1 --2- 172.21.15.96:0/2901468788 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f2ec80afc10 0x7f2ec80b2120 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:30.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:30.186+0000 7f2ee3eda700 1 --2- 172.21.15.96:0/2901468788 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2edc108640 0x7f2edc19c650 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:30.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:30.186+0000 7f2ee3eda700 1 --2- 172.21.15.96:0/2901468788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2edc1033c0 0x7f2edc19bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:30.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:30.186+0000 7f2ee3eda700 1 --2- 172.21.15.96:0/2901468788 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2edc107cf0 0x7f2edc19c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:30.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:30.186+0000 7f2ee3eda700 1 -- 172.21.15.96:0/2901468788 >> 172.21.15.96:0/2901468788 conn(0x7f2edc0fedb0 msgr2=0x7f2edc107460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:30.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:30.186+0000 7f2ee3eda700 1 -- 172.21.15.96:0/2901468788 shutdown_connections 2024-04-25T14:12:30.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:30.186+0000 7f2ee3eda700 1 -- 172.21.15.96:0/2901468788 wait complete. 2024-04-25T14:12:30.543 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411393 got 77309411398 for osd.2 2024-04-25T14:12:30.543 DEBUG:teuthology.parallel:result is None 2024-04-25T14:12:30.845 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:30 smithi096 ceph-mon[29207]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 92 B/s, 1 keys/s, 1 objects/s recovering 2024-04-25T14:12:30.845 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:30 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:12:30.845 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:30 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:12:30.845 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:30 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/2901468788' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T14:12:30.845 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:30 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:12:30.845 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:30 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:12:30.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:30 smithi097 ceph-mon[32720]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 92 B/s, 1 keys/s, 1 objects/s recovering 2024-04-25T14:12:30.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:30 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:12:30.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:30 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:12:30.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:30 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/2901468788' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T14:12:30.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:30 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:12:30.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:30 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:12:30.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:30 smithi151 ceph-mon[32990]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 92 B/s, 1 keys/s, 1 objects/s recovering 2024-04-25T14:12:30.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:30 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:12:30.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:30 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:12:30.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:30 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/2901468788' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T14:12:30.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:30 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:12:30.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:30 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:12:31.252 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593776 got 184683593780 for osd.6 2024-04-25T14:12:31.252 DEBUG:teuthology.parallel:result is None 2024-04-25T14:12:32.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.274+0000 7f0381208700 1 -- 172.21.15.96:0/2342374909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f037c108db0 msgr2=0x7f037c1091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:32.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.274+0000 7f0381208700 1 --2- 172.21.15.96:0/2342374909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f037c108db0 0x7f037c1091c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f03700099c0 tx=0x7f0370009cd0 comp rx=0 tx=0).stop 2024-04-25T14:12:32.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.274+0000 7f0381208700 1 -- 172.21.15.96:0/2342374909 shutdown_connections 2024-04-25T14:12:32.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.274+0000 7f0381208700 1 --2- 172.21.15.96:0/2342374909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f037c109700 0x7f037c111060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:32.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.274+0000 7f0381208700 1 --2- 172.21.15.96:0/2342374909 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f037c104480 0x7f037c104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:32.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.274+0000 7f0381208700 1 --2- 172.21.15.96:0/2342374909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f037c108db0 0x7f037c1091c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:32.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.274+0000 7f0381208700 1 -- 172.21.15.96:0/2342374909 >> 172.21.15.96:0/2342374909 conn(0x7f037c0ffd70 msgr2=0x7f037c1021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:32.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.275+0000 7f0381208700 1 -- 172.21.15.96:0/2342374909 shutdown_connections 2024-04-25T14:12:32.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.275+0000 7f0381208700 1 -- 172.21.15.96:0/2342374909 wait complete. 2024-04-25T14:12:32.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.275+0000 7f0381208700 1 Processor -- start 2024-04-25T14:12:32.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.276+0000 7f0381208700 1 -- start start 2024-04-25T14:12:32.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.276+0000 7f0381208700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f037c104480 0x7f037c19cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:32.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.276+0000 7f0381208700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f037c108db0 0x7f037c19d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:32.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.276+0000 7f0381208700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f037c109700 0x7f037c1a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:32.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.276+0000 7f0381208700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f037c114310 con 0x7f037c104480 2024-04-25T14:12:32.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.276+0000 7f0381208700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f037c114190 con 0x7f037c109700 2024-04-25T14:12:32.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.276+0000 7f0381208700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f037c114490 con 0x7f037c108db0 2024-04-25T14:12:32.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.276+0000 7f037ad9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f037c104480 0x7f037c19cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:32.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.276+0000 7f037ad9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f037c104480 0x7f037c19cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49584/0 (socket says 172.21.15.96:49584) 2024-04-25T14:12:32.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.276+0000 7f037ad9d700 1 -- 172.21.15.96:0/2706803109 learned_addr learned my addr 172.21.15.96:0/2706803109 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:12:32.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.276+0000 7f037a59c700 1 --2- 172.21.15.96:0/2706803109 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f037c108db0 0x7f037c19d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:32.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.276+0000 7f037b59e700 1 --2- 172.21.15.96:0/2706803109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f037c109700 0x7f037c1a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:32.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.276+0000 7f037ad9d700 1 -- 172.21.15.96:0/2706803109 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f037c108db0 msgr2=0x7f037c19d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:32.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.276+0000 7f037ad9d700 1 --2- 172.21.15.96:0/2706803109 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f037c108db0 0x7f037c19d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:32.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.276+0000 7f037ad9d700 1 -- 172.21.15.96:0/2706803109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f037c109700 msgr2=0x7f037c1a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:32.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.276+0000 7f037ad9d700 1 --2- 172.21.15.96:0/2706803109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f037c109700 0x7f037c1a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:32.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.277+0000 7f037ad9d700 1 -- 172.21.15.96:0/2706803109 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f037c1a1d70 con 0x7f037c104480 2024-04-25T14:12:32.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.277+0000 7f037ad9d700 1 --2- 172.21.15.96:0/2706803109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f037c104480 0x7f037c19cea0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f036c00d980 tx=0x7f036c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:32.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.277+0000 7f0363fff700 1 -- 172.21.15.96:0/2706803109 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c016070 con 0x7f037c104480 2024-04-25T14:12:32.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.277+0000 7f0381208700 1 -- 172.21.15.96:0/2706803109 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f037c1a2060 con 0x7f037c104480 2024-04-25T14:12:32.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.277+0000 7f0363fff700 1 -- 172.21.15.96:0/2706803109 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f036c004830 con 0x7f037c104480 2024-04-25T14:12:32.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.277+0000 7f0381208700 1 -- 172.21.15.96:0/2706803109 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f037c1a9980 con 0x7f037c104480 2024-04-25T14:12:32.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.277+0000 7f0363fff700 1 -- 172.21.15.96:0/2706803109 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c005540 con 0x7f037c104480 2024-04-25T14:12:32.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.278+0000 7f0381208700 1 -- 172.21.15.96:0/2706803109 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0368005330 con 0x7f037c104480 2024-04-25T14:12:32.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.280+0000 7f0363fff700 1 -- 172.21.15.96:0/2706803109 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f036c004220 con 0x7f037c104480 2024-04-25T14:12:32.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.281+0000 7f0363fff700 1 --2- 172.21.15.96:0/2706803109 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f03640afc90 0x7f03640b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:32.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.281+0000 7f037a59c700 1 --2- 172.21.15.96:0/2706803109 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f03640afc90 0x7f03640b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:32.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.281+0000 7f0363fff700 1 -- 172.21.15.96:0/2706803109 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f036c0defd0 con 0x7f037c104480 2024-04-25T14:12:32.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.281+0000 7f037a59c700 1 --2- 172.21.15.96:0/2706803109 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f03640afc90 0x7f03640b21a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0370006010 tx=0x7f0370000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:32.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.285+0000 7f0363fff700 1 -- 172.21.15.96:0/2706803109 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f036c01f020 con 0x7f037c104480 2024-04-25T14:12:32.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.491+0000 7f0381208700 1 -- 172.21.15.96:0/2706803109 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f03680058f0 con 0x7f037c104480 2024-04-25T14:12:32.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.491+0000 7f0363fff700 1 -- 172.21.15.96:0/2706803109 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f036c0a7a30 con 0x7f037c104480 2024-04-25T14:12:32.492 INFO:teuthology.orchestra.run.smithi096.stdout:206158430254 2024-04-25T14:12:32.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.494+0000 7f0381208700 1 -- 172.21.15.96:0/2706803109 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f03640afc90 msgr2=0x7f03640b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:32.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.494+0000 7f0381208700 1 --2- 172.21.15.96:0/2706803109 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f03640afc90 0x7f03640b21a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0370006010 tx=0x7f0370000bc0 comp rx=0 tx=0).stop 2024-04-25T14:12:32.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.494+0000 7f0381208700 1 -- 172.21.15.96:0/2706803109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f037c104480 msgr2=0x7f037c19cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:32.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.494+0000 7f0381208700 1 --2- 172.21.15.96:0/2706803109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f037c104480 0x7f037c19cea0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f036c00d980 tx=0x7f036c00dc90 comp rx=0 tx=0).stop 2024-04-25T14:12:32.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.494+0000 7f0381208700 1 -- 172.21.15.96:0/2706803109 shutdown_connections 2024-04-25T14:12:32.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.494+0000 7f0381208700 1 --2- 172.21.15.96:0/2706803109 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f03640afc90 0x7f03640b21a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:32.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.494+0000 7f0381208700 1 --2- 172.21.15.96:0/2706803109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f037c109700 0x7f037c1a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:32.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.494+0000 7f0381208700 1 --2- 172.21.15.96:0/2706803109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f037c104480 0x7f037c19cea0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:32.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.494+0000 7f0381208700 1 --2- 172.21.15.96:0/2706803109 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f037c108db0 0x7f037c19d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:32.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.494+0000 7f0381208700 1 -- 172.21.15.96:0/2706803109 >> 172.21.15.96:0/2706803109 conn(0x7f037c0ffd70 msgr2=0x7f037c10a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:32.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.494+0000 7f0381208700 1 -- 172.21.15.96:0/2706803109 shutdown_connections 2024-04-25T14:12:32.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.494+0000 7f0381208700 1 -- 172.21.15.96:0/2706803109 wait complete. 2024-04-25T14:12:32.835 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:32 smithi096 ceph-mon[29207]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s; 74 B/s, 1 keys/s, 1 objects/s recovering 2024-04-25T14:12:32.835 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:32 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/2147267665' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T14:12:32.835 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:32 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/2706803109' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T14:12:32.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:32 smithi097 ceph-mon[32720]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s; 74 B/s, 1 keys/s, 1 objects/s recovering 2024-04-25T14:12:32.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:32 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/2147267665' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T14:12:32.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:32 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/2706803109' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T14:12:32.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:32 smithi151 ceph-mon[32990]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s; 74 B/s, 1 keys/s, 1 objects/s recovering 2024-04-25T14:12:32.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:32 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/2147267665' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T14:12:32.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:32 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/2706803109' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T14:12:32.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.972+0000 7ff43dc34700 1 -- 172.21.15.96:0/3309373275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4381085d0 msgr2=0x7ff43810ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:32.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.972+0000 7ff43dc34700 1 --2- 172.21.15.96:0/3309373275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4381085d0 0x7ff43810ff30 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff43000b0d0 tx=0x7ff43000b3e0 comp rx=0 tx=0).stop 2024-04-25T14:12:32.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.972+0000 7ff43dc34700 1 -- 172.21.15.96:0/3309373275 shutdown_connections 2024-04-25T14:12:32.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.972+0000 7ff43dc34700 1 --2- 172.21.15.96:0/3309373275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4381085d0 0x7ff43810ff30 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:32.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.972+0000 7ff43dc34700 1 --2- 172.21.15.96:0/3309373275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff438103350 0x7ff438103760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:32.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.972+0000 7ff43dc34700 1 --2- 172.21.15.96:0/3309373275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff438107c80 0x7ff438108090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:32.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.972+0000 7ff43dc34700 1 -- 172.21.15.96:0/3309373275 >> 172.21.15.96:0/3309373275 conn(0x7ff4380fec30 msgr2=0x7ff438101050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:32.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.972+0000 7ff43dc34700 1 -- 172.21.15.96:0/3309373275 shutdown_connections 2024-04-25T14:12:32.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.972+0000 7ff43dc34700 1 -- 172.21.15.96:0/3309373275 wait complete. 2024-04-25T14:12:32.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.973+0000 7ff43dc34700 1 Processor -- start 2024-04-25T14:12:32.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.973+0000 7ff43dc34700 1 -- start start 2024-04-25T14:12:32.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.973+0000 7ff43dc34700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff438103350 0x7ff43819bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:32.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.973+0000 7ff43dc34700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff438107c80 0x7ff43819c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:32.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.973+0000 7ff43dc34700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4381085d0 0x7ff43819c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:32.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.973+0000 7ff43dc34700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff438113200 con 0x7ff4381085d0 2024-04-25T14:12:32.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.973+0000 7ff43dc34700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff438113080 con 0x7ff438103350 2024-04-25T14:12:32.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.973+0000 7ff43dc34700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff438113380 con 0x7ff438107c80 2024-04-25T14:12:32.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.973+0000 7ff437fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4381085d0 0x7ff43819c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:32.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.973+0000 7ff437fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4381085d0 0x7ff43819c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49592/0 (socket says 172.21.15.96:49592) 2024-04-25T14:12:32.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.973+0000 7ff437fff700 1 -- 172.21.15.96:0/1353606501 learned_addr learned my addr 172.21.15.96:0/1353606501 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:12:32.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.974+0000 7ff436ffd700 1 --2- 172.21.15.96:0/1353606501 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff438107c80 0x7ff43819c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:32.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.974+0000 7ff4377fe700 1 --2- 172.21.15.96:0/1353606501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff438103350 0x7ff43819bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:32.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.974+0000 7ff437fff700 1 -- 172.21.15.96:0/1353606501 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff438107c80 msgr2=0x7ff43819c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:32.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.974+0000 7ff437fff700 1 --2- 172.21.15.96:0/1353606501 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff438107c80 0x7ff43819c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:32.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.974+0000 7ff437fff700 1 -- 172.21.15.96:0/1353606501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff438103350 msgr2=0x7ff43819bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:32.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.974+0000 7ff437fff700 1 --2- 172.21.15.96:0/1353606501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff438103350 0x7ff43819bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:32.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.974+0000 7ff437fff700 1 -- 172.21.15.96:0/1353606501 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4381a6de0 con 0x7ff4381085d0 2024-04-25T14:12:32.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.974+0000 7ff436ffd700 1 --2- 172.21.15.96:0/1353606501 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff438107c80 0x7ff43819c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:12:32.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.974+0000 7ff437fff700 1 --2- 172.21.15.96:0/1353606501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4381085d0 0x7ff43819c5c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff430009fd0 tx=0x7ff430007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:32.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.974+0000 7ff434ff9700 1 -- 172.21.15.96:0/1353606501 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43001f070 con 0x7ff4381085d0 2024-04-25T14:12:32.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.974+0000 7ff434ff9700 1 -- 172.21.15.96:0/1353606501 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff430003c00 con 0x7ff4381085d0 2024-04-25T14:12:32.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.974+0000 7ff43dc34700 1 -- 172.21.15.96:0/1353606501 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4381a7070 con 0x7ff4381085d0 2024-04-25T14:12:32.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.974+0000 7ff434ff9700 1 -- 172.21.15.96:0/1353606501 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43001b8e0 con 0x7ff4381085d0 2024-04-25T14:12:32.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.974+0000 7ff43dc34700 1 -- 172.21.15.96:0/1353606501 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4381a74c0 con 0x7ff4381085d0 2024-04-25T14:12:32.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.975+0000 7ff43dc34700 1 -- 172.21.15.96:0/1353606501 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff438051190 con 0x7ff4381085d0 2024-04-25T14:12:32.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.977+0000 7ff434ff9700 1 -- 172.21.15.96:0/1353606501 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7ff430022070 con 0x7ff4381085d0 2024-04-25T14:12:32.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.978+0000 7ff434ff9700 1 --2- 172.21.15.96:0/1353606501 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ff4200afc10 0x7ff4200b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:32.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.978+0000 7ff4377fe700 1 --2- 172.21.15.96:0/1353606501 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ff4200afc10 0x7ff4200b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:32.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.978+0000 7ff434ff9700 1 -- 172.21.15.96:0/1353606501 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7ff43000f020 con 0x7ff4381085d0 2024-04-25T14:12:32.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.978+0000 7ff4377fe700 1 --2- 172.21.15.96:0/1353606501 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ff4200afc10 0x7ff4200b2120 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff428005ea0 tx=0x7ff428005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:32.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.981+0000 7ff434ff9700 1 -- 172.21.15.96:0/1353606501 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff430029020 con 0x7ff4381085d0 2024-04-25T14:12:33.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.186+0000 7ff43dc34700 1 -- 172.21.15.96:0/1353606501 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7ff43802e510 con 0x7ff4381085d0 2024-04-25T14:12:33.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.187+0000 7ff434ff9700 1 -- 172.21.15.96:0/1353606501 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff4300a9050 con 0x7ff4381085d0 2024-04-25T14:12:33.187 INFO:teuthology.orchestra.run.smithi096.stdout:34359738450 2024-04-25T14:12:33.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.189+0000 7ff43dc34700 1 -- 172.21.15.96:0/1353606501 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ff4200afc10 msgr2=0x7ff4200b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:33.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.189+0000 7ff43dc34700 1 --2- 172.21.15.96:0/1353606501 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ff4200afc10 0x7ff4200b2120 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff428005ea0 tx=0x7ff428005e30 comp rx=0 tx=0).stop 2024-04-25T14:12:33.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.189+0000 7ff43dc34700 1 -- 172.21.15.96:0/1353606501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4381085d0 msgr2=0x7ff43819c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:33.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.189+0000 7ff43dc34700 1 --2- 172.21.15.96:0/1353606501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4381085d0 0x7ff43819c5c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff430009fd0 tx=0x7ff430007ad0 comp rx=0 tx=0).stop 2024-04-25T14:12:33.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.189+0000 7ff43dc34700 1 -- 172.21.15.96:0/1353606501 shutdown_connections 2024-04-25T14:12:33.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.189+0000 7ff43dc34700 1 --2- 172.21.15.96:0/1353606501 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ff4200afc10 0x7ff4200b2120 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:33.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.189+0000 7ff43dc34700 1 --2- 172.21.15.96:0/1353606501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4381085d0 0x7ff43819c5c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:33.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.189+0000 7ff43dc34700 1 --2- 172.21.15.96:0/1353606501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff438103350 0x7ff43819bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:33.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.189+0000 7ff43dc34700 1 --2- 172.21.15.96:0/1353606501 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff438107c80 0x7ff43819c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:33.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.189+0000 7ff43dc34700 1 -- 172.21.15.96:0/1353606501 >> 172.21.15.96:0/1353606501 conn(0x7ff4380fec30 msgr2=0x7ff4381073f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:33.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.189+0000 7ff43dc34700 1 -- 172.21.15.96:0/1353606501 shutdown_connections 2024-04-25T14:12:33.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.189+0000 7ff43dc34700 1 -- 172.21.15.96:0/1353606501 wait complete. 2024-04-25T14:12:33.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.776+0000 7f4011695700 1 -- 172.21.15.96:0/3897525075 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f400c102f90 msgr2=0x7f400c111020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:33.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.776+0000 7f4011695700 1 --2- 172.21.15.96:0/3897525075 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f400c102f90 0x7f400c111020 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f400400b0a0 tx=0x7f400400b3b0 comp rx=0 tx=0).stop 2024-04-25T14:12:33.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.776+0000 7f4011695700 1 -- 172.21.15.96:0/3897525075 shutdown_connections 2024-04-25T14:12:33.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.776+0000 7f4011695700 1 --2- 172.21.15.96:0/3897525075 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f400c102f90 0x7f400c111020 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:33.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.776+0000 7f4011695700 1 --2- 172.21.15.96:0/3897525075 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f400c1091b0 0x7f400c1095c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:33.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.776+0000 7f4011695700 1 --2- 172.21.15.96:0/3897525075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f400c102640 0x7f400c102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:33.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.776+0000 7f4011695700 1 -- 172.21.15.96:0/3897525075 >> 172.21.15.96:0/3897525075 conn(0x7f400c0fe370 msgr2=0x7f400c1007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:33.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.776+0000 7f4011695700 1 -- 172.21.15.96:0/3897525075 shutdown_connections 2024-04-25T14:12:33.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.776+0000 7f4011695700 1 -- 172.21.15.96:0/3897525075 wait complete. 2024-04-25T14:12:33.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.777+0000 7f4011695700 1 Processor -- start 2024-04-25T14:12:33.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.778+0000 7f4011695700 1 -- start start 2024-04-25T14:12:33.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.778+0000 7f4011695700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f400c102640 0x7f400c1a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:33.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.778+0000 7f4011695700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f400c102f90 0x7f400c1a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:33.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.778+0000 7f4011695700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f400c1091b0 0x7f400c19aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:33.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.778+0000 7f4011695700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f400c114450 con 0x7f400c1091b0 2024-04-25T14:12:33.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.778+0000 7f4011695700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f400c1142d0 con 0x7f400c102f90 2024-04-25T14:12:33.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.778+0000 7f4011695700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f400c1145d0 con 0x7f400c102640 2024-04-25T14:12:33.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.778+0000 7f400b7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f400c1091b0 0x7f400c19aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:33.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.778+0000 7f400b7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f400c1091b0 0x7f400c19aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49606/0 (socket says 172.21.15.96:49606) 2024-04-25T14:12:33.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.778+0000 7f400b7fe700 1 -- 172.21.15.96:0/3703370373 learned_addr learned my addr 172.21.15.96:0/3703370373 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:12:33.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.778+0000 7f400a7fc700 1 --2- 172.21.15.96:0/3703370373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f400c102f90 0x7f400c1a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:33.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.778+0000 7f400affd700 1 --2- 172.21.15.96:0/3703370373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f400c102640 0x7f400c1a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:33.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.778+0000 7f400b7fe700 1 -- 172.21.15.96:0/3703370373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f400c102640 msgr2=0x7f400c1a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:33.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.778+0000 7f400b7fe700 1 --2- 172.21.15.96:0/3703370373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f400c102640 0x7f400c1a08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:33.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.778+0000 7f400b7fe700 1 -- 172.21.15.96:0/3703370373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f400c102f90 msgr2=0x7f400c1a0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:33.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.778+0000 7f400b7fe700 1 --2- 172.21.15.96:0/3703370373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f400c102f90 0x7f400c1a0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:33.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.778+0000 7f400b7fe700 1 -- 172.21.15.96:0/3703370373 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f400c19b1d0 con 0x7f400c1091b0 2024-04-25T14:12:33.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.778+0000 7f400b7fe700 1 --2- 172.21.15.96:0/3703370373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f400c1091b0 0x7f400c19aaf0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f400400b830 tx=0x7f4004004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:33.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.779+0000 7f3ff3fff700 1 -- 172.21.15.96:0/3703370373 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4004021070 con 0x7f400c1091b0 2024-04-25T14:12:33.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.779+0000 7f4011695700 1 -- 172.21.15.96:0/3703370373 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f400c19b460 con 0x7f400c1091b0 2024-04-25T14:12:33.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.779+0000 7f3ff3fff700 1 -- 172.21.15.96:0/3703370373 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4004007e30 con 0x7f400c1091b0 2024-04-25T14:12:33.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.779+0000 7f3ff3fff700 1 -- 172.21.15.96:0/3703370373 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4004004700 con 0x7f400c1091b0 2024-04-25T14:12:33.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.779+0000 7f4011695700 1 -- 172.21.15.96:0/3703370373 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f400c1a7760 con 0x7f400c1091b0 2024-04-25T14:12:33.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.780+0000 7f4011695700 1 -- 172.21.15.96:0/3703370373 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f400c105960 con 0x7f400c1091b0 2024-04-25T14:12:33.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.781+0000 7f3ff3fff700 1 -- 172.21.15.96:0/3703370373 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f4004007980 con 0x7f400c1091b0 2024-04-25T14:12:33.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.782+0000 7f3ff3fff700 1 --2- 172.21.15.96:0/3703370373 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f3ff40afc10 0x7f3ff40b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:33.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.782+0000 7f400affd700 1 --2- 172.21.15.96:0/3703370373 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f3ff40afc10 0x7f3ff40b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:33.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.782+0000 7f3ff3fff700 1 -- 172.21.15.96:0/3703370373 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f400400f020 con 0x7f400c1091b0 2024-04-25T14:12:33.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.782+0000 7f400affd700 1 --2- 172.21.15.96:0/3703370373 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f3ff40afc10 0x7f3ff40b2120 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f400c19bc30 tx=0x7f3ffc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:33.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.786+0000 7f3ff3fff700 1 -- 172.21.15.96:0/3703370373 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f400402a020 con 0x7f400c1091b0 2024-04-25T14:12:33.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:33 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/1353606501' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T14:12:33.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:33 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/1353606501' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T14:12:33.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:33 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/1353606501' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T14:12:33.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.990+0000 7f4011695700 1 -- 172.21.15.96:0/3703370373 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f400c19bfe0 con 0x7f400c1091b0 2024-04-25T14:12:33.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.991+0000 7f3ff3fff700 1 -- 172.21.15.96:0/3703370373 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f40040a8e00 con 0x7f400c1091b0 2024-04-25T14:12:33.990 INFO:teuthology.orchestra.run.smithi096.stdout:55834574926 2024-04-25T14:12:33.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.993+0000 7f4011695700 1 -- 172.21.15.96:0/3703370373 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f3ff40afc10 msgr2=0x7f3ff40b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:33.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.993+0000 7f4011695700 1 --2- 172.21.15.96:0/3703370373 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f3ff40afc10 0x7f3ff40b2120 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f400c19bc30 tx=0x7f3ffc00b410 comp rx=0 tx=0).stop 2024-04-25T14:12:33.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.993+0000 7f4011695700 1 -- 172.21.15.96:0/3703370373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f400c1091b0 msgr2=0x7f400c19aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:33.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.993+0000 7f4011695700 1 --2- 172.21.15.96:0/3703370373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f400c1091b0 0x7f400c19aaf0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f400400b830 tx=0x7f4004004ce0 comp rx=0 tx=0).stop 2024-04-25T14:12:33.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.993+0000 7f4011695700 1 -- 172.21.15.96:0/3703370373 shutdown_connections 2024-04-25T14:12:33.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.993+0000 7f4011695700 1 --2- 172.21.15.96:0/3703370373 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f3ff40afc10 0x7f3ff40b2120 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:33.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.993+0000 7f4011695700 1 --2- 172.21.15.96:0/3703370373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f400c1091b0 0x7f400c19aaf0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:33.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.993+0000 7f4011695700 1 --2- 172.21.15.96:0/3703370373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f400c102640 0x7f400c1a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:33.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.993+0000 7f4011695700 1 --2- 172.21.15.96:0/3703370373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f400c102f90 0x7f400c1a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:33.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.993+0000 7f4011695700 1 -- 172.21.15.96:0/3703370373 >> 172.21.15.96:0/3703370373 conn(0x7f400c0fe370 msgr2=0x7f400c0fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:33.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.993+0000 7f4011695700 1 -- 172.21.15.96:0/3703370373 shutdown_connections 2024-04-25T14:12:33.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:33.993+0000 7f4011695700 1 -- 172.21.15.96:0/3703370373 wait complete. 2024-04-25T14:12:34.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.164+0000 7f0efd5b9700 1 -- 172.21.15.96:0/3049769927 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ef8107c90 msgr2=0x7f0ef81080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:34.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.164+0000 7f0efd5b9700 1 --2- 172.21.15.96:0/3049769927 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ef8107c90 0x7f0ef81080a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0eec0099c0 tx=0x7f0eec009cd0 comp rx=0 tx=0).stop 2024-04-25T14:12:34.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.164+0000 7f0efd5b9700 1 -- 172.21.15.96:0/3049769927 shutdown_connections 2024-04-25T14:12:34.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.164+0000 7f0efd5b9700 1 --2- 172.21.15.96:0/3049769927 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ef81085e0 0x7f0ef810ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:34.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.164+0000 7f0efd5b9700 1 --2- 172.21.15.96:0/3049769927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ef8103360 0x7f0ef8103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:34.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.164+0000 7f0efd5b9700 1 --2- 172.21.15.96:0/3049769927 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ef8107c90 0x7f0ef81080a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:34.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.164+0000 7f0efd5b9700 1 -- 172.21.15.96:0/3049769927 >> 172.21.15.96:0/3049769927 conn(0x7f0ef80fec90 msgr2=0x7f0ef81010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:34.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.164+0000 7f0efd5b9700 1 -- 172.21.15.96:0/3049769927 shutdown_connections 2024-04-25T14:12:34.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.164+0000 7f0efd5b9700 1 -- 172.21.15.96:0/3049769927 wait complete. 2024-04-25T14:12:34.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.164+0000 7f0efd5b9700 1 Processor -- start 2024-04-25T14:12:34.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.165+0000 7f0efd5b9700 1 -- start start 2024-04-25T14:12:34.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.165+0000 7f0efd5b9700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ef8103360 0x7f0ef81a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:34.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.165+0000 7f0efd5b9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ef8107c90 0x7f0ef81a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:34.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.165+0000 7f0efd5b9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ef81085e0 0x7f0ef819aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:34.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.165+0000 7f0efd5b9700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef81131f0 con 0x7f0ef81085e0 2024-04-25T14:12:34.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.165+0000 7f0efd5b9700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef8113070 con 0x7f0ef8107c90 2024-04-25T14:12:34.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.165+0000 7f0efd5b9700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef8113370 con 0x7f0ef8103360 2024-04-25T14:12:34.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.165+0000 7f0ef77fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ef81085e0 0x7f0ef819aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:34.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.165+0000 7f0ef77fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ef81085e0 0x7f0ef819aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49568/0 (socket says 172.21.15.96:49568) 2024-04-25T14:12:34.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.165+0000 7f0ef77fe700 1 -- 172.21.15.96:0/2147267665 learned_addr learned my addr 172.21.15.96:0/2147267665 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:12:34.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.165+0000 7f0ef6ffd700 1 --2- 172.21.15.96:0/2147267665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ef8103360 0x7f0ef81a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:34.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.165+0000 7f0ef67fc700 1 --2- 172.21.15.96:0/2147267665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ef8107c90 0x7f0ef81a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:34.315 INFO:teuthology.orchestra.run.smithi096.stdout:137438953534 2024-04-25T14:12:34.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.165+0000 7f0ef77fe700 1 -- 172.21.15.96:0/2147267665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ef8103360 msgr2=0x7f0ef81a08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:34.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.165+0000 7f0ef77fe700 1 --2- 172.21.15.96:0/2147267665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ef8103360 0x7f0ef81a08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:34.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.166+0000 7f0ef77fe700 1 -- 172.21.15.96:0/2147267665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ef8107c90 msgr2=0x7f0ef81a0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:34.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.166+0000 7f0ef77fe700 1 --2- 172.21.15.96:0/2147267665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ef8107c90 0x7f0ef81a0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:34.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.166+0000 7f0ef77fe700 1 -- 172.21.15.96:0/2147267665 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef819b100 con 0x7f0ef81085e0 2024-04-25T14:12:34.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.166+0000 7f0ef77fe700 1 --2- 172.21.15.96:0/2147267665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ef81085e0 0x7f0ef819aac0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0007d70 tx=0x7f0ef000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:34.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.166+0000 7f0edffff700 1 -- 172.21.15.96:0/2147267665 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef0016020 con 0x7f0ef81085e0 2024-04-25T14:12:34.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.166+0000 7f0efd5b9700 1 -- 172.21.15.96:0/2147267665 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ef819b3f0 con 0x7f0ef81085e0 2024-04-25T14:12:34.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.166+0000 7f0edffff700 1 -- 172.21.15.96:0/2147267665 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ef000ac90 con 0x7f0ef81085e0 2024-04-25T14:12:34.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.166+0000 7f0edffff700 1 -- 172.21.15.96:0/2147267665 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef00115e0 con 0x7f0ef81085e0 2024-04-25T14:12:34.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.166+0000 7f0efd5b9700 1 -- 172.21.15.96:0/2147267665 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ef819b900 con 0x7f0ef81085e0 2024-04-25T14:12:34.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.167+0000 7f0efd5b9700 1 -- 172.21.15.96:0/2147267665 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ef8051190 con 0x7f0ef81085e0 2024-04-25T14:12:34.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.170+0000 7f0edffff700 1 -- 172.21.15.96:0/2147267665 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f0ef0019020 con 0x7f0ef81085e0 2024-04-25T14:12:34.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.171+0000 7f0edffff700 1 --2- 172.21.15.96:0/2147267665 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f0ee00afc10 0x7f0ee00b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:34.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.171+0000 7f0ef6ffd700 1 --2- 172.21.15.96:0/2147267665 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f0ee00afc10 0x7f0ee00b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:34.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.171+0000 7f0edffff700 1 -- 172.21.15.96:0/2147267665 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f0ef00df050 con 0x7f0ef81085e0 2024-04-25T14:12:34.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.171+0000 7f0ef6ffd700 1 --2- 172.21.15.96:0/2147267665 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f0ee00afc10 0x7f0ee00b2120 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0ef819bc00 tx=0x7f0ee800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:34.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.174+0000 7f0edffff700 1 -- 172.21.15.96:0/2147267665 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0ef0020020 con 0x7f0ef81085e0 2024-04-25T14:12:34.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.385+0000 7f0efd5b9700 1 -- 172.21.15.96:0/2147267665 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f0ef819bff0 con 0x7f0ef81085e0 2024-04-25T14:12:34.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.385+0000 7f0edffff700 1 -- 172.21.15.96:0/2147267665 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0ef00a7bc0 con 0x7f0ef81085e0 2024-04-25T14:12:34.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.387+0000 7f0efd5b9700 1 -- 172.21.15.96:0/2147267665 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f0ee00afc10 msgr2=0x7f0ee00b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:34.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.387+0000 7f0efd5b9700 1 --2- 172.21.15.96:0/2147267665 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f0ee00afc10 0x7f0ee00b2120 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0ef819bc00 tx=0x7f0ee800b410 comp rx=0 tx=0).stop 2024-04-25T14:12:34.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.387+0000 7f0efd5b9700 1 -- 172.21.15.96:0/2147267665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ef81085e0 msgr2=0x7f0ef819aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:34.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.387+0000 7f0efd5b9700 1 --2- 172.21.15.96:0/2147267665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ef81085e0 0x7f0ef819aac0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0007d70 tx=0x7f0ef000a340 comp rx=0 tx=0).stop 2024-04-25T14:12:34.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.387+0000 7f0efd5b9700 1 -- 172.21.15.96:0/2147267665 shutdown_connections 2024-04-25T14:12:34.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.387+0000 7f0efd5b9700 1 --2- 172.21.15.96:0/2147267665 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f0ee00afc10 0x7f0ee00b2120 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:34.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.387+0000 7f0efd5b9700 1 --2- 172.21.15.96:0/2147267665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ef81085e0 0x7f0ef819aac0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:34.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.387+0000 7f0efd5b9700 1 --2- 172.21.15.96:0/2147267665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ef8103360 0x7f0ef81a08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:34.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.387+0000 7f0efd5b9700 1 --2- 172.21.15.96:0/2147267665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ef8107c90 0x7f0ef81a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:34.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.387+0000 7f0efd5b9700 1 -- 172.21.15.96:0/2147267665 >> 172.21.15.96:0/2147267665 conn(0x7f0ef80fec90 msgr2=0x7f0ef806b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:34.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.387+0000 7f0efd5b9700 1 -- 172.21.15.96:0/2147267665 shutdown_connections 2024-04-25T14:12:34.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:32.387+0000 7f0efd5b9700 1 -- 172.21.15.96:0/2147267665 wait complete. 2024-04-25T14:12:34.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:34 smithi096 ceph-mon[29207]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s; 67 B/s, 0 keys/s, 1 objects/s recovering 2024-04-25T14:12:34.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:34 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/3703370373' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T14:12:34.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:34 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/3647352628' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T14:12:34.912 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:34 smithi097 ceph-mon[32720]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s; 67 B/s, 0 keys/s, 1 objects/s recovering 2024-04-25T14:12:34.913 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:34 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/3703370373' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T14:12:34.913 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:34 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/3647352628' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T14:12:34.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:34 smithi151 ceph-mon[32990]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s; 67 B/s, 0 keys/s, 1 objects/s recovering 2024-04-25T14:12:34.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:34 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/3703370373' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T14:12:34.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:34 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/3647352628' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T14:12:35.020 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953528 got 137438953534 for osd.4 2024-04-25T14:12:35.021 DEBUG:teuthology.parallel:result is None 2024-04-25T14:12:35.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.297+0000 7fa116699700 1 -- 172.21.15.96:0/749381719 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa110107c90 msgr2=0x7fa1101080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:35.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.297+0000 7fa116699700 1 --2- 172.21.15.96:0/749381719 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa110107c90 0x7fa1101080a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa1040099c0 tx=0x7fa104009cd0 comp rx=0 tx=0).stop 2024-04-25T14:12:35.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.298+0000 7fa116699700 1 -- 172.21.15.96:0/749381719 shutdown_connections 2024-04-25T14:12:35.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.298+0000 7fa116699700 1 --2- 172.21.15.96:0/749381719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1101085e0 0x7fa11010ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:35.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.298+0000 7fa116699700 1 --2- 172.21.15.96:0/749381719 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa110103360 0x7fa110103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:35.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.298+0000 7fa116699700 1 --2- 172.21.15.96:0/749381719 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa110107c90 0x7fa1101080a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:35.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.298+0000 7fa116699700 1 -- 172.21.15.96:0/749381719 >> 172.21.15.96:0/749381719 conn(0x7fa1100fec90 msgr2=0x7fa1101010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:35.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.298+0000 7fa116699700 1 -- 172.21.15.96:0/749381719 shutdown_connections 2024-04-25T14:12:35.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.298+0000 7fa116699700 1 -- 172.21.15.96:0/749381719 wait complete. 2024-04-25T14:12:35.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.298+0000 7fa116699700 1 Processor -- start 2024-04-25T14:12:35.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.299+0000 7fa116699700 1 -- start start 2024-04-25T14:12:35.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.299+0000 7fa116699700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa110103360 0x7fa11019f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:35.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.299+0000 7fa116699700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa110107c90 0x7fa11019fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:35.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.299+0000 7fa116699700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1101085e0 0x7fa1101999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:35.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.299+0000 7fa116699700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa1101131f0 con 0x7fa110107c90 2024-04-25T14:12:35.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.299+0000 7fa116699700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa110113070 con 0x7fa110103360 2024-04-25T14:12:35.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.299+0000 7fa116699700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa110113370 con 0x7fa1101085e0 2024-04-25T14:12:35.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.299+0000 7fa10f7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa110107c90 0x7fa11019fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:35.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.299+0000 7fa10f7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa110107c90 0x7fa11019fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49626/0 (socket says 172.21.15.96:49626) 2024-04-25T14:12:35.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.299+0000 7fa10ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa110103360 0x7fa11019f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:35.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.299+0000 7fa10f7fe700 1 -- 172.21.15.96:0/3647352628 learned_addr learned my addr 172.21.15.96:0/3647352628 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:12:35.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.299+0000 7fa114c36700 1 --2- 172.21.15.96:0/3647352628 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1101085e0 0x7fa1101999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:35.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.299+0000 7fa10f7fe700 1 -- 172.21.15.96:0/3647352628 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1101085e0 msgr2=0x7fa1101999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:35.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.299+0000 7fa10f7fe700 1 --2- 172.21.15.96:0/3647352628 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1101085e0 0x7fa1101999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:35.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.299+0000 7fa10f7fe700 1 -- 172.21.15.96:0/3647352628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa110103360 msgr2=0x7fa11019f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:35.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.299+0000 7fa10f7fe700 1 --2- 172.21.15.96:0/3647352628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa110103360 0x7fa11019f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:35.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.299+0000 7fa10f7fe700 1 -- 172.21.15.96:0/3647352628 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa11019a080 con 0x7fa110107c90 2024-04-25T14:12:35.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.300+0000 7fa10ffff700 1 --2- 172.21.15.96:0/3647352628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa110103360 0x7fa11019f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:12:35.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.300+0000 7fa10f7fe700 1 --2- 172.21.15.96:0/3647352628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa110107c90 0x7fa11019fcf0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa104005f50 tx=0x7fa104004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:35.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.300+0000 7fa10d7fa700 1 -- 172.21.15.96:0/3647352628 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa104026070 con 0x7fa110107c90 2024-04-25T14:12:35.129 INFO:teuthology.orchestra.run.smithi096.stdout:257698037796 2024-04-25T14:12:35.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.300+0000 7fa116699700 1 -- 172.21.15.96:0/3647352628 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa11019a310 con 0x7fa110107c90 2024-04-25T14:12:35.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.300+0000 7fa10d7fa700 1 -- 172.21.15.96:0/3647352628 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa104003c70 con 0x7fa110107c90 2024-04-25T14:12:35.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.300+0000 7fa10d7fa700 1 -- 172.21.15.96:0/3647352628 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10400f9c0 con 0x7fa110107c90 2024-04-25T14:12:35.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.300+0000 7fa116699700 1 -- 172.21.15.96:0/3647352628 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1101a6640 con 0x7fa110107c90 2024-04-25T14:12:35.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.301+0000 7fa116699700 1 -- 172.21.15.96:0/3647352628 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa110051190 con 0x7fa110107c90 2024-04-25T14:12:35.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.303+0000 7fa10d7fa700 1 -- 172.21.15.96:0/3647352628 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fa104029070 con 0x7fa110107c90 2024-04-25T14:12:35.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.303+0000 7fa10d7fa700 1 --2- 172.21.15.96:0/3647352628 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa0f80afc10 0x7fa0f80b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:35.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.304+0000 7fa10ffff700 1 --2- 172.21.15.96:0/3647352628 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa0f80afc10 0x7fa0f80b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:35.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.304+0000 7fa10d7fa700 1 -- 172.21.15.96:0/3647352628 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fa1040df9c0 con 0x7fa110107c90 2024-04-25T14:12:35.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.304+0000 7fa10ffff700 1 --2- 172.21.15.96:0/3647352628 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa0f80afc10 0x7fa0f80b2120 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa11019ab10 tx=0x7fa100009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:35.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.307+0000 7fa10d7fa700 1 -- 172.21.15.96:0/3647352628 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa104030080 con 0x7fa110107c90 2024-04-25T14:12:35.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.507+0000 7fa116699700 1 -- 172.21.15.96:0/3647352628 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fa11002e510 con 0x7fa110107c90 2024-04-25T14:12:35.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.507+0000 7fa10d7fa700 1 -- 172.21.15.96:0/3647352628 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa104013070 con 0x7fa110107c90 2024-04-25T14:12:35.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.510+0000 7fa116699700 1 -- 172.21.15.96:0/3647352628 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa0f80afc10 msgr2=0x7fa0f80b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:35.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.510+0000 7fa116699700 1 --2- 172.21.15.96:0/3647352628 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa0f80afc10 0x7fa0f80b2120 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa11019ab10 tx=0x7fa100009260 comp rx=0 tx=0).stop 2024-04-25T14:12:35.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.510+0000 7fa116699700 1 -- 172.21.15.96:0/3647352628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa110107c90 msgr2=0x7fa11019fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:35.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.510+0000 7fa116699700 1 --2- 172.21.15.96:0/3647352628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa110107c90 0x7fa11019fcf0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa104005f50 tx=0x7fa104004270 comp rx=0 tx=0).stop 2024-04-25T14:12:35.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.510+0000 7fa116699700 1 -- 172.21.15.96:0/3647352628 shutdown_connections 2024-04-25T14:12:35.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.510+0000 7fa116699700 1 --2- 172.21.15.96:0/3647352628 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa0f80afc10 0x7fa0f80b2120 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:35.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.510+0000 7fa116699700 1 --2- 172.21.15.96:0/3647352628 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1101085e0 0x7fa1101999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:35.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.510+0000 7fa116699700 1 --2- 172.21.15.96:0/3647352628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa110103360 0x7fa11019f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:35.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.510+0000 7fa116699700 1 --2- 172.21.15.96:0/3647352628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa110107c90 0x7fa11019fcf0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:35.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.510+0000 7fa116699700 1 -- 172.21.15.96:0/3647352628 >> 172.21.15.96:0/3647352628 conn(0x7fa1100fec90 msgr2=0x7fa1101010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:35.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.510+0000 7fa116699700 1 -- 172.21.15.96:0/3647352628 shutdown_connections 2024-04-25T14:12:35.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:34.511+0000 7fa116699700 1 -- 172.21.15.96:0/3647352628 wait complete. 2024-04-25T14:12:35.361 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:12:35.849 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037792 got 257698037796 for osd.9 2024-04-25T14:12:35.849 DEBUG:teuthology.parallel:result is None 2024-04-25T14:12:35.918 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430249 got 206158430254 for osd.7 2024-04-25T14:12:35.918 DEBUG:teuthology.parallel:result is None 2024-04-25T14:12:36.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:36 smithi096 ceph-mon[29207]: pgmap v315: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 61 B/s, 0 keys/s, 1 objects/s recovering 2024-04-25T14:12:36.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:36 smithi097 ceph-mon[32720]: pgmap v315: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 61 B/s, 0 keys/s, 1 objects/s recovering 2024-04-25T14:12:36.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:36 smithi151 ceph-mon[32990]: pgmap v315: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 61 B/s, 0 keys/s, 1 objects/s recovering 2024-04-25T14:12:37.281 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574921 got 55834574926 for osd.1 2024-04-25T14:12:37.281 DEBUG:teuthology.parallel:result is None 2024-04-25T14:12:37.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.397+0000 7fe0fc383700 1 -- 172.21.15.96:0/2837392437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0f4103410 msgr2=0x7fe0f4103820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:37.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.397+0000 7fe0fc383700 1 --2- 172.21.15.96:0/2837392437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0f4103410 0x7fe0f4103820 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec00b0d0 tx=0x7fe0ec00b3e0 comp rx=0 tx=0).stop 2024-04-25T14:12:37.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.397+0000 7fe0fc383700 1 -- 172.21.15.96:0/2837392437 shutdown_connections 2024-04-25T14:12:37.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.397+0000 7fe0fc383700 1 --2- 172.21.15.96:0/2837392437 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0f4108640 0x7fe0f410ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:37.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.397+0000 7fe0fc383700 1 --2- 172.21.15.96:0/2837392437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0f4103410 0x7fe0f4103820 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:37.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.397+0000 7fe0fc383700 1 --2- 172.21.15.96:0/2837392437 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0f4107cf0 0x7fe0f4108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:37.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.397+0000 7fe0fc383700 1 -- 172.21.15.96:0/2837392437 >> 172.21.15.96:0/2837392437 conn(0x7fe0f4078d20 msgr2=0x7fe0f4079160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:37.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.397+0000 7fe0fc383700 1 -- 172.21.15.96:0/2837392437 shutdown_connections 2024-04-25T14:12:37.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.397+0000 7fe0fc383700 1 -- 172.21.15.96:0/2837392437 wait complete. 2024-04-25T14:12:37.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.398+0000 7fe0fc383700 1 Processor -- start 2024-04-25T14:12:37.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.398+0000 7fe0fc383700 1 -- start start 2024-04-25T14:12:37.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.398+0000 7fe0fc383700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0f4103410 0x7fe0f419f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:37.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.398+0000 7fe0fc383700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0f4107cf0 0x7fe0f419fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:37.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.398+0000 7fe0fc383700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0f4108640 0x7fe0f4199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:37.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.398+0000 7fe0fc383700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f4113270 con 0x7fe0f4103410 2024-04-25T14:12:37.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.398+0000 7fe0fc383700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f41130f0 con 0x7fe0f4107cf0 2024-04-25T14:12:37.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.398+0000 7fe0fc383700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f41133f0 con 0x7fe0f4108640 2024-04-25T14:12:37.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.398+0000 7fe0fa11f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0f4103410 0x7fe0f419f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:37.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.398+0000 7fe0fa11f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0f4103410 0x7fe0f419f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49640/0 (socket says 172.21.15.96:49640) 2024-04-25T14:12:37.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.398+0000 7fe0fa11f700 1 -- 172.21.15.96:0/1549335471 learned_addr learned my addr 172.21.15.96:0/1549335471 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:12:37.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.398+0000 7fe0f991e700 1 --2- 172.21.15.96:0/1549335471 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0f4107cf0 0x7fe0f419fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:37.818 INFO:teuthology.orchestra.run.smithi096.stdout:279172874272 2024-04-25T14:12:37.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.398+0000 7fe0fa920700 1 --2- 172.21.15.96:0/1549335471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0f4108640 0x7fe0f4199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:37.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.398+0000 7fe0fa11f700 1 -- 172.21.15.96:0/1549335471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0f4108640 msgr2=0x7fe0f4199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:37.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.398+0000 7fe0fa11f700 1 --2- 172.21.15.96:0/1549335471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0f4108640 0x7fe0f4199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:37.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.398+0000 7fe0fa11f700 1 -- 172.21.15.96:0/1549335471 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0f4107cf0 msgr2=0x7fe0f419fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:37.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.399+0000 7fe0fa11f700 1 --2- 172.21.15.96:0/1549335471 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0f4107cf0 0x7fe0f419fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:37.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.399+0000 7fe0fa11f700 1 -- 172.21.15.96:0/1549335471 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0f419a090 con 0x7fe0f4103410 2024-04-25T14:12:37.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.399+0000 7fe0fa920700 1 --2- 172.21.15.96:0/1549335471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0f4108640 0x7fe0f4199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:12:37.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.399+0000 7fe0fa11f700 1 --2- 172.21.15.96:0/1549335471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0f4103410 0x7fe0f419f830 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec007a20 tx=0x7fe0ec007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:37.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.399+0000 7fe0eb7fe700 1 -- 172.21.15.96:0/1549335471 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec01f070 con 0x7fe0f4103410 2024-04-25T14:12:37.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.399+0000 7fe0eb7fe700 1 -- 172.21.15.96:0/1549335471 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0ec003c00 con 0x7fe0f4103410 2024-04-25T14:12:37.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.399+0000 7fe0fc383700 1 -- 172.21.15.96:0/1549335471 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0f419a320 con 0x7fe0f4103410 2024-04-25T14:12:37.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.399+0000 7fe0eb7fe700 1 -- 172.21.15.96:0/1549335471 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec01b730 con 0x7fe0f4103410 2024-04-25T14:12:37.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.399+0000 7fe0fc383700 1 -- 172.21.15.96:0/1549335471 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0f419a850 con 0x7fe0f4103410 2024-04-25T14:12:37.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.400+0000 7fe0fc383700 1 -- 172.21.15.96:0/1549335471 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0f4051190 con 0x7fe0f4103410 2024-04-25T14:12:37.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.401+0000 7fe0eb7fe700 1 -- 172.21.15.96:0/1549335471 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fe0ec022070 con 0x7fe0f4103410 2024-04-25T14:12:37.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.402+0000 7fe0eb7fe700 1 --2- 172.21.15.96:0/1549335471 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fe0e00afc90 0x7fe0e00b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:37.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.402+0000 7fe0f991e700 1 --2- 172.21.15.96:0/1549335471 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fe0e00afc90 0x7fe0e00b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:37.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.402+0000 7fe0eb7fe700 1 -- 172.21.15.96:0/1549335471 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fe0ec00f020 con 0x7fe0f4103410 2024-04-25T14:12:37.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.402+0000 7fe0f991e700 1 --2- 172.21.15.96:0/1549335471 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fe0e00afc90 0x7fe0e00b21a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe0f406b890 tx=0x7fe0e400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:37.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.406+0000 7fe0eb7fe700 1 -- 172.21.15.96:0/1549335471 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe0ec029020 con 0x7fe0f4103410 2024-04-25T14:12:37.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.612+0000 7fe0fc383700 1 -- 172.21.15.96:0/1549335471 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fe0f419af80 con 0x7fe0f4103410 2024-04-25T14:12:37.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.612+0000 7fe0eb7fe700 1 -- 172.21.15.96:0/1549335471 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fe0ec0a8f70 con 0x7fe0f4103410 2024-04-25T14:12:37.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.614+0000 7fe0fc383700 1 -- 172.21.15.96:0/1549335471 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fe0e00afc90 msgr2=0x7fe0e00b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:37.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.614+0000 7fe0fc383700 1 --2- 172.21.15.96:0/1549335471 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fe0e00afc90 0x7fe0e00b21a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe0f406b890 tx=0x7fe0e400a380 comp rx=0 tx=0).stop 2024-04-25T14:12:37.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.614+0000 7fe0fc383700 1 -- 172.21.15.96:0/1549335471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0f4103410 msgr2=0x7fe0f419f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:37.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.614+0000 7fe0fc383700 1 --2- 172.21.15.96:0/1549335471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0f4103410 0x7fe0f419f830 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec007a20 tx=0x7fe0ec007a50 comp rx=0 tx=0).stop 2024-04-25T14:12:37.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.614+0000 7fe0fc383700 1 -- 172.21.15.96:0/1549335471 shutdown_connections 2024-04-25T14:12:37.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.614+0000 7fe0fc383700 1 --2- 172.21.15.96:0/1549335471 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fe0e00afc90 0x7fe0e00b21a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:37.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.614+0000 7fe0fc383700 1 --2- 172.21.15.96:0/1549335471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0f4108640 0x7fe0f4199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:37.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.614+0000 7fe0fc383700 1 --2- 172.21.15.96:0/1549335471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0f4103410 0x7fe0f419f830 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:37.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.614+0000 7fe0fc383700 1 --2- 172.21.15.96:0/1549335471 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0f4107cf0 0x7fe0f419fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:37.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.614+0000 7fe0fc383700 1 -- 172.21.15.96:0/1549335471 >> 172.21.15.96:0/1549335471 conn(0x7fe0f4078d20 msgr2=0x7fe0f41007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:37.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.614+0000 7fe0fc383700 1 -- 172.21.15.96:0/1549335471 shutdown_connections 2024-04-25T14:12:37.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:36.614+0000 7fe0fc383700 1 -- 172.21.15.96:0/1549335471 wait complete. 2024-04-25T14:12:37.825 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:37 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/1549335471' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T14:12:37.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:37 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/1549335471' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T14:12:37.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:37 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/1549335471' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T14:12:38.804 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:38 smithi096 ceph-mon[29207]: pgmap v316: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2024-04-25T14:12:38.835 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738444 got 34359738450 for osd.0 2024-04-25T14:12:38.835 DEBUG:teuthology.parallel:result is None 2024-04-25T14:12:38.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:38 smithi097 ceph-mon[32720]: pgmap v316: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2024-04-25T14:12:38.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:38 smithi151 ceph-mon[32990]: pgmap v316: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2024-04-25T14:12:39.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.026+0000 7f9656b66700 1 -- 172.21.15.96:0/1199251334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96501085d0 msgr2=0x7f965010ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:39.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.026+0000 7f9656b66700 1 --2- 172.21.15.96:0/1199251334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96501085d0 0x7f965010ff30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f964800b0d0 tx=0x7f964800b3e0 comp rx=0 tx=0).stop 2024-04-25T14:12:39.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.027+0000 7f9656b66700 1 -- 172.21.15.96:0/1199251334 shutdown_connections 2024-04-25T14:12:39.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.027+0000 7f9656b66700 1 --2- 172.21.15.96:0/1199251334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96501085d0 0x7f965010ff30 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:39.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.027+0000 7f9656b66700 1 --2- 172.21.15.96:0/1199251334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9650103350 0x7f9650103760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:39.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.027+0000 7f9656b66700 1 --2- 172.21.15.96:0/1199251334 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9650107c80 0x7f9650108090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:39.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.027+0000 7f9656b66700 1 -- 172.21.15.96:0/1199251334 >> 172.21.15.96:0/1199251334 conn(0x7f96500fec30 msgr2=0x7f9650101050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:39.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.027+0000 7f9656b66700 1 -- 172.21.15.96:0/1199251334 shutdown_connections 2024-04-25T14:12:39.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.027+0000 7f9656b66700 1 -- 172.21.15.96:0/1199251334 wait complete. 2024-04-25T14:12:39.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.027+0000 7f9656b66700 1 Processor -- start 2024-04-25T14:12:39.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.027+0000 7f9656b66700 1 -- start start 2024-04-25T14:12:39.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.027+0000 7f9656b66700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9650103350 0x7f965019bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:39.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.028+0000 7f9656b66700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9650107c80 0x7f965019c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:39.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.028+0000 7f9656b66700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96501085d0 0x7f965019c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:39.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.028+0000 7f9656b66700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9650113270 con 0x7f9650103350 2024-04-25T14:12:39.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.028+0000 7f9656b66700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f96501130f0 con 0x7f96501085d0 2024-04-25T14:12:39.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.028+0000 7f9656b66700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f96501133f0 con 0x7f9650107c80 2024-04-25T14:12:39.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.028+0000 7f9654902700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9650103350 0x7f965019bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:39.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.028+0000 7f9654902700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9650103350 0x7f965019bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49644/0 (socket says 172.21.15.96:49644) 2024-04-25T14:12:39.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.028+0000 7f9654902700 1 -- 172.21.15.96:0/137095753 learned_addr learned my addr 172.21.15.96:0/137095753 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:12:39.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.028+0000 7f9655103700 1 --2- 172.21.15.96:0/137095753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96501085d0 0x7f965019c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:39.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.028+0000 7f964ffff700 1 --2- 172.21.15.96:0/137095753 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9650107c80 0x7f965019c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:39.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.028+0000 7f9654902700 1 -- 172.21.15.96:0/137095753 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9650107c80 msgr2=0x7f965019c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:39.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.028+0000 7f9654902700 1 --2- 172.21.15.96:0/137095753 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9650107c80 0x7f965019c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:39.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.028+0000 7f9654902700 1 -- 172.21.15.96:0/137095753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96501085d0 msgr2=0x7f965019c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:39.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.028+0000 7f9654902700 1 --2- 172.21.15.96:0/137095753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96501085d0 0x7f965019c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:39.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.028+0000 7f9654902700 1 -- 172.21.15.96:0/137095753 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96501a6ec0 con 0x7f9650103350 2024-04-25T14:12:39.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.028+0000 7f9654902700 1 --2- 172.21.15.96:0/137095753 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9650103350 0x7f965019bb50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f964400d950 tx=0x7f964400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:39.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.028+0000 7f964dffb700 1 -- 172.21.15.96:0/137095753 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9644016070 con 0x7f9650103350 2024-04-25T14:12:39.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.028+0000 7f964dffb700 1 -- 172.21.15.96:0/137095753 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9644009df0 con 0x7f9650103350 2024-04-25T14:12:39.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.028+0000 7f9656b66700 1 -- 172.21.15.96:0/137095753 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96501a71b0 con 0x7f9650103350 2024-04-25T14:12:39.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.028+0000 7f964dffb700 1 -- 172.21.15.96:0/137095753 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964400f3c0 con 0x7f9650103350 2024-04-25T14:12:39.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.029+0000 7f9656b66700 1 -- 172.21.15.96:0/137095753 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96501a76f0 con 0x7f9650103350 2024-04-25T14:12:39.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.030+0000 7f9656b66700 1 -- 172.21.15.96:0/137095753 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9650051190 con 0x7f9650103350 2024-04-25T14:12:39.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.032+0000 7f964dffb700 1 -- 172.21.15.96:0/137095753 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f9644019070 con 0x7f9650103350 2024-04-25T14:12:39.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.032+0000 7f964dffb700 1 --2- 172.21.15.96:0/137095753 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f96400afc10 0x7f96400b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:39.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.032+0000 7f964ffff700 1 --2- 172.21.15.96:0/137095753 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f96400afc10 0x7f96400b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:39.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.033+0000 7f964dffb700 1 -- 172.21.15.96:0/137095753 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f96440dea00 con 0x7f9650103350 2024-04-25T14:12:39.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.033+0000 7f964ffff700 1 --2- 172.21.15.96:0/137095753 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f96400afc10 0x7f96400b2120 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f963c006fd0 tx=0x7f963c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:39.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.036+0000 7f964dffb700 1 -- 172.21.15.96:0/137095753 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f96440a7460 con 0x7f9650103350 2024-04-25T14:12:39.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.245+0000 7f9656b66700 1 -- 172.21.15.96:0/137095753 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f965002e510 con 0x7f9650103350 2024-04-25T14:12:39.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.245+0000 7f964dffb700 1 -- 172.21.15.96:0/137095753 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f96440a6d60 con 0x7f9650103350 2024-04-25T14:12:39.245 INFO:teuthology.orchestra.run.smithi096.stdout:231928234027 2024-04-25T14:12:39.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.247+0000 7f9656b66700 1 -- 172.21.15.96:0/137095753 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f96400afc10 msgr2=0x7f96400b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:39.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.247+0000 7f9656b66700 1 --2- 172.21.15.96:0/137095753 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f96400afc10 0x7f96400b2120 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f963c006fd0 tx=0x7f963c008040 comp rx=0 tx=0).stop 2024-04-25T14:12:39.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.247+0000 7f9656b66700 1 -- 172.21.15.96:0/137095753 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9650103350 msgr2=0x7f965019bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:39.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.247+0000 7f9656b66700 1 --2- 172.21.15.96:0/137095753 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9650103350 0x7f965019bb50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f964400d950 tx=0x7f964400dc60 comp rx=0 tx=0).stop 2024-04-25T14:12:39.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.247+0000 7f9656b66700 1 -- 172.21.15.96:0/137095753 shutdown_connections 2024-04-25T14:12:39.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.247+0000 7f9656b66700 1 --2- 172.21.15.96:0/137095753 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f96400afc10 0x7f96400b2120 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:39.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.247+0000 7f9656b66700 1 --2- 172.21.15.96:0/137095753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96501085d0 0x7f965019c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:39.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.247+0000 7f9656b66700 1 --2- 172.21.15.96:0/137095753 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9650103350 0x7f965019bb50 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:39.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.247+0000 7f9656b66700 1 --2- 172.21.15.96:0/137095753 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9650107c80 0x7f965019c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:39.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.247+0000 7f9656b66700 1 -- 172.21.15.96:0/137095753 >> 172.21.15.96:0/137095753 conn(0x7f96500fec30 msgr2=0x7f96501073f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:39.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.247+0000 7f9656b66700 1 -- 172.21.15.96:0/137095753 shutdown_connections 2024-04-25T14:12:39.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:39.247+0000 7f9656b66700 1 -- 172.21.15.96:0/137095753 wait complete. 2024-04-25T14:12:39.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:39 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/137095753' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T14:12:39.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:39 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/137095753' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T14:12:39.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:39 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/137095753' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T14:12:39.967 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:12:39.978 INFO:tasks.cephadm.ceph_manager.ceph:need seq 279172874268 got 279172874272 for osd.10 2024-04-25T14:12:39.978 DEBUG:teuthology.parallel:result is None 2024-04-25T14:12:40.827 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:40 smithi096 ceph-mon[29207]: pgmap v317: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T14:12:40.855 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234022 got 231928234027 for osd.8 2024-04-25T14:12:40.856 DEBUG:teuthology.parallel:result is None 2024-04-25T14:12:40.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:40 smithi097 ceph-mon[32720]: pgmap v317: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T14:12:40.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:40 smithi151 ceph-mon[32990]: pgmap v317: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T14:12:42.640 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:42 smithi096 ceph-mon[29207]: pgmap v318: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T14:12:42.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:42.862+0000 7ffaaa100700 1 -- 172.21.15.96:0/2043297977 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaa4103410 msgr2=0x7ffaa4103820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:42.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:42.862+0000 7ffaaa100700 1 --2- 172.21.15.96:0/2043297977 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaa4103410 0x7ffaa4103820 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c00b0d0 tx=0x7ffa9c00b3e0 comp rx=0 tx=0).stop 2024-04-25T14:12:42.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:42.862+0000 7ffaaa100700 1 -- 172.21.15.96:0/2043297977 shutdown_connections 2024-04-25T14:12:42.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:42.862+0000 7ffaaa100700 1 --2- 172.21.15.96:0/2043297977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffaa4108640 0x7ffaa410ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:42.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:42.862+0000 7ffaaa100700 1 --2- 172.21.15.96:0/2043297977 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaa4103410 0x7ffaa4103820 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:42.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:42.862+0000 7ffaaa100700 1 --2- 172.21.15.96:0/2043297977 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffaa4107cf0 0x7ffaa4108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:42.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:42.862+0000 7ffaaa100700 1 -- 172.21.15.96:0/2043297977 >> 172.21.15.96:0/2043297977 conn(0x7ffaa4078d20 msgr2=0x7ffaa4079160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:42.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:42.862+0000 7ffaaa100700 1 -- 172.21.15.96:0/2043297977 shutdown_connections 2024-04-25T14:12:42.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:42.862+0000 7ffaaa100700 1 -- 172.21.15.96:0/2043297977 wait complete. 2024-04-25T14:12:42.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:42.863+0000 7ffaaa100700 1 Processor -- start 2024-04-25T14:12:42.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:42.863+0000 7ffaaa100700 1 -- start start 2024-04-25T14:12:42.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:42.863+0000 7ffaaa100700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffaa4103410 0x7ffaa419bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:42.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:42.863+0000 7ffaaa100700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaa4107cf0 0x7ffaa419c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:42.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:42.863+0000 7ffaaa100700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffaa4108640 0x7ffaa419c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:42.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:42.863+0000 7ffaaa100700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa4113360 con 0x7ffaa4107cf0 2024-04-25T14:12:42.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:42.863+0000 7ffaaa100700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa41131e0 con 0x7ffaa4103410 2024-04-25T14:12:42.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:42.863+0000 7ffaaa100700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa41134e0 con 0x7ffaa4108640 2024-04-25T14:12:42.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:42.863+0000 7ffaa2ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaa4107cf0 0x7ffaa419c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:42.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:42.863+0000 7ffaa2ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaa4107cf0 0x7ffaa419c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56154/0 (socket says 172.21.15.96:56154) 2024-04-25T14:12:42.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:42.863+0000 7ffaa2ffd700 1 -- 172.21.15.96:0/190034501 learned_addr learned my addr 172.21.15.96:0/190034501 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:12:42.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:42.864+0000 7ffaa3fff700 1 --2- 172.21.15.96:0/190034501 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffaa4108640 0x7ffaa419c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:42.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:42.864+0000 7ffaa37fe700 1 --2- 172.21.15.96:0/190034501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffaa4103410 0x7ffaa419bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:42.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:42.864+0000 7ffaa2ffd700 1 -- 172.21.15.96:0/190034501 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffaa4108640 msgr2=0x7ffaa419c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:42.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:42.864+0000 7ffaa2ffd700 1 --2- 172.21.15.96:0/190034501 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffaa4108640 0x7ffaa419c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:42.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:42.864+0000 7ffaa2ffd700 1 -- 172.21.15.96:0/190034501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffaa4103410 msgr2=0x7ffaa419bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:42.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:42.864+0000 7ffaa2ffd700 1 --2- 172.21.15.96:0/190034501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffaa4103410 0x7ffaa419bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:42.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:42.864+0000 7ffaa2ffd700 1 -- 172.21.15.96:0/190034501 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaa41a7010 con 0x7ffaa4107cf0 2024-04-25T14:12:42.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:42.864+0000 7ffaa2ffd700 1 --2- 172.21.15.96:0/190034501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaa4107cf0 0x7ffaa419c1e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ffa9800b500 tx=0x7ffa9800b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:42.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:42.864+0000 7ffaa0ff9700 1 -- 172.21.15.96:0/190034501 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa98015020 con 0x7ffaa4107cf0 2024-04-25T14:12:42.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:42.864+0000 7ffaa0ff9700 1 -- 172.21.15.96:0/190034501 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa98011ce0 con 0x7ffaa4107cf0 2024-04-25T14:12:42.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:42.864+0000 7ffaaa100700 1 -- 172.21.15.96:0/190034501 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaa41a7300 con 0x7ffaa4107cf0 2024-04-25T14:12:42.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:42.864+0000 7ffaa0ff9700 1 -- 172.21.15.96:0/190034501 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa98010cb0 con 0x7ffaa4107cf0 2024-04-25T14:12:42.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:42.864+0000 7ffaaa100700 1 -- 172.21.15.96:0/190034501 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaa41a7840 con 0x7ffaa4107cf0 2024-04-25T14:12:42.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:42.865+0000 7ffaaa100700 1 -- 172.21.15.96:0/190034501 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffaa4051190 con 0x7ffaa4107cf0 2024-04-25T14:12:42.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:42.867+0000 7ffaa0ff9700 1 -- 172.21.15.96:0/190034501 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7ffa98018020 con 0x7ffaa4107cf0 2024-04-25T14:12:42.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:42.867+0000 7ffaa0ff9700 1 --2- 172.21.15.96:0/190034501 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ffa8c0afc10 0x7ffa8c0b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:42.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:42.867+0000 7ffaa37fe700 1 --2- 172.21.15.96:0/190034501 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ffa8c0afc10 0x7ffa8c0b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:42.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:42.867+0000 7ffaa0ff9700 1 -- 172.21.15.96:0/190034501 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7ffa980de900 con 0x7ffaa4107cf0 2024-04-25T14:12:42.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:42.868+0000 7ffaa37fe700 1 --2- 172.21.15.96:0/190034501 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ffa8c0afc10 0x7ffa8c0b2120 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c009900 tx=0x7ffa9c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:42.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:42.871+0000 7ffaa0ff9700 1 -- 172.21.15.96:0/190034501 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffa9801f020 con 0x7ffaa4107cf0 2024-04-25T14:12:42.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:42 smithi097 ceph-mon[32720]: pgmap v318: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T14:12:42.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:42 smithi151 ceph-mon[32990]: pgmap v318: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T14:12:43.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.076+0000 7ffaaa100700 1 -- 172.21.15.96:0/190034501 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7ffaa402e510 con 0x7ffaa4107cf0 2024-04-25T14:12:43.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.077+0000 7ffaa0ff9700 1 -- 172.21.15.96:0/190034501 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ffa980a7470 con 0x7ffaa4107cf0 2024-04-25T14:12:43.077 INFO:teuthology.orchestra.run.smithi096.stdout:158913790012 2024-04-25T14:12:43.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.079+0000 7ffaaa100700 1 -- 172.21.15.96:0/190034501 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ffa8c0afc10 msgr2=0x7ffa8c0b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:43.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.079+0000 7ffaaa100700 1 --2- 172.21.15.96:0/190034501 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ffa8c0afc10 0x7ffa8c0b2120 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c009900 tx=0x7ffa9c00b040 comp rx=0 tx=0).stop 2024-04-25T14:12:43.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.079+0000 7ffaaa100700 1 -- 172.21.15.96:0/190034501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaa4107cf0 msgr2=0x7ffaa419c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:43.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.079+0000 7ffaaa100700 1 --2- 172.21.15.96:0/190034501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaa4107cf0 0x7ffaa419c1e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ffa9800b500 tx=0x7ffa9800b810 comp rx=0 tx=0).stop 2024-04-25T14:12:43.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.079+0000 7ffaaa100700 1 -- 172.21.15.96:0/190034501 shutdown_connections 2024-04-25T14:12:43.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.079+0000 7ffaaa100700 1 --2- 172.21.15.96:0/190034501 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ffa8c0afc10 0x7ffa8c0b2120 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:43.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.079+0000 7ffaaa100700 1 --2- 172.21.15.96:0/190034501 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffaa4108640 0x7ffaa419c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:43.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.079+0000 7ffaaa100700 1 --2- 172.21.15.96:0/190034501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffaa4103410 0x7ffaa419bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:43.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.079+0000 7ffaaa100700 1 --2- 172.21.15.96:0/190034501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaa4107cf0 0x7ffaa419c1e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:43.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.079+0000 7ffaaa100700 1 -- 172.21.15.96:0/190034501 >> 172.21.15.96:0/190034501 conn(0x7ffaa4078d20 msgr2=0x7ffaa41008b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:43.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.079+0000 7ffaaa100700 1 -- 172.21.15.96:0/190034501 shutdown_connections 2024-04-25T14:12:43.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.079+0000 7ffaaa100700 1 -- 172.21.15.96:0/190034501 wait complete. 2024-04-25T14:12:43.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.576+0000 7fd28e2cd700 1 -- 172.21.15.96:0/1258961996 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd288102fb0 msgr2=0x7fd288111040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:43.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.576+0000 7fd28e2cd700 1 --2- 172.21.15.96:0/1258961996 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd288102fb0 0x7fd288111040 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd28000b0a0 tx=0x7fd28000b3b0 comp rx=0 tx=0).stop 2024-04-25T14:12:43.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.577+0000 7fd28e2cd700 1 -- 172.21.15.96:0/1258961996 shutdown_connections 2024-04-25T14:12:43.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.577+0000 7fd28e2cd700 1 --2- 172.21.15.96:0/1258961996 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd288102fb0 0x7fd288111040 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:43.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.577+0000 7fd28e2cd700 1 --2- 172.21.15.96:0/1258961996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2881091d0 0x7fd2881095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:43.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.577+0000 7fd28e2cd700 1 --2- 172.21.15.96:0/1258961996 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd288102660 0x7fd288102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:43.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.577+0000 7fd28e2cd700 1 -- 172.21.15.96:0/1258961996 >> 172.21.15.96:0/1258961996 conn(0x7fd2880fe3f0 msgr2=0x7fd288100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:43.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.577+0000 7fd28e2cd700 1 -- 172.21.15.96:0/1258961996 shutdown_connections 2024-04-25T14:12:43.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.577+0000 7fd28e2cd700 1 -- 172.21.15.96:0/1258961996 wait complete. 2024-04-25T14:12:43.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.578+0000 7fd28e2cd700 1 Processor -- start 2024-04-25T14:12:43.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.578+0000 7fd28e2cd700 1 -- start start 2024-04-25T14:12:43.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.578+0000 7fd28e2cd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd288102660 0x7fd28819cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:43.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.578+0000 7fd28e2cd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd288102fb0 0x7fd28819d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:43.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.578+0000 7fd28e2cd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2881091d0 0x7fd2881a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:43.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.579+0000 7fd28e2cd700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd288114480 con 0x7fd288102fb0 2024-04-25T14:12:43.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.579+0000 7fd28e2cd700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd288114300 con 0x7fd2881091d0 2024-04-25T14:12:43.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.579+0000 7fd28e2cd700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd288114600 con 0x7fd288102660 2024-04-25T14:12:43.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.579+0000 7fd2877fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd288102fb0 0x7fd28819d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:43.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.579+0000 7fd2877fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd288102fb0 0x7fd28819d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56172/0 (socket says 172.21.15.96:56172) 2024-04-25T14:12:43.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.579+0000 7fd2877fe700 1 -- 172.21.15.96:0/231222767 learned_addr learned my addr 172.21.15.96:0/231222767 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:12:43.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.579+0000 7fd287fff700 1 --2- 172.21.15.96:0/231222767 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd288102660 0x7fd28819cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:43.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.579+0000 7fd28c86a700 1 --2- 172.21.15.96:0/231222767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2881091d0 0x7fd2881a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:43.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.579+0000 7fd2877fe700 1 -- 172.21.15.96:0/231222767 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd288102660 msgr2=0x7fd28819cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:43.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.579+0000 7fd2877fe700 1 --2- 172.21.15.96:0/231222767 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd288102660 0x7fd28819cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:43.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.579+0000 7fd2877fe700 1 -- 172.21.15.96:0/231222767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2881091d0 msgr2=0x7fd2881a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:43.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.579+0000 7fd2877fe700 1 --2- 172.21.15.96:0/231222767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2881091d0 0x7fd2881a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:43.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.579+0000 7fd2877fe700 1 -- 172.21.15.96:0/231222767 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2881a1dc0 con 0x7fd288102fb0 2024-04-25T14:12:43.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.579+0000 7fd2877fe700 1 --2- 172.21.15.96:0/231222767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd288102fb0 0x7fd28819d490 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd27c00d950 tx=0x7fd27c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:43.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.579+0000 7fd2857fa700 1 -- 172.21.15.96:0/231222767 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27c017070 con 0x7fd288102fb0 2024-04-25T14:12:43.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.579+0000 7fd2857fa700 1 -- 172.21.15.96:0/231222767 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd27c005020 con 0x7fd288102fb0 2024-04-25T14:12:43.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.579+0000 7fd28e2cd700 1 -- 172.21.15.96:0/231222767 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2881a20b0 con 0x7fd288102fb0 2024-04-25T14:12:43.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.579+0000 7fd2857fa700 1 -- 172.21.15.96:0/231222767 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27c00fa10 con 0x7fd288102fb0 2024-04-25T14:12:43.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.580+0000 7fd28e2cd700 1 -- 172.21.15.96:0/231222767 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2881a99d0 con 0x7fd288102fb0 2024-04-25T14:12:43.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.580+0000 7fd28e2cd700 1 -- 172.21.15.96:0/231222767 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd288051190 con 0x7fd288102fb0 2024-04-25T14:12:43.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.582+0000 7fd2857fa700 1 -- 172.21.15.96:0/231222767 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fd27c004a30 con 0x7fd288102fb0 2024-04-25T14:12:43.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.583+0000 7fd2857fa700 1 --2- 172.21.15.96:0/231222767 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd2700afc10 0x7fd2700b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:43.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.583+0000 7fd287fff700 1 --2- 172.21.15.96:0/231222767 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd2700afc10 0x7fd2700b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:43.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.583+0000 7fd2857fa700 1 -- 172.21.15.96:0/231222767 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fd27c0dea30 con 0x7fd288102fb0 2024-04-25T14:12:43.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.583+0000 7fd287fff700 1 --2- 172.21.15.96:0/231222767 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd2700afc10 0x7fd2700b2120 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd28806c940 tx=0x7fd27800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:43.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.586+0000 7fd2857fa700 1 -- 172.21.15.96:0/231222767 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd27c0a7490 con 0x7fd288102fb0 2024-04-25T14:12:43.759 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:43 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/190034501' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T14:12:43.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.794+0000 7fd28e2cd700 1 -- 172.21.15.96:0/231222767 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fd2881a9ba0 con 0x7fd288102fb0 2024-04-25T14:12:43.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.794+0000 7fd2857fa700 1 -- 172.21.15.96:0/231222767 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fd27c0a6d90 con 0x7fd288102fb0 2024-04-25T14:12:43.795 INFO:teuthology.orchestra.run.smithi096.stdout:304942678045 2024-04-25T14:12:43.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.797+0000 7fd28e2cd700 1 -- 172.21.15.96:0/231222767 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd2700afc10 msgr2=0x7fd2700b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:43.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.797+0000 7fd28e2cd700 1 --2- 172.21.15.96:0/231222767 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd2700afc10 0x7fd2700b2120 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd28806c940 tx=0x7fd27800a380 comp rx=0 tx=0).stop 2024-04-25T14:12:43.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.797+0000 7fd28e2cd700 1 -- 172.21.15.96:0/231222767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd288102fb0 msgr2=0x7fd28819d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:43.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.797+0000 7fd28e2cd700 1 --2- 172.21.15.96:0/231222767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd288102fb0 0x7fd28819d490 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd27c00d950 tx=0x7fd27c00dc60 comp rx=0 tx=0).stop 2024-04-25T14:12:43.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.797+0000 7fd28e2cd700 1 -- 172.21.15.96:0/231222767 shutdown_connections 2024-04-25T14:12:43.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.797+0000 7fd28e2cd700 1 --2- 172.21.15.96:0/231222767 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd2700afc10 0x7fd2700b2120 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:43.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.797+0000 7fd28e2cd700 1 --2- 172.21.15.96:0/231222767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2881091d0 0x7fd2881a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:43.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.797+0000 7fd28e2cd700 1 --2- 172.21.15.96:0/231222767 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd288102660 0x7fd28819cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:43.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.797+0000 7fd28e2cd700 1 --2- 172.21.15.96:0/231222767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd288102fb0 0x7fd28819d490 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:43.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.797+0000 7fd28e2cd700 1 -- 172.21.15.96:0/231222767 >> 172.21.15.96:0/231222767 conn(0x7fd2880fe3f0 msgr2=0x7fd28810fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:43.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.797+0000 7fd28e2cd700 1 -- 172.21.15.96:0/231222767 shutdown_connections 2024-04-25T14:12:43.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:43.797+0000 7fd28e2cd700 1 -- 172.21.15.96:0/231222767 wait complete. 2024-04-25T14:12:43.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:43 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/190034501' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T14:12:43.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:43 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/190034501' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T14:12:44.564 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678039 got 304942678045 for osd.11 2024-04-25T14:12:44.564 DEBUG:teuthology.parallel:result is None 2024-04-25T14:12:44.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:44 smithi097 ceph-mon[32720]: pgmap v319: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T14:12:44.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:44 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/231222767' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T14:12:44.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:44 smithi096 ceph-mon[29207]: pgmap v319: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T14:12:44.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:44 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/231222767' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T14:12:44.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:44 smithi151 ceph-mon[32990]: pgmap v319: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T14:12:44.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:44 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/231222767' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T14:12:45.832 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790006 got 158913790012 for osd.5 2024-04-25T14:12:45.832 DEBUG:teuthology.parallel:result is None 2024-04-25T14:12:46.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.365+0000 7f777630b700 1 -- 172.21.15.96:0/2252909449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7770103380 msgr2=0x7f7770103790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:46.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.365+0000 7f777630b700 1 --2- 172.21.15.96:0/2252909449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7770103380 0x7f7770103790 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f77600099c0 tx=0x7f7760009cd0 comp rx=0 tx=0).stop 2024-04-25T14:12:46.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.365+0000 7f777630b700 1 -- 172.21.15.96:0/2252909449 shutdown_connections 2024-04-25T14:12:46.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.365+0000 7f777630b700 1 --2- 172.21.15.96:0/2252909449 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7770108600 0x7f777010ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:46.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.365+0000 7f777630b700 1 --2- 172.21.15.96:0/2252909449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7770103380 0x7f7770103790 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:46.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.365+0000 7f777630b700 1 --2- 172.21.15.96:0/2252909449 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7770107cb0 0x7f77701080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:46.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.365+0000 7f777630b700 1 -- 172.21.15.96:0/2252909449 >> 172.21.15.96:0/2252909449 conn(0x7f77700fecb0 msgr2=0x7f77701010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:46.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.365+0000 7f777630b700 1 -- 172.21.15.96:0/2252909449 shutdown_connections 2024-04-25T14:12:46.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.365+0000 7f777630b700 1 -- 172.21.15.96:0/2252909449 wait complete. 2024-04-25T14:12:46.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.366+0000 7f777630b700 1 Processor -- start 2024-04-25T14:12:46.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.366+0000 7f777630b700 1 -- start start 2024-04-25T14:12:46.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.366+0000 7f777630b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7770103380 0x7f777019bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:46.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.366+0000 7f777630b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7770107cb0 0x7f777019c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:46.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.366+0000 7f777630b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7770108600 0x7f777019c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:46.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.366+0000 7f777630b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f77701132a0 con 0x7f7770107cb0 2024-04-25T14:12:46.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.366+0000 7f777630b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7770113120 con 0x7f7770108600 2024-04-25T14:12:46.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.366+0000 7f777630b700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7770113420 con 0x7f7770103380 2024-04-25T14:12:46.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.366+0000 7f776f7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7770107cb0 0x7f777019c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:46.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.366+0000 7f776f7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7770107cb0 0x7f777019c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56196/0 (socket says 172.21.15.96:56196) 2024-04-25T14:12:46.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.367+0000 7f776f7fe700 1 -- 172.21.15.96:0/3385871193 learned_addr learned my addr 172.21.15.96:0/3385871193 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:12:46.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.367+0000 7f776ffff700 1 --2- 172.21.15.96:0/3385871193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7770103380 0x7f777019bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:46.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.367+0000 7f77748a8700 1 --2- 172.21.15.96:0/3385871193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7770108600 0x7f777019c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:46.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.367+0000 7f776f7fe700 1 -- 172.21.15.96:0/3385871193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7770103380 msgr2=0x7f777019bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:46.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.367+0000 7f776f7fe700 1 --2- 172.21.15.96:0/3385871193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7770103380 0x7f777019bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:46.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.367+0000 7f776f7fe700 1 -- 172.21.15.96:0/3385871193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7770108600 msgr2=0x7f777019c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:46.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.367+0000 7f776f7fe700 1 --2- 172.21.15.96:0/3385871193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7770108600 0x7f777019c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:46.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.367+0000 7f776f7fe700 1 -- 172.21.15.96:0/3385871193 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77701a6e40 con 0x7f7770107cb0 2024-04-25T14:12:46.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.367+0000 7f77748a8700 1 --2- 172.21.15.96:0/3385871193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7770108600 0x7f777019c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:12:46.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.367+0000 7f776f7fe700 1 --2- 172.21.15.96:0/3385871193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7770107cb0 0x7f777019c0c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f776400b500 tx=0x7f776400b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:46.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.367+0000 7f776d7fa700 1 -- 172.21.15.96:0/3385871193 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7764015020 con 0x7f7770107cb0 2024-04-25T14:12:46.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.367+0000 7f777630b700 1 -- 172.21.15.96:0/3385871193 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77701a7130 con 0x7f7770107cb0 2024-04-25T14:12:46.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.367+0000 7f776d7fa700 1 -- 172.21.15.96:0/3385871193 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7764011ce0 con 0x7f7770107cb0 2024-04-25T14:12:46.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.367+0000 7f776d7fa700 1 -- 172.21.15.96:0/3385871193 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7764010d40 con 0x7f7770107cb0 2024-04-25T14:12:46.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.367+0000 7f777630b700 1 -- 172.21.15.96:0/3385871193 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77701a76c0 con 0x7f7770107cb0 2024-04-25T14:12:46.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.368+0000 7f777630b700 1 -- 172.21.15.96:0/3385871193 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7770104830 con 0x7f7770107cb0 2024-04-25T14:12:46.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.370+0000 7f776d7fa700 1 -- 172.21.15.96:0/3385871193 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f7764018020 con 0x7f7770107cb0 2024-04-25T14:12:46.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.370+0000 7f776d7fa700 1 --2- 172.21.15.96:0/3385871193 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f77580afc10 0x7f77580b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:46.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.370+0000 7f776ffff700 1 --2- 172.21.15.96:0/3385871193 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f77580afc10 0x7f77580b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:46.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.370+0000 7f776d7fa700 1 -- 172.21.15.96:0/3385871193 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f77640de410 con 0x7f7770107cb0 2024-04-25T14:12:46.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.371+0000 7f776ffff700 1 --2- 172.21.15.96:0/3385871193 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f77580afc10 0x7f77580b2120 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7760009990 tx=0x7f7760000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:46.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.375+0000 7f776d7fa700 1 -- 172.21.15.96:0/3385871193 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f776401f020 con 0x7f7770107cb0 2024-04-25T14:12:46.568 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:46 smithi096 ceph-mon[29207]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T14:12:46.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.615+0000 7f777630b700 1 -- 172.21.15.96:0/3385871193 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f777002e510 con 0x7f7770107cb0 2024-04-25T14:12:46.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.615+0000 7f776d7fa700 1 -- 172.21.15.96:0/3385871193 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f77640a6f80 con 0x7f7770107cb0 2024-04-25T14:12:46.616 INFO:teuthology.orchestra.run.smithi096.stdout:111669149766 2024-04-25T14:12:46.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.618+0000 7f777630b700 1 -- 172.21.15.96:0/3385871193 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f77580afc10 msgr2=0x7f77580b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:46.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.618+0000 7f777630b700 1 --2- 172.21.15.96:0/3385871193 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f77580afc10 0x7f77580b2120 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7760009990 tx=0x7f7760000bc0 comp rx=0 tx=0).stop 2024-04-25T14:12:46.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.618+0000 7f777630b700 1 -- 172.21.15.96:0/3385871193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7770107cb0 msgr2=0x7f777019c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:46.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.618+0000 7f777630b700 1 --2- 172.21.15.96:0/3385871193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7770107cb0 0x7f777019c0c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f776400b500 tx=0x7f776400b810 comp rx=0 tx=0).stop 2024-04-25T14:12:46.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.618+0000 7f777630b700 1 -- 172.21.15.96:0/3385871193 shutdown_connections 2024-04-25T14:12:46.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.618+0000 7f777630b700 1 --2- 172.21.15.96:0/3385871193 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f77580afc10 0x7f77580b2120 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:46.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.618+0000 7f777630b700 1 --2- 172.21.15.96:0/3385871193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7770108600 0x7f777019c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:46.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.618+0000 7f777630b700 1 --2- 172.21.15.96:0/3385871193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7770103380 0x7f777019bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:46.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.618+0000 7f777630b700 1 --2- 172.21.15.96:0/3385871193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7770107cb0 0x7f777019c0c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:46.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.618+0000 7f777630b700 1 -- 172.21.15.96:0/3385871193 >> 172.21.15.96:0/3385871193 conn(0x7f77700fecb0 msgr2=0x7f7770106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:46.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.618+0000 7f777630b700 1 -- 172.21.15.96:0/3385871193 shutdown_connections 2024-04-25T14:12:46.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:46.618+0000 7f777630b700 1 -- 172.21.15.96:0/3385871193 wait complete. 2024-04-25T14:12:46.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:46 smithi097 ceph-mon[32720]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T14:12:46.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:46 smithi151 ceph-mon[32990]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T14:12:47.314 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149760 got 111669149766 for osd.3 2024-04-25T14:12:47.314 DEBUG:teuthology.parallel:result is None 2024-04-25T14:12:47.314 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T14:12:47.315 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T14:12:47.567 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:12:47.847 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:47 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/3385871193' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T14:12:47.847 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:47 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:12:47.847 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:47 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:12:47.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:47 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/3385871193' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T14:12:47.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:47 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:12:47.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:47 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:12:47.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:47 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/3385871193' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T14:12:47.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:47 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:12:47.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:47 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:12:48.734 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:48 smithi096 ceph-mon[29207]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-04-25T14:12:48.734 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:48 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:12:48.734 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:48 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:12:48.735 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:48 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:12:48.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:48 smithi097 ceph-mon[32720]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-04-25T14:12:48.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:48 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:12:48.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:48 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:12:48.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:48 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:12:48.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:48 smithi151 ceph-mon[32990]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-04-25T14:12:48.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:48 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:12:48.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:48 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:12:48.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:48 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:12:49.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.256+0000 7fa4e403a700 1 -- 172.21.15.96:0/3106171375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4dc103350 msgr2=0x7fa4dc103760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:49.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.256+0000 7fa4e403a700 1 --2- 172.21.15.96:0/3106171375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4dc103350 0x7fa4dc103760 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa4d80099c0 tx=0x7fa4d8009cd0 comp rx=0 tx=0).stop 2024-04-25T14:12:49.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.256+0000 7fa4e403a700 1 -- 172.21.15.96:0/3106171375 shutdown_connections 2024-04-25T14:12:49.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.256+0000 7fa4e403a700 1 --2- 172.21.15.96:0/3106171375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4dc1085d0 0x7fa4dc10ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:49.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.256+0000 7fa4e403a700 1 --2- 172.21.15.96:0/3106171375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4dc103350 0x7fa4dc103760 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:49.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.256+0000 7fa4e403a700 1 --2- 172.21.15.96:0/3106171375 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4dc107c80 0x7fa4dc108090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:49.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.256+0000 7fa4e403a700 1 -- 172.21.15.96:0/3106171375 >> 172.21.15.96:0/3106171375 conn(0x7fa4dc0fec30 msgr2=0x7fa4dc101050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:49.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.256+0000 7fa4e403a700 1 -- 172.21.15.96:0/3106171375 shutdown_connections 2024-04-25T14:12:49.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.256+0000 7fa4e403a700 1 -- 172.21.15.96:0/3106171375 wait complete. 2024-04-25T14:12:49.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.256+0000 7fa4e403a700 1 Processor -- start 2024-04-25T14:12:49.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.256+0000 7fa4e403a700 1 -- start start 2024-04-25T14:12:49.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.257+0000 7fa4e403a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4dc103350 0x7fa4dc1a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:49.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.257+0000 7fa4e403a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4dc107c80 0x7fa4dc1a0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:49.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.257+0000 7fa4e403a700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4dc1085d0 0x7fa4dc19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:49.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.257+0000 7fa4e403a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc113250 con 0x7fa4dc107c80 2024-04-25T14:12:49.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.257+0000 7fa4e403a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc1130d0 con 0x7fa4dc103350 2024-04-25T14:12:49.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.257+0000 7fa4e403a700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc1133d0 con 0x7fa4dc1085d0 2024-04-25T14:12:49.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.257+0000 7fa4e15d5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4dc107c80 0x7fa4dc1a0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:49.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.257+0000 7fa4e15d5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4dc107c80 0x7fa4dc1a0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56218/0 (socket says 172.21.15.96:56218) 2024-04-25T14:12:49.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.257+0000 7fa4e15d5700 1 -- 172.21.15.96:0/1143750891 learned_addr learned my addr 172.21.15.96:0/1143750891 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:12:49.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.257+0000 7fa4e1dd6700 1 --2- 172.21.15.96:0/1143750891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4dc103350 0x7fa4dc1a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:49.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.257+0000 7fa4e25d7700 1 --2- 172.21.15.96:0/1143750891 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4dc1085d0 0x7fa4dc19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:49.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.257+0000 7fa4e15d5700 1 -- 172.21.15.96:0/1143750891 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4dc1085d0 msgr2=0x7fa4dc19ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:49.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.257+0000 7fa4e15d5700 1 --2- 172.21.15.96:0/1143750891 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4dc1085d0 0x7fa4dc19ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:49.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.257+0000 7fa4e15d5700 1 -- 172.21.15.96:0/1143750891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4dc103350 msgr2=0x7fa4dc1a09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:49.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.257+0000 7fa4e15d5700 1 --2- 172.21.15.96:0/1143750891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4dc103350 0x7fa4dc1a09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:49.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.257+0000 7fa4e15d5700 1 -- 172.21.15.96:0/1143750891 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4dc19b1e0 con 0x7fa4dc107c80 2024-04-25T14:12:49.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.257+0000 7fa4e15d5700 1 --2- 172.21.15.96:0/1143750891 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4dc107c80 0x7fa4dc1a0ef0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa4d000c9e0 tx=0x7fa4d000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:49.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.257+0000 7fa4ceffd700 1 -- 172.21.15.96:0/1143750891 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d0007aa0 con 0x7fa4dc107c80 2024-04-25T14:12:49.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.258+0000 7fa4ceffd700 1 -- 172.21.15.96:0/1143750891 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4d0007c40 con 0x7fa4dc107c80 2024-04-25T14:12:49.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.258+0000 7fa4e403a700 1 -- 172.21.15.96:0/1143750891 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4dc19b4d0 con 0x7fa4dc107c80 2024-04-25T14:12:49.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.258+0000 7fa4ceffd700 1 -- 172.21.15.96:0/1143750891 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d000fe60 con 0x7fa4dc107c80 2024-04-25T14:12:49.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.258+0000 7fa4e403a700 1 -- 172.21.15.96:0/1143750891 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4dc1a7760 con 0x7fa4dc107c80 2024-04-25T14:12:49.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.259+0000 7fa4e403a700 1 -- 172.21.15.96:0/1143750891 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4dc104800 con 0x7fa4dc107c80 2024-04-25T14:12:49.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.260+0000 7fa4ceffd700 1 -- 172.21.15.96:0/1143750891 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fa4d000f450 con 0x7fa4dc107c80 2024-04-25T14:12:49.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.261+0000 7fa4ceffd700 1 --2- 172.21.15.96:0/1143750891 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa4c80afc10 0x7fa4c80b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:49.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.261+0000 7fa4e1dd6700 1 --2- 172.21.15.96:0/1143750891 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa4c80afc10 0x7fa4c80b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:49.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.261+0000 7fa4ceffd700 1 -- 172.21.15.96:0/1143750891 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fa4d00defb0 con 0x7fa4dc107c80 2024-04-25T14:12:49.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.261+0000 7fa4e1dd6700 1 --2- 172.21.15.96:0/1143750891 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa4c80afc10 0x7fa4c80b2120 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8009990 tx=0x7fa4d8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:49.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.265+0000 7fa4ceffd700 1 -- 172.21.15.96:0/1143750891 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa4d0014030 con 0x7fa4dc107c80 2024-04-25T14:12:49.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.458+0000 7fa4e403a700 1 -- 172.21.15.96:0/1143750891 --> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa4dc19bfe0 con 0x7fa4c80afc10 2024-04-25T14:12:49.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.465+0000 7fa4ceffd700 1 -- 172.21.15.96:0/1143750891 <== mgr.14152 v2:172.21.15.96:6800/536332425 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259045 (secure 0 0 0) 0x7fa4d8067400 con 0x7fa4c80afc10 2024-04-25T14:12:49.469 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:12:49.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.468+0000 7fa4e403a700 1 -- 172.21.15.96:0/1143750891 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa4c80afc10 msgr2=0x7fa4c80b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:49.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.468+0000 7fa4e403a700 1 --2- 172.21.15.96:0/1143750891 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa4c80afc10 0x7fa4c80b2120 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8009990 tx=0x7fa4d8000bc0 comp rx=0 tx=0).stop 2024-04-25T14:12:49.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.468+0000 7fa4e403a700 1 -- 172.21.15.96:0/1143750891 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4dc107c80 msgr2=0x7fa4dc1a0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:49.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.468+0000 7fa4e403a700 1 --2- 172.21.15.96:0/1143750891 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4dc107c80 0x7fa4dc1a0ef0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa4d000c9e0 tx=0x7fa4d000ccf0 comp rx=0 tx=0).stop 2024-04-25T14:12:49.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.468+0000 7fa4e403a700 1 -- 172.21.15.96:0/1143750891 shutdown_connections 2024-04-25T14:12:49.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.468+0000 7fa4e403a700 1 --2- 172.21.15.96:0/1143750891 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa4c80afc10 0x7fa4c80b2120 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:49.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.468+0000 7fa4e403a700 1 --2- 172.21.15.96:0/1143750891 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4dc1085d0 0x7fa4dc19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:49.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.468+0000 7fa4e403a700 1 --2- 172.21.15.96:0/1143750891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4dc103350 0x7fa4dc1a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:49.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.468+0000 7fa4e403a700 1 --2- 172.21.15.96:0/1143750891 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4dc107c80 0x7fa4dc1a0ef0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:49.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.468+0000 7fa4e403a700 1 -- 172.21.15.96:0/1143750891 >> 172.21.15.96:0/1143750891 conn(0x7fa4dc0fec30 msgr2=0x7fa4dc06c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:49.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.468+0000 7fa4e403a700 1 -- 172.21.15.96:0/1143750891 shutdown_connections 2024-04-25T14:12:49.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:49.468+0000 7fa4e403a700 1 -- 172.21.15.96:0/1143750891 wait complete. 2024-04-25T14:12:49.472 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-04-25T14:12:50.187 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-04-25T14:12:49.420508+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657200,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":828,"num_read_kb":510,"num_write":361,"num_write_kb":2283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":2952580,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":299,"ondisk_log_size":299,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359684,"kb_used_data":10452,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124582140,"statfs":{"total":1151940427776,"available":1151572111360,"internally_reserved":0,"allocated":10702848,"data_stored":6820572,"data_compressed":182373,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":100,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003478"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.511167+0000","last_change":"2024-04-25T14:11:29.569615+0000","last_active":"2024-04-25T14:12:22.511167+0000","last_peered":"2024-04-25T14:12:22.511167+0000","last_clean":"2024-04-25T14:12:22.511167+0000","last_became_active":"2024-04-25T14:11:29.569237+0000","last_became_peered":"2024-04-25T14:11:29.569237+0000","last_unstale":"2024-04-25T14:12:22.511167+0000","last_undegraded":"2024-04-25T14:12:22.511167+0000","last_fullsized":"2024-04-25T14:12:22.511167+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:24.367533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.509445+0000","last_change":"2024-04-25T14:11:29.571820+0000","last_active":"2024-04-25T14:12:22.509445+0000","last_peered":"2024-04-25T14:12:22.509445+0000","last_clean":"2024-04-25T14:12:22.509445+0000","last_became_active":"2024-04-25T14:11:29.571456+0000","last_became_peered":"2024-04-25T14:11:29.571456+0000","last_unstale":"2024-04-25T14:12:22.509445+0000","last_undegraded":"2024-04-25T14:12:22.509445+0000","last_fullsized":"2024-04-25T14:12:22.509445+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:33.442301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.509287+0000","last_change":"2024-04-25T14:11:29.579302+0000","last_active":"2024-04-25T14:12:22.509287+0000","last_peered":"2024-04-25T14:12:22.509287+0000","last_clean":"2024-04-25T14:12:22.509287+0000","last_became_active":"2024-04-25T14:11:29.577734+0000","last_became_peered":"2024-04-25T14:11:29.577734+0000","last_unstale":"2024-04-25T14:12:22.509287+0000","last_undegraded":"2024-04-25T14:12:22.509287+0000","last_fullsized":"2024-04-25T14:12:22.509287+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:59.147554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.512471+0000","last_change":"2024-04-25T14:11:29.574942+0000","last_active":"2024-04-25T14:12:22.512471+0000","last_peered":"2024-04-25T14:12:22.512471+0000","last_clean":"2024-04-25T14:12:22.512471+0000","last_became_active":"2024-04-25T14:11:29.573774+0000","last_became_peered":"2024-04-25T14:11:29.573774+0000","last_unstale":"2024-04-25T14:12:22.512471+0000","last_undegraded":"2024-04-25T14:12:22.512471+0000","last_fullsized":"2024-04-25T14:12:22.512471+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:10.626265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514141+0000","last_change":"2024-04-25T14:11:29.557199+0000","last_active":"2024-04-25T14:12:22.514141+0000","last_peered":"2024-04-25T14:12:22.514141+0000","last_clean":"2024-04-25T14:12:22.514141+0000","last_became_active":"2024-04-25T14:11:29.556836+0000","last_became_peered":"2024-04-25T14:11:29.556836+0000","last_unstale":"2024-04-25T14:12:22.514141+0000","last_undegraded":"2024-04-25T14:12:22.514141+0000","last_fullsized":"2024-04-25T14:12:22.514141+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:04:45.882935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.515973+0000","last_change":"2024-04-25T14:11:29.557988+0000","last_active":"2024-04-25T14:12:22.515973+0000","last_peered":"2024-04-25T14:12:22.515973+0000","last_clean":"2024-04-25T14:12:22.515973+0000","last_became_active":"2024-04-25T14:11:29.557622+0000","last_became_peered":"2024-04-25T14:11:29.557622+0000","last_unstale":"2024-04-25T14:12:22.515973+0000","last_undegraded":"2024-04-25T14:12:22.515973+0000","last_fullsized":"2024-04-25T14:12:22.515973+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:28:37.578368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.509027+0000","last_change":"2024-04-25T14:11:29.565049+0000","last_active":"2024-04-25T14:12:22.509027+0000","last_peered":"2024-04-25T14:12:22.509027+0000","last_clean":"2024-04-25T14:12:22.509027+0000","last_became_active":"2024-04-25T14:11:29.564688+0000","last_became_peered":"2024-04-25T14:11:29.564688+0000","last_unstale":"2024-04-25T14:12:22.509027+0000","last_undegraded":"2024-04-25T14:12:22.509027+0000","last_fullsized":"2024-04-25T14:12:22.509027+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:48.128778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.513942+0000","last_change":"2024-04-25T14:11:29.565879+0000","last_active":"2024-04-25T14:12:22.513942+0000","last_peered":"2024-04-25T14:12:22.513942+0000","last_clean":"2024-04-25T14:12:22.513942+0000","last_became_active":"2024-04-25T14:11:29.565525+0000","last_became_peered":"2024-04-25T14:11:29.565525+0000","last_unstale":"2024-04-25T14:12:22.513942+0000","last_undegraded":"2024-04-25T14:12:22.513942+0000","last_fullsized":"2024-04-25T14:12:22.513942+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:06.956334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.517776+0000","last_change":"2024-04-25T14:12:22.517776+0000","last_active":"2024-04-25T14:12:22.517776+0000","last_peered":"2024-04-25T14:12:22.517776+0000","last_clean":"2024-04-25T14:12:22.517776+0000","last_became_active":"2024-04-25T14:12:22.517232+0000","last_became_peered":"2024-04-25T14:12:22.517232+0000","last_unstale":"2024-04-25T14:12:22.517776+0000","last_undegraded":"2024-04-25T14:12:22.517776+0000","last_fullsized":"2024-04-25T14:12:22.517776+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:51.912445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,11],"acting":[1,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.516507+0000","last_change":"2024-04-25T14:11:29.561841+0000","last_active":"2024-04-25T14:12:22.516507+0000","last_peered":"2024-04-25T14:12:22.516507+0000","last_clean":"2024-04-25T14:12:22.516507+0000","last_became_active":"2024-04-25T14:11:29.560145+0000","last_became_peered":"2024-04-25T14:11:29.560145+0000","last_unstale":"2024-04-25T14:12:22.516507+0000","last_undegraded":"2024-04-25T14:12:22.516507+0000","last_fullsized":"2024-04-25T14:12:22.516507+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:11.654692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.509434+0000","last_change":"2024-04-25T14:11:29.570954+0000","last_active":"2024-04-25T14:12:22.509434+0000","last_peered":"2024-04-25T14:12:22.509434+0000","last_clean":"2024-04-25T14:12:22.509434+0000","last_became_active":"2024-04-25T14:11:29.570368+0000","last_became_peered":"2024-04-25T14:11:29.570368+0000","last_unstale":"2024-04-25T14:12:22.509434+0000","last_undegraded":"2024-04-25T14:12:22.509434+0000","last_fullsized":"2024-04-25T14:12:22.509434+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:50.153695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.512357+0000","last_change":"2024-04-25T14:11:29.571911+0000","last_active":"2024-04-25T14:12:22.512357+0000","last_peered":"2024-04-25T14:12:22.512357+0000","last_clean":"2024-04-25T14:12:22.512357+0000","last_became_active":"2024-04-25T14:11:29.568557+0000","last_became_peered":"2024-04-25T14:11:29.568557+0000","last_unstale":"2024-04-25T14:12:22.512357+0000","last_undegraded":"2024-04-25T14:12:22.512357+0000","last_fullsized":"2024-04-25T14:12:22.512357+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:24.221527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.505150+0000","last_change":"2024-04-25T14:11:29.566826+0000","last_active":"2024-04-25T14:12:22.505150+0000","last_peered":"2024-04-25T14:12:22.505150+0000","last_clean":"2024-04-25T14:12:22.505150+0000","last_became_active":"2024-04-25T14:11:29.565825+0000","last_became_peered":"2024-04-25T14:11:29.565825+0000","last_unstale":"2024-04-25T14:12:22.505150+0000","last_undegraded":"2024-04-25T14:12:22.505150+0000","last_fullsized":"2024-04-25T14:12:22.505150+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:46.711809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.509114+0000","last_change":"2024-04-25T14:11:29.577359+0000","last_active":"2024-04-25T14:12:22.509114+0000","last_peered":"2024-04-25T14:12:22.509114+0000","last_clean":"2024-04-25T14:12:22.509114+0000","last_became_active":"2024-04-25T14:11:29.574643+0000","last_became_peered":"2024-04-25T14:11:29.574643+0000","last_unstale":"2024-04-25T14:12:22.509114+0000","last_undegraded":"2024-04-25T14:12:22.509114+0000","last_fullsized":"2024-04-25T14:12:22.509114+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:07.582154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.508363+0000","last_change":"2024-04-25T14:11:29.583606+0000","last_active":"2024-04-25T14:12:22.508363+0000","last_peered":"2024-04-25T14:12:22.508363+0000","last_clean":"2024-04-25T14:12:22.508363+0000","last_became_active":"2024-04-25T14:11:29.582091+0000","last_became_peered":"2024-04-25T14:11:29.582091+0000","last_unstale":"2024-04-25T14:12:22.508363+0000","last_undegraded":"2024-04-25T14:12:22.508363+0000","last_fullsized":"2024-04-25T14:12:22.508363+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:58.711591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.511077+0000","last_change":"2024-04-25T14:11:29.571223+0000","last_active":"2024-04-25T14:12:22.511077+0000","last_peered":"2024-04-25T14:12:22.511077+0000","last_clean":"2024-04-25T14:12:22.511077+0000","last_became_active":"2024-04-25T14:11:29.570856+0000","last_became_peered":"2024-04-25T14:11:29.570856+0000","last_unstale":"2024-04-25T14:12:22.511077+0000","last_undegraded":"2024-04-25T14:12:22.511077+0000","last_fullsized":"2024-04-25T14:12:22.511077+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:53.058088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.511297+0000","last_change":"2024-04-25T14:11:29.574467+0000","last_active":"2024-04-25T14:12:22.511297+0000","last_peered":"2024-04-25T14:12:22.511297+0000","last_clean":"2024-04-25T14:12:22.511297+0000","last_became_active":"2024-04-25T14:11:29.573878+0000","last_became_peered":"2024-04-25T14:11:29.573878+0000","last_unstale":"2024-04-25T14:12:22.511297+0000","last_undegraded":"2024-04-25T14:12:22.511297+0000","last_fullsized":"2024-04-25T14:12:22.511297+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:22:02.087606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.509373+0000","last_change":"2024-04-25T14:11:29.561612+0000","last_active":"2024-04-25T14:12:22.509373+0000","last_peered":"2024-04-25T14:12:22.509373+0000","last_clean":"2024-04-25T14:12:22.509373+0000","last_became_active":"2024-04-25T14:11:29.561211+0000","last_became_peered":"2024-04-25T14:11:29.561211+0000","last_unstale":"2024-04-25T14:12:22.509373+0000","last_undegraded":"2024-04-25T14:12:22.509373+0000","last_fullsized":"2024-04-25T14:12:22.509373+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:29.195093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.512413+0000","last_change":"2024-04-25T14:11:29.575231+0000","last_active":"2024-04-25T14:12:22.512413+0000","last_peered":"2024-04-25T14:12:22.512413+0000","last_clean":"2024-04-25T14:12:22.512413+0000","last_became_active":"2024-04-25T14:11:29.573928+0000","last_became_peered":"2024-04-25T14:11:29.573928+0000","last_unstale":"2024-04-25T14:12:22.512413+0000","last_undegraded":"2024-04-25T14:12:22.512413+0000","last_fullsized":"2024-04-25T14:12:22.512413+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:10.025584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"76'13","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:42.543391+0000","last_change":"2024-04-25T14:12:22.527280+0000","last_active":"2024-04-25T14:12:42.543391+0000","last_peered":"2024-04-25T14:12:42.543391+0000","last_clean":"2024-04-25T14:12:42.543391+0000","last_became_active":"2024-04-25T14:12:22.515420+0000","last_became_peered":"2024-04-25T14:12:22.515420+0000","last_unstale":"2024-04-25T14:12:42.543391+0000","last_undegraded":"2024-04-25T14:12:42.543391+0000","last_fullsized":"2024-04-25T14:12:42.543391+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:53.882212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":64,"num_read_kb":64,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514275+0000","last_change":"2024-04-25T14:11:29.576855+0000","last_active":"2024-04-25T14:12:22.514275+0000","last_peered":"2024-04-25T14:12:22.514275+0000","last_clean":"2024-04-25T14:12:22.514275+0000","last_became_active":"2024-04-25T14:11:29.574566+0000","last_became_peered":"2024-04-25T14:11:29.574566+0000","last_unstale":"2024-04-25T14:12:22.514275+0000","last_undegraded":"2024-04-25T14:12:22.514275+0000","last_fullsized":"2024-04-25T14:12:22.514275+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:31.325583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"76'2","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514429+0000","last_change":"2024-04-25T14:11:28.552763+0000","last_active":"2024-04-25T14:12:22.514429+0000","last_peered":"2024-04-25T14:12:22.514429+0000","last_clean":"2024-04-25T14:12:22.514429+0000","last_became_active":"2024-04-25T14:11:28.552403+0000","last_became_peered":"2024-04-25T14:11:28.552403+0000","last_unstale":"2024-04-25T14:12:22.514429+0000","last_undegraded":"2024-04-25T14:12:22.514429+0000","last_fullsized":"2024-04-25T14:12:22.514429+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:31.473779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.509492+0000","last_change":"2024-04-25T14:11:29.571019+0000","last_active":"2024-04-25T14:12:22.509492+0000","last_peered":"2024-04-25T14:12:22.509492+0000","last_clean":"2024-04-25T14:12:22.509492+0000","last_became_active":"2024-04-25T14:11:29.570633+0000","last_became_peered":"2024-04-25T14:11:29.570633+0000","last_unstale":"2024-04-25T14:12:22.509492+0000","last_undegraded":"2024-04-25T14:12:22.509492+0000","last_fullsized":"2024-04-25T14:12:22.509492+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:38:32.664318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"76'12","reported_seq":32,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514608+0000","last_change":"2024-04-25T14:11:28.545189+0000","last_active":"2024-04-25T14:12:22.514608+0000","last_peered":"2024-04-25T14:12:22.514608+0000","last_clean":"2024-04-25T14:12:22.514608+0000","last_became_active":"2024-04-25T14:11:28.544857+0000","last_became_peered":"2024-04-25T14:11:28.544857+0000","last_unstale":"2024-04-25T14:12:22.514608+0000","last_undegraded":"2024-04-25T14:12:22.514608+0000","last_fullsized":"2024-04-25T14:12:22.514608+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:58.464226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.511770+0000","last_change":"2024-04-25T14:11:29.576696+0000","last_active":"2024-04-25T14:12:22.511770+0000","last_peered":"2024-04-25T14:12:22.511770+0000","last_clean":"2024-04-25T14:12:22.511770+0000","last_became_active":"2024-04-25T14:11:29.576337+0000","last_became_peered":"2024-04-25T14:11:29.576337+0000","last_unstale":"2024-04-25T14:12:22.511770+0000","last_undegraded":"2024-04-25T14:12:22.511770+0000","last_fullsized":"2024-04-25T14:12:22.511770+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:31:30.410434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"76'5","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514457+0000","last_change":"2024-04-25T14:11:28.542466+0000","last_active":"2024-04-25T14:12:22.514457+0000","last_peered":"2024-04-25T14:12:22.514457+0000","last_clean":"2024-04-25T14:12:22.514457+0000","last_became_active":"2024-04-25T14:11:28.542092+0000","last_became_peered":"2024-04-25T14:11:28.542092+0000","last_unstale":"2024-04-25T14:12:22.514457+0000","last_undegraded":"2024-04-25T14:12:22.514457+0000","last_fullsized":"2024-04-25T14:12:22.514457+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:33.810111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.508895+0000","last_change":"2024-04-25T14:11:29.567842+0000","last_active":"2024-04-25T14:12:22.508895+0000","last_peered":"2024-04-25T14:12:22.508895+0000","last_clean":"2024-04-25T14:12:22.508895+0000","last_became_active":"2024-04-25T14:11:29.566269+0000","last_became_peered":"2024-04-25T14:11:29.566269+0000","last_unstale":"2024-04-25T14:12:22.508895+0000","last_undegraded":"2024-04-25T14:12:22.508895+0000","last_fullsized":"2024-04-25T14:12:22.508895+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:31.009318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"76'6","reported_seq":102,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:44.619290+0000","last_change":"2024-04-25T14:11:28.546215+0000","last_active":"2024-04-25T14:12:44.619290+0000","last_peered":"2024-04-25T14:12:44.619290+0000","last_clean":"2024-04-25T14:12:44.619290+0000","last_became_active":"2024-04-25T14:11:28.545828+0000","last_became_peered":"2024-04-25T14:11:28.545828+0000","last_unstale":"2024-04-25T14:12:44.619290+0000","last_undegraded":"2024-04-25T14:12:44.619290+0000","last_fullsized":"2024-04-25T14:12:44.619290+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:27.152582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514346+0000","last_change":"2024-04-25T14:11:29.559566+0000","last_active":"2024-04-25T14:12:22.514346+0000","last_peered":"2024-04-25T14:12:22.514346+0000","last_clean":"2024-04-25T14:12:22.514346+0000","last_became_active":"2024-04-25T14:11:29.558560+0000","last_became_peered":"2024-04-25T14:11:29.558560+0000","last_unstale":"2024-04-25T14:12:22.514346+0000","last_undegraded":"2024-04-25T14:12:22.514346+0000","last_fullsized":"2024-04-25T14:12:22.514346+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:23.677099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"76'7","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.515784+0000","last_change":"2024-04-25T14:11:28.544020+0000","last_active":"2024-04-25T14:12:22.515784+0000","last_peered":"2024-04-25T14:12:22.515784+0000","last_clean":"2024-04-25T14:12:22.515784+0000","last_became_active":"2024-04-25T14:11:28.543598+0000","last_became_peered":"2024-04-25T14:11:28.543598+0000","last_unstale":"2024-04-25T14:12:22.515784+0000","last_undegraded":"2024-04-25T14:12:22.515784+0000","last_fullsized":"2024-04-25T14:12:22.515784+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:49:06.543876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.508777+0000","last_change":"2024-04-25T14:11:29.576122+0000","last_active":"2024-04-25T14:12:22.508777+0000","last_peered":"2024-04-25T14:12:22.508777+0000","last_clean":"2024-04-25T14:12:22.508777+0000","last_became_active":"2024-04-25T14:11:29.573668+0000","last_became_peered":"2024-04-25T14:11:29.573668+0000","last_unstale":"2024-04-25T14:12:22.508777+0000","last_undegraded":"2024-04-25T14:12:22.508777+0000","last_fullsized":"2024-04-25T14:12:22.508777+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:06.998876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"76'3","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.505016+0000","last_change":"2024-04-25T14:11:28.541554+0000","last_active":"2024-04-25T14:12:22.505016+0000","last_peered":"2024-04-25T14:12:22.505016+0000","last_clean":"2024-04-25T14:12:22.505016+0000","last_became_active":"2024-04-25T14:11:28.541062+0000","last_became_peered":"2024-04-25T14:11:28.541062+0000","last_unstale":"2024-04-25T14:12:22.505016+0000","last_undegraded":"2024-04-25T14:12:22.505016+0000","last_fullsized":"2024-04-25T14:12:22.505016+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:16.719820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.513986+0000","last_change":"2024-04-25T14:11:29.564550+0000","last_active":"2024-04-25T14:12:22.513986+0000","last_peered":"2024-04-25T14:12:22.513986+0000","last_clean":"2024-04-25T14:12:22.513986+0000","last_became_active":"2024-04-25T14:11:29.564193+0000","last_became_peered":"2024-04-25T14:11:29.564193+0000","last_unstale":"2024-04-25T14:12:22.513986+0000","last_undegraded":"2024-04-25T14:12:22.513986+0000","last_fullsized":"2024-04-25T14:12:22.513986+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:59.357327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"76'3","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.508406+0000","last_change":"2024-04-25T14:11:28.540504+0000","last_active":"2024-04-25T14:12:22.508406+0000","last_peered":"2024-04-25T14:12:22.508406+0000","last_clean":"2024-04-25T14:12:22.508406+0000","last_became_active":"2024-04-25T14:11:28.540150+0000","last_became_peered":"2024-04-25T14:11:28.540150+0000","last_unstale":"2024-04-25T14:12:22.508406+0000","last_undegraded":"2024-04-25T14:12:22.508406+0000","last_fullsized":"2024-04-25T14:12:22.508406+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:35.331489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.508786+0000","last_change":"2024-04-25T14:11:29.561226+0000","last_active":"2024-04-25T14:12:22.508786+0000","last_peered":"2024-04-25T14:12:22.508786+0000","last_clean":"2024-04-25T14:12:22.508786+0000","last_became_active":"2024-04-25T14:11:29.560847+0000","last_became_peered":"2024-04-25T14:11:29.560847+0000","last_unstale":"2024-04-25T14:12:22.508786+0000","last_undegraded":"2024-04-25T14:12:22.508786+0000","last_fullsized":"2024-04-25T14:12:22.508786+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:05.299388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"76'4","reported_seq":168,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:45.620029+0000","last_change":"2024-04-25T14:11:28.548580+0000","last_active":"2024-04-25T14:12:45.620029+0000","last_peered":"2024-04-25T14:12:45.620029+0000","last_clean":"2024-04-25T14:12:45.620029+0000","last_became_active":"2024-04-25T14:11:28.548250+0000","last_became_peered":"2024-04-25T14:11:28.548250+0000","last_unstale":"2024-04-25T14:12:45.620029+0000","last_undegraded":"2024-04-25T14:12:45.620029+0000","last_fullsized":"2024-04-25T14:12:45.620029+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:01.166285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":145,"num_read_kb":145,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.511606+0000","last_change":"2024-04-25T14:11:29.575983+0000","last_active":"2024-04-25T14:12:22.511606+0000","last_peered":"2024-04-25T14:12:22.511606+0000","last_clean":"2024-04-25T14:12:22.511606+0000","last_became_active":"2024-04-25T14:11:29.575147+0000","last_became_peered":"2024-04-25T14:11:29.575147+0000","last_unstale":"2024-04-25T14:12:22.511606+0000","last_undegraded":"2024-04-25T14:12:22.511606+0000","last_fullsized":"2024-04-25T14:12:22.511606+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:40:45.366736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"76'4","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.530419+0000","last_change":"2024-04-25T14:12:22.530419+0000","last_active":"2024-04-25T14:12:22.530419+0000","last_peered":"2024-04-25T14:12:22.530419+0000","last_clean":"2024-04-25T14:12:22.530419+0000","last_became_active":"2024-04-25T14:12:22.514999+0000","last_became_peered":"2024-04-25T14:12:22.514999+0000","last_unstale":"2024-04-25T14:12:22.530419+0000","last_undegraded":"2024-04-25T14:12:22.530419+0000","last_fullsized":"2024-04-25T14:12:22.530419+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:02.075270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.509571+0000","last_change":"2024-04-25T14:11:29.581424+0000","last_active":"2024-04-25T14:12:22.509571+0000","last_peered":"2024-04-25T14:12:22.509571+0000","last_clean":"2024-04-25T14:12:22.509571+0000","last_became_active":"2024-04-25T14:11:29.581086+0000","last_became_peered":"2024-04-25T14:11:29.581086+0000","last_unstale":"2024-04-25T14:12:22.509571+0000","last_undegraded":"2024-04-25T14:12:22.509571+0000","last_fullsized":"2024-04-25T14:12:22.509571+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:30.468728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"76'5","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.508977+0000","last_change":"2024-04-25T14:11:28.540771+0000","last_active":"2024-04-25T14:12:22.508977+0000","last_peered":"2024-04-25T14:12:22.508977+0000","last_clean":"2024-04-25T14:12:22.508977+0000","last_became_active":"2024-04-25T14:11:28.540442+0000","last_became_peered":"2024-04-25T14:11:28.540442+0000","last_unstale":"2024-04-25T14:12:22.508977+0000","last_undegraded":"2024-04-25T14:12:22.508977+0000","last_fullsized":"2024-04-25T14:12:22.508977+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:52.479423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.517721+0000","last_change":"2024-04-25T14:12:22.517721+0000","last_active":"2024-04-25T14:12:22.517721+0000","last_peered":"2024-04-25T14:12:22.517721+0000","last_clean":"2024-04-25T14:12:22.517721+0000","last_became_active":"2024-04-25T14:12:22.517232+0000","last_became_peered":"2024-04-25T14:12:22.517232+0000","last_unstale":"2024-04-25T14:12:22.517721+0000","last_undegraded":"2024-04-25T14:12:22.517721+0000","last_fullsized":"2024-04-25T14:12:22.517721+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:46.150626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"76'29","reported_seq":203,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:45.620056+0000","last_change":"2024-04-25T14:11:28.543093+0000","last_active":"2024-04-25T14:12:45.620056+0000","last_peered":"2024-04-25T14:12:45.620056+0000","last_clean":"2024-04-25T14:12:45.620056+0000","last_became_active":"2024-04-25T14:11:28.542407+0000","last_became_peered":"2024-04-25T14:11:28.542407+0000","last_unstale":"2024-04-25T14:12:45.620056+0000","last_undegraded":"2024-04-25T14:12:45.620056+0000","last_fullsized":"2024-04-25T14:12:45.620056+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:43.767375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59394,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":155,"num_read_kb":135,"num_write":31,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514402+0000","last_change":"2024-04-25T14:11:29.571896+0000","last_active":"2024-04-25T14:12:22.514402+0000","last_peered":"2024-04-25T14:12:22.514402+0000","last_clean":"2024-04-25T14:12:22.514402+0000","last_became_active":"2024-04-25T14:11:29.571537+0000","last_became_peered":"2024-04-25T14:11:29.571537+0000","last_unstale":"2024-04-25T14:12:22.514402+0000","last_undegraded":"2024-04-25T14:12:22.514402+0000","last_fullsized":"2024-04-25T14:12:22.514402+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:32:07.914841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"76'9","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.530152+0000","last_change":"2024-04-25T14:12:22.529692+0000","last_active":"2024-04-25T14:12:22.530152+0000","last_peered":"2024-04-25T14:12:22.530152+0000","last_clean":"2024-04-25T14:12:22.530152+0000","last_became_active":"2024-04-25T14:12:22.517225+0000","last_became_peered":"2024-04-25T14:12:22.517225+0000","last_unstale":"2024-04-25T14:12:22.530152+0000","last_undegraded":"2024-04-25T14:12:22.530152+0000","last_fullsized":"2024-04-25T14:12:22.530152+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:36.610551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.504867+0000","last_change":"2024-04-25T14:11:29.563877+0000","last_active":"2024-04-25T14:12:22.504867+0000","last_peered":"2024-04-25T14:12:22.504867+0000","last_clean":"2024-04-25T14:12:22.504867+0000","last_became_active":"2024-04-25T14:11:29.563535+0000","last_became_peered":"2024-04-25T14:11:29.563535+0000","last_unstale":"2024-04-25T14:12:22.504867+0000","last_undegraded":"2024-04-25T14:12:22.504867+0000","last_fullsized":"2024-04-25T14:12:22.504867+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:07:38.805863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"76'5","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.508526+0000","last_change":"2024-04-25T14:11:28.543787+0000","last_active":"2024-04-25T14:12:22.508526+0000","last_peered":"2024-04-25T14:12:22.508526+0000","last_clean":"2024-04-25T14:12:22.508526+0000","last_became_active":"2024-04-25T14:11:28.540816+0000","last_became_peered":"2024-04-25T14:11:28.540816+0000","last_unstale":"2024-04-25T14:12:22.508526+0000","last_undegraded":"2024-04-25T14:12:22.508526+0000","last_fullsized":"2024-04-25T14:12:22.508526+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:18.776280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514028+0000","last_change":"2024-04-25T14:12:22.514028+0000","last_active":"2024-04-25T14:12:22.514028+0000","last_peered":"2024-04-25T14:12:22.514028+0000","last_clean":"2024-04-25T14:12:22.514028+0000","last_became_active":"2024-04-25T14:12:22.513536+0000","last_became_peered":"2024-04-25T14:12:22.513536+0000","last_unstale":"2024-04-25T14:12:22.514028+0000","last_undegraded":"2024-04-25T14:12:22.514028+0000","last_fullsized":"2024-04-25T14:12:22.514028+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:31.921284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"76'6","reported_seq":98,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:44.636267+0000","last_change":"2024-04-25T14:11:28.537450+0000","last_active":"2024-04-25T14:12:44.636267+0000","last_peered":"2024-04-25T14:12:44.636267+0000","last_clean":"2024-04-25T14:12:44.636267+0000","last_became_active":"2024-04-25T14:11:28.537088+0000","last_became_peered":"2024-04-25T14:11:28.537088+0000","last_unstale":"2024-04-25T14:12:44.636267+0000","last_undegraded":"2024-04-25T14:12:44.636267+0000","last_fullsized":"2024-04-25T14:12:44.636267+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:05.779706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.508849+0000","last_change":"2024-04-25T14:11:29.572789+0000","last_active":"2024-04-25T14:12:22.508849+0000","last_peered":"2024-04-25T14:12:22.508849+0000","last_clean":"2024-04-25T14:12:22.508849+0000","last_became_active":"2024-04-25T14:11:29.572201+0000","last_became_peered":"2024-04-25T14:11:29.572201+0000","last_unstale":"2024-04-25T14:12:22.508849+0000","last_undegraded":"2024-04-25T14:12:22.508849+0000","last_fullsized":"2024-04-25T14:12:22.508849+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:39:58.237070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"76'17","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514338+0000","last_change":"2024-04-25T14:11:28.540577+0000","last_active":"2024-04-25T14:12:22.514338+0000","last_peered":"2024-04-25T14:12:22.514338+0000","last_clean":"2024-04-25T14:12:22.514338+0000","last_became_active":"2024-04-25T14:11:28.540212+0000","last_became_peered":"2024-04-25T14:11:28.540212+0000","last_unstale":"2024-04-25T14:12:22.514338+0000","last_undegraded":"2024-04-25T14:12:22.514338+0000","last_fullsized":"2024-04-25T14:12:22.514338+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:31.422332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.508850+0000","last_change":"2024-04-25T14:11:29.574952+0000","last_active":"2024-04-25T14:12:22.508850+0000","last_peered":"2024-04-25T14:12:22.508850+0000","last_clean":"2024-04-25T14:12:22.508850+0000","last_became_active":"2024-04-25T14:11:29.573677+0000","last_became_peered":"2024-04-25T14:11:29.573677+0000","last_unstale":"2024-04-25T14:12:22.508850+0000","last_undegraded":"2024-04-25T14:12:22.508850+0000","last_fullsized":"2024-04-25T14:12:22.508850+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:35.896103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"76'2","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514622+0000","last_change":"2024-04-25T14:11:28.541356+0000","last_active":"2024-04-25T14:12:22.514622+0000","last_peered":"2024-04-25T14:12:22.514622+0000","last_clean":"2024-04-25T14:12:22.514622+0000","last_became_active":"2024-04-25T14:11:28.540976+0000","last_became_peered":"2024-04-25T14:11:28.540976+0000","last_unstale":"2024-04-25T14:12:22.514622+0000","last_undegraded":"2024-04-25T14:12:22.514622+0000","last_fullsized":"2024-04-25T14:12:22.514622+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:32:50.599900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.511031+0000","last_change":"2024-04-25T14:11:29.554317+0000","last_active":"2024-04-25T14:12:22.511031+0000","last_peered":"2024-04-25T14:12:22.511031+0000","last_clean":"2024-04-25T14:12:22.511031+0000","last_became_active":"2024-04-25T14:11:29.553935+0000","last_became_peered":"2024-04-25T14:11:29.553935+0000","last_unstale":"2024-04-25T14:12:22.511031+0000","last_undegraded":"2024-04-25T14:12:22.511031+0000","last_fullsized":"2024-04-25T14:12:22.511031+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:01.405964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"76'1","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.509026+0000","last_change":"2024-04-25T14:11:28.546892+0000","last_active":"2024-04-25T14:12:22.509026+0000","last_peered":"2024-04-25T14:12:22.509026+0000","last_clean":"2024-04-25T14:12:22.509026+0000","last_became_active":"2024-04-25T14:11:28.546566+0000","last_became_peered":"2024-04-25T14:11:28.546566+0000","last_unstale":"2024-04-25T14:12:22.509026+0000","last_undegraded":"2024-04-25T14:12:22.509026+0000","last_fullsized":"2024-04-25T14:12:22.509026+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:05.239635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"76'1","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.516285+0000","last_change":"2024-04-25T14:11:29.565083+0000","last_active":"2024-04-25T14:12:22.516285+0000","last_peered":"2024-04-25T14:12:22.516285+0000","last_clean":"2024-04-25T14:12:22.516285+0000","last_became_active":"2024-04-25T14:11:29.564062+0000","last_became_peered":"2024-04-25T14:11:29.564062+0000","last_unstale":"2024-04-25T14:12:22.516285+0000","last_undegraded":"2024-04-25T14:12:22.516285+0000","last_fullsized":"2024-04-25T14:12:22.516285+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:07:10.664113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"76'5","reported_seq":96,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:45.548074+0000","last_change":"2024-04-25T14:11:28.538649+0000","last_active":"2024-04-25T14:12:45.548074+0000","last_peered":"2024-04-25T14:12:45.548074+0000","last_clean":"2024-04-25T14:12:45.548074+0000","last_became_active":"2024-04-25T14:11:28.538353+0000","last_became_peered":"2024-04-25T14:11:28.538353+0000","last_unstale":"2024-04-25T14:12:45.548074+0000","last_undegraded":"2024-04-25T14:12:45.548074+0000","last_fullsized":"2024-04-25T14:12:45.548074+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:06.513039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.516066+0000","last_change":"2024-04-25T14:11:29.576263+0000","last_active":"2024-04-25T14:12:22.516066+0000","last_peered":"2024-04-25T14:12:22.516066+0000","last_clean":"2024-04-25T14:12:22.516066+0000","last_became_active":"2024-04-25T14:11:29.575877+0000","last_became_peered":"2024-04-25T14:11:29.575877+0000","last_unstale":"2024-04-25T14:12:22.516066+0000","last_undegraded":"2024-04-25T14:12:22.516066+0000","last_fullsized":"2024-04-25T14:12:22.516066+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:33:37.639537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.508779+0000","last_change":"2024-04-25T14:11:29.568220+0000","last_active":"2024-04-25T14:12:22.508779+0000","last_peered":"2024-04-25T14:12:22.508779+0000","last_clean":"2024-04-25T14:12:22.508779+0000","last_became_active":"2024-04-25T14:11:29.567741+0000","last_became_peered":"2024-04-25T14:11:29.567741+0000","last_unstale":"2024-04-25T14:12:22.508779+0000","last_undegraded":"2024-04-25T14:12:22.508779+0000","last_fullsized":"2024-04-25T14:12:22.508779+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:05.134466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"76'5","reported_seq":97,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:45.637173+0000","last_change":"2024-04-25T14:11:28.537503+0000","last_active":"2024-04-25T14:12:45.637173+0000","last_peered":"2024-04-25T14:12:45.637173+0000","last_clean":"2024-04-25T14:12:45.637173+0000","last_became_active":"2024-04-25T14:11:28.537169+0000","last_became_peered":"2024-04-25T14:11:28.537169+0000","last_unstale":"2024-04-25T14:12:45.637173+0000","last_undegraded":"2024-04-25T14:12:45.637173+0000","last_fullsized":"2024-04-25T14:12:45.637173+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:09.752988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.516965+0000","last_change":"2024-04-25T14:12:22.516273+0000","last_active":"2024-04-25T14:12:22.516965+0000","last_peered":"2024-04-25T14:12:22.516965+0000","last_clean":"2024-04-25T14:12:22.516965+0000","last_became_active":"2024-04-25T14:12:22.515737+0000","last_became_peered":"2024-04-25T14:12:22.515737+0000","last_unstale":"2024-04-25T14:12:22.516965+0000","last_undegraded":"2024-04-25T14:12:22.516965+0000","last_fullsized":"2024-04-25T14:12:22.516965+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:36.241587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"76'6","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.512277+0000","last_change":"2024-04-25T14:11:28.538216+0000","last_active":"2024-04-25T14:12:22.512277+0000","last_peered":"2024-04-25T14:12:22.512277+0000","last_clean":"2024-04-25T14:12:22.512277+0000","last_became_active":"2024-04-25T14:11:28.537772+0000","last_became_peered":"2024-04-25T14:11:28.537772+0000","last_unstale":"2024-04-25T14:12:22.512277+0000","last_undegraded":"2024-04-25T14:12:22.512277+0000","last_fullsized":"2024-04-25T14:12:22.512277+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:55.460966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":93,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.508948+0000","last_change":"2024-04-25T14:10:12.194216+0000","last_active":"2024-04-25T14:12:22.508948+0000","last_peered":"2024-04-25T14:12:22.508948+0000","last_clean":"2024-04-25T14:12:22.508948+0000","last_became_active":"2024-04-25T14:10:12.189616+0000","last_became_peered":"2024-04-25T14:10:12.189616+0000","last_unstale":"2024-04-25T14:12:22.508948+0000","last_undegraded":"2024-04-25T14:12:22.508948+0000","last_fullsized":"2024-04-25T14:12:22.508948+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:47.553264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:47.553264+0000","last_clean_scrub_stamp":"2024-04-25T14:06:47.553264+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:47.576058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.504798+0000","last_change":"2024-04-25T14:11:29.574985+0000","last_active":"2024-04-25T14:12:22.504798+0000","last_peered":"2024-04-25T14:12:22.504798+0000","last_clean":"2024-04-25T14:12:22.504798+0000","last_became_active":"2024-04-25T14:11:29.574629+0000","last_became_peered":"2024-04-25T14:11:29.574629+0000","last_unstale":"2024-04-25T14:12:22.504798+0000","last_undegraded":"2024-04-25T14:12:22.504798+0000","last_fullsized":"2024-04-25T14:12:22.504798+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:01.899889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"76'4","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.520398+0000","last_change":"2024-04-25T14:12:22.519851+0000","last_active":"2024-04-25T14:12:22.520398+0000","last_peered":"2024-04-25T14:12:22.520398+0000","last_clean":"2024-04-25T14:12:22.520398+0000","last_became_active":"2024-04-25T14:12:22.515791+0000","last_became_peered":"2024-04-25T14:12:22.515791+0000","last_unstale":"2024-04-25T14:12:22.520398+0000","last_undegraded":"2024-04-25T14:12:22.520398+0000","last_fullsized":"2024-04-25T14:12:22.520398+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:47:01.242259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"76'4","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514394+0000","last_change":"2024-04-25T14:11:28.544269+0000","last_active":"2024-04-25T14:12:22.514394+0000","last_peered":"2024-04-25T14:12:22.514394+0000","last_clean":"2024-04-25T14:12:22.514394+0000","last_became_active":"2024-04-25T14:11:28.543897+0000","last_became_peered":"2024-04-25T14:11:28.543897+0000","last_unstale":"2024-04-25T14:12:22.514394+0000","last_undegraded":"2024-04-25T14:12:22.514394+0000","last_fullsized":"2024-04-25T14:12:22.514394+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:37.668187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.505336+0000","last_change":"2024-04-25T14:11:29.566428+0000","last_active":"2024-04-25T14:12:22.505336+0000","last_peered":"2024-04-25T14:12:22.505336+0000","last_clean":"2024-04-25T14:12:22.505336+0000","last_became_active":"2024-04-25T14:11:29.562598+0000","last_became_peered":"2024-04-25T14:11:29.562598+0000","last_unstale":"2024-04-25T14:12:22.505336+0000","last_undegraded":"2024-04-25T14:12:22.505336+0000","last_fullsized":"2024-04-25T14:12:22.505336+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:46.611359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"76'12","reported_seq":31,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.509486+0000","last_change":"2024-04-25T14:11:28.542571+0000","last_active":"2024-04-25T14:12:22.509486+0000","last_peered":"2024-04-25T14:12:22.509486+0000","last_clean":"2024-04-25T14:12:22.509486+0000","last_became_active":"2024-04-25T14:11:28.542272+0000","last_became_peered":"2024-04-25T14:11:28.542272+0000","last_unstale":"2024-04-25T14:12:22.509486+0000","last_undegraded":"2024-04-25T14:12:22.509486+0000","last_fullsized":"2024-04-25T14:12:22.509486+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:41:05.128068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514749+0000","last_change":"2024-04-25T14:11:29.578565+0000","last_active":"2024-04-25T14:12:22.514749+0000","last_peered":"2024-04-25T14:12:22.514749+0000","last_clean":"2024-04-25T14:12:22.514749+0000","last_became_active":"2024-04-25T14:11:29.576782+0000","last_became_peered":"2024-04-25T14:11:29.576782+0000","last_unstale":"2024-04-25T14:12:22.514749+0000","last_undegraded":"2024-04-25T14:12:22.514749+0000","last_fullsized":"2024-04-25T14:12:22.514749+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:31.634602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"76'7","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.511746+0000","last_change":"2024-04-25T14:11:28.543838+0000","last_active":"2024-04-25T14:12:22.511746+0000","last_peered":"2024-04-25T14:12:22.511746+0000","last_clean":"2024-04-25T14:12:22.511746+0000","last_became_active":"2024-04-25T14:11:28.543511+0000","last_became_peered":"2024-04-25T14:11:28.543511+0000","last_unstale":"2024-04-25T14:12:22.511746+0000","last_undegraded":"2024-04-25T14:12:22.511746+0000","last_fullsized":"2024-04-25T14:12:22.511746+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:34.440234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.516416+0000","last_change":"2024-04-25T14:12:22.516416+0000","last_active":"2024-04-25T14:12:22.516416+0000","last_peered":"2024-04-25T14:12:22.516416+0000","last_clean":"2024-04-25T14:12:22.516416+0000","last_became_active":"2024-04-25T14:12:22.515928+0000","last_became_peered":"2024-04-25T14:12:22.515928+0000","last_unstale":"2024-04-25T14:12:22.516416+0000","last_undegraded":"2024-04-25T14:12:22.516416+0000","last_fullsized":"2024-04-25T14:12:22.516416+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:26.989293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"76'2","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.512719+0000","last_change":"2024-04-25T14:11:28.539443+0000","last_active":"2024-04-25T14:12:22.512719+0000","last_peered":"2024-04-25T14:12:22.512719+0000","last_clean":"2024-04-25T14:12:22.512719+0000","last_became_active":"2024-04-25T14:11:28.539113+0000","last_became_peered":"2024-04-25T14:11:28.539113+0000","last_unstale":"2024-04-25T14:12:22.512719+0000","last_undegraded":"2024-04-25T14:12:22.512719+0000","last_fullsized":"2024-04-25T14:12:22.512719+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:36.664972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514061+0000","last_change":"2024-04-25T14:11:29.575451+0000","last_active":"2024-04-25T14:12:22.514061+0000","last_peered":"2024-04-25T14:12:22.514061+0000","last_clean":"2024-04-25T14:12:22.514061+0000","last_became_active":"2024-04-25T14:11:29.574983+0000","last_became_peered":"2024-04-25T14:11:29.574983+0000","last_unstale":"2024-04-25T14:12:22.514061+0000","last_undegraded":"2024-04-25T14:12:22.514061+0000","last_fullsized":"2024-04-25T14:12:22.514061+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:53.317608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"76'7","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514234+0000","last_change":"2024-04-25T14:11:28.546394+0000","last_active":"2024-04-25T14:12:22.514234+0000","last_peered":"2024-04-25T14:12:22.514234+0000","last_clean":"2024-04-25T14:12:22.514234+0000","last_became_active":"2024-04-25T14:11:28.546064+0000","last_became_peered":"2024-04-25T14:11:28.546064+0000","last_unstale":"2024-04-25T14:12:22.514234+0000","last_undegraded":"2024-04-25T14:12:22.514234+0000","last_fullsized":"2024-04-25T14:12:22.514234+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:15.301492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.512276+0000","last_change":"2024-04-25T14:11:29.569584+0000","last_active":"2024-04-25T14:12:22.512276+0000","last_peered":"2024-04-25T14:12:22.512276+0000","last_clean":"2024-04-25T14:12:22.512276+0000","last_became_active":"2024-04-25T14:11:29.569217+0000","last_became_peered":"2024-04-25T14:11:29.569217+0000","last_unstale":"2024-04-25T14:12:22.512276+0000","last_undegraded":"2024-04-25T14:12:22.512276+0000","last_fullsized":"2024-04-25T14:12:22.512276+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:05.708277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"76'3","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.512047+0000","last_change":"2024-04-25T14:11:28.544399+0000","last_active":"2024-04-25T14:12:22.512047+0000","last_peered":"2024-04-25T14:12:22.512047+0000","last_clean":"2024-04-25T14:12:22.512047+0000","last_became_active":"2024-04-25T14:11:28.544098+0000","last_became_peered":"2024-04-25T14:11:28.544098+0000","last_unstale":"2024-04-25T14:12:22.512047+0000","last_undegraded":"2024-04-25T14:12:22.512047+0000","last_fullsized":"2024-04-25T14:12:22.512047+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:05.148386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.518268+0000","last_change":"2024-04-25T14:12:22.517727+0000","last_active":"2024-04-25T14:12:22.518268+0000","last_peered":"2024-04-25T14:12:22.518268+0000","last_clean":"2024-04-25T14:12:22.518268+0000","last_became_active":"2024-04-25T14:12:22.517364+0000","last_became_peered":"2024-04-25T14:12:22.517364+0000","last_unstale":"2024-04-25T14:12:22.518268+0000","last_undegraded":"2024-04-25T14:12:22.518268+0000","last_fullsized":"2024-04-25T14:12:22.518268+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:30:11.023308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"76'4","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514211+0000","last_change":"2024-04-25T14:11:28.542488+0000","last_active":"2024-04-25T14:12:22.514211+0000","last_peered":"2024-04-25T14:12:22.514211+0000","last_clean":"2024-04-25T14:12:22.514211+0000","last_became_active":"2024-04-25T14:11:28.542171+0000","last_became_peered":"2024-04-25T14:11:28.542171+0000","last_unstale":"2024-04-25T14:12:22.514211+0000","last_undegraded":"2024-04-25T14:12:22.514211+0000","last_fullsized":"2024-04-25T14:12:22.514211+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:48.201762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.509671+0000","last_change":"2024-04-25T14:11:29.573805+0000","last_active":"2024-04-25T14:12:22.509671+0000","last_peered":"2024-04-25T14:12:22.509671+0000","last_clean":"2024-04-25T14:12:22.509671+0000","last_became_active":"2024-04-25T14:11:29.573420+0000","last_became_peered":"2024-04-25T14:11:29.573420+0000","last_unstale":"2024-04-25T14:12:22.509671+0000","last_undegraded":"2024-04-25T14:12:22.509671+0000","last_fullsized":"2024-04-25T14:12:22.509671+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:18.721977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"76'5","reported_seq":96,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:43.637431+0000","last_change":"2024-04-25T14:11:28.552863+0000","last_active":"2024-04-25T14:12:43.637431+0000","last_peered":"2024-04-25T14:12:43.637431+0000","last_clean":"2024-04-25T14:12:43.637431+0000","last_became_active":"2024-04-25T14:11:28.552461+0000","last_became_peered":"2024-04-25T14:11:28.552461+0000","last_unstale":"2024-04-25T14:12:43.637431+0000","last_undegraded":"2024-04-25T14:12:43.637431+0000","last_fullsized":"2024-04-25T14:12:43.637431+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:46.264473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":72,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.509551+0000","last_change":"2024-04-25T14:11:29.571268+0000","last_active":"2024-04-25T14:12:22.509551+0000","last_peered":"2024-04-25T14:12:22.509551+0000","last_clean":"2024-04-25T14:12:22.509551+0000","last_became_active":"2024-04-25T14:11:29.570876+0000","last_became_peered":"2024-04-25T14:11:29.570876+0000","last_unstale":"2024-04-25T14:12:22.509551+0000","last_undegraded":"2024-04-25T14:12:22.509551+0000","last_fullsized":"2024-04-25T14:12:22.509551+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:15.093635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"76'3","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.508764+0000","last_change":"2024-04-25T14:11:28.543343+0000","last_active":"2024-04-25T14:12:22.508764+0000","last_peered":"2024-04-25T14:12:22.508764+0000","last_clean":"2024-04-25T14:12:22.508764+0000","last_became_active":"2024-04-25T14:11:28.540790+0000","last_became_peered":"2024-04-25T14:11:28.540790+0000","last_unstale":"2024-04-25T14:12:22.508764+0000","last_undegraded":"2024-04-25T14:12:22.508764+0000","last_fullsized":"2024-04-25T14:12:22.508764+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:20.498016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"76'1","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.509418+0000","last_change":"2024-04-25T14:11:29.566161+0000","last_active":"2024-04-25T14:12:22.509418+0000","last_peered":"2024-04-25T14:12:22.509418+0000","last_clean":"2024-04-25T14:12:22.509418+0000","last_became_active":"2024-04-25T14:11:29.564197+0000","last_became_peered":"2024-04-25T14:11:29.564197+0000","last_unstale":"2024-04-25T14:12:22.509418+0000","last_undegraded":"2024-04-25T14:12:22.509418+0000","last_fullsized":"2024-04-25T14:12:22.509418+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:35.291546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"76'5","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.505214+0000","last_change":"2024-04-25T14:11:28.543479+0000","last_active":"2024-04-25T14:12:22.505214+0000","last_peered":"2024-04-25T14:12:22.505214+0000","last_clean":"2024-04-25T14:12:22.505214+0000","last_became_active":"2024-04-25T14:11:28.543167+0000","last_became_peered":"2024-04-25T14:11:28.543167+0000","last_unstale":"2024-04-25T14:12:22.505214+0000","last_undegraded":"2024-04-25T14:12:22.505214+0000","last_fullsized":"2024-04-25T14:12:22.505214+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:54:53.572809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.509700+0000","last_change":"2024-04-25T14:11:29.572548+0000","last_active":"2024-04-25T14:12:22.509700+0000","last_peered":"2024-04-25T14:12:22.509700+0000","last_clean":"2024-04-25T14:12:22.509700+0000","last_became_active":"2024-04-25T14:11:29.572167+0000","last_became_peered":"2024-04-25T14:11:29.572167+0000","last_unstale":"2024-04-25T14:12:22.509700+0000","last_undegraded":"2024-04-25T14:12:22.509700+0000","last_fullsized":"2024-04-25T14:12:22.509700+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:38.423331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.512408+0000","last_change":"2024-04-25T14:11:29.560956+0000","last_active":"2024-04-25T14:12:22.512408+0000","last_peered":"2024-04-25T14:12:22.512408+0000","last_clean":"2024-04-25T14:12:22.512408+0000","last_became_active":"2024-04-25T14:11:29.560519+0000","last_became_peered":"2024-04-25T14:11:29.560519+0000","last_unstale":"2024-04-25T14:12:22.512408+0000","last_undegraded":"2024-04-25T14:12:22.512408+0000","last_fullsized":"2024-04-25T14:12:22.512408+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:53:47.677923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.512828+0000","last_change":"2024-04-25T14:11:29.574372+0000","last_active":"2024-04-25T14:12:22.512828+0000","last_peered":"2024-04-25T14:12:22.512828+0000","last_clean":"2024-04-25T14:12:22.512828+0000","last_became_active":"2024-04-25T14:11:29.573995+0000","last_became_peered":"2024-04-25T14:11:29.573995+0000","last_unstale":"2024-04-25T14:12:22.512828+0000","last_undegraded":"2024-04-25T14:12:22.512828+0000","last_fullsized":"2024-04-25T14:12:22.512828+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:12.381512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.512034+0000","last_change":"2024-04-25T14:11:29.578324+0000","last_active":"2024-04-25T14:12:22.512034+0000","last_peered":"2024-04-25T14:12:22.512034+0000","last_clean":"2024-04-25T14:12:22.512034+0000","last_became_active":"2024-04-25T14:11:29.577917+0000","last_became_peered":"2024-04-25T14:11:29.577917+0000","last_unstale":"2024-04-25T14:12:22.512034+0000","last_undegraded":"2024-04-25T14:12:22.512034+0000","last_fullsized":"2024-04-25T14:12:22.512034+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:20:27.070720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.509075+0000","last_change":"2024-04-25T14:11:29.578879+0000","last_active":"2024-04-25T14:12:22.509075+0000","last_peered":"2024-04-25T14:12:22.509075+0000","last_clean":"2024-04-25T14:12:22.509075+0000","last_became_active":"2024-04-25T14:11:29.578551+0000","last_became_peered":"2024-04-25T14:11:29.578551+0000","last_unstale":"2024-04-25T14:12:22.509075+0000","last_undegraded":"2024-04-25T14:12:22.509075+0000","last_fullsized":"2024-04-25T14:12:22.509075+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:41.083794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.505252+0000","last_change":"2024-04-25T14:11:29.574054+0000","last_active":"2024-04-25T14:12:22.505252+0000","last_peered":"2024-04-25T14:12:22.505252+0000","last_clean":"2024-04-25T14:12:22.505252+0000","last_became_active":"2024-04-25T14:11:29.573700+0000","last_became_peered":"2024-04-25T14:11:29.573700+0000","last_unstale":"2024-04-25T14:12:22.505252+0000","last_undegraded":"2024-04-25T14:12:22.505252+0000","last_fullsized":"2024-04-25T14:12:22.505252+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:23.304580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.510858+0000","last_change":"2024-04-25T14:11:29.553768+0000","last_active":"2024-04-25T14:12:22.510858+0000","last_peered":"2024-04-25T14:12:22.510858+0000","last_clean":"2024-04-25T14:12:22.510858+0000","last_became_active":"2024-04-25T14:11:29.553376+0000","last_became_peered":"2024-04-25T14:11:29.553376+0000","last_unstale":"2024-04-25T14:12:22.510858+0000","last_undegraded":"2024-04-25T14:12:22.510858+0000","last_fullsized":"2024-04-25T14:12:22.510858+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:46.688368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.512688+0000","last_change":"2024-04-25T14:11:29.563122+0000","last_active":"2024-04-25T14:12:22.512688+0000","last_peered":"2024-04-25T14:12:22.512688+0000","last_clean":"2024-04-25T14:12:22.512688+0000","last_became_active":"2024-04-25T14:11:29.562717+0000","last_became_peered":"2024-04-25T14:11:29.562717+0000","last_unstale":"2024-04-25T14:12:22.512688+0000","last_undegraded":"2024-04-25T14:12:22.512688+0000","last_fullsized":"2024-04-25T14:12:22.512688+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:37.585352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514599+0000","last_change":"2024-04-25T14:11:29.576487+0000","last_active":"2024-04-25T14:12:22.514599+0000","last_peered":"2024-04-25T14:12:22.514599+0000","last_clean":"2024-04-25T14:12:22.514599+0000","last_became_active":"2024-04-25T14:11:29.575070+0000","last_became_peered":"2024-04-25T14:11:29.575070+0000","last_unstale":"2024-04-25T14:12:22.514599+0000","last_undegraded":"2024-04-25T14:12:22.514599+0000","last_fullsized":"2024-04-25T14:12:22.514599+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:13.803505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.508462+0000","last_change":"2024-04-25T14:11:29.568733+0000","last_active":"2024-04-25T14:12:22.508462+0000","last_peered":"2024-04-25T14:12:22.508462+0000","last_clean":"2024-04-25T14:12:22.508462+0000","last_became_active":"2024-04-25T14:11:29.567665+0000","last_became_peered":"2024-04-25T14:11:29.567665+0000","last_unstale":"2024-04-25T14:12:22.508462+0000","last_undegraded":"2024-04-25T14:12:22.508462+0000","last_fullsized":"2024-04-25T14:12:22.508462+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:57.956199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514141+0000","last_change":"2024-04-25T14:11:29.574648+0000","last_active":"2024-04-25T14:12:22.514141+0000","last_peered":"2024-04-25T14:12:22.514141+0000","last_clean":"2024-04-25T14:12:22.514141+0000","last_became_active":"2024-04-25T14:11:29.574247+0000","last_became_peered":"2024-04-25T14:11:29.574247+0000","last_unstale":"2024-04-25T14:12:22.514141+0000","last_undegraded":"2024-04-25T14:12:22.514141+0000","last_fullsized":"2024-04-25T14:12:22.514141+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:57.110017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.511877+0000","last_change":"2024-04-25T14:11:29.573633+0000","last_active":"2024-04-25T14:12:22.511877+0000","last_peered":"2024-04-25T14:12:22.511877+0000","last_clean":"2024-04-25T14:12:22.511877+0000","last_became_active":"2024-04-25T14:11:29.573303+0000","last_became_peered":"2024-04-25T14:11:29.573303+0000","last_unstale":"2024-04-25T14:12:22.511877+0000","last_undegraded":"2024-04-25T14:12:22.511877+0000","last_fullsized":"2024-04-25T14:12:22.511877+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:11.458117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.505212+0000","last_change":"2024-04-25T14:11:29.569136+0000","last_active":"2024-04-25T14:12:22.505212+0000","last_peered":"2024-04-25T14:12:22.505212+0000","last_clean":"2024-04-25T14:12:22.505212+0000","last_became_active":"2024-04-25T14:11:29.568801+0000","last_became_peered":"2024-04-25T14:11:29.568801+0000","last_unstale":"2024-04-25T14:12:22.505212+0000","last_undegraded":"2024-04-25T14:12:22.505212+0000","last_fullsized":"2024-04-25T14:12:22.505212+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:30.969081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.510724+0000","last_change":"2024-04-25T14:11:29.566754+0000","last_active":"2024-04-25T14:12:22.510724+0000","last_peered":"2024-04-25T14:12:22.510724+0000","last_clean":"2024-04-25T14:12:22.510724+0000","last_became_active":"2024-04-25T14:11:29.566368+0000","last_became_peered":"2024-04-25T14:11:29.566368+0000","last_unstale":"2024-04-25T14:12:22.510724+0000","last_undegraded":"2024-04-25T14:12:22.510724+0000","last_fullsized":"2024-04-25T14:12:22.510724+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:37.057718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281277,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66598,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":752,"num_read_kb":446,"num_write":236,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":740,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":199794,"data_compressed":5550,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678046,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":851968,"data_stored":405341,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874275,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":995328,"data_stored":1056087,"data_compressed":43412,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037799,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":868352,"data_stored":405203,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234030,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32020,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713132,"statfs":{"total":95995035648,"available":95962247168,"internally_reserved":0,"allocated":856064,"data_stored":404909,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430258,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":917504,"data_stored":998490,"data_compressed":41562,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593784,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":851968,"data_stored":404920,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":465227,"data_compressed":7643,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738454,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":872448,"data_stored":406259,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":465770,"data_compressed":7643,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149767,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":868352,"data_stored":404745,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953537,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":860160,"data_stored":405844,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790013,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":997777,"data_compressed":41562,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":61146,"data_compressed":1850,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":61689,"data_compressed":1850,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":61521,"data_compressed":1850,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:12:50.192 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T14:12:50.430 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:12:50.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:50 smithi096 ceph-mon[29207]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T14:12:50.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:50 smithi096 ceph-mon[29207]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:12:50.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:50 smithi097 ceph-mon[32720]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T14:12:50.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:50 smithi097 ceph-mon[32720]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:12:50.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:50 smithi151 ceph-mon[32990]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T14:12:50.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:50 smithi151 ceph-mon[32990]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:12:51.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:51 smithi096 ceph-mon[29207]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T14:12:51.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:51 smithi096 ceph-mon[29207]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T14:12:51.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:51 smithi097 ceph-mon[32720]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T14:12:51.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:51 smithi097 ceph-mon[32720]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T14:12:51.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:51 smithi151 ceph-mon[32990]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T14:12:51.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:51 smithi151 ceph-mon[32990]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T14:12:52.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.183+0000 7fface920700 1 -- 172.21.15.96:0/2609292583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffac8102890 msgr2=0x7ffac8102ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:52.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.183+0000 7fface920700 1 --2- 172.21.15.96:0/2609292583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffac8102890 0x7ffac8102ca0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ffabc00aaa0 tx=0x7ffabc00adb0 comp rx=0 tx=0).stop 2024-04-25T14:12:52.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.183+0000 7fface920700 1 -- 172.21.15.96:0/2609292583 shutdown_connections 2024-04-25T14:12:52.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.183+0000 7fface920700 1 --2- 172.21.15.96:0/2609292583 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffac81031e0 0x7ffac810f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:52.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.183+0000 7fface920700 1 --2- 172.21.15.96:0/2609292583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffac81088c0 0x7ffac8108cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:52.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.183+0000 7fface920700 1 --2- 172.21.15.96:0/2609292583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffac8102890 0x7ffac8102ca0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:52.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.183+0000 7fface920700 1 -- 172.21.15.96:0/2609292583 >> 172.21.15.96:0/2609292583 conn(0x7ffac80fe3f0 msgr2=0x7ffac8100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:52.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.183+0000 7fface920700 1 -- 172.21.15.96:0/2609292583 shutdown_connections 2024-04-25T14:12:52.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.183+0000 7fface920700 1 -- 172.21.15.96:0/2609292583 wait complete. 2024-04-25T14:12:52.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.184+0000 7fface920700 1 Processor -- start 2024-04-25T14:12:52.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.184+0000 7fface920700 1 -- start start 2024-04-25T14:12:52.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.184+0000 7fface920700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffac8102890 0x7ffac819e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:52.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.184+0000 7fface920700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffac81031e0 0x7ffac819ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:52.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.184+0000 7fface920700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffac81088c0 0x7ffac8198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:52.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.184+0000 7fface920700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffac81121b0 con 0x7ffac81031e0 2024-04-25T14:12:52.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.184+0000 7fface920700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ffac8112030 con 0x7ffac81088c0 2024-04-25T14:12:52.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.184+0000 7fface920700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffac8112330 con 0x7ffac8102890 2024-04-25T14:12:52.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.185+0000 7ffac77fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffac81031e0 0x7ffac819ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:52.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.185+0000 7ffac77fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffac81031e0 0x7ffac819ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51492/0 (socket says 172.21.15.96:51492) 2024-04-25T14:12:52.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.185+0000 7ffac77fe700 1 -- 172.21.15.96:0/1314280435 learned_addr learned my addr 172.21.15.96:0/1314280435 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:12:52.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.185+0000 7ffac7fff700 1 --2- 172.21.15.96:0/1314280435 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffac8102890 0x7ffac819e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:52.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.185+0000 7ffaccebd700 1 --2- 172.21.15.96:0/1314280435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffac81088c0 0x7ffac8198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:52.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.185+0000 7ffac77fe700 1 -- 172.21.15.96:0/1314280435 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffac8102890 msgr2=0x7ffac819e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:52.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.185+0000 7ffac77fe700 1 --2- 172.21.15.96:0/1314280435 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffac8102890 0x7ffac819e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:52.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.185+0000 7ffac77fe700 1 -- 172.21.15.96:0/1314280435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffac81088c0 msgr2=0x7ffac8198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:52.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.185+0000 7ffac77fe700 1 --2- 172.21.15.96:0/1314280435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffac81088c0 0x7ffac8198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:52.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.185+0000 7ffac77fe700 1 -- 172.21.15.96:0/1314280435 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffac8198f70 con 0x7ffac81031e0 2024-04-25T14:12:52.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.185+0000 7ffaccebd700 1 --2- 172.21.15.96:0/1314280435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffac81088c0 0x7ffac8198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:12:52.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.185+0000 7ffac7fff700 1 --2- 172.21.15.96:0/1314280435 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffac8102890 0x7ffac819e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:12:52.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.185+0000 7ffac77fe700 1 --2- 172.21.15.96:0/1314280435 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffac81031e0 0x7ffac819ec50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ffabc0059b0 tx=0x7ffabc0057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:52.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.185+0000 7ffac57fa700 1 -- 172.21.15.96:0/1314280435 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffabc024020 con 0x7ffac81031e0 2024-04-25T14:12:52.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.185+0000 7ffac57fa700 1 -- 172.21.15.96:0/1314280435 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffabc0041e0 con 0x7ffac81031e0 2024-04-25T14:12:52.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.185+0000 7fface920700 1 -- 172.21.15.96:0/1314280435 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffac8199200 con 0x7ffac81031e0 2024-04-25T14:12:52.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.185+0000 7ffac57fa700 1 -- 172.21.15.96:0/1314280435 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffabc01b040 con 0x7ffac81031e0 2024-04-25T14:12:52.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.185+0000 7fface920700 1 -- 172.21.15.96:0/1314280435 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffac8199730 con 0x7ffac81031e0 2024-04-25T14:12:52.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.186+0000 7fface920700 1 -- 172.21.15.96:0/1314280435 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffac8051190 con 0x7ffac81031e0 2024-04-25T14:12:52.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.188+0000 7ffac57fa700 1 -- 172.21.15.96:0/1314280435 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7ffabc027020 con 0x7ffac81031e0 2024-04-25T14:12:52.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.188+0000 7ffac57fa700 1 --2- 172.21.15.96:0/1314280435 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ffab00afc00 0x7ffab00b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:52.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.188+0000 7ffac7fff700 1 --2- 172.21.15.96:0/1314280435 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ffab00afc00 0x7ffab00b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:52.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.189+0000 7ffac57fa700 1 -- 172.21.15.96:0/1314280435 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7ffabc0e2030 con 0x7ffac81031e0 2024-04-25T14:12:52.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.189+0000 7ffac7fff700 1 --2- 172.21.15.96:0/1314280435 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ffab00afc00 0x7ffab00b2110 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ffab8005e10 tx=0x7ffab8005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:52.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.192+0000 7ffac57fa700 1 -- 172.21.15.96:0/1314280435 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffabc02e020 con 0x7ffac81031e0 2024-04-25T14:12:52.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.384+0000 7fface920700 1 -- 172.21.15.96:0/1314280435 --> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ffac8199e60 con 0x7ffab00afc00 2024-04-25T14:12:52.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.392+0000 7ffac57fa700 1 -- 172.21.15.96:0/1314280435 <== mgr.14152 v2:172.21.15.96:6800/536332425 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259047 (secure 0 0 0) 0x7ffac8199e60 con 0x7ffab00afc00 2024-04-25T14:12:52.394 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:12:52.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.394+0000 7fface920700 1 -- 172.21.15.96:0/1314280435 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ffab00afc00 msgr2=0x7ffab00b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:52.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.394+0000 7fface920700 1 --2- 172.21.15.96:0/1314280435 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ffab00afc00 0x7ffab00b2110 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ffab8005e10 tx=0x7ffab8005d60 comp rx=0 tx=0).stop 2024-04-25T14:12:52.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.395+0000 7fface920700 1 -- 172.21.15.96:0/1314280435 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffac81031e0 msgr2=0x7ffac819ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:52.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.395+0000 7fface920700 1 --2- 172.21.15.96:0/1314280435 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffac81031e0 0x7ffac819ec50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ffabc0059b0 tx=0x7ffabc0057d0 comp rx=0 tx=0).stop 2024-04-25T14:12:52.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.395+0000 7fface920700 1 -- 172.21.15.96:0/1314280435 shutdown_connections 2024-04-25T14:12:52.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.395+0000 7fface920700 1 --2- 172.21.15.96:0/1314280435 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ffab00afc00 0x7ffab00b2110 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:52.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.395+0000 7fface920700 1 --2- 172.21.15.96:0/1314280435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffac81088c0 0x7ffac8198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:52.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.395+0000 7fface920700 1 --2- 172.21.15.96:0/1314280435 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffac8102890 0x7ffac819e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:52.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.395+0000 7fface920700 1 --2- 172.21.15.96:0/1314280435 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffac81031e0 0x7ffac819ec50 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:52.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.395+0000 7fface920700 1 -- 172.21.15.96:0/1314280435 >> 172.21.15.96:0/1314280435 conn(0x7ffac80fe3f0 msgr2=0x7ffac8100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:52.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.395+0000 7fface920700 1 -- 172.21.15.96:0/1314280435 shutdown_connections 2024-04-25T14:12:52.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:52.395+0000 7fface920700 1 -- 172.21.15.96:0/1314280435 wait complete. 2024-04-25T14:12:52.397 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-04-25T14:12:52.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:52 smithi097 ceph-mon[32720]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:12:52.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:52 smithi151 ceph-mon[32990]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:12:53.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:52 smithi096 ceph-mon[29207]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:12:53.213 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2024-04-25T14:12:51.420927+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657200,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":868,"num_read_kb":530,"num_write":361,"num_write_kb":2283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":2952580,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":299,"ondisk_log_size":299,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359684,"kb_used_data":10452,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124582140,"statfs":{"total":1151940427776,"available":1151572111360,"internally_reserved":0,"allocated":10702848,"data_stored":6820572,"data_compressed":182373,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":140,"num_read_kb":70,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003193"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.511167+0000","last_change":"2024-04-25T14:11:29.569615+0000","last_active":"2024-04-25T14:12:22.511167+0000","last_peered":"2024-04-25T14:12:22.511167+0000","last_clean":"2024-04-25T14:12:22.511167+0000","last_became_active":"2024-04-25T14:11:29.569237+0000","last_became_peered":"2024-04-25T14:11:29.569237+0000","last_unstale":"2024-04-25T14:12:22.511167+0000","last_undegraded":"2024-04-25T14:12:22.511167+0000","last_fullsized":"2024-04-25T14:12:22.511167+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:24.367533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.509445+0000","last_change":"2024-04-25T14:11:29.571820+0000","last_active":"2024-04-25T14:12:22.509445+0000","last_peered":"2024-04-25T14:12:22.509445+0000","last_clean":"2024-04-25T14:12:22.509445+0000","last_became_active":"2024-04-25T14:11:29.571456+0000","last_became_peered":"2024-04-25T14:11:29.571456+0000","last_unstale":"2024-04-25T14:12:22.509445+0000","last_undegraded":"2024-04-25T14:12:22.509445+0000","last_fullsized":"2024-04-25T14:12:22.509445+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:33.442301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.509287+0000","last_change":"2024-04-25T14:11:29.579302+0000","last_active":"2024-04-25T14:12:22.509287+0000","last_peered":"2024-04-25T14:12:22.509287+0000","last_clean":"2024-04-25T14:12:22.509287+0000","last_became_active":"2024-04-25T14:11:29.577734+0000","last_became_peered":"2024-04-25T14:11:29.577734+0000","last_unstale":"2024-04-25T14:12:22.509287+0000","last_undegraded":"2024-04-25T14:12:22.509287+0000","last_fullsized":"2024-04-25T14:12:22.509287+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:59.147554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.512471+0000","last_change":"2024-04-25T14:11:29.574942+0000","last_active":"2024-04-25T14:12:22.512471+0000","last_peered":"2024-04-25T14:12:22.512471+0000","last_clean":"2024-04-25T14:12:22.512471+0000","last_became_active":"2024-04-25T14:11:29.573774+0000","last_became_peered":"2024-04-25T14:11:29.573774+0000","last_unstale":"2024-04-25T14:12:22.512471+0000","last_undegraded":"2024-04-25T14:12:22.512471+0000","last_fullsized":"2024-04-25T14:12:22.512471+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:10.626265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514141+0000","last_change":"2024-04-25T14:11:29.557199+0000","last_active":"2024-04-25T14:12:22.514141+0000","last_peered":"2024-04-25T14:12:22.514141+0000","last_clean":"2024-04-25T14:12:22.514141+0000","last_became_active":"2024-04-25T14:11:29.556836+0000","last_became_peered":"2024-04-25T14:11:29.556836+0000","last_unstale":"2024-04-25T14:12:22.514141+0000","last_undegraded":"2024-04-25T14:12:22.514141+0000","last_fullsized":"2024-04-25T14:12:22.514141+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:04:45.882935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.515973+0000","last_change":"2024-04-25T14:11:29.557988+0000","last_active":"2024-04-25T14:12:22.515973+0000","last_peered":"2024-04-25T14:12:22.515973+0000","last_clean":"2024-04-25T14:12:22.515973+0000","last_became_active":"2024-04-25T14:11:29.557622+0000","last_became_peered":"2024-04-25T14:11:29.557622+0000","last_unstale":"2024-04-25T14:12:22.515973+0000","last_undegraded":"2024-04-25T14:12:22.515973+0000","last_fullsized":"2024-04-25T14:12:22.515973+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:28:37.578368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.509027+0000","last_change":"2024-04-25T14:11:29.565049+0000","last_active":"2024-04-25T14:12:22.509027+0000","last_peered":"2024-04-25T14:12:22.509027+0000","last_clean":"2024-04-25T14:12:22.509027+0000","last_became_active":"2024-04-25T14:11:29.564688+0000","last_became_peered":"2024-04-25T14:11:29.564688+0000","last_unstale":"2024-04-25T14:12:22.509027+0000","last_undegraded":"2024-04-25T14:12:22.509027+0000","last_fullsized":"2024-04-25T14:12:22.509027+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:48.128778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.513942+0000","last_change":"2024-04-25T14:11:29.565879+0000","last_active":"2024-04-25T14:12:22.513942+0000","last_peered":"2024-04-25T14:12:22.513942+0000","last_clean":"2024-04-25T14:12:22.513942+0000","last_became_active":"2024-04-25T14:11:29.565525+0000","last_became_peered":"2024-04-25T14:11:29.565525+0000","last_unstale":"2024-04-25T14:12:22.513942+0000","last_undegraded":"2024-04-25T14:12:22.513942+0000","last_fullsized":"2024-04-25T14:12:22.513942+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:06.956334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.517776+0000","last_change":"2024-04-25T14:12:22.517776+0000","last_active":"2024-04-25T14:12:22.517776+0000","last_peered":"2024-04-25T14:12:22.517776+0000","last_clean":"2024-04-25T14:12:22.517776+0000","last_became_active":"2024-04-25T14:12:22.517232+0000","last_became_peered":"2024-04-25T14:12:22.517232+0000","last_unstale":"2024-04-25T14:12:22.517776+0000","last_undegraded":"2024-04-25T14:12:22.517776+0000","last_fullsized":"2024-04-25T14:12:22.517776+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:51.912445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,11],"acting":[1,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.516507+0000","last_change":"2024-04-25T14:11:29.561841+0000","last_active":"2024-04-25T14:12:22.516507+0000","last_peered":"2024-04-25T14:12:22.516507+0000","last_clean":"2024-04-25T14:12:22.516507+0000","last_became_active":"2024-04-25T14:11:29.560145+0000","last_became_peered":"2024-04-25T14:11:29.560145+0000","last_unstale":"2024-04-25T14:12:22.516507+0000","last_undegraded":"2024-04-25T14:12:22.516507+0000","last_fullsized":"2024-04-25T14:12:22.516507+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:11.654692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.509434+0000","last_change":"2024-04-25T14:11:29.570954+0000","last_active":"2024-04-25T14:12:22.509434+0000","last_peered":"2024-04-25T14:12:22.509434+0000","last_clean":"2024-04-25T14:12:22.509434+0000","last_became_active":"2024-04-25T14:11:29.570368+0000","last_became_peered":"2024-04-25T14:11:29.570368+0000","last_unstale":"2024-04-25T14:12:22.509434+0000","last_undegraded":"2024-04-25T14:12:22.509434+0000","last_fullsized":"2024-04-25T14:12:22.509434+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:50.153695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.512357+0000","last_change":"2024-04-25T14:11:29.571911+0000","last_active":"2024-04-25T14:12:22.512357+0000","last_peered":"2024-04-25T14:12:22.512357+0000","last_clean":"2024-04-25T14:12:22.512357+0000","last_became_active":"2024-04-25T14:11:29.568557+0000","last_became_peered":"2024-04-25T14:11:29.568557+0000","last_unstale":"2024-04-25T14:12:22.512357+0000","last_undegraded":"2024-04-25T14:12:22.512357+0000","last_fullsized":"2024-04-25T14:12:22.512357+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:24.221527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.505150+0000","last_change":"2024-04-25T14:11:29.566826+0000","last_active":"2024-04-25T14:12:22.505150+0000","last_peered":"2024-04-25T14:12:22.505150+0000","last_clean":"2024-04-25T14:12:22.505150+0000","last_became_active":"2024-04-25T14:11:29.565825+0000","last_became_peered":"2024-04-25T14:11:29.565825+0000","last_unstale":"2024-04-25T14:12:22.505150+0000","last_undegraded":"2024-04-25T14:12:22.505150+0000","last_fullsized":"2024-04-25T14:12:22.505150+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:46.711809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.509114+0000","last_change":"2024-04-25T14:11:29.577359+0000","last_active":"2024-04-25T14:12:22.509114+0000","last_peered":"2024-04-25T14:12:22.509114+0000","last_clean":"2024-04-25T14:12:22.509114+0000","last_became_active":"2024-04-25T14:11:29.574643+0000","last_became_peered":"2024-04-25T14:11:29.574643+0000","last_unstale":"2024-04-25T14:12:22.509114+0000","last_undegraded":"2024-04-25T14:12:22.509114+0000","last_fullsized":"2024-04-25T14:12:22.509114+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:07.582154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.508363+0000","last_change":"2024-04-25T14:11:29.583606+0000","last_active":"2024-04-25T14:12:22.508363+0000","last_peered":"2024-04-25T14:12:22.508363+0000","last_clean":"2024-04-25T14:12:22.508363+0000","last_became_active":"2024-04-25T14:11:29.582091+0000","last_became_peered":"2024-04-25T14:11:29.582091+0000","last_unstale":"2024-04-25T14:12:22.508363+0000","last_undegraded":"2024-04-25T14:12:22.508363+0000","last_fullsized":"2024-04-25T14:12:22.508363+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:58.711591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.511077+0000","last_change":"2024-04-25T14:11:29.571223+0000","last_active":"2024-04-25T14:12:22.511077+0000","last_peered":"2024-04-25T14:12:22.511077+0000","last_clean":"2024-04-25T14:12:22.511077+0000","last_became_active":"2024-04-25T14:11:29.570856+0000","last_became_peered":"2024-04-25T14:11:29.570856+0000","last_unstale":"2024-04-25T14:12:22.511077+0000","last_undegraded":"2024-04-25T14:12:22.511077+0000","last_fullsized":"2024-04-25T14:12:22.511077+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:53.058088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.511297+0000","last_change":"2024-04-25T14:11:29.574467+0000","last_active":"2024-04-25T14:12:22.511297+0000","last_peered":"2024-04-25T14:12:22.511297+0000","last_clean":"2024-04-25T14:12:22.511297+0000","last_became_active":"2024-04-25T14:11:29.573878+0000","last_became_peered":"2024-04-25T14:11:29.573878+0000","last_unstale":"2024-04-25T14:12:22.511297+0000","last_undegraded":"2024-04-25T14:12:22.511297+0000","last_fullsized":"2024-04-25T14:12:22.511297+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:22:02.087606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.509373+0000","last_change":"2024-04-25T14:11:29.561612+0000","last_active":"2024-04-25T14:12:22.509373+0000","last_peered":"2024-04-25T14:12:22.509373+0000","last_clean":"2024-04-25T14:12:22.509373+0000","last_became_active":"2024-04-25T14:11:29.561211+0000","last_became_peered":"2024-04-25T14:11:29.561211+0000","last_unstale":"2024-04-25T14:12:22.509373+0000","last_undegraded":"2024-04-25T14:12:22.509373+0000","last_fullsized":"2024-04-25T14:12:22.509373+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:29.195093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.512413+0000","last_change":"2024-04-25T14:11:29.575231+0000","last_active":"2024-04-25T14:12:22.512413+0000","last_peered":"2024-04-25T14:12:22.512413+0000","last_clean":"2024-04-25T14:12:22.512413+0000","last_became_active":"2024-04-25T14:11:29.573928+0000","last_became_peered":"2024-04-25T14:11:29.573928+0000","last_unstale":"2024-04-25T14:12:22.512413+0000","last_undegraded":"2024-04-25T14:12:22.512413+0000","last_fullsized":"2024-04-25T14:12:22.512413+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:10.025584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"76'13","reported_seq":126,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:47.550016+0000","last_change":"2024-04-25T14:12:22.527280+0000","last_active":"2024-04-25T14:12:47.550016+0000","last_peered":"2024-04-25T14:12:47.550016+0000","last_clean":"2024-04-25T14:12:47.550016+0000","last_became_active":"2024-04-25T14:12:22.515420+0000","last_became_peered":"2024-04-25T14:12:22.515420+0000","last_unstale":"2024-04-25T14:12:47.550016+0000","last_undegraded":"2024-04-25T14:12:47.550016+0000","last_fullsized":"2024-04-25T14:12:47.550016+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:53.882212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":69,"num_read_kb":69,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514275+0000","last_change":"2024-04-25T14:11:29.576855+0000","last_active":"2024-04-25T14:12:22.514275+0000","last_peered":"2024-04-25T14:12:22.514275+0000","last_clean":"2024-04-25T14:12:22.514275+0000","last_became_active":"2024-04-25T14:11:29.574566+0000","last_became_peered":"2024-04-25T14:11:29.574566+0000","last_unstale":"2024-04-25T14:12:22.514275+0000","last_undegraded":"2024-04-25T14:12:22.514275+0000","last_fullsized":"2024-04-25T14:12:22.514275+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:31.325583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"76'2","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514429+0000","last_change":"2024-04-25T14:11:28.552763+0000","last_active":"2024-04-25T14:12:22.514429+0000","last_peered":"2024-04-25T14:12:22.514429+0000","last_clean":"2024-04-25T14:12:22.514429+0000","last_became_active":"2024-04-25T14:11:28.552403+0000","last_became_peered":"2024-04-25T14:11:28.552403+0000","last_unstale":"2024-04-25T14:12:22.514429+0000","last_undegraded":"2024-04-25T14:12:22.514429+0000","last_fullsized":"2024-04-25T14:12:22.514429+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:31.473779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.509492+0000","last_change":"2024-04-25T14:11:29.571019+0000","last_active":"2024-04-25T14:12:22.509492+0000","last_peered":"2024-04-25T14:12:22.509492+0000","last_clean":"2024-04-25T14:12:22.509492+0000","last_became_active":"2024-04-25T14:11:29.570633+0000","last_became_peered":"2024-04-25T14:11:29.570633+0000","last_unstale":"2024-04-25T14:12:22.509492+0000","last_undegraded":"2024-04-25T14:12:22.509492+0000","last_fullsized":"2024-04-25T14:12:22.509492+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:38:32.664318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"76'12","reported_seq":32,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514608+0000","last_change":"2024-04-25T14:11:28.545189+0000","last_active":"2024-04-25T14:12:22.514608+0000","last_peered":"2024-04-25T14:12:22.514608+0000","last_clean":"2024-04-25T14:12:22.514608+0000","last_became_active":"2024-04-25T14:11:28.544857+0000","last_became_peered":"2024-04-25T14:11:28.544857+0000","last_unstale":"2024-04-25T14:12:22.514608+0000","last_undegraded":"2024-04-25T14:12:22.514608+0000","last_fullsized":"2024-04-25T14:12:22.514608+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:58.464226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.511770+0000","last_change":"2024-04-25T14:11:29.576696+0000","last_active":"2024-04-25T14:12:22.511770+0000","last_peered":"2024-04-25T14:12:22.511770+0000","last_clean":"2024-04-25T14:12:22.511770+0000","last_became_active":"2024-04-25T14:11:29.576337+0000","last_became_peered":"2024-04-25T14:11:29.576337+0000","last_unstale":"2024-04-25T14:12:22.511770+0000","last_undegraded":"2024-04-25T14:12:22.511770+0000","last_fullsized":"2024-04-25T14:12:22.511770+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:31:30.410434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"76'5","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514457+0000","last_change":"2024-04-25T14:11:28.542466+0000","last_active":"2024-04-25T14:12:22.514457+0000","last_peered":"2024-04-25T14:12:22.514457+0000","last_clean":"2024-04-25T14:12:22.514457+0000","last_became_active":"2024-04-25T14:11:28.542092+0000","last_became_peered":"2024-04-25T14:11:28.542092+0000","last_unstale":"2024-04-25T14:12:22.514457+0000","last_undegraded":"2024-04-25T14:12:22.514457+0000","last_fullsized":"2024-04-25T14:12:22.514457+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:33.810111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.508895+0000","last_change":"2024-04-25T14:11:29.567842+0000","last_active":"2024-04-25T14:12:22.508895+0000","last_peered":"2024-04-25T14:12:22.508895+0000","last_clean":"2024-04-25T14:12:22.508895+0000","last_became_active":"2024-04-25T14:11:29.566269+0000","last_became_peered":"2024-04-25T14:11:29.566269+0000","last_unstale":"2024-04-25T14:12:22.508895+0000","last_undegraded":"2024-04-25T14:12:22.508895+0000","last_fullsized":"2024-04-25T14:12:22.508895+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:31.009318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"76'6","reported_seq":107,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:49.626672+0000","last_change":"2024-04-25T14:11:28.546215+0000","last_active":"2024-04-25T14:12:49.626672+0000","last_peered":"2024-04-25T14:12:49.626672+0000","last_clean":"2024-04-25T14:12:49.626672+0000","last_became_active":"2024-04-25T14:11:28.545828+0000","last_became_peered":"2024-04-25T14:11:28.545828+0000","last_unstale":"2024-04-25T14:12:49.626672+0000","last_undegraded":"2024-04-25T14:12:49.626672+0000","last_fullsized":"2024-04-25T14:12:49.626672+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:27.152582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514346+0000","last_change":"2024-04-25T14:11:29.559566+0000","last_active":"2024-04-25T14:12:22.514346+0000","last_peered":"2024-04-25T14:12:22.514346+0000","last_clean":"2024-04-25T14:12:22.514346+0000","last_became_active":"2024-04-25T14:11:29.558560+0000","last_became_peered":"2024-04-25T14:11:29.558560+0000","last_unstale":"2024-04-25T14:12:22.514346+0000","last_undegraded":"2024-04-25T14:12:22.514346+0000","last_fullsized":"2024-04-25T14:12:22.514346+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:23.677099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"76'7","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.515784+0000","last_change":"2024-04-25T14:11:28.544020+0000","last_active":"2024-04-25T14:12:22.515784+0000","last_peered":"2024-04-25T14:12:22.515784+0000","last_clean":"2024-04-25T14:12:22.515784+0000","last_became_active":"2024-04-25T14:11:28.543598+0000","last_became_peered":"2024-04-25T14:11:28.543598+0000","last_unstale":"2024-04-25T14:12:22.515784+0000","last_undegraded":"2024-04-25T14:12:22.515784+0000","last_fullsized":"2024-04-25T14:12:22.515784+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:49:06.543876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.508777+0000","last_change":"2024-04-25T14:11:29.576122+0000","last_active":"2024-04-25T14:12:22.508777+0000","last_peered":"2024-04-25T14:12:22.508777+0000","last_clean":"2024-04-25T14:12:22.508777+0000","last_became_active":"2024-04-25T14:11:29.573668+0000","last_became_peered":"2024-04-25T14:11:29.573668+0000","last_unstale":"2024-04-25T14:12:22.508777+0000","last_undegraded":"2024-04-25T14:12:22.508777+0000","last_fullsized":"2024-04-25T14:12:22.508777+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:06.998876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"76'3","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.505016+0000","last_change":"2024-04-25T14:11:28.541554+0000","last_active":"2024-04-25T14:12:22.505016+0000","last_peered":"2024-04-25T14:12:22.505016+0000","last_clean":"2024-04-25T14:12:22.505016+0000","last_became_active":"2024-04-25T14:11:28.541062+0000","last_became_peered":"2024-04-25T14:11:28.541062+0000","last_unstale":"2024-04-25T14:12:22.505016+0000","last_undegraded":"2024-04-25T14:12:22.505016+0000","last_fullsized":"2024-04-25T14:12:22.505016+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:16.719820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.513986+0000","last_change":"2024-04-25T14:11:29.564550+0000","last_active":"2024-04-25T14:12:22.513986+0000","last_peered":"2024-04-25T14:12:22.513986+0000","last_clean":"2024-04-25T14:12:22.513986+0000","last_became_active":"2024-04-25T14:11:29.564193+0000","last_became_peered":"2024-04-25T14:11:29.564193+0000","last_unstale":"2024-04-25T14:12:22.513986+0000","last_undegraded":"2024-04-25T14:12:22.513986+0000","last_fullsized":"2024-04-25T14:12:22.513986+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:59.357327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"76'3","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.508406+0000","last_change":"2024-04-25T14:11:28.540504+0000","last_active":"2024-04-25T14:12:22.508406+0000","last_peered":"2024-04-25T14:12:22.508406+0000","last_clean":"2024-04-25T14:12:22.508406+0000","last_became_active":"2024-04-25T14:11:28.540150+0000","last_became_peered":"2024-04-25T14:11:28.540150+0000","last_unstale":"2024-04-25T14:12:22.508406+0000","last_undegraded":"2024-04-25T14:12:22.508406+0000","last_fullsized":"2024-04-25T14:12:22.508406+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:35.331489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.508786+0000","last_change":"2024-04-25T14:11:29.561226+0000","last_active":"2024-04-25T14:12:22.508786+0000","last_peered":"2024-04-25T14:12:22.508786+0000","last_clean":"2024-04-25T14:12:22.508786+0000","last_became_active":"2024-04-25T14:11:29.560847+0000","last_became_peered":"2024-04-25T14:11:29.560847+0000","last_unstale":"2024-04-25T14:12:22.508786+0000","last_undegraded":"2024-04-25T14:12:22.508786+0000","last_fullsized":"2024-04-25T14:12:22.508786+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:05.299388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"76'4","reported_seq":178,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:50.627433+0000","last_change":"2024-04-25T14:11:28.548580+0000","last_active":"2024-04-25T14:12:50.627433+0000","last_peered":"2024-04-25T14:12:50.627433+0000","last_clean":"2024-04-25T14:12:50.627433+0000","last_became_active":"2024-04-25T14:11:28.548250+0000","last_became_peered":"2024-04-25T14:11:28.548250+0000","last_unstale":"2024-04-25T14:12:50.627433+0000","last_undegraded":"2024-04-25T14:12:50.627433+0000","last_fullsized":"2024-04-25T14:12:50.627433+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:01.166285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":155,"num_read_kb":155,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.511606+0000","last_change":"2024-04-25T14:11:29.575983+0000","last_active":"2024-04-25T14:12:22.511606+0000","last_peered":"2024-04-25T14:12:22.511606+0000","last_clean":"2024-04-25T14:12:22.511606+0000","last_became_active":"2024-04-25T14:11:29.575147+0000","last_became_peered":"2024-04-25T14:11:29.575147+0000","last_unstale":"2024-04-25T14:12:22.511606+0000","last_undegraded":"2024-04-25T14:12:22.511606+0000","last_fullsized":"2024-04-25T14:12:22.511606+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:40:45.366736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"76'4","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.530419+0000","last_change":"2024-04-25T14:12:22.530419+0000","last_active":"2024-04-25T14:12:22.530419+0000","last_peered":"2024-04-25T14:12:22.530419+0000","last_clean":"2024-04-25T14:12:22.530419+0000","last_became_active":"2024-04-25T14:12:22.514999+0000","last_became_peered":"2024-04-25T14:12:22.514999+0000","last_unstale":"2024-04-25T14:12:22.530419+0000","last_undegraded":"2024-04-25T14:12:22.530419+0000","last_fullsized":"2024-04-25T14:12:22.530419+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:02.075270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.509571+0000","last_change":"2024-04-25T14:11:29.581424+0000","last_active":"2024-04-25T14:12:22.509571+0000","last_peered":"2024-04-25T14:12:22.509571+0000","last_clean":"2024-04-25T14:12:22.509571+0000","last_became_active":"2024-04-25T14:11:29.581086+0000","last_became_peered":"2024-04-25T14:11:29.581086+0000","last_unstale":"2024-04-25T14:12:22.509571+0000","last_undegraded":"2024-04-25T14:12:22.509571+0000","last_fullsized":"2024-04-25T14:12:22.509571+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:30.468728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"76'5","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.508977+0000","last_change":"2024-04-25T14:11:28.540771+0000","last_active":"2024-04-25T14:12:22.508977+0000","last_peered":"2024-04-25T14:12:22.508977+0000","last_clean":"2024-04-25T14:12:22.508977+0000","last_became_active":"2024-04-25T14:11:28.540442+0000","last_became_peered":"2024-04-25T14:11:28.540442+0000","last_unstale":"2024-04-25T14:12:22.508977+0000","last_undegraded":"2024-04-25T14:12:22.508977+0000","last_fullsized":"2024-04-25T14:12:22.508977+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:52.479423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.517721+0000","last_change":"2024-04-25T14:12:22.517721+0000","last_active":"2024-04-25T14:12:22.517721+0000","last_peered":"2024-04-25T14:12:22.517721+0000","last_clean":"2024-04-25T14:12:22.517721+0000","last_became_active":"2024-04-25T14:12:22.517232+0000","last_became_peered":"2024-04-25T14:12:22.517232+0000","last_unstale":"2024-04-25T14:12:22.517721+0000","last_undegraded":"2024-04-25T14:12:22.517721+0000","last_fullsized":"2024-04-25T14:12:22.517721+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:46.150626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"76'29","reported_seq":213,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:50.627357+0000","last_change":"2024-04-25T14:11:28.543093+0000","last_active":"2024-04-25T14:12:50.627357+0000","last_peered":"2024-04-25T14:12:50.627357+0000","last_clean":"2024-04-25T14:12:50.627357+0000","last_became_active":"2024-04-25T14:11:28.542407+0000","last_became_peered":"2024-04-25T14:11:28.542407+0000","last_unstale":"2024-04-25T14:12:50.627357+0000","last_undegraded":"2024-04-25T14:12:50.627357+0000","last_fullsized":"2024-04-25T14:12:50.627357+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:43.767375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59394,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":165,"num_read_kb":140,"num_write":31,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514402+0000","last_change":"2024-04-25T14:11:29.571896+0000","last_active":"2024-04-25T14:12:22.514402+0000","last_peered":"2024-04-25T14:12:22.514402+0000","last_clean":"2024-04-25T14:12:22.514402+0000","last_became_active":"2024-04-25T14:11:29.571537+0000","last_became_peered":"2024-04-25T14:11:29.571537+0000","last_unstale":"2024-04-25T14:12:22.514402+0000","last_undegraded":"2024-04-25T14:12:22.514402+0000","last_fullsized":"2024-04-25T14:12:22.514402+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:32:07.914841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"76'9","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.530152+0000","last_change":"2024-04-25T14:12:22.529692+0000","last_active":"2024-04-25T14:12:22.530152+0000","last_peered":"2024-04-25T14:12:22.530152+0000","last_clean":"2024-04-25T14:12:22.530152+0000","last_became_active":"2024-04-25T14:12:22.517225+0000","last_became_peered":"2024-04-25T14:12:22.517225+0000","last_unstale":"2024-04-25T14:12:22.530152+0000","last_undegraded":"2024-04-25T14:12:22.530152+0000","last_fullsized":"2024-04-25T14:12:22.530152+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:36.610551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.504867+0000","last_change":"2024-04-25T14:11:29.563877+0000","last_active":"2024-04-25T14:12:22.504867+0000","last_peered":"2024-04-25T14:12:22.504867+0000","last_clean":"2024-04-25T14:12:22.504867+0000","last_became_active":"2024-04-25T14:11:29.563535+0000","last_became_peered":"2024-04-25T14:11:29.563535+0000","last_unstale":"2024-04-25T14:12:22.504867+0000","last_undegraded":"2024-04-25T14:12:22.504867+0000","last_fullsized":"2024-04-25T14:12:22.504867+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:07:38.805863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"76'5","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.508526+0000","last_change":"2024-04-25T14:11:28.543787+0000","last_active":"2024-04-25T14:12:22.508526+0000","last_peered":"2024-04-25T14:12:22.508526+0000","last_clean":"2024-04-25T14:12:22.508526+0000","last_became_active":"2024-04-25T14:11:28.540816+0000","last_became_peered":"2024-04-25T14:11:28.540816+0000","last_unstale":"2024-04-25T14:12:22.508526+0000","last_undegraded":"2024-04-25T14:12:22.508526+0000","last_fullsized":"2024-04-25T14:12:22.508526+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:18.776280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514028+0000","last_change":"2024-04-25T14:12:22.514028+0000","last_active":"2024-04-25T14:12:22.514028+0000","last_peered":"2024-04-25T14:12:22.514028+0000","last_clean":"2024-04-25T14:12:22.514028+0000","last_became_active":"2024-04-25T14:12:22.513536+0000","last_became_peered":"2024-04-25T14:12:22.513536+0000","last_unstale":"2024-04-25T14:12:22.514028+0000","last_undegraded":"2024-04-25T14:12:22.514028+0000","last_fullsized":"2024-04-25T14:12:22.514028+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:31.921284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"76'6","reported_seq":103,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:49.644089+0000","last_change":"2024-04-25T14:11:28.537450+0000","last_active":"2024-04-25T14:12:49.644089+0000","last_peered":"2024-04-25T14:12:49.644089+0000","last_clean":"2024-04-25T14:12:49.644089+0000","last_became_active":"2024-04-25T14:11:28.537088+0000","last_became_peered":"2024-04-25T14:11:28.537088+0000","last_unstale":"2024-04-25T14:12:49.644089+0000","last_undegraded":"2024-04-25T14:12:49.644089+0000","last_fullsized":"2024-04-25T14:12:49.644089+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:05.779706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":79,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.508849+0000","last_change":"2024-04-25T14:11:29.572789+0000","last_active":"2024-04-25T14:12:22.508849+0000","last_peered":"2024-04-25T14:12:22.508849+0000","last_clean":"2024-04-25T14:12:22.508849+0000","last_became_active":"2024-04-25T14:11:29.572201+0000","last_became_peered":"2024-04-25T14:11:29.572201+0000","last_unstale":"2024-04-25T14:12:22.508849+0000","last_undegraded":"2024-04-25T14:12:22.508849+0000","last_fullsized":"2024-04-25T14:12:22.508849+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:39:58.237070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"76'17","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514338+0000","last_change":"2024-04-25T14:11:28.540577+0000","last_active":"2024-04-25T14:12:22.514338+0000","last_peered":"2024-04-25T14:12:22.514338+0000","last_clean":"2024-04-25T14:12:22.514338+0000","last_became_active":"2024-04-25T14:11:28.540212+0000","last_became_peered":"2024-04-25T14:11:28.540212+0000","last_unstale":"2024-04-25T14:12:22.514338+0000","last_undegraded":"2024-04-25T14:12:22.514338+0000","last_fullsized":"2024-04-25T14:12:22.514338+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:31.422332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.508850+0000","last_change":"2024-04-25T14:11:29.574952+0000","last_active":"2024-04-25T14:12:22.508850+0000","last_peered":"2024-04-25T14:12:22.508850+0000","last_clean":"2024-04-25T14:12:22.508850+0000","last_became_active":"2024-04-25T14:11:29.573677+0000","last_became_peered":"2024-04-25T14:11:29.573677+0000","last_unstale":"2024-04-25T14:12:22.508850+0000","last_undegraded":"2024-04-25T14:12:22.508850+0000","last_fullsized":"2024-04-25T14:12:22.508850+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:35.896103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"76'2","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514622+0000","last_change":"2024-04-25T14:11:28.541356+0000","last_active":"2024-04-25T14:12:22.514622+0000","last_peered":"2024-04-25T14:12:22.514622+0000","last_clean":"2024-04-25T14:12:22.514622+0000","last_became_active":"2024-04-25T14:11:28.540976+0000","last_became_peered":"2024-04-25T14:11:28.540976+0000","last_unstale":"2024-04-25T14:12:22.514622+0000","last_undegraded":"2024-04-25T14:12:22.514622+0000","last_fullsized":"2024-04-25T14:12:22.514622+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:32:50.599900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.511031+0000","last_change":"2024-04-25T14:11:29.554317+0000","last_active":"2024-04-25T14:12:22.511031+0000","last_peered":"2024-04-25T14:12:22.511031+0000","last_clean":"2024-04-25T14:12:22.511031+0000","last_became_active":"2024-04-25T14:11:29.553935+0000","last_became_peered":"2024-04-25T14:11:29.553935+0000","last_unstale":"2024-04-25T14:12:22.511031+0000","last_undegraded":"2024-04-25T14:12:22.511031+0000","last_fullsized":"2024-04-25T14:12:22.511031+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:01.405964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"76'1","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.509026+0000","last_change":"2024-04-25T14:11:28.546892+0000","last_active":"2024-04-25T14:12:22.509026+0000","last_peered":"2024-04-25T14:12:22.509026+0000","last_clean":"2024-04-25T14:12:22.509026+0000","last_became_active":"2024-04-25T14:11:28.546566+0000","last_became_peered":"2024-04-25T14:11:28.546566+0000","last_unstale":"2024-04-25T14:12:22.509026+0000","last_undegraded":"2024-04-25T14:12:22.509026+0000","last_fullsized":"2024-04-25T14:12:22.509026+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:05.239635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"76'1","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.516285+0000","last_change":"2024-04-25T14:11:29.565083+0000","last_active":"2024-04-25T14:12:22.516285+0000","last_peered":"2024-04-25T14:12:22.516285+0000","last_clean":"2024-04-25T14:12:22.516285+0000","last_became_active":"2024-04-25T14:11:29.564062+0000","last_became_peered":"2024-04-25T14:11:29.564062+0000","last_unstale":"2024-04-25T14:12:22.516285+0000","last_undegraded":"2024-04-25T14:12:22.516285+0000","last_fullsized":"2024-04-25T14:12:22.516285+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:07:10.664113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"76'5","reported_seq":96,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:45.548074+0000","last_change":"2024-04-25T14:11:28.538649+0000","last_active":"2024-04-25T14:12:45.548074+0000","last_peered":"2024-04-25T14:12:45.548074+0000","last_clean":"2024-04-25T14:12:45.548074+0000","last_became_active":"2024-04-25T14:11:28.538353+0000","last_became_peered":"2024-04-25T14:11:28.538353+0000","last_unstale":"2024-04-25T14:12:45.548074+0000","last_undegraded":"2024-04-25T14:12:45.548074+0000","last_fullsized":"2024-04-25T14:12:45.548074+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:06.513039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.516066+0000","last_change":"2024-04-25T14:11:29.576263+0000","last_active":"2024-04-25T14:12:22.516066+0000","last_peered":"2024-04-25T14:12:22.516066+0000","last_clean":"2024-04-25T14:12:22.516066+0000","last_became_active":"2024-04-25T14:11:29.575877+0000","last_became_peered":"2024-04-25T14:11:29.575877+0000","last_unstale":"2024-04-25T14:12:22.516066+0000","last_undegraded":"2024-04-25T14:12:22.516066+0000","last_fullsized":"2024-04-25T14:12:22.516066+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:33:37.639537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.508779+0000","last_change":"2024-04-25T14:11:29.568220+0000","last_active":"2024-04-25T14:12:22.508779+0000","last_peered":"2024-04-25T14:12:22.508779+0000","last_clean":"2024-04-25T14:12:22.508779+0000","last_became_active":"2024-04-25T14:11:29.567741+0000","last_became_peered":"2024-04-25T14:11:29.567741+0000","last_unstale":"2024-04-25T14:12:22.508779+0000","last_undegraded":"2024-04-25T14:12:22.508779+0000","last_fullsized":"2024-04-25T14:12:22.508779+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:05.134466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"76'5","reported_seq":97,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:45.637173+0000","last_change":"2024-04-25T14:11:28.537503+0000","last_active":"2024-04-25T14:12:45.637173+0000","last_peered":"2024-04-25T14:12:45.637173+0000","last_clean":"2024-04-25T14:12:45.637173+0000","last_became_active":"2024-04-25T14:11:28.537169+0000","last_became_peered":"2024-04-25T14:11:28.537169+0000","last_unstale":"2024-04-25T14:12:45.637173+0000","last_undegraded":"2024-04-25T14:12:45.637173+0000","last_fullsized":"2024-04-25T14:12:45.637173+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:09.752988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.516965+0000","last_change":"2024-04-25T14:12:22.516273+0000","last_active":"2024-04-25T14:12:22.516965+0000","last_peered":"2024-04-25T14:12:22.516965+0000","last_clean":"2024-04-25T14:12:22.516965+0000","last_became_active":"2024-04-25T14:12:22.515737+0000","last_became_peered":"2024-04-25T14:12:22.515737+0000","last_unstale":"2024-04-25T14:12:22.516965+0000","last_undegraded":"2024-04-25T14:12:22.516965+0000","last_fullsized":"2024-04-25T14:12:22.516965+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:36.241587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"76'6","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.512277+0000","last_change":"2024-04-25T14:11:28.538216+0000","last_active":"2024-04-25T14:12:22.512277+0000","last_peered":"2024-04-25T14:12:22.512277+0000","last_clean":"2024-04-25T14:12:22.512277+0000","last_became_active":"2024-04-25T14:11:28.537772+0000","last_became_peered":"2024-04-25T14:11:28.537772+0000","last_unstale":"2024-04-25T14:12:22.512277+0000","last_undegraded":"2024-04-25T14:12:22.512277+0000","last_fullsized":"2024-04-25T14:12:22.512277+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:55.460966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":93,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.508948+0000","last_change":"2024-04-25T14:10:12.194216+0000","last_active":"2024-04-25T14:12:22.508948+0000","last_peered":"2024-04-25T14:12:22.508948+0000","last_clean":"2024-04-25T14:12:22.508948+0000","last_became_active":"2024-04-25T14:10:12.189616+0000","last_became_peered":"2024-04-25T14:10:12.189616+0000","last_unstale":"2024-04-25T14:12:22.508948+0000","last_undegraded":"2024-04-25T14:12:22.508948+0000","last_fullsized":"2024-04-25T14:12:22.508948+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:47.553264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:47.553264+0000","last_clean_scrub_stamp":"2024-04-25T14:06:47.553264+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:47.576058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.504798+0000","last_change":"2024-04-25T14:11:29.574985+0000","last_active":"2024-04-25T14:12:22.504798+0000","last_peered":"2024-04-25T14:12:22.504798+0000","last_clean":"2024-04-25T14:12:22.504798+0000","last_became_active":"2024-04-25T14:11:29.574629+0000","last_became_peered":"2024-04-25T14:11:29.574629+0000","last_unstale":"2024-04-25T14:12:22.504798+0000","last_undegraded":"2024-04-25T14:12:22.504798+0000","last_fullsized":"2024-04-25T14:12:22.504798+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:01.899889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"76'4","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.520398+0000","last_change":"2024-04-25T14:12:22.519851+0000","last_active":"2024-04-25T14:12:22.520398+0000","last_peered":"2024-04-25T14:12:22.520398+0000","last_clean":"2024-04-25T14:12:22.520398+0000","last_became_active":"2024-04-25T14:12:22.515791+0000","last_became_peered":"2024-04-25T14:12:22.515791+0000","last_unstale":"2024-04-25T14:12:22.520398+0000","last_undegraded":"2024-04-25T14:12:22.520398+0000","last_fullsized":"2024-04-25T14:12:22.520398+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:47:01.242259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"76'4","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514394+0000","last_change":"2024-04-25T14:11:28.544269+0000","last_active":"2024-04-25T14:12:22.514394+0000","last_peered":"2024-04-25T14:12:22.514394+0000","last_clean":"2024-04-25T14:12:22.514394+0000","last_became_active":"2024-04-25T14:11:28.543897+0000","last_became_peered":"2024-04-25T14:11:28.543897+0000","last_unstale":"2024-04-25T14:12:22.514394+0000","last_undegraded":"2024-04-25T14:12:22.514394+0000","last_fullsized":"2024-04-25T14:12:22.514394+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:37.668187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.505336+0000","last_change":"2024-04-25T14:11:29.566428+0000","last_active":"2024-04-25T14:12:22.505336+0000","last_peered":"2024-04-25T14:12:22.505336+0000","last_clean":"2024-04-25T14:12:22.505336+0000","last_became_active":"2024-04-25T14:11:29.562598+0000","last_became_peered":"2024-04-25T14:11:29.562598+0000","last_unstale":"2024-04-25T14:12:22.505336+0000","last_undegraded":"2024-04-25T14:12:22.505336+0000","last_fullsized":"2024-04-25T14:12:22.505336+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:46.611359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"76'12","reported_seq":31,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.509486+0000","last_change":"2024-04-25T14:11:28.542571+0000","last_active":"2024-04-25T14:12:22.509486+0000","last_peered":"2024-04-25T14:12:22.509486+0000","last_clean":"2024-04-25T14:12:22.509486+0000","last_became_active":"2024-04-25T14:11:28.542272+0000","last_became_peered":"2024-04-25T14:11:28.542272+0000","last_unstale":"2024-04-25T14:12:22.509486+0000","last_undegraded":"2024-04-25T14:12:22.509486+0000","last_fullsized":"2024-04-25T14:12:22.509486+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:41:05.128068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514749+0000","last_change":"2024-04-25T14:11:29.578565+0000","last_active":"2024-04-25T14:12:22.514749+0000","last_peered":"2024-04-25T14:12:22.514749+0000","last_clean":"2024-04-25T14:12:22.514749+0000","last_became_active":"2024-04-25T14:11:29.576782+0000","last_became_peered":"2024-04-25T14:11:29.576782+0000","last_unstale":"2024-04-25T14:12:22.514749+0000","last_undegraded":"2024-04-25T14:12:22.514749+0000","last_fullsized":"2024-04-25T14:12:22.514749+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:31.634602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"76'7","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.511746+0000","last_change":"2024-04-25T14:11:28.543838+0000","last_active":"2024-04-25T14:12:22.511746+0000","last_peered":"2024-04-25T14:12:22.511746+0000","last_clean":"2024-04-25T14:12:22.511746+0000","last_became_active":"2024-04-25T14:11:28.543511+0000","last_became_peered":"2024-04-25T14:11:28.543511+0000","last_unstale":"2024-04-25T14:12:22.511746+0000","last_undegraded":"2024-04-25T14:12:22.511746+0000","last_fullsized":"2024-04-25T14:12:22.511746+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:34.440234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.516416+0000","last_change":"2024-04-25T14:12:22.516416+0000","last_active":"2024-04-25T14:12:22.516416+0000","last_peered":"2024-04-25T14:12:22.516416+0000","last_clean":"2024-04-25T14:12:22.516416+0000","last_became_active":"2024-04-25T14:12:22.515928+0000","last_became_peered":"2024-04-25T14:12:22.515928+0000","last_unstale":"2024-04-25T14:12:22.516416+0000","last_undegraded":"2024-04-25T14:12:22.516416+0000","last_fullsized":"2024-04-25T14:12:22.516416+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:26.989293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"76'2","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.512719+0000","last_change":"2024-04-25T14:11:28.539443+0000","last_active":"2024-04-25T14:12:22.512719+0000","last_peered":"2024-04-25T14:12:22.512719+0000","last_clean":"2024-04-25T14:12:22.512719+0000","last_became_active":"2024-04-25T14:11:28.539113+0000","last_became_peered":"2024-04-25T14:11:28.539113+0000","last_unstale":"2024-04-25T14:12:22.512719+0000","last_undegraded":"2024-04-25T14:12:22.512719+0000","last_fullsized":"2024-04-25T14:12:22.512719+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:36.664972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514061+0000","last_change":"2024-04-25T14:11:29.575451+0000","last_active":"2024-04-25T14:12:22.514061+0000","last_peered":"2024-04-25T14:12:22.514061+0000","last_clean":"2024-04-25T14:12:22.514061+0000","last_became_active":"2024-04-25T14:11:29.574983+0000","last_became_peered":"2024-04-25T14:11:29.574983+0000","last_unstale":"2024-04-25T14:12:22.514061+0000","last_undegraded":"2024-04-25T14:12:22.514061+0000","last_fullsized":"2024-04-25T14:12:22.514061+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:53.317608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"76'7","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514234+0000","last_change":"2024-04-25T14:11:28.546394+0000","last_active":"2024-04-25T14:12:22.514234+0000","last_peered":"2024-04-25T14:12:22.514234+0000","last_clean":"2024-04-25T14:12:22.514234+0000","last_became_active":"2024-04-25T14:11:28.546064+0000","last_became_peered":"2024-04-25T14:11:28.546064+0000","last_unstale":"2024-04-25T14:12:22.514234+0000","last_undegraded":"2024-04-25T14:12:22.514234+0000","last_fullsized":"2024-04-25T14:12:22.514234+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:15.301492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.512276+0000","last_change":"2024-04-25T14:11:29.569584+0000","last_active":"2024-04-25T14:12:22.512276+0000","last_peered":"2024-04-25T14:12:22.512276+0000","last_clean":"2024-04-25T14:12:22.512276+0000","last_became_active":"2024-04-25T14:11:29.569217+0000","last_became_peered":"2024-04-25T14:11:29.569217+0000","last_unstale":"2024-04-25T14:12:22.512276+0000","last_undegraded":"2024-04-25T14:12:22.512276+0000","last_fullsized":"2024-04-25T14:12:22.512276+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:05.708277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"76'3","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.512047+0000","last_change":"2024-04-25T14:11:28.544399+0000","last_active":"2024-04-25T14:12:22.512047+0000","last_peered":"2024-04-25T14:12:22.512047+0000","last_clean":"2024-04-25T14:12:22.512047+0000","last_became_active":"2024-04-25T14:11:28.544098+0000","last_became_peered":"2024-04-25T14:11:28.544098+0000","last_unstale":"2024-04-25T14:12:22.512047+0000","last_undegraded":"2024-04-25T14:12:22.512047+0000","last_fullsized":"2024-04-25T14:12:22.512047+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:05.148386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.518268+0000","last_change":"2024-04-25T14:12:22.517727+0000","last_active":"2024-04-25T14:12:22.518268+0000","last_peered":"2024-04-25T14:12:22.518268+0000","last_clean":"2024-04-25T14:12:22.518268+0000","last_became_active":"2024-04-25T14:12:22.517364+0000","last_became_peered":"2024-04-25T14:12:22.517364+0000","last_unstale":"2024-04-25T14:12:22.518268+0000","last_undegraded":"2024-04-25T14:12:22.518268+0000","last_fullsized":"2024-04-25T14:12:22.518268+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:30:11.023308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"76'4","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514211+0000","last_change":"2024-04-25T14:11:28.542488+0000","last_active":"2024-04-25T14:12:22.514211+0000","last_peered":"2024-04-25T14:12:22.514211+0000","last_clean":"2024-04-25T14:12:22.514211+0000","last_became_active":"2024-04-25T14:11:28.542171+0000","last_became_peered":"2024-04-25T14:11:28.542171+0000","last_unstale":"2024-04-25T14:12:22.514211+0000","last_undegraded":"2024-04-25T14:12:22.514211+0000","last_fullsized":"2024-04-25T14:12:22.514211+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:48.201762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.509671+0000","last_change":"2024-04-25T14:11:29.573805+0000","last_active":"2024-04-25T14:12:22.509671+0000","last_peered":"2024-04-25T14:12:22.509671+0000","last_clean":"2024-04-25T14:12:22.509671+0000","last_became_active":"2024-04-25T14:11:29.573420+0000","last_became_peered":"2024-04-25T14:11:29.573420+0000","last_unstale":"2024-04-25T14:12:22.509671+0000","last_undegraded":"2024-04-25T14:12:22.509671+0000","last_fullsized":"2024-04-25T14:12:22.509671+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:18.721977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"76'5","reported_seq":101,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:48.644890+0000","last_change":"2024-04-25T14:11:28.552863+0000","last_active":"2024-04-25T14:12:48.644890+0000","last_peered":"2024-04-25T14:12:48.644890+0000","last_clean":"2024-04-25T14:12:48.644890+0000","last_became_active":"2024-04-25T14:11:28.552461+0000","last_became_peered":"2024-04-25T14:11:28.552461+0000","last_unstale":"2024-04-25T14:12:48.644890+0000","last_undegraded":"2024-04-25T14:12:48.644890+0000","last_fullsized":"2024-04-25T14:12:48.644890+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:46.264473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":77,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.509551+0000","last_change":"2024-04-25T14:11:29.571268+0000","last_active":"2024-04-25T14:12:22.509551+0000","last_peered":"2024-04-25T14:12:22.509551+0000","last_clean":"2024-04-25T14:12:22.509551+0000","last_became_active":"2024-04-25T14:11:29.570876+0000","last_became_peered":"2024-04-25T14:11:29.570876+0000","last_unstale":"2024-04-25T14:12:22.509551+0000","last_undegraded":"2024-04-25T14:12:22.509551+0000","last_fullsized":"2024-04-25T14:12:22.509551+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:15.093635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"76'3","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.508764+0000","last_change":"2024-04-25T14:11:28.543343+0000","last_active":"2024-04-25T14:12:22.508764+0000","last_peered":"2024-04-25T14:12:22.508764+0000","last_clean":"2024-04-25T14:12:22.508764+0000","last_became_active":"2024-04-25T14:11:28.540790+0000","last_became_peered":"2024-04-25T14:11:28.540790+0000","last_unstale":"2024-04-25T14:12:22.508764+0000","last_undegraded":"2024-04-25T14:12:22.508764+0000","last_fullsized":"2024-04-25T14:12:22.508764+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:20.498016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"76'1","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.509418+0000","last_change":"2024-04-25T14:11:29.566161+0000","last_active":"2024-04-25T14:12:22.509418+0000","last_peered":"2024-04-25T14:12:22.509418+0000","last_clean":"2024-04-25T14:12:22.509418+0000","last_became_active":"2024-04-25T14:11:29.564197+0000","last_became_peered":"2024-04-25T14:11:29.564197+0000","last_unstale":"2024-04-25T14:12:22.509418+0000","last_undegraded":"2024-04-25T14:12:22.509418+0000","last_fullsized":"2024-04-25T14:12:22.509418+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:35.291546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"76'5","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.505214+0000","last_change":"2024-04-25T14:11:28.543479+0000","last_active":"2024-04-25T14:12:22.505214+0000","last_peered":"2024-04-25T14:12:22.505214+0000","last_clean":"2024-04-25T14:12:22.505214+0000","last_became_active":"2024-04-25T14:11:28.543167+0000","last_became_peered":"2024-04-25T14:11:28.543167+0000","last_unstale":"2024-04-25T14:12:22.505214+0000","last_undegraded":"2024-04-25T14:12:22.505214+0000","last_fullsized":"2024-04-25T14:12:22.505214+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:54:53.572809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.509700+0000","last_change":"2024-04-25T14:11:29.572548+0000","last_active":"2024-04-25T14:12:22.509700+0000","last_peered":"2024-04-25T14:12:22.509700+0000","last_clean":"2024-04-25T14:12:22.509700+0000","last_became_active":"2024-04-25T14:11:29.572167+0000","last_became_peered":"2024-04-25T14:11:29.572167+0000","last_unstale":"2024-04-25T14:12:22.509700+0000","last_undegraded":"2024-04-25T14:12:22.509700+0000","last_fullsized":"2024-04-25T14:12:22.509700+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:38.423331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.512408+0000","last_change":"2024-04-25T14:11:29.560956+0000","last_active":"2024-04-25T14:12:22.512408+0000","last_peered":"2024-04-25T14:12:22.512408+0000","last_clean":"2024-04-25T14:12:22.512408+0000","last_became_active":"2024-04-25T14:11:29.560519+0000","last_became_peered":"2024-04-25T14:11:29.560519+0000","last_unstale":"2024-04-25T14:12:22.512408+0000","last_undegraded":"2024-04-25T14:12:22.512408+0000","last_fullsized":"2024-04-25T14:12:22.512408+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:53:47.677923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.512828+0000","last_change":"2024-04-25T14:11:29.574372+0000","last_active":"2024-04-25T14:12:22.512828+0000","last_peered":"2024-04-25T14:12:22.512828+0000","last_clean":"2024-04-25T14:12:22.512828+0000","last_became_active":"2024-04-25T14:11:29.573995+0000","last_became_peered":"2024-04-25T14:11:29.573995+0000","last_unstale":"2024-04-25T14:12:22.512828+0000","last_undegraded":"2024-04-25T14:12:22.512828+0000","last_fullsized":"2024-04-25T14:12:22.512828+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:12.381512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.512034+0000","last_change":"2024-04-25T14:11:29.578324+0000","last_active":"2024-04-25T14:12:22.512034+0000","last_peered":"2024-04-25T14:12:22.512034+0000","last_clean":"2024-04-25T14:12:22.512034+0000","last_became_active":"2024-04-25T14:11:29.577917+0000","last_became_peered":"2024-04-25T14:11:29.577917+0000","last_unstale":"2024-04-25T14:12:22.512034+0000","last_undegraded":"2024-04-25T14:12:22.512034+0000","last_fullsized":"2024-04-25T14:12:22.512034+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:20:27.070720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.509075+0000","last_change":"2024-04-25T14:11:29.578879+0000","last_active":"2024-04-25T14:12:22.509075+0000","last_peered":"2024-04-25T14:12:22.509075+0000","last_clean":"2024-04-25T14:12:22.509075+0000","last_became_active":"2024-04-25T14:11:29.578551+0000","last_became_peered":"2024-04-25T14:11:29.578551+0000","last_unstale":"2024-04-25T14:12:22.509075+0000","last_undegraded":"2024-04-25T14:12:22.509075+0000","last_fullsized":"2024-04-25T14:12:22.509075+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:41.083794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.505252+0000","last_change":"2024-04-25T14:11:29.574054+0000","last_active":"2024-04-25T14:12:22.505252+0000","last_peered":"2024-04-25T14:12:22.505252+0000","last_clean":"2024-04-25T14:12:22.505252+0000","last_became_active":"2024-04-25T14:11:29.573700+0000","last_became_peered":"2024-04-25T14:11:29.573700+0000","last_unstale":"2024-04-25T14:12:22.505252+0000","last_undegraded":"2024-04-25T14:12:22.505252+0000","last_fullsized":"2024-04-25T14:12:22.505252+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:23.304580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.510858+0000","last_change":"2024-04-25T14:11:29.553768+0000","last_active":"2024-04-25T14:12:22.510858+0000","last_peered":"2024-04-25T14:12:22.510858+0000","last_clean":"2024-04-25T14:12:22.510858+0000","last_became_active":"2024-04-25T14:11:29.553376+0000","last_became_peered":"2024-04-25T14:11:29.553376+0000","last_unstale":"2024-04-25T14:12:22.510858+0000","last_undegraded":"2024-04-25T14:12:22.510858+0000","last_fullsized":"2024-04-25T14:12:22.510858+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:46.688368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.512688+0000","last_change":"2024-04-25T14:11:29.563122+0000","last_active":"2024-04-25T14:12:22.512688+0000","last_peered":"2024-04-25T14:12:22.512688+0000","last_clean":"2024-04-25T14:12:22.512688+0000","last_became_active":"2024-04-25T14:11:29.562717+0000","last_became_peered":"2024-04-25T14:11:29.562717+0000","last_unstale":"2024-04-25T14:12:22.512688+0000","last_undegraded":"2024-04-25T14:12:22.512688+0000","last_fullsized":"2024-04-25T14:12:22.512688+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:37.585352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514599+0000","last_change":"2024-04-25T14:11:29.576487+0000","last_active":"2024-04-25T14:12:22.514599+0000","last_peered":"2024-04-25T14:12:22.514599+0000","last_clean":"2024-04-25T14:12:22.514599+0000","last_became_active":"2024-04-25T14:11:29.575070+0000","last_became_peered":"2024-04-25T14:11:29.575070+0000","last_unstale":"2024-04-25T14:12:22.514599+0000","last_undegraded":"2024-04-25T14:12:22.514599+0000","last_fullsized":"2024-04-25T14:12:22.514599+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:13.803505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.508462+0000","last_change":"2024-04-25T14:11:29.568733+0000","last_active":"2024-04-25T14:12:22.508462+0000","last_peered":"2024-04-25T14:12:22.508462+0000","last_clean":"2024-04-25T14:12:22.508462+0000","last_became_active":"2024-04-25T14:11:29.567665+0000","last_became_peered":"2024-04-25T14:11:29.567665+0000","last_unstale":"2024-04-25T14:12:22.508462+0000","last_undegraded":"2024-04-25T14:12:22.508462+0000","last_fullsized":"2024-04-25T14:12:22.508462+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:57.956199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514141+0000","last_change":"2024-04-25T14:11:29.574648+0000","last_active":"2024-04-25T14:12:22.514141+0000","last_peered":"2024-04-25T14:12:22.514141+0000","last_clean":"2024-04-25T14:12:22.514141+0000","last_became_active":"2024-04-25T14:11:29.574247+0000","last_became_peered":"2024-04-25T14:11:29.574247+0000","last_unstale":"2024-04-25T14:12:22.514141+0000","last_undegraded":"2024-04-25T14:12:22.514141+0000","last_fullsized":"2024-04-25T14:12:22.514141+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:57.110017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.511877+0000","last_change":"2024-04-25T14:11:29.573633+0000","last_active":"2024-04-25T14:12:22.511877+0000","last_peered":"2024-04-25T14:12:22.511877+0000","last_clean":"2024-04-25T14:12:22.511877+0000","last_became_active":"2024-04-25T14:11:29.573303+0000","last_became_peered":"2024-04-25T14:11:29.573303+0000","last_unstale":"2024-04-25T14:12:22.511877+0000","last_undegraded":"2024-04-25T14:12:22.511877+0000","last_fullsized":"2024-04-25T14:12:22.511877+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:11.458117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.505212+0000","last_change":"2024-04-25T14:11:29.569136+0000","last_active":"2024-04-25T14:12:22.505212+0000","last_peered":"2024-04-25T14:12:22.505212+0000","last_clean":"2024-04-25T14:12:22.505212+0000","last_became_active":"2024-04-25T14:11:29.568801+0000","last_became_peered":"2024-04-25T14:11:29.568801+0000","last_unstale":"2024-04-25T14:12:22.505212+0000","last_undegraded":"2024-04-25T14:12:22.505212+0000","last_fullsized":"2024-04-25T14:12:22.505212+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:30.969081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.510724+0000","last_change":"2024-04-25T14:11:29.566754+0000","last_active":"2024-04-25T14:12:22.510724+0000","last_peered":"2024-04-25T14:12:22.510724+0000","last_clean":"2024-04-25T14:12:22.510724+0000","last_became_active":"2024-04-25T14:11:29.566368+0000","last_became_peered":"2024-04-25T14:11:29.566368+0000","last_unstale":"2024-04-25T14:12:22.510724+0000","last_undegraded":"2024-04-25T14:12:22.510724+0000","last_fullsized":"2024-04-25T14:12:22.510724+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:37.057718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281277,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66598,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":792,"num_read_kb":466,"num_write":236,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":740,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":199794,"data_compressed":5550,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678046,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":851968,"data_stored":405341,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874276,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":995328,"data_stored":1056087,"data_compressed":43412,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037800,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":868352,"data_stored":405203,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234030,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32020,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713132,"statfs":{"total":95995035648,"available":95962247168,"internally_reserved":0,"allocated":856064,"data_stored":404909,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430258,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":917504,"data_stored":998490,"data_compressed":41562,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593784,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":851968,"data_stored":404920,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":465227,"data_compressed":7643,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738454,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":872448,"data_stored":406259,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411404,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":465770,"data_compressed":7643,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149768,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":868352,"data_stored":404745,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953538,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":860160,"data_stored":405844,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790014,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":997777,"data_compressed":41562,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":61146,"data_compressed":1850,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":61689,"data_compressed":1850,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":61521,"data_compressed":1850,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:12:53.217 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T14:12:53.217 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T14:12:53.217 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-25T14:12:53.218 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-25T14:12:53.458 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:12:53.855 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:53 smithi096 ceph-mon[29207]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-25T14:12:53.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:53 smithi097 ceph-mon[32720]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-25T14:12:53.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:53 smithi151 ceph-mon[32990]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-25T14:12:55.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.182+0000 7fa7f0599700 1 -- 172.21.15.96:0/587260274 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7e81044a0 msgr2=0x7fa7e81048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:55.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.182+0000 7fa7f0599700 1 --2- 172.21.15.96:0/587260274 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7e81044a0 0x7fa7e81048b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa7e40099c0 tx=0x7fa7e4009cd0 comp rx=0 tx=0).stop 2024-04-25T14:12:55.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.182+0000 7fa7f0599700 1 -- 172.21.15.96:0/587260274 shutdown_connections 2024-04-25T14:12:55.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.182+0000 7fa7f0599700 1 --2- 172.21.15.96:0/587260274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7e8109720 0x7fa7e8111080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:55.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.182+0000 7fa7f0599700 1 --2- 172.21.15.96:0/587260274 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7e81044a0 0x7fa7e81048b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:55.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.182+0000 7fa7f0599700 1 --2- 172.21.15.96:0/587260274 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa7e8108dd0 0x7fa7e81091e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:55.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.182+0000 7fa7f0599700 1 -- 172.21.15.96:0/587260274 >> 172.21.15.96:0/587260274 conn(0x7fa7e80ffd70 msgr2=0x7fa7e81021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:55.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.182+0000 7fa7f0599700 1 -- 172.21.15.96:0/587260274 shutdown_connections 2024-04-25T14:12:55.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.182+0000 7fa7f0599700 1 -- 172.21.15.96:0/587260274 wait complete. 2024-04-25T14:12:55.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.183+0000 7fa7f0599700 1 Processor -- start 2024-04-25T14:12:55.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.183+0000 7fa7f0599700 1 -- start start 2024-04-25T14:12:55.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.183+0000 7fa7f0599700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7e81044a0 0x7fa7e8076aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:55.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.183+0000 7fa7f0599700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7e8108dd0 0x7fa7e8076fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:55.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.183+0000 7fa7f0599700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa7e8109720 0x7fa7e8077520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:55.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.183+0000 7fa7f0599700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e81143a0 con 0x7fa7e8108dd0 2024-04-25T14:12:55.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.183+0000 7fa7f0599700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e8114220 con 0x7fa7e81044a0 2024-04-25T14:12:55.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.183+0000 7fa7f0599700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e8114520 con 0x7fa7e8109720 2024-04-25T14:12:55.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.183+0000 7fa7edb34700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7e8108dd0 0x7fa7e8076fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:55.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.183+0000 7fa7edb34700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7e8108dd0 0x7fa7e8076fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51520/0 (socket says 172.21.15.96:51520) 2024-04-25T14:12:55.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.183+0000 7fa7edb34700 1 -- 172.21.15.96:0/618746549 learned_addr learned my addr 172.21.15.96:0/618746549 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:12:55.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.183+0000 7fa7ee335700 1 --2- 172.21.15.96:0/618746549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7e81044a0 0x7fa7e8076aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:55.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.183+0000 7fa7eeb36700 1 --2- 172.21.15.96:0/618746549 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa7e8109720 0x7fa7e8077520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:55.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.184+0000 7fa7edb34700 1 -- 172.21.15.96:0/618746549 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa7e8109720 msgr2=0x7fa7e8077520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:55.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.184+0000 7fa7edb34700 1 --2- 172.21.15.96:0/618746549 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa7e8109720 0x7fa7e8077520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:55.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.184+0000 7fa7edb34700 1 -- 172.21.15.96:0/618746549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7e81044a0 msgr2=0x7fa7e8076aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:55.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.184+0000 7fa7edb34700 1 --2- 172.21.15.96:0/618746549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7e81044a0 0x7fa7e8076aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:55.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.184+0000 7fa7edb34700 1 -- 172.21.15.96:0/618746549 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7e81a9000 con 0x7fa7e8108dd0 2024-04-25T14:12:55.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.184+0000 7fa7eeb36700 1 --2- 172.21.15.96:0/618746549 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa7e8109720 0x7fa7e8077520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:12:55.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.184+0000 7fa7edb34700 1 --2- 172.21.15.96:0/618746549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7e8108dd0 0x7fa7e8076fe0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa7d800d980 tx=0x7fa7d800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:55.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.184+0000 7fa7df7fe700 1 -- 172.21.15.96:0/618746549 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d8017070 con 0x7fa7e8108dd0 2024-04-25T14:12:55.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.184+0000 7fa7df7fe700 1 -- 172.21.15.96:0/618746549 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7d8009de0 con 0x7fa7e8108dd0 2024-04-25T14:12:55.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.184+0000 7fa7df7fe700 1 -- 172.21.15.96:0/618746549 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d800f3c0 con 0x7fa7e8108dd0 2024-04-25T14:12:55.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.184+0000 7fa7f0599700 1 -- 172.21.15.96:0/618746549 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7e81a92f0 con 0x7fa7e8108dd0 2024-04-25T14:12:55.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.184+0000 7fa7f0599700 1 -- 172.21.15.96:0/618746549 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7e81a9880 con 0x7fa7e8108dd0 2024-04-25T14:12:55.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.185+0000 7fa7f0599700 1 -- 172.21.15.96:0/618746549 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7e8051190 con 0x7fa7e8108dd0 2024-04-25T14:12:55.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.187+0000 7fa7df7fe700 1 -- 172.21.15.96:0/618746549 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fa7d8009930 con 0x7fa7e8108dd0 2024-04-25T14:12:55.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.187+0000 7fa7df7fe700 1 --2- 172.21.15.96:0/618746549 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa7d40afc00 0x7fa7d40b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:55.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.187+0000 7fa7df7fe700 1 -- 172.21.15.96:0/618746549 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fa7d80de680 con 0x7fa7e8108dd0 2024-04-25T14:12:55.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.187+0000 7fa7ee335700 1 --2- 172.21.15.96:0/618746549 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa7d40afc00 0x7fa7d40b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:55.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.188+0000 7fa7ee335700 1 --2- 172.21.15.96:0/618746549 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa7d40afc00 0x7fa7d40b2110 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa7e400b580 tx=0x7fa7e40036e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:55.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.191+0000 7fa7df7fe700 1 -- 172.21.15.96:0/618746549 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa7d80a72c0 con 0x7fa7e8108dd0 2024-04-25T14:12:55.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.436+0000 7fa7f0599700 1 -- 172.21.15.96:0/618746549 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fa7e802e510 con 0x7fa7e8108dd0 2024-04-25T14:12:55.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.436+0000 7fa7df7fe700 1 -- 172.21.15.96:0/618746549 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fa7d80af340 con 0x7fa7e8108dd0 2024-04-25T14:12:55.437 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:12:55.437 INFO:teuthology.orchestra.run.smithi096.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T14:12:55.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.439+0000 7fa7f0599700 1 -- 172.21.15.96:0/618746549 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa7d40afc00 msgr2=0x7fa7d40b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:55.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.439+0000 7fa7f0599700 1 --2- 172.21.15.96:0/618746549 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa7d40afc00 0x7fa7d40b2110 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa7e400b580 tx=0x7fa7e40036e0 comp rx=0 tx=0).stop 2024-04-25T14:12:55.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.439+0000 7fa7f0599700 1 -- 172.21.15.96:0/618746549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7e8108dd0 msgr2=0x7fa7e8076fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:55.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.439+0000 7fa7f0599700 1 --2- 172.21.15.96:0/618746549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7e8108dd0 0x7fa7e8076fe0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa7d800d980 tx=0x7fa7d800dc90 comp rx=0 tx=0).stop 2024-04-25T14:12:55.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.439+0000 7fa7f0599700 1 -- 172.21.15.96:0/618746549 shutdown_connections 2024-04-25T14:12:55.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.439+0000 7fa7f0599700 1 --2- 172.21.15.96:0/618746549 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa7d40afc00 0x7fa7d40b2110 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:55.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.439+0000 7fa7f0599700 1 --2- 172.21.15.96:0/618746549 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa7e8109720 0x7fa7e8077520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:55.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.439+0000 7fa7f0599700 1 --2- 172.21.15.96:0/618746549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7e81044a0 0x7fa7e8076aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:55.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.439+0000 7fa7f0599700 1 --2- 172.21.15.96:0/618746549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7e8108dd0 0x7fa7e8076fe0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:55.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.439+0000 7fa7f0599700 1 -- 172.21.15.96:0/618746549 >> 172.21.15.96:0/618746549 conn(0x7fa7e80ffd70 msgr2=0x7fa7e8102190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:55.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.439+0000 7fa7f0599700 1 -- 172.21.15.96:0/618746549 shutdown_connections 2024-04-25T14:12:55.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:55.439+0000 7fa7f0599700 1 -- 172.21.15.96:0/618746549 wait complete. 2024-04-25T14:12:55.641 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:55 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/618746549' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T14:12:55.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:55 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/618746549' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T14:12:55.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:55 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/618746549' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T14:12:56.172 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-25T14:12:56.173 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:12:56.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.286+0000 7f47ed40c640 1 -- 172.21.15.96:0/1486143620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47e8104470 msgr2=0x7f47e810abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:56.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.286+0000 7f47ed40c640 1 --2- 172.21.15.96:0/1486143620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47e8104470 0x7f47e810abc0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f47e0009ca0 tx=0x7f47e002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:12:56.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.286+0000 7f47ed40c640 1 -- 172.21.15.96:0/1486143620 shutdown_connections 2024-04-25T14:12:56.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.286+0000 7f47ed40c640 1 --2- 172.21.15.96:0/1486143620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47e8104470 0x7f47e810abc0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:56.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.286+0000 7f47ed40c640 1 --2- 172.21.15.96:0/1486143620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47e8103950 0x7f47e8103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:56.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.286+0000 7f47ed40c640 1 --2- 172.21.15.96:0/1486143620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47e8102f70 0x7f47e8103380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:56.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.286+0000 7f47ed40c640 1 -- 172.21.15.96:0/1486143620 >> 172.21.15.96:0/1486143620 conn(0x7f47e80fd3b0 msgr2=0x7f47e80ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:56.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.286+0000 7f47ed40c640 1 -- 172.21.15.96:0/1486143620 shutdown_connections 2024-04-25T14:12:56.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.286+0000 7f47ed40c640 1 -- 172.21.15.96:0/1486143620 wait complete. 2024-04-25T14:12:56.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.287+0000 7f47ed40c640 1 Processor -- start 2024-04-25T14:12:56.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.287+0000 7f47ed40c640 1 -- start start 2024-04-25T14:12:56.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.287+0000 7f47ed40c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47e8102f70 0x7f47e8102230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:56.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.287+0000 7f47ed40c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47e8103950 0x7f47e819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:56.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.287+0000 7f47ed40c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47e8104470 0x7f47e81a1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:56.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.287+0000 7f47ed40c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f47e810e540 con 0x7f47e8103950 2024-04-25T14:12:56.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.287+0000 7f47ed40c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f47e810e3c0 con 0x7f47e8102f70 2024-04-25T14:12:56.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.287+0000 7f47ed40c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f47e810e6c0 con 0x7f47e8104470 2024-04-25T14:12:56.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.287+0000 7f47e67fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47e8103950 0x7f47e819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:56.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.287+0000 7f47e67fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47e8103950 0x7f47e819eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51548/0 (socket says 172.21.15.96:51548) 2024-04-25T14:12:56.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.287+0000 7f47e67fc640 1 -- 172.21.15.96:0/3283407522 learned_addr learned my addr 172.21.15.96:0/3283407522 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:12:56.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.287+0000 7f47e6ffd640 1 --2- 172.21.15.96:0/3283407522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47e8102f70 0x7f47e8102230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:56.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.287+0000 7f47e77fe640 1 --2- 172.21.15.96:0/3283407522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47e8104470 0x7f47e81a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:56.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.288+0000 7f47e67fc640 1 -- 172.21.15.96:0/3283407522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47e8104470 msgr2=0x7f47e81a1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:56.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.288+0000 7f47e67fc640 1 --2- 172.21.15.96:0/3283407522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47e8104470 0x7f47e81a1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:56.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.288+0000 7f47e67fc640 1 -- 172.21.15.96:0/3283407522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47e8102f70 msgr2=0x7f47e8102230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:56.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.288+0000 7f47e67fc640 1 --2- 172.21.15.96:0/3283407522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47e8102f70 0x7f47e8102230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:56.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.288+0000 7f47e67fc640 1 -- 172.21.15.96:0/3283407522 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47e81a1c70 con 0x7f47e8103950 2024-04-25T14:12:56.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.288+0000 7f47e67fc640 1 --2- 172.21.15.96:0/3283407522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47e8103950 0x7f47e819eee0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f47dc00d770 tx=0x7f47dc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:56.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.288+0000 7f47cffff640 1 -- 172.21.15.96:0/3283407522 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc017070 con 0x7f47e8103950 2024-04-25T14:12:56.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.288+0000 7f47cffff640 1 -- 172.21.15.96:0/3283407522 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47dc0054d0 con 0x7f47e8103950 2024-04-25T14:12:56.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.288+0000 7f47ed40c640 1 -- 172.21.15.96:0/3283407522 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47e81a1f60 con 0x7f47e8103950 2024-04-25T14:12:56.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.288+0000 7f47cffff640 1 -- 172.21.15.96:0/3283407522 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc004c30 con 0x7f47e8103950 2024-04-25T14:12:56.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.288+0000 7f47ed40c640 1 -- 172.21.15.96:0/3283407522 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47e81a23c0 con 0x7f47e8103950 2024-04-25T14:12:56.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.289+0000 7f47ed40c640 1 -- 172.21.15.96:0/3283407522 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47e8109920 con 0x7f47e8103950 2024-04-25T14:12:56.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.291+0000 7f47cffff640 1 -- 172.21.15.96:0/3283407522 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f47dc004e10 con 0x7f47e8103950 2024-04-25T14:12:56.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.292+0000 7f47cffff640 1 --2- 172.21.15.96:0/3283407522 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f47d00afaf0 0x7f47d00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:56.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.292+0000 7f47e6ffd640 1 --2- 172.21.15.96:0/3283407522 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f47d00afaf0 0x7f47d00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:56.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.292+0000 7f47cffff640 1 -- 172.21.15.96:0/3283407522 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f47dc0de650 con 0x7f47e8103950 2024-04-25T14:12:56.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.293+0000 7f47e6ffd640 1 --2- 172.21.15.96:0/3283407522 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f47d00afaf0 0x7f47d00b2000 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f47e806b320 tx=0x7f47d800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:56.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.297+0000 7f47cffff640 1 -- 172.21.15.96:0/3283407522 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f47dc0a6bc0 con 0x7f47e8103950 2024-04-25T14:12:56.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.482+0000 7f47ed40c640 1 -- 172.21.15.96:0/3283407522 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f47e80637a0 con 0x7f47e8103950 2024-04-25T14:12:56.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.483+0000 7f47cffff640 1 -- 172.21.15.96:0/3283407522 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22616 (secure 0 0 0) 0x7f47dc0aaec0 con 0x7f47e8103950 2024-04-25T14:12:56.483 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:12:56.483 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":78,"fsid":"59334182-030c-11ef-bc93-c7b262605968","created":"2024-04-25T14:03:25.680745+0000","modified":"2024-04-25T14:12:22.496143+0000","last_up_change":"2024-04-25T14:10:35.894312+0000","last_in_change":"2024-04-25T14:10:15.583440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:06:47.402927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:11:27.434331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:11:27.854004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cf4ba3ac-142e-4e8e-96d4-410d77157eb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6803","nonce":3832506168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6805","nonce":3832506168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832506168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6807","nonce":3832506168}]},"public_addr":"172.21.15.96:6803/3832506168","cluster_addr":"172.21.15.96:6805/3832506168","heartbeat_back_addr":"172.21.15.96:6809/3832506168","heartbeat_front_addr":"172.21.15.96:6807/3832506168","state":["exists","up"]},{"osd":1,"uuid":"866e73b9-4a26-48ba-badf-704fe22f71cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6801","nonce":1083311494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6803","nonce":1083311494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6807","nonce":1083311494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6805","nonce":1083311494}]},"public_addr":"172.21.15.97:6801/1083311494","cluster_addr":"172.21.15.97:6803/1083311494","heartbeat_back_addr":"172.21.15.97:6807/1083311494","heartbeat_front_addr":"172.21.15.97:6805/1083311494","state":["exists","up"]},{"osd":2,"uuid":"45c79ecb-abc4-4dba-bae8-74804e4bc1bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6801","nonce":981902966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6803","nonce":981902966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6807","nonce":981902966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6805","nonce":981902966}]},"public_addr":"172.21.15.151:6801/981902966","cluster_addr":"172.21.15.151:6803/981902966","heartbeat_back_addr":"172.21.15.151:6807/981902966","heartbeat_front_addr":"172.21.15.151:6805/981902966","state":["exists","up"]},{"osd":3,"uuid":"9fc39468-a7ea-4ccf-b365-335ad2ea4c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6811","nonce":2274158748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6813","nonce":2274158748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6817","nonce":2274158748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6815","nonce":2274158748}]},"public_addr":"172.21.15.96:6811/2274158748","cluster_addr":"172.21.15.96:6813/2274158748","heartbeat_back_addr":"172.21.15.96:6817/2274158748","heartbeat_front_addr":"172.21.15.96:6815/2274158748","state":["exists","up"]},{"osd":4,"uuid":"616582f1-12d2-456b-b7b3-e5517800e8a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6809","nonce":3727031962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6811","nonce":3727031962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6815","nonce":3727031962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6813","nonce":3727031962}]},"public_addr":"172.21.15.97:6809/3727031962","cluster_addr":"172.21.15.97:6811/3727031962","heartbeat_back_addr":"172.21.15.97:6815/3727031962","heartbeat_front_addr":"172.21.15.97:6813/3727031962","state":["exists","up"]},{"osd":5,"uuid":"3d277784-ab0f-4c90-81a1-2a4034c52a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6809","nonce":223882546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6811","nonce":223882546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6815","nonce":223882546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6813","nonce":223882546}]},"public_addr":"172.21.15.151:6809/223882546","cluster_addr":"172.21.15.151:6811/223882546","heartbeat_back_addr":"172.21.15.151:6815/223882546","heartbeat_front_addr":"172.21.15.151:6813/223882546","state":["exists","up"]},{"osd":6,"uuid":"9a9ad585-cb1a-4239-9d18-055f59164049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6819","nonce":96197407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6821","nonce":96197407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6825","nonce":96197407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6823","nonce":96197407}]},"public_addr":"172.21.15.96:6819/96197407","cluster_addr":"172.21.15.96:6821/96197407","heartbeat_back_addr":"172.21.15.96:6825/96197407","heartbeat_front_addr":"172.21.15.96:6823/96197407","state":["exists","up"]},{"osd":7,"uuid":"303551c8-6af0-42f2-844f-45bf8db5baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6817","nonce":2247216400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6819","nonce":2247216400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6823","nonce":2247216400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6821","nonce":2247216400}]},"public_addr":"172.21.15.97:6817/2247216400","cluster_addr":"172.21.15.97:6819/2247216400","heartbeat_back_addr":"172.21.15.97:6823/2247216400","heartbeat_front_addr":"172.21.15.97:6821/2247216400","state":["exists","up"]},{"osd":8,"uuid":"81f6cc66-e4ca-430c-b336-b6678d968818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6817","nonce":2549822185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6819","nonce":2549822185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6823","nonce":2549822185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6821","nonce":2549822185}]},"public_addr":"172.21.15.151:6817/2549822185","cluster_addr":"172.21.15.151:6819/2549822185","heartbeat_back_addr":"172.21.15.151:6823/2549822185","heartbeat_front_addr":"172.21.15.151:6821/2549822185","state":["exists","up"]},{"osd":9,"uuid":"29d4da16-feab-4d1c-9d1d-de4f799bd724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6827","nonce":1099399643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6829","nonce":1099399643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6833","nonce":1099399643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6831","nonce":1099399643}]},"public_addr":"172.21.15.96:6827/1099399643","cluster_addr":"172.21.15.96:6829/1099399643","heartbeat_back_addr":"172.21.15.96:6833/1099399643","heartbeat_front_addr":"172.21.15.96:6831/1099399643","state":["exists","up"]},{"osd":10,"uuid":"197fee9b-eee4-4893-be1c-3d57f42250e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6825","nonce":155754520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6827","nonce":155754520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6831","nonce":155754520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6829","nonce":155754520}]},"public_addr":"172.21.15.97:6825/155754520","cluster_addr":"172.21.15.97:6827/155754520","heartbeat_back_addr":"172.21.15.97:6831/155754520","heartbeat_front_addr":"172.21.15.97:6829/155754520","state":["exists","up"]},{"osd":11,"uuid":"9c1d0a83-fc12-41a2-8eb0-80c4abdd2770","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6825","nonce":2027958746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6827","nonce":2027958746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6831","nonce":2027958746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6829","nonce":2027958746}]},"public_addr":"172.21.15.151:6825/2027958746","cluster_addr":"172.21.15.151:6827/2027958746","heartbeat_back_addr":"172.21.15.151:6831/2027958746","heartbeat_front_addr":"172.21.15.151:6829/2027958746","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:05:56.177390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:06:19.917663+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:06:43.608052+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:08.540382+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:33.474072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:58.242306+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:08:24.120442+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:08:49.661064+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:09:15.097271+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:09:41.904505+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:10:08.233252+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:10:33.712296+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/3839807484":"2024-04-26T14:04:21.307574+0000","172.21.15.96:0/1495237551":"2024-04-26T14:04:21.307574+0000","172.21.15.96:0/2793751450":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/129575834":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/2233075139":"2024-04-26T14:04:21.307574+0000","172.21.15.96:6801/2348397481":"2024-04-26T14:03:54.249537+0000","172.21.15.96:6801/433591062":"2024-04-26T14:04:21.307574+0000","172.21.15.96:6800/2348397481":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/3517674924":"2024-04-26T14:03:54.249537+0000","172.21.15.96:6800/433591062":"2024-04-26T14:04:21.307574+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:12:56.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.485+0000 7f47ed40c640 1 -- 172.21.15.96:0/3283407522 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f47d00afaf0 msgr2=0x7f47d00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:56.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.485+0000 7f47ed40c640 1 --2- 172.21.15.96:0/3283407522 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f47d00afaf0 0x7f47d00b2000 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f47e806b320 tx=0x7f47d800b390 comp rx=0 tx=0).stop 2024-04-25T14:12:56.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.485+0000 7f47ed40c640 1 -- 172.21.15.96:0/3283407522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47e8103950 msgr2=0x7f47e819eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:56.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.485+0000 7f47ed40c640 1 --2- 172.21.15.96:0/3283407522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47e8103950 0x7f47e819eee0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f47dc00d770 tx=0x7f47dc00dc30 comp rx=0 tx=0).stop 2024-04-25T14:12:56.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.485+0000 7f47ed40c640 1 -- 172.21.15.96:0/3283407522 shutdown_connections 2024-04-25T14:12:56.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.485+0000 7f47ed40c640 1 --2- 172.21.15.96:0/3283407522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47e8104470 0x7f47e81a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:56.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.485+0000 7f47ed40c640 1 --2- 172.21.15.96:0/3283407522 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f47d00afaf0 0x7f47d00b2000 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:56.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.485+0000 7f47ed40c640 1 --2- 172.21.15.96:0/3283407522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47e8103950 0x7f47e819eee0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:56.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.485+0000 7f47ed40c640 1 --2- 172.21.15.96:0/3283407522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47e8102f70 0x7f47e8102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:56.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.485+0000 7f47ed40c640 1 -- 172.21.15.96:0/3283407522 >> 172.21.15.96:0/3283407522 conn(0x7f47e80fd3b0 msgr2=0x7f47e8106af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:56.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.485+0000 7f47ed40c640 1 -- 172.21.15.96:0/3283407522 shutdown_connections 2024-04-25T14:12:56.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.485+0000 7f47ed40c640 1 -- 172.21.15.96:0/3283407522 wait complete. 2024-04-25T14:12:56.498 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:06:47.402927+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:11:27.434331+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:11:27.854004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:12:56.499 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:12:56.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.613+0000 7fea86ab8640 1 -- 172.21.15.96:0/3664334035 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea80105090 msgr2=0x7fea801054a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:56.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.613+0000 7fea86ab8640 1 --2- 172.21.15.96:0/3664334035 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea80105090 0x7fea801054a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fea74009900 tx=0x7fea7402a410 comp rx=0 tx=0).stop 2024-04-25T14:12:56.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.613+0000 7fea86ab8640 1 -- 172.21.15.96:0/3664334035 shutdown_connections 2024-04-25T14:12:56.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.613+0000 7fea86ab8640 1 --2- 172.21.15.96:0/3664334035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea80102240 0x7fea8010bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:56.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.613+0000 7fea86ab8640 1 --2- 172.21.15.96:0/3664334035 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea80101740 0x7fea80101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:56.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.613+0000 7fea86ab8640 1 --2- 172.21.15.96:0/3664334035 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea80105090 0x7fea801054a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:56.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.613+0000 7fea86ab8640 1 -- 172.21.15.96:0/3664334035 >> 172.21.15.96:0/3664334035 conn(0x7fea800fd340 msgr2=0x7fea800ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:56.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.613+0000 7fea86ab8640 1 -- 172.21.15.96:0/3664334035 shutdown_connections 2024-04-25T14:12:56.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.613+0000 7fea86ab8640 1 -- 172.21.15.96:0/3664334035 wait complete. 2024-04-25T14:12:56.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.614+0000 7fea86ab8640 1 Processor -- start 2024-04-25T14:12:56.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.614+0000 7fea86ab8640 1 -- start start 2024-04-25T14:12:56.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.614+0000 7fea86ab8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea80101740 0x7fea8019a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:56.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.614+0000 7fea86ab8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea80102240 0x7fea8019a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:56.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.614+0000 7fea86ab8640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea80105090 0x7fea801a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:56.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.614+0000 7fea86ab8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fea8010f600 con 0x7fea80101740 2024-04-25T14:12:56.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.614+0000 7fea86ab8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fea8010f480 con 0x7fea80102240 2024-04-25T14:12:56.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.614+0000 7fea86ab8640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fea8010f780 con 0x7fea80105090 2024-04-25T14:12:56.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.614+0000 7fea84854640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea80101740 0x7fea8019a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:56.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.614+0000 7fea84854640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea80101740 0x7fea8019a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51566/0 (socket says 172.21.15.96:51566) 2024-04-25T14:12:56.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.614+0000 7fea84854640 1 -- 172.21.15.96:0/1888894827 learned_addr learned my addr 172.21.15.96:0/1888894827 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:12:56.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.614+0000 7fea7ffff640 1 --2- 172.21.15.96:0/1888894827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea80102240 0x7fea8019a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:56.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.615+0000 7fea85055640 1 --2- 172.21.15.96:0/1888894827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea80105090 0x7fea801a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:56.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.615+0000 7fea84854640 1 -- 172.21.15.96:0/1888894827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea80105090 msgr2=0x7fea801a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:56.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.615+0000 7fea84854640 1 --2- 172.21.15.96:0/1888894827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea80105090 0x7fea801a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:56.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.615+0000 7fea84854640 1 -- 172.21.15.96:0/1888894827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea80102240 msgr2=0x7fea8019a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:56.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.615+0000 7fea84854640 1 --2- 172.21.15.96:0/1888894827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea80102240 0x7fea8019a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:56.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.615+0000 7fea84854640 1 -- 172.21.15.96:0/1888894827 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea801a1d90 con 0x7fea80101740 2024-04-25T14:12:56.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.615+0000 7fea85055640 1 --2- 172.21.15.96:0/1888894827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea80105090 0x7fea801a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:12:56.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.615+0000 7fea84854640 1 --2- 172.21.15.96:0/1888894827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea80101740 0x7fea8019a250 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fea74032fd0 tx=0x7fea740042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:56.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.615+0000 7fea7dffb640 1 -- 172.21.15.96:0/1888894827 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7403f070 con 0x7fea80101740 2024-04-25T14:12:56.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.615+0000 7fea7dffb640 1 -- 172.21.15.96:0/1888894827 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea740044d0 con 0x7fea80101740 2024-04-25T14:12:56.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.615+0000 7fea86ab8640 1 -- 172.21.15.96:0/1888894827 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea801a2020 con 0x7fea80101740 2024-04-25T14:12:56.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.615+0000 7fea7dffb640 1 -- 172.21.15.96:0/1888894827 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea74035070 con 0x7fea80101740 2024-04-25T14:12:56.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.615+0000 7fea86ab8640 1 -- 172.21.15.96:0/1888894827 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea801a2510 con 0x7fea80101740 2024-04-25T14:12:56.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.616+0000 7fea86ab8640 1 -- 172.21.15.96:0/1888894827 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea80108a20 con 0x7fea80101740 2024-04-25T14:12:56.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.618+0000 7fea7dffb640 1 -- 172.21.15.96:0/1888894827 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fea7403b5f0 con 0x7fea80101740 2024-04-25T14:12:56.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.619+0000 7fea7dffb640 1 --2- 172.21.15.96:0/1888894827 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fea700afaf0 0x7fea700b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:56.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.619+0000 7fea7ffff640 1 --2- 172.21.15.96:0/1888894827 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fea700afaf0 0x7fea700b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:56.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.619+0000 7fea7dffb640 1 -- 172.21.15.96:0/1888894827 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fea740ff950 con 0x7fea80101740 2024-04-25T14:12:56.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.620+0000 7fea7ffff640 1 --2- 172.21.15.96:0/1888894827 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fea700afaf0 0x7fea700b2000 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fea6c0058c0 tx=0x7fea6c00a250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:56.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.623+0000 7fea7dffb640 1 -- 172.21.15.96:0/1888894827 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fea740c84b0 con 0x7fea80101740 2024-04-25T14:12:56.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.800+0000 7fea86ab8640 1 -- 172.21.15.96:0/1888894827 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fea8002dce0 con 0x7fea80101740 2024-04-25T14:12:56.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.800+0000 7fea7dffb640 1 -- 172.21.15.96:0/1888894827 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) ==== 93+0+10 (secure 0 0 0) 0x7fea8002dce0 con 0x7fea80101740 2024-04-25T14:12:56.800 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-04-25T14:12:56.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.802+0000 7fea86ab8640 1 -- 172.21.15.96:0/1888894827 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fea700afaf0 msgr2=0x7fea700b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:56.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.802+0000 7fea86ab8640 1 --2- 172.21.15.96:0/1888894827 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fea700afaf0 0x7fea700b2000 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fea6c0058c0 tx=0x7fea6c00a250 comp rx=0 tx=0).stop 2024-04-25T14:12:56.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.802+0000 7fea86ab8640 1 -- 172.21.15.96:0/1888894827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea80101740 msgr2=0x7fea8019a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:56.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.802+0000 7fea86ab8640 1 --2- 172.21.15.96:0/1888894827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea80101740 0x7fea8019a250 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fea74032fd0 tx=0x7fea740042e0 comp rx=0 tx=0).stop 2024-04-25T14:12:56.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.802+0000 7fea86ab8640 1 -- 172.21.15.96:0/1888894827 shutdown_connections 2024-04-25T14:12:56.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.802+0000 7fea86ab8640 1 --2- 172.21.15.96:0/1888894827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea80105090 0x7fea801a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:56.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.803+0000 7fea86ab8640 1 --2- 172.21.15.96:0/1888894827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea80102240 0x7fea8019a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:56.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.803+0000 7fea86ab8640 1 --2- 172.21.15.96:0/1888894827 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fea700afaf0 0x7fea700b2000 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:56.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.803+0000 7fea86ab8640 1 --2- 172.21.15.96:0/1888894827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea80101740 0x7fea8019a250 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:56.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.803+0000 7fea86ab8640 1 -- 172.21.15.96:0/1888894827 >> 172.21.15.96:0/1888894827 conn(0x7fea800fd340 msgr2=0x7fea80105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:56.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.803+0000 7fea86ab8640 1 -- 172.21.15.96:0/1888894827 shutdown_connections 2024-04-25T14:12:56.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.803+0000 7fea86ab8640 1 -- 172.21.15.96:0/1888894827 wait complete. 2024-04-25T14:12:56.813 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:56 smithi096 ceph-mon[29207]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T14:12:56.813 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:56 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/3283407522' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:12:56.814 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:12:56.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:56 smithi097 ceph-mon[32720]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T14:12:56.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:56 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/3283407522' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:12:56.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.924+0000 7f5154a29640 1 -- 172.21.15.96:0/655257690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5150102fa0 msgr2=0x7f51501033b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:56.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.924+0000 7f5154a29640 1 --2- 172.21.15.96:0/655257690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5150102fa0 0x7f51501033b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5140009900 tx=0x7f514002a410 comp rx=0 tx=0).stop 2024-04-25T14:12:56.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.924+0000 7f5154a29640 1 -- 172.21.15.96:0/655257690 shutdown_connections 2024-04-25T14:12:56.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.924+0000 7f5154a29640 1 --2- 172.21.15.96:0/655257690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5150104480 0x7f515010abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:56.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.924+0000 7f5154a29640 1 --2- 172.21.15.96:0/655257690 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5150103980 0x7f5150103df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:56.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.924+0000 7f5154a29640 1 --2- 172.21.15.96:0/655257690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5150102fa0 0x7f51501033b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:56.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.924+0000 7f5154a29640 1 -- 172.21.15.96:0/655257690 >> 172.21.15.96:0/655257690 conn(0x7f51500fd380 msgr2=0x7f51500ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:56.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.924+0000 7f5154a29640 1 -- 172.21.15.96:0/655257690 shutdown_connections 2024-04-25T14:12:56.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.924+0000 7f5154a29640 1 -- 172.21.15.96:0/655257690 wait complete. 2024-04-25T14:12:56.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.925+0000 7f5154a29640 1 Processor -- start 2024-04-25T14:12:56.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.925+0000 7f5154a29640 1 -- start start 2024-04-25T14:12:56.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.925+0000 7f5154a29640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5150102fa0 0x7f5150102240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:56.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.925+0000 7f5154a29640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5150103980 0x7f515019f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:56.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.925+0000 7f5154a29640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5150104480 0x7f51501a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:56.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.925+0000 7f5154a29640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f515010e520 con 0x7f5150104480 2024-04-25T14:12:56.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.925+0000 7f5154a29640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f515010e3a0 con 0x7f5150102fa0 2024-04-25T14:12:56.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.925+0000 7f5154a29640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f515010e6a0 con 0x7f5150103980 2024-04-25T14:12:56.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.926+0000 7f514ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5150104480 0x7f51501a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:56.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.926+0000 7f514ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5150104480 0x7f51501a15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51572/0 (socket says 172.21.15.96:51572) 2024-04-25T14:12:56.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.926+0000 7f514ed9d640 1 -- 172.21.15.96:0/3091498719 learned_addr learned my addr 172.21.15.96:0/3091498719 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:12:56.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.926+0000 7f514e59c640 1 --2- 172.21.15.96:0/3091498719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5150102fa0 0x7f5150102240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:56.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.926+0000 7f514dd9b640 1 --2- 172.21.15.96:0/3091498719 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5150103980 0x7f515019f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:56.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.926+0000 7f514ed9d640 1 -- 172.21.15.96:0/3091498719 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5150103980 msgr2=0x7f515019f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:56.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.926+0000 7f514ed9d640 1 --2- 172.21.15.96:0/3091498719 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5150103980 0x7f515019f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:56.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.926+0000 7f514ed9d640 1 -- 172.21.15.96:0/3091498719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5150102fa0 msgr2=0x7f5150102240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:56.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.926+0000 7f514ed9d640 1 --2- 172.21.15.96:0/3091498719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5150102fa0 0x7f5150102240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:56.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.926+0000 7f514ed9d640 1 -- 172.21.15.96:0/3091498719 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51501a1dd0 con 0x7f5150104480 2024-04-25T14:12:56.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.926+0000 7f514dd9b640 1 --2- 172.21.15.96:0/3091498719 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5150103980 0x7f515019f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:12:56.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.926+0000 7f514ed9d640 1 --2- 172.21.15.96:0/3091498719 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5150104480 0x7f51501a15b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f514800cad0 tx=0x7f514800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:56.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.926+0000 7f513f7fe640 1 -- 172.21.15.96:0/3091498719 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514800ae20 con 0x7f5150104480 2024-04-25T14:12:56.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.926+0000 7f5154a29640 1 -- 172.21.15.96:0/3091498719 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51501a20c0 con 0x7f5150104480 2024-04-25T14:12:56.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.926+0000 7f513f7fe640 1 -- 172.21.15.96:0/3091498719 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5148005020 con 0x7f5150104480 2024-04-25T14:12:56.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.926+0000 7f513f7fe640 1 -- 172.21.15.96:0/3091498719 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514800f6b0 con 0x7f5150104480 2024-04-25T14:12:56.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.926+0000 7f5154a29640 1 -- 172.21.15.96:0/3091498719 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51501a25c0 con 0x7f5150104480 2024-04-25T14:12:56.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.927+0000 7f5154a29640 1 -- 172.21.15.96:0/3091498719 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5150107940 con 0x7f5150104480 2024-04-25T14:12:56.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.928+0000 7f513f7fe640 1 -- 172.21.15.96:0/3091498719 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f51480051d0 con 0x7f5150104480 2024-04-25T14:12:56.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.929+0000 7f513f7fe640 1 --2- 172.21.15.96:0/3091498719 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f51380afaf0 0x7f51380b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:56.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.929+0000 7f514e59c640 1 --2- 172.21.15.96:0/3091498719 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f51380afaf0 0x7f51380b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:56.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.929+0000 7f513f7fe640 1 -- 172.21.15.96:0/3091498719 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f51480df000 con 0x7f5150104480 2024-04-25T14:12:56.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.930+0000 7f514e59c640 1 --2- 172.21.15.96:0/3091498719 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f51380afaf0 0x7f51380b2000 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5140005ab0 tx=0x7f5140031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:56.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:56.933+0000 7f513f7fe640 1 -- 172.21.15.96:0/3091498719 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5148014030 con 0x7f5150104480 2024-04-25T14:12:56.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:56 smithi151 ceph-mon[32990]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T14:12:56.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:56 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/3283407522' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:12:57.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.109+0000 7f5154a29640 1 -- 172.21.15.96:0/3091498719 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f51500637a0 con 0x7f5150104480 2024-04-25T14:12:57.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.110+0000 7f513f7fe640 1 -- 172.21.15.96:0/3091498719 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v78) ==== 104+0+11 (secure 0 0 0) 0x7f51480a7c60 con 0x7f5150104480 2024-04-25T14:12:57.110 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-04-25T14:12:57.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.112+0000 7f5154a29640 1 -- 172.21.15.96:0/3091498719 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f51380afaf0 msgr2=0x7f51380b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:57.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.112+0000 7f5154a29640 1 --2- 172.21.15.96:0/3091498719 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f51380afaf0 0x7f51380b2000 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5140005ab0 tx=0x7f5140031400 comp rx=0 tx=0).stop 2024-04-25T14:12:57.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.112+0000 7f5154a29640 1 -- 172.21.15.96:0/3091498719 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5150104480 msgr2=0x7f51501a15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:57.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.112+0000 7f5154a29640 1 --2- 172.21.15.96:0/3091498719 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5150104480 0x7f51501a15b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f514800cad0 tx=0x7f514800cf90 comp rx=0 tx=0).stop 2024-04-25T14:12:57.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.112+0000 7f5154a29640 1 -- 172.21.15.96:0/3091498719 shutdown_connections 2024-04-25T14:12:57.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.112+0000 7f5154a29640 1 --2- 172.21.15.96:0/3091498719 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f51380afaf0 0x7f51380b2000 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:57.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.112+0000 7f5154a29640 1 --2- 172.21.15.96:0/3091498719 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5150104480 0x7f51501a15b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:57.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.112+0000 7f5154a29640 1 --2- 172.21.15.96:0/3091498719 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5150103980 0x7f515019f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:57.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.112+0000 7f5154a29640 1 --2- 172.21.15.96:0/3091498719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5150102fa0 0x7f5150102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:57.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.112+0000 7f5154a29640 1 -- 172.21.15.96:0/3091498719 >> 172.21.15.96:0/3091498719 conn(0x7f51500fd380 msgr2=0x7f5150104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:57.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.112+0000 7f5154a29640 1 -- 172.21.15.96:0/3091498719 shutdown_connections 2024-04-25T14:12:57.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.112+0000 7f5154a29640 1 -- 172.21.15.96:0/3091498719 wait complete. 2024-04-25T14:12:57.123 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:12:57.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.234+0000 7f388cba8640 1 -- 172.21.15.96:0/1404006278 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3888109780 msgr2=0x7f388810bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:57.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.234+0000 7f388cba8640 1 --2- 172.21.15.96:0/1404006278 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3888109780 0x7f388810bc90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3880009ca0 tx=0x7f388002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:12:57.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.234+0000 7f388cba8640 1 -- 172.21.15.96:0/1404006278 shutdown_connections 2024-04-25T14:12:57.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.234+0000 7f388cba8640 1 --2- 172.21.15.96:0/1404006278 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3888109780 0x7f388810bc90 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:57.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.234+0000 7f388cba8640 1 --2- 172.21.15.96:0/1404006278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3888102360 0x7f38881090f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:57.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.234+0000 7f388cba8640 1 --2- 172.21.15.96:0/1404006278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3888101e60 0x7f3888106b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:57.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.234+0000 7f388cba8640 1 -- 172.21.15.96:0/1404006278 >> 172.21.15.96:0/1404006278 conn(0x7f38880fd350 msgr2=0x7f38880ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:57.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.234+0000 7f388cba8640 1 -- 172.21.15.96:0/1404006278 shutdown_connections 2024-04-25T14:12:57.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.234+0000 7f388cba8640 1 -- 172.21.15.96:0/1404006278 wait complete. 2024-04-25T14:12:57.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.235+0000 7f388cba8640 1 Processor -- start 2024-04-25T14:12:57.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.235+0000 7f388cba8640 1 -- start start 2024-04-25T14:12:57.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.235+0000 7f388cba8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3888101e60 0x7f388819a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:57.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.235+0000 7f388cba8640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3888102360 0x7f388819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:57.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.235+0000 7f388cba8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3888109780 0x7f38881a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:57.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.235+0000 7f388cba8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f388810f640 con 0x7f3888109780 2024-04-25T14:12:57.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.235+0000 7f388cba8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f388810f4c0 con 0x7f3888101e60 2024-04-25T14:12:57.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.235+0000 7f388cba8640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f388810f7c0 con 0x7f3888102360 2024-04-25T14:12:57.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.235+0000 7f3886d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3888109780 0x7f38881a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:57.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.235+0000 7f3886d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3888109780 0x7f38881a1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51586/0 (socket says 172.21.15.96:51586) 2024-04-25T14:12:57.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.235+0000 7f3886d9d640 1 -- 172.21.15.96:0/2687772276 learned_addr learned my addr 172.21.15.96:0/2687772276 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:12:57.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.235+0000 7f388659c640 1 --2- 172.21.15.96:0/2687772276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3888101e60 0x7f388819a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:57.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.236+0000 7f3885d9b640 1 --2- 172.21.15.96:0/2687772276 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3888102360 0x7f388819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:57.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.236+0000 7f3886d9d640 1 -- 172.21.15.96:0/2687772276 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3888102360 msgr2=0x7f388819a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:57.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.236+0000 7f3886d9d640 1 --2- 172.21.15.96:0/2687772276 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3888102360 0x7f388819a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:57.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.236+0000 7f3886d9d640 1 -- 172.21.15.96:0/2687772276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3888101e60 msgr2=0x7f388819a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:57.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.236+0000 7f3886d9d640 1 --2- 172.21.15.96:0/2687772276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3888101e60 0x7f388819a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:57.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.236+0000 7f3886d9d640 1 -- 172.21.15.96:0/2687772276 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38881a1cf0 con 0x7f3888109780 2024-04-25T14:12:57.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.236+0000 7f3886d9d640 1 --2- 172.21.15.96:0/2687772276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3888109780 0x7f38881a1680 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3880003ba0 tx=0x7f3880003bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:57.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.236+0000 7f38777fe640 1 -- 172.21.15.96:0/2687772276 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38800317c0 con 0x7f3888109780 2024-04-25T14:12:57.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.236+0000 7f38777fe640 1 -- 172.21.15.96:0/2687772276 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38800044f0 con 0x7f3888109780 2024-04-25T14:12:57.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.236+0000 7f38777fe640 1 -- 172.21.15.96:0/2687772276 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388002fd00 con 0x7f3888109780 2024-04-25T14:12:57.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.236+0000 7f388cba8640 1 -- 172.21.15.96:0/2687772276 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38881a1f80 con 0x7f3888109780 2024-04-25T14:12:57.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.236+0000 7f388cba8640 1 -- 172.21.15.96:0/2687772276 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38881a23e0 con 0x7f3888109780 2024-04-25T14:12:57.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.237+0000 7f388cba8640 1 -- 172.21.15.96:0/2687772276 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f388806a160 con 0x7f3888109780 2024-04-25T14:12:57.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.239+0000 7f38777fe640 1 -- 172.21.15.96:0/2687772276 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f3880004040 con 0x7f3888109780 2024-04-25T14:12:57.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.240+0000 7f38777fe640 1 --2- 172.21.15.96:0/2687772276 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f38700afaf0 0x7f38700b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:57.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.240+0000 7f388659c640 1 --2- 172.21.15.96:0/2687772276 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f38700afaf0 0x7f38700b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:57.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.240+0000 7f38777fe640 1 -- 172.21.15.96:0/2687772276 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f38801005f0 con 0x7f3888109780 2024-04-25T14:12:57.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.240+0000 7f388659c640 1 --2- 172.21.15.96:0/2687772276 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f38700afaf0 0x7f38700b2000 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f387c00bbe0 tx=0x7f387c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:57.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.244+0000 7f38777fe640 1 -- 172.21.15.96:0/2687772276 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f38800c9150 con 0x7f3888109780 2024-04-25T14:12:57.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.419+0000 7f388cba8640 1 -- 172.21.15.96:0/2687772276 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f38880637a0 con 0x7f3888109780 2024-04-25T14:12:57.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.420+0000 7f38777fe640 1 -- 172.21.15.96:0/2687772276 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v78) ==== 100+0+11 (secure 0 0 0) 0x7f38800098b0 con 0x7f3888109780 2024-04-25T14:12:57.420 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-04-25T14:12:57.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.421+0000 7f388cba8640 1 -- 172.21.15.96:0/2687772276 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f38700afaf0 msgr2=0x7f38700b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:57.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.421+0000 7f388cba8640 1 --2- 172.21.15.96:0/2687772276 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f38700afaf0 0x7f38700b2000 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f387c00bbe0 tx=0x7f387c00b390 comp rx=0 tx=0).stop 2024-04-25T14:12:57.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.422+0000 7f388cba8640 1 -- 172.21.15.96:0/2687772276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3888109780 msgr2=0x7f38881a1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:57.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.422+0000 7f388cba8640 1 --2- 172.21.15.96:0/2687772276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3888109780 0x7f38881a1680 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3880003ba0 tx=0x7f3880003bd0 comp rx=0 tx=0).stop 2024-04-25T14:12:57.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.422+0000 7f388cba8640 1 -- 172.21.15.96:0/2687772276 shutdown_connections 2024-04-25T14:12:57.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.422+0000 7f388cba8640 1 --2- 172.21.15.96:0/2687772276 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f38700afaf0 0x7f38700b2000 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:57.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.422+0000 7f388cba8640 1 --2- 172.21.15.96:0/2687772276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3888109780 0x7f38881a1680 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:57.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.422+0000 7f388cba8640 1 --2- 172.21.15.96:0/2687772276 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3888102360 0x7f388819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:57.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.422+0000 7f388cba8640 1 --2- 172.21.15.96:0/2687772276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3888101e60 0x7f388819a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:57.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.422+0000 7f388cba8640 1 -- 172.21.15.96:0/2687772276 >> 172.21.15.96:0/2687772276 conn(0x7f38880fd350 msgr2=0x7f38881055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:57.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.422+0000 7f388cba8640 1 -- 172.21.15.96:0/2687772276 shutdown_connections 2024-04-25T14:12:57.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.422+0000 7f388cba8640 1 -- 172.21.15.96:0/2687772276 wait complete. 2024-04-25T14:12:57.433 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:12:57.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.541+0000 7fe82e58f640 1 -- 172.21.15.96:0/813443243 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe828109870 msgr2=0x7fe82810bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:57.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.541+0000 7fe82e58f640 1 --2- 172.21.15.96:0/813443243 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe828109870 0x7fe82810bc90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe820009ca0 tx=0x7fe82002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:12:57.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.541+0000 7fe82e58f640 1 -- 172.21.15.96:0/813443243 shutdown_connections 2024-04-25T14:12:57.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.541+0000 7fe82e58f640 1 --2- 172.21.15.96:0/813443243 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe828109870 0x7fe82810bc90 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:57.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.541+0000 7fe82e58f640 1 --2- 172.21.15.96:0/813443243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe828106f10 0x7fe828109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:57.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.541+0000 7fe82e58f640 1 --2- 172.21.15.96:0/813443243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8281045b0 0x7fe8281069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:57.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.541+0000 7fe82e58f640 1 -- 172.21.15.96:0/813443243 >> 172.21.15.96:0/813443243 conn(0x7fe8280fdbd0 msgr2=0x7fe828100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:57.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.541+0000 7fe82e58f640 1 -- 172.21.15.96:0/813443243 shutdown_connections 2024-04-25T14:12:57.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.541+0000 7fe82e58f640 1 -- 172.21.15.96:0/813443243 wait complete. 2024-04-25T14:12:57.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.542+0000 7fe82e58f640 1 Processor -- start 2024-04-25T14:12:57.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.542+0000 7fe82e58f640 1 -- start start 2024-04-25T14:12:57.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.542+0000 7fe82e58f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8281045b0 0x7fe82819a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:57.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.542+0000 7fe82e58f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe828106f10 0x7fe82819a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:57.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.542+0000 7fe82e58f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe828109870 0x7fe8281a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:57.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.542+0000 7fe82e58f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe82810f5f0 con 0x7fe8281045b0 2024-04-25T14:12:57.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.542+0000 7fe82e58f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe82810f470 con 0x7fe828109870 2024-04-25T14:12:57.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.542+0000 7fe82e58f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe82810f770 con 0x7fe828106f10 2024-04-25T14:12:57.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.542+0000 7fe827fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8281045b0 0x7fe82819a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:57.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.543+0000 7fe827fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8281045b0 0x7fe82819a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51608/0 (socket says 172.21.15.96:51608) 2024-04-25T14:12:57.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.543+0000 7fe827fff640 1 -- 172.21.15.96:0/291871372 learned_addr learned my addr 172.21.15.96:0/291871372 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:12:57.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.543+0000 7fe8277fe640 1 --2- 172.21.15.96:0/291871372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe828106f10 0x7fe82819a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:57.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.543+0000 7fe827fff640 1 -- 172.21.15.96:0/291871372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe828106f10 msgr2=0x7fe82819a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:57.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.543+0000 7fe827fff640 1 --2- 172.21.15.96:0/291871372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe828106f10 0x7fe82819a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:57.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.543+0000 7fe827fff640 1 -- 172.21.15.96:0/291871372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe828109870 msgr2=0x7fe8281a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:57.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.543+0000 7fe827fff640 1 --2- 172.21.15.96:0/291871372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe828109870 0x7fe8281a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:57.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.543+0000 7fe827fff640 1 -- 172.21.15.96:0/291871372 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8281a1c50 con 0x7fe8281045b0 2024-04-25T14:12:57.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.543+0000 7fe827fff640 1 --2- 172.21.15.96:0/291871372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8281045b0 0x7fe82819a1c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe81c00c970 tx=0x7fe81c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:57.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.543+0000 7fe8257fa640 1 -- 172.21.15.96:0/291871372 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81c014070 con 0x7fe8281045b0 2024-04-25T14:12:57.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.543+0000 7fe8257fa640 1 -- 172.21.15.96:0/291871372 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe81c004440 con 0x7fe8281045b0 2024-04-25T14:12:57.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.543+0000 7fe82e58f640 1 -- 172.21.15.96:0/291871372 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8281a1f40 con 0x7fe8281045b0 2024-04-25T14:12:57.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.543+0000 7fe8257fa640 1 -- 172.21.15.96:0/291871372 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81c003e50 con 0x7fe8281045b0 2024-04-25T14:12:57.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.543+0000 7fe82e58f640 1 -- 172.21.15.96:0/291871372 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8281a23a0 con 0x7fe8281045b0 2024-04-25T14:12:57.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.544+0000 7fe82e58f640 1 -- 172.21.15.96:0/291871372 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe82806afa0 con 0x7fe8281045b0 2024-04-25T14:12:57.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.546+0000 7fe8257fa640 1 -- 172.21.15.96:0/291871372 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fe81c017070 con 0x7fe8281045b0 2024-04-25T14:12:57.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.546+0000 7fe8257fa640 1 --2- 172.21.15.96:0/291871372 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fe8100afc90 0x7fe8100b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:57.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.546+0000 7fe8277fe640 1 --2- 172.21.15.96:0/291871372 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fe8100afc90 0x7fe8100b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:57.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.546+0000 7fe8257fa640 1 -- 172.21.15.96:0/291871372 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fe81c0defc0 con 0x7fe8281045b0 2024-04-25T14:12:57.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.547+0000 7fe8277fe640 1 --2- 172.21.15.96:0/291871372 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fe8100afc90 0x7fe8100b21a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe818005fd0 tx=0x7fe818005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:57.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.550+0000 7fe8257fa640 1 -- 172.21.15.96:0/291871372 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe81c01e030 con 0x7fe8281045b0 2024-04-25T14:12:57.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.768+0000 7fe82e58f640 1 -- 172.21.15.96:0/291871372 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe82802dc30 con 0x7fe8281045b0 2024-04-25T14:12:57.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.769+0000 7fe8257fa640 1 -- 172.21.15.96:0/291871372 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11927 (secure 0 0 0) 0x7fe81c0a7b80 con 0x7fe8281045b0 2024-04-25T14:12:57.769 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:12:57.769 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/606227917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":606227917},{"type":"v1","addr":"172.21.15.96:6835","nonce":606227917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/3265300710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3265300710},{"type":"v1","addr":"172.21.15.96:6837","nonce":3265300710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T14:11:28.886961+0000","modified":"2024-04-25T14:11:35.532775+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24517,"mds_2":24478,"mds_3":24455,"mds_4":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/4080308509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":4080308509},{"type":"v1","addr":"172.21.15.96:6839","nonce":4080308509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/1606748971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":1606748971},{"type":"v1","addr":"172.21.15.96:6841","nonce":1606748971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6833/1995324831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1995324831},{"type":"v1","addr":"172.21.15.97:6833","nonce":1995324831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":4,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.151:6833/1382721608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1382721608},{"type":"v1","addr":"172.21.15.151:6833","nonce":1382721608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6835/3872301726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3872301726},{"type":"v1","addr":"172.21.15.97:6835","nonce":3872301726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.151:6835/1313823638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":1313823638},{"type":"v1","addr":"172.21.15.151:6835","nonce":1313823638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/2691348366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2691348366},{"type":"v1","addr":"172.21.15.151:6837","nonce":2691348366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3306780081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3306780081},{"type":"v1","addr":"172.21.15.97:6837","nonce":3306780081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.151:6839/245699981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":245699981},{"type":"v1","addr":"172.21.15.151:6839","nonce":245699981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6839/2850442269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":2850442269},{"type":"v1","addr":"172.21.15.97:6839","nonce":2850442269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24517,24536,24478,24455,24427]},"id":1}]} 2024-04-25T14:12:57.769 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 25 2024-04-25T14:12:57.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.771+0000 7fe82e58f640 1 -- 172.21.15.96:0/291871372 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fe8100afc90 msgr2=0x7fe8100b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:57.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.771+0000 7fe82e58f640 1 --2- 172.21.15.96:0/291871372 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fe8100afc90 0x7fe8100b21a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe818005fd0 tx=0x7fe818005850 comp rx=0 tx=0).stop 2024-04-25T14:12:57.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.771+0000 7fe82e58f640 1 -- 172.21.15.96:0/291871372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8281045b0 msgr2=0x7fe82819a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:57.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.771+0000 7fe82e58f640 1 --2- 172.21.15.96:0/291871372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8281045b0 0x7fe82819a1c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe81c00c970 tx=0x7fe81c00ce30 comp rx=0 tx=0).stop 2024-04-25T14:12:57.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.771+0000 7fe82e58f640 1 -- 172.21.15.96:0/291871372 shutdown_connections 2024-04-25T14:12:57.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.771+0000 7fe82e58f640 1 --2- 172.21.15.96:0/291871372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe828109870 0x7fe8281a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:57.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.771+0000 7fe82e58f640 1 --2- 172.21.15.96:0/291871372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe828106f10 0x7fe82819a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:57.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.771+0000 7fe82e58f640 1 --2- 172.21.15.96:0/291871372 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fe8100afc90 0x7fe8100b21a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:57.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.771+0000 7fe82e58f640 1 --2- 172.21.15.96:0/291871372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8281045b0 0x7fe82819a1c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:57.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.771+0000 7fe82e58f640 1 -- 172.21.15.96:0/291871372 >> 172.21.15.96:0/291871372 conn(0x7fe8280fdbd0 msgr2=0x7fe828105180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:57.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.771+0000 7fe82e58f640 1 -- 172.21.15.96:0/291871372 shutdown_connections 2024-04-25T14:12:57.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.771+0000 7fe82e58f640 1 -- 172.21.15.96:0/291871372 wait complete. 2024-04-25T14:12:57.782 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:57 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/1888894827' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:12:57.782 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:57 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/3091498719' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:12:57.782 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:57 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/2687772276' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:12:57.783 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:12:57.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.890+0000 7ff68bc0c640 1 -- 172.21.15.96:0/1640631437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff684102230 msgr2=0x7ff68410bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:57.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.890+0000 7ff68bc0c640 1 --2- 172.21.15.96:0/1640631437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff684102230 0x7ff68410bc90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff67c009900 tx=0x7ff67c02a410 comp rx=0 tx=0).stop 2024-04-25T14:12:57.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.891+0000 7ff68bc0c640 1 -- 172.21.15.96:0/1640631437 shutdown_connections 2024-04-25T14:12:57.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.891+0000 7ff68bc0c640 1 --2- 172.21.15.96:0/1640631437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff684102230 0x7ff68410bc90 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:57.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.891+0000 7ff68bc0c640 1 --2- 172.21.15.96:0/1640631437 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff684101730 0x7ff684101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:57.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.891+0000 7ff68bc0c640 1 --2- 172.21.15.96:0/1640631437 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff684105080 0x7ff684105490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:57.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.891+0000 7ff68bc0c640 1 -- 172.21.15.96:0/1640631437 >> 172.21.15.96:0/1640631437 conn(0x7ff6840fd330 msgr2=0x7ff6840ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:57.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.891+0000 7ff68bc0c640 1 -- 172.21.15.96:0/1640631437 shutdown_connections 2024-04-25T14:12:57.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.891+0000 7ff68bc0c640 1 -- 172.21.15.96:0/1640631437 wait complete. 2024-04-25T14:12:57.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.891+0000 7ff68bc0c640 1 Processor -- start 2024-04-25T14:12:57.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.891+0000 7ff68bc0c640 1 -- start start 2024-04-25T14:12:57.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.891+0000 7ff68bc0c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff684101730 0x7ff68419a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:57.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.891+0000 7ff68bc0c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff684102230 0x7ff68419a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:57.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.891+0000 7ff68bc0c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff684105080 0x7ff6841a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:57.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.891+0000 7ff68bc0c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff68410f690 con 0x7ff684102230 2024-04-25T14:12:57.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.891+0000 7ff68bc0c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff68410f510 con 0x7ff684101730 2024-04-25T14:12:57.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.891+0000 7ff68bc0c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff68410f810 con 0x7ff684105080 2024-04-25T14:12:57.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.892+0000 7ff6891a7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff684102230 0x7ff68419a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:57.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.892+0000 7ff6891a7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff684102230 0x7ff68419a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51630/0 (socket says 172.21.15.96:51630) 2024-04-25T14:12:57.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.892+0000 7ff6891a7640 1 -- 172.21.15.96:0/3658765910 learned_addr learned my addr 172.21.15.96:0/3658765910 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:12:57.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.892+0000 7ff6891a7640 1 -- 172.21.15.96:0/3658765910 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff684105080 msgr2=0x7ff6841a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:57.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.892+0000 7ff6891a7640 1 --2- 172.21.15.96:0/3658765910 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff684105080 0x7ff6841a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:57.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.892+0000 7ff6899a8640 1 --2- 172.21.15.96:0/3658765910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff684101730 0x7ff68419a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:57.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.892+0000 7ff6891a7640 1 -- 172.21.15.96:0/3658765910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff684101730 msgr2=0x7ff68419a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:57.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.892+0000 7ff6891a7640 1 --2- 172.21.15.96:0/3658765910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff684101730 0x7ff68419a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:57.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.892+0000 7ff6891a7640 1 -- 172.21.15.96:0/3658765910 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6841a1d90 con 0x7ff684102230 2024-04-25T14:12:57.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.892+0000 7ff6899a8640 1 --2- 172.21.15.96:0/3658765910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff684101730 0x7ff68419a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:12:57.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.892+0000 7ff6891a7640 1 --2- 172.21.15.96:0/3658765910 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff684102230 0x7ff68419a790 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff67400d770 tx=0x7ff67400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:57.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.892+0000 7ff67affd640 1 -- 172.21.15.96:0/3658765910 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff674017070 con 0x7ff684102230 2024-04-25T14:12:57.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.892+0000 7ff67affd640 1 -- 172.21.15.96:0/3658765910 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6740054d0 con 0x7ff684102230 2024-04-25T14:12:57.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.892+0000 7ff67affd640 1 -- 172.21.15.96:0/3658765910 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff674004c30 con 0x7ff684102230 2024-04-25T14:12:57.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.892+0000 7ff68bc0c640 1 -- 172.21.15.96:0/3658765910 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6841a2080 con 0x7ff684102230 2024-04-25T14:12:57.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.892+0000 7ff68bc0c640 1 -- 172.21.15.96:0/3658765910 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6841a2600 con 0x7ff684102230 2024-04-25T14:12:57.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.893+0000 7ff68bc0c640 1 -- 172.21.15.96:0/3658765910 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff68410aa20 con 0x7ff684102230 2024-04-25T14:12:57.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.894+0000 7ff67affd640 1 -- 172.21.15.96:0/3658765910 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7ff674004dd0 con 0x7ff684102230 2024-04-25T14:12:57.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.895+0000 7ff67affd640 1 --2- 172.21.15.96:0/3658765910 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ff6700afaf0 0x7ff6700b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:57.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.895+0000 7ff6899a8640 1 --2- 172.21.15.96:0/3658765910 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ff6700afaf0 0x7ff6700b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:57.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.895+0000 7ff67affd640 1 -- 172.21.15.96:0/3658765910 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7ff6740debc0 con 0x7ff684102230 2024-04-25T14:12:57.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.895+0000 7ff6899a8640 1 --2- 172.21.15.96:0/3658765910 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ff6700afaf0 0x7ff6700b2000 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff684069bf0 tx=0x7ff6800073c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:57.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:57.898+0000 7ff67affd640 1 -- 172.21.15.96:0/3658765910 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff6740a7780 con 0x7ff684102230 2024-04-25T14:12:57.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:57 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/1888894827' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:12:57.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:57 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/3091498719' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:12:57.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:57 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/2687772276' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:12:57.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:57 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/1888894827' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:12:57.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:57 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/3091498719' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:12:57.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:57 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/2687772276' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:12:58.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.076+0000 7ff68bc0c640 1 -- 172.21.15.96:0/3658765910 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff6840637a0 con 0x7ff684102230 2024-04-25T14:12:58.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.078+0000 7ff67affd640 1 -- 172.21.15.96:0/3658765910 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22616 (secure 0 0 0) 0x7ff6740a7080 con 0x7ff684102230 2024-04-25T14:12:58.078 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:12:58.078 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":78,"fsid":"59334182-030c-11ef-bc93-c7b262605968","created":"2024-04-25T14:03:25.680745+0000","modified":"2024-04-25T14:12:22.496143+0000","last_up_change":"2024-04-25T14:10:35.894312+0000","last_in_change":"2024-04-25T14:10:15.583440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:06:47.402927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:11:27.434331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:11:27.854004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cf4ba3ac-142e-4e8e-96d4-410d77157eb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6803","nonce":3832506168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6805","nonce":3832506168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832506168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6807","nonce":3832506168}]},"public_addr":"172.21.15.96:6803/3832506168","cluster_addr":"172.21.15.96:6805/3832506168","heartbeat_back_addr":"172.21.15.96:6809/3832506168","heartbeat_front_addr":"172.21.15.96:6807/3832506168","state":["exists","up"]},{"osd":1,"uuid":"866e73b9-4a26-48ba-badf-704fe22f71cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6801","nonce":1083311494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6803","nonce":1083311494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6807","nonce":1083311494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6805","nonce":1083311494}]},"public_addr":"172.21.15.97:6801/1083311494","cluster_addr":"172.21.15.97:6803/1083311494","heartbeat_back_addr":"172.21.15.97:6807/1083311494","heartbeat_front_addr":"172.21.15.97:6805/1083311494","state":["exists","up"]},{"osd":2,"uuid":"45c79ecb-abc4-4dba-bae8-74804e4bc1bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6801","nonce":981902966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6803","nonce":981902966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6807","nonce":981902966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6805","nonce":981902966}]},"public_addr":"172.21.15.151:6801/981902966","cluster_addr":"172.21.15.151:6803/981902966","heartbeat_back_addr":"172.21.15.151:6807/981902966","heartbeat_front_addr":"172.21.15.151:6805/981902966","state":["exists","up"]},{"osd":3,"uuid":"9fc39468-a7ea-4ccf-b365-335ad2ea4c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6811","nonce":2274158748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6813","nonce":2274158748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6817","nonce":2274158748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6815","nonce":2274158748}]},"public_addr":"172.21.15.96:6811/2274158748","cluster_addr":"172.21.15.96:6813/2274158748","heartbeat_back_addr":"172.21.15.96:6817/2274158748","heartbeat_front_addr":"172.21.15.96:6815/2274158748","state":["exists","up"]},{"osd":4,"uuid":"616582f1-12d2-456b-b7b3-e5517800e8a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6809","nonce":3727031962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6811","nonce":3727031962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6815","nonce":3727031962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6813","nonce":3727031962}]},"public_addr":"172.21.15.97:6809/3727031962","cluster_addr":"172.21.15.97:6811/3727031962","heartbeat_back_addr":"172.21.15.97:6815/3727031962","heartbeat_front_addr":"172.21.15.97:6813/3727031962","state":["exists","up"]},{"osd":5,"uuid":"3d277784-ab0f-4c90-81a1-2a4034c52a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6809","nonce":223882546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6811","nonce":223882546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6815","nonce":223882546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6813","nonce":223882546}]},"public_addr":"172.21.15.151:6809/223882546","cluster_addr":"172.21.15.151:6811/223882546","heartbeat_back_addr":"172.21.15.151:6815/223882546","heartbeat_front_addr":"172.21.15.151:6813/223882546","state":["exists","up"]},{"osd":6,"uuid":"9a9ad585-cb1a-4239-9d18-055f59164049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6819","nonce":96197407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6821","nonce":96197407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6825","nonce":96197407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6823","nonce":96197407}]},"public_addr":"172.21.15.96:6819/96197407","cluster_addr":"172.21.15.96:6821/96197407","heartbeat_back_addr":"172.21.15.96:6825/96197407","heartbeat_front_addr":"172.21.15.96:6823/96197407","state":["exists","up"]},{"osd":7,"uuid":"303551c8-6af0-42f2-844f-45bf8db5baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6817","nonce":2247216400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6819","nonce":2247216400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6823","nonce":2247216400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6821","nonce":2247216400}]},"public_addr":"172.21.15.97:6817/2247216400","cluster_addr":"172.21.15.97:6819/2247216400","heartbeat_back_addr":"172.21.15.97:6823/2247216400","heartbeat_front_addr":"172.21.15.97:6821/2247216400","state":["exists","up"]},{"osd":8,"uuid":"81f6cc66-e4ca-430c-b336-b6678d968818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6817","nonce":2549822185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6819","nonce":2549822185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6823","nonce":2549822185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6821","nonce":2549822185}]},"public_addr":"172.21.15.151:6817/2549822185","cluster_addr":"172.21.15.151:6819/2549822185","heartbeat_back_addr":"172.21.15.151:6823/2549822185","heartbeat_front_addr":"172.21.15.151:6821/2549822185","state":["exists","up"]},{"osd":9,"uuid":"29d4da16-feab-4d1c-9d1d-de4f799bd724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6827","nonce":1099399643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6829","nonce":1099399643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6833","nonce":1099399643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6831","nonce":1099399643}]},"public_addr":"172.21.15.96:6827/1099399643","cluster_addr":"172.21.15.96:6829/1099399643","heartbeat_back_addr":"172.21.15.96:6833/1099399643","heartbeat_front_addr":"172.21.15.96:6831/1099399643","state":["exists","up"]},{"osd":10,"uuid":"197fee9b-eee4-4893-be1c-3d57f42250e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6825","nonce":155754520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6827","nonce":155754520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6831","nonce":155754520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6829","nonce":155754520}]},"public_addr":"172.21.15.97:6825/155754520","cluster_addr":"172.21.15.97:6827/155754520","heartbeat_back_addr":"172.21.15.97:6831/155754520","heartbeat_front_addr":"172.21.15.97:6829/155754520","state":["exists","up"]},{"osd":11,"uuid":"9c1d0a83-fc12-41a2-8eb0-80c4abdd2770","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6825","nonce":2027958746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6827","nonce":2027958746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6831","nonce":2027958746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6829","nonce":2027958746}]},"public_addr":"172.21.15.151:6825/2027958746","cluster_addr":"172.21.15.151:6827/2027958746","heartbeat_back_addr":"172.21.15.151:6831/2027958746","heartbeat_front_addr":"172.21.15.151:6829/2027958746","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:05:56.177390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:06:19.917663+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:06:43.608052+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:08.540382+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:33.474072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:58.242306+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:08:24.120442+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:08:49.661064+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:09:15.097271+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:09:41.904505+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:10:08.233252+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:10:33.712296+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/3839807484":"2024-04-26T14:04:21.307574+0000","172.21.15.96:0/1495237551":"2024-04-26T14:04:21.307574+0000","172.21.15.96:0/2793751450":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/129575834":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/2233075139":"2024-04-26T14:04:21.307574+0000","172.21.15.96:6801/2348397481":"2024-04-26T14:03:54.249537+0000","172.21.15.96:6801/433591062":"2024-04-26T14:04:21.307574+0000","172.21.15.96:6800/2348397481":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/3517674924":"2024-04-26T14:03:54.249537+0000","172.21.15.96:6800/433591062":"2024-04-26T14:04:21.307574+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:12:58.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.080+0000 7ff68bc0c640 1 -- 172.21.15.96:0/3658765910 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ff6700afaf0 msgr2=0x7ff6700b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:58.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.080+0000 7ff68bc0c640 1 --2- 172.21.15.96:0/3658765910 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ff6700afaf0 0x7ff6700b2000 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff684069bf0 tx=0x7ff6800073c0 comp rx=0 tx=0).stop 2024-04-25T14:12:58.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.080+0000 7ff68bc0c640 1 -- 172.21.15.96:0/3658765910 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff684102230 msgr2=0x7ff68419a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:58.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.080+0000 7ff68bc0c640 1 --2- 172.21.15.96:0/3658765910 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff684102230 0x7ff68419a790 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff67400d770 tx=0x7ff67400dc30 comp rx=0 tx=0).stop 2024-04-25T14:12:58.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.080+0000 7ff68bc0c640 1 -- 172.21.15.96:0/3658765910 shutdown_connections 2024-04-25T14:12:58.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.080+0000 7ff68bc0c640 1 --2- 172.21.15.96:0/3658765910 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff684105080 0x7ff6841a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:58.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.080+0000 7ff68bc0c640 1 --2- 172.21.15.96:0/3658765910 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ff6700afaf0 0x7ff6700b2000 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:58.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.080+0000 7ff68bc0c640 1 --2- 172.21.15.96:0/3658765910 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff684102230 0x7ff68419a790 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:58.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.080+0000 7ff68bc0c640 1 --2- 172.21.15.96:0/3658765910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff684101730 0x7ff68419a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:58.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.080+0000 7ff68bc0c640 1 -- 172.21.15.96:0/3658765910 >> 172.21.15.96:0/3658765910 conn(0x7ff6840fd330 msgr2=0x7ff684107bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:58.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.080+0000 7ff68bc0c640 1 -- 172.21.15.96:0/3658765910 shutdown_connections 2024-04-25T14:12:58.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.080+0000 7ff68bc0c640 1 -- 172.21.15.96:0/3658765910 wait complete. 2024-04-25T14:12:58.092 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:06:47.402927+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:11:27.434331+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:11:27.854004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:12:58.092 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:12:58.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.203+0000 7f1ab57a8640 1 -- 172.21.15.96:0/956494205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ab01080b0 msgr2=0x7f1ab010a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:58.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.203+0000 7f1ab57a8640 1 --2- 172.21.15.96:0/956494205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ab01080b0 0x7f1ab010a4d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8009ca0 tx=0x7f1aa802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:12:58.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.203+0000 7f1ab57a8640 1 -- 172.21.15.96:0/956494205 shutdown_connections 2024-04-25T14:12:58.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.203+0000 7f1ab57a8640 1 --2- 172.21.15.96:0/956494205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ab01080b0 0x7f1ab010a4d0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:58.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.203+0000 7f1ab57a8640 1 --2- 172.21.15.96:0/956494205 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ab01033b0 0x7f1ab01037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:58.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.203+0000 7f1ab57a8640 1 --2- 172.21.15.96:0/956494205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ab0102170 0x7f1ab0102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:58.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.203+0000 7f1ab57a8640 1 -- 172.21.15.96:0/956494205 >> 172.21.15.96:0/956494205 conn(0x7f1ab00fd340 msgr2=0x7f1ab00ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:58.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.203+0000 7f1ab57a8640 1 -- 172.21.15.96:0/956494205 shutdown_connections 2024-04-25T14:12:58.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.203+0000 7f1ab57a8640 1 -- 172.21.15.96:0/956494205 wait complete. 2024-04-25T14:12:58.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.204+0000 7f1ab57a8640 1 Processor -- start 2024-04-25T14:12:58.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.204+0000 7f1ab57a8640 1 -- start start 2024-04-25T14:12:58.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.204+0000 7f1ab57a8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ab0102170 0x7f1ab0108f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:58.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.204+0000 7f1ab57a8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ab01033b0 0x7f1ab01a1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:58.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.204+0000 7f1ab57a8640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ab01080b0 0x7f1ab01a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:58.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.204+0000 7f1ab57a8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab010dc90 con 0x7f1ab0102170 2024-04-25T14:12:58.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.204+0000 7f1ab57a8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab010db10 con 0x7f1ab01033b0 2024-04-25T14:12:58.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.204+0000 7f1ab57a8640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab010de10 con 0x7f1ab01080b0 2024-04-25T14:12:58.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.204+0000 7f1aaeffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ab0102170 0x7f1ab0108f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:58.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.204+0000 7f1aaeffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ab0102170 0x7f1ab0108f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51636/0 (socket says 172.21.15.96:51636) 2024-04-25T14:12:58.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.204+0000 7f1aaeffd640 1 -- 172.21.15.96:0/838922960 learned_addr learned my addr 172.21.15.96:0/838922960 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:12:58.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.204+0000 7f1aa7fff640 1 --2- 172.21.15.96:0/838922960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ab01033b0 0x7f1ab01a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:58.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.204+0000 7f1aaeffd640 1 -- 172.21.15.96:0/838922960 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ab01080b0 msgr2=0x7f1ab01a3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:58.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.204+0000 7f1aaeffd640 1 --2- 172.21.15.96:0/838922960 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ab01080b0 0x7f1ab01a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:58.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.204+0000 7f1aaeffd640 1 -- 172.21.15.96:0/838922960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ab01033b0 msgr2=0x7f1ab01a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:58.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.204+0000 7f1aaeffd640 1 --2- 172.21.15.96:0/838922960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ab01033b0 0x7f1ab01a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:58.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.204+0000 7f1aaeffd640 1 -- 172.21.15.96:0/838922960 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab01a3e20 con 0x7f1ab0102170 2024-04-25T14:12:58.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.204+0000 7f1aa7fff640 1 --2- 172.21.15.96:0/838922960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ab01033b0 0x7f1ab01a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:12:58.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.205+0000 7f1aaeffd640 1 --2- 172.21.15.96:0/838922960 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ab0102170 0x7f1ab0108f30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1aa000c970 tx=0x7f1aa000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:58.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.205+0000 7f1aacff9640 1 -- 172.21.15.96:0/838922960 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa0007c70 con 0x7f1ab0102170 2024-04-25T14:12:58.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.205+0000 7f1ab57a8640 1 -- 172.21.15.96:0/838922960 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ab01a4110 con 0x7f1ab0102170 2024-04-25T14:12:58.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.205+0000 7f1aacff9640 1 -- 172.21.15.96:0/838922960 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1aa0007e10 con 0x7f1ab0102170 2024-04-25T14:12:58.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.205+0000 7f1aacff9640 1 -- 172.21.15.96:0/838922960 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa0018760 con 0x7f1ab0102170 2024-04-25T14:12:58.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.205+0000 7f1ab57a8640 1 -- 172.21.15.96:0/838922960 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ab01a4660 con 0x7f1ab0102170 2024-04-25T14:12:58.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.206+0000 7f1ab57a8640 1 -- 172.21.15.96:0/838922960 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ab01077b0 con 0x7f1ab0102170 2024-04-25T14:12:58.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.207+0000 7f1aacff9640 1 -- 172.21.15.96:0/838922960 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f1aa000fab0 con 0x7f1ab0102170 2024-04-25T14:12:58.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.208+0000 7f1aacff9640 1 --2- 172.21.15.96:0/838922960 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1a9c0afaf0 0x7f1a9c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:58.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.208+0000 7f1aa7fff640 1 --2- 172.21.15.96:0/838922960 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1a9c0afaf0 0x7f1a9c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:58.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.208+0000 7f1aacff9640 1 -- 172.21.15.96:0/838922960 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f1aa00deec0 con 0x7f1ab0102170 2024-04-25T14:12:58.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.208+0000 7f1aa7fff640 1 --2- 172.21.15.96:0/838922960 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1a9c0afaf0 0x7f1a9c0b2000 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1a98005fd0 tx=0x7f1a98005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:58.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.212+0000 7f1aacff9640 1 -- 172.21.15.96:0/838922960 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1aa0014030 con 0x7f1ab0102170 2024-04-25T14:12:58.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.388+0000 7f1ab57a8640 1 -- 172.21.15.96:0/838922960 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1ab002dd00 con 0x7f1ab0102170 2024-04-25T14:12:58.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.389+0000 7f1aacff9640 1 -- 172.21.15.96:0/838922960 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) ==== 93+0+10 (secure 0 0 0) 0x7f1aa00a7ab0 con 0x7f1ab0102170 2024-04-25T14:12:58.389 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-04-25T14:12:58.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.391+0000 7f1ab57a8640 1 -- 172.21.15.96:0/838922960 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1a9c0afaf0 msgr2=0x7f1a9c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:58.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.391+0000 7f1ab57a8640 1 --2- 172.21.15.96:0/838922960 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1a9c0afaf0 0x7f1a9c0b2000 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1a98005fd0 tx=0x7f1a98005850 comp rx=0 tx=0).stop 2024-04-25T14:12:58.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.391+0000 7f1ab57a8640 1 -- 172.21.15.96:0/838922960 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ab0102170 msgr2=0x7f1ab0108f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:58.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.391+0000 7f1ab57a8640 1 --2- 172.21.15.96:0/838922960 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ab0102170 0x7f1ab0108f30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1aa000c970 tx=0x7f1aa000ce30 comp rx=0 tx=0).stop 2024-04-25T14:12:58.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.391+0000 7f1ab57a8640 1 -- 172.21.15.96:0/838922960 shutdown_connections 2024-04-25T14:12:58.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.391+0000 7f1ab57a8640 1 --2- 172.21.15.96:0/838922960 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ab01080b0 0x7f1ab01a3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:58.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.391+0000 7f1ab57a8640 1 --2- 172.21.15.96:0/838922960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ab01033b0 0x7f1ab01a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:58.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.391+0000 7f1ab57a8640 1 --2- 172.21.15.96:0/838922960 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1a9c0afaf0 0x7f1a9c0b2000 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:58.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.391+0000 7f1ab57a8640 1 --2- 172.21.15.96:0/838922960 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ab0102170 0x7f1ab0108f30 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:58.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.391+0000 7f1ab57a8640 1 -- 172.21.15.96:0/838922960 >> 172.21.15.96:0/838922960 conn(0x7f1ab00fd340 msgr2=0x7f1ab00fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:58.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.391+0000 7f1ab57a8640 1 -- 172.21.15.96:0/838922960 shutdown_connections 2024-04-25T14:12:58.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.391+0000 7f1ab57a8640 1 -- 172.21.15.96:0/838922960 wait complete. 2024-04-25T14:12:58.402 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:12:58.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.513+0000 7f4316beb640 1 -- 172.21.15.96:0/1748638186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4310104470 msgr2=0x7f431010abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:58.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.513+0000 7f4316beb640 1 --2- 172.21.15.96:0/1748638186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4310104470 0x7f431010abb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4308009ca0 tx=0x7f430802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:12:58.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.513+0000 7f4316beb640 1 -- 172.21.15.96:0/1748638186 shutdown_connections 2024-04-25T14:12:58.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.513+0000 7f4316beb640 1 --2- 172.21.15.96:0/1748638186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4310104470 0x7f431010abb0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:58.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.513+0000 7f4316beb640 1 --2- 172.21.15.96:0/1748638186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4310103930 0x7f4310103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:58.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.513+0000 7f4316beb640 1 --2- 172.21.15.96:0/1748638186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4310102f50 0x7f4310103360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:58.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.513+0000 7f4316beb640 1 -- 172.21.15.96:0/1748638186 >> 172.21.15.96:0/1748638186 conn(0x7f43100fd350 msgr2=0x7f43100ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:58.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.513+0000 7f4316beb640 1 -- 172.21.15.96:0/1748638186 shutdown_connections 2024-04-25T14:12:58.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.513+0000 7f4316beb640 1 -- 172.21.15.96:0/1748638186 wait complete. 2024-04-25T14:12:58.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.514+0000 7f4316beb640 1 Processor -- start 2024-04-25T14:12:58.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.514+0000 7f4316beb640 1 -- start start 2024-04-25T14:12:58.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.514+0000 7f4316beb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4310102f50 0x7f4310199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:58.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.514+0000 7f4316beb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4310103930 0x7f43101995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:58.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.514+0000 7f4316beb640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4310104470 0x7f43101a0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:58.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.514+0000 7f4316beb640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f431010e540 con 0x7f4310102f50 2024-04-25T14:12:58.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.514+0000 7f4316beb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f431010e3c0 con 0x7f4310103930 2024-04-25T14:12:58.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.514+0000 7f4316beb640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f431010e6c0 con 0x7f4310104470 2024-04-25T14:12:58.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.514+0000 7f4314987640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4310102f50 0x7f4310199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:58.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.514+0000 7f4314987640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4310102f50 0x7f4310199090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51650/0 (socket says 172.21.15.96:51650) 2024-04-25T14:12:58.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.514+0000 7f4314987640 1 -- 172.21.15.96:0/910227700 learned_addr learned my addr 172.21.15.96:0/910227700 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:12:58.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.514+0000 7f4315188640 1 --2- 172.21.15.96:0/910227700 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4310104470 0x7f43101a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:58.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.514+0000 7f430ffff640 1 --2- 172.21.15.96:0/910227700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4310103930 0x7f43101995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:58.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.515+0000 7f4314987640 1 -- 172.21.15.96:0/910227700 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4310104470 msgr2=0x7f43101a0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:58.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.515+0000 7f4314987640 1 --2- 172.21.15.96:0/910227700 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4310104470 0x7f43101a0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:58.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.515+0000 7f4314987640 1 -- 172.21.15.96:0/910227700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4310103930 msgr2=0x7f43101995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:58.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.515+0000 7f4314987640 1 --2- 172.21.15.96:0/910227700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4310103930 0x7f43101995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:58.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.515+0000 7f4314987640 1 -- 172.21.15.96:0/910227700 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43101a0bd0 con 0x7f4310102f50 2024-04-25T14:12:58.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.515+0000 7f4314987640 1 --2- 172.21.15.96:0/910227700 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4310102f50 0x7f4310199090 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f430400c970 tx=0x7f430400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:58.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.515+0000 7f430dffb640 1 -- 172.21.15.96:0/910227700 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4304007bb0 con 0x7f4310102f50 2024-04-25T14:12:58.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.515+0000 7f430dffb640 1 -- 172.21.15.96:0/910227700 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4304007d50 con 0x7f4310102f50 2024-04-25T14:12:58.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.515+0000 7f4316beb640 1 -- 172.21.15.96:0/910227700 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43101a0ec0 con 0x7f4310102f50 2024-04-25T14:12:58.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.515+0000 7f430dffb640 1 -- 172.21.15.96:0/910227700 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430400f450 con 0x7f4310102f50 2024-04-25T14:12:58.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.515+0000 7f4316beb640 1 -- 172.21.15.96:0/910227700 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43101a1330 con 0x7f4310102f50 2024-04-25T14:12:58.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.516+0000 7f4316beb640 1 -- 172.21.15.96:0/910227700 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4310109900 con 0x7f4310102f50 2024-04-25T14:12:58.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.517+0000 7f430dffb640 1 -- 172.21.15.96:0/910227700 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f4304004eb0 con 0x7f4310102f50 2024-04-25T14:12:58.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.518+0000 7f430dffb640 1 --2- 172.21.15.96:0/910227700 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f43000afb70 0x7f43000b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:58.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.518+0000 7f430ffff640 1 --2- 172.21.15.96:0/910227700 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f43000afb70 0x7f43000b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:58.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.518+0000 7f430dffb640 1 -- 172.21.15.96:0/910227700 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f43040dec70 con 0x7f4310102f50 2024-04-25T14:12:58.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.519+0000 7f430ffff640 1 --2- 172.21.15.96:0/910227700 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f43000afb70 0x7f43000b2080 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f431019a130 tx=0x7f42fc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:58.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.522+0000 7f430dffb640 1 -- 172.21.15.96:0/910227700 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4304014030 con 0x7f4310102f50 2024-04-25T14:12:58.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.701+0000 7f4316beb640 1 -- 172.21.15.96:0/910227700 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f431002dda0 con 0x7f4310102f50 2024-04-25T14:12:58.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.701+0000 7f430dffb640 1 -- 172.21.15.96:0/910227700 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v78) ==== 104+0+11 (secure 0 0 0) 0x7f43040a7260 con 0x7f4310102f50 2024-04-25T14:12:58.702 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-04-25T14:12:58.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.703+0000 7f4316beb640 1 -- 172.21.15.96:0/910227700 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f43000afb70 msgr2=0x7f43000b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:58.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.703+0000 7f4316beb640 1 --2- 172.21.15.96:0/910227700 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f43000afb70 0x7f43000b2080 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f431019a130 tx=0x7f42fc00b390 comp rx=0 tx=0).stop 2024-04-25T14:12:58.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.703+0000 7f4316beb640 1 -- 172.21.15.96:0/910227700 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4310102f50 msgr2=0x7f4310199090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:58.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.703+0000 7f4316beb640 1 --2- 172.21.15.96:0/910227700 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4310102f50 0x7f4310199090 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f430400c970 tx=0x7f430400ce30 comp rx=0 tx=0).stop 2024-04-25T14:12:58.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.703+0000 7f4316beb640 1 -- 172.21.15.96:0/910227700 shutdown_connections 2024-04-25T14:12:58.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.703+0000 7f4316beb640 1 --2- 172.21.15.96:0/910227700 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4310104470 0x7f43101a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:58.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.703+0000 7f4316beb640 1 --2- 172.21.15.96:0/910227700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4310103930 0x7f43101995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:58.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.703+0000 7f4316beb640 1 --2- 172.21.15.96:0/910227700 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f43000afb70 0x7f43000b2080 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:58.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.703+0000 7f4316beb640 1 --2- 172.21.15.96:0/910227700 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4310102f50 0x7f4310199090 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:58.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.703+0000 7f4316beb640 1 -- 172.21.15.96:0/910227700 >> 172.21.15.96:0/910227700 conn(0x7f43100fd350 msgr2=0x7f4310106ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:58.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.703+0000 7f4316beb640 1 -- 172.21.15.96:0/910227700 shutdown_connections 2024-04-25T14:12:58.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.703+0000 7f4316beb640 1 -- 172.21.15.96:0/910227700 wait complete. 2024-04-25T14:12:58.713 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:58 smithi096 ceph-mon[29207]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 13 op/s 2024-04-25T14:12:58.714 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:58 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/291871372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:12:58.714 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:58 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/3658765910' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:12:58.714 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:58 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/838922960' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:12:58.715 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:12:58.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.825+0000 7f5c31fa8640 1 -- 172.21.15.96:0/3701554234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c2c102180 msgr2=0x7f5c2c102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:58.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.825+0000 7f5c31fa8640 1 --2- 172.21.15.96:0/3701554234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c2c102180 0x7f5c2c102e80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c009900 tx=0x7f5c1c02a410 comp rx=0 tx=0).stop 2024-04-25T14:12:58.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.825+0000 7f5c31fa8640 1 -- 172.21.15.96:0/3701554234 shutdown_connections 2024-04-25T14:12:58.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.825+0000 7f5c31fa8640 1 --2- 172.21.15.96:0/3701554234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c2c1080c0 0x7f5c2c10a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:58.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.825+0000 7f5c31fa8640 1 --2- 172.21.15.96:0/3701554234 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c2c1033c0 0x7f5c2c1037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:58.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.825+0000 7f5c31fa8640 1 --2- 172.21.15.96:0/3701554234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c2c102180 0x7f5c2c102e80 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:58.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.825+0000 7f5c31fa8640 1 -- 172.21.15.96:0/3701554234 >> 172.21.15.96:0/3701554234 conn(0x7f5c2c0fd350 msgr2=0x7f5c2c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:58.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.825+0000 7f5c31fa8640 1 -- 172.21.15.96:0/3701554234 shutdown_connections 2024-04-25T14:12:58.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.825+0000 7f5c31fa8640 1 -- 172.21.15.96:0/3701554234 wait complete. 2024-04-25T14:12:58.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.826+0000 7f5c31fa8640 1 Processor -- start 2024-04-25T14:12:58.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.826+0000 7f5c31fa8640 1 -- start start 2024-04-25T14:12:58.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.826+0000 7f5c31fa8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c2c102180 0x7f5c2c19a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:58.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.826+0000 7f5c31fa8640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c2c1033c0 0x7f5c2c19a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:58.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.826+0000 7f5c31fa8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c2c1080c0 0x7f5c2c1a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:58.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.826+0000 7f5c31fa8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2c10f620 con 0x7f5c2c1080c0 2024-04-25T14:12:58.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.826+0000 7f5c31fa8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2c10f4a0 con 0x7f5c2c102180 2024-04-25T14:12:58.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.826+0000 7f5c31fa8640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2c10f7a0 con 0x7f5c2c1033c0 2024-04-25T14:12:58.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.826+0000 7f5c2bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c2c1080c0 0x7f5c2c1a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:58.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.826+0000 7f5c2bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c2c1080c0 0x7f5c2c1a16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51680/0 (socket says 172.21.15.96:51680) 2024-04-25T14:12:58.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.826+0000 7f5c2bfff640 1 -- 172.21.15.96:0/2871493694 learned_addr learned my addr 172.21.15.96:0/2871493694 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:12:58.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.826+0000 7f5c2b7fe640 1 --2- 172.21.15.96:0/2871493694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c2c102180 0x7f5c2c19a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:58.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.827+0000 7f5c2affd640 1 --2- 172.21.15.96:0/2871493694 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c2c1033c0 0x7f5c2c19a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:58.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.827+0000 7f5c2bfff640 1 -- 172.21.15.96:0/2871493694 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c2c1033c0 msgr2=0x7f5c2c19a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:58.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.827+0000 7f5c2bfff640 1 --2- 172.21.15.96:0/2871493694 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c2c1033c0 0x7f5c2c19a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:58.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.827+0000 7f5c2bfff640 1 -- 172.21.15.96:0/2871493694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c2c102180 msgr2=0x7f5c2c19a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:58.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.827+0000 7f5c2bfff640 1 --2- 172.21.15.96:0/2871493694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c2c102180 0x7f5c2c19a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:58.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.827+0000 7f5c2bfff640 1 -- 172.21.15.96:0/2871493694 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c2c1a1d40 con 0x7f5c2c1080c0 2024-04-25T14:12:58.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.827+0000 7f5c2bfff640 1 --2- 172.21.15.96:0/2871493694 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c2c1080c0 0x7f5c2c1a16d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5c2400eaa0 tx=0x7f5c2400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:58.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.827+0000 7f5c28ff9640 1 -- 172.21.15.96:0/2871493694 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2400cdb0 con 0x7f5c2c1080c0 2024-04-25T14:12:58.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.827+0000 7f5c28ff9640 1 -- 172.21.15.96:0/2871493694 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c24005020 con 0x7f5c2c1080c0 2024-04-25T14:12:58.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.827+0000 7f5c31fa8640 1 -- 172.21.15.96:0/2871493694 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c2c1a1fd0 con 0x7f5c2c1080c0 2024-04-25T14:12:58.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.827+0000 7f5c28ff9640 1 -- 172.21.15.96:0/2871493694 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c240116d0 con 0x7f5c2c1080c0 2024-04-25T14:12:58.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.827+0000 7f5c31fa8640 1 -- 172.21.15.96:0/2871493694 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c2c1a24e0 con 0x7f5c2c1080c0 2024-04-25T14:12:58.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.828+0000 7f5c31fa8640 1 -- 172.21.15.96:0/2871493694 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c2c106ed0 con 0x7f5c2c1080c0 2024-04-25T14:12:58.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.829+0000 7f5c28ff9640 1 -- 172.21.15.96:0/2871493694 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f5c240051d0 con 0x7f5c2c1080c0 2024-04-25T14:12:58.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.830+0000 7f5c28ff9640 1 --2- 172.21.15.96:0/2871493694 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f5c140afb70 0x7f5c140b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:58.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.830+0000 7f5c2b7fe640 1 --2- 172.21.15.96:0/2871493694 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f5c140afb70 0x7f5c140b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:58.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.830+0000 7f5c28ff9640 1 -- 172.21.15.96:0/2871493694 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f5c240df040 con 0x7f5c2c1080c0 2024-04-25T14:12:58.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.830+0000 7f5c2b7fe640 1 --2- 172.21.15.96:0/2871493694 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f5c140afb70 0x7f5c140b2080 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c005ab0 tx=0x7f5c1c031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:58.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:58.834+0000 7f5c28ff9640 1 -- 172.21.15.96:0/2871493694 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5c24016030 con 0x7f5c2c1080c0 2024-04-25T14:12:58.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:58 smithi097 ceph-mon[32720]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 13 op/s 2024-04-25T14:12:58.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:58 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/291871372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:12:58.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:58 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/3658765910' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:12:58.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:58 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/838922960' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:12:58.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:58 smithi151 ceph-mon[32990]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 13 op/s 2024-04-25T14:12:58.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:58 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/291871372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:12:58.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:58 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/3658765910' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:12:58.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:58 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/838922960' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:12:59.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.012+0000 7f5c31fa8640 1 -- 172.21.15.96:0/2871493694 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5c2c0637a0 con 0x7f5c2c1080c0 2024-04-25T14:12:59.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.013+0000 7f5c28ff9640 1 -- 172.21.15.96:0/2871493694 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v78) ==== 100+0+11 (secure 0 0 0) 0x7f5c240a7c80 con 0x7f5c2c1080c0 2024-04-25T14:12:59.013 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-04-25T14:12:59.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.015+0000 7f5c31fa8640 1 -- 172.21.15.96:0/2871493694 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f5c140afb70 msgr2=0x7f5c140b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:59.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.015+0000 7f5c31fa8640 1 --2- 172.21.15.96:0/2871493694 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f5c140afb70 0x7f5c140b2080 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c005ab0 tx=0x7f5c1c031400 comp rx=0 tx=0).stop 2024-04-25T14:12:59.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.015+0000 7f5c31fa8640 1 -- 172.21.15.96:0/2871493694 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c2c1080c0 msgr2=0x7f5c2c1a16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:59.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.015+0000 7f5c31fa8640 1 --2- 172.21.15.96:0/2871493694 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c2c1080c0 0x7f5c2c1a16d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5c2400eaa0 tx=0x7f5c2400ef60 comp rx=0 tx=0).stop 2024-04-25T14:12:59.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.015+0000 7f5c31fa8640 1 -- 172.21.15.96:0/2871493694 shutdown_connections 2024-04-25T14:12:59.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.015+0000 7f5c31fa8640 1 --2- 172.21.15.96:0/2871493694 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f5c140afb70 0x7f5c140b2080 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:59.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.015+0000 7f5c31fa8640 1 --2- 172.21.15.96:0/2871493694 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c2c1080c0 0x7f5c2c1a16d0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:59.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.015+0000 7f5c31fa8640 1 --2- 172.21.15.96:0/2871493694 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c2c1033c0 0x7f5c2c19a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:59.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.015+0000 7f5c31fa8640 1 --2- 172.21.15.96:0/2871493694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c2c102180 0x7f5c2c19a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:59.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.015+0000 7f5c31fa8640 1 -- 172.21.15.96:0/2871493694 >> 172.21.15.96:0/2871493694 conn(0x7f5c2c0fd350 msgr2=0x7f5c2c1040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:59.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.015+0000 7f5c31fa8640 1 -- 172.21.15.96:0/2871493694 shutdown_connections 2024-04-25T14:12:59.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.015+0000 7f5c31fa8640 1 -- 172.21.15.96:0/2871493694 wait complete. 2024-04-25T14:12:59.026 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:12:59.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.133+0000 7f3d2d492640 1 -- 172.21.15.96:0/622038414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d28102fb0 msgr2=0x7f3d281033c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:59.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.133+0000 7f3d2d492640 1 --2- 172.21.15.96:0/622038414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d28102fb0 0x7f3d281033c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3d18009900 tx=0x7f3d1802a410 comp rx=0 tx=0).stop 2024-04-25T14:12:59.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.134+0000 7f3d2d492640 1 -- 172.21.15.96:0/622038414 shutdown_connections 2024-04-25T14:12:59.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.134+0000 7f3d2d492640 1 --2- 172.21.15.96:0/622038414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d281044d0 0x7f3d2810ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:59.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.134+0000 7f3d2d492640 1 --2- 172.21.15.96:0/622038414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d28103990 0x7f3d28103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:59.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.134+0000 7f3d2d492640 1 --2- 172.21.15.96:0/622038414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d28102fb0 0x7f3d281033c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:59.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.134+0000 7f3d2d492640 1 -- 172.21.15.96:0/622038414 >> 172.21.15.96:0/622038414 conn(0x7f3d280fd3b0 msgr2=0x7f3d280ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:59.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.134+0000 7f3d2d492640 1 -- 172.21.15.96:0/622038414 shutdown_connections 2024-04-25T14:12:59.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.134+0000 7f3d2d492640 1 -- 172.21.15.96:0/622038414 wait complete. 2024-04-25T14:12:59.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.134+0000 7f3d2d492640 1 Processor -- start 2024-04-25T14:12:59.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.134+0000 7f3d2d492640 1 -- start start 2024-04-25T14:12:59.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.134+0000 7f3d2d492640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d28102fb0 0x7f3d28102250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:59.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.134+0000 7f3d2d492640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d28103990 0x7f3d2819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:59.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.134+0000 7f3d2d492640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d281044d0 0x7f3d281a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:59.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.134+0000 7f3d2d492640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2810e4f0 con 0x7f3d28102fb0 2024-04-25T14:12:59.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.134+0000 7f3d2d492640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2810e370 con 0x7f3d28103990 2024-04-25T14:12:59.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.134+0000 7f3d2d492640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2810e670 con 0x7f3d281044d0 2024-04-25T14:12:59.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.134+0000 7f3d26ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d28102fb0 0x7f3d28102250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:59.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.134+0000 7f3d26ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d28102fb0 0x7f3d28102250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51702/0 (socket says 172.21.15.96:51702) 2024-04-25T14:12:59.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.135+0000 7f3d26ffd640 1 -- 172.21.15.96:0/3065982956 learned_addr learned my addr 172.21.15.96:0/3065982956 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:12:59.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.135+0000 7f3d1ffff640 1 --2- 172.21.15.96:0/3065982956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d28103990 0x7f3d2819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:59.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.135+0000 7f3d26ffd640 1 -- 172.21.15.96:0/3065982956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d281044d0 msgr2=0x7f3d281a1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:59.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.135+0000 7f3d26ffd640 1 --2- 172.21.15.96:0/3065982956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d281044d0 0x7f3d281a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:59.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.135+0000 7f3d26ffd640 1 -- 172.21.15.96:0/3065982956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d28103990 msgr2=0x7f3d2819efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:59.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.135+0000 7f3d26ffd640 1 --2- 172.21.15.96:0/3065982956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d28103990 0x7f3d2819efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:59.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.135+0000 7f3d26ffd640 1 -- 172.21.15.96:0/3065982956 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d281a1d20 con 0x7f3d28102fb0 2024-04-25T14:12:59.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.135+0000 7f3d1ffff640 1 --2- 172.21.15.96:0/3065982956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d28103990 0x7f3d2819efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:12:59.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.135+0000 7f3d26ffd640 1 --2- 172.21.15.96:0/3065982956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d28102fb0 0x7f3d28102250 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3d18006010 tx=0x7f3d18004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:59.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.135+0000 7f3d24ff9640 1 -- 172.21.15.96:0/3065982956 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1803f030 con 0x7f3d28102fb0 2024-04-25T14:12:59.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.135+0000 7f3d24ff9640 1 -- 172.21.15.96:0/3065982956 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d18003aa0 con 0x7f3d28102fb0 2024-04-25T14:12:59.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.135+0000 7f3d24ff9640 1 -- 172.21.15.96:0/3065982956 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d18035070 con 0x7f3d28102fb0 2024-04-25T14:12:59.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.135+0000 7f3d2d492640 1 -- 172.21.15.96:0/3065982956 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d281a1fb0 con 0x7f3d28102fb0 2024-04-25T14:12:59.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.135+0000 7f3d2d492640 1 -- 172.21.15.96:0/3065982956 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d281a2410 con 0x7f3d28102fb0 2024-04-25T14:12:59.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.136+0000 7f3d2d492640 1 -- 172.21.15.96:0/3065982956 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d28107950 con 0x7f3d28102fb0 2024-04-25T14:12:59.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.138+0000 7f3d24ff9640 1 -- 172.21.15.96:0/3065982956 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f3d18042030 con 0x7f3d28102fb0 2024-04-25T14:12:59.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.138+0000 7f3d24ff9640 1 --2- 172.21.15.96:0/3065982956 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f3d140afaf0 0x7f3d140b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:59.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.138+0000 7f3d1ffff640 1 --2- 172.21.15.96:0/3065982956 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f3d140afaf0 0x7f3d140b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:59.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.138+0000 7f3d24ff9640 1 -- 172.21.15.96:0/3065982956 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f3d180ff7f0 con 0x7f3d28102fb0 2024-04-25T14:12:59.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.139+0000 7f3d1ffff640 1 --2- 172.21.15.96:0/3065982956 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f3d140afaf0 0x7f3d140b2000 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3d10005fd0 tx=0x7f3d10005ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:59.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.142+0000 7f3d24ff9640 1 -- 172.21.15.96:0/3065982956 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3d180c83b0 con 0x7f3d28102fb0 2024-04-25T14:12:59.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.356+0000 7f3d2d492640 1 -- 172.21.15.96:0/3065982956 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3d2802dcb0 con 0x7f3d28102fb0 2024-04-25T14:12:59.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.357+0000 7f3d24ff9640 1 -- 172.21.15.96:0/3065982956 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11927 (secure 0 0 0) 0x7f3d180c7cb0 con 0x7f3d28102fb0 2024-04-25T14:12:59.357 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:12:59.357 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/606227917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":606227917},{"type":"v1","addr":"172.21.15.96:6835","nonce":606227917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/3265300710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3265300710},{"type":"v1","addr":"172.21.15.96:6837","nonce":3265300710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T14:11:28.886961+0000","modified":"2024-04-25T14:11:35.532775+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24517,"mds_2":24478,"mds_3":24455,"mds_4":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/4080308509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":4080308509},{"type":"v1","addr":"172.21.15.96:6839","nonce":4080308509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/1606748971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":1606748971},{"type":"v1","addr":"172.21.15.96:6841","nonce":1606748971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6833/1995324831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1995324831},{"type":"v1","addr":"172.21.15.97:6833","nonce":1995324831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":4,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.151:6833/1382721608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1382721608},{"type":"v1","addr":"172.21.15.151:6833","nonce":1382721608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6835/3872301726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3872301726},{"type":"v1","addr":"172.21.15.97:6835","nonce":3872301726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.151:6835/1313823638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":1313823638},{"type":"v1","addr":"172.21.15.151:6835","nonce":1313823638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/2691348366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2691348366},{"type":"v1","addr":"172.21.15.151:6837","nonce":2691348366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3306780081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3306780081},{"type":"v1","addr":"172.21.15.97:6837","nonce":3306780081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.151:6839/245699981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":245699981},{"type":"v1","addr":"172.21.15.151:6839","nonce":245699981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6839/2850442269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":2850442269},{"type":"v1","addr":"172.21.15.97:6839","nonce":2850442269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24517,24536,24478,24455,24427]},"id":1}]} 2024-04-25T14:12:59.357 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 25 2024-04-25T14:12:59.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.359+0000 7f3d2d492640 1 -- 172.21.15.96:0/3065982956 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f3d140afaf0 msgr2=0x7f3d140b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:59.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.359+0000 7f3d2d492640 1 --2- 172.21.15.96:0/3065982956 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f3d140afaf0 0x7f3d140b2000 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3d10005fd0 tx=0x7f3d10005ec0 comp rx=0 tx=0).stop 2024-04-25T14:12:59.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.359+0000 7f3d2d492640 1 -- 172.21.15.96:0/3065982956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d28102fb0 msgr2=0x7f3d28102250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:59.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.359+0000 7f3d2d492640 1 --2- 172.21.15.96:0/3065982956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d28102fb0 0x7f3d28102250 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3d18006010 tx=0x7f3d18004270 comp rx=0 tx=0).stop 2024-04-25T14:12:59.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.359+0000 7f3d2d492640 1 -- 172.21.15.96:0/3065982956 shutdown_connections 2024-04-25T14:12:59.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.359+0000 7f3d2d492640 1 --2- 172.21.15.96:0/3065982956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d281044d0 0x7f3d281a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:59.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.359+0000 7f3d2d492640 1 --2- 172.21.15.96:0/3065982956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d28103990 0x7f3d2819efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:59.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.359+0000 7f3d2d492640 1 --2- 172.21.15.96:0/3065982956 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f3d140afaf0 0x7f3d140b2000 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:59.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.359+0000 7f3d2d492640 1 --2- 172.21.15.96:0/3065982956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d28102fb0 0x7f3d28102250 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:59.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.359+0000 7f3d2d492640 1 -- 172.21.15.96:0/3065982956 >> 172.21.15.96:0/3065982956 conn(0x7f3d280fd3b0 msgr2=0x7f3d28104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:59.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.359+0000 7f3d2d492640 1 -- 172.21.15.96:0/3065982956 shutdown_connections 2024-04-25T14:12:59.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.359+0000 7f3d2d492640 1 -- 172.21.15.96:0/3065982956 wait complete. 2024-04-25T14:12:59.371 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:12:59.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.478+0000 7fa76feaf640 1 -- 172.21.15.96:0/1235569129 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7681050a0 msgr2=0x7fa7681054b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:59.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.478+0000 7fa76feaf640 1 --2- 172.21.15.96:0/1235569129 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7681050a0 0x7fa7681054b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa764009900 tx=0x7fa76402a410 comp rx=0 tx=0).stop 2024-04-25T14:12:59.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.478+0000 7fa76feaf640 1 -- 172.21.15.96:0/1235569129 shutdown_connections 2024-04-25T14:12:59.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.478+0000 7fa76feaf640 1 --2- 172.21.15.96:0/1235569129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa768102250 0x7fa76810bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:59.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.478+0000 7fa76feaf640 1 --2- 172.21.15.96:0/1235569129 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa768101750 0x7fa768101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:59.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.478+0000 7fa76feaf640 1 --2- 172.21.15.96:0/1235569129 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7681050a0 0x7fa7681054b0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:59.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.478+0000 7fa76feaf640 1 -- 172.21.15.96:0/1235569129 >> 172.21.15.96:0/1235569129 conn(0x7fa7680fd370 msgr2=0x7fa7680ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:59.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.478+0000 7fa76feaf640 1 -- 172.21.15.96:0/1235569129 shutdown_connections 2024-04-25T14:12:59.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.478+0000 7fa76feaf640 1 -- 172.21.15.96:0/1235569129 wait complete. 2024-04-25T14:12:59.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.479+0000 7fa76feaf640 1 Processor -- start 2024-04-25T14:12:59.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.479+0000 7fa76feaf640 1 -- start start 2024-04-25T14:12:59.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.479+0000 7fa76feaf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa768101750 0x7fa76819a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:59.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.479+0000 7fa76feaf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa768102250 0x7fa76819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:59.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.479+0000 7fa76feaf640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa7681050a0 0x7fa7681a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:59.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.479+0000 7fa76feaf640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa76810f670 con 0x7fa768102250 2024-04-25T14:12:59.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.479+0000 7fa76feaf640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa76810f4f0 con 0x7fa768101750 2024-04-25T14:12:59.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.479+0000 7fa76feaf640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa76810f7f0 con 0x7fa7681050a0 2024-04-25T14:12:59.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.479+0000 7fa76d44a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa768102250 0x7fa76819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:59.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.479+0000 7fa76d44a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa768102250 0x7fa76819a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51726/0 (socket says 172.21.15.96:51726) 2024-04-25T14:12:59.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.479+0000 7fa76d44a640 1 -- 172.21.15.96:0/2794028549 learned_addr learned my addr 172.21.15.96:0/2794028549 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:12:59.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.479+0000 7fa76d44a640 1 -- 172.21.15.96:0/2794028549 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa7681050a0 msgr2=0x7fa7681a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:59.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.479+0000 7fa76dc4b640 1 --2- 172.21.15.96:0/2794028549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa768101750 0x7fa76819a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:59.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.479+0000 7fa76d44a640 1 --2- 172.21.15.96:0/2794028549 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa7681050a0 0x7fa7681a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:59.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.479+0000 7fa76d44a640 1 -- 172.21.15.96:0/2794028549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa768101750 msgr2=0x7fa76819a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:59.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.479+0000 7fa76d44a640 1 --2- 172.21.15.96:0/2794028549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa768101750 0x7fa76819a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:59.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.479+0000 7fa76d44a640 1 -- 172.21.15.96:0/2794028549 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7681a1cc0 con 0x7fa768102250 2024-04-25T14:12:59.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.479+0000 7fa76dc4b640 1 --2- 172.21.15.96:0/2794028549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa768101750 0x7fa76819a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:12:59.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.479+0000 7fa76d44a640 1 --2- 172.21.15.96:0/2794028549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa768102250 0x7fa76819a770 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa75800c920 tx=0x7fa75800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:59.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.479+0000 7fa75effd640 1 -- 172.21.15.96:0/2794028549 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa758007c70 con 0x7fa768102250 2024-04-25T14:12:59.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.479+0000 7fa75effd640 1 -- 172.21.15.96:0/2794028549 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa758007e10 con 0x7fa768102250 2024-04-25T14:12:59.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.479+0000 7fa75effd640 1 -- 172.21.15.96:0/2794028549 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa758018820 con 0x7fa768102250 2024-04-25T14:12:59.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.480+0000 7fa76feaf640 1 -- 172.21.15.96:0/2794028549 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7681a1fb0 con 0x7fa768102250 2024-04-25T14:12:59.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.480+0000 7fa76feaf640 1 -- 172.21.15.96:0/2794028549 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7681a24a0 con 0x7fa768102250 2024-04-25T14:12:59.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.480+0000 7fa76feaf640 1 -- 172.21.15.96:0/2794028549 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa768108a30 con 0x7fa768102250 2024-04-25T14:12:59.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.482+0000 7fa75effd640 1 -- 172.21.15.96:0/2794028549 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fa75800f450 con 0x7fa768102250 2024-04-25T14:12:59.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.482+0000 7fa75effd640 1 --2- 172.21.15.96:0/2794028549 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa7540afaf0 0x7fa7540b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:59.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.483+0000 7fa76dc4b640 1 --2- 172.21.15.96:0/2794028549 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa7540afaf0 0x7fa7540b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:59.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.483+0000 7fa75effd640 1 -- 172.21.15.96:0/2794028549 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fa7580df0e0 con 0x7fa768102250 2024-04-25T14:12:59.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.483+0000 7fa76dc4b640 1 --2- 172.21.15.96:0/2794028549 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa7540afaf0 0x7fa7540b2000 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa764005f70 tx=0x7fa764000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:59.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.486+0000 7fa75effd640 1 -- 172.21.15.96:0/2794028549 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa758014030 con 0x7fa768102250 2024-04-25T14:12:59.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.670+0000 7fa76feaf640 1 -- 172.21.15.96:0/2794028549 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa76802dd20 con 0x7fa768102250 2024-04-25T14:12:59.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.672+0000 7fa75effd640 1 -- 172.21.15.96:0/2794028549 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22616 (secure 0 0 0) 0x7fa7580a7d30 con 0x7fa768102250 2024-04-25T14:12:59.672 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:12:59.672 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":78,"fsid":"59334182-030c-11ef-bc93-c7b262605968","created":"2024-04-25T14:03:25.680745+0000","modified":"2024-04-25T14:12:22.496143+0000","last_up_change":"2024-04-25T14:10:35.894312+0000","last_in_change":"2024-04-25T14:10:15.583440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:06:47.402927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:11:27.434331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:11:27.854004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cf4ba3ac-142e-4e8e-96d4-410d77157eb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6803","nonce":3832506168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6805","nonce":3832506168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832506168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6807","nonce":3832506168}]},"public_addr":"172.21.15.96:6803/3832506168","cluster_addr":"172.21.15.96:6805/3832506168","heartbeat_back_addr":"172.21.15.96:6809/3832506168","heartbeat_front_addr":"172.21.15.96:6807/3832506168","state":["exists","up"]},{"osd":1,"uuid":"866e73b9-4a26-48ba-badf-704fe22f71cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6801","nonce":1083311494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6803","nonce":1083311494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6807","nonce":1083311494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6805","nonce":1083311494}]},"public_addr":"172.21.15.97:6801/1083311494","cluster_addr":"172.21.15.97:6803/1083311494","heartbeat_back_addr":"172.21.15.97:6807/1083311494","heartbeat_front_addr":"172.21.15.97:6805/1083311494","state":["exists","up"]},{"osd":2,"uuid":"45c79ecb-abc4-4dba-bae8-74804e4bc1bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6801","nonce":981902966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6803","nonce":981902966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6807","nonce":981902966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6805","nonce":981902966}]},"public_addr":"172.21.15.151:6801/981902966","cluster_addr":"172.21.15.151:6803/981902966","heartbeat_back_addr":"172.21.15.151:6807/981902966","heartbeat_front_addr":"172.21.15.151:6805/981902966","state":["exists","up"]},{"osd":3,"uuid":"9fc39468-a7ea-4ccf-b365-335ad2ea4c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6811","nonce":2274158748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6813","nonce":2274158748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6817","nonce":2274158748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6815","nonce":2274158748}]},"public_addr":"172.21.15.96:6811/2274158748","cluster_addr":"172.21.15.96:6813/2274158748","heartbeat_back_addr":"172.21.15.96:6817/2274158748","heartbeat_front_addr":"172.21.15.96:6815/2274158748","state":["exists","up"]},{"osd":4,"uuid":"616582f1-12d2-456b-b7b3-e5517800e8a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6809","nonce":3727031962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6811","nonce":3727031962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6815","nonce":3727031962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6813","nonce":3727031962}]},"public_addr":"172.21.15.97:6809/3727031962","cluster_addr":"172.21.15.97:6811/3727031962","heartbeat_back_addr":"172.21.15.97:6815/3727031962","heartbeat_front_addr":"172.21.15.97:6813/3727031962","state":["exists","up"]},{"osd":5,"uuid":"3d277784-ab0f-4c90-81a1-2a4034c52a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6809","nonce":223882546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6811","nonce":223882546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6815","nonce":223882546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6813","nonce":223882546}]},"public_addr":"172.21.15.151:6809/223882546","cluster_addr":"172.21.15.151:6811/223882546","heartbeat_back_addr":"172.21.15.151:6815/223882546","heartbeat_front_addr":"172.21.15.151:6813/223882546","state":["exists","up"]},{"osd":6,"uuid":"9a9ad585-cb1a-4239-9d18-055f59164049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6819","nonce":96197407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6821","nonce":96197407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6825","nonce":96197407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6823","nonce":96197407}]},"public_addr":"172.21.15.96:6819/96197407","cluster_addr":"172.21.15.96:6821/96197407","heartbeat_back_addr":"172.21.15.96:6825/96197407","heartbeat_front_addr":"172.21.15.96:6823/96197407","state":["exists","up"]},{"osd":7,"uuid":"303551c8-6af0-42f2-844f-45bf8db5baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6817","nonce":2247216400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6819","nonce":2247216400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6823","nonce":2247216400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6821","nonce":2247216400}]},"public_addr":"172.21.15.97:6817/2247216400","cluster_addr":"172.21.15.97:6819/2247216400","heartbeat_back_addr":"172.21.15.97:6823/2247216400","heartbeat_front_addr":"172.21.15.97:6821/2247216400","state":["exists","up"]},{"osd":8,"uuid":"81f6cc66-e4ca-430c-b336-b6678d968818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6817","nonce":2549822185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6819","nonce":2549822185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6823","nonce":2549822185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6821","nonce":2549822185}]},"public_addr":"172.21.15.151:6817/2549822185","cluster_addr":"172.21.15.151:6819/2549822185","heartbeat_back_addr":"172.21.15.151:6823/2549822185","heartbeat_front_addr":"172.21.15.151:6821/2549822185","state":["exists","up"]},{"osd":9,"uuid":"29d4da16-feab-4d1c-9d1d-de4f799bd724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6827","nonce":1099399643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6829","nonce":1099399643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6833","nonce":1099399643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6831","nonce":1099399643}]},"public_addr":"172.21.15.96:6827/1099399643","cluster_addr":"172.21.15.96:6829/1099399643","heartbeat_back_addr":"172.21.15.96:6833/1099399643","heartbeat_front_addr":"172.21.15.96:6831/1099399643","state":["exists","up"]},{"osd":10,"uuid":"197fee9b-eee4-4893-be1c-3d57f42250e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6825","nonce":155754520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6827","nonce":155754520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6831","nonce":155754520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6829","nonce":155754520}]},"public_addr":"172.21.15.97:6825/155754520","cluster_addr":"172.21.15.97:6827/155754520","heartbeat_back_addr":"172.21.15.97:6831/155754520","heartbeat_front_addr":"172.21.15.97:6829/155754520","state":["exists","up"]},{"osd":11,"uuid":"9c1d0a83-fc12-41a2-8eb0-80c4abdd2770","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6825","nonce":2027958746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6827","nonce":2027958746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6831","nonce":2027958746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6829","nonce":2027958746}]},"public_addr":"172.21.15.151:6825/2027958746","cluster_addr":"172.21.15.151:6827/2027958746","heartbeat_back_addr":"172.21.15.151:6831/2027958746","heartbeat_front_addr":"172.21.15.151:6829/2027958746","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:05:56.177390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:06:19.917663+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:06:43.608052+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:08.540382+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:33.474072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:58.242306+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:08:24.120442+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:08:49.661064+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:09:15.097271+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:09:41.904505+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:10:08.233252+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:10:33.712296+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/3839807484":"2024-04-26T14:04:21.307574+0000","172.21.15.96:0/1495237551":"2024-04-26T14:04:21.307574+0000","172.21.15.96:0/2793751450":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/129575834":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/2233075139":"2024-04-26T14:04:21.307574+0000","172.21.15.96:6801/2348397481":"2024-04-26T14:03:54.249537+0000","172.21.15.96:6801/433591062":"2024-04-26T14:04:21.307574+0000","172.21.15.96:6800/2348397481":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/3517674924":"2024-04-26T14:03:54.249537+0000","172.21.15.96:6800/433591062":"2024-04-26T14:04:21.307574+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:12:59.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.674+0000 7fa76feaf640 1 -- 172.21.15.96:0/2794028549 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa7540afaf0 msgr2=0x7fa7540b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:59.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.674+0000 7fa76feaf640 1 --2- 172.21.15.96:0/2794028549 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa7540afaf0 0x7fa7540b2000 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa764005f70 tx=0x7fa764000bc0 comp rx=0 tx=0).stop 2024-04-25T14:12:59.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.674+0000 7fa76feaf640 1 -- 172.21.15.96:0/2794028549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa768102250 msgr2=0x7fa76819a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:59.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.674+0000 7fa76feaf640 1 --2- 172.21.15.96:0/2794028549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa768102250 0x7fa76819a770 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa75800c920 tx=0x7fa75800cde0 comp rx=0 tx=0).stop 2024-04-25T14:12:59.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.674+0000 7fa76feaf640 1 -- 172.21.15.96:0/2794028549 shutdown_connections 2024-04-25T14:12:59.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.674+0000 7fa76feaf640 1 --2- 172.21.15.96:0/2794028549 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa7681050a0 0x7fa7681a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:59.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.674+0000 7fa76feaf640 1 --2- 172.21.15.96:0/2794028549 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa7540afaf0 0x7fa7540b2000 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:59.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.674+0000 7fa76feaf640 1 --2- 172.21.15.96:0/2794028549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa768102250 0x7fa76819a770 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:59.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.674+0000 7fa76feaf640 1 --2- 172.21.15.96:0/2794028549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa768101750 0x7fa76819a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:59.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.674+0000 7fa76feaf640 1 -- 172.21.15.96:0/2794028549 >> 172.21.15.96:0/2794028549 conn(0x7fa7680fd370 msgr2=0x7fa768105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:59.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.674+0000 7fa76feaf640 1 -- 172.21.15.96:0/2794028549 shutdown_connections 2024-04-25T14:12:59.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.674+0000 7fa76feaf640 1 -- 172.21.15.96:0/2794028549 wait complete. 2024-04-25T14:12:59.684 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:59 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/910227700' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:12:59.684 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:59 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/2871493694' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:12:59.684 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:12:59 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/3065982956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:12:59.686 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:12:59.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.794+0000 7f560934e640 1 -- 172.21.15.96:0/888740274 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5604103c00 msgr2=0x7f5604104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:59.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.794+0000 7f560934e640 1 --2- 172.21.15.96:0/888740274 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5604103c00 0x7f5604104010 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f55ec009900 tx=0x7f55ec02a410 comp rx=0 tx=0).stop 2024-04-25T14:12:59.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.794+0000 7f560934e640 1 -- 172.21.15.96:0/888740274 shutdown_connections 2024-04-25T14:12:59.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.794+0000 7f560934e640 1 --2- 172.21.15.96:0/888740274 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5604108900 0x7f560410ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:59.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.794+0000 7f560934e640 1 --2- 172.21.15.96:0/888740274 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5604103c00 0x7f5604104010 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:59.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.794+0000 7f560934e640 1 --2- 172.21.15.96:0/888740274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56041029c0 0x7f56041036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:59.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.794+0000 7f560934e640 1 -- 172.21.15.96:0/888740274 >> 172.21.15.96:0/888740274 conn(0x7f56040fdbd0 msgr2=0x7f5604100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:59.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.794+0000 7f560934e640 1 -- 172.21.15.96:0/888740274 shutdown_connections 2024-04-25T14:12:59.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.794+0000 7f560934e640 1 -- 172.21.15.96:0/888740274 wait complete. 2024-04-25T14:12:59.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.795+0000 7f560934e640 1 Processor -- start 2024-04-25T14:12:59.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.795+0000 7f560934e640 1 -- start start 2024-04-25T14:12:59.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.795+0000 7f560934e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56041029c0 0x7f560419a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:59.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.795+0000 7f560934e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5604103c00 0x7f560419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:59.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.795+0000 7f560934e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5604108900 0x7f56041a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:59.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.795+0000 7f560934e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f560410e4c0 con 0x7f5604108900 2024-04-25T14:12:59.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.795+0000 7f560934e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f560410e340 con 0x7f5604103c00 2024-04-25T14:12:59.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.795+0000 7f560934e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f560410e640 con 0x7f56041029c0 2024-04-25T14:12:59.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.795+0000 7f56037fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5604108900 0x7f56041a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:59.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.795+0000 7f56037fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5604108900 0x7f56041a16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51736/0 (socket says 172.21.15.96:51736) 2024-04-25T14:12:59.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.795+0000 7f56037fe640 1 -- 172.21.15.96:0/655939224 learned_addr learned my addr 172.21.15.96:0/655939224 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:12:59.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.796+0000 7f55fbfff640 1 --2- 172.21.15.96:0/655939224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5604103c00 0x7f560419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:59.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.796+0000 7f5602ffd640 1 --2- 172.21.15.96:0/655939224 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56041029c0 0x7f560419a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:59.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.796+0000 7f56037fe640 1 -- 172.21.15.96:0/655939224 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56041029c0 msgr2=0x7f560419a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:59.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.796+0000 7f56037fe640 1 --2- 172.21.15.96:0/655939224 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56041029c0 0x7f560419a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:59.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.796+0000 7f56037fe640 1 -- 172.21.15.96:0/655939224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5604103c00 msgr2=0x7f560419a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:59.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.796+0000 7f56037fe640 1 --2- 172.21.15.96:0/655939224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5604103c00 0x7f560419a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:59.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.796+0000 7f56037fe640 1 -- 172.21.15.96:0/655939224 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56041a1ca0 con 0x7f5604108900 2024-04-25T14:12:59.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.796+0000 7f56037fe640 1 --2- 172.21.15.96:0/655939224 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5604108900 0x7f56041a16e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f55fc00cce0 tx=0x7f55fc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:59.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.796+0000 7f5600ff9640 1 -- 172.21.15.96:0/655939224 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc00f450 con 0x7f5604108900 2024-04-25T14:12:59.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.796+0000 7f5600ff9640 1 -- 172.21.15.96:0/655939224 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55fc00faa0 con 0x7f5604108900 2024-04-25T14:12:59.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.796+0000 7f560934e640 1 -- 172.21.15.96:0/655939224 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56041a1f90 con 0x7f5604108900 2024-04-25T14:12:59.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.796+0000 7f5600ff9640 1 -- 172.21.15.96:0/655939224 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc018b70 con 0x7f5604108900 2024-04-25T14:12:59.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.796+0000 7f560934e640 1 -- 172.21.15.96:0/655939224 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56041a23f0 con 0x7f5604108900 2024-04-25T14:12:59.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.797+0000 7f560934e640 1 -- 172.21.15.96:0/655939224 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56041a26b0 con 0x7f5604108900 2024-04-25T14:12:59.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.799+0000 7f5600ff9640 1 -- 172.21.15.96:0/655939224 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f55fc018d10 con 0x7f5604108900 2024-04-25T14:12:59.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.799+0000 7f5600ff9640 1 --2- 172.21.15.96:0/655939224 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f55f00afbf0 0x7f55f00b2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:59.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.799+0000 7f5602ffd640 1 --2- 172.21.15.96:0/655939224 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f55f00afbf0 0x7f55f00b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:59.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.799+0000 7f5600ff9640 1 -- 172.21.15.96:0/655939224 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f55fc0df1e0 con 0x7f5604108900 2024-04-25T14:12:59.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.800+0000 7f5602ffd640 1 --2- 172.21.15.96:0/655939224 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f55f00afbf0 0x7f55f00b2100 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f56041035c0 tx=0x7f55f4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:59.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:12:59.803+0000 7f5600ff9640 1 -- 172.21.15.96:0/655939224 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f55fc014030 con 0x7f5604108900 2024-04-25T14:12:59.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:59 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/910227700' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:12:59.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:59 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/2871493694' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:12:59.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:12:59 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/3065982956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:12:59.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:59 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/910227700' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:12:59.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:59 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/2871493694' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:12:59.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:12:59 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/3065982956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:13:00.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.015+0000 7f560934e640 1 -- 172.21.15.96:0/655939224 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56041a2880 con 0x7f5604108900 2024-04-25T14:13:00.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.016+0000 7f5600ff9640 1 -- 172.21.15.96:0/655939224 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11927 (secure 0 0 0) 0x7f55fc0a7e50 con 0x7f5604108900 2024-04-25T14:13:00.016 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:13:00.016 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/606227917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":606227917},{"type":"v1","addr":"172.21.15.96:6835","nonce":606227917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/3265300710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3265300710},{"type":"v1","addr":"172.21.15.96:6837","nonce":3265300710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T14:11:28.886961+0000","modified":"2024-04-25T14:11:35.532775+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24517,"mds_2":24478,"mds_3":24455,"mds_4":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/4080308509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":4080308509},{"type":"v1","addr":"172.21.15.96:6839","nonce":4080308509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/1606748971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":1606748971},{"type":"v1","addr":"172.21.15.96:6841","nonce":1606748971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6833/1995324831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1995324831},{"type":"v1","addr":"172.21.15.97:6833","nonce":1995324831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":4,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.151:6833/1382721608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1382721608},{"type":"v1","addr":"172.21.15.151:6833","nonce":1382721608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6835/3872301726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3872301726},{"type":"v1","addr":"172.21.15.97:6835","nonce":3872301726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.151:6835/1313823638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":1313823638},{"type":"v1","addr":"172.21.15.151:6835","nonce":1313823638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/2691348366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2691348366},{"type":"v1","addr":"172.21.15.151:6837","nonce":2691348366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3306780081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3306780081},{"type":"v1","addr":"172.21.15.97:6837","nonce":3306780081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.151:6839/245699981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":245699981},{"type":"v1","addr":"172.21.15.151:6839","nonce":245699981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6839/2850442269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":2850442269},{"type":"v1","addr":"172.21.15.97:6839","nonce":2850442269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24517,24536,24478,24455,24427]},"id":1}]} 2024-04-25T14:13:00.016 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 25 2024-04-25T14:13:00.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.018+0000 7f560934e640 1 -- 172.21.15.96:0/655939224 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f55f00afbf0 msgr2=0x7f55f00b2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:00.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.018+0000 7f560934e640 1 --2- 172.21.15.96:0/655939224 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f55f00afbf0 0x7f55f00b2100 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f56041035c0 tx=0x7f55f4005f90 comp rx=0 tx=0).stop 2024-04-25T14:13:00.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.018+0000 7f560934e640 1 -- 172.21.15.96:0/655939224 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5604108900 msgr2=0x7f56041a16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:00.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.018+0000 7f560934e640 1 --2- 172.21.15.96:0/655939224 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5604108900 0x7f56041a16e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f55fc00cce0 tx=0x7f55fc00a620 comp rx=0 tx=0).stop 2024-04-25T14:13:00.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.018+0000 7f560934e640 1 -- 172.21.15.96:0/655939224 shutdown_connections 2024-04-25T14:13:00.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.018+0000 7f560934e640 1 --2- 172.21.15.96:0/655939224 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f55f00afbf0 0x7f55f00b2100 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.018+0000 7f560934e640 1 --2- 172.21.15.96:0/655939224 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5604108900 0x7f56041a16e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.018+0000 7f560934e640 1 --2- 172.21.15.96:0/655939224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5604103c00 0x7f560419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.018+0000 7f560934e640 1 --2- 172.21.15.96:0/655939224 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56041029c0 0x7f560419a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.018+0000 7f560934e640 1 -- 172.21.15.96:0/655939224 >> 172.21.15.96:0/655939224 conn(0x7f56040fdbd0 msgr2=0x7f5604109e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:00.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.018+0000 7f560934e640 1 -- 172.21.15.96:0/655939224 shutdown_connections 2024-04-25T14:13:00.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.018+0000 7f560934e640 1 -- 172.21.15.96:0/655939224 wait complete. 2024-04-25T14:13:00.030 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T14:11:28.886961+0000', 'modified': '2024-04-25T14:11:35.532775+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24536, 'mds_1': 24517, 'mds_2': 24478, 'mds_3': 24455, 'mds_4': 24427}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14640': {'gid': 14640, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.96:6839/4080308509', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6838', 'nonce': 4080308509}, {'type': 'v1', 'addr': '172.21.15.96:6839', 'nonce': 4080308509}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14652': {'gid': 14652, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.96:6841/1606748971', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6840', 'nonce': 1606748971}, {'type': 'v1', 'addr': '172.21.15.96:6841', 'nonce': 1606748971}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24419': {'gid': 24419, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.97:6833/1995324831', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6832', 'nonce': 1995324831}, {'type': 'v1', 'addr': '172.21.15.97:6833', 'nonce': 1995324831}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24427': {'gid': 24427, 'name': 'l', 'rank': 4, 'incarnation': 23, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.151:6833/1382721608', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6832', 'nonce': 1382721608}, {'type': 'v1', 'addr': '172.21.15.151:6833', 'nonce': 1382721608}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'e', 'rank': 3, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.97:6835/3872301726', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6834', 'nonce': 3872301726}, {'type': 'v1', 'addr': '172.21.15.97:6835', 'nonce': 3872301726}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'f', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.151:6835/1313823638', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6834', 'nonce': 1313823638}, {'type': 'v1', 'addr': '172.21.15.151:6835', 'nonce': 1313823638}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24490': {'gid': 24490, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.151:6837/2691348366', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6836', 'nonce': 2691348366}, {'type': 'v1', 'addr': '172.21.15.151:6837', 'nonce': 2691348366}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24497': {'gid': 24497, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.97:6837/3306780081', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6836', 'nonce': 3306780081}, {'type': 'v1', 'addr': '172.21.15.97:6837', 'nonce': 3306780081}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24517': {'gid': 24517, 'name': 'i', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.151:6839/245699981', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6838', 'nonce': 245699981}, {'type': 'v1', 'addr': '172.21.15.151:6839', 'nonce': 245699981}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24536': {'gid': 24536, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.97:6839/2850442269', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6838', 'nonce': 2850442269}, {'type': 'v1', 'addr': '172.21.15.97:6839', 'nonce': 2850442269}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24536, 'qdb_cluster': [24517, 24536, 24478, 24455, 24427]} 2024-04-25T14:13:00.030 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-25T14:13:00.031 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2024-04-25T14:13:00.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.139+0000 7f360794d640 1 -- 172.21.15.96:0/3911376947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3600102260 msgr2=0x7f360010bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:00.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.139+0000 7f360794d640 1 --2- 172.21.15.96:0/3911376947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3600102260 0x7f360010bcc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f35f0009900 tx=0x7f35f002a410 comp rx=0 tx=0).stop 2024-04-25T14:13:00.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.139+0000 7f360794d640 1 -- 172.21.15.96:0/3911376947 shutdown_connections 2024-04-25T14:13:00.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.139+0000 7f360794d640 1 --2- 172.21.15.96:0/3911376947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3600102260 0x7f360010bcc0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.139+0000 7f360794d640 1 --2- 172.21.15.96:0/3911376947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3600101760 0x7f3600101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.139+0000 7f360794d640 1 --2- 172.21.15.96:0/3911376947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f36001050b0 0x7f36001054c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.139+0000 7f360794d640 1 -- 172.21.15.96:0/3911376947 >> 172.21.15.96:0/3911376947 conn(0x7f36000fd380 msgr2=0x7f36000ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:00.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.139+0000 7f360794d640 1 -- 172.21.15.96:0/3911376947 shutdown_connections 2024-04-25T14:13:00.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.139+0000 7f360794d640 1 -- 172.21.15.96:0/3911376947 wait complete. 2024-04-25T14:13:00.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.140+0000 7f360794d640 1 Processor -- start 2024-04-25T14:13:00.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.140+0000 7f360794d640 1 -- start start 2024-04-25T14:13:00.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.140+0000 7f360794d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3600101760 0x7f360019a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.140+0000 7f360794d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3600102260 0x7f360019a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.140+0000 7f360794d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f36001050b0 0x7f36001a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.140+0000 7f360794d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f360010f640 con 0x7f3600102260 2024-04-25T14:13:00.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.140+0000 7f360794d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f360010f4c0 con 0x7f3600101760 2024-04-25T14:13:00.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.140+0000 7f360794d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f360010f7c0 con 0x7f36001050b0 2024-04-25T14:13:00.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.140+0000 7f3604ee8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3600102260 0x7f360019a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.140+0000 7f3604ee8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3600102260 0x7f360019a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51748/0 (socket says 172.21.15.96:51748) 2024-04-25T14:13:00.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.140+0000 7f3604ee8640 1 -- 172.21.15.96:0/2002599194 learned_addr learned my addr 172.21.15.96:0/2002599194 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:00.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.140+0000 7f3605eea640 1 --2- 172.21.15.96:0/2002599194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f36001050b0 0x7f36001a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.140+0000 7f36056e9640 1 --2- 172.21.15.96:0/2002599194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3600101760 0x7f360019a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.140+0000 7f3604ee8640 1 -- 172.21.15.96:0/2002599194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f36001050b0 msgr2=0x7f36001a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:00.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.140+0000 7f3604ee8640 1 --2- 172.21.15.96:0/2002599194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f36001050b0 0x7f36001a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.140+0000 7f3604ee8640 1 -- 172.21.15.96:0/2002599194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3600101760 msgr2=0x7f360019a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:00.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.140+0000 7f3604ee8640 1 --2- 172.21.15.96:0/2002599194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3600101760 0x7f360019a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.140+0000 7f3604ee8640 1 -- 172.21.15.96:0/2002599194 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36001a1d00 con 0x7f3600102260 2024-04-25T14:13:00.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.140+0000 7f3605eea640 1 --2- 172.21.15.96:0/2002599194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f36001050b0 0x7f36001a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:00.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.140+0000 7f36056e9640 1 --2- 172.21.15.96:0/2002599194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3600101760 0x7f360019a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:00.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.141+0000 7f3604ee8640 1 --2- 172.21.15.96:0/2002599194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3600102260 0x7f360019a7b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f35f400cce0 tx=0x7f35f400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:00.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.141+0000 7f35fe7fc640 1 -- 172.21.15.96:0/2002599194 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f400f650 con 0x7f3600102260 2024-04-25T14:13:00.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.141+0000 7f35fe7fc640 1 -- 172.21.15.96:0/2002599194 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35f400fca0 con 0x7f3600102260 2024-04-25T14:13:00.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.141+0000 7f360794d640 1 -- 172.21.15.96:0/2002599194 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36001a1ff0 con 0x7f3600102260 2024-04-25T14:13:00.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.141+0000 7f35fe7fc640 1 -- 172.21.15.96:0/2002599194 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f4018b40 con 0x7f3600102260 2024-04-25T14:13:00.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.141+0000 7f360794d640 1 -- 172.21.15.96:0/2002599194 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36001a2460 con 0x7f3600102260 2024-04-25T14:13:00.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.143+0000 7f35fe7fc640 1 -- 172.21.15.96:0/2002599194 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f35f400f7f0 con 0x7f3600102260 2024-04-25T14:13:00.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.143+0000 7f360e7fe740 1 Processor -- start 2024-04-25T14:13:00.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.143+0000 7f360e7fe740 1 -- start start 2024-04-25T14:13:00.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.143+0000 7f360e7fe740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e1736d270 0x560e1770e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.144+0000 7f360e7fe740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e173443b0 0x560e1770e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.144+0000 7f360e7fe740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560e1734c590 0x560e177157f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.144+0000 7f360e7fe740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560e17673740 con 0x560e173443b0 2024-04-25T14:13:00.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.144+0000 7f360e7fe740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560e1760f3a0 con 0x560e1734c590 2024-04-25T14:13:00.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.144+0000 7f360e7fe740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560e1766f700 con 0x560e1736d270 2024-04-25T14:13:00.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.144+0000 7f36056e9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e173443b0 0x560e1770e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.144+0000 7f36056e9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e173443b0 0x560e1770e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51754/0 (socket says 172.21.15.96:51754) 2024-04-25T14:13:00.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.144+0000 7f36056e9640 1 -- 172.21.15.96:0/390400316 learned_addr learned my addr 172.21.15.96:0/390400316 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:00.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.144+0000 7f3604ee8640 1 --2- 172.21.15.96:0/390400316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560e1734c590 0x560e177157f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.144+0000 7f3605eea640 1 --2- 172.21.15.96:0/390400316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e1736d270 0x560e1770e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.144+0000 7f35fe7fc640 1 --2- 172.21.15.96:0/2002599194 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f35e80afaf0 0x7f35e80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.144+0000 7f36056e9640 1 -- 172.21.15.96:0/390400316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e1736d270 msgr2=0x560e1770e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:00.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.144+0000 7f36056e9640 1 --2- 172.21.15.96:0/390400316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e1736d270 0x560e1770e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.144+0000 7f36056e9640 1 -- 172.21.15.96:0/390400316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560e1734c590 msgr2=0x560e177157f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:00.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.144+0000 7f36056e9640 1 --2- 172.21.15.96:0/390400316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560e1734c590 0x560e177157f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.144+0000 7f36056e9640 1 -- 172.21.15.96:0/390400316 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e1762b750 con 0x560e173443b0 2024-04-25T14:13:00.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.144+0000 7f3605eea640 1 --2- 172.21.15.96:0/2002599194 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f35e80afaf0 0x7f35e80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.144+0000 7f35fe7fc640 1 -- 172.21.15.96:0/2002599194 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f35f40df210 con 0x7f3600102260 2024-04-25T14:13:00.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.144+0000 7f36056e9640 1 --2- 172.21.15.96:0/390400316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e173443b0 0x560e1770e8f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f35f800ba90 tx=0x7f35f800bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a233a3364f74832d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:00.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.144+0000 7f35e77fe640 1 -- 172.21.15.96:0/390400316 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f8017420 con 0x560e173443b0 2024-04-25T14:13:00.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.144+0000 7f35e77fe640 1 -- 172.21.15.96:0/390400316 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35f8017a70 con 0x560e173443b0 2024-04-25T14:13:00.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.144+0000 7f35e77fe640 1 -- 172.21.15.96:0/390400316 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f800f9f0 con 0x560e173443b0 2024-04-25T14:13:00.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.145+0000 7f360e7fe740 1 -- 172.21.15.96:0/390400316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e173443b0 msgr2=0x560e1770e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:00.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.145+0000 7f3605eea640 1 --2- 172.21.15.96:0/2002599194 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f35e80afaf0 0x7f35e80b2000 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x560e1770f540 tx=0x7f35f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:00.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.145+0000 7f360e7fe740 1 --2- 172.21.15.96:0/390400316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e173443b0 0x560e1770e8f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f35f800ba90 tx=0x7f35f800bf50 comp rx=0 tx=0).stop 2024-04-25T14:13:00.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.145+0000 7f360e7fe740 1 -- 172.21.15.96:0/390400316 shutdown_connections 2024-04-25T14:13:00.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.145+0000 7f360e7fe740 1 --2- 172.21.15.96:0/390400316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560e1734c590 0x560e177157f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.145+0000 7f360e7fe740 1 --2- 172.21.15.96:0/390400316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e173443b0 0x560e1770e8f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.145+0000 7f360e7fe740 1 --2- 172.21.15.96:0/390400316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e1736d270 0x560e1770e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.145+0000 7f360e7fe740 1 -- 172.21.15.96:0/390400316 >> 172.21.15.96:0/390400316 conn(0x560e173613c0 msgr2=0x560e17709d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:00.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.145+0000 7f360e7fe740 1 -- 172.21.15.96:0/390400316 shutdown_connections 2024-04-25T14:13:00.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.145+0000 7f360e7fe740 1 -- 172.21.15.96:0/390400316 wait complete. 2024-04-25T14:13:00.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.146+0000 7f360e7fe740 1 -- start start 2024-04-25T14:13:00.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.146+0000 7f360e7fe740 1 Processor -- start 2024-04-25T14:13:00.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.146+0000 7f360e7fe740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560e173443b0 0x560e17718280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.146+0000 7f360e7fe740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e1734c590 0x560e177187c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.146+0000 7f360e7fe740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e1736d270 0x560e1771f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.146+0000 7f360e7fe740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560e17716240 con 0x560e1736d270 2024-04-25T14:13:00.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.146+0000 7f360e7fe740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560e175fafe0 con 0x560e173443b0 2024-04-25T14:13:00.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.146+0000 7f360e7fe740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560e177163c0 con 0x560e1734c590 2024-04-25T14:13:00.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.146+0000 7f360e7fe740 10 client.0 _ll_register_callbacks cb 0x560e17611150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:13:00.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.146+0000 7f3605eea640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e1736d270 0x560e1771f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.146+0000 7f3605eea640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e1736d270 0x560e1771f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51764/0 (socket says 172.21.15.96:51764) 2024-04-25T14:13:00.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.146+0000 7f3605eea640 1 -- 172.21.15.96:0/477048372 learned_addr learned my addr 172.21.15.96:0/477048372 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:00.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.146+0000 7f3604ee8640 1 --2- 172.21.15.96:0/477048372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e1734c590 0x560e177187c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.147+0000 7f36056e9640 1 --2- 172.21.15.96:0/477048372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560e173443b0 0x560e17718280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.147+0000 7f3605eea640 1 -- 172.21.15.96:0/477048372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e1734c590 msgr2=0x560e177187c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:00.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.147+0000 7f3605eea640 1 --2- 172.21.15.96:0/477048372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e1734c590 0x560e177187c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.147+0000 7f3605eea640 1 -- 172.21.15.96:0/477048372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560e173443b0 msgr2=0x560e17718280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:00.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.147+0000 7f3605eea640 1 --2- 172.21.15.96:0/477048372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560e173443b0 0x560e17718280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.147+0000 7f3605eea640 1 -- 172.21.15.96:0/477048372 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e173852d0 con 0x560e1736d270 2024-04-25T14:13:00.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.147+0000 7f3604ee8640 1 --2- 172.21.15.96:0/477048372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e1734c590 0x560e177187c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:00.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.147+0000 7f3605eea640 1 --2- 172.21.15.96:0/477048372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e1736d270 0x560e1771f720 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f35f0033000 tx=0x7f35f002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ab002fdc89614f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:00.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.147+0000 7f35e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:13:00.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.147+0000 7f35e4ff9640 1 -- 172.21.15.96:0/477048372 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f0035040 con 0x560e1736d270 2024-04-25T14:13:00.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.147+0000 7f35e4ff9640 1 -- 172.21.15.96:0/477048372 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f35f00035f0 con 0x560e1736d270 2024-04-25T14:13:00.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.147+0000 7f360e7fe740 1 -- 172.21.15.96:0/477048372 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35f8017a70 con 0x560e1736d270 2024-04-25T14:13:00.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.147+0000 7f35e4ff9640 1 -- 172.21.15.96:0/477048372 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f35f0058070 con 0x560e1736d270 2024-04-25T14:13:00.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.147+0000 7f360e7fe740 10 client.15072 fetch_fsmap learned FSMap version 25 2024-04-25T14:13:00.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.147+0000 7f360e7fe740 1 -- 172.21.15.96:0/477048372 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560e1744f6b0 con 0x560e1736d270 2024-04-25T14:13:00.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.147+0000 7f35e4ff9640 1 -- 172.21.15.96:0/477048372 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35f8017a70 con 0x560e1736d270 2024-04-25T14:13:00.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.147+0000 7f35e4ff9640 1 -- 172.21.15.96:0/477048372 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f35f0048990 con 0x560e1736d270 2024-04-25T14:13:00.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.147+0000 7f360e7fe740 10 client.15072 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T14:13:00.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.148+0000 7f360e7fe740 10 client.15072 resolve_mds: resolved name 'l' to daemon mds.l 2024-04-25T14:13:00.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.148+0000 7f360e7fe740 20 client.15072 populate_metadata read hostname 'smithi096' 2024-04-25T14:13:00.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.148+0000 7f360e7fe740 1 --2- 172.21.15.96:0/477048372 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x560e1732be40 0x560e17726030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.148+0000 7f360e7fe740 4 client.15072 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:13:00.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.148+0000 7f360e7fe740 1 -- 172.21.15.96:0/477048372 --> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e17681350 con 0x560e1732be40 2024-04-25T14:13:00.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.148+0000 7f36056e9640 1 --2- 172.21.15.96:0/477048372 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x560e1732be40 0x560e17726030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.148+0000 7f35e4ff9640 10 client.15072 ms_handle_connect on v2:172.21.15.151:6832/1382721608 2024-04-25T14:13:00.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.148+0000 7f36056e9640 1 --2- 172.21.15.96:0/477048372 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x560e1732be40 0x560e17726030 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5875cf6f5b356611 server_cookie=3a9a81a080ff594e in_seq=0 out_seq=0 2024-04-25T14:13:00.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.149+0000 7f35e4ff9640 1 -- 172.21.15.96:0/477048372 <== mds.4 v2:172.21.15.151:6832/1382721608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560e17681350 con 0x560e1732be40 2024-04-25T14:13:00.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.149+0000 7f35e4ff9640 10 client.15072 handle_command_reply: tid=0 2024-04-25T14:13:00.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.149+0000 7f360e7fe740 1 client.15072 shutdown 2024-04-25T14:13:00.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.149+0000 7f360e7fe740 1 -- 172.21.15.96:0/477048372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e1736d270 msgr2=0x560e1771f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:00.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.149+0000 7f360e7fe740 1 --2- 172.21.15.96:0/477048372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e1736d270 0x560e1771f720 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f35f0033000 tx=0x7f35f002a410 comp rx=0 tx=0).stop 2024-04-25T14:13:00.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.149+0000 7f360e7fe740 1 -- 172.21.15.96:0/477048372 shutdown_connections 2024-04-25T14:13:00.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.149+0000 7f360e7fe740 1 --2- 172.21.15.96:0/477048372 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x560e1732be40 0x560e17726030 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.149+0000 7f360e7fe740 1 --2- 172.21.15.96:0/477048372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e1736d270 0x560e1771f720 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.149+0000 7f360e7fe740 1 --2- 172.21.15.96:0/477048372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e1734c590 0x560e177187c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.149+0000 7f360e7fe740 1 --2- 172.21.15.96:0/477048372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560e173443b0 0x560e17718280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.149+0000 7f35e4ff9640 0 client.15072 ms_handle_reset on v2:172.21.15.151:6832/1382721608 2024-04-25T14:13:00.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.149+0000 7f360e7fe740 1 -- 172.21.15.96:0/477048372 >> 172.21.15.96:0/477048372 conn(0x560e173613c0 msgr2=0x560e17708d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:00.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.149+0000 7f360e7fe740 1 -- 172.21.15.96:0/477048372 shutdown_connections 2024-04-25T14:13:00.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.149+0000 7f360e7fe740 1 -- 172.21.15.96:0/477048372 wait complete. 2024-04-25T14:13:00.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.150+0000 7f360e7fe740 20 client.15072 trim_cache size 0 max 16384 2024-04-25T14:13:00.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.168+0000 7f360e7fe740 1 Processor -- start 2024-04-25T14:13:00.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.168+0000 7f360e7fe740 1 -- start start 2024-04-25T14:13:00.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.168+0000 7f360e7fe740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e173443b0 0x560e17711970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.168+0000 7f360e7fe740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560e1734c590 0x560e17704a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.168+0000 7f360e7fe740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e1736d270 0x560e17712690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.168+0000 7f360e7fe740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560e17724370 con 0x560e173443b0 2024-04-25T14:13:00.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.168+0000 7f360e7fe740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560e17724a40 con 0x560e1734c590 2024-04-25T14:13:00.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.168+0000 7f360e7fe740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560e17725110 con 0x560e1736d270 2024-04-25T14:13:00.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.168+0000 7f36056e9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e173443b0 0x560e17711970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.168+0000 7f36056e9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e173443b0 0x560e17711970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51778/0 (socket says 172.21.15.96:51778) 2024-04-25T14:13:00.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.168+0000 7f36056e9640 1 -- 172.21.15.96:0/2682639917 learned_addr learned my addr 172.21.15.96:0/2682639917 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:00.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.169+0000 7f36056e9640 1 -- 172.21.15.96:0/2682639917 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e1736d270 msgr2=0x560e17712690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:00.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.169+0000 7f36056e9640 1 --2- 172.21.15.96:0/2682639917 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e1736d270 0x560e17712690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.169+0000 7f36056e9640 1 -- 172.21.15.96:0/2682639917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560e1734c590 msgr2=0x560e17704a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:00.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.169+0000 7f3604ee8640 1 --2- 172.21.15.96:0/2682639917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560e1734c590 0x560e17704a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.169+0000 7f36056e9640 1 --2- 172.21.15.96:0/2682639917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560e1734c590 0x560e17704a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.169+0000 7f36056e9640 1 -- 172.21.15.96:0/2682639917 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e1770df20 con 0x560e173443b0 2024-04-25T14:13:00.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.169+0000 7f3604ee8640 1 --2- 172.21.15.96:0/2682639917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560e1734c590 0x560e17704a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:00.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.169+0000 7f36056e9640 1 --2- 172.21.15.96:0/2682639917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e173443b0 0x560e17711970 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f35f800b9e0 tx=0x7f35f8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b839bf4ad0cc96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:00.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.169+0000 7f35e6ffd640 1 -- 172.21.15.96:0/2682639917 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f802c070 con 0x560e173443b0 2024-04-25T14:13:00.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.169+0000 7f35e6ffd640 1 -- 172.21.15.96:0/2682639917 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35f8028780 con 0x560e173443b0 2024-04-25T14:13:00.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.169+0000 7f35e6ffd640 1 -- 172.21.15.96:0/2682639917 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f800f9e0 con 0x560e173443b0 2024-04-25T14:13:00.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.170+0000 7f360e7fe740 1 -- 172.21.15.96:0/2682639917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e173443b0 msgr2=0x560e17711970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:00.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.170+0000 7f360e7fe740 1 --2- 172.21.15.96:0/2682639917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e173443b0 0x560e17711970 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f35f800b9e0 tx=0x7f35f8005fd0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.170+0000 7f360e7fe740 1 -- 172.21.15.96:0/2682639917 shutdown_connections 2024-04-25T14:13:00.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.170+0000 7f360e7fe740 1 --2- 172.21.15.96:0/2682639917 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e1736d270 0x560e17712690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.170+0000 7f360e7fe740 1 --2- 172.21.15.96:0/2682639917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560e1734c590 0x560e17704a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.170+0000 7f360e7fe740 1 --2- 172.21.15.96:0/2682639917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e173443b0 0x560e17711970 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.170+0000 7f360e7fe740 1 -- 172.21.15.96:0/2682639917 >> 172.21.15.96:0/2682639917 conn(0x560e173613c0 msgr2=0x560e177239b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:00.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.170+0000 7f360e7fe740 1 -- 172.21.15.96:0/2682639917 shutdown_connections 2024-04-25T14:13:00.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.170+0000 7f360e7fe740 1 -- 172.21.15.96:0/2682639917 wait complete. 2024-04-25T14:13:00.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.170+0000 7f360e7fe740 1 -- start start 2024-04-25T14:13:00.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.170+0000 7f360e7fe740 1 Processor -- start 2024-04-25T14:13:00.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.171+0000 7f360e7fe740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e173443b0 0x560e17719d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.171+0000 7f360e7fe740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560e1734c590 0x560e1771a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.171+0000 7f360e7fe740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e1736d270 0x560e1778cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.171+0000 7f360e7fe740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560e17725eb0 con 0x560e173443b0 2024-04-25T14:13:00.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.171+0000 7f360e7fe740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560e177257e0 con 0x560e1734c590 2024-04-25T14:13:00.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.171+0000 7f360e7fe740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560e17726580 con 0x560e1736d270 2024-04-25T14:13:00.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.171+0000 7f360e7fe740 10 client.0 _ll_register_callbacks cb 0x7f35f0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:13:00.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.171+0000 7f36056e9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e173443b0 0x560e17719d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.171+0000 7f36056e9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e173443b0 0x560e17719d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51780/0 (socket says 172.21.15.96:51780) 2024-04-25T14:13:00.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.171+0000 7f36056e9640 1 -- 172.21.15.96:0/2806399385 learned_addr learned my addr 172.21.15.96:0/2806399385 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:00.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.171+0000 7f36056e9640 1 -- 172.21.15.96:0/2806399385 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e1736d270 msgr2=0x560e1778cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:00.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.171+0000 7f36056e9640 1 --2- 172.21.15.96:0/2806399385 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e1736d270 0x560e1778cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.171+0000 7f36056e9640 1 -- 172.21.15.96:0/2806399385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560e1734c590 msgr2=0x560e1771a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:00.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.171+0000 7f36056e9640 1 --2- 172.21.15.96:0/2806399385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560e1734c590 0x560e1771a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.171+0000 7f36056e9640 1 -- 172.21.15.96:0/2806399385 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e1771a910 con 0x560e173443b0 2024-04-25T14:13:00.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.171+0000 7f36056e9640 1 --2- 172.21.15.96:0/2806399385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e173443b0 0x560e17719d60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f35f800b850 tx=0x7f35f8005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7a91571fcdd48b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:00.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.171+0000 7f35e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:13:00.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.171+0000 7f35e4ff9640 1 -- 172.21.15.96:0/2806399385 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f8037070 con 0x560e173443b0 2024-04-25T14:13:00.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.171+0000 7f35e4ff9640 1 -- 172.21.15.96:0/2806399385 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f35f80282d0 con 0x560e173443b0 2024-04-25T14:13:00.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.171+0000 7f360e7fe740 1 -- 172.21.15.96:0/2806399385 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35f8028780 con 0x560e173443b0 2024-04-25T14:13:00.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.171+0000 7f35e4ff9640 1 -- 172.21.15.96:0/2806399385 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f35f802c040 con 0x560e173443b0 2024-04-25T14:13:00.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.172+0000 7f360e7fe740 10 client.15078 fetch_fsmap learned FSMap version 25 2024-04-25T14:13:00.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.172+0000 7f360e7fe740 1 -- 172.21.15.96:0/2806399385 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560e1778e3e0 con 0x560e173443b0 2024-04-25T14:13:00.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.172+0000 7f35e4ff9640 1 -- 172.21.15.96:0/2806399385 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35f8028780 con 0x560e173443b0 2024-04-25T14:13:00.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.172+0000 7f35e4ff9640 1 -- 172.21.15.96:0/2806399385 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f35f8028ce0 con 0x560e173443b0 2024-04-25T14:13:00.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.172+0000 7f360e7fe740 10 client.15078 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T14:13:00.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.172+0000 7f360e7fe740 10 client.15078 resolve_mds: resolved name 'l' to daemon mds.l 2024-04-25T14:13:00.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.172+0000 7f360e7fe740 20 client.15078 populate_metadata read hostname 'smithi096' 2024-04-25T14:13:00.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.172+0000 7f360e7fe740 1 --2- 172.21.15.96:0/2806399385 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x560e17301280 0x560e177916a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.172+0000 7f360e7fe740 4 client.15078 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T14:13:00.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.172+0000 7f360e7fe740 1 -- 172.21.15.96:0/2806399385 --> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] -- command(tid 0: {"prefix": "status"}) -- 0x560e1767b1c0 con 0x560e17301280 2024-04-25T14:13:00.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.172+0000 7f3604ee8640 1 --2- 172.21.15.96:0/2806399385 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x560e17301280 0x560e177916a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.173+0000 7f35e4ff9640 10 client.15078 ms_handle_connect on v2:172.21.15.151:6832/1382721608 2024-04-25T14:13:00.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.173+0000 7f3604ee8640 1 --2- 172.21.15.96:0/2806399385 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x560e17301280 0x560e177916a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=aa3f449d1668e834 server_cookie=e2cadd205dcf0604 in_seq=0 out_seq=0 2024-04-25T14:13:00.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.173+0000 7f35e4ff9640 1 -- 172.21.15.96:0/2806399385 <== mds.4 v2:172.21.15.151:6832/1382721608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x560e1767b1c0 con 0x560e17301280 2024-04-25T14:13:00.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.173+0000 7f35e4ff9640 10 client.15078 handle_command_reply: tid=0 2024-04-25T14:13:00.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.173+0000 7f360e7fe740 1 client.15078 shutdown 2024-04-25T14:13:00.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.173+0000 7f360e7fe740 1 -- 172.21.15.96:0/2806399385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e173443b0 msgr2=0x560e17719d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:00.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.173+0000 7f360e7fe740 1 --2- 172.21.15.96:0/2806399385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e173443b0 0x560e17719d60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f35f800b850 tx=0x7f35f8005bb0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.173+0000 7f360e7fe740 1 -- 172.21.15.96:0/2806399385 shutdown_connections 2024-04-25T14:13:00.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.173+0000 7f360e7fe740 1 --2- 172.21.15.96:0/2806399385 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x560e17301280 0x560e177916a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.173+0000 7f360e7fe740 1 --2- 172.21.15.96:0/2806399385 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e1736d270 0x560e1778cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.173+0000 7f360e7fe740 1 --2- 172.21.15.96:0/2806399385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560e1734c590 0x560e1771a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.173+0000 7f35e4ff9640 0 client.15078 ms_handle_reset on v2:172.21.15.151:6832/1382721608 2024-04-25T14:13:00.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.173+0000 7f360e7fe740 1 --2- 172.21.15.96:0/2806399385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e173443b0 0x560e17719d60 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.173+0000 7f360e7fe740 1 -- 172.21.15.96:0/2806399385 >> 172.21.15.96:0/2806399385 conn(0x560e173613c0 msgr2=0x560e17704a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:00.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.173+0000 7f360e7fe740 1 -- 172.21.15.96:0/2806399385 shutdown_connections 2024-04-25T14:13:00.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.173+0000 7f360e7fe740 1 -- 172.21.15.96:0/2806399385 wait complete. 2024-04-25T14:13:00.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.173+0000 7f360e7fe740 20 client.15078 trim_cache size 0 max 16384 2024-04-25T14:13:00.180 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T14:13:00.180 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "59334182-030c-11ef-bc93-c7b262605968", 2024-04-25T14:13:00.180 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 4, 2024-04-25T14:13:00.180 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 24427, 2024-04-25T14:13:00.180 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-04-25T14:13:00.180 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-04-25T14:13:00.181 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-04-25T14:13:00.181 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 85.646898910999994, 2024-04-25T14:13:00.181 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 25, 2024-04-25T14:13:00.181 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 78, 2024-04-25T14:13:00.181 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 76, 2024-04-25T14:13:00.181 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 128.370853603 2024-04-25T14:13:00.181 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T14:13:00.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.174+0000 7f360794d640 1 -- 172.21.15.96:0/2002599194 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f35e80afaf0 msgr2=0x7f35e80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:00.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.174+0000 7f360794d640 1 --2- 172.21.15.96:0/2002599194 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f35e80afaf0 0x7f35e80b2000 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x560e1770f540 tx=0x7f35f0009260 comp rx=0 tx=0).stop 2024-04-25T14:13:00.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.174+0000 7f360794d640 1 -- 172.21.15.96:0/2002599194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3600102260 msgr2=0x7f360019a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:00.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.174+0000 7f360794d640 1 --2- 172.21.15.96:0/2002599194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3600102260 0x7f360019a7b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f35f400cce0 tx=0x7f35f400a620 comp rx=0 tx=0).stop 2024-04-25T14:13:00.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.174+0000 7f360794d640 1 -- 172.21.15.96:0/2002599194 shutdown_connections 2024-04-25T14:13:00.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.174+0000 7f360794d640 1 --2- 172.21.15.96:0/2002599194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f36001050b0 0x7f36001a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.174+0000 7f360794d640 1 --2- 172.21.15.96:0/2002599194 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f35e80afaf0 0x7f35e80b2000 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.174+0000 7f360794d640 1 --2- 172.21.15.96:0/2002599194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3600102260 0x7f360019a7b0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.174+0000 7f360794d640 1 --2- 172.21.15.96:0/2002599194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3600101760 0x7f360019a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.174+0000 7f360794d640 1 -- 172.21.15.96:0/2002599194 >> 172.21.15.96:0/2002599194 conn(0x7f36000fd380 msgr2=0x7f3600107c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:00.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.174+0000 7f360794d640 1 -- 172.21.15.96:0/2002599194 shutdown_connections 2024-04-25T14:13:00.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.174+0000 7f360794d640 1 -- 172.21.15.96:0/2002599194 wait complete. 2024-04-25T14:13:00.185 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-04-25T14:13:00.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.293+0000 7f77d4842640 1 -- 172.21.15.96:0/2352317993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77d0103c20 msgr2=0x7f77d0104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:00.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.293+0000 7f77d4842640 1 --2- 172.21.15.96:0/2352317993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77d0103c20 0x7f77d0104030 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f77c0009900 tx=0x7f77c002a410 comp rx=0 tx=0).stop 2024-04-25T14:13:00.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.294+0000 7f77d4842640 1 -- 172.21.15.96:0/2352317993 shutdown_connections 2024-04-25T14:13:00.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.294+0000 7f77d4842640 1 --2- 172.21.15.96:0/2352317993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77d0108920 0x7f77d010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.294+0000 7f77d4842640 1 --2- 172.21.15.96:0/2352317993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77d0103c20 0x7f77d0104030 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.294+0000 7f77d4842640 1 --2- 172.21.15.96:0/2352317993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77d01029e0 0x7f77d01036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.294+0000 7f77d4842640 1 -- 172.21.15.96:0/2352317993 >> 172.21.15.96:0/2352317993 conn(0x7f77d00fdbd0 msgr2=0x7f77d0100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:00.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.294+0000 7f77d4842640 1 -- 172.21.15.96:0/2352317993 shutdown_connections 2024-04-25T14:13:00.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.294+0000 7f77d4842640 1 -- 172.21.15.96:0/2352317993 wait complete. 2024-04-25T14:13:00.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.294+0000 7f77d4842640 1 Processor -- start 2024-04-25T14:13:00.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.294+0000 7f77d4842640 1 -- start start 2024-04-25T14:13:00.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.294+0000 7f77d4842640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77d01029e0 0x7f77d006a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.295+0000 7f77d4842640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77d0103c20 0x7f77d019f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.295+0000 7f77d4842640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77d0108920 0x7f77d01a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.295+0000 7f77d4842640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f77d010e560 con 0x7f77d0108920 2024-04-25T14:13:00.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.295+0000 7f77d4842640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f77d010e3e0 con 0x7f77d0103c20 2024-04-25T14:13:00.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.295+0000 7f77d4842640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f77d010e6e0 con 0x7f77d01029e0 2024-04-25T14:13:00.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.295+0000 7f77ced9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77d0108920 0x7f77d01a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.295+0000 7f77ced9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77d0108920 0x7f77d01a18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51796/0 (socket says 172.21.15.96:51796) 2024-04-25T14:13:00.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.295+0000 7f77ced9d640 1 -- 172.21.15.96:0/4114960393 learned_addr learned my addr 172.21.15.96:0/4114960393 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:00.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.295+0000 7f77ce59c640 1 --2- 172.21.15.96:0/4114960393 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77d01029e0 0x7f77d006a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.295+0000 7f77cdd9b640 1 --2- 172.21.15.96:0/4114960393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77d0103c20 0x7f77d019f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.295+0000 7f77ced9d640 1 -- 172.21.15.96:0/4114960393 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77d01029e0 msgr2=0x7f77d006a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:00.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.295+0000 7f77ced9d640 1 --2- 172.21.15.96:0/4114960393 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77d01029e0 0x7f77d006a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.295+0000 7f77ced9d640 1 -- 172.21.15.96:0/4114960393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77d0103c20 msgr2=0x7f77d019f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:00.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.295+0000 7f77ced9d640 1 --2- 172.21.15.96:0/4114960393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77d0103c20 0x7f77d019f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.295+0000 7f77ced9d640 1 -- 172.21.15.96:0/4114960393 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d01a2050 con 0x7f77d0108920 2024-04-25T14:13:00.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.295+0000 7f77cdd9b640 1 --2- 172.21.15.96:0/4114960393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77d0103c20 0x7f77d019f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:00.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.295+0000 7f77ced9d640 1 --2- 172.21.15.96:0/4114960393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77d0108920 0x7f77d01a18c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f77c800cce0 tx=0x7f77c800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:00.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.295+0000 7f77bf7fe640 1 -- 172.21.15.96:0/4114960393 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c800f450 con 0x7f77d0108920 2024-04-25T14:13:00.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.295+0000 7f77d4842640 1 -- 172.21.15.96:0/4114960393 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77d01a2340 con 0x7f77d0108920 2024-04-25T14:13:00.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.295+0000 7f77bf7fe640 1 -- 172.21.15.96:0/4114960393 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77c800faa0 con 0x7f77d0108920 2024-04-25T14:13:00.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.295+0000 7f77bf7fe640 1 -- 172.21.15.96:0/4114960393 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c8018a80 con 0x7f77d0108920 2024-04-25T14:13:00.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.295+0000 7f77d4842640 1 -- 172.21.15.96:0/4114960393 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77d01a2910 con 0x7f77d0108920 2024-04-25T14:13:00.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.298+0000 7f77db7c6740 1 Processor -- start 2024-04-25T14:13:00.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.298+0000 7f77bf7fe640 1 -- 172.21.15.96:0/4114960393 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f77c800fc50 con 0x7f77d0108920 2024-04-25T14:13:00.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.298+0000 7f77db7c6740 1 -- start start 2024-04-25T14:13:00.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.298+0000 7f77db7c6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c96c3b6270 0x55c96c757370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.298+0000 7f77db7c6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c96c38d3b0 0x55c96c7578f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.298+0000 7f77db7c6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c96c395590 0x55c96c75e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.298+0000 7f77db7c6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c96c6bc740 con 0x55c96c395590 2024-04-25T14:13:00.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.298+0000 7f77db7c6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c96c6583a0 con 0x55c96c3b6270 2024-04-25T14:13:00.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.298+0000 7f77db7c6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c96c6b8700 con 0x55c96c38d3b0 2024-04-25T14:13:00.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.298+0000 7f77cdd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c96c395590 0x55c96c75e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.299+0000 7f77cdd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c96c395590 0x55c96c75e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51808/0 (socket says 172.21.15.96:51808) 2024-04-25T14:13:00.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.299+0000 7f77cdd9b640 1 -- 172.21.15.96:0/2918229604 learned_addr learned my addr 172.21.15.96:0/2918229604 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:00.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.299+0000 7f77ce59c640 1 --2- 172.21.15.96:0/2918229604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c96c38d3b0 0x55c96c7578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.299+0000 7f77cdd9b640 1 --2- 172.21.15.96:0/2918229604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c96c3b6270 0x55c96c757370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.299+0000 7f77bf7fe640 1 --2- 172.21.15.96:0/4114960393 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f77b80afaf0 0x7f77b80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.299+0000 7f77cdd9b640 1 -- 172.21.15.96:0/2918229604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c96c38d3b0 msgr2=0x55c96c7578f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:00.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.299+0000 7f77cdd9b640 1 --2- 172.21.15.96:0/2918229604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c96c38d3b0 0x55c96c7578f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.299+0000 7f77cdd9b640 1 -- 172.21.15.96:0/2918229604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c96c3b6270 msgr2=0x55c96c757370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:00.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.299+0000 7f77cdd9b640 1 --2- 172.21.15.96:0/2918229604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c96c3b6270 0x55c96c757370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.299+0000 7f77cdd9b640 1 -- 172.21.15.96:0/2918229604 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c96c674750 con 0x55c96c395590 2024-04-25T14:13:00.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.299+0000 7f77cdd9b640 1 --2- 172.21.15.96:0/2918229604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c96c395590 0x55c96c75e7f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f77c0005f70 tx=0x7f77c0031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d80111fcfc871830 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:00.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.299+0000 7f77bf7fe640 1 -- 172.21.15.96:0/4114960393 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f77c80df0c0 con 0x7f77d0108920 2024-04-25T14:13:00.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.299+0000 7f77ced9d640 1 --2- 172.21.15.96:0/4114960393 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f77b80afaf0 0x7f77b80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.299+0000 7f77bcff9640 1 -- 172.21.15.96:0/2918229604 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c0035040 con 0x55c96c395590 2024-04-25T14:13:00.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.299+0000 7f77bcff9640 1 -- 172.21.15.96:0/2918229604 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77c0031770 con 0x55c96c395590 2024-04-25T14:13:00.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.299+0000 7f77bcff9640 1 -- 172.21.15.96:0/2918229604 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c0004920 con 0x55c96c395590 2024-04-25T14:13:00.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.300+0000 7f77db7c6740 1 -- 172.21.15.96:0/2918229604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c96c395590 msgr2=0x55c96c75e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:00.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.300+0000 7f77ced9d640 1 --2- 172.21.15.96:0/4114960393 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f77b80afaf0 0x7f77b80b2000 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f77c80df420 tx=0x7f77c8021000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:00.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.300+0000 7f77db7c6740 1 --2- 172.21.15.96:0/2918229604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c96c395590 0x55c96c75e7f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f77c0005f70 tx=0x7f77c0031400 comp rx=0 tx=0).stop 2024-04-25T14:13:00.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.300+0000 7f77db7c6740 1 -- 172.21.15.96:0/2918229604 shutdown_connections 2024-04-25T14:13:00.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.300+0000 7f77db7c6740 1 --2- 172.21.15.96:0/2918229604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c96c395590 0x55c96c75e7f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.300+0000 7f77db7c6740 1 --2- 172.21.15.96:0/2918229604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c96c38d3b0 0x55c96c7578f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.300+0000 7f77db7c6740 1 --2- 172.21.15.96:0/2918229604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c96c3b6270 0x55c96c757370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.300+0000 7f77db7c6740 1 -- 172.21.15.96:0/2918229604 >> 172.21.15.96:0/2918229604 conn(0x55c96c3aa3c0 msgr2=0x55c96c752d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:00.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.300+0000 7f77db7c6740 1 -- 172.21.15.96:0/2918229604 shutdown_connections 2024-04-25T14:13:00.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.300+0000 7f77db7c6740 1 -- 172.21.15.96:0/2918229604 wait complete. 2024-04-25T14:13:00.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.301+0000 7f77db7c6740 1 -- start start 2024-04-25T14:13:00.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.301+0000 7f77db7c6740 1 Processor -- start 2024-04-25T14:13:00.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.301+0000 7f77db7c6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c96c38d3b0 0x55c96c7611f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.301+0000 7f77db7c6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c96c395590 0x55c96c761730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.301+0000 7f77db7c6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c96c3b6270 0x55c96c768690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.301+0000 7f77db7c6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c96c75f1e0 con 0x55c96c3b6270 2024-04-25T14:13:00.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.301+0000 7f77db7c6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c96c643fe0 con 0x55c96c395590 2024-04-25T14:13:00.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.301+0000 7f77db7c6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c96c75f360 con 0x55c96c38d3b0 2024-04-25T14:13:00.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.301+0000 7f77db7c6740 10 client.0 _ll_register_callbacks cb 0x55c96c65a150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:13:00.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.301+0000 7f77cdd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c96c3b6270 0x55c96c768690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.301+0000 7f77cdd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c96c3b6270 0x55c96c768690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51820/0 (socket says 172.21.15.96:51820) 2024-04-25T14:13:00.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.301+0000 7f77cdd9b640 1 -- 172.21.15.96:0/1794060236 learned_addr learned my addr 172.21.15.96:0/1794060236 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:00.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.301+0000 7f77cdd9b640 1 -- 172.21.15.96:0/1794060236 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c96c38d3b0 msgr2=0x55c96c7611f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:00.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.301+0000 7f77cdd9b640 1 --2- 172.21.15.96:0/1794060236 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c96c38d3b0 0x55c96c7611f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.301+0000 7f77cdd9b640 1 -- 172.21.15.96:0/1794060236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c96c395590 msgr2=0x55c96c761730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:00.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.301+0000 7f77ce59c640 1 --2- 172.21.15.96:0/1794060236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c96c395590 0x55c96c761730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.301+0000 7f77cdd9b640 1 --2- 172.21.15.96:0/1794060236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c96c395590 0x55c96c761730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.301+0000 7f77cdd9b640 1 -- 172.21.15.96:0/1794060236 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c96c3ce2d0 con 0x55c96c3b6270 2024-04-25T14:13:00.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.301+0000 7f77cdd9b640 1 --2- 172.21.15.96:0/1794060236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c96c3b6270 0x55c96c768690 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f77c0032e20 tx=0x7f77c0031920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd8392bb1c212c49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:00.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.301+0000 7f77b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:13:00.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.302+0000 7f77b5ffb640 1 -- 172.21.15.96:0/1794060236 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c0005530 con 0x55c96c3b6270 2024-04-25T14:13:00.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.302+0000 7f77b5ffb640 1 -- 172.21.15.96:0/1794060236 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f77c00048f0 con 0x55c96c3b6270 2024-04-25T14:13:00.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.302+0000 7f77db7c6740 1 -- 172.21.15.96:0/1794060236 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77c0031770 con 0x55c96c3b6270 2024-04-25T14:13:00.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.302+0000 7f77b5ffb640 1 -- 172.21.15.96:0/1794060236 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f77c0004d10 con 0x55c96c3b6270 2024-04-25T14:13:00.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.302+0000 7f77db7c6740 10 client.15090 fetch_fsmap learned FSMap version 25 2024-04-25T14:13:00.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.302+0000 7f77db7c6740 1 -- 172.21.15.96:0/1794060236 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c96c4986b0 con 0x55c96c3b6270 2024-04-25T14:13:00.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.302+0000 7f77b5ffb640 1 -- 172.21.15.96:0/1794060236 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77c0031770 con 0x55c96c3b6270 2024-04-25T14:13:00.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.302+0000 7f77b5ffb640 1 -- 172.21.15.96:0/1794060236 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f77c003a050 con 0x55c96c3b6270 2024-04-25T14:13:00.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.302+0000 7f77db7c6740 10 client.15090 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T14:13:00.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.302+0000 7f77db7c6740 10 client.15090 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-25T14:13:00.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.302+0000 7f77db7c6740 20 client.15090 populate_metadata read hostname 'smithi096' 2024-04-25T14:13:00.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.302+0000 7f77db7c6740 1 --2- 172.21.15.96:0/1794060236 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x55c96c374e40 0x55c96c76ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.302+0000 7f77db7c6740 4 client.15090 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:13:00.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.302+0000 7f77db7c6740 1 -- 172.21.15.96:0/1794060236 --> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c96c6ca350 con 0x55c96c374e40 2024-04-25T14:13:00.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.302+0000 7f77ce59c640 1 --2- 172.21.15.96:0/1794060236 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x55c96c374e40 0x55c96c76ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.303+0000 7f77b5ffb640 10 client.15090 ms_handle_connect on v2:172.21.15.97:6834/3872301726 2024-04-25T14:13:00.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.303+0000 7f77ce59c640 1 --2- 172.21.15.96:0/1794060236 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x55c96c374e40 0x55c96c76ea00 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7f008b527d7547dd server_cookie=20be7d7e7148654e in_seq=0 out_seq=0 2024-04-25T14:13:00.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.304+0000 7f77b5ffb640 1 -- 172.21.15.96:0/1794060236 <== mds.3 v2:172.21.15.97:6834/3872301726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c96c6ca350 con 0x55c96c374e40 2024-04-25T14:13:00.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.304+0000 7f77b5ffb640 10 client.15090 handle_command_reply: tid=0 2024-04-25T14:13:00.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.304+0000 7f77db7c6740 1 client.15090 shutdown 2024-04-25T14:13:00.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.304+0000 7f77db7c6740 1 -- 172.21.15.96:0/1794060236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c96c3b6270 msgr2=0x55c96c768690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:00.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.304+0000 7f77db7c6740 1 --2- 172.21.15.96:0/1794060236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c96c3b6270 0x55c96c768690 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f77c0032e20 tx=0x7f77c0031920 comp rx=0 tx=0).stop 2024-04-25T14:13:00.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.304+0000 7f77db7c6740 1 -- 172.21.15.96:0/1794060236 shutdown_connections 2024-04-25T14:13:00.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.304+0000 7f77db7c6740 1 --2- 172.21.15.96:0/1794060236 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x55c96c374e40 0x55c96c76ea00 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.304+0000 7f77db7c6740 1 --2- 172.21.15.96:0/1794060236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c96c3b6270 0x55c96c768690 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.304+0000 7f77db7c6740 1 --2- 172.21.15.96:0/1794060236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c96c395590 0x55c96c761730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.304+0000 7f77b5ffb640 0 client.15090 ms_handle_reset on v2:172.21.15.97:6834/3872301726 2024-04-25T14:13:00.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.304+0000 7f77db7c6740 1 --2- 172.21.15.96:0/1794060236 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c96c38d3b0 0x55c96c7611f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.304+0000 7f77db7c6740 1 -- 172.21.15.96:0/1794060236 >> 172.21.15.96:0/1794060236 conn(0x55c96c3aa3c0 msgr2=0x55c96c751d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:00.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.304+0000 7f77db7c6740 1 -- 172.21.15.96:0/1794060236 shutdown_connections 2024-04-25T14:13:00.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.304+0000 7f77db7c6740 1 -- 172.21.15.96:0/1794060236 wait complete. 2024-04-25T14:13:00.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.304+0000 7f77db7c6740 20 client.15090 trim_cache size 0 max 16384 2024-04-25T14:13:00.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.322+0000 7f77db7c6740 1 Processor -- start 2024-04-25T14:13:00.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.322+0000 7f77db7c6740 1 -- start start 2024-04-25T14:13:00.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.322+0000 7f77db7c6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c96c374e40 0x55c96c75f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.322+0000 7f77db7c6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c96c38d3b0 0x55c96c75fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.322+0000 7f77db7c6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c96c395590 0x55c96c7526b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.323+0000 7f77db7c6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c96c76cd40 con 0x55c96c395590 2024-04-25T14:13:00.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.323+0000 7f77db7c6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c96c76d410 con 0x55c96c38d3b0 2024-04-25T14:13:00.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.323+0000 7f77db7c6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c96c76dae0 con 0x55c96c374e40 2024-04-25T14:13:00.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.323+0000 7f77cdd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c96c395590 0x55c96c7526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.323+0000 7f77cdd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c96c395590 0x55c96c7526b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51832/0 (socket says 172.21.15.96:51832) 2024-04-25T14:13:00.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.323+0000 7f77cdd9b640 1 -- 172.21.15.96:0/2847557991 learned_addr learned my addr 172.21.15.96:0/2847557991 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:00.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.323+0000 7f77cdd9b640 1 -- 172.21.15.96:0/2847557991 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c96c374e40 msgr2=0x55c96c75f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:00.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.323+0000 7f77cdd9b640 1 --2- 172.21.15.96:0/2847557991 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c96c374e40 0x55c96c75f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.323+0000 7f77cdd9b640 1 -- 172.21.15.96:0/2847557991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c96c38d3b0 msgr2=0x55c96c75fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:00.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.323+0000 7f77cdd9b640 1 --2- 172.21.15.96:0/2847557991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c96c38d3b0 0x55c96c75fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.323+0000 7f77cdd9b640 1 -- 172.21.15.96:0/2847557991 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c96c75c0c0 con 0x55c96c395590 2024-04-25T14:13:00.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.323+0000 7f77cdd9b640 1 --2- 172.21.15.96:0/2847557991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c96c395590 0x55c96c7526b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f77c0009900 tx=0x7f77c0031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=872f4af6c70a24f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:00.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.323+0000 7f77b7fff640 1 -- 172.21.15.96:0/2847557991 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c0035030 con 0x55c96c395590 2024-04-25T14:13:00.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.323+0000 7f77b7fff640 1 -- 172.21.15.96:0/2847557991 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77c004e070 con 0x55c96c395590 2024-04-25T14:13:00.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.323+0000 7f77b7fff640 1 -- 172.21.15.96:0/2847557991 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c0031bf0 con 0x55c96c395590 2024-04-25T14:13:00.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.324+0000 7f77db7c6740 1 -- 172.21.15.96:0/2847557991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c96c395590 msgr2=0x55c96c7526b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:00.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.324+0000 7f77db7c6740 1 --2- 172.21.15.96:0/2847557991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c96c395590 0x55c96c7526b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f77c0009900 tx=0x7f77c0031400 comp rx=0 tx=0).stop 2024-04-25T14:13:00.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.324+0000 7f77db7c6740 1 -- 172.21.15.96:0/2847557991 shutdown_connections 2024-04-25T14:13:00.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.324+0000 7f77db7c6740 1 --2- 172.21.15.96:0/2847557991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c96c395590 0x55c96c7526b0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.324+0000 7f77db7c6740 1 --2- 172.21.15.96:0/2847557991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c96c38d3b0 0x55c96c75fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.324+0000 7f77db7c6740 1 --2- 172.21.15.96:0/2847557991 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c96c374e40 0x55c96c75f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.324+0000 7f77db7c6740 1 -- 172.21.15.96:0/2847557991 >> 172.21.15.96:0/2847557991 conn(0x55c96c3aa3c0 msgr2=0x55c96c75a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:00.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.324+0000 7f77db7c6740 1 -- 172.21.15.96:0/2847557991 shutdown_connections 2024-04-25T14:13:00.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.324+0000 7f77db7c6740 1 -- 172.21.15.96:0/2847557991 wait complete. 2024-04-25T14:13:00.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.324+0000 7f77db7c6740 1 -- start start 2024-04-25T14:13:00.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.324+0000 7f77db7c6740 1 Processor -- start 2024-04-25T14:13:00.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.324+0000 7f77db7c6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c96c374e40 0x55c96c7633d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.324+0000 7f77db7c6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c96c38d3b0 0x55c96c763910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.324+0000 7f77db7c6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c96c395590 0x55c96c766850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.324+0000 7f77db7c6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c96c76e880 con 0x55c96c374e40 2024-04-25T14:13:00.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.324+0000 7f77db7c6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c96c76e1b0 con 0x55c96c395590 2024-04-25T14:13:00.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.324+0000 7f77db7c6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c96c76ef50 con 0x55c96c38d3b0 2024-04-25T14:13:00.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.324+0000 7f77db7c6740 10 client.0 _ll_register_callbacks cb 0x7f77c40093c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:13:00.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.324+0000 7f77cdd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c96c374e40 0x55c96c7633d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.324+0000 7f77cdd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c96c374e40 0x55c96c7633d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51838/0 (socket says 172.21.15.96:51838) 2024-04-25T14:13:00.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.324+0000 7f77cdd9b640 1 -- 172.21.15.96:0/2606516773 learned_addr learned my addr 172.21.15.96:0/2606516773 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:00.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.325+0000 7f77cdd9b640 1 --2- 172.21.15.96:0/2606516773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c96c395590 0x55c96c766850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.325+0000 7f77ce59c640 1 --2- 172.21.15.96:0/2606516773 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c96c38d3b0 0x55c96c763910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.325+0000 7f77cdd9b640 1 -- 172.21.15.96:0/2606516773 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c96c38d3b0 msgr2=0x55c96c763910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:00.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.325+0000 7f77cdd9b640 1 --2- 172.21.15.96:0/2606516773 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c96c38d3b0 0x55c96c763910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.325+0000 7f77cdd9b640 1 -- 172.21.15.96:0/2606516773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c96c395590 msgr2=0x55c96c766850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:00.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.325+0000 7f77cdd9b640 1 --2- 172.21.15.96:0/2606516773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c96c395590 0x55c96c766850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.325+0000 7f77cdd9b640 1 -- 172.21.15.96:0/2606516773 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c96c766e10 con 0x55c96c374e40 2024-04-25T14:13:00.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.325+0000 7f77ce59c640 1 --2- 172.21.15.96:0/2606516773 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c96c38d3b0 0x55c96c763910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:00.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.325+0000 7f77cdd9b640 1 --2- 172.21.15.96:0/2606516773 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c96c374e40 0x55c96c7633d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f77c0003d30 tx=0x7f77c0032c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f6d990d6b2dcb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:00.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.325+0000 7f77b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:13:00.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.325+0000 7f77b5ffb640 1 -- 172.21.15.96:0/2606516773 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c0065020 con 0x55c96c374e40 2024-04-25T14:13:00.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.325+0000 7f77b5ffb640 1 -- 172.21.15.96:0/2606516773 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f77c0003800 con 0x55c96c374e40 2024-04-25T14:13:00.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.325+0000 7f77db7c6740 1 -- 172.21.15.96:0/2606516773 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77c004e070 con 0x55c96c374e40 2024-04-25T14:13:00.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.325+0000 7f77b5ffb640 1 -- 172.21.15.96:0/2606516773 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f77c0035070 con 0x55c96c374e40 2024-04-25T14:13:00.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.325+0000 7f77db7c6740 10 client.15096 fetch_fsmap learned FSMap version 25 2024-04-25T14:13:00.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.325+0000 7f77db7c6740 1 -- 172.21.15.96:0/2606516773 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c96c7d7880 con 0x55c96c374e40 2024-04-25T14:13:00.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.325+0000 7f77b5ffb640 1 -- 172.21.15.96:0/2606516773 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77c004e070 con 0x55c96c374e40 2024-04-25T14:13:00.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.326+0000 7f77b5ffb640 1 -- 172.21.15.96:0/2606516773 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f77c005c070 con 0x55c96c374e40 2024-04-25T14:13:00.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.326+0000 7f77db7c6740 10 client.15096 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T14:13:00.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.326+0000 7f77db7c6740 10 client.15096 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-25T14:13:00.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.326+0000 7f77db7c6740 20 client.15096 populate_metadata read hostname 'smithi096' 2024-04-25T14:13:00.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.326+0000 7f77db7c6740 1 --2- 172.21.15.96:0/2606516773 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x55c96c3b6270 0x55c96c7daab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.326+0000 7f77db7c6740 4 client.15096 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T14:13:00.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.326+0000 7f77db7c6740 1 -- 172.21.15.96:0/2606516773 --> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] -- command(tid 0: {"prefix": "status"}) -- 0x55c96c6c41c0 con 0x55c96c3b6270 2024-04-25T14:13:00.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.326+0000 7f77ce59c640 1 --2- 172.21.15.96:0/2606516773 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x55c96c3b6270 0x55c96c7daab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.326+0000 7f77b5ffb640 10 client.15096 ms_handle_connect on v2:172.21.15.97:6834/3872301726 2024-04-25T14:13:00.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.326+0000 7f77ce59c640 1 --2- 172.21.15.96:0/2606516773 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x55c96c3b6270 0x55c96c7daab0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1eccddd3bcbbfc87 server_cookie=6c508f5208a21b0d in_seq=0 out_seq=0 2024-04-25T14:13:00.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.327+0000 7f77b5ffb640 1 -- 172.21.15.96:0/2606516773 <== mds.3 v2:172.21.15.97:6834/3872301726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55c96c6c41c0 con 0x55c96c3b6270 2024-04-25T14:13:00.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.327+0000 7f77b5ffb640 10 client.15096 handle_command_reply: tid=0 2024-04-25T14:13:00.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.327+0000 7f77db7c6740 1 client.15096 shutdown 2024-04-25T14:13:00.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.327+0000 7f77db7c6740 1 -- 172.21.15.96:0/2606516773 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c96c374e40 msgr2=0x55c96c7633d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:00.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.327+0000 7f77db7c6740 1 --2- 172.21.15.96:0/2606516773 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c96c374e40 0x55c96c7633d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f77c0003d30 tx=0x7f77c0032c00 comp rx=0 tx=0).stop 2024-04-25T14:13:00.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.327+0000 7f77db7c6740 1 -- 172.21.15.96:0/2606516773 shutdown_connections 2024-04-25T14:13:00.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.327+0000 7f77db7c6740 1 --2- 172.21.15.96:0/2606516773 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x55c96c3b6270 0x55c96c7daab0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.327+0000 7f77db7c6740 1 --2- 172.21.15.96:0/2606516773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c96c395590 0x55c96c766850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.327+0000 7f77b5ffb640 0 client.15096 ms_handle_reset on v2:172.21.15.97:6834/3872301726 2024-04-25T14:13:00.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.327+0000 7f77db7c6740 1 --2- 172.21.15.96:0/2606516773 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c96c38d3b0 0x55c96c763910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.327+0000 7f77db7c6740 1 --2- 172.21.15.96:0/2606516773 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c96c374e40 0x55c96c7633d0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.327+0000 7f77db7c6740 1 -- 172.21.15.96:0/2606516773 >> 172.21.15.96:0/2606516773 conn(0x55c96c3aa3c0 msgr2=0x55c96c75f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:00.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.327+0000 7f77db7c6740 1 -- 172.21.15.96:0/2606516773 shutdown_connections 2024-04-25T14:13:00.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.327+0000 7f77db7c6740 1 -- 172.21.15.96:0/2606516773 wait complete. 2024-04-25T14:13:00.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.327+0000 7f77db7c6740 20 client.15096 trim_cache size 0 max 16384 2024-04-25T14:13:00.334 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T14:13:00.334 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "59334182-030c-11ef-bc93-c7b262605968", 2024-04-25T14:13:00.334 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 3, 2024-04-25T14:13:00.334 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 24455, 2024-04-25T14:13:00.334 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-04-25T14:13:00.334 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-04-25T14:13:00.334 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-04-25T14:13:00.335 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 86.808374494000006, 2024-04-25T14:13:00.335 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 25, 2024-04-25T14:13:00.335 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 78, 2024-04-25T14:13:00.335 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 76, 2024-04-25T14:13:00.335 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 116.98340543899999 2024-04-25T14:13:00.335 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T14:13:00.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.327+0000 7f77d4842640 1 -- 172.21.15.96:0/4114960393 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f77b80afaf0 msgr2=0x7f77b80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:00.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.327+0000 7f77d4842640 1 --2- 172.21.15.96:0/4114960393 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f77b80afaf0 0x7f77b80b2000 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f77c80df420 tx=0x7f77c8021000 comp rx=0 tx=0).stop 2024-04-25T14:13:00.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.327+0000 7f77d4842640 1 -- 172.21.15.96:0/4114960393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77d0108920 msgr2=0x7f77d01a18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:00.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.327+0000 7f77d4842640 1 --2- 172.21.15.96:0/4114960393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77d0108920 0x7f77d01a18c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f77c800cce0 tx=0x7f77c800a620 comp rx=0 tx=0).stop 2024-04-25T14:13:00.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.327+0000 7f77d4842640 1 -- 172.21.15.96:0/4114960393 shutdown_connections 2024-04-25T14:13:00.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.327+0000 7f77d4842640 1 --2- 172.21.15.96:0/4114960393 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f77b80afaf0 0x7f77b80b2000 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.327+0000 7f77d4842640 1 --2- 172.21.15.96:0/4114960393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77d0108920 0x7f77d01a18c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.327+0000 7f77d4842640 1 --2- 172.21.15.96:0/4114960393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77d0103c20 0x7f77d019f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.327+0000 7f77d4842640 1 --2- 172.21.15.96:0/4114960393 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77d01029e0 0x7f77d006a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.327+0000 7f77d4842640 1 -- 172.21.15.96:0/4114960393 >> 172.21.15.96:0/4114960393 conn(0x7f77d00fdbd0 msgr2=0x7f77d0104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:00.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.327+0000 7f77d4842640 1 -- 172.21.15.96:0/4114960393 shutdown_connections 2024-04-25T14:13:00.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.327+0000 7f77d4842640 1 -- 172.21.15.96:0/4114960393 wait complete. 2024-04-25T14:13:00.338 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-04-25T14:13:00.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.447+0000 7fbf6ed82640 1 -- 172.21.15.96:0/233875789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf68109a40 msgr2=0x7fbf6810be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:00.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.447+0000 7fbf6ed82640 1 --2- 172.21.15.96:0/233875789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf68109a40 0x7fbf6810be60 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fbf60009ca0 tx=0x7fbf6002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.447+0000 7fbf6ed82640 1 -- 172.21.15.96:0/233875789 shutdown_connections 2024-04-25T14:13:00.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.447+0000 7fbf6ed82640 1 --2- 172.21.15.96:0/233875789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf68109a40 0x7fbf6810be60 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.447+0000 7fbf6ed82640 1 --2- 172.21.15.96:0/233875789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf68104d40 0x7fbf68105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.447+0000 7fbf6ed82640 1 --2- 172.21.15.96:0/233875789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf68103b00 0x7fbf68104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.447+0000 7fbf6ed82640 1 -- 172.21.15.96:0/233875789 >> 172.21.15.96:0/233875789 conn(0x7fbf680fed10 msgr2=0x7fbf681011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:00.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.447+0000 7fbf6ed82640 1 -- 172.21.15.96:0/233875789 shutdown_connections 2024-04-25T14:13:00.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.447+0000 7fbf6ed82640 1 -- 172.21.15.96:0/233875789 wait complete. 2024-04-25T14:13:00.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.447+0000 7fbf6ed82640 1 Processor -- start 2024-04-25T14:13:00.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.447+0000 7fbf6ed82640 1 -- start start 2024-04-25T14:13:00.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.448+0000 7fbf6ed82640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf68103b00 0x7fbf6819a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.448+0000 7fbf6ed82640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf68104d40 0x7fbf6819a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.448+0000 7fbf6ed82640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf68109a40 0x7fbf681a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.448+0000 7fbf6ed82640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6810f630 con 0x7fbf68104d40 2024-04-25T14:13:00.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.448+0000 7fbf6ed82640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6810f4b0 con 0x7fbf68109a40 2024-04-25T14:13:00.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.448+0000 7fbf6ed82640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6810f7b0 con 0x7fbf68103b00 2024-04-25T14:13:00.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.448+0000 7fbf67fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf68104d40 0x7fbf6819a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.448+0000 7fbf67fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf68104d40 0x7fbf6819a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51848/0 (socket says 172.21.15.96:51848) 2024-04-25T14:13:00.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.448+0000 7fbf67fff640 1 -- 172.21.15.96:0/788836626 learned_addr learned my addr 172.21.15.96:0/788836626 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:00.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.448+0000 7fbf6cb1e640 1 --2- 172.21.15.96:0/788836626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf68103b00 0x7fbf6819a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.448+0000 7fbf67fff640 1 -- 172.21.15.96:0/788836626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf68103b00 msgr2=0x7fbf6819a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:00.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.448+0000 7fbf67fff640 1 --2- 172.21.15.96:0/788836626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf68103b00 0x7fbf6819a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.448+0000 7fbf67fff640 1 -- 172.21.15.96:0/788836626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf68109a40 msgr2=0x7fbf681a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:00.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.448+0000 7fbf67fff640 1 --2- 172.21.15.96:0/788836626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf68109a40 0x7fbf681a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.448+0000 7fbf67fff640 1 -- 172.21.15.96:0/788836626 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf681a1d70 con 0x7fbf68104d40 2024-04-25T14:13:00.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.448+0000 7fbf67fff640 1 --2- 172.21.15.96:0/788836626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf68104d40 0x7fbf6819a7e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbf5400e9f0 tx=0x7fbf5400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:00.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.448+0000 7fbf65ffb640 1 -- 172.21.15.96:0/788836626 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5400cd80 con 0x7fbf68104d40 2024-04-25T14:13:00.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.448+0000 7fbf65ffb640 1 -- 172.21.15.96:0/788836626 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf540108e0 con 0x7fbf68104d40 2024-04-25T14:13:00.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.448+0000 7fbf6ed82640 1 -- 172.21.15.96:0/788836626 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf681a2060 con 0x7fbf68104d40 2024-04-25T14:13:00.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.448+0000 7fbf65ffb640 1 -- 172.21.15.96:0/788836626 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf54018980 con 0x7fbf68104d40 2024-04-25T14:13:00.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.449+0000 7fbf6ed82640 1 -- 172.21.15.96:0/788836626 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf681a25b0 con 0x7fbf68104d40 2024-04-25T14:13:00.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.451+0000 7fbf65ffb640 1 -- 172.21.15.96:0/788836626 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fbf54010430 con 0x7fbf68104d40 2024-04-25T14:13:00.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.451+0000 7fbf75c36740 1 Processor -- start 2024-04-25T14:13:00.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.451+0000 7fbf75c36740 1 -- start start 2024-04-25T14:13:00.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.451+0000 7fbf75c36740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567700b3270 0x556770458780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.451+0000 7fbf75c36740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55677008a3b0 0x556770458d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.451+0000 7fbf75c36740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556770092590 0x55677045fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.451+0000 7fbf75c36740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5567703b9740 con 0x55677008a3b0 2024-04-25T14:13:00.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.451+0000 7fbf75c36740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5567703553a0 con 0x5567700b3270 2024-04-25T14:13:00.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.451+0000 7fbf75c36740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5567703b5700 con 0x556770092590 2024-04-25T14:13:00.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.452+0000 7fbf6cb1e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55677008a3b0 0x556770458d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.452+0000 7fbf6cb1e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55677008a3b0 0x556770458d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51850/0 (socket says 172.21.15.96:51850) 2024-04-25T14:13:00.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.452+0000 7fbf6cb1e640 1 -- 172.21.15.96:0/2206073856 learned_addr learned my addr 172.21.15.96:0/2206073856 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:00.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.452+0000 7fbf67fff640 1 --2- 172.21.15.96:0/2206073856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556770092590 0x55677045fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.452+0000 7fbf65ffb640 1 --2- 172.21.15.96:0/788836626 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fbf580afaf0 0x7fbf580b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.452+0000 7fbf6d31f640 1 --2- 172.21.15.96:0/2206073856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567700b3270 0x556770458780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.452+0000 7fbf6cb1e640 1 -- 172.21.15.96:0/2206073856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556770092590 msgr2=0x55677045fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:00.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.452+0000 7fbf6cb1e640 1 --2- 172.21.15.96:0/2206073856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556770092590 0x55677045fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.452+0000 7fbf6cb1e640 1 -- 172.21.15.96:0/2206073856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567700b3270 msgr2=0x556770458780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:00.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.452+0000 7fbf6cb1e640 1 --2- 172.21.15.96:0/2206073856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567700b3270 0x556770458780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.452+0000 7fbf6cb1e640 1 -- 172.21.15.96:0/2206073856 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556770371750 con 0x55677008a3b0 2024-04-25T14:13:00.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.452+0000 7fbf6d31f640 1 --2- 172.21.15.96:0/2206073856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567700b3270 0x556770458780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:00.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.452+0000 7fbf6d31f640 1 --2- 172.21.15.96:0/788836626 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fbf580afaf0 0x7fbf580b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.452+0000 7fbf65ffb640 1 -- 172.21.15.96:0/788836626 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fbf540defd0 con 0x7fbf68104d40 2024-04-25T14:13:00.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.452+0000 7fbf6cb1e640 1 --2- 172.21.15.96:0/2206073856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55677008a3b0 0x556770458d00 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c00bc40 tx=0x7fbf5c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8479d922a557bee7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:00.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.452+0000 7fbf52ffd640 1 -- 172.21.15.96:0/2206073856 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c015070 con 0x55677008a3b0 2024-04-25T14:13:00.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.452+0000 7fbf52ffd640 1 -- 172.21.15.96:0/2206073856 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf5c01a910 con 0x55677008a3b0 2024-04-25T14:13:00.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.452+0000 7fbf52ffd640 1 -- 172.21.15.96:0/2206073856 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c0118e0 con 0x55677008a3b0 2024-04-25T14:13:00.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.453+0000 7fbf75c36740 1 -- 172.21.15.96:0/2206073856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55677008a3b0 msgr2=0x556770458d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:00.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.453+0000 7fbf6d31f640 1 --2- 172.21.15.96:0/788836626 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fbf580afaf0 0x7fbf580b2000 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbf6002ade0 tx=0x7fbf60009430 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:00.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.453+0000 7fbf75c36740 1 --2- 172.21.15.96:0/2206073856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55677008a3b0 0x556770458d00 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c00bc40 tx=0x7fbf5c00f730 comp rx=0 tx=0).stop 2024-04-25T14:13:00.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.453+0000 7fbf75c36740 1 -- 172.21.15.96:0/2206073856 shutdown_connections 2024-04-25T14:13:00.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.453+0000 7fbf75c36740 1 --2- 172.21.15.96:0/2206073856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556770092590 0x55677045fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.453+0000 7fbf75c36740 1 --2- 172.21.15.96:0/2206073856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55677008a3b0 0x556770458d00 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.453+0000 7fbf75c36740 1 --2- 172.21.15.96:0/2206073856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567700b3270 0x556770458780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.453+0000 7fbf75c36740 1 -- 172.21.15.96:0/2206073856 >> 172.21.15.96:0/2206073856 conn(0x5567700a73c0 msgr2=0x55677044fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:00.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.453+0000 7fbf75c36740 1 -- 172.21.15.96:0/2206073856 shutdown_connections 2024-04-25T14:13:00.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.453+0000 7fbf75c36740 1 -- 172.21.15.96:0/2206073856 wait complete. 2024-04-25T14:13:00.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.453+0000 7fbf75c36740 1 -- start start 2024-04-25T14:13:00.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.454+0000 7fbf75c36740 1 Processor -- start 2024-04-25T14:13:00.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.454+0000 7fbf75c36740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55677008a3b0 0x5567704626c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.454+0000 7fbf75c36740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556770092590 0x556770462c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.454+0000 7fbf75c36740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567700b3270 0x556770469b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.454+0000 7fbf75c36740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556770460650 con 0x55677008a3b0 2024-04-25T14:13:00.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.454+0000 7fbf75c36740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556770340fe0 con 0x5567700b3270 2024-04-25T14:13:00.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.454+0000 7fbf75c36740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5567704607d0 con 0x556770092590 2024-04-25T14:13:00.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.454+0000 7fbf6cb1e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55677008a3b0 0x5567704626c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.454+0000 7fbf75c36740 10 client.0 _ll_register_callbacks cb 0x556770357150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:13:00.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.454+0000 7fbf6cb1e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55677008a3b0 0x5567704626c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51856/0 (socket says 172.21.15.96:51856) 2024-04-25T14:13:00.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.454+0000 7fbf6cb1e640 1 -- 172.21.15.96:0/890861063 learned_addr learned my addr 172.21.15.96:0/890861063 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:00.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.454+0000 7fbf6cb1e640 1 -- 172.21.15.96:0/890861063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556770092590 msgr2=0x556770462c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:00.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.454+0000 7fbf6cb1e640 1 --2- 172.21.15.96:0/890861063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556770092590 0x556770462c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.454+0000 7fbf6cb1e640 1 -- 172.21.15.96:0/890861063 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567700b3270 msgr2=0x556770469b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:00.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.454+0000 7fbf6cb1e640 1 --2- 172.21.15.96:0/890861063 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567700b3270 0x556770469b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.454+0000 7fbf6cb1e640 1 -- 172.21.15.96:0/890861063 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567700cb2d0 con 0x55677008a3b0 2024-04-25T14:13:00.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.455+0000 7fbf6cb1e640 1 --2- 172.21.15.96:0/890861063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55677008a3b0 0x5567704626c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c01a690 tx=0x7fbf5c01a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e01c15212708060 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:00.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.455+0000 7fbf47fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:13:00.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.455+0000 7fbf47fff640 1 -- 172.21.15.96:0/890861063 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c015040 con 0x55677008a3b0 2024-04-25T14:13:00.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.455+0000 7fbf47fff640 1 -- 172.21.15.96:0/890861063 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fbf5c004040 con 0x55677008a3b0 2024-04-25T14:13:00.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.455+0000 7fbf75c36740 1 -- 172.21.15.96:0/890861063 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf5c01a910 con 0x55677008a3b0 2024-04-25T14:13:00.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.455+0000 7fbf47fff640 1 -- 172.21.15.96:0/890861063 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbf5c004240 con 0x55677008a3b0 2024-04-25T14:13:00.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.455+0000 7fbf75c36740 10 client.15108 fetch_fsmap learned FSMap version 25 2024-04-25T14:13:00.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.455+0000 7fbf75c36740 1 -- 172.21.15.96:0/890861063 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5567701956b0 con 0x55677008a3b0 2024-04-25T14:13:00.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.455+0000 7fbf47fff640 1 -- 172.21.15.96:0/890861063 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf5c01a910 con 0x55677008a3b0 2024-04-25T14:13:00.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.455+0000 7fbf47fff640 1 -- 172.21.15.96:0/890861063 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fbf5c004870 con 0x55677008a3b0 2024-04-25T14:13:00.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.455+0000 7fbf75c36740 10 client.15108 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T14:13:00.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.455+0000 7fbf75c36740 10 client.15108 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-25T14:13:00.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.455+0000 7fbf75c36740 20 client.15108 populate_metadata read hostname 'smithi096' 2024-04-25T14:13:00.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.455+0000 7fbf75c36740 1 --2- 172.21.15.96:0/890861063 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x556770071e40 0x556770470160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.455+0000 7fbf75c36740 4 client.15108 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:13:00.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.455+0000 7fbf75c36740 1 -- 172.21.15.96:0/890861063 --> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567703c7350 con 0x556770071e40 2024-04-25T14:13:00.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.455+0000 7fbf67fff640 1 --2- 172.21.15.96:0/890861063 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x556770071e40 0x556770470160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.456+0000 7fbf67fff640 1 --2- 172.21.15.96:0/890861063 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x556770071e40 0x556770470160 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9556a04eff152890 server_cookie=c0c4c2eee8a3baa9 in_seq=0 out_seq=0 2024-04-25T14:13:00.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.456+0000 7fbf47fff640 10 client.15108 ms_handle_connect on v2:172.21.15.151:6834/1313823638 2024-04-25T14:13:00.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.457+0000 7fbf47fff640 1 -- 172.21.15.96:0/890861063 <== mds.2 v2:172.21.15.151:6834/1313823638 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5567703c7350 con 0x556770071e40 2024-04-25T14:13:00.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.457+0000 7fbf47fff640 10 client.15108 handle_command_reply: tid=0 2024-04-25T14:13:00.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.457+0000 7fbf75c36740 1 client.15108 shutdown 2024-04-25T14:13:00.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.457+0000 7fbf75c36740 1 -- 172.21.15.96:0/890861063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55677008a3b0 msgr2=0x5567704626c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:00.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.457+0000 7fbf75c36740 1 --2- 172.21.15.96:0/890861063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55677008a3b0 0x5567704626c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c01a690 tx=0x7fbf5c01a6c0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.457+0000 7fbf75c36740 1 -- 172.21.15.96:0/890861063 shutdown_connections 2024-04-25T14:13:00.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.457+0000 7fbf75c36740 1 --2- 172.21.15.96:0/890861063 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567700b3270 0x556770469b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.457+0000 7fbf75c36740 1 --2- 172.21.15.96:0/890861063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556770092590 0x556770462c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.457+0000 7fbf75c36740 1 --2- 172.21.15.96:0/890861063 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x556770071e40 0x556770470160 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.457+0000 7fbf75c36740 1 --2- 172.21.15.96:0/890861063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55677008a3b0 0x5567704626c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.457+0000 7fbf47fff640 0 client.15108 ms_handle_reset on v2:172.21.15.151:6834/1313823638 2024-04-25T14:13:00.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.457+0000 7fbf75c36740 1 -- 172.21.15.96:0/890861063 >> 172.21.15.96:0/890861063 conn(0x5567700a73c0 msgr2=0x55677044ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:00.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.457+0000 7fbf75c36740 1 -- 172.21.15.96:0/890861063 shutdown_connections 2024-04-25T14:13:00.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.457+0000 7fbf75c36740 1 -- 172.21.15.96:0/890861063 wait complete. 2024-04-25T14:13:00.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.457+0000 7fbf75c36740 20 client.15108 trim_cache size 0 max 16384 2024-04-25T14:13:00.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.476+0000 7fbf75c36740 1 Processor -- start 2024-04-25T14:13:00.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.476+0000 7fbf75c36740 1 -- start start 2024-04-25T14:13:00.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.476+0000 7fbf75c36740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55677008a3b0 0x55677044f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.476+0000 7fbf75c36740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556770092590 0x55677044fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.476+0000 7fbf75c36740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567700b3270 0x556770469a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.476+0000 7fbf75c36740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55677046e4a0 con 0x556770092590 2024-04-25T14:13:00.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.476+0000 7fbf75c36740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55677046eb70 con 0x5567700b3270 2024-04-25T14:13:00.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.476+0000 7fbf75c36740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55677046f240 con 0x55677008a3b0 2024-04-25T14:13:00.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.476+0000 7fbf67fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556770092590 0x55677044fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.476+0000 7fbf67fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556770092590 0x55677044fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51872/0 (socket says 172.21.15.96:51872) 2024-04-25T14:13:00.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.476+0000 7fbf67fff640 1 -- 172.21.15.96:0/440330711 learned_addr learned my addr 172.21.15.96:0/440330711 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:00.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.476+0000 7fbf6d31f640 1 --2- 172.21.15.96:0/440330711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567700b3270 0x556770469a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.476+0000 7fbf67fff640 1 -- 172.21.15.96:0/440330711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55677008a3b0 msgr2=0x55677044f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:00.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.476+0000 7fbf67fff640 1 --2- 172.21.15.96:0/440330711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55677008a3b0 0x55677044f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.476+0000 7fbf67fff640 1 -- 172.21.15.96:0/440330711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567700b3270 msgr2=0x556770469a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:00.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.476+0000 7fbf67fff640 1 --2- 172.21.15.96:0/440330711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567700b3270 0x556770469a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.476+0000 7fbf67fff640 1 -- 172.21.15.96:0/440330711 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55677045d4d0 con 0x556770092590 2024-04-25T14:13:00.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.476+0000 7fbf67fff640 1 --2- 172.21.15.96:0/440330711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556770092590 0x55677044fbf0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fbf540a7230 tx=0x7fbf540dca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdc3e4b098a97e02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:00.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.476+0000 7fbf527fc640 1 -- 172.21.15.96:0/440330711 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf540a7040 con 0x556770092590 2024-04-25T14:13:00.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.476+0000 7fbf527fc640 1 -- 172.21.15.96:0/440330711 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf540a7c30 con 0x556770092590 2024-04-25T14:13:00.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.476+0000 7fbf527fc640 1 -- 172.21.15.96:0/440330711 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf540e10b0 con 0x556770092590 2024-04-25T14:13:00.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.477+0000 7fbf75c36740 1 -- 172.21.15.96:0/440330711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556770092590 msgr2=0x55677044fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:00.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.477+0000 7fbf75c36740 1 --2- 172.21.15.96:0/440330711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556770092590 0x55677044fbf0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fbf540a7230 tx=0x7fbf540dca20 comp rx=0 tx=0).stop 2024-04-25T14:13:00.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.477+0000 7fbf75c36740 1 -- 172.21.15.96:0/440330711 shutdown_connections 2024-04-25T14:13:00.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.477+0000 7fbf75c36740 1 --2- 172.21.15.96:0/440330711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567700b3270 0x556770469a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.477+0000 7fbf75c36740 1 --2- 172.21.15.96:0/440330711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556770092590 0x55677044fbf0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.477+0000 7fbf75c36740 1 --2- 172.21.15.96:0/440330711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55677008a3b0 0x55677044f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.477+0000 7fbf75c36740 1 -- 172.21.15.96:0/440330711 >> 172.21.15.96:0/440330711 conn(0x5567700a73c0 msgr2=0x556770460c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:00.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.477+0000 7fbf75c36740 1 -- 172.21.15.96:0/440330711 shutdown_connections 2024-04-25T14:13:00.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.477+0000 7fbf75c36740 1 -- 172.21.15.96:0/440330711 wait complete. 2024-04-25T14:13:00.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.478+0000 7fbf75c36740 1 -- start start 2024-04-25T14:13:00.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.478+0000 7fbf75c36740 1 Processor -- start 2024-04-25T14:13:00.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.478+0000 7fbf75c36740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55677008a3b0 0x556770465a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.478+0000 7fbf75c36740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556770092590 0x556770465fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.478+0000 7fbf75c36740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567700b3270 0x556770468f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.478+0000 7fbf75c36740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55677046ffe0 con 0x5567700b3270 2024-04-25T14:13:00.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.478+0000 7fbf75c36740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55677046f910 con 0x556770092590 2024-04-25T14:13:00.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.478+0000 7fbf75c36740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5567704706b0 con 0x55677008a3b0 2024-04-25T14:13:00.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.478+0000 7fbf75c36740 10 client.0 _ll_register_callbacks cb 0x7fbf5c00ba90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:13:00.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.478+0000 7fbf6d31f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567700b3270 0x556770468f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.478+0000 7fbf6d31f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567700b3270 0x556770468f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51882/0 (socket says 172.21.15.96:51882) 2024-04-25T14:13:00.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.478+0000 7fbf6d31f640 1 -- 172.21.15.96:0/3942781370 learned_addr learned my addr 172.21.15.96:0/3942781370 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:00.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.478+0000 7fbf67fff640 1 --2- 172.21.15.96:0/3942781370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556770092590 0x556770465fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.478+0000 7fbf6d31f640 1 -- 172.21.15.96:0/3942781370 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55677008a3b0 msgr2=0x556770465a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:00.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.478+0000 7fbf6d31f640 1 --2- 172.21.15.96:0/3942781370 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55677008a3b0 0x556770465a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.478+0000 7fbf6d31f640 1 -- 172.21.15.96:0/3942781370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556770092590 msgr2=0x556770465fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:00.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.478+0000 7fbf6d31f640 1 --2- 172.21.15.96:0/3942781370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556770092590 0x556770465fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.478+0000 7fbf6d31f640 1 -- 172.21.15.96:0/3942781370 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567704694c0 con 0x5567700b3270 2024-04-25T14:13:00.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.479+0000 7fbf6d31f640 1 --2- 172.21.15.96:0/3942781370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567700b3270 0x556770468f00 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbf6002f910 tx=0x7fbf60005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=790b5d48ce071682 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:00.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.479+0000 7fbf47fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:13:00.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.479+0000 7fbf47fff640 1 -- 172.21.15.96:0/3942781370 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf60041070 con 0x5567700b3270 2024-04-25T14:13:00.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.479+0000 7fbf47fff640 1 -- 172.21.15.96:0/3942781370 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fbf6002fdd0 con 0x5567700b3270 2024-04-25T14:13:00.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.479+0000 7fbf75c36740 1 -- 172.21.15.96:0/3942781370 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf540a7c30 con 0x5567700b3270 2024-04-25T14:13:00.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.479+0000 7fbf47fff640 1 -- 172.21.15.96:0/3942781370 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbf600318e0 con 0x5567700b3270 2024-04-25T14:13:00.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.479+0000 7fbf75c36740 10 client.15114 fetch_fsmap learned FSMap version 25 2024-04-25T14:13:00.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.479+0000 7fbf75c36740 1 -- 172.21.15.96:0/3942781370 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5567704d8fe0 con 0x5567700b3270 2024-04-25T14:13:00.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.479+0000 7fbf47fff640 1 -- 172.21.15.96:0/3942781370 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf540a7c30 con 0x5567700b3270 2024-04-25T14:13:00.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.479+0000 7fbf47fff640 1 -- 172.21.15.96:0/3942781370 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fbf60050bc0 con 0x5567700b3270 2024-04-25T14:13:00.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.479+0000 7fbf75c36740 10 client.15114 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T14:13:00.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.479+0000 7fbf75c36740 10 client.15114 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-25T14:13:00.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.479+0000 7fbf75c36740 20 client.15114 populate_metadata read hostname 'smithi096' 2024-04-25T14:13:00.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.479+0000 7fbf75c36740 1 --2- 172.21.15.96:0/3942781370 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x556770047280 0x5567704dc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.479+0000 7fbf75c36740 4 client.15114 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T14:13:00.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.479+0000 7fbf75c36740 1 -- 172.21.15.96:0/3942781370 --> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] -- command(tid 0: {"prefix": "status"}) -- 0x5567703c11c0 con 0x556770047280 2024-04-25T14:13:00.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.480+0000 7fbf6cb1e640 1 --2- 172.21.15.96:0/3942781370 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x556770047280 0x5567704dc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.480+0000 7fbf47fff640 10 client.15114 ms_handle_connect on v2:172.21.15.151:6834/1313823638 2024-04-25T14:13:00.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.480+0000 7fbf6cb1e640 1 --2- 172.21.15.96:0/3942781370 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x556770047280 0x5567704dc8c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=47d15d11e5266d78 server_cookie=577d8a3a880744c0 in_seq=0 out_seq=0 2024-04-25T14:13:00.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.480+0000 7fbf47fff640 1 -- 172.21.15.96:0/3942781370 <== mds.2 v2:172.21.15.151:6834/1313823638 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5567703c11c0 con 0x556770047280 2024-04-25T14:13:00.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.480+0000 7fbf47fff640 10 client.15114 handle_command_reply: tid=0 2024-04-25T14:13:00.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.480+0000 7fbf75c36740 1 client.15114 shutdown 2024-04-25T14:13:00.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.480+0000 7fbf75c36740 1 -- 172.21.15.96:0/3942781370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567700b3270 msgr2=0x556770468f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:00.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.480+0000 7fbf75c36740 1 --2- 172.21.15.96:0/3942781370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567700b3270 0x556770468f00 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbf6002f910 tx=0x7fbf60005fd0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.480+0000 7fbf75c36740 1 -- 172.21.15.96:0/3942781370 shutdown_connections 2024-04-25T14:13:00.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.480+0000 7fbf75c36740 1 --2- 172.21.15.96:0/3942781370 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x556770047280 0x5567704dc8c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.480+0000 7fbf75c36740 1 --2- 172.21.15.96:0/3942781370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567700b3270 0x556770468f00 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.480+0000 7fbf75c36740 1 --2- 172.21.15.96:0/3942781370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556770092590 0x556770465fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.480+0000 7fbf47fff640 0 client.15114 ms_handle_reset on v2:172.21.15.151:6834/1313823638 2024-04-25T14:13:00.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.480+0000 7fbf75c36740 1 --2- 172.21.15.96:0/3942781370 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55677008a3b0 0x556770465a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.480+0000 7fbf75c36740 1 -- 172.21.15.96:0/3942781370 >> 172.21.15.96:0/3942781370 conn(0x5567700a73c0 msgr2=0x55677044f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:00.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.480+0000 7fbf75c36740 1 -- 172.21.15.96:0/3942781370 shutdown_connections 2024-04-25T14:13:00.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.481+0000 7fbf75c36740 1 -- 172.21.15.96:0/3942781370 wait complete. 2024-04-25T14:13:00.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.481+0000 7fbf75c36740 20 client.15114 trim_cache size 0 max 16384 2024-04-25T14:13:00.489 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T14:13:00.489 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "59334182-030c-11ef-bc93-c7b262605968", 2024-04-25T14:13:00.489 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 2, 2024-04-25T14:13:00.489 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 24478, 2024-04-25T14:13:00.490 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-04-25T14:13:00.490 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-04-25T14:13:00.490 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-04-25T14:13:00.490 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 87.967815302999995, 2024-04-25T14:13:00.490 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 25, 2024-04-25T14:13:00.490 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 78, 2024-04-25T14:13:00.490 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 76, 2024-04-25T14:13:00.490 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 114.320416632 2024-04-25T14:13:00.490 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T14:13:00.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.481+0000 7fbf6ed82640 1 -- 172.21.15.96:0/788836626 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fbf580afaf0 msgr2=0x7fbf580b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:00.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.481+0000 7fbf6ed82640 1 --2- 172.21.15.96:0/788836626 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fbf580afaf0 0x7fbf580b2000 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbf6002ade0 tx=0x7fbf60009430 comp rx=0 tx=0).stop 2024-04-25T14:13:00.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.481+0000 7fbf6ed82640 1 -- 172.21.15.96:0/788836626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf68104d40 msgr2=0x7fbf6819a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:00.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.481+0000 7fbf6ed82640 1 --2- 172.21.15.96:0/788836626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf68104d40 0x7fbf6819a7e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbf5400e9f0 tx=0x7fbf5400eeb0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.481+0000 7fbf6ed82640 1 -- 172.21.15.96:0/788836626 shutdown_connections 2024-04-25T14:13:00.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.481+0000 7fbf6ed82640 1 --2- 172.21.15.96:0/788836626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf68109a40 0x7fbf681a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.481+0000 7fbf6ed82640 1 --2- 172.21.15.96:0/788836626 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fbf580afaf0 0x7fbf580b2000 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.481+0000 7fbf6ed82640 1 --2- 172.21.15.96:0/788836626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf68104d40 0x7fbf6819a7e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.481+0000 7fbf6ed82640 1 --2- 172.21.15.96:0/788836626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf68103b00 0x7fbf6819a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.481+0000 7fbf6ed82640 1 -- 172.21.15.96:0/788836626 >> 172.21.15.96:0/788836626 conn(0x7fbf680fed10 msgr2=0x7fbf68105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:00.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.481+0000 7fbf6ed82640 1 -- 172.21.15.96:0/788836626 shutdown_connections 2024-04-25T14:13:00.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.481+0000 7fbf6ed82640 1 -- 172.21.15.96:0/788836626 wait complete. 2024-04-25T14:13:00.493 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-25T14:13:00.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.603+0000 7f711efce640 1 -- 172.21.15.96:0/2523049705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7118102f60 msgr2=0x7f7118103370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:00.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.603+0000 7f711efce640 1 --2- 172.21.15.96:0/2523049705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7118102f60 0x7f7118103370 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7108009900 tx=0x7f710802a410 comp rx=0 tx=0).stop 2024-04-25T14:13:00.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.603+0000 7f711efce640 1 -- 172.21.15.96:0/2523049705 shutdown_connections 2024-04-25T14:13:00.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.603+0000 7f711efce640 1 --2- 172.21.15.96:0/2523049705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7118104480 0x7f711810abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.603+0000 7f711efce640 1 --2- 172.21.15.96:0/2523049705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7118103940 0x7f7118103df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.603+0000 7f711efce640 1 --2- 172.21.15.96:0/2523049705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7118102f60 0x7f7118103370 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.603+0000 7f711efce640 1 -- 172.21.15.96:0/2523049705 >> 172.21.15.96:0/2523049705 conn(0x7f71180fd380 msgr2=0x7f71180ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:00.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.603+0000 7f711efce640 1 -- 172.21.15.96:0/2523049705 shutdown_connections 2024-04-25T14:13:00.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.603+0000 7f711efce640 1 -- 172.21.15.96:0/2523049705 wait complete. 2024-04-25T14:13:00.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.604+0000 7f711efce640 1 Processor -- start 2024-04-25T14:13:00.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.604+0000 7f711efce640 1 -- start start 2024-04-25T14:13:00.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.604+0000 7f711efce640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7118102f60 0x7f7118102200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.604+0000 7f711efce640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7118103940 0x7f711819efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.604+0000 7f711efce640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7118104480 0x7f71181a1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.604+0000 7f711efce640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f711810e4f0 con 0x7f7118102f60 2024-04-25T14:13:00.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.604+0000 7f711efce640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f711810e370 con 0x7f7118104480 2024-04-25T14:13:00.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.604+0000 7f711efce640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f711810e670 con 0x7f7118103940 2024-04-25T14:13:00.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.604+0000 7f711cd6a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7118102f60 0x7f7118102200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.604+0000 7f711cd6a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7118102f60 0x7f7118102200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51904/0 (socket says 172.21.15.96:51904) 2024-04-25T14:13:00.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.604+0000 7f711cd6a640 1 -- 172.21.15.96:0/2562688174 learned_addr learned my addr 172.21.15.96:0/2562688174 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:00.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.604+0000 7f711d56b640 1 --2- 172.21.15.96:0/2562688174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7118104480 0x7f71181a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.604+0000 7f7117fff640 1 --2- 172.21.15.96:0/2562688174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7118103940 0x7f711819efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.604+0000 7f711cd6a640 1 -- 172.21.15.96:0/2562688174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7118103940 msgr2=0x7f711819efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:00.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.604+0000 7f711cd6a640 1 --2- 172.21.15.96:0/2562688174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7118103940 0x7f711819efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.604+0000 7f711cd6a640 1 -- 172.21.15.96:0/2562688174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7118104480 msgr2=0x7f71181a1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:00.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.604+0000 7f711cd6a640 1 --2- 172.21.15.96:0/2562688174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7118104480 0x7f71181a1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.604+0000 7f711cd6a640 1 -- 172.21.15.96:0/2562688174 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71181a1cc0 con 0x7f7118102f60 2024-04-25T14:13:00.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.604+0000 7f7117fff640 1 --2- 172.21.15.96:0/2562688174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7118103940 0x7f711819efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:00.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.604+0000 7f711cd6a640 1 --2- 172.21.15.96:0/2562688174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7118102f60 0x7f7118102200 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7108006010 tx=0x7f7108004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:00.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.605+0000 7f7115ffb640 1 -- 172.21.15.96:0/2562688174 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710803f030 con 0x7f7118102f60 2024-04-25T14:13:00.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.605+0000 7f711efce640 1 -- 172.21.15.96:0/2562688174 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71181a1f50 con 0x7f7118102f60 2024-04-25T14:13:00.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.605+0000 7f7115ffb640 1 -- 172.21.15.96:0/2562688174 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7108003aa0 con 0x7f7118102f60 2024-04-25T14:13:00.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.605+0000 7f7115ffb640 1 -- 172.21.15.96:0/2562688174 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7108035070 con 0x7f7118102f60 2024-04-25T14:13:00.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.605+0000 7f711efce640 1 -- 172.21.15.96:0/2562688174 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71181a23b0 con 0x7f7118102f60 2024-04-25T14:13:00.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.607+0000 7f7125e83740 1 Processor -- start 2024-04-25T14:13:00.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.607+0000 7f7115ffb640 1 -- 172.21.15.96:0/2562688174 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f7108042030 con 0x7f7118102f60 2024-04-25T14:13:00.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.607+0000 7f7125e83740 1 -- start start 2024-04-25T14:13:00.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.607+0000 7f7125e83740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651dcef1270 0x5651dd292370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.607+0000 7f7125e83740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651dcec83b0 0x5651dd2928f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.607+0000 7f7125e83740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651dced0590 0x5651dd2997f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.607+0000 7f7125e83740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5651dd1f7740 con 0x5651dcef1270 2024-04-25T14:13:00.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.607+0000 7f7125e83740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5651dd1933a0 con 0x5651dcec83b0 2024-04-25T14:13:00.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.607+0000 7f7125e83740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5651dd1f3700 con 0x5651dced0590 2024-04-25T14:13:00.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.607+0000 7f711d56b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651dcef1270 0x5651dd292370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.608+0000 7f711d56b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651dcef1270 0x5651dd292370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51914/0 (socket says 172.21.15.96:51914) 2024-04-25T14:13:00.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.608+0000 7f711d56b640 1 -- 172.21.15.96:0/733641733 learned_addr learned my addr 172.21.15.96:0/733641733 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:00.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.608+0000 7f7117fff640 1 --2- 172.21.15.96:0/733641733 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651dced0590 0x5651dd2997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.608+0000 7f711cd6a640 1 --2- 172.21.15.96:0/733641733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651dcec83b0 0x5651dd2928f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.608+0000 7f711d56b640 1 -- 172.21.15.96:0/733641733 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651dced0590 msgr2=0x5651dd2997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:00.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.608+0000 7f711d56b640 1 --2- 172.21.15.96:0/733641733 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651dced0590 0x5651dd2997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.608+0000 7f711d56b640 1 -- 172.21.15.96:0/733641733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651dcec83b0 msgr2=0x5651dd2928f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:00.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.608+0000 7f711d56b640 1 --2- 172.21.15.96:0/733641733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651dcec83b0 0x5651dd2928f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.608+0000 7f711d56b640 1 -- 172.21.15.96:0/733641733 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651dd1af750 con 0x5651dcef1270 2024-04-25T14:13:00.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.608+0000 7f7115ffb640 1 --2- 172.21.15.96:0/2562688174 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f71000afaf0 0x7f71000b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.608+0000 7f711d56b640 1 --2- 172.21.15.96:0/733641733 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651dcef1270 0x5651dd292370 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7110007f40 tx=0x7f711000f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93693991f4e2bcf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:00.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.608+0000 7f711cd6a640 1 --2- 172.21.15.96:0/2562688174 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f71000afaf0 0x7f71000b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.608+0000 7f70feffd640 1 -- 172.21.15.96:0/733641733 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7110010900 con 0x5651dcef1270 2024-04-25T14:13:00.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.608+0000 7f7115ffb640 1 -- 172.21.15.96:0/2562688174 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f71080feff0 con 0x7f7118102f60 2024-04-25T14:13:00.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.608+0000 7f70feffd640 1 -- 172.21.15.96:0/733641733 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71100183f0 con 0x5651dcef1270 2024-04-25T14:13:00.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.608+0000 7f70feffd640 1 -- 172.21.15.96:0/733641733 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7110017600 con 0x5651dcef1270 2024-04-25T14:13:00.612 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:00 smithi096 ceph-mon[29207]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-25T14:13:00.612 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:00 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/2794028549' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:13:00.612 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:00 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/655939224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:13:00.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.609+0000 7f711cd6a640 1 --2- 172.21.15.96:0/2562688174 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f71000afaf0 0x7f71000b2000 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f71080c47c0 tx=0x7f7108102aa0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:00.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.609+0000 7f7125e83740 1 -- 172.21.15.96:0/733641733 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651dcef1270 msgr2=0x5651dd292370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:00.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.609+0000 7f7125e83740 1 --2- 172.21.15.96:0/733641733 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651dcef1270 0x5651dd292370 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7110007f40 tx=0x7f711000f850 comp rx=0 tx=0).stop 2024-04-25T14:13:00.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.609+0000 7f7125e83740 1 -- 172.21.15.96:0/733641733 shutdown_connections 2024-04-25T14:13:00.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.609+0000 7f7125e83740 1 --2- 172.21.15.96:0/733641733 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651dced0590 0x5651dd2997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.609+0000 7f7125e83740 1 --2- 172.21.15.96:0/733641733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651dcec83b0 0x5651dd2928f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.609+0000 7f7125e83740 1 --2- 172.21.15.96:0/733641733 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651dcef1270 0x5651dd292370 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.609+0000 7f7125e83740 1 -- 172.21.15.96:0/733641733 >> 172.21.15.96:0/733641733 conn(0x5651dcee53c0 msgr2=0x5651dd28dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:00.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.609+0000 7f7125e83740 1 -- 172.21.15.96:0/733641733 shutdown_connections 2024-04-25T14:13:00.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.609+0000 7f7125e83740 1 -- 172.21.15.96:0/733641733 wait complete. 2024-04-25T14:13:00.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.610+0000 7f7125e83740 1 -- start start 2024-04-25T14:13:00.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.610+0000 7f7125e83740 1 Processor -- start 2024-04-25T14:13:00.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.610+0000 7f7125e83740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651dcec83b0 0x5651dd29c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.610+0000 7f7125e83740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651dced0590 0x5651dd29c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.610+0000 7f7125e83740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651dcef1270 0x5651dd2a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.610+0000 7f7125e83740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5651dd29a240 con 0x5651dcec83b0 2024-04-25T14:13:00.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.610+0000 7f7125e83740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5651dd17efe0 con 0x5651dcef1270 2024-04-25T14:13:00.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.610+0000 7f7125e83740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5651dd29a3c0 con 0x5651dced0590 2024-04-25T14:13:00.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.610+0000 7f7125e83740 10 client.0 _ll_register_callbacks cb 0x5651dd195150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:13:00.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.610+0000 7f711d56b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651dcec83b0 0x5651dd29c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.610+0000 7f711d56b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651dcec83b0 0x5651dd29c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51920/0 (socket says 172.21.15.96:51920) 2024-04-25T14:13:00.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.610+0000 7f711d56b640 1 -- 172.21.15.96:0/1893962340 learned_addr learned my addr 172.21.15.96:0/1893962340 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:00.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.610+0000 7f7117fff640 1 --2- 172.21.15.96:0/1893962340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651dced0590 0x5651dd29c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.610+0000 7f711d56b640 1 --2- 172.21.15.96:0/1893962340 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651dcef1270 0x5651dd2a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.611+0000 7f711d56b640 1 -- 172.21.15.96:0/1893962340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651dced0590 msgr2=0x5651dd29c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:00.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.611+0000 7f711d56b640 1 --2- 172.21.15.96:0/1893962340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651dced0590 0x5651dd29c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.611+0000 7f711d56b640 1 -- 172.21.15.96:0/1893962340 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651dcef1270 msgr2=0x5651dd2a36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:00.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.611+0000 7f711d56b640 1 --2- 172.21.15.96:0/1893962340 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651dcef1270 0x5651dd2a36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.611+0000 7f711d56b640 1 -- 172.21.15.96:0/1893962340 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651dcf092d0 con 0x5651dcec83b0 2024-04-25T14:13:00.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.611+0000 7f7117fff640 1 --2- 172.21.15.96:0/1893962340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651dced0590 0x5651dd29c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:00.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.611+0000 7f711d56b640 1 --2- 172.21.15.96:0/1893962340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651dcec83b0 0x5651dd29c250 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7110005c40 tx=0x7f71100072a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99063544690fdbcf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:00.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.611+0000 7f70f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:13:00.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.611+0000 7f70f7fff640 1 -- 172.21.15.96:0/1893962340 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711002b060 con 0x5651dcec83b0 2024-04-25T14:13:00.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.611+0000 7f7125e83740 1 -- 172.21.15.96:0/1893962340 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71100183f0 con 0x5651dcec83b0 2024-04-25T14:13:00.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.611+0000 7f70f7fff640 1 -- 172.21.15.96:0/1893962340 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f7110004c40 con 0x5651dcec83b0 2024-04-25T14:13:00.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.611+0000 7f70f7fff640 1 -- 172.21.15.96:0/1893962340 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7110004e40 con 0x5651dcec83b0 2024-04-25T14:13:00.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.611+0000 7f7125e83740 10 client.15126 fetch_fsmap learned FSMap version 25 2024-04-25T14:13:00.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.611+0000 7f7125e83740 1 -- 172.21.15.96:0/1893962340 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5651dcfd36b0 con 0x5651dcec83b0 2024-04-25T14:13:00.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.611+0000 7f70f7fff640 1 -- 172.21.15.96:0/1893962340 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71100183f0 con 0x5651dcec83b0 2024-04-25T14:13:00.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.611+0000 7f70f7fff640 1 -- 172.21.15.96:0/1893962340 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f71100042a0 con 0x5651dcec83b0 2024-04-25T14:13:00.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.611+0000 7f7125e83740 10 client.15126 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T14:13:00.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.611+0000 7f7125e83740 10 client.15126 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T14:13:00.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.612+0000 7f7125e83740 20 client.15126 populate_metadata read hostname 'smithi096' 2024-04-25T14:13:00.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.612+0000 7f7125e83740 1 --2- 172.21.15.96:0/1893962340 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x5651dceafe40 0x5651dd2a9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.612+0000 7f7125e83740 4 client.15126 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:13:00.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.612+0000 7f7125e83740 1 -- 172.21.15.96:0/1893962340 --> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651dd205350 con 0x5651dceafe40 2024-04-25T14:13:00.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.612+0000 7f7117fff640 1 --2- 172.21.15.96:0/1893962340 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x5651dceafe40 0x5651dd2a9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.612+0000 7f70f7fff640 10 client.15126 ms_handle_connect on v2:172.21.15.151:6838/245699981 2024-04-25T14:13:00.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.612+0000 7f7117fff640 1 --2- 172.21.15.96:0/1893962340 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x5651dceafe40 0x5651dd2a9c10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c926a7bc6d0550cf server_cookie=b22f786c4c5b18d4 in_seq=0 out_seq=0 2024-04-25T14:13:00.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.613+0000 7f70f7fff640 1 -- 172.21.15.96:0/1893962340 <== mds.1 v2:172.21.15.151:6838/245699981 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5651dd205350 con 0x5651dceafe40 2024-04-25T14:13:00.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.613+0000 7f70f7fff640 10 client.15126 handle_command_reply: tid=0 2024-04-25T14:13:00.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.613+0000 7f7125e83740 1 client.15126 shutdown 2024-04-25T14:13:00.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.613+0000 7f7125e83740 1 -- 172.21.15.96:0/1893962340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651dcec83b0 msgr2=0x5651dd29c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:00.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.613+0000 7f7125e83740 1 --2- 172.21.15.96:0/1893962340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651dcec83b0 0x5651dd29c250 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7110005c40 tx=0x7f71100072a0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.613+0000 7f7125e83740 1 -- 172.21.15.96:0/1893962340 shutdown_connections 2024-04-25T14:13:00.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.613+0000 7f7125e83740 1 --2- 172.21.15.96:0/1893962340 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x5651dceafe40 0x5651dd2a9c10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.613+0000 7f7125e83740 1 --2- 172.21.15.96:0/1893962340 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651dcef1270 0x5651dd2a36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.613+0000 7f7125e83740 1 --2- 172.21.15.96:0/1893962340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651dced0590 0x5651dd29c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.613+0000 7f7125e83740 1 --2- 172.21.15.96:0/1893962340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651dcec83b0 0x5651dd29c250 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.613+0000 7f70f7fff640 0 client.15126 ms_handle_reset on v2:172.21.15.151:6838/245699981 2024-04-25T14:13:00.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.613+0000 7f7125e83740 1 -- 172.21.15.96:0/1893962340 >> 172.21.15.96:0/1893962340 conn(0x5651dcee53c0 msgr2=0x5651dd28cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:00.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.613+0000 7f7125e83740 1 -- 172.21.15.96:0/1893962340 shutdown_connections 2024-04-25T14:13:00.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.613+0000 7f7125e83740 1 -- 172.21.15.96:0/1893962340 wait complete. 2024-04-25T14:13:00.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.614+0000 7f7125e83740 20 client.15126 trim_cache size 0 max 16384 2024-04-25T14:13:00.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.632+0000 7f7125e83740 1 Processor -- start 2024-04-25T14:13:00.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.632+0000 7f7125e83740 1 -- start start 2024-04-25T14:13:00.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.632+0000 7f7125e83740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651dcec83b0 0x5651dd29af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.632+0000 7f7125e83740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651dced0590 0x5651dd292f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.632+0000 7f7125e83740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651dcef1270 0x5651dd2a3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.632+0000 7f7125e83740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5651dd2a7f50 con 0x5651dcef1270 2024-04-25T14:13:00.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.632+0000 7f7125e83740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5651dd2a8620 con 0x5651dced0590 2024-04-25T14:13:00.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.632+0000 7f7125e83740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5651dd2a8cf0 con 0x5651dcec83b0 2024-04-25T14:13:00.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.632+0000 7f711d56b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651dcef1270 0x5651dd2a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.632+0000 7f711d56b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651dcef1270 0x5651dd2a3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51932/0 (socket says 172.21.15.96:51932) 2024-04-25T14:13:00.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.632+0000 7f711d56b640 1 -- 172.21.15.96:0/1829058670 learned_addr learned my addr 172.21.15.96:0/1829058670 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:00.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.633+0000 7f7117fff640 1 --2- 172.21.15.96:0/1829058670 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651dced0590 0x5651dd292f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.633+0000 7f711d56b640 1 --2- 172.21.15.96:0/1829058670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651dcec83b0 0x5651dd29af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.633+0000 7f711d56b640 1 -- 172.21.15.96:0/1829058670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651dcec83b0 msgr2=0x5651dd29af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:00.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.633+0000 7f711d56b640 1 --2- 172.21.15.96:0/1829058670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651dcec83b0 0x5651dd29af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.633+0000 7f711d56b640 1 -- 172.21.15.96:0/1829058670 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651dced0590 msgr2=0x5651dd292f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:00.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.633+0000 7f711d56b640 1 --2- 172.21.15.96:0/1829058670 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651dced0590 0x5651dd292f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.633+0000 7f711d56b640 1 -- 172.21.15.96:0/1829058670 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651dd2a6530 con 0x5651dcef1270 2024-04-25T14:13:00.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.633+0000 7f711d56b640 1 --2- 172.21.15.96:0/1829058670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651dcef1270 0x5651dd2a3680 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7110007390 tx=0x7f7110019fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=176a7dbf4321b187 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:00.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.633+0000 7f70fe7fc640 1 -- 172.21.15.96:0/1829058670 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711002b040 con 0x5651dcef1270 2024-04-25T14:13:00.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.633+0000 7f70fe7fc640 1 -- 172.21.15.96:0/1829058670 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71100183f0 con 0x5651dcef1270 2024-04-25T14:13:00.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.633+0000 7f70fe7fc640 1 -- 172.21.15.96:0/1829058670 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7110010700 con 0x5651dcef1270 2024-04-25T14:13:00.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.634+0000 7f7125e83740 1 -- 172.21.15.96:0/1829058670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651dcef1270 msgr2=0x5651dd2a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:00.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.634+0000 7f7125e83740 1 --2- 172.21.15.96:0/1829058670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651dcef1270 0x5651dd2a3680 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7110007390 tx=0x7f7110019fe0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.634+0000 7f7125e83740 1 -- 172.21.15.96:0/1829058670 shutdown_connections 2024-04-25T14:13:00.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.634+0000 7f7125e83740 1 --2- 172.21.15.96:0/1829058670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651dcef1270 0x5651dd2a3680 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.634+0000 7f7125e83740 1 --2- 172.21.15.96:0/1829058670 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651dced0590 0x5651dd292f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.634+0000 7f7125e83740 1 --2- 172.21.15.96:0/1829058670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651dcec83b0 0x5651dd29af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.634+0000 7f7125e83740 1 -- 172.21.15.96:0/1829058670 >> 172.21.15.96:0/1829058670 conn(0x5651dcee53c0 msgr2=0x5651dd29a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:00.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.634+0000 7f7125e83740 1 -- 172.21.15.96:0/1829058670 shutdown_connections 2024-04-25T14:13:00.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.634+0000 7f7125e83740 1 -- 172.21.15.96:0/1829058670 wait complete. 2024-04-25T14:13:00.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.634+0000 7f7125e83740 1 -- start start 2024-04-25T14:13:00.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.634+0000 7f7125e83740 1 Processor -- start 2024-04-25T14:13:00.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.634+0000 7f7125e83740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651dcec83b0 0x5651dd29f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.634+0000 7f7125e83740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651dced0590 0x5651dd29f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.634+0000 7f7125e83740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651dcef1270 0x5651dd2a26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.634+0000 7f7125e83740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5651dd2a9a90 con 0x5651dcec83b0 2024-04-25T14:13:00.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.634+0000 7f7125e83740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5651dd2a93c0 con 0x5651dcef1270 2024-04-25T14:13:00.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.634+0000 7f7125e83740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5651dd2aa160 con 0x5651dced0590 2024-04-25T14:13:00.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.635+0000 7f7125e83740 10 client.0 _ll_register_callbacks cb 0x7f710c0095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:13:00.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.635+0000 7f711d56b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651dcec83b0 0x5651dd29f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.635+0000 7f711d56b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651dcec83b0 0x5651dd29f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51944/0 (socket says 172.21.15.96:51944) 2024-04-25T14:13:00.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.635+0000 7f711d56b640 1 -- 172.21.15.96:0/793992785 learned_addr learned my addr 172.21.15.96:0/793992785 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:00.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.635+0000 7f711d56b640 1 -- 172.21.15.96:0/793992785 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651dced0590 msgr2=0x5651dd29f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:00.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.635+0000 7f7117fff640 1 --2- 172.21.15.96:0/793992785 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651dced0590 0x5651dd29f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.635+0000 7f711d56b640 1 --2- 172.21.15.96:0/793992785 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651dced0590 0x5651dd29f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.635+0000 7f711d56b640 1 -- 172.21.15.96:0/793992785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651dcef1270 msgr2=0x5651dd2a26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:00.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.635+0000 7f711d56b640 1 --2- 172.21.15.96:0/793992785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651dcef1270 0x5651dd2a26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.635+0000 7f711d56b640 1 -- 172.21.15.96:0/793992785 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651dd297070 con 0x5651dcec83b0 2024-04-25T14:13:00.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.635+0000 7f711d56b640 1 --2- 172.21.15.96:0/793992785 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651dcec83b0 0x5651dd29f2c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f711000f890 tx=0x7f7110004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1e84a804b480e53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:00.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.635+0000 7f70f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:13:00.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.635+0000 7f70f7fff640 1 -- 172.21.15.96:0/793992785 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711002b040 con 0x5651dcec83b0 2024-04-25T14:13:00.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.635+0000 7f70f7fff640 1 -- 172.21.15.96:0/793992785 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f7110004b50 con 0x5651dcec83b0 2024-04-25T14:13:00.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.635+0000 7f7125e83740 1 -- 172.21.15.96:0/793992785 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71100183f0 con 0x5651dcec83b0 2024-04-25T14:13:00.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.635+0000 7f70f7fff640 1 -- 172.21.15.96:0/793992785 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f711004b050 con 0x5651dcec83b0 2024-04-25T14:13:00.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.635+0000 7f7125e83740 10 client.15132 fetch_fsmap learned FSMap version 25 2024-04-25T14:13:00.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.635+0000 7f7125e83740 1 -- 172.21.15.96:0/793992785 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5651dd2a3dd0 con 0x5651dcec83b0 2024-04-25T14:13:00.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.635+0000 7f70f7fff640 1 -- 172.21.15.96:0/793992785 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71100183f0 con 0x5651dcec83b0 2024-04-25T14:13:00.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.636+0000 7f70f7fff640 1 -- 172.21.15.96:0/793992785 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f711003a030 con 0x5651dcec83b0 2024-04-25T14:13:00.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.636+0000 7f7125e83740 10 client.15132 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T14:13:00.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.636+0000 7f7125e83740 10 client.15132 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T14:13:00.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.636+0000 7f7125e83740 20 client.15132 populate_metadata read hostname 'smithi096' 2024-04-25T14:13:00.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.636+0000 7f7125e83740 1 --2- 172.21.15.96:0/793992785 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x5651dce85280 0x5651dd315b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.636+0000 7f7125e83740 4 client.15132 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T14:13:00.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.636+0000 7f7125e83740 1 -- 172.21.15.96:0/793992785 --> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] -- command(tid 0: {"prefix": "status"}) -- 0x5651dd1ff1c0 con 0x5651dce85280 2024-04-25T14:13:00.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.636+0000 7f7117fff640 1 --2- 172.21.15.96:0/793992785 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x5651dce85280 0x5651dd315b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.636+0000 7f7117fff640 1 --2- 172.21.15.96:0/793992785 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x5651dce85280 0x5651dd315b80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4dca57ba0762e028 server_cookie=7e37782a5e88457f in_seq=0 out_seq=0 2024-04-25T14:13:00.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.636+0000 7f70f7fff640 10 client.15132 ms_handle_connect on v2:172.21.15.151:6838/245699981 2024-04-25T14:13:00.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.637+0000 7f70f7fff640 1 -- 172.21.15.96:0/793992785 <== mds.1 v2:172.21.15.151:6838/245699981 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5651dd1ff1c0 con 0x5651dce85280 2024-04-25T14:13:00.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.637+0000 7f70f7fff640 10 client.15132 handle_command_reply: tid=0 2024-04-25T14:13:00.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.637+0000 7f7125e83740 1 client.15132 shutdown 2024-04-25T14:13:00.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.637+0000 7f7125e83740 1 -- 172.21.15.96:0/793992785 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651dcec83b0 msgr2=0x5651dd29f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:00.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.637+0000 7f7125e83740 1 --2- 172.21.15.96:0/793992785 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651dcec83b0 0x5651dd29f2c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f711000f890 tx=0x7f7110004580 comp rx=0 tx=0).stop 2024-04-25T14:13:00.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.637+0000 7f7125e83740 1 -- 172.21.15.96:0/793992785 shutdown_connections 2024-04-25T14:13:00.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.637+0000 7f7125e83740 1 --2- 172.21.15.96:0/793992785 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x5651dce85280 0x5651dd315b80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.637+0000 7f7125e83740 1 --2- 172.21.15.96:0/793992785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651dcef1270 0x5651dd2a26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.637+0000 7f7125e83740 1 --2- 172.21.15.96:0/793992785 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651dced0590 0x5651dd29f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.637+0000 7f7125e83740 1 --2- 172.21.15.96:0/793992785 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651dcec83b0 0x5651dd29f2c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.637+0000 7f70f7fff640 0 client.15132 ms_handle_reset on v2:172.21.15.151:6838/245699981 2024-04-25T14:13:00.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.637+0000 7f7125e83740 1 -- 172.21.15.96:0/793992785 >> 172.21.15.96:0/793992785 conn(0x5651dcee53c0 msgr2=0x5651dd28d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:00.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.637+0000 7f7125e83740 1 -- 172.21.15.96:0/793992785 shutdown_connections 2024-04-25T14:13:00.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.637+0000 7f7125e83740 1 -- 172.21.15.96:0/793992785 wait complete. 2024-04-25T14:13:00.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.637+0000 7f7125e83740 20 client.15132 trim_cache size 0 max 16384 2024-04-25T14:13:00.644 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T14:13:00.644 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "59334182-030c-11ef-bc93-c7b262605968", 2024-04-25T14:13:00.644 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 1, 2024-04-25T14:13:00.644 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 24517, 2024-04-25T14:13:00.644 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-04-25T14:13:00.644 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-04-25T14:13:00.644 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-04-25T14:13:00.644 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 89.060002632999996, 2024-04-25T14:13:00.644 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 25, 2024-04-25T14:13:00.644 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 78, 2024-04-25T14:13:00.644 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 76, 2024-04-25T14:13:00.644 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 105.742215437 2024-04-25T14:13:00.645 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T14:13:00.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.638+0000 7f711efce640 1 -- 172.21.15.96:0/2562688174 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f71000afaf0 msgr2=0x7f71000b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:00.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.638+0000 7f711efce640 1 --2- 172.21.15.96:0/2562688174 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f71000afaf0 0x7f71000b2000 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f71080c47c0 tx=0x7f7108102aa0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.638+0000 7f711efce640 1 -- 172.21.15.96:0/2562688174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7118102f60 msgr2=0x7f7118102200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:00.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.638+0000 7f711efce640 1 --2- 172.21.15.96:0/2562688174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7118102f60 0x7f7118102200 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7108006010 tx=0x7f7108004270 comp rx=0 tx=0).stop 2024-04-25T14:13:00.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.638+0000 7f711efce640 1 -- 172.21.15.96:0/2562688174 shutdown_connections 2024-04-25T14:13:00.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.638+0000 7f711efce640 1 --2- 172.21.15.96:0/2562688174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7118104480 0x7f71181a1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.638+0000 7f711efce640 1 --2- 172.21.15.96:0/2562688174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7118103940 0x7f711819efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.638+0000 7f711efce640 1 --2- 172.21.15.96:0/2562688174 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f71000afaf0 0x7f71000b2000 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.638+0000 7f711efce640 1 --2- 172.21.15.96:0/2562688174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7118102f60 0x7f7118102200 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.638+0000 7f711efce640 1 -- 172.21.15.96:0/2562688174 >> 172.21.15.96:0/2562688174 conn(0x7f71180fd380 msgr2=0x7f7118104ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:00.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.638+0000 7f711efce640 1 -- 172.21.15.96:0/2562688174 shutdown_connections 2024-04-25T14:13:00.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.638+0000 7f711efce640 1 -- 172.21.15.96:0/2562688174 wait complete. 2024-04-25T14:13:00.649 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T14:13:00.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.757+0000 7f0760b46640 1 -- 172.21.15.96:0/4156913924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f075c1043c0 msgr2=0x7f075c102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:00.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.757+0000 7f0760b46640 1 --2- 172.21.15.96:0/4156913924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f075c1043c0 0x7f075c102dc0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0750009900 tx=0x7f075002a410 comp rx=0 tx=0).stop 2024-04-25T14:13:00.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.757+0000 7f0760b46640 1 -- 172.21.15.96:0/4156913924 shutdown_connections 2024-04-25T14:13:00.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.757+0000 7f0760b46640 1 --2- 172.21.15.96:0/4156913924 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f075c108a10 0x7f075c10ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.757+0000 7f0760b46640 1 --2- 172.21.15.96:0/4156913924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f075c1043c0 0x7f075c102dc0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.757+0000 7f0760b46640 1 --2- 172.21.15.96:0/4156913924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f075c103ee0 0x7f075c1027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.757+0000 7f0760b46640 1 -- 172.21.15.96:0/4156913924 >> 172.21.15.96:0/4156913924 conn(0x7f075c0fd340 msgr2=0x7f075c0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:00.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.757+0000 7f0760b46640 1 -- 172.21.15.96:0/4156913924 shutdown_connections 2024-04-25T14:13:00.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.757+0000 7f0760b46640 1 -- 172.21.15.96:0/4156913924 wait complete. 2024-04-25T14:13:00.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.757+0000 7f0760b46640 1 Processor -- start 2024-04-25T14:13:00.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.758+0000 7f0760b46640 1 -- start start 2024-04-25T14:13:00.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.758+0000 7f0760b46640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f075c103ee0 0x7f075c0ff1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.758+0000 7f0760b46640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f075c108a10 0x7f075c0fd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.758+0000 7f0760b46640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f075c0fddb0 0x7f075c0fe1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.758+0000 7f0760b46640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f075c10e570 con 0x7f075c103ee0 2024-04-25T14:13:00.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.758+0000 7f0760b46640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f075c10e3f0 con 0x7f075c108a10 2024-04-25T14:13:00.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.758+0000 7f0760b46640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f075c10e6f0 con 0x7f075c0fddb0 2024-04-25T14:13:00.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.758+0000 7f075a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f075c103ee0 0x7f075c0ff1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.758+0000 7f075a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f075c103ee0 0x7f075c0ff1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51954/0 (socket says 172.21.15.96:51954) 2024-04-25T14:13:00.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.758+0000 7f075a59c640 1 -- 172.21.15.96:0/1699421310 learned_addr learned my addr 172.21.15.96:0/1699421310 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:00.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.758+0000 7f075a59c640 1 -- 172.21.15.96:0/1699421310 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f075c0fddb0 msgr2=0x7f075c0fe1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:00.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.758+0000 7f075a59c640 1 --2- 172.21.15.96:0/1699421310 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f075c0fddb0 0x7f075c0fe1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.758+0000 7f075a59c640 1 -- 172.21.15.96:0/1699421310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f075c108a10 msgr2=0x7f075c0fd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:00.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.758+0000 7f075a59c640 1 --2- 172.21.15.96:0/1699421310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f075c108a10 0x7f075c0fd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.758+0000 7f075a59c640 1 -- 172.21.15.96:0/1699421310 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f075c1a3f60 con 0x7f075c103ee0 2024-04-25T14:13:00.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.758+0000 7f075a59c640 1 --2- 172.21.15.96:0/1699421310 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f075c103ee0 0x7f075c0ff1d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f074c00cce0 tx=0x7f074c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:00.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.758+0000 7f074b7fe640 1 -- 172.21.15.96:0/1699421310 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c004b20 con 0x7f075c103ee0 2024-04-25T14:13:00.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.758+0000 7f074b7fe640 1 -- 172.21.15.96:0/1699421310 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f074c004cc0 con 0x7f075c103ee0 2024-04-25T14:13:00.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.758+0000 7f074b7fe640 1 -- 172.21.15.96:0/1699421310 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c00f6b0 con 0x7f075c103ee0 2024-04-25T14:13:00.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.758+0000 7f0760b46640 1 -- 172.21.15.96:0/1699421310 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f075c1a4250 con 0x7f075c103ee0 2024-04-25T14:13:00.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.759+0000 7f0760b46640 1 -- 172.21.15.96:0/1699421310 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f075c1a47d0 con 0x7f075c103ee0 2024-04-25T14:13:00.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.761+0000 7f074b7fe640 1 -- 172.21.15.96:0/1699421310 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f074c00f850 con 0x7f075c103ee0 2024-04-25T14:13:00.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.761+0000 7f07679fc740 1 Processor -- start 2024-04-25T14:13:00.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.761+0000 7f07679fc740 1 -- start start 2024-04-25T14:13:00.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.762+0000 7f07679fc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfc6001270 0x55dfc63a2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.762+0000 7f07679fc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc5fd83b0 0x55dfc63a28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.762+0000 7f07679fc740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfc5fe0590 0x55dfc63a97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.762+0000 7f07679fc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55dfc6307740 con 0x55dfc5fd83b0 2024-04-25T14:13:00.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.762+0000 7f07679fc740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55dfc62a33a0 con 0x55dfc5fe0590 2024-04-25T14:13:00.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.762+0000 7f07679fc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55dfc6303700 con 0x55dfc6001270 2024-04-25T14:13:00.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.762+0000 7f075a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc5fd83b0 0x55dfc63a28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.762+0000 7f075a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc5fd83b0 0x55dfc63a28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51962/0 (socket says 172.21.15.96:51962) 2024-04-25T14:13:00.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.762+0000 7f075a59c640 1 -- 172.21.15.96:0/4285786120 learned_addr learned my addr 172.21.15.96:0/4285786120 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:00.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.762+0000 7f074b7fe640 1 --2- 172.21.15.96:0/1699421310 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f07440afb70 0x7f07440b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.762+0000 7f0759d9b640 1 --2- 172.21.15.96:0/4285786120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfc5fe0590 0x55dfc63a97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.762+0000 7f075ad9d640 1 --2- 172.21.15.96:0/4285786120 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfc6001270 0x55dfc63a2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.762+0000 7f075a59c640 1 -- 172.21.15.96:0/4285786120 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfc6001270 msgr2=0x55dfc63a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:00.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.762+0000 7f075a59c640 1 --2- 172.21.15.96:0/4285786120 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfc6001270 0x55dfc63a2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.762+0000 7f075a59c640 1 -- 172.21.15.96:0/4285786120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfc5fe0590 msgr2=0x55dfc63a97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:00.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.762+0000 7f075a59c640 1 --2- 172.21.15.96:0/4285786120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfc5fe0590 0x55dfc63a97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.762+0000 7f075a59c640 1 -- 172.21.15.96:0/4285786120 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfc62bf750 con 0x55dfc5fd83b0 2024-04-25T14:13:00.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.762+0000 7f075ad9d640 1 --2- 172.21.15.96:0/4285786120 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfc6001270 0x55dfc63a2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:00.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.762+0000 7f075ad9d640 1 --2- 172.21.15.96:0/1699421310 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f07440afb70 0x7f07440b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.762+0000 7f074b7fe640 1 -- 172.21.15.96:0/1699421310 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f074c0ded00 con 0x7f075c103ee0 2024-04-25T14:13:00.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.762+0000 7f075a59c640 1 --2- 172.21.15.96:0/4285786120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc5fd83b0 0x55dfc63a28f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f074c0805b0 tx=0x7f074c06d630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1588aacb28e9e65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:00.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.763+0000 7f075ad9d640 1 --2- 172.21.15.96:0/1699421310 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f07440afb70 0x7f07440b2080 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x55dfc63a3540 tx=0x7f07540047d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:00.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.763+0000 7f0748ff9640 1 -- 172.21.15.96:0/4285786120 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c0a6b10 con 0x55dfc5fd83b0 2024-04-25T14:13:00.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.763+0000 7f0748ff9640 1 -- 172.21.15.96:0/4285786120 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f074c0a6cd0 con 0x55dfc5fd83b0 2024-04-25T14:13:00.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.763+0000 7f0748ff9640 1 -- 172.21.15.96:0/4285786120 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c0e24f0 con 0x55dfc5fd83b0 2024-04-25T14:13:00.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.763+0000 7f07679fc740 1 -- 172.21.15.96:0/4285786120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc5fd83b0 msgr2=0x55dfc63a28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:00.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.763+0000 7f07679fc740 1 --2- 172.21.15.96:0/4285786120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc5fd83b0 0x55dfc63a28f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f074c0805b0 tx=0x7f074c06d630 comp rx=0 tx=0).stop 2024-04-25T14:13:00.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.764+0000 7f07679fc740 1 -- 172.21.15.96:0/4285786120 shutdown_connections 2024-04-25T14:13:00.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.764+0000 7f07679fc740 1 --2- 172.21.15.96:0/4285786120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfc5fe0590 0x55dfc63a97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.764+0000 7f07679fc740 1 --2- 172.21.15.96:0/4285786120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc5fd83b0 0x55dfc63a28f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.764+0000 7f07679fc740 1 --2- 172.21.15.96:0/4285786120 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfc6001270 0x55dfc63a2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.764+0000 7f07679fc740 1 -- 172.21.15.96:0/4285786120 >> 172.21.15.96:0/4285786120 conn(0x55dfc5ff53c0 msgr2=0x55dfc639dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:00.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.764+0000 7f07679fc740 1 -- 172.21.15.96:0/4285786120 shutdown_connections 2024-04-25T14:13:00.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.764+0000 7f07679fc740 1 -- 172.21.15.96:0/4285786120 wait complete. 2024-04-25T14:13:00.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.764+0000 7f07679fc740 1 -- start start 2024-04-25T14:13:00.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.764+0000 7f07679fc740 1 Processor -- start 2024-04-25T14:13:00.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.764+0000 7f07679fc740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfc5fd83b0 0x55dfc63ac1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.764+0000 7f07679fc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc5fe0590 0x55dfc63ac710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.764+0000 7f07679fc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfc6001270 0x55dfc63b3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.764+0000 7f07679fc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55dfc63aa1e0 con 0x55dfc5fe0590 2024-04-25T14:13:00.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.764+0000 7f07679fc740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55dfc628efe0 con 0x55dfc5fd83b0 2024-04-25T14:13:00.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.764+0000 7f07679fc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55dfc63aa360 con 0x55dfc6001270 2024-04-25T14:13:00.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.764+0000 7f07679fc740 10 client.0 _ll_register_callbacks cb 0x55dfc62a5150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:13:00.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.764+0000 7f0759d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc5fe0590 0x55dfc63ac710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.764+0000 7f0759d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc5fe0590 0x55dfc63ac710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51974/0 (socket says 172.21.15.96:51974) 2024-04-25T14:13:00.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.764+0000 7f0759d9b640 1 -- 172.21.15.96:0/3795771348 learned_addr learned my addr 172.21.15.96:0/3795771348 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:00.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.765+0000 7f075ad9d640 1 --2- 172.21.15.96:0/3795771348 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfc6001270 0x55dfc63b3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.765+0000 7f075a59c640 1 --2- 172.21.15.96:0/3795771348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfc5fd83b0 0x55dfc63ac1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.765+0000 7f0759d9b640 1 -- 172.21.15.96:0/3795771348 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfc6001270 msgr2=0x55dfc63b3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:00.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.765+0000 7f0759d9b640 1 --2- 172.21.15.96:0/3795771348 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfc6001270 0x55dfc63b3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.765+0000 7f0759d9b640 1 -- 172.21.15.96:0/3795771348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfc5fd83b0 msgr2=0x55dfc63ac1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:00.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.765+0000 7f0759d9b640 1 --2- 172.21.15.96:0/3795771348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfc5fd83b0 0x55dfc63ac1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.765+0000 7f0759d9b640 1 -- 172.21.15.96:0/3795771348 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfc60192d0 con 0x55dfc5fe0590 2024-04-25T14:13:00.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.765+0000 7f0759d9b640 1 --2- 172.21.15.96:0/3795771348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc5fe0590 0x55dfc63ac710 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0750009900 tx=0x7f07500317d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc82986b51c1e021 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:00.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.765+0000 7f0741ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:13:00.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.765+0000 7f0741ffb640 1 -- 172.21.15.96:0/3795771348 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0750040070 con 0x55dfc5fe0590 2024-04-25T14:13:00.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.765+0000 7f0741ffb640 1 -- 172.21.15.96:0/3795771348 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f0750004450 con 0x55dfc5fe0590 2024-04-25T14:13:00.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.765+0000 7f07679fc740 1 -- 172.21.15.96:0/3795771348 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f074c0a6cd0 con 0x55dfc5fe0590 2024-04-25T14:13:00.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.765+0000 7f0741ffb640 1 -- 172.21.15.96:0/3795771348 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f075004c050 con 0x55dfc5fe0590 2024-04-25T14:13:00.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.765+0000 7f07679fc740 10 client.15144 fetch_fsmap learned FSMap version 25 2024-04-25T14:13:00.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.765+0000 7f07679fc740 1 -- 172.21.15.96:0/3795771348 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dfc60e36b0 con 0x55dfc5fe0590 2024-04-25T14:13:00.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.765+0000 7f0741ffb640 1 -- 172.21.15.96:0/3795771348 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f074c0a6cd0 con 0x55dfc5fe0590 2024-04-25T14:13:00.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.766+0000 7f0741ffb640 1 -- 172.21.15.96:0/3795771348 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f075003a040 con 0x55dfc5fe0590 2024-04-25T14:13:00.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.766+0000 7f07679fc740 10 client.15144 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T14:13:00.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.766+0000 7f07679fc740 10 client.15144 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T14:13:00.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.766+0000 7f07679fc740 20 client.15144 populate_metadata read hostname 'smithi096' 2024-04-25T14:13:00.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.766+0000 7f07679fc740 1 --2- 172.21.15.96:0/3795771348 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55dfc5fbfe40 0x55dfc63b9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.766+0000 7f07679fc740 4 client.15144 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:13:00.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.766+0000 7f07679fc740 1 -- 172.21.15.96:0/3795771348 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dfc6315350 con 0x55dfc5fbfe40 2024-04-25T14:13:00.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.766+0000 7f075a59c640 1 --2- 172.21.15.96:0/3795771348 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55dfc5fbfe40 0x55dfc63b9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.766+0000 7f0741ffb640 10 client.15144 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:13:00.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.766+0000 7f075a59c640 1 --2- 172.21.15.96:0/3795771348 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55dfc5fbfe40 0x55dfc63b9e70 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63e91f0b2c4748e8 server_cookie=42a3c77a126c6238 in_seq=0 out_seq=0 2024-04-25T14:13:00.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.767+0000 7f0741ffb640 1 -- 172.21.15.96:0/3795771348 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dfc6315350 con 0x55dfc5fbfe40 2024-04-25T14:13:00.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.767+0000 7f0741ffb640 10 client.15144 handle_command_reply: tid=0 2024-04-25T14:13:00.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.767+0000 7f07679fc740 1 client.15144 shutdown 2024-04-25T14:13:00.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.767+0000 7f07679fc740 1 -- 172.21.15.96:0/3795771348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc5fe0590 msgr2=0x55dfc63ac710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:00.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.767+0000 7f07679fc740 1 --2- 172.21.15.96:0/3795771348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc5fe0590 0x55dfc63ac710 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0750009900 tx=0x7f07500317d0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.767+0000 7f07679fc740 1 -- 172.21.15.96:0/3795771348 shutdown_connections 2024-04-25T14:13:00.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.767+0000 7f07679fc740 1 --2- 172.21.15.96:0/3795771348 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfc6001270 0x55dfc63b3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.767+0000 7f07679fc740 1 --2- 172.21.15.96:0/3795771348 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55dfc5fbfe40 0x55dfc63b9e70 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.767+0000 7f07679fc740 1 --2- 172.21.15.96:0/3795771348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc5fe0590 0x55dfc63ac710 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.767+0000 7f07679fc740 1 --2- 172.21.15.96:0/3795771348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfc5fd83b0 0x55dfc63ac1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.767+0000 7f0741ffb640 0 client.15144 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:13:00.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.767+0000 7f07679fc740 1 -- 172.21.15.96:0/3795771348 >> 172.21.15.96:0/3795771348 conn(0x55dfc5ff53c0 msgr2=0x55dfc639cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:00.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.768+0000 7f07679fc740 1 -- 172.21.15.96:0/3795771348 shutdown_connections 2024-04-25T14:13:00.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.768+0000 7f07679fc740 1 -- 172.21.15.96:0/3795771348 wait complete. 2024-04-25T14:13:00.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.768+0000 7f07679fc740 20 client.15144 trim_cache size 0 max 16384 2024-04-25T14:13:00.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.786+0000 7f07679fc740 1 Processor -- start 2024-04-25T14:13:00.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.786+0000 7f07679fc740 1 -- start start 2024-04-25T14:13:00.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.786+0000 7f07679fc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc5fd83b0 0x55dfc63ab7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.786+0000 7f07679fc740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfc5fe0590 0x55dfc63ac170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.786+0000 7f07679fc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfc6001270 0x55dfc63ac6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.786+0000 7f07679fc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55dfc63b81b0 con 0x55dfc5fd83b0 2024-04-25T14:13:00.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.786+0000 7f07679fc740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55dfc63b8880 con 0x55dfc5fe0590 2024-04-25T14:13:00.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.786+0000 7f07679fc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55dfc63b8f50 con 0x55dfc6001270 2024-04-25T14:13:00.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.786+0000 7f075a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc5fd83b0 0x55dfc63ab7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.786+0000 7f075a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc5fd83b0 0x55dfc63ab7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51978/0 (socket says 172.21.15.96:51978) 2024-04-25T14:13:00.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.786+0000 7f075a59c640 1 -- 172.21.15.96:0/2255149384 learned_addr learned my addr 172.21.15.96:0/2255149384 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:00.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.786+0000 7f075ad9d640 1 --2- 172.21.15.96:0/2255149384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfc6001270 0x55dfc63ac6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.786+0000 7f075a59c640 1 -- 172.21.15.96:0/2255149384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfc6001270 msgr2=0x55dfc63ac6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:00.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.786+0000 7f075a59c640 1 --2- 172.21.15.96:0/2255149384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfc6001270 0x55dfc63ac6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.786+0000 7f075a59c640 1 -- 172.21.15.96:0/2255149384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfc5fe0590 msgr2=0x55dfc63ac170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:00.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.786+0000 7f075a59c640 1 --2- 172.21.15.96:0/2255149384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfc5fe0590 0x55dfc63ac170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.786+0000 7f075a59c640 1 -- 172.21.15.96:0/2255149384 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfc63abf80 con 0x55dfc5fd83b0 2024-04-25T14:13:00.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.786+0000 7f075a59c640 1 --2- 172.21.15.96:0/2255149384 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc5fd83b0 0x55dfc63ab7b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f074c0a4430 tx=0x7f074c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=674b309dd59e5839 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:00.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.787+0000 7f0743fff640 1 -- 172.21.15.96:0/2255149384 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c01d040 con 0x55dfc5fd83b0 2024-04-25T14:13:00.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.787+0000 7f0743fff640 1 -- 172.21.15.96:0/2255149384 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f074c0a6cd0 con 0x55dfc5fd83b0 2024-04-25T14:13:00.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.787+0000 7f0743fff640 1 -- 172.21.15.96:0/2255149384 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c075b70 con 0x55dfc5fd83b0 2024-04-25T14:13:00.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.787+0000 7f07679fc740 1 -- 172.21.15.96:0/2255149384 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc5fd83b0 msgr2=0x55dfc63ab7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:00.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.787+0000 7f07679fc740 1 --2- 172.21.15.96:0/2255149384 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc5fd83b0 0x55dfc63ab7b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f074c0a4430 tx=0x7f074c000f40 comp rx=0 tx=0).stop 2024-04-25T14:13:00.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.787+0000 7f07679fc740 1 -- 172.21.15.96:0/2255149384 shutdown_connections 2024-04-25T14:13:00.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.787+0000 7f07679fc740 1 --2- 172.21.15.96:0/2255149384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfc6001270 0x55dfc63ac6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.787+0000 7f07679fc740 1 --2- 172.21.15.96:0/2255149384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfc5fe0590 0x55dfc63ac170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.787+0000 7f07679fc740 1 --2- 172.21.15.96:0/2255149384 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc5fd83b0 0x55dfc63ab7b0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.787+0000 7f07679fc740 1 -- 172.21.15.96:0/2255149384 >> 172.21.15.96:0/2255149384 conn(0x55dfc5ff53c0 msgr2=0x55dfc63a66b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:00.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.787+0000 7f07679fc740 1 -- 172.21.15.96:0/2255149384 shutdown_connections 2024-04-25T14:13:00.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.787+0000 7f07679fc740 1 -- 172.21.15.96:0/2255149384 wait complete. 2024-04-25T14:13:00.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.788+0000 7f07679fc740 1 -- start start 2024-04-25T14:13:00.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.788+0000 7f07679fc740 1 Processor -- start 2024-04-25T14:13:00.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.788+0000 7f07679fc740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfc5fd83b0 0x55dfc63b1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.788+0000 7f07679fc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc5fe0590 0x55dfc63b1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.788+0000 7f07679fc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfc6001270 0x55dfc63b2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.788+0000 7f07679fc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55dfc63b9cf0 con 0x55dfc5fe0590 2024-04-25T14:13:00.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.788+0000 7f07679fc740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55dfc63b9620 con 0x55dfc5fd83b0 2024-04-25T14:13:00.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.788+0000 7f07679fc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55dfc63ba3c0 con 0x55dfc6001270 2024-04-25T14:13:00.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.788+0000 7f07679fc740 10 client.0 _ll_register_callbacks cb 0x7f0750009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:13:00.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.788+0000 7f0759d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc5fe0590 0x55dfc63b1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.788+0000 7f0759d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc5fe0590 0x55dfc63b1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51986/0 (socket says 172.21.15.96:51986) 2024-04-25T14:13:00.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.788+0000 7f0759d9b640 1 -- 172.21.15.96:0/1205105293 learned_addr learned my addr 172.21.15.96:0/1205105293 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:00.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.788+0000 7f0759d9b640 1 -- 172.21.15.96:0/1205105293 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfc6001270 msgr2=0x55dfc63b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:00.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.788+0000 7f0759d9b640 1 --2- 172.21.15.96:0/1205105293 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfc6001270 0x55dfc63b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.788+0000 7f075a59c640 1 --2- 172.21.15.96:0/1205105293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfc5fd83b0 0x55dfc63b1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.788+0000 7f0759d9b640 1 -- 172.21.15.96:0/1205105293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfc5fd83b0 msgr2=0x55dfc63b1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:00.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.788+0000 7f0759d9b640 1 --2- 172.21.15.96:0/1205105293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfc5fd83b0 0x55dfc63b1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.788+0000 7f0759d9b640 1 -- 172.21.15.96:0/1205105293 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfc63b2a50 con 0x55dfc5fe0590 2024-04-25T14:13:00.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.788+0000 7f0759d9b640 1 --2- 172.21.15.96:0/1205105293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc5fe0590 0x55dfc63b1560 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0750009850 tx=0x7f075002aa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29b9b3c796adbc44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:00.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.788+0000 7f0741ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:13:00.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.789+0000 7f0741ffb640 1 -- 172.21.15.96:0/1205105293 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0750040070 con 0x55dfc5fe0590 2024-04-25T14:13:00.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.789+0000 7f0741ffb640 1 -- 172.21.15.96:0/1205105293 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f07500046c0 con 0x55dfc5fe0590 2024-04-25T14:13:00.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.789+0000 7f07679fc740 1 -- 172.21.15.96:0/1205105293 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f074c0a6cd0 con 0x55dfc5fe0590 2024-04-25T14:13:00.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.789+0000 7f0741ffb640 1 -- 172.21.15.96:0/1205105293 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0750035040 con 0x55dfc5fe0590 2024-04-25T14:13:00.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.789+0000 7f07679fc740 10 client.15150 fetch_fsmap learned FSMap version 25 2024-04-25T14:13:00.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.789+0000 7f07679fc740 1 -- 172.21.15.96:0/1205105293 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dfc63b3d70 con 0x55dfc5fe0590 2024-04-25T14:13:00.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.789+0000 7f0741ffb640 1 -- 172.21.15.96:0/1205105293 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f074c0a6cd0 con 0x55dfc5fe0590 2024-04-25T14:13:00.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.789+0000 7f0741ffb640 1 -- 172.21.15.96:0/1205105293 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f0750003680 con 0x55dfc5fe0590 2024-04-25T14:13:00.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.789+0000 7f07679fc740 10 client.15150 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T14:13:00.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.789+0000 7f07679fc740 10 client.15150 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T14:13:00.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.789+0000 7f07679fc740 20 client.15150 populate_metadata read hostname 'smithi096' 2024-04-25T14:13:00.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.789+0000 7f07679fc740 1 --2- 172.21.15.96:0/1205105293 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55dfc5f95280 0x55dfc6426060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:00.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.789+0000 7f07679fc740 4 client.15150 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T14:13:00.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.789+0000 7f07679fc740 1 -- 172.21.15.96:0/1205105293 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "status"}) -- 0x55dfc630f1c0 con 0x55dfc5f95280 2024-04-25T14:13:00.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.789+0000 7f075a59c640 1 --2- 172.21.15.96:0/1205105293 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55dfc5f95280 0x55dfc6426060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:00.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.790+0000 7f0741ffb640 10 client.15150 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:13:00.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.790+0000 7f075a59c640 1 --2- 172.21.15.96:0/1205105293 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55dfc5f95280 0x55dfc6426060 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=399921f302a3e79f server_cookie=92d9ddd8ef828de7 in_seq=0 out_seq=0 2024-04-25T14:13:00.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.790+0000 7f0741ffb640 1 -- 172.21.15.96:0/1205105293 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55dfc630f1c0 con 0x55dfc5f95280 2024-04-25T14:13:00.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.790+0000 7f0741ffb640 10 client.15150 handle_command_reply: tid=0 2024-04-25T14:13:00.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.790+0000 7f07679fc740 1 client.15150 shutdown 2024-04-25T14:13:00.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.790+0000 7f07679fc740 1 -- 172.21.15.96:0/1205105293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc5fe0590 msgr2=0x55dfc63b1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:00.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.790+0000 7f07679fc740 1 --2- 172.21.15.96:0/1205105293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc5fe0590 0x55dfc63b1560 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0750009850 tx=0x7f075002aa90 comp rx=0 tx=0).stop 2024-04-25T14:13:00.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.790+0000 7f07679fc740 1 -- 172.21.15.96:0/1205105293 shutdown_connections 2024-04-25T14:13:00.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.790+0000 7f07679fc740 1 --2- 172.21.15.96:0/1205105293 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfc6001270 0x55dfc63b2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.790+0000 7f07679fc740 1 --2- 172.21.15.96:0/1205105293 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55dfc5f95280 0x55dfc6426060 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.790+0000 7f07679fc740 1 --2- 172.21.15.96:0/1205105293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc5fe0590 0x55dfc63b1560 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.790+0000 7f07679fc740 1 --2- 172.21.15.96:0/1205105293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfc5fd83b0 0x55dfc63b1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.790+0000 7f0741ffb640 0 client.15150 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:13:00.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.790+0000 7f07679fc740 1 -- 172.21.15.96:0/1205105293 >> 172.21.15.96:0/1205105293 conn(0x55dfc5ff53c0 msgr2=0x55dfc63aa760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:00.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.790+0000 7f07679fc740 1 -- 172.21.15.96:0/1205105293 shutdown_connections 2024-04-25T14:13:00.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.790+0000 7f07679fc740 1 -- 172.21.15.96:0/1205105293 wait complete. 2024-04-25T14:13:00.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.790+0000 7f07679fc740 20 client.15150 trim_cache size 0 max 16384 2024-04-25T14:13:00.797 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T14:13:00.797 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "59334182-030c-11ef-bc93-c7b262605968", 2024-04-25T14:13:00.797 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 0, 2024-04-25T14:13:00.797 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 24536, 2024-04-25T14:13:00.797 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-04-25T14:13:00.797 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-04-25T14:13:00.797 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-04-25T14:13:00.797 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 91.247133267999999, 2024-04-25T14:13:00.797 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 25, 2024-04-25T14:13:00.797 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 78, 2024-04-25T14:13:00.797 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 76, 2024-04-25T14:13:00.798 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 96.984948935000006 2024-04-25T14:13:00.798 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T14:13:00.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.791+0000 7f0760b46640 1 -- 172.21.15.96:0/1699421310 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f07440afb70 msgr2=0x7f07440b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:00.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.791+0000 7f0760b46640 1 --2- 172.21.15.96:0/1699421310 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f07440afb70 0x7f07440b2080 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x55dfc63a3540 tx=0x7f07540047d0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.791+0000 7f0760b46640 1 -- 172.21.15.96:0/1699421310 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f075c103ee0 msgr2=0x7f075c0ff1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:00.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.791+0000 7f0760b46640 1 --2- 172.21.15.96:0/1699421310 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f075c103ee0 0x7f075c0ff1d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f074c00cce0 tx=0x7f074c00a620 comp rx=0 tx=0).stop 2024-04-25T14:13:00.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.791+0000 7f0760b46640 1 -- 172.21.15.96:0/1699421310 shutdown_connections 2024-04-25T14:13:00.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.791+0000 7f0760b46640 1 --2- 172.21.15.96:0/1699421310 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f075c0fddb0 0x7f075c0fe1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.791+0000 7f0760b46640 1 --2- 172.21.15.96:0/1699421310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f075c108a10 0x7f075c0fd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.791+0000 7f0760b46640 1 --2- 172.21.15.96:0/1699421310 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f07440afb70 0x7f07440b2080 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.791+0000 7f0760b46640 1 --2- 172.21.15.96:0/1699421310 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f075c103ee0 0x7f075c0ff1d0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:00.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.791+0000 7f0760b46640 1 -- 172.21.15.96:0/1699421310 >> 172.21.15.96:0/1699421310 conn(0x7f075c0fd340 msgr2=0x7f075c101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:00.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.791+0000 7f0760b46640 1 -- 172.21.15.96:0/1699421310 shutdown_connections 2024-04-25T14:13:00.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:00.791+0000 7f0760b46640 1 -- 172.21.15.96:0/1699421310 wait complete. 2024-04-25T14:13:00.802 INFO:tasks.cephadm:Setup complete, yielding 2024-04-25T14:13:00.802 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T14:13:00.812 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi096.front.sepia.ceph.com 2024-04-25T14:13:00.813 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-25T14:13:00.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:00 smithi097 ceph-mon[32720]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-25T14:13:00.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:00 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/2794028549' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:13:00.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:00 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/655939224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:13:00.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:00 smithi151 ceph-mon[32990]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-25T14:13:00.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:00 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/2794028549' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:13:00.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:00 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/655939224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:13:01.050 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:13:02.799 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:02 smithi096 ceph-mon[29207]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2024-04-25T14:13:02.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:02.912+0000 7fcb3222e700 1 -- 172.21.15.96:0/1619435750 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb2c105ce0 msgr2=0x7fcb2c10c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:02.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:02.912+0000 7fcb3222e700 1 --2- 172.21.15.96:0/1619435750 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb2c105ce0 0x7fcb2c10c590 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcb2400b0a0 tx=0x7fcb2400b3b0 comp rx=0 tx=0).stop 2024-04-25T14:13:02.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:02.912+0000 7fcb3222e700 1 -- 172.21.15.96:0/1619435750 shutdown_connections 2024-04-25T14:13:02.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:02.912+0000 7fcb3222e700 1 --2- 172.21.15.96:0/1619435750 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb2c105ce0 0x7fcb2c10c590 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:02.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:02.912+0000 7fcb3222e700 1 --2- 172.21.15.96:0/1619435750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb2c104210 0x7fcb2c104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:02.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:02.912+0000 7fcb3222e700 1 --2- 172.21.15.96:0/1619435750 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb2c105340 0x7fcb2c105750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:02.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:02.912+0000 7fcb3222e700 1 -- 172.21.15.96:0/1619435750 >> 172.21.15.96:0/1619435750 conn(0x7fcb2c0ff870 msgr2=0x7fcb2c101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:02.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:02.912+0000 7fcb3222e700 1 -- 172.21.15.96:0/1619435750 shutdown_connections 2024-04-25T14:13:02.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:02.912+0000 7fcb3222e700 1 -- 172.21.15.96:0/1619435750 wait complete. 2024-04-25T14:13:02.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:02.913+0000 7fcb3222e700 1 Processor -- start 2024-04-25T14:13:02.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:02.913+0000 7fcb3222e700 1 -- start start 2024-04-25T14:13:02.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:02.913+0000 7fcb3222e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb2c104210 0x7fcb2c19a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:02.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:02.913+0000 7fcb3222e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb2c105340 0x7fcb2c19ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:02.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:02.913+0000 7fcb3222e700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb2c105ce0 0x7fcb2c1a1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:02.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:02.913+0000 7fcb3222e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c10fc90 con 0x7fcb2c104210 2024-04-25T14:13:02.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:02.913+0000 7fcb3222e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c10fb10 con 0x7fcb2c105340 2024-04-25T14:13:02.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:02.913+0000 7fcb3222e700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c10fe10 con 0x7fcb2c105ce0 2024-04-25T14:13:02.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:02.913+0000 7fcb2b7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb2c104210 0x7fcb2c19a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:02.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:02.913+0000 7fcb2b7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb2c104210 0x7fcb2c19a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49212/0 (socket says 172.21.15.96:49212) 2024-04-25T14:13:02.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:02.913+0000 7fcb2b7fe700 1 -- 172.21.15.96:0/2880305448 learned_addr learned my addr 172.21.15.96:0/2880305448 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:02.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:02.913+0000 7fcb22ffd700 1 --2- 172.21.15.96:0/2880305448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb2c105340 0x7fcb2c19ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:02.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:02.913+0000 7fcb2bfff700 1 --2- 172.21.15.96:0/2880305448 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb2c105ce0 0x7fcb2c1a1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:02.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:02.914+0000 7fcb2b7fe700 1 -- 172.21.15.96:0/2880305448 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb2c105ce0 msgr2=0x7fcb2c1a1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:02.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:02.914+0000 7fcb2b7fe700 1 --2- 172.21.15.96:0/2880305448 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb2c105ce0 0x7fcb2c1a1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:02.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:02.914+0000 7fcb2b7fe700 1 -- 172.21.15.96:0/2880305448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb2c105340 msgr2=0x7fcb2c19ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:02.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:02.914+0000 7fcb2b7fe700 1 --2- 172.21.15.96:0/2880305448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb2c105340 0x7fcb2c19ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:02.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:02.914+0000 7fcb2b7fe700 1 -- 172.21.15.96:0/2880305448 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb2c1a2280 con 0x7fcb2c104210 2024-04-25T14:13:02.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:02.914+0000 7fcb22ffd700 1 --2- 172.21.15.96:0/2880305448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb2c105340 0x7fcb2c19ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:02.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:02.914+0000 7fcb2b7fe700 1 --2- 172.21.15.96:0/2880305448 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb2c104210 0x7fcb2c19a720 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c00c970 tx=0x7fcb1c00cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:02.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:02.914+0000 7fcb297fa700 1 -- 172.21.15.96:0/2880305448 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c004220 con 0x7fcb2c104210 2024-04-25T14:13:02.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:02.914+0000 7fcb297fa700 1 -- 172.21.15.96:0/2880305448 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb1c007db0 con 0x7fcb2c104210 2024-04-25T14:13:02.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:02.914+0000 7fcb3222e700 1 -- 172.21.15.96:0/2880305448 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb2c1a2570 con 0x7fcb2c104210 2024-04-25T14:13:02.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:02.914+0000 7fcb297fa700 1 -- 172.21.15.96:0/2880305448 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c003d00 con 0x7fcb2c104210 2024-04-25T14:13:02.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:02.914+0000 7fcb3222e700 1 -- 172.21.15.96:0/2880305448 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb2c1a2b00 con 0x7fcb2c104210 2024-04-25T14:13:02.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:02.915+0000 7fcb3222e700 1 -- 172.21.15.96:0/2880305448 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb2c051190 con 0x7fcb2c104210 2024-04-25T14:13:02.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:02.917+0000 7fcb297fa700 1 -- 172.21.15.96:0/2880305448 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fcb1c007900 con 0x7fcb2c104210 2024-04-25T14:13:02.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:02.918+0000 7fcb297fa700 1 --2- 172.21.15.96:0/2880305448 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fcb180afc00 0x7fcb180b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:02.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:02.918+0000 7fcb22ffd700 1 --2- 172.21.15.96:0/2880305448 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fcb180afc00 0x7fcb180b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:02.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:02.918+0000 7fcb297fa700 1 -- 172.21.15.96:0/2880305448 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fcb1c0dec60 con 0x7fcb2c104210 2024-04-25T14:13:02.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:02.918+0000 7fcb22ffd700 1 --2- 172.21.15.96:0/2880305448 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fcb180afc00 0x7fcb180b2110 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c19b7d0 tx=0x7fcb1400b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:02.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:02.921+0000 7fcb297fa700 1 -- 172.21.15.96:0/2880305448 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcb1c014030 con 0x7fcb2c104210 2024-04-25T14:13:02.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:02 smithi097 ceph-mon[32720]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2024-04-25T14:13:02.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:02 smithi151 ceph-mon[32990]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2024-04-25T14:13:03.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:03.138+0000 7fcb3222e700 1 -- 172.21.15.96:0/2880305448 --> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fcb2c063ad0 con 0x7fcb180afc00 2024-04-25T14:13:03.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:03.139+0000 7fcb297fa700 1 -- 172.21.15.96:0/2880305448 <== mgr.14152 v2:172.21.15.96:6800/536332425 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fcb2c063ad0 con 0x7fcb180afc00 2024-04-25T14:13:03.139 INFO:teuthology.orchestra.run.smithi096.stdout:Backend: cephadm 2024-04-25T14:13:03.139 INFO:teuthology.orchestra.run.smithi096.stdout:Available: Yes 2024-04-25T14:13:03.139 INFO:teuthology.orchestra.run.smithi096.stdout:Paused: No 2024-04-25T14:13:03.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:03.141+0000 7fcb3222e700 1 -- 172.21.15.96:0/2880305448 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fcb180afc00 msgr2=0x7fcb180b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:03.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:03.141+0000 7fcb3222e700 1 --2- 172.21.15.96:0/2880305448 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fcb180afc00 0x7fcb180b2110 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c19b7d0 tx=0x7fcb1400b2c0 comp rx=0 tx=0).stop 2024-04-25T14:13:03.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:03.141+0000 7fcb3222e700 1 -- 172.21.15.96:0/2880305448 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb2c104210 msgr2=0x7fcb2c19a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:03.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:03.141+0000 7fcb3222e700 1 --2- 172.21.15.96:0/2880305448 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb2c104210 0x7fcb2c19a720 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c00c970 tx=0x7fcb1c00cd30 comp rx=0 tx=0).stop 2024-04-25T14:13:03.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:03.141+0000 7fcb3222e700 1 -- 172.21.15.96:0/2880305448 shutdown_connections 2024-04-25T14:13:03.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:03.141+0000 7fcb3222e700 1 --2- 172.21.15.96:0/2880305448 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fcb180afc00 0x7fcb180b2110 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:03.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:03.141+0000 7fcb3222e700 1 --2- 172.21.15.96:0/2880305448 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb2c105ce0 0x7fcb2c1a1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:03.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:03.141+0000 7fcb3222e700 1 --2- 172.21.15.96:0/2880305448 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb2c104210 0x7fcb2c19a720 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:03.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:03.141+0000 7fcb3222e700 1 --2- 172.21.15.96:0/2880305448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb2c105340 0x7fcb2c19ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:03.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:03.141+0000 7fcb3222e700 1 -- 172.21.15.96:0/2880305448 >> 172.21.15.96:0/2880305448 conn(0x7fcb2c0ff870 msgr2=0x7fcb2c108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:03.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:03.141+0000 7fcb3222e700 1 -- 172.21.15.96:0/2880305448 shutdown_connections 2024-04-25T14:13:03.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:03.141+0000 7fcb3222e700 1 -- 172.21.15.96:0/2880305448 wait complete. 2024-04-25T14:13:03.955 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-25T14:13:04.199 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:13:04.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:04 smithi096 ceph-mon[29207]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:13:04.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:04 smithi096 ceph-mon[29207]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-25T14:13:04.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:04 smithi097 ceph-mon[32720]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:13:04.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:04 smithi097 ceph-mon[32720]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-25T14:13:04.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:04 smithi151 ceph-mon[32990]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:13:04.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:04 smithi151 ceph-mon[32990]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-25T14:13:05.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:05.972+0000 7fc86d9f1700 1 -- 172.21.15.96:0/4235642897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc868104be0 msgr2=0x7fc86810b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:05.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:05.972+0000 7fc86d9f1700 1 --2- 172.21.15.96:0/4235642897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc868104be0 0x7fc86810b490 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc86000b0a0 tx=0x7fc86000b3b0 comp rx=0 tx=0).stop 2024-04-25T14:13:05.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:05.972+0000 7fc86d9f1700 1 -- 172.21.15.96:0/4235642897 shutdown_connections 2024-04-25T14:13:05.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:05.972+0000 7fc86d9f1700 1 --2- 172.21.15.96:0/4235642897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc868104be0 0x7fc86810b490 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:05.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:05.972+0000 7fc86d9f1700 1 --2- 172.21.15.96:0/4235642897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc868103110 0x7fc868103520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:05.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:05.972+0000 7fc86d9f1700 1 --2- 172.21.15.96:0/4235642897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc868104240 0x7fc868104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:05.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:05.972+0000 7fc86d9f1700 1 -- 172.21.15.96:0/4235642897 >> 172.21.15.96:0/4235642897 conn(0x7fc8680fe750 msgr2=0x7fc868100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:05.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:05.972+0000 7fc86d9f1700 1 -- 172.21.15.96:0/4235642897 shutdown_connections 2024-04-25T14:13:05.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:05.972+0000 7fc86d9f1700 1 -- 172.21.15.96:0/4235642897 wait complete. 2024-04-25T14:13:05.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:05.973+0000 7fc86d9f1700 1 Processor -- start 2024-04-25T14:13:05.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:05.973+0000 7fc86d9f1700 1 -- start start 2024-04-25T14:13:05.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:05.973+0000 7fc86d9f1700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc868103110 0x7fc86806bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:05.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:05.973+0000 7fc86d9f1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc868104240 0x7fc86819f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:05.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:05.973+0000 7fc86d9f1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc868104be0 0x7fc8681a1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:05.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:05.973+0000 7fc86d9f1700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc86810ebf0 con 0x7fc868104be0 2024-04-25T14:13:05.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:05.973+0000 7fc86d9f1700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc86810ea70 con 0x7fc868104240 2024-04-25T14:13:05.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:05.973+0000 7fc86d9f1700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc86810ed70 con 0x7fc868103110 2024-04-25T14:13:05.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:05.973+0000 7fc8677fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc868104be0 0x7fc8681a1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:05.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:05.973+0000 7fc8677fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc868104be0 0x7fc8681a1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49230/0 (socket says 172.21.15.96:49230) 2024-04-25T14:13:05.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:05.973+0000 7fc8677fe700 1 -- 172.21.15.96:0/1649478479 learned_addr learned my addr 172.21.15.96:0/1649478479 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:05.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:05.973+0000 7fc85ffff700 1 --2- 172.21.15.96:0/1649478479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc868104240 0x7fc86819f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:05.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:05.973+0000 7fc866ffd700 1 --2- 172.21.15.96:0/1649478479 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc868103110 0x7fc86806bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:05.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:05.973+0000 7fc8677fe700 1 -- 172.21.15.96:0/1649478479 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc868103110 msgr2=0x7fc86806bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:05.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:05.973+0000 7fc8677fe700 1 --2- 172.21.15.96:0/1649478479 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc868103110 0x7fc86806bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:05.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:05.973+0000 7fc8677fe700 1 -- 172.21.15.96:0/1649478479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc868104240 msgr2=0x7fc86819f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:05.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:05.973+0000 7fc8677fe700 1 --2- 172.21.15.96:0/1649478479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc868104240 0x7fc86819f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:05.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:05.973+0000 7fc8677fe700 1 -- 172.21.15.96:0/1649478479 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8681a2330 con 0x7fc868104be0 2024-04-25T14:13:05.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:05.974+0000 7fc8677fe700 1 --2- 172.21.15.96:0/1649478479 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc868104be0 0x7fc8681a1b00 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc86000b830 tx=0x7fc860004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:05.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:05.974+0000 7fc86c9ef700 1 -- 172.21.15.96:0/1649478479 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc860021070 con 0x7fc868104be0 2024-04-25T14:13:05.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:05.974+0000 7fc86c9ef700 1 -- 172.21.15.96:0/1649478479 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc860007c80 con 0x7fc868104be0 2024-04-25T14:13:05.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:05.974+0000 7fc86d9f1700 1 -- 172.21.15.96:0/1649478479 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8681a25c0 con 0x7fc868104be0 2024-04-25T14:13:05.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:05.974+0000 7fc86c9ef700 1 -- 172.21.15.96:0/1649478479 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86001d450 con 0x7fc868104be0 2024-04-25T14:13:05.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:05.974+0000 7fc86d9f1700 1 -- 172.21.15.96:0/1649478479 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8681a2b80 con 0x7fc868104be0 2024-04-25T14:13:05.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:05.975+0000 7fc86d9f1700 1 -- 172.21.15.96:0/1649478479 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc868051190 con 0x7fc868104be0 2024-04-25T14:13:05.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:05.976+0000 7fc86c9ef700 1 -- 172.21.15.96:0/1649478479 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fc860004020 con 0x7fc868104be0 2024-04-25T14:13:05.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:05.977+0000 7fc86c9ef700 1 --2- 172.21.15.96:0/1649478479 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fc8540afb80 0x7fc8540b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:05.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:05.977+0000 7fc866ffd700 1 --2- 172.21.15.96:0/1649478479 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fc8540afb80 0x7fc8540b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:05.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:05.977+0000 7fc86c9ef700 1 -- 172.21.15.96:0/1649478479 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fc86000f020 con 0x7fc868104be0 2024-04-25T14:13:05.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:05.977+0000 7fc866ffd700 1 --2- 172.21.15.96:0/1649478479 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fc8540afb80 0x7fc8540b2090 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc858006fd0 tx=0x7fc858008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:05.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:05.983+0000 7fc86c9ef700 1 -- 172.21.15.96:0/1649478479 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc86002a020 con 0x7fc868104be0 2024-04-25T14:13:06.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:06.192+0000 7fc86d9f1700 1 -- 172.21.15.96:0/1649478479 --> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fc868063ad0 con 0x7fc8540afb80 2024-04-25T14:13:06.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:06.206+0000 7fc86c9ef700 1 -- 172.21.15.96:0/1649478479 <== mgr.14152 v2:172.21.15.96:6800/536332425 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7fc868063ad0 con 0x7fc8540afb80 2024-04-25T14:13:06.206 INFO:teuthology.orchestra.run.smithi096.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T14:13:06.206 INFO:teuthology.orchestra.run.smithi096.stdout:mds.a smithi096 running (105s) 18s ago 105s 87.5M - 19.0.0-3291-g994e2f20 d53c96779a29 e15473296981 2024-04-25T14:13:06.206 INFO:teuthology.orchestra.run.smithi096.stdout:mds.b smithi097 running (102s) 36s ago 102s 96.2M - 19.0.0-3291-g994e2f20 d53c96779a29 7bfcbedc3a91 2024-04-25T14:13:06.206 INFO:teuthology.orchestra.run.smithi096.stdout:mds.c smithi151 running (117s) 35s ago 117s 87.2M - 19.0.0-3291-g994e2f20 d53c96779a29 6b065404a694 2024-04-25T14:13:06.206 INFO:teuthology.orchestra.run.smithi096.stdout:mds.d smithi096 running (2m) 18s ago 2m 81.6M - 19.0.0-3291-g994e2f20 d53c96779a29 d0538fbe5a3c 2024-04-25T14:13:06.206 INFO:teuthology.orchestra.run.smithi096.stdout:mds.e smithi097 running (2m) 36s ago 2m 87.8M - 19.0.0-3291-g994e2f20 d53c96779a29 5fa8c1d3bf3a 2024-04-25T14:13:06.206 INFO:teuthology.orchestra.run.smithi096.stdout:mds.f smithi151 running (2m) 35s ago 119s 87.4M - 19.0.0-3291-g994e2f20 d53c96779a29 629f47d1bed2 2024-04-25T14:13:06.206 INFO:teuthology.orchestra.run.smithi096.stdout:mds.g smithi096 running (108s) 18s ago 108s 83.9M - 19.0.0-3291-g994e2f20 d53c96779a29 64e700cd8c53 2024-04-25T14:13:06.206 INFO:teuthology.orchestra.run.smithi096.stdout:mds.h smithi097 running (114s) 36s ago 114s 85.4M - 19.0.0-3291-g994e2f20 d53c96779a29 84a82aacbfa9 2024-04-25T14:13:06.206 INFO:teuthology.orchestra.run.smithi096.stdout:mds.i smithi151 running (111s) 35s ago 111s 85.0M - 19.0.0-3291-g994e2f20 d53c96779a29 e8f5258ca3f5 2024-04-25T14:13:06.206 INFO:teuthology.orchestra.run.smithi096.stdout:mds.j smithi096 running (2m) 18s ago 2m 81.6M - 19.0.0-3291-g994e2f20 d53c96779a29 dc649cef1a74 2024-04-25T14:13:06.207 INFO:teuthology.orchestra.run.smithi096.stdout:mds.k smithi097 running (2m) 36s ago 2m 83.7M - 19.0.0-3291-g994e2f20 d53c96779a29 e7a56b736f0f 2024-04-25T14:13:06.207 INFO:teuthology.orchestra.run.smithi096.stdout:mds.l smithi151 running (2m) 35s ago 2m 85.4M - 19.0.0-3291-g994e2f20 d53c96779a29 545a1bff6ffd 2024-04-25T14:13:06.207 INFO:teuthology.orchestra.run.smithi096.stdout:mgr.x smithi096 *:9283,8765 running (9m) 18s ago 9m 582M - 19.0.0-3291-g994e2f20 d53c96779a29 de6d45d4d236 2024-04-25T14:13:06.207 INFO:teuthology.orchestra.run.smithi096.stdout:mgr.y smithi097 *:8443,8765 running (7m) 36s ago 7m 523M - 19.0.0-3291-g994e2f20 d53c96779a29 38308e63f649 2024-04-25T14:13:06.207 INFO:teuthology.orchestra.run.smithi096.stdout:mgr.z smithi151 *:8443,8765 running (7m) 35s ago 7m 520M - 19.0.0-3291-g994e2f20 d53c96779a29 e616504caf4a 2024-04-25T14:13:06.207 INFO:teuthology.orchestra.run.smithi096.stdout:mon.a smithi096 running (9m) 18s ago 9m 117M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 18d82172990b 2024-04-25T14:13:06.207 INFO:teuthology.orchestra.run.smithi096.stdout:mon.b smithi097 running (7m) 36s ago 7m 107M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 affb5a16a9fc 2024-04-25T14:13:06.207 INFO:teuthology.orchestra.run.smithi096.stdout:mon.c smithi151 running (7m) 35s ago 7m 106M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 412fdde6b2cb 2024-04-25T14:13:06.207 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0 smithi096 running (7m) 18s ago 7m 125M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 14666124012b 2024-04-25T14:13:06.207 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1 smithi097 running (6m) 36s ago 6m 112M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 25f071ec389f 2024-04-25T14:13:06.207 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2 smithi151 running (6m) 35s ago 6m 113M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 442f7d1ce4db 2024-04-25T14:13:06.207 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3 smithi096 running (6m) 18s ago 6m 110M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 8f7622abc3a1 2024-04-25T14:13:06.207 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4 smithi097 running (5m) 36s ago 5m 113M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 7ee9cc2432a6 2024-04-25T14:13:06.207 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5 smithi151 running (5m) 35s ago 5m 110M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 91b32bc02409 2024-04-25T14:13:06.207 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6 smithi096 running (4m) 18s ago 4m 121M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 dfe424cc4634 2024-04-25T14:13:06.208 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7 smithi097 running (4m) 36s ago 4m 124M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 bcf004867959 2024-04-25T14:13:06.208 INFO:teuthology.orchestra.run.smithi096.stdout:osd.8 smithi151 running (3m) 35s ago 3m 125M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 9b0962026659 2024-04-25T14:13:06.208 INFO:teuthology.orchestra.run.smithi096.stdout:osd.9 smithi096 running (3m) 18s ago 3m 110M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 f57826b9403e 2024-04-25T14:13:06.208 INFO:teuthology.orchestra.run.smithi096.stdout:osd.10 smithi097 running (3m) 36s ago 3m 122M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 d292f35c4984 2024-04-25T14:13:06.208 INFO:teuthology.orchestra.run.smithi096.stdout:osd.11 smithi151 running (2m) 35s ago 2m 117M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 27e6574c58ea 2024-04-25T14:13:06.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:06.208+0000 7fc86d9f1700 1 -- 172.21.15.96:0/1649478479 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fc8540afb80 msgr2=0x7fc8540b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:06.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:06.208+0000 7fc86d9f1700 1 --2- 172.21.15.96:0/1649478479 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fc8540afb80 0x7fc8540b2090 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc858006fd0 tx=0x7fc858008040 comp rx=0 tx=0).stop 2024-04-25T14:13:06.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:06.208+0000 7fc86d9f1700 1 -- 172.21.15.96:0/1649478479 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc868104be0 msgr2=0x7fc8681a1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:06.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:06.208+0000 7fc86d9f1700 1 --2- 172.21.15.96:0/1649478479 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc868104be0 0x7fc8681a1b00 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc86000b830 tx=0x7fc860004dc0 comp rx=0 tx=0).stop 2024-04-25T14:13:06.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:06.208+0000 7fc86d9f1700 1 -- 172.21.15.96:0/1649478479 shutdown_connections 2024-04-25T14:13:06.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:06.208+0000 7fc86d9f1700 1 --2- 172.21.15.96:0/1649478479 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fc8540afb80 0x7fc8540b2090 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:06.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:06.208+0000 7fc86d9f1700 1 --2- 172.21.15.96:0/1649478479 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc868104be0 0x7fc8681a1b00 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:06.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:06.208+0000 7fc86d9f1700 1 --2- 172.21.15.96:0/1649478479 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc868103110 0x7fc86806bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:06.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:06.208+0000 7fc86d9f1700 1 --2- 172.21.15.96:0/1649478479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc868104240 0x7fc86819f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:06.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:06.208+0000 7fc86d9f1700 1 -- 172.21.15.96:0/1649478479 >> 172.21.15.96:0/1649478479 conn(0x7fc8680fe750 msgr2=0x7fc868107470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:06.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:06.208+0000 7fc86d9f1700 1 -- 172.21.15.96:0/1649478479 shutdown_connections 2024-04-25T14:13:06.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:06.208+0000 7fc86d9f1700 1 -- 172.21.15.96:0/1649478479 wait complete. 2024-04-25T14:13:06.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:06 smithi096 ceph-mon[29207]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-25T14:13:06.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:06 smithi097 ceph-mon[32720]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-25T14:13:06.935 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-25T14:13:06.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:06 smithi151 ceph-mon[32990]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-25T14:13:07.176 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:13:07.577 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:07 smithi096 ceph-mon[29207]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:13:07.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:07 smithi097 ceph-mon[32720]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:13:07.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:07 smithi151 ceph-mon[32990]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:13:08.736 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:08 smithi096 ceph-mon[29207]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 511 B/s wr, 12 op/s 2024-04-25T14:13:08.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:08 smithi097 ceph-mon[32720]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 511 B/s wr, 12 op/s 2024-04-25T14:13:08.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:08 smithi151 ceph-mon[32990]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 511 B/s wr, 12 op/s 2024-04-25T14:13:08.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:08.958+0000 7f1274348700 1 -- 172.21.15.96:0/1918542277 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f126c105340 msgr2=0x7f126c105750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:08.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:08.958+0000 7f1274348700 1 --2- 172.21.15.96:0/1918542277 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f126c105340 0x7f126c105750 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f125c009990 tx=0x7f125c009ca0 comp rx=0 tx=0).stop 2024-04-25T14:13:08.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:08.958+0000 7f1274348700 1 -- 172.21.15.96:0/1918542277 shutdown_connections 2024-04-25T14:13:08.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:08.958+0000 7f1274348700 1 --2- 172.21.15.96:0/1918542277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f126c105ce0 0x7f126c10c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:08.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:08.958+0000 7f1274348700 1 --2- 172.21.15.96:0/1918542277 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f126c104210 0x7f126c104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:08.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:08.958+0000 7f1274348700 1 --2- 172.21.15.96:0/1918542277 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f126c105340 0x7f126c105750 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:08.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:08.958+0000 7f1274348700 1 -- 172.21.15.96:0/1918542277 >> 172.21.15.96:0/1918542277 conn(0x7f126c0ff870 msgr2=0x7f126c101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:08.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:08.958+0000 7f1274348700 1 -- 172.21.15.96:0/1918542277 shutdown_connections 2024-04-25T14:13:08.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:08.958+0000 7f1274348700 1 -- 172.21.15.96:0/1918542277 wait complete. 2024-04-25T14:13:08.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:08.958+0000 7f1274348700 1 Processor -- start 2024-04-25T14:13:08.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:08.958+0000 7f1274348700 1 -- start start 2024-04-25T14:13:08.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:08.959+0000 7f1274348700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f126c104210 0x7f126c19a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:08.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:08.959+0000 7f1274348700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f126c105340 0x7f126c19acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:08.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:08.959+0000 7f1274348700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f126c105ce0 0x7f126c1a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:08.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:08.959+0000 7f1274348700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f126c10fc70 con 0x7f126c105ce0 2024-04-25T14:13:08.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:08.959+0000 7f1274348700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f126c10faf0 con 0x7f126c104210 2024-04-25T14:13:08.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:08.959+0000 7f1274348700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f126c10fdf0 con 0x7f126c105340 2024-04-25T14:13:08.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:08.959+0000 7f12728e5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f126c105ce0 0x7f126c1a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:08.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:08.959+0000 7f12728e5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f126c105ce0 0x7f126c1a1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49242/0 (socket says 172.21.15.96:49242) 2024-04-25T14:13:08.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:08.959+0000 7f12728e5700 1 -- 172.21.15.96:0/1292296313 learned_addr learned my addr 172.21.15.96:0/1292296313 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:08.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:08.959+0000 7f12718e3700 1 --2- 172.21.15.96:0/1292296313 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f126c105340 0x7f126c19acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:08.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:08.959+0000 7f12720e4700 1 --2- 172.21.15.96:0/1292296313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f126c104210 0x7f126c19a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:08.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:08.959+0000 7f12728e5700 1 -- 172.21.15.96:0/1292296313 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f126c105340 msgr2=0x7f126c19acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:08.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:08.959+0000 7f12728e5700 1 --2- 172.21.15.96:0/1292296313 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f126c105340 0x7f126c19acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:08.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:08.959+0000 7f12728e5700 1 -- 172.21.15.96:0/1292296313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f126c104210 msgr2=0x7f126c19a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:08.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:08.959+0000 7f12728e5700 1 --2- 172.21.15.96:0/1292296313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f126c104210 0x7f126c19a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:08.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:08.959+0000 7f12728e5700 1 -- 172.21.15.96:0/1292296313 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f126c0769d0 con 0x7f126c105ce0 2024-04-25T14:13:08.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:08.959+0000 7f12728e5700 1 --2- 172.21.15.96:0/1292296313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f126c105ce0 0x7f126c1a1c90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1264007d70 tx=0x7f126400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:08.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:08.959+0000 7f12637fe700 1 -- 172.21.15.96:0/1292296313 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1264016020 con 0x7f126c105ce0 2024-04-25T14:13:08.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:08.960+0000 7f1274348700 1 -- 172.21.15.96:0/1292296313 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f126c076cc0 con 0x7f126c105ce0 2024-04-25T14:13:08.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:08.960+0000 7f12637fe700 1 -- 172.21.15.96:0/1292296313 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f126400ad40 con 0x7f126c105ce0 2024-04-25T14:13:08.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:08.960+0000 7f12637fe700 1 -- 172.21.15.96:0/1292296313 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12640115e0 con 0x7f126c105ce0 2024-04-25T14:13:08.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:08.960+0000 7f1274348700 1 -- 172.21.15.96:0/1292296313 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f126c0771b0 con 0x7f126c105ce0 2024-04-25T14:13:08.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:08.960+0000 7f1274348700 1 -- 172.21.15.96:0/1292296313 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f126c051190 con 0x7f126c105ce0 2024-04-25T14:13:08.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:08.963+0000 7f12637fe700 1 -- 172.21.15.96:0/1292296313 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f1264019020 con 0x7f126c105ce0 2024-04-25T14:13:08.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:08.963+0000 7f12637fe700 1 --2- 172.21.15.96:0/1292296313 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f12580afb80 0x7f12580b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:08.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:08.963+0000 7f12720e4700 1 --2- 172.21.15.96:0/1292296313 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f12580afb80 0x7f12580b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:08.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:08.963+0000 7f12637fe700 1 -- 172.21.15.96:0/1292296313 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f12640df000 con 0x7f126c105ce0 2024-04-25T14:13:08.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:08.964+0000 7f12720e4700 1 --2- 172.21.15.96:0/1292296313 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f12580afb80 0x7f12580b2090 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1268005950 tx=0x7f12680058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:08.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:08.967+0000 7f12637fe700 1 -- 172.21.15.96:0/1292296313 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1264020020 con 0x7f126c105ce0 2024-04-25T14:13:09.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:09.177+0000 7f1274348700 1 -- 172.21.15.96:0/1292296313 --> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f126c10af20 con 0x7f12580afb80 2024-04-25T14:13:09.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:09.183+0000 7f12637fe700 1 -- 172.21.15.96:0/1292296313 <== mgr.14152 v2:172.21.15.96:6800/536332425 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f126c10af20 con 0x7f12580afb80 2024-04-25T14:13:09.183 INFO:teuthology.orchestra.run.smithi096.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-25T14:13:09.183 INFO:teuthology.orchestra.run.smithi096.stdout:mds.all 12/12 39s ago 2m smithi096=a;smithi096=d;smithi096=g;smithi096=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi151=c;smithi151=f;smithi151=i;smithi151=l;count:12 2024-04-25T14:13:09.183 INFO:teuthology.orchestra.run.smithi096.stdout:mgr 3/3 39s ago 7m smithi096=x;smithi097=y;smithi151=z;count:3 2024-04-25T14:13:09.183 INFO:teuthology.orchestra.run.smithi096.stdout:mon 3/3 39s ago 8m smithi096:172.21.15.96=a;smithi097:172.21.15.97=b;smithi151:172.21.15.151=c;count:3 2024-04-25T14:13:09.183 INFO:teuthology.orchestra.run.smithi096.stdout:osd 12 39s ago - 2024-04-25T14:13:09.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:09.185+0000 7f1274348700 1 -- 172.21.15.96:0/1292296313 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f12580afb80 msgr2=0x7f12580b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:09.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:09.185+0000 7f1274348700 1 --2- 172.21.15.96:0/1292296313 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f12580afb80 0x7f12580b2090 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1268005950 tx=0x7f12680058e0 comp rx=0 tx=0).stop 2024-04-25T14:13:09.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:09.185+0000 7f1274348700 1 -- 172.21.15.96:0/1292296313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f126c105ce0 msgr2=0x7f126c1a1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:09.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:09.185+0000 7f1274348700 1 --2- 172.21.15.96:0/1292296313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f126c105ce0 0x7f126c1a1c90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1264007d70 tx=0x7f126400a340 comp rx=0 tx=0).stop 2024-04-25T14:13:09.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:09.185+0000 7f1274348700 1 -- 172.21.15.96:0/1292296313 shutdown_connections 2024-04-25T14:13:09.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:09.185+0000 7f1274348700 1 --2- 172.21.15.96:0/1292296313 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f12580afb80 0x7f12580b2090 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:09.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:09.185+0000 7f1274348700 1 --2- 172.21.15.96:0/1292296313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f126c105ce0 0x7f126c1a1c90 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:09.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:09.185+0000 7f1274348700 1 --2- 172.21.15.96:0/1292296313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f126c104210 0x7f126c19a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:09.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:09.185+0000 7f1274348700 1 --2- 172.21.15.96:0/1292296313 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f126c105340 0x7f126c19acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:09.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:09.185+0000 7f1274348700 1 -- 172.21.15.96:0/1292296313 >> 172.21.15.96:0/1292296313 conn(0x7f126c0ff870 msgr2=0x7f126c108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:09.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:09.185+0000 7f1274348700 1 -- 172.21.15.96:0/1292296313 shutdown_connections 2024-04-25T14:13:09.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:09.185+0000 7f1274348700 1 -- 172.21.15.96:0/1292296313 wait complete. 2024-04-25T14:13:10.114 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-25T14:13:10.354 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:13:10.761 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:10 smithi096 ceph-mon[29207]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:13:10.761 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:10 smithi096 ceph-mon[29207]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T14:13:10.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:10 smithi097 ceph-mon[32720]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:13:10.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:10 smithi097 ceph-mon[32720]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T14:13:10.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:10 smithi151 ceph-mon[32990]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:13:10.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:10 smithi151 ceph-mon[32990]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T14:13:12.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.210+0000 7f0ec79d9700 1 -- 172.21.15.96:0/2024468206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ec0104880 msgr2=0x7f0ec010c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:12.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.210+0000 7f0ec79d9700 1 --2- 172.21.15.96:0/2024468206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ec0104880 0x7f0ec010c590 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0eb800b0a0 tx=0x7f0eb800b3b0 comp rx=0 tx=0).stop 2024-04-25T14:13:12.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.210+0000 7f0ec79d9700 1 -- 172.21.15.96:0/2024468206 shutdown_connections 2024-04-25T14:13:12.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.210+0000 7f0ec79d9700 1 --2- 172.21.15.96:0/2024468206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ec0104880 0x7f0ec010c590 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:12.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.210+0000 7f0ec79d9700 1 --2- 172.21.15.96:0/2024468206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ec0105850 0x7f0ec0105c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:12.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.210+0000 7f0ec79d9700 1 --2- 172.21.15.96:0/2024468206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ec0103e00 0x7f0ec0104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:12.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.210+0000 7f0ec79d9700 1 -- 172.21.15.96:0/2024468206 >> 172.21.15.96:0/2024468206 conn(0x7f0ec00ff870 msgr2=0x7f0ec0101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:12.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.210+0000 7f0ec79d9700 1 -- 172.21.15.96:0/2024468206 shutdown_connections 2024-04-25T14:13:12.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.210+0000 7f0ec79d9700 1 -- 172.21.15.96:0/2024468206 wait complete. 2024-04-25T14:13:12.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.211+0000 7f0ec79d9700 1 Processor -- start 2024-04-25T14:13:12.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.211+0000 7f0ec79d9700 1 -- start start 2024-04-25T14:13:12.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.211+0000 7f0ec79d9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ec0103e00 0x7f0ec019a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:12.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.211+0000 7f0ec79d9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ec0104880 0x7f0ec019ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:12.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.211+0000 7f0ec79d9700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ec0105850 0x7f0ec01a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:12.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.211+0000 7f0ec79d9700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec010fd90 con 0x7f0ec0104880 2024-04-25T14:13:12.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.211+0000 7f0ec79d9700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec010fc10 con 0x7f0ec0103e00 2024-04-25T14:13:12.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.211+0000 7f0ec79d9700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec010ff10 con 0x7f0ec0105850 2024-04-25T14:13:12.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.211+0000 7f0ec4f74700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ec0104880 0x7f0ec019ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:12.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.211+0000 7f0ec4f74700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ec0104880 0x7f0ec019ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56316/0 (socket says 172.21.15.96:56316) 2024-04-25T14:13:12.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.211+0000 7f0ec4f74700 1 -- 172.21.15.96:0/1576577424 learned_addr learned my addr 172.21.15.96:0/1576577424 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:12.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.211+0000 7f0ec5f76700 1 --2- 172.21.15.96:0/1576577424 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ec0105850 0x7f0ec01a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:12.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.212+0000 7f0ec5775700 1 --2- 172.21.15.96:0/1576577424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ec0103e00 0x7f0ec019a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:12.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.212+0000 7f0ec4f74700 1 -- 172.21.15.96:0/1576577424 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ec0105850 msgr2=0x7f0ec01a1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:12.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.212+0000 7f0ec4f74700 1 --2- 172.21.15.96:0/1576577424 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ec0105850 0x7f0ec01a1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:12.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.212+0000 7f0ec4f74700 1 -- 172.21.15.96:0/1576577424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ec0103e00 msgr2=0x7f0ec019a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:12.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.212+0000 7f0ec4f74700 1 --2- 172.21.15.96:0/1576577424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ec0103e00 0x7f0ec019a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:12.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.212+0000 7f0ec4f74700 1 -- 172.21.15.96:0/1576577424 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ec01a2350 con 0x7f0ec0104880 2024-04-25T14:13:12.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.212+0000 7f0ec5775700 1 --2- 172.21.15.96:0/1576577424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ec0103e00 0x7f0ec019a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:12.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.212+0000 7f0ec4f74700 1 --2- 172.21.15.96:0/1576577424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ec0104880 0x7f0ec019ad30 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0eb000e9a0 tx=0x7f0eb000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:12.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.212+0000 7f0eb67fc700 1 -- 172.21.15.96:0/1576577424 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb000cbd0 con 0x7f0ec0104880 2024-04-25T14:13:12.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.212+0000 7f0eb67fc700 1 -- 172.21.15.96:0/1576577424 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0eb000cd70 con 0x7f0ec0104880 2024-04-25T14:13:12.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.212+0000 7f0ec79d9700 1 -- 172.21.15.96:0/1576577424 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ec01a2640 con 0x7f0ec0104880 2024-04-25T14:13:12.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.212+0000 7f0eb67fc700 1 -- 172.21.15.96:0/1576577424 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0018680 con 0x7f0ec0104880 2024-04-25T14:13:12.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.212+0000 7f0ec79d9700 1 -- 172.21.15.96:0/1576577424 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ec01a2bd0 con 0x7f0ec0104880 2024-04-25T14:13:12.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.213+0000 7f0ec79d9700 1 -- 172.21.15.96:0/1576577424 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ec0051190 con 0x7f0ec0104880 2024-04-25T14:13:12.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.214+0000 7f0eb67fc700 1 -- 172.21.15.96:0/1576577424 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f0eb0010430 con 0x7f0ec0104880 2024-04-25T14:13:12.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.215+0000 7f0eb67fc700 1 --2- 172.21.15.96:0/1576577424 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f0eac0afc00 0x7f0eac0b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:12.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.215+0000 7f0ec5775700 1 --2- 172.21.15.96:0/1576577424 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f0eac0afc00 0x7f0eac0b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:12.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.215+0000 7f0eb67fc700 1 -- 172.21.15.96:0/1576577424 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f0eb00df310 con 0x7f0ec0104880 2024-04-25T14:13:12.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.215+0000 7f0ec5775700 1 --2- 172.21.15.96:0/1576577424 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f0eac0afc00 0x7f0eac0b2110 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0105240 tx=0x7f0ebc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:12.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.219+0000 7f0eb67fc700 1 -- 172.21.15.96:0/1576577424 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0eb00a7ed0 con 0x7f0ec0104880 2024-04-25T14:13:12.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.422+0000 7f0ec79d9700 1 -- 172.21.15.96:0/1576577424 --> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f0ec002e930 con 0x7f0eac0afc00 2024-04-25T14:13:12.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.423+0000 7f0eb67fc700 1 -- 172.21.15.96:0/1576577424 <== mgr.14152 v2:172.21.15.96:6800/536332425 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f0ec002e930 con 0x7f0eac0afc00 2024-04-25T14:13:12.423 INFO:teuthology.orchestra.run.smithi096.stdout:HOST ADDR LABELS STATUS 2024-04-25T14:13:12.423 INFO:teuthology.orchestra.run.smithi096.stdout:smithi096 172.21.15.96 2024-04-25T14:13:12.423 INFO:teuthology.orchestra.run.smithi096.stdout:smithi097 172.21.15.97 2024-04-25T14:13:12.424 INFO:teuthology.orchestra.run.smithi096.stdout:smithi151 172.21.15.151 2024-04-25T14:13:12.424 INFO:teuthology.orchestra.run.smithi096.stdout:3 hosts in cluster 2024-04-25T14:13:12.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.426+0000 7f0ec79d9700 1 -- 172.21.15.96:0/1576577424 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f0eac0afc00 msgr2=0x7f0eac0b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:12.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.426+0000 7f0ec79d9700 1 --2- 172.21.15.96:0/1576577424 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f0eac0afc00 0x7f0eac0b2110 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0105240 tx=0x7f0ebc005f90 comp rx=0 tx=0).stop 2024-04-25T14:13:12.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.426+0000 7f0ec79d9700 1 -- 172.21.15.96:0/1576577424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ec0104880 msgr2=0x7f0ec019ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:12.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.426+0000 7f0ec79d9700 1 --2- 172.21.15.96:0/1576577424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ec0104880 0x7f0ec019ad30 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0eb000e9a0 tx=0x7f0eb000ecb0 comp rx=0 tx=0).stop 2024-04-25T14:13:12.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.426+0000 7f0ec79d9700 1 -- 172.21.15.96:0/1576577424 shutdown_connections 2024-04-25T14:13:12.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.426+0000 7f0ec79d9700 1 --2- 172.21.15.96:0/1576577424 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f0eac0afc00 0x7f0eac0b2110 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:12.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.426+0000 7f0ec79d9700 1 --2- 172.21.15.96:0/1576577424 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ec0105850 0x7f0ec01a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:12.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.426+0000 7f0ec79d9700 1 --2- 172.21.15.96:0/1576577424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ec0103e00 0x7f0ec019a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:12.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.426+0000 7f0ec79d9700 1 --2- 172.21.15.96:0/1576577424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ec0104880 0x7f0ec019ad30 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:12.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.426+0000 7f0ec79d9700 1 -- 172.21.15.96:0/1576577424 >> 172.21.15.96:0/1576577424 conn(0x7f0ec00ff870 msgr2=0x7f0ec0108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:12.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.426+0000 7f0ec79d9700 1 -- 172.21.15.96:0/1576577424 shutdown_connections 2024-04-25T14:13:12.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:12.426+0000 7f0ec79d9700 1 -- 172.21.15.96:0/1576577424 wait complete. 2024-04-25T14:13:12.799 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:12 smithi096 ceph-mon[29207]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T14:13:12.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:12 smithi097 ceph-mon[32720]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T14:13:12.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:12 smithi151 ceph-mon[32990]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T14:13:13.210 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-25T14:13:13.449 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:13:13.846 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:13 smithi096 ceph-mon[29207]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:13:13.925 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:13 smithi097 ceph-mon[32720]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:13:13.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:13 smithi151 ceph-mon[32990]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:13:14.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:14 smithi096 ceph-mon[29207]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T14:13:14.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:14 smithi097 ceph-mon[32720]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T14:13:14.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:14 smithi151 ceph-mon[32990]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T14:13:15.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.276+0000 7fd0abf54700 1 -- 172.21.15.96:0/3934451334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0a4077670 msgr2=0x7fd0a4112de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:15.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.276+0000 7fd0abf54700 1 --2- 172.21.15.96:0/3934451334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0a4077670 0x7fd0a4112de0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00b0a0 tx=0x7fd09c00b3b0 comp rx=0 tx=0).stop 2024-04-25T14:13:15.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.276+0000 7fd0abf54700 1 -- 172.21.15.96:0/3934451334 shutdown_connections 2024-04-25T14:13:15.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.276+0000 7fd0abf54700 1 --2- 172.21.15.96:0/3934451334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0a4077670 0x7fd0a4112de0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:15.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.276+0000 7fd0abf54700 1 --2- 172.21.15.96:0/3934451334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0a40787c0 0x7fd0a4076b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:15.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.276+0000 7fd0abf54700 1 --2- 172.21.15.96:0/3934451334 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0a40771a0 0x7fd0a4110890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:15.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.276+0000 7fd0abf54700 1 -- 172.21.15.96:0/3934451334 >> 172.21.15.96:0/3934451334 conn(0x7fd0a40ff8a0 msgr2=0x7fd0a4101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:15.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.276+0000 7fd0abf54700 1 -- 172.21.15.96:0/3934451334 shutdown_connections 2024-04-25T14:13:15.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.276+0000 7fd0abf54700 1 -- 172.21.15.96:0/3934451334 wait complete. 2024-04-25T14:13:15.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.277+0000 7fd0abf54700 1 Processor -- start 2024-04-25T14:13:15.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.277+0000 7fd0abf54700 1 -- start start 2024-04-25T14:13:15.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.277+0000 7fd0abf54700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0a40771a0 0x7fd0a41a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:15.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.277+0000 7fd0abf54700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0a4077670 0x7fd0a41a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:15.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.277+0000 7fd0abf54700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0a41a5fd0 0x7fd0a41a83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:15.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.277+0000 7fd0abf54700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a4116430 con 0x7fd0a40771a0 2024-04-25T14:13:15.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.277+0000 7fd0abf54700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a41162b0 con 0x7fd0a4077670 2024-04-25T14:13:15.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.277+0000 7fd0abf54700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a41165b0 con 0x7fd0a41a5fd0 2024-04-25T14:13:15.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.277+0000 7fd0a9cf0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0a40771a0 0x7fd0a41a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:15.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.277+0000 7fd0a9cf0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0a40771a0 0x7fd0a41a0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56332/0 (socket says 172.21.15.96:56332) 2024-04-25T14:13:15.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.277+0000 7fd0a9cf0700 1 -- 172.21.15.96:0/3619940182 learned_addr learned my addr 172.21.15.96:0/3619940182 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:15.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.277+0000 7fd0a94ef700 1 --2- 172.21.15.96:0/3619940182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0a4077670 0x7fd0a41a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:15.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.277+0000 7fd0a9cf0700 1 -- 172.21.15.96:0/3619940182 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0a41a5fd0 msgr2=0x7fd0a41a83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:15.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.277+0000 7fd0a9cf0700 1 --2- 172.21.15.96:0/3619940182 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0a41a5fd0 0x7fd0a41a83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:15.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.277+0000 7fd0a9cf0700 1 -- 172.21.15.96:0/3619940182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0a4077670 msgr2=0x7fd0a41a1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:15.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.277+0000 7fd0a9cf0700 1 --2- 172.21.15.96:0/3619940182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0a4077670 0x7fd0a41a1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:15.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.277+0000 7fd0a9cf0700 1 -- 172.21.15.96:0/3619940182 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a41a8a70 con 0x7fd0a40771a0 2024-04-25T14:13:15.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.277+0000 7fd0a94ef700 1 --2- 172.21.15.96:0/3619940182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0a4077670 0x7fd0a41a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:15.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.278+0000 7fd0a9cf0700 1 --2- 172.21.15.96:0/3619940182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0a40771a0 0x7fd0a41a0b00 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd0a000d980 tx=0x7fd0a000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:15.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.278+0000 7fd09affd700 1 -- 172.21.15.96:0/3619940182 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0017070 con 0x7fd0a40771a0 2024-04-25T14:13:15.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.278+0000 7fd0abf54700 1 -- 172.21.15.96:0/3619940182 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0a41a8d60 con 0x7fd0a40771a0 2024-04-25T14:13:15.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.278+0000 7fd09affd700 1 -- 172.21.15.96:0/3619940182 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0a0010460 con 0x7fd0a40771a0 2024-04-25T14:13:15.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.278+0000 7fd09affd700 1 -- 172.21.15.96:0/3619940182 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a000de60 con 0x7fd0a40771a0 2024-04-25T14:13:15.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.278+0000 7fd0abf54700 1 -- 172.21.15.96:0/3619940182 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0a41a92e0 con 0x7fd0a40771a0 2024-04-25T14:13:15.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.279+0000 7fd0abf54700 1 -- 172.21.15.96:0/3619940182 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0a4051190 con 0x7fd0a40771a0 2024-04-25T14:13:15.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.280+0000 7fd09affd700 1 -- 172.21.15.96:0/3619940182 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fd0a0010610 con 0x7fd0a40771a0 2024-04-25T14:13:15.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.281+0000 7fd09affd700 1 --2- 172.21.15.96:0/3619940182 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd0900afb80 0x7fd0900b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:15.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.281+0000 7fd0a94ef700 1 --2- 172.21.15.96:0/3619940182 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd0900afb80 0x7fd0900b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:15.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.281+0000 7fd09affd700 1 -- 172.21.15.96:0/3619940182 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fd0a00de7b0 con 0x7fd0a40771a0 2024-04-25T14:13:15.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.281+0000 7fd0a94ef700 1 --2- 172.21.15.96:0/3619940182 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd0900afb80 0x7fd0900b2090 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd0a41a1bb0 tx=0x7fd094008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:15.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.285+0000 7fd09affd700 1 -- 172.21.15.96:0/3619940182 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd0a00a7370 con 0x7fd0a40771a0 2024-04-25T14:13:15.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.493+0000 7fd0abf54700 1 -- 172.21.15.96:0/3619940182 --> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fd0a4063ad0 con 0x7fd0900afb80 2024-04-25T14:13:15.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.494+0000 7fd09affd700 1 -- 172.21.15.96:0/3619940182 <== mgr.14152 v2:172.21.15.96:6800/536332425 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fd0a4063ad0 con 0x7fd0900afb80 2024-04-25T14:13:15.495 INFO:teuthology.orchestra.run.smithi096.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-25T14:13:15.495 INFO:teuthology.orchestra.run.smithi096.stdout:smithi096 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NH400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-25T14:13:15.495 INFO:teuthology.orchestra.run.smithi096.stdout:smithi097 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204013T400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-25T14:13:15.495 INFO:teuthology.orchestra.run.smithi096.stdout:smithi151 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300JJ400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-25T14:13:15.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.497+0000 7fd0abf54700 1 -- 172.21.15.96:0/3619940182 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd0900afb80 msgr2=0x7fd0900b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:15.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.497+0000 7fd0abf54700 1 --2- 172.21.15.96:0/3619940182 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd0900afb80 0x7fd0900b2090 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd0a41a1bb0 tx=0x7fd094008040 comp rx=0 tx=0).stop 2024-04-25T14:13:15.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.497+0000 7fd0abf54700 1 -- 172.21.15.96:0/3619940182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0a40771a0 msgr2=0x7fd0a41a0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:15.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.497+0000 7fd0abf54700 1 --2- 172.21.15.96:0/3619940182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0a40771a0 0x7fd0a41a0b00 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd0a000d980 tx=0x7fd0a000dd40 comp rx=0 tx=0).stop 2024-04-25T14:13:15.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.497+0000 7fd0abf54700 1 -- 172.21.15.96:0/3619940182 shutdown_connections 2024-04-25T14:13:15.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.497+0000 7fd0abf54700 1 --2- 172.21.15.96:0/3619940182 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd0900afb80 0x7fd0900b2090 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:15.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.497+0000 7fd0abf54700 1 --2- 172.21.15.96:0/3619940182 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0a41a5fd0 0x7fd0a41a83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:15.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.497+0000 7fd0abf54700 1 --2- 172.21.15.96:0/3619940182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0a40771a0 0x7fd0a41a0b00 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:15.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.497+0000 7fd0abf54700 1 --2- 172.21.15.96:0/3619940182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0a4077670 0x7fd0a41a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:15.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.497+0000 7fd0abf54700 1 -- 172.21.15.96:0/3619940182 >> 172.21.15.96:0/3619940182 conn(0x7fd0a40ff8a0 msgr2=0x7fd0a41128a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:15.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.497+0000 7fd0abf54700 1 -- 172.21.15.96:0/3619940182 shutdown_connections 2024-04-25T14:13:15.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:15.497+0000 7fd0abf54700 1 -- 172.21.15.96:0/3619940182 wait complete. 2024-04-25T14:13:16.319 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T14:13:16.327 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi096.front.sepia.ceph.com 2024-04-25T14:13:16.327 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-25T14:13:16.574 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:13:16.836 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:16 smithi096 ceph-mon[29207]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T14:13:16.836 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:16 smithi096 ceph-mon[29207]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:13:16.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:16 smithi097 ceph-mon[32720]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T14:13:16.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:16 smithi097 ceph-mon[32720]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:13:16.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:16 smithi151 ceph-mon[32990]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T14:13:16.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:16 smithi151 ceph-mon[32990]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:13:18.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.317+0000 7f1f10c31700 1 -- 172.21.15.96:0/1719289576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f0c107310 msgr2=0x7f1f0c1097f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:18.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.317+0000 7f1f10c31700 1 --2- 172.21.15.96:0/1719289576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f0c107310 0x7f1f0c1097f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1f00009990 tx=0x7f1f00009ca0 comp rx=0 tx=0).stop 2024-04-25T14:13:18.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.317+0000 7f1f10c31700 1 -- 172.21.15.96:0/1719289576 shutdown_connections 2024-04-25T14:13:18.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.317+0000 7f1f10c31700 1 --2- 172.21.15.96:0/1719289576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f0c109e80 0x7f1f0c10c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:18.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.317+0000 7f1f10c31700 1 --2- 172.21.15.96:0/1719289576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1f0c076580 0x7f1f0c106d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:18.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.317+0000 7f1f10c31700 1 --2- 172.21.15.96:0/1719289576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f0c107310 0x7f1f0c1097f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:18.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.317+0000 7f1f10c31700 1 -- 172.21.15.96:0/1719289576 >> 172.21.15.96:0/1719289576 conn(0x7f1f0c0fe730 msgr2=0x7f1f0c100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:18.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.317+0000 7f1f10c31700 1 -- 172.21.15.96:0/1719289576 shutdown_connections 2024-04-25T14:13:18.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.317+0000 7f1f10c31700 1 -- 172.21.15.96:0/1719289576 wait complete. 2024-04-25T14:13:18.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.318+0000 7f1f10c31700 1 Processor -- start 2024-04-25T14:13:18.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.318+0000 7f1f10c31700 1 -- start start 2024-04-25T14:13:18.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.318+0000 7f1f10c31700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f0c076580 0x7f1f0c19a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:18.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.318+0000 7f1f10c31700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1f0c107310 0x7f1f0c19acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:18.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.318+0000 7f1f10c31700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f0c109e80 0x7f1f0c1a1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:18.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.318+0000 7f1f10c31700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1f0c10fc70 con 0x7f1f0c109e80 2024-04-25T14:13:18.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.318+0000 7f1f10c31700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1f0c10faf0 con 0x7f1f0c076580 2024-04-25T14:13:18.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.318+0000 7f1f10c31700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1f0c10fdf0 con 0x7f1f0c107310 2024-04-25T14:13:18.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.318+0000 7f1f0ad9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f0c109e80 0x7f1f0c1a1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:18.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.318+0000 7f1f0ad9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f0c109e80 0x7f1f0c1a1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56352/0 (socket says 172.21.15.96:56352) 2024-04-25T14:13:18.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.318+0000 7f1f0ad9d700 1 -- 172.21.15.96:0/3240190817 learned_addr learned my addr 172.21.15.96:0/3240190817 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:18.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.318+0000 7f1f0a59c700 1 --2- 172.21.15.96:0/3240190817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f0c076580 0x7f1f0c19a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:18.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.318+0000 7f1f09d9b700 1 --2- 172.21.15.96:0/3240190817 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1f0c107310 0x7f1f0c19acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:18.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.318+0000 7f1f0ad9d700 1 -- 172.21.15.96:0/3240190817 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1f0c107310 msgr2=0x7f1f0c19acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:18.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.318+0000 7f1f0ad9d700 1 --2- 172.21.15.96:0/3240190817 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1f0c107310 0x7f1f0c19acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:18.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.318+0000 7f1f0ad9d700 1 -- 172.21.15.96:0/3240190817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f0c076580 msgr2=0x7f1f0c19a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:18.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.318+0000 7f1f0ad9d700 1 --2- 172.21.15.96:0/3240190817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f0c076580 0x7f1f0c19a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:18.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.319+0000 7f1f0ad9d700 1 -- 172.21.15.96:0/3240190817 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f0c1a22e0 con 0x7f1f0c109e80 2024-04-25T14:13:18.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.319+0000 7f1f0ad9d700 1 --2- 172.21.15.96:0/3240190817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f0c109e80 0x7f1f0c1a1c60 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1f0400b970 tx=0x7f1f0400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:18.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.319+0000 7f1efb7fe700 1 -- 172.21.15.96:0/3240190817 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f04019070 con 0x7f1f0c109e80 2024-04-25T14:13:18.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.319+0000 7f1efb7fe700 1 -- 172.21.15.96:0/3240190817 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f0400cde0 con 0x7f1f0c109e80 2024-04-25T14:13:18.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.319+0000 7f1f10c31700 1 -- 172.21.15.96:0/3240190817 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f0c1a25d0 con 0x7f1f0c109e80 2024-04-25T14:13:18.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.319+0000 7f1efb7fe700 1 -- 172.21.15.96:0/3240190817 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f04007370 con 0x7f1f0c109e80 2024-04-25T14:13:18.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.319+0000 7f1f10c31700 1 -- 172.21.15.96:0/3240190817 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f0c1a2b90 con 0x7f1f0c109e80 2024-04-25T14:13:18.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.320+0000 7f1f10c31700 1 -- 172.21.15.96:0/3240190817 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f0c051190 con 0x7f1f0c109e80 2024-04-25T14:13:18.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.321+0000 7f1efb7fe700 1 -- 172.21.15.96:0/3240190817 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f1f0401c070 con 0x7f1f0c109e80 2024-04-25T14:13:18.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.322+0000 7f1efb7fe700 1 --2- 172.21.15.96:0/3240190817 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1ef40afb80 0x7f1ef40b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:18.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.322+0000 7f1f0a59c700 1 --2- 172.21.15.96:0/3240190817 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1ef40afb80 0x7f1ef40b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:18.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.322+0000 7f1efb7fe700 1 -- 172.21.15.96:0/3240190817 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f1f040de830 con 0x7f1f0c109e80 2024-04-25T14:13:18.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.322+0000 7f1f0a59c700 1 --2- 172.21.15.96:0/3240190817 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1ef40afb80 0x7f1ef40b2090 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1efc005950 tx=0x7f1efc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:18.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.325+0000 7f1efb7fe700 1 -- 172.21.15.96:0/3240190817 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1f04023070 con 0x7f1f0c109e80 2024-04-25T14:13:18.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.571+0000 7f1f10c31700 1 -- 172.21.15.96:0/3240190817 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f1f0c02e960 con 0x7f1f0c109e80 2024-04-25T14:13:18.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.571+0000 7f1efb7fe700 1 -- 172.21.15.96:0/3240190817 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+3087 (secure 0 0 0) 0x7f1f040a73f0 con 0x7f1f0c109e80 2024-04-25T14:13:18.571 INFO:teuthology.orchestra.run.smithi096.stdout:e25 2024-04-25T14:13:18.571 INFO:teuthology.orchestra.run.smithi096.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T14:13:18.571 INFO:teuthology.orchestra.run.smithi096.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T14:13:18.571 INFO:teuthology.orchestra.run.smithi096.stdout:legacy client fscid: 1 2024-04-25T14:13:18.571 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:13:18.572 INFO:teuthology.orchestra.run.smithi096.stdout:Filesystem 'cephfs' (1) 2024-04-25T14:13:18.572 INFO:teuthology.orchestra.run.smithi096.stdout:fs_name cephfs 2024-04-25T14:13:18.572 INFO:teuthology.orchestra.run.smithi096.stdout:epoch 25 2024-04-25T14:13:18.572 INFO:teuthology.orchestra.run.smithi096.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-25T14:13:18.572 INFO:teuthology.orchestra.run.smithi096.stdout:created 2024-04-25T14:11:28.886961+0000 2024-04-25T14:13:18.572 INFO:teuthology.orchestra.run.smithi096.stdout:modified 2024-04-25T14:11:35.532775+0000 2024-04-25T14:13:18.572 INFO:teuthology.orchestra.run.smithi096.stdout:tableserver 0 2024-04-25T14:13:18.572 INFO:teuthology.orchestra.run.smithi096.stdout:root 0 2024-04-25T14:13:18.572 INFO:teuthology.orchestra.run.smithi096.stdout:session_timeout 300 2024-04-25T14:13:18.572 INFO:teuthology.orchestra.run.smithi096.stdout:session_autoclose 300 2024-04-25T14:13:18.572 INFO:teuthology.orchestra.run.smithi096.stdout:max_file_size 1099511627776 2024-04-25T14:13:18.572 INFO:teuthology.orchestra.run.smithi096.stdout:max_xattr_size 65536 2024-04-25T14:13:18.572 INFO:teuthology.orchestra.run.smithi096.stdout:required_client_features {} 2024-04-25T14:13:18.572 INFO:teuthology.orchestra.run.smithi096.stdout:last_failure 0 2024-04-25T14:13:18.573 INFO:teuthology.orchestra.run.smithi096.stdout:last_failure_osd_epoch 0 2024-04-25T14:13:18.573 INFO:teuthology.orchestra.run.smithi096.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T14:13:18.573 INFO:teuthology.orchestra.run.smithi096.stdout:max_mds 5 2024-04-25T14:13:18.573 INFO:teuthology.orchestra.run.smithi096.stdout:in 0,1,2,3,4 2024-04-25T14:13:18.573 INFO:teuthology.orchestra.run.smithi096.stdout:up {0=24536,1=24517,2=24478,3=24455,4=24427} 2024-04-25T14:13:18.573 INFO:teuthology.orchestra.run.smithi096.stdout:failed 2024-04-25T14:13:18.573 INFO:teuthology.orchestra.run.smithi096.stdout:damaged 2024-04-25T14:13:18.573 INFO:teuthology.orchestra.run.smithi096.stdout:stopped 2024-04-25T14:13:18.573 INFO:teuthology.orchestra.run.smithi096.stdout:data_pools [3] 2024-04-25T14:13:18.573 INFO:teuthology.orchestra.run.smithi096.stdout:metadata_pool 2 2024-04-25T14:13:18.573 INFO:teuthology.orchestra.run.smithi096.stdout:inline_data disabled 2024-04-25T14:13:18.573 INFO:teuthology.orchestra.run.smithi096.stdout:balancer 2024-04-25T14:13:18.573 INFO:teuthology.orchestra.run.smithi096.stdout:bal_rank_mask -1 2024-04-25T14:13:18.573 INFO:teuthology.orchestra.run.smithi096.stdout:standby_count_wanted 0 2024-04-25T14:13:18.574 INFO:teuthology.orchestra.run.smithi096.stdout:qdb_cluster leader: 24536 members: 24517,24536,24478,24455,24427 2024-04-25T14:13:18.574 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.b{0:24536} state up:active seq 3 addr [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T14:13:18.574 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.h{0:24497} state up:standby-replay seq 1 addr [v2:172.21.15.97:6836/3306780081,v1:172.21.15.97:6837/3306780081] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T14:13:18.574 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.i{1:24517} state up:active seq 6 addr [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T14:13:18.574 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.c{1:24490} state up:standby-replay seq 1 addr [v2:172.21.15.151:6836/2691348366,v1:172.21.15.151:6837/2691348366] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T14:13:18.574 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.f{2:24478} state up:active seq 8 addr [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T14:13:18.574 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.k{2:24419} state up:standby-replay seq 1 addr [v2:172.21.15.97:6832/1995324831,v1:172.21.15.97:6833/1995324831] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T14:13:18.574 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.e{3:24455} state up:active seq 9 addr [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T14:13:18.574 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.a{3:14652} state up:standby-replay seq 1 addr [v2:172.21.15.96:6840/1606748971,v1:172.21.15.96:6841/1606748971] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T14:13:18.574 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.l{4:24427} state up:active seq 12 addr [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T14:13:18.574 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.g{4:14640} state up:standby-replay seq 1 addr [v2:172.21.15.96:6838/4080308509,v1:172.21.15.96:6839/4080308509] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T14:13:18.574 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:13:18.575 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:13:18.575 INFO:teuthology.orchestra.run.smithi096.stdout:Standby daemons: 2024-04-25T14:13:18.575 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:13:18.575 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.j{-1:14556} state up:standby seq 1 addr [v2:172.21.15.96:6834/606227917,v1:172.21.15.96:6835/606227917] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T14:13:18.575 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.d{-1:14568} state up:standby seq 1 addr [v2:172.21.15.96:6836/3265300710,v1:172.21.15.96:6837/3265300710] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T14:13:18.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.573+0000 7f1f10c31700 1 -- 172.21.15.96:0/3240190817 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1ef40afb80 msgr2=0x7f1ef40b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:18.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.573+0000 7f1f10c31700 1 --2- 172.21.15.96:0/3240190817 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1ef40afb80 0x7f1ef40b2090 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1efc005950 tx=0x7f1efc00b410 comp rx=0 tx=0).stop 2024-04-25T14:13:18.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.573+0000 7f1f10c31700 1 -- 172.21.15.96:0/3240190817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f0c109e80 msgr2=0x7f1f0c1a1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:18.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.573+0000 7f1f10c31700 1 --2- 172.21.15.96:0/3240190817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f0c109e80 0x7f1f0c1a1c60 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1f0400b970 tx=0x7f1f0400bc80 comp rx=0 tx=0).stop 2024-04-25T14:13:18.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.573+0000 7f1f10c31700 1 -- 172.21.15.96:0/3240190817 shutdown_connections 2024-04-25T14:13:18.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.573+0000 7f1f10c31700 1 --2- 172.21.15.96:0/3240190817 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1ef40afb80 0x7f1ef40b2090 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:18.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.573+0000 7f1f10c31700 1 --2- 172.21.15.96:0/3240190817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f0c109e80 0x7f1f0c1a1c60 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:18.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.573+0000 7f1f10c31700 1 --2- 172.21.15.96:0/3240190817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f0c076580 0x7f1f0c19a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:18.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.573+0000 7f1f10c31700 1 --2- 172.21.15.96:0/3240190817 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1f0c107310 0x7f1f0c19acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:18.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.573+0000 7f1f10c31700 1 -- 172.21.15.96:0/3240190817 >> 172.21.15.96:0/3240190817 conn(0x7f1f0c0fe730 msgr2=0x7f1f0c105760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:18.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.573+0000 7f1f10c31700 1 -- 172.21.15.96:0/3240190817 shutdown_connections 2024-04-25T14:13:18.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:18.573+0000 7f1f10c31700 1 -- 172.21.15.96:0/3240190817 wait complete. 2024-04-25T14:13:18.577 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 25 2024-04-25T14:13:18.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:18 smithi096 ceph-mon[29207]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-04-25T14:13:18.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:18 smithi097 ceph-mon[32720]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-04-25T14:13:18.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:18 smithi151 ceph-mon[32990]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-04-25T14:13:19.452 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 59334182-030c-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-25T14:13:19.698 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:13:19.821 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:19 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/3240190817' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T14:13:19.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:19 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/3240190817' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T14:13:19.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:19 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/3240190817' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T14:13:20.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:20 smithi096 ceph-mon[29207]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T14:13:20.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:20 smithi097 ceph-mon[32720]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T14:13:20.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:20 smithi151 ceph-mon[32990]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T14:13:21.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.521+0000 7f4008779700 1 -- 172.21.15.96:0/694843571 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4000104010 msgr2=0x7f40001044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:21.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.521+0000 7f4008779700 1 --2- 172.21.15.96:0/694843571 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4000104010 0x7f40001044e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0009990 tx=0x7f3ff0009ca0 comp rx=0 tx=0).stop 2024-04-25T14:13:21.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.521+0000 7f4008779700 1 -- 172.21.15.96:0/694843571 shutdown_connections 2024-04-25T14:13:21.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.521+0000 7f4008779700 1 --2- 172.21.15.96:0/694843571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4000104b70 0x7f400010b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:21.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.521+0000 7f4008779700 1 --2- 172.21.15.96:0/694843571 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4000103630 0x7f4000103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:21.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.521+0000 7f4008779700 1 --2- 172.21.15.96:0/694843571 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4000104010 0x7f40001044e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:21.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.521+0000 7f4008779700 1 -- 172.21.15.96:0/694843571 >> 172.21.15.96:0/694843571 conn(0x7f40000fde80 msgr2=0x7f4000100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:21.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.521+0000 7f4008779700 1 -- 172.21.15.96:0/694843571 shutdown_connections 2024-04-25T14:13:21.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.521+0000 7f4008779700 1 -- 172.21.15.96:0/694843571 wait complete. 2024-04-25T14:13:21.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.522+0000 7f4008779700 1 Processor -- start 2024-04-25T14:13:21.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.522+0000 7f4008779700 1 -- start start 2024-04-25T14:13:21.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.522+0000 7f4008779700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4000103630 0x7f400019a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:21.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.522+0000 7f4008779700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4000104010 0x7f400019ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:21.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.522+0000 7f4008779700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4000104b70 0x7f40001a1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:21.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.522+0000 7f4008779700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f400010ec10 con 0x7f4000104b70 2024-04-25T14:13:21.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.522+0000 7f4008779700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f400010ea90 con 0x7f4000103630 2024-04-25T14:13:21.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.522+0000 7f4008779700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f400010ed90 con 0x7f4000104010 2024-04-25T14:13:21.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.522+0000 7f4006d16700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4000104b70 0x7f40001a1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:21.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.522+0000 7f4006d16700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4000104b70 0x7f40001a1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51386/0 (socket says 172.21.15.96:51386) 2024-04-25T14:13:21.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.522+0000 7f4006d16700 1 -- 172.21.15.96:0/2209640217 learned_addr learned my addr 172.21.15.96:0/2209640217 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:21.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.522+0000 7f4006515700 1 --2- 172.21.15.96:0/2209640217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4000103630 0x7f400019a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:21.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.523+0000 7f4005d14700 1 --2- 172.21.15.96:0/2209640217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4000104010 0x7f400019ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:21.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.523+0000 7f4006d16700 1 -- 172.21.15.96:0/2209640217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4000104010 msgr2=0x7f400019ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:21.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.523+0000 7f4006d16700 1 --2- 172.21.15.96:0/2209640217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4000104010 0x7f400019ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:21.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.523+0000 7f4006d16700 1 -- 172.21.15.96:0/2209640217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4000103630 msgr2=0x7f400019a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:21.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.523+0000 7f4006d16700 1 --2- 172.21.15.96:0/2209640217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4000103630 0x7f400019a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:21.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.523+0000 7f4006d16700 1 -- 172.21.15.96:0/2209640217 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40001a2460 con 0x7f4000104b70 2024-04-25T14:13:21.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.523+0000 7f4006d16700 1 --2- 172.21.15.96:0/2209640217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4000104b70 0x7f40001a1de0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3ff800ada0 tx=0x7f3ff800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:21.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.523+0000 7f3ff77fe700 1 -- 172.21.15.96:0/2209640217 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff8017020 con 0x7f4000104b70 2024-04-25T14:13:21.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.523+0000 7f3ff77fe700 1 -- 172.21.15.96:0/2209640217 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ff800cd80 con 0x7f4000104b70 2024-04-25T14:13:21.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.523+0000 7f4008779700 1 -- 172.21.15.96:0/2209640217 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40001a2750 con 0x7f4000104b70 2024-04-25T14:13:21.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.523+0000 7f3ff77fe700 1 -- 172.21.15.96:0/2209640217 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff8005260 con 0x7f4000104b70 2024-04-25T14:13:21.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.523+0000 7f4008779700 1 -- 172.21.15.96:0/2209640217 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40001a2ce0 con 0x7f4000104b70 2024-04-25T14:13:21.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.524+0000 7f4008779700 1 -- 172.21.15.96:0/2209640217 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4000051190 con 0x7f4000104b70 2024-04-25T14:13:21.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.526+0000 7f3ff77fe700 1 -- 172.21.15.96:0/2209640217 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f3ff800a090 con 0x7f4000104b70 2024-04-25T14:13:21.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.526+0000 7f3ff77fe700 1 --2- 172.21.15.96:0/2209640217 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f3fec0afc00 0x7f3fec0b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:21.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.526+0000 7f4006515700 1 --2- 172.21.15.96:0/2209640217 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f3fec0afc00 0x7f3fec0b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:21.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.526+0000 7f3ff77fe700 1 -- 172.21.15.96:0/2209640217 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f3ff80de890 con 0x7f4000104b70 2024-04-25T14:13:21.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.527+0000 7f4006515700 1 --2- 172.21.15.96:0/2209640217 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f3fec0afc00 0x7f3fec0b2110 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f400006b450 tx=0x7f3ffc009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:21.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.530+0000 7f3ff77fe700 1 -- 172.21.15.96:0/2209640217 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3ff8021020 con 0x7f4000104b70 2024-04-25T14:13:21.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.738+0000 7f4008779700 1 -- 172.21.15.96:0/2209640217 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f40001a3100 con 0x7f4000104b70 2024-04-25T14:13:21.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.739+0000 7f3ff77fe700 1 -- 172.21.15.96:0/2209640217 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v78) ==== 56+0+5309 (secure 0 0 0) 0x7f3ff80a74e0 con 0x7f4000104b70 2024-04-25T14:13:21.739 INFO:teuthology.orchestra.run.smithi096.stdout:epoch 78 2024-04-25T14:13:21.739 INFO:teuthology.orchestra.run.smithi096.stdout:fsid 59334182-030c-11ef-bc93-c7b262605968 2024-04-25T14:13:21.739 INFO:teuthology.orchestra.run.smithi096.stdout:created 2024-04-25T14:03:25.680745+0000 2024-04-25T14:13:21.739 INFO:teuthology.orchestra.run.smithi096.stdout:modified 2024-04-25T14:12:22.496143+0000 2024-04-25T14:13:21.739 INFO:teuthology.orchestra.run.smithi096.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-25T14:13:21.739 INFO:teuthology.orchestra.run.smithi096.stdout:crush_version 26 2024-04-25T14:13:21.739 INFO:teuthology.orchestra.run.smithi096.stdout:full_ratio 0.95 2024-04-25T14:13:21.740 INFO:teuthology.orchestra.run.smithi096.stdout:backfillfull_ratio 0.9 2024-04-25T14:13:21.740 INFO:teuthology.orchestra.run.smithi096.stdout:nearfull_ratio 0.85 2024-04-25T14:13:21.740 INFO:teuthology.orchestra.run.smithi096.stdout:require_min_compat_client luminous 2024-04-25T14:13:21.740 INFO:teuthology.orchestra.run.smithi096.stdout:min_compat_client luminous 2024-04-25T14:13:21.740 INFO:teuthology.orchestra.run.smithi096.stdout:require_osd_release squid 2024-04-25T14:13:21.740 INFO:teuthology.orchestra.run.smithi096.stdout:stretch_mode_enabled false 2024-04-25T14:13:21.740 INFO:teuthology.orchestra.run.smithi096.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-25T14:13:21.740 INFO:teuthology.orchestra.run.smithi096.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 75 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-04-25T14:13:21.740 INFO:teuthology.orchestra.run.smithi096.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 76 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-25T14:13:21.740 INFO:teuthology.orchestra.run.smithi096.stdout:max_osd 12 2024-04-25T14:13:21.740 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0 up in weight 1 up_from 8 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6802/3832506168,v1:172.21.15.96:6803/3832506168] [v2:172.21.15.96:6804/3832506168,v1:172.21.15.96:6805/3832506168] exists,up cf4ba3ac-142e-4e8e-96d4-410d77157eb0 2024-04-25T14:13:21.740 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1 up in weight 1 up_from 13 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6800/1083311494,v1:172.21.15.97:6801/1083311494] [v2:172.21.15.97:6802/1083311494,v1:172.21.15.97:6803/1083311494] exists,up 866e73b9-4a26-48ba-badf-704fe22f71cc 2024-04-25T14:13:21.740 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2 up in weight 1 up_from 18 up_thru 74 down_at 0 last_clean_interval [0,0) [v2:172.21.15.151:6800/981902966,v1:172.21.15.151:6801/981902966] [v2:172.21.15.151:6802/981902966,v1:172.21.15.151:6803/981902966] exists,up 45c79ecb-abc4-4dba-bae8-74804e4bc1bd 2024-04-25T14:13:21.741 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3 up in weight 1 up_from 26 up_thru 74 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6810/2274158748,v1:172.21.15.96:6811/2274158748] [v2:172.21.15.96:6812/2274158748,v1:172.21.15.96:6813/2274158748] exists,up 9fc39468-a7ea-4ccf-b365-335ad2ea4c7a 2024-04-25T14:13:21.741 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4 up in weight 1 up_from 32 up_thru 74 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6808/3727031962,v1:172.21.15.97:6809/3727031962] [v2:172.21.15.97:6810/3727031962,v1:172.21.15.97:6811/3727031962] exists,up 616582f1-12d2-456b-b7b3-e5517800e8a3 2024-04-25T14:13:21.741 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5 up in weight 1 up_from 37 up_thru 74 down_at 0 last_clean_interval [0,0) [v2:172.21.15.151:6808/223882546,v1:172.21.15.151:6809/223882546] [v2:172.21.15.151:6810/223882546,v1:172.21.15.151:6811/223882546] exists,up 3d277784-ab0f-4c90-81a1-2a4034c52a27 2024-04-25T14:13:21.741 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6 up in weight 1 up_from 43 up_thru 74 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6818/96197407,v1:172.21.15.96:6819/96197407] [v2:172.21.15.96:6820/96197407,v1:172.21.15.96:6821/96197407] exists,up 9a9ad585-cb1a-4239-9d18-055f59164049 2024-04-25T14:13:21.741 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7 up in weight 1 up_from 48 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6816/2247216400,v1:172.21.15.97:6817/2247216400] [v2:172.21.15.97:6818/2247216400,v1:172.21.15.97:6819/2247216400] exists,up 303551c8-6af0-42f2-844f-45bf8db5baa9 2024-04-25T14:13:21.741 INFO:teuthology.orchestra.run.smithi096.stdout:osd.8 up in weight 1 up_from 54 up_thru 74 down_at 0 last_clean_interval [0,0) [v2:172.21.15.151:6816/2549822185,v1:172.21.15.151:6817/2549822185] [v2:172.21.15.151:6818/2549822185,v1:172.21.15.151:6819/2549822185] exists,up 81f6cc66-e4ca-430c-b336-b6678d968818 2024-04-25T14:13:21.741 INFO:teuthology.orchestra.run.smithi096.stdout:osd.9 up in weight 1 up_from 60 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6826/1099399643,v1:172.21.15.96:6827/1099399643] [v2:172.21.15.96:6828/1099399643,v1:172.21.15.96:6829/1099399643] exists,up 29d4da16-feab-4d1c-9d1d-de4f799bd724 2024-04-25T14:13:21.741 INFO:teuthology.orchestra.run.smithi096.stdout:osd.10 up in weight 1 up_from 65 up_thru 74 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6824/155754520,v1:172.21.15.97:6825/155754520] [v2:172.21.15.97:6826/155754520,v1:172.21.15.97:6827/155754520] exists,up 197fee9b-eee4-4893-be1c-3d57f42250e9 2024-04-25T14:13:21.741 INFO:teuthology.orchestra.run.smithi096.stdout:osd.11 up in weight 1 up_from 71 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.151:6824/2027958746,v1:172.21.15.151:6825/2027958746] [v2:172.21.15.151:6826/2027958746,v1:172.21.15.151:6827/2027958746] exists,up 9c1d0a83-fc12-41a2-8eb0-80c4abdd2770 2024-04-25T14:13:21.741 INFO:teuthology.orchestra.run.smithi096.stdout:pg_upmap_items 2.0 [7,11] 2024-04-25T14:13:21.741 INFO:teuthology.orchestra.run.smithi096.stdout:pg_upmap_items 2.13 [2,11] 2024-04-25T14:13:21.741 INFO:teuthology.orchestra.run.smithi096.stdout:pg_upmap_items 2.16 [2,4] 2024-04-25T14:13:21.741 INFO:teuthology.orchestra.run.smithi096.stdout:pg_upmap_items 2.1f [8,4] 2024-04-25T14:13:21.742 INFO:teuthology.orchestra.run.smithi096.stdout:pg_upmap_items 3.2 [3,9] 2024-04-25T14:13:21.742 INFO:teuthology.orchestra.run.smithi096.stdout:pg_upmap_items 3.6 [4,6] 2024-04-25T14:13:21.742 INFO:teuthology.orchestra.run.smithi096.stdout:pg_upmap_items 3.b [3,9] 2024-04-25T14:13:21.742 INFO:teuthology.orchestra.run.smithi096.stdout:pg_upmap_items 3.13 [4,9] 2024-04-25T14:13:21.742 INFO:teuthology.orchestra.run.smithi096.stdout:pg_upmap_items 3.14 [4,7] 2024-04-25T14:13:21.742 INFO:teuthology.orchestra.run.smithi096.stdout:pg_upmap_items 3.37 [5,8] 2024-04-25T14:13:21.742 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:0/3839807484 expires 2024-04-26T14:04:21.307574+0000 2024-04-25T14:13:21.742 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:0/1495237551 expires 2024-04-26T14:04:21.307574+0000 2024-04-25T14:13:21.742 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:0/2793751450 expires 2024-04-26T14:03:54.249537+0000 2024-04-25T14:13:21.742 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:0/129575834 expires 2024-04-26T14:03:54.249537+0000 2024-04-25T14:13:21.742 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:0/2233075139 expires 2024-04-26T14:04:21.307574+0000 2024-04-25T14:13:21.742 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:6801/2348397481 expires 2024-04-26T14:03:54.249537+0000 2024-04-25T14:13:21.742 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:6801/433591062 expires 2024-04-26T14:04:21.307574+0000 2024-04-25T14:13:21.743 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:6800/2348397481 expires 2024-04-26T14:03:54.249537+0000 2024-04-25T14:13:21.743 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:0/3517674924 expires 2024-04-26T14:03:54.249537+0000 2024-04-25T14:13:21.743 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:6800/433591062 expires 2024-04-26T14:04:21.307574+0000 2024-04-25T14:13:21.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.741+0000 7f4008779700 1 -- 172.21.15.96:0/2209640217 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f3fec0afc00 msgr2=0x7f3fec0b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:21.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.741+0000 7f4008779700 1 --2- 172.21.15.96:0/2209640217 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f3fec0afc00 0x7f3fec0b2110 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f400006b450 tx=0x7f3ffc009190 comp rx=0 tx=0).stop 2024-04-25T14:13:21.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.741+0000 7f4008779700 1 -- 172.21.15.96:0/2209640217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4000104b70 msgr2=0x7f40001a1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:21.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.741+0000 7f4008779700 1 --2- 172.21.15.96:0/2209640217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4000104b70 0x7f40001a1de0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3ff800ada0 tx=0x7f3ff800c330 comp rx=0 tx=0).stop 2024-04-25T14:13:21.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.741+0000 7f4008779700 1 -- 172.21.15.96:0/2209640217 shutdown_connections 2024-04-25T14:13:21.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.741+0000 7f4008779700 1 --2- 172.21.15.96:0/2209640217 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f3fec0afc00 0x7f3fec0b2110 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:21.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.741+0000 7f4008779700 1 --2- 172.21.15.96:0/2209640217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4000104b70 0x7f40001a1de0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:21.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.741+0000 7f4008779700 1 --2- 172.21.15.96:0/2209640217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4000103630 0x7f400019a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:21.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.741+0000 7f4008779700 1 --2- 172.21.15.96:0/2209640217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4000104010 0x7f400019ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:21.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.741+0000 7f4008779700 1 -- 172.21.15.96:0/2209640217 >> 172.21.15.96:0/2209640217 conn(0x7f40000fde80 msgr2=0x7f40000ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:21.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.742+0000 7f4008779700 1 -- 172.21.15.96:0/2209640217 shutdown_connections 2024-04-25T14:13:21.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:21.742+0000 7f4008779700 1 -- 172.21.15.96:0/2209640217 wait complete. 2024-04-25T14:13:22.591 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-25T14:13:22.603 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:13:22.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.721+0000 7f3ac1140640 1 -- 172.21.15.96:0/3696429828 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3abc103b10 msgr2=0x7f3abc104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:22.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.721+0000 7f3ac1140640 1 --2- 172.21.15.96:0/3696429828 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3abc103b10 0x7f3abc104810 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0009900 tx=0x7f3ab002a410 comp rx=0 tx=0).stop 2024-04-25T14:13:22.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.721+0000 7f3ac1140640 1 -- 172.21.15.96:0/3696429828 shutdown_connections 2024-04-25T14:13:22.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.721+0000 7f3ac1140640 1 --2- 172.21.15.96:0/3696429828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3abc109a50 0x7f3abc10be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:22.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.721+0000 7f3ac1140640 1 --2- 172.21.15.96:0/3696429828 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3abc104d50 0x7f3abc105160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:22.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.721+0000 7f3ac1140640 1 --2- 172.21.15.96:0/3696429828 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3abc103b10 0x7f3abc104810 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:22.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.721+0000 7f3ac1140640 1 -- 172.21.15.96:0/3696429828 >> 172.21.15.96:0/3696429828 conn(0x7f3abc0fed40 msgr2=0x7f3abc1011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:22.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.721+0000 7f3ac1140640 1 -- 172.21.15.96:0/3696429828 shutdown_connections 2024-04-25T14:13:22.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.721+0000 7f3ac1140640 1 -- 172.21.15.96:0/3696429828 wait complete. 2024-04-25T14:13:22.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.722+0000 7f3ac1140640 1 Processor -- start 2024-04-25T14:13:22.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.722+0000 7f3ac1140640 1 -- start start 2024-04-25T14:13:22.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.722+0000 7f3ac1140640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3abc103b10 0x7f3abc19a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:22.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.722+0000 7f3ac1140640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3abc104d50 0x7f3abc19a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:22.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.722+0000 7f3ac1140640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3abc109a50 0x7f3abc1a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:22.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.722+0000 7f3ac1140640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3abc10f610 con 0x7f3abc104d50 2024-04-25T14:13:22.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.722+0000 7f3ac1140640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3abc10f490 con 0x7f3abc103b10 2024-04-25T14:13:22.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.722+0000 7f3ac1140640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3abc10f790 con 0x7f3abc109a50 2024-04-25T14:13:22.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.722+0000 7f3aba59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3abc104d50 0x7f3abc19a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:22.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.722+0000 7f3aba59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3abc104d50 0x7f3abc19a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51406/0 (socket says 172.21.15.96:51406) 2024-04-25T14:13:22.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.722+0000 7f3aba59c640 1 -- 172.21.15.96:0/224515528 learned_addr learned my addr 172.21.15.96:0/224515528 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:22.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.722+0000 7f3abad9d640 1 --2- 172.21.15.96:0/224515528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3abc103b10 0x7f3abc19a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:22.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.722+0000 7f3abb59e640 1 --2- 172.21.15.96:0/224515528 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3abc109a50 0x7f3abc1a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:22.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.723+0000 7f3aba59c640 1 -- 172.21.15.96:0/224515528 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3abc109a50 msgr2=0x7f3abc1a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:22.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.723+0000 7f3aba59c640 1 --2- 172.21.15.96:0/224515528 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3abc109a50 0x7f3abc1a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:22.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.723+0000 7f3aba59c640 1 -- 172.21.15.96:0/224515528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3abc103b10 msgr2=0x7f3abc19a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:22.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.723+0000 7f3aba59c640 1 --2- 172.21.15.96:0/224515528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3abc103b10 0x7f3abc19a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:22.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.723+0000 7f3aba59c640 1 -- 172.21.15.96:0/224515528 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3abc1a1dc0 con 0x7f3abc104d50 2024-04-25T14:13:22.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.723+0000 7f3aba59c640 1 --2- 172.21.15.96:0/224515528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3abc104d50 0x7f3abc19a7c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3aac00d770 tx=0x7f3aac00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:22.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.723+0000 7f3aa3fff640 1 -- 172.21.15.96:0/224515528 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac017070 con 0x7f3abc104d50 2024-04-25T14:13:22.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.723+0000 7f3ac1140640 1 -- 172.21.15.96:0/224515528 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3abc1a20b0 con 0x7f3abc104d50 2024-04-25T14:13:22.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.723+0000 7f3aa3fff640 1 -- 172.21.15.96:0/224515528 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3aac0054d0 con 0x7f3abc104d50 2024-04-25T14:13:22.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.723+0000 7f3aa3fff640 1 -- 172.21.15.96:0/224515528 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac004c30 con 0x7f3abc104d50 2024-04-25T14:13:22.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.723+0000 7f3ac1140640 1 -- 172.21.15.96:0/224515528 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3abc1a25b0 con 0x7f3abc104d50 2024-04-25T14:13:22.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.724+0000 7f3ac1140640 1 -- 172.21.15.96:0/224515528 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3abc108860 con 0x7f3abc104d50 2024-04-25T14:13:22.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.725+0000 7f3aa3fff640 1 -- 172.21.15.96:0/224515528 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f3aac004e10 con 0x7f3abc104d50 2024-04-25T14:13:22.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.726+0000 7f3aa3fff640 1 --2- 172.21.15.96:0/224515528 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f3aa40afaf0 0x7f3aa40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:22.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.726+0000 7f3abad9d640 1 --2- 172.21.15.96:0/224515528 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f3aa40afaf0 0x7f3aa40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:22.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.726+0000 7f3aa3fff640 1 -- 172.21.15.96:0/224515528 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f3aac0de650 con 0x7f3abc104d50 2024-04-25T14:13:22.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.726+0000 7f3abad9d640 1 --2- 172.21.15.96:0/224515528 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f3aa40afaf0 0x7f3aa40b2000 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0005ab0 tx=0x7f3ab0031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:22.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.731+0000 7f3aa3fff640 1 -- 172.21.15.96:0/224515528 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3aac0a6bc0 con 0x7f3abc104d50 2024-04-25T14:13:22.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:22 smithi096 ceph-mon[29207]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T14:13:22.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:22 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/2209640217' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T14:13:22.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.910+0000 7f3ac1140640 1 -- 172.21.15.96:0/224515528 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3abc0637a0 con 0x7f3abc104d50 2024-04-25T14:13:22.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.912+0000 7f3aa3fff640 1 -- 172.21.15.96:0/224515528 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22616 (secure 0 0 0) 0x7f3aac0aaec0 con 0x7f3abc104d50 2024-04-25T14:13:22.912 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:13:22.912 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":78,"fsid":"59334182-030c-11ef-bc93-c7b262605968","created":"2024-04-25T14:03:25.680745+0000","modified":"2024-04-25T14:12:22.496143+0000","last_up_change":"2024-04-25T14:10:35.894312+0000","last_in_change":"2024-04-25T14:10:15.583440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:06:47.402927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:11:27.434331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:11:27.854004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cf4ba3ac-142e-4e8e-96d4-410d77157eb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6803","nonce":3832506168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6805","nonce":3832506168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832506168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6807","nonce":3832506168}]},"public_addr":"172.21.15.96:6803/3832506168","cluster_addr":"172.21.15.96:6805/3832506168","heartbeat_back_addr":"172.21.15.96:6809/3832506168","heartbeat_front_addr":"172.21.15.96:6807/3832506168","state":["exists","up"]},{"osd":1,"uuid":"866e73b9-4a26-48ba-badf-704fe22f71cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6801","nonce":1083311494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6803","nonce":1083311494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6807","nonce":1083311494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6805","nonce":1083311494}]},"public_addr":"172.21.15.97:6801/1083311494","cluster_addr":"172.21.15.97:6803/1083311494","heartbeat_back_addr":"172.21.15.97:6807/1083311494","heartbeat_front_addr":"172.21.15.97:6805/1083311494","state":["exists","up"]},{"osd":2,"uuid":"45c79ecb-abc4-4dba-bae8-74804e4bc1bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6801","nonce":981902966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6803","nonce":981902966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6807","nonce":981902966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6805","nonce":981902966}]},"public_addr":"172.21.15.151:6801/981902966","cluster_addr":"172.21.15.151:6803/981902966","heartbeat_back_addr":"172.21.15.151:6807/981902966","heartbeat_front_addr":"172.21.15.151:6805/981902966","state":["exists","up"]},{"osd":3,"uuid":"9fc39468-a7ea-4ccf-b365-335ad2ea4c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6811","nonce":2274158748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6813","nonce":2274158748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6817","nonce":2274158748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6815","nonce":2274158748}]},"public_addr":"172.21.15.96:6811/2274158748","cluster_addr":"172.21.15.96:6813/2274158748","heartbeat_back_addr":"172.21.15.96:6817/2274158748","heartbeat_front_addr":"172.21.15.96:6815/2274158748","state":["exists","up"]},{"osd":4,"uuid":"616582f1-12d2-456b-b7b3-e5517800e8a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6809","nonce":3727031962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6811","nonce":3727031962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6815","nonce":3727031962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6813","nonce":3727031962}]},"public_addr":"172.21.15.97:6809/3727031962","cluster_addr":"172.21.15.97:6811/3727031962","heartbeat_back_addr":"172.21.15.97:6815/3727031962","heartbeat_front_addr":"172.21.15.97:6813/3727031962","state":["exists","up"]},{"osd":5,"uuid":"3d277784-ab0f-4c90-81a1-2a4034c52a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6809","nonce":223882546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6811","nonce":223882546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6815","nonce":223882546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6813","nonce":223882546}]},"public_addr":"172.21.15.151:6809/223882546","cluster_addr":"172.21.15.151:6811/223882546","heartbeat_back_addr":"172.21.15.151:6815/223882546","heartbeat_front_addr":"172.21.15.151:6813/223882546","state":["exists","up"]},{"osd":6,"uuid":"9a9ad585-cb1a-4239-9d18-055f59164049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6819","nonce":96197407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6821","nonce":96197407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6825","nonce":96197407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6823","nonce":96197407}]},"public_addr":"172.21.15.96:6819/96197407","cluster_addr":"172.21.15.96:6821/96197407","heartbeat_back_addr":"172.21.15.96:6825/96197407","heartbeat_front_addr":"172.21.15.96:6823/96197407","state":["exists","up"]},{"osd":7,"uuid":"303551c8-6af0-42f2-844f-45bf8db5baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6817","nonce":2247216400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6819","nonce":2247216400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6823","nonce":2247216400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6821","nonce":2247216400}]},"public_addr":"172.21.15.97:6817/2247216400","cluster_addr":"172.21.15.97:6819/2247216400","heartbeat_back_addr":"172.21.15.97:6823/2247216400","heartbeat_front_addr":"172.21.15.97:6821/2247216400","state":["exists","up"]},{"osd":8,"uuid":"81f6cc66-e4ca-430c-b336-b6678d968818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6817","nonce":2549822185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6819","nonce":2549822185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6823","nonce":2549822185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6821","nonce":2549822185}]},"public_addr":"172.21.15.151:6817/2549822185","cluster_addr":"172.21.15.151:6819/2549822185","heartbeat_back_addr":"172.21.15.151:6823/2549822185","heartbeat_front_addr":"172.21.15.151:6821/2549822185","state":["exists","up"]},{"osd":9,"uuid":"29d4da16-feab-4d1c-9d1d-de4f799bd724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6827","nonce":1099399643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6829","nonce":1099399643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6833","nonce":1099399643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6831","nonce":1099399643}]},"public_addr":"172.21.15.96:6827/1099399643","cluster_addr":"172.21.15.96:6829/1099399643","heartbeat_back_addr":"172.21.15.96:6833/1099399643","heartbeat_front_addr":"172.21.15.96:6831/1099399643","state":["exists","up"]},{"osd":10,"uuid":"197fee9b-eee4-4893-be1c-3d57f42250e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6825","nonce":155754520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6827","nonce":155754520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6831","nonce":155754520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6829","nonce":155754520}]},"public_addr":"172.21.15.97:6825/155754520","cluster_addr":"172.21.15.97:6827/155754520","heartbeat_back_addr":"172.21.15.97:6831/155754520","heartbeat_front_addr":"172.21.15.97:6829/155754520","state":["exists","up"]},{"osd":11,"uuid":"9c1d0a83-fc12-41a2-8eb0-80c4abdd2770","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6825","nonce":2027958746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6827","nonce":2027958746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6831","nonce":2027958746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6829","nonce":2027958746}]},"public_addr":"172.21.15.151:6825/2027958746","cluster_addr":"172.21.15.151:6827/2027958746","heartbeat_back_addr":"172.21.15.151:6831/2027958746","heartbeat_front_addr":"172.21.15.151:6829/2027958746","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:05:56.177390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:06:19.917663+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:06:43.608052+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:08.540382+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:33.474072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:58.242306+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:08:24.120442+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:08:49.661064+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:09:15.097271+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:09:41.904505+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:10:08.233252+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:10:33.712296+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/3839807484":"2024-04-26T14:04:21.307574+0000","172.21.15.96:0/1495237551":"2024-04-26T14:04:21.307574+0000","172.21.15.96:0/2793751450":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/129575834":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/2233075139":"2024-04-26T14:04:21.307574+0000","172.21.15.96:6801/2348397481":"2024-04-26T14:03:54.249537+0000","172.21.15.96:6801/433591062":"2024-04-26T14:04:21.307574+0000","172.21.15.96:6800/2348397481":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/3517674924":"2024-04-26T14:03:54.249537+0000","172.21.15.96:6800/433591062":"2024-04-26T14:04:21.307574+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:13:22.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.914+0000 7f3ac1140640 1 -- 172.21.15.96:0/224515528 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f3aa40afaf0 msgr2=0x7f3aa40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:22.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.914+0000 7f3ac1140640 1 --2- 172.21.15.96:0/224515528 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f3aa40afaf0 0x7f3aa40b2000 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0005ab0 tx=0x7f3ab0031400 comp rx=0 tx=0).stop 2024-04-25T14:13:22.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.914+0000 7f3ac1140640 1 -- 172.21.15.96:0/224515528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3abc104d50 msgr2=0x7f3abc19a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:22.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.914+0000 7f3ac1140640 1 --2- 172.21.15.96:0/224515528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3abc104d50 0x7f3abc19a7c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3aac00d770 tx=0x7f3aac00dc30 comp rx=0 tx=0).stop 2024-04-25T14:13:22.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.914+0000 7f3ac1140640 1 -- 172.21.15.96:0/224515528 shutdown_connections 2024-04-25T14:13:22.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.914+0000 7f3ac1140640 1 --2- 172.21.15.96:0/224515528 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3abc109a50 0x7f3abc1a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:22.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.914+0000 7f3ac1140640 1 --2- 172.21.15.96:0/224515528 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f3aa40afaf0 0x7f3aa40b2000 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:22.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.914+0000 7f3ac1140640 1 --2- 172.21.15.96:0/224515528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3abc104d50 0x7f3abc19a7c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:22.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.914+0000 7f3ac1140640 1 --2- 172.21.15.96:0/224515528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3abc103b10 0x7f3abc19a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:22.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.914+0000 7f3ac1140640 1 -- 172.21.15.96:0/224515528 >> 172.21.15.96:0/224515528 conn(0x7f3abc0fed40 msgr2=0x7f3abc105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:22.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.914+0000 7f3ac1140640 1 -- 172.21.15.96:0/224515528 shutdown_connections 2024-04-25T14:13:22.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:22.914+0000 7f3ac1140640 1 -- 172.21.15.96:0/224515528 wait complete. 2024-04-25T14:13:22.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:22 smithi097 ceph-mon[32720]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T14:13:22.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:22 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/2209640217' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T14:13:22.926 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:06:47.402927+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:11:27.434331+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:11:27.854004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:13:22.926 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:13:22.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:22 smithi151 ceph-mon[32990]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T14:13:22.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:22 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/2209640217' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T14:13:23.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.037+0000 7fc08c8f9640 1 -- 172.21.15.96:0/4181938350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc088108920 msgr2=0x7fc08810ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:23.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.037+0000 7fc08c8f9640 1 --2- 172.21.15.96:0/4181938350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc088108920 0x7fc08810ad40 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc080009ca0 tx=0x7fc08002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:13:23.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.037+0000 7fc08c8f9640 1 -- 172.21.15.96:0/4181938350 shutdown_connections 2024-04-25T14:13:23.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.037+0000 7fc08c8f9640 1 --2- 172.21.15.96:0/4181938350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc088108920 0x7fc08810ad40 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:23.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.037+0000 7fc08c8f9640 1 --2- 172.21.15.96:0/4181938350 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc088103c20 0x7fc088104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:23.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.037+0000 7fc08c8f9640 1 --2- 172.21.15.96:0/4181938350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0881029e0 0x7fc0881036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:23.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.037+0000 7fc08c8f9640 1 -- 172.21.15.96:0/4181938350 >> 172.21.15.96:0/4181938350 conn(0x7fc0880fdbd0 msgr2=0x7fc088100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:23.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.037+0000 7fc08c8f9640 1 -- 172.21.15.96:0/4181938350 shutdown_connections 2024-04-25T14:13:23.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.037+0000 7fc08c8f9640 1 -- 172.21.15.96:0/4181938350 wait complete. 2024-04-25T14:13:23.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.038+0000 7fc08c8f9640 1 Processor -- start 2024-04-25T14:13:23.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.038+0000 7fc08c8f9640 1 -- start start 2024-04-25T14:13:23.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.038+0000 7fc08c8f9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0881029e0 0x7fc08819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:23.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.038+0000 7fc08c8f9640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc088103c20 0x7fc08819aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:23.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.038+0000 7fc08c8f9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc088108920 0x7fc0881a19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:23.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.038+0000 7fc08c8f9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc08810e510 con 0x7fc0881029e0 2024-04-25T14:13:23.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.038+0000 7fc08c8f9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc08810e390 con 0x7fc088108920 2024-04-25T14:13:23.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.038+0000 7fc08c8f9640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc08810e690 con 0x7fc088103c20 2024-04-25T14:13:23.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.038+0000 7fc08659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0881029e0 0x7fc08819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:23.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.038+0000 7fc08659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0881029e0 0x7fc08819a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51424/0 (socket says 172.21.15.96:51424) 2024-04-25T14:13:23.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.038+0000 7fc08659c640 1 -- 172.21.15.96:0/2709760329 learned_addr learned my addr 172.21.15.96:0/2709760329 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:23.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.039+0000 7fc07ffff640 1 --2- 172.21.15.96:0/2709760329 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc088103c20 0x7fc08819aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:23.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.039+0000 7fc086d9d640 1 --2- 172.21.15.96:0/2709760329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc088108920 0x7fc0881a19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:23.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.039+0000 7fc08659c640 1 -- 172.21.15.96:0/2709760329 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc088103c20 msgr2=0x7fc08819aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:23.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.039+0000 7fc08659c640 1 --2- 172.21.15.96:0/2709760329 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc088103c20 0x7fc08819aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:23.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.039+0000 7fc08659c640 1 -- 172.21.15.96:0/2709760329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc088108920 msgr2=0x7fc0881a19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:23.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.039+0000 7fc08659c640 1 --2- 172.21.15.96:0/2709760329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc088108920 0x7fc0881a19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:23.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.039+0000 7fc08659c640 1 -- 172.21.15.96:0/2709760329 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0881a2030 con 0x7fc0881029e0 2024-04-25T14:13:23.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.039+0000 7fc086d9d640 1 --2- 172.21.15.96:0/2709760329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc088108920 0x7fc0881a19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:23.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.039+0000 7fc08659c640 1 --2- 172.21.15.96:0/2709760329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0881029e0 0x7fc08819a4f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc07800c970 tx=0x7fc07800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:23.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.039+0000 7fc07f7fe640 1 -- 172.21.15.96:0/2709760329 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc078004220 con 0x7fc0881029e0 2024-04-25T14:13:23.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.039+0000 7fc07f7fe640 1 -- 172.21.15.96:0/2709760329 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0780043c0 con 0x7fc0881029e0 2024-04-25T14:13:23.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.039+0000 7fc08c8f9640 1 -- 172.21.15.96:0/2709760329 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0881a2320 con 0x7fc0881029e0 2024-04-25T14:13:23.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.039+0000 7fc07f7fe640 1 -- 172.21.15.96:0/2709760329 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07800f450 con 0x7fc0881029e0 2024-04-25T14:13:23.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.039+0000 7fc08c8f9640 1 -- 172.21.15.96:0/2709760329 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0881a2870 con 0x7fc0881029e0 2024-04-25T14:13:23.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.040+0000 7fc08c8f9640 1 -- 172.21.15.96:0/2709760329 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc08806c2a0 con 0x7fc0881029e0 2024-04-25T14:13:23.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.042+0000 7fc07f7fe640 1 -- 172.21.15.96:0/2709760329 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fc0780079b0 con 0x7fc0881029e0 2024-04-25T14:13:23.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.043+0000 7fc07f7fe640 1 --2- 172.21.15.96:0/2709760329 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fc0740afaf0 0x7fc0740b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:23.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.043+0000 7fc07ffff640 1 --2- 172.21.15.96:0/2709760329 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fc0740afaf0 0x7fc0740b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:23.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.043+0000 7fc07f7fe640 1 -- 172.21.15.96:0/2709760329 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fc0780dedb0 con 0x7fc0881029e0 2024-04-25T14:13:23.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.043+0000 7fc07ffff640 1 --2- 172.21.15.96:0/2709760329 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fc0740afaf0 0x7fc0740b2000 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc08819b590 tx=0x7fc07000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:23.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.047+0000 7fc07f7fe640 1 -- 172.21.15.96:0/2709760329 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc078014030 con 0x7fc0881029e0 2024-04-25T14:13:23.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.225+0000 7fc08c8f9640 1 -- 172.21.15.96:0/2709760329 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc0881a2b30 con 0x7fc0881029e0 2024-04-25T14:13:23.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.226+0000 7fc07f7fe640 1 -- 172.21.15.96:0/2709760329 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) ==== 93+0+10 (secure 0 0 0) 0x7fc0780a7970 con 0x7fc0881029e0 2024-04-25T14:13:23.226 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-04-25T14:13:23.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.227+0000 7fc08c8f9640 1 -- 172.21.15.96:0/2709760329 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fc0740afaf0 msgr2=0x7fc0740b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:23.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.227+0000 7fc08c8f9640 1 --2- 172.21.15.96:0/2709760329 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fc0740afaf0 0x7fc0740b2000 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc08819b590 tx=0x7fc07000b390 comp rx=0 tx=0).stop 2024-04-25T14:13:23.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.227+0000 7fc08c8f9640 1 -- 172.21.15.96:0/2709760329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0881029e0 msgr2=0x7fc08819a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:23.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.227+0000 7fc08c8f9640 1 --2- 172.21.15.96:0/2709760329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0881029e0 0x7fc08819a4f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc07800c970 tx=0x7fc07800ce30 comp rx=0 tx=0).stop 2024-04-25T14:13:23.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.228+0000 7fc08c8f9640 1 -- 172.21.15.96:0/2709760329 shutdown_connections 2024-04-25T14:13:23.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.228+0000 7fc08c8f9640 1 --2- 172.21.15.96:0/2709760329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc088108920 0x7fc0881a19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:23.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.228+0000 7fc08c8f9640 1 --2- 172.21.15.96:0/2709760329 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc088103c20 0x7fc08819aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:23.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.228+0000 7fc08c8f9640 1 --2- 172.21.15.96:0/2709760329 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fc0740afaf0 0x7fc0740b2000 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:23.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.228+0000 7fc08c8f9640 1 --2- 172.21.15.96:0/2709760329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0881029e0 0x7fc08819a4f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:23.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.228+0000 7fc08c8f9640 1 -- 172.21.15.96:0/2709760329 >> 172.21.15.96:0/2709760329 conn(0x7fc0880fdbd0 msgr2=0x7fc088069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:23.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.228+0000 7fc08c8f9640 1 -- 172.21.15.96:0/2709760329 shutdown_connections 2024-04-25T14:13:23.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.228+0000 7fc08c8f9640 1 -- 172.21.15.96:0/2709760329 wait complete. 2024-04-25T14:13:23.239 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:13:23.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.349+0000 7f90b8430640 1 -- 172.21.15.96:0/2844398498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90b0102f80 msgr2=0x7f90b0103390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:23.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.349+0000 7f90b8430640 1 --2- 172.21.15.96:0/2844398498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90b0102f80 0x7f90b0103390 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f90ac009900 tx=0x7f90ac02a410 comp rx=0 tx=0).stop 2024-04-25T14:13:23.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.349+0000 7f90b8430640 1 -- 172.21.15.96:0/2844398498 shutdown_connections 2024-04-25T14:13:23.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.349+0000 7f90b8430640 1 --2- 172.21.15.96:0/2844398498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f90b0104480 0x7f90b010abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:23.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.349+0000 7f90b8430640 1 --2- 172.21.15.96:0/2844398498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90b0103960 0x7f90b0103df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:23.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.349+0000 7f90b8430640 1 --2- 172.21.15.96:0/2844398498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90b0102f80 0x7f90b0103390 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:23.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.349+0000 7f90b8430640 1 -- 172.21.15.96:0/2844398498 >> 172.21.15.96:0/2844398498 conn(0x7f90b00fd380 msgr2=0x7f90b00ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:23.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.349+0000 7f90b8430640 1 -- 172.21.15.96:0/2844398498 shutdown_connections 2024-04-25T14:13:23.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.349+0000 7f90b8430640 1 -- 172.21.15.96:0/2844398498 wait complete. 2024-04-25T14:13:23.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.349+0000 7f90b8430640 1 Processor -- start 2024-04-25T14:13:23.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.349+0000 7f90b8430640 1 -- start start 2024-04-25T14:13:23.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.350+0000 7f90b8430640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90b0102f80 0x7f90b019a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:23.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.350+0000 7f90b8430640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90b0103960 0x7f90b019a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:23.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.350+0000 7f90b8430640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f90b0104480 0x7f90b01a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:23.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.350+0000 7f90b8430640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f90b010f650 con 0x7f90b0102f80 2024-04-25T14:13:23.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.350+0000 7f90b8430640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f90b010f4d0 con 0x7f90b0103960 2024-04-25T14:13:23.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.350+0000 7f90b8430640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f90b010f7d0 con 0x7f90b0104480 2024-04-25T14:13:23.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.350+0000 7f90b61cc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90b0102f80 0x7f90b019a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:23.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.350+0000 7f90b61cc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90b0102f80 0x7f90b019a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51444/0 (socket says 172.21.15.96:51444) 2024-04-25T14:13:23.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.350+0000 7f90b61cc640 1 -- 172.21.15.96:0/2538282732 learned_addr learned my addr 172.21.15.96:0/2538282732 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:23.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.350+0000 7f90b69cd640 1 --2- 172.21.15.96:0/2538282732 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f90b0104480 0x7f90b01a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:23.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.350+0000 7f90b59cb640 1 --2- 172.21.15.96:0/2538282732 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90b0103960 0x7f90b019a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:23.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.350+0000 7f90b61cc640 1 -- 172.21.15.96:0/2538282732 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f90b0104480 msgr2=0x7f90b01a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:23.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.350+0000 7f90b61cc640 1 --2- 172.21.15.96:0/2538282732 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f90b0104480 0x7f90b01a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:23.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.350+0000 7f90b61cc640 1 -- 172.21.15.96:0/2538282732 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90b0103960 msgr2=0x7f90b019a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:23.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.350+0000 7f90b61cc640 1 --2- 172.21.15.96:0/2538282732 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90b0103960 0x7f90b019a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:23.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.350+0000 7f90b61cc640 1 -- 172.21.15.96:0/2538282732 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b01a1d70 con 0x7f90b0102f80 2024-04-25T14:13:23.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.350+0000 7f90b61cc640 1 --2- 172.21.15.96:0/2538282732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90b0102f80 0x7f90b019a2a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f90ac032fd0 tx=0x7f90ac0042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:23.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.350+0000 7f90a77fe640 1 -- 172.21.15.96:0/2538282732 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac03f030 con 0x7f90b0102f80 2024-04-25T14:13:23.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.350+0000 7f90a77fe640 1 -- 172.21.15.96:0/2538282732 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90ac0044d0 con 0x7f90b0102f80 2024-04-25T14:13:23.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.350+0000 7f90b8430640 1 -- 172.21.15.96:0/2538282732 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90b01a2000 con 0x7f90b0102f80 2024-04-25T14:13:23.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.350+0000 7f90a77fe640 1 -- 172.21.15.96:0/2538282732 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac035070 con 0x7f90b0102f80 2024-04-25T14:13:23.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.351+0000 7f90b8430640 1 -- 172.21.15.96:0/2538282732 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90b01a24f0 con 0x7f90b0102f80 2024-04-25T14:13:23.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.351+0000 7f90b8430640 1 -- 172.21.15.96:0/2538282732 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90b0107920 con 0x7f90b0102f80 2024-04-25T14:13:23.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.353+0000 7f90a77fe640 1 -- 172.21.15.96:0/2538282732 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f90ac042030 con 0x7f90b0102f80 2024-04-25T14:13:23.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.354+0000 7f90a77fe640 1 --2- 172.21.15.96:0/2538282732 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f909c0afaf0 0x7f909c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:23.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.354+0000 7f90b59cb640 1 --2- 172.21.15.96:0/2538282732 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f909c0afaf0 0x7f909c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:23.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.354+0000 7f90a77fe640 1 -- 172.21.15.96:0/2538282732 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f90ac0ff960 con 0x7f90b0102f80 2024-04-25T14:13:23.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.354+0000 7f90b59cb640 1 --2- 172.21.15.96:0/2538282732 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f909c0afaf0 0x7f909c0b2000 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f90a00058c0 tx=0x7f90a0005c80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:23.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.357+0000 7f90a77fe640 1 -- 172.21.15.96:0/2538282732 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f90ac0c8520 con 0x7f90b0102f80 2024-04-25T14:13:23.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.536+0000 7f90b8430640 1 -- 172.21.15.96:0/2538282732 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f90b002dce0 con 0x7f90b0102f80 2024-04-25T14:13:23.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.537+0000 7f90a77fe640 1 -- 172.21.15.96:0/2538282732 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v78) ==== 104+0+11 (secure 0 0 0) 0x7f90b002dce0 con 0x7f90b0102f80 2024-04-25T14:13:23.537 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-04-25T14:13:23.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.539+0000 7f90b8430640 1 -- 172.21.15.96:0/2538282732 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f909c0afaf0 msgr2=0x7f909c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:23.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.539+0000 7f90b8430640 1 --2- 172.21.15.96:0/2538282732 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f909c0afaf0 0x7f909c0b2000 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f90a00058c0 tx=0x7f90a0005c80 comp rx=0 tx=0).stop 2024-04-25T14:13:23.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.539+0000 7f90b8430640 1 -- 172.21.15.96:0/2538282732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90b0102f80 msgr2=0x7f90b019a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:23.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.539+0000 7f90b8430640 1 --2- 172.21.15.96:0/2538282732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90b0102f80 0x7f90b019a2a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f90ac032fd0 tx=0x7f90ac0042e0 comp rx=0 tx=0).stop 2024-04-25T14:13:23.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.539+0000 7f90b8430640 1 -- 172.21.15.96:0/2538282732 shutdown_connections 2024-04-25T14:13:23.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.539+0000 7f90b8430640 1 --2- 172.21.15.96:0/2538282732 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f90b0104480 0x7f90b01a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:23.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.539+0000 7f90b8430640 1 --2- 172.21.15.96:0/2538282732 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90b0103960 0x7f90b019a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:23.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.539+0000 7f90b8430640 1 --2- 172.21.15.96:0/2538282732 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f909c0afaf0 0x7f909c0b2000 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:23.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.539+0000 7f90b8430640 1 --2- 172.21.15.96:0/2538282732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90b0102f80 0x7f90b019a2a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:23.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.539+0000 7f90b8430640 1 -- 172.21.15.96:0/2538282732 >> 172.21.15.96:0/2538282732 conn(0x7f90b00fd380 msgr2=0x7f90b0104af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:23.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.540+0000 7f90b8430640 1 -- 172.21.15.96:0/2538282732 shutdown_connections 2024-04-25T14:13:23.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.540+0000 7f90b8430640 1 -- 172.21.15.96:0/2538282732 wait complete. 2024-04-25T14:13:23.551 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:13:23.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.665+0000 7f82a0a05640 1 -- 172.21.15.96:0/226889948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f829c108a20 msgr2=0x7f829c10ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:23.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.665+0000 7f82a0a05640 1 --2- 172.21.15.96:0/226889948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f829c108a20 0x7f829c10ae40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8294009ca0 tx=0x7f829402a8c0 comp rx=0 tx=0).stop 2024-04-25T14:13:23.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.665+0000 7f82a0a05640 1 -- 172.21.15.96:0/226889948 shutdown_connections 2024-04-25T14:13:23.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.665+0000 7f82a0a05640 1 --2- 172.21.15.96:0/226889948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f829c108a20 0x7f829c10ae40 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:23.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.665+0000 7f82a0a05640 1 --2- 172.21.15.96:0/226889948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f829c1043d0 0x7f829c102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:23.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.665+0000 7f82a0a05640 1 --2- 172.21.15.96:0/226889948 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f829c103ef0 0x7f829c1027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:23.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.665+0000 7f82a0a05640 1 -- 172.21.15.96:0/226889948 >> 172.21.15.96:0/226889948 conn(0x7f829c0fd350 msgr2=0x7f829c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:23.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.665+0000 7f82a0a05640 1 -- 172.21.15.96:0/226889948 shutdown_connections 2024-04-25T14:13:23.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.665+0000 7f82a0a05640 1 -- 172.21.15.96:0/226889948 wait complete. 2024-04-25T14:13:23.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.666+0000 7f82a0a05640 1 Processor -- start 2024-04-25T14:13:23.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.666+0000 7f82a0a05640 1 -- start start 2024-04-25T14:13:23.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.666+0000 7f82a0a05640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f829c103ef0 0x7f829c199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:23.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.666+0000 7f82a0a05640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f829c108a20 0x7f829c19a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:23.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.666+0000 7f82a0a05640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f829c19f2b0 0x7f829c1a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:23.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.666+0000 7f82a0a05640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f829c10f690 con 0x7f829c108a20 2024-04-25T14:13:23.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.666+0000 7f82a0a05640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f829c10f510 con 0x7f829c103ef0 2024-04-25T14:13:23.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.666+0000 7f82a0a05640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f829c10f810 con 0x7f829c19f2b0 2024-04-25T14:13:23.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.666+0000 7f8299d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f829c108a20 0x7f829c19a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:23.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.666+0000 7f8299d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f829c108a20 0x7f829c19a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51458/0 (socket says 172.21.15.96:51458) 2024-04-25T14:13:23.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.666+0000 7f8299d9b640 1 -- 172.21.15.96:0/1860959174 learned_addr learned my addr 172.21.15.96:0/1860959174 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:23.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.666+0000 7f829a59c640 1 --2- 172.21.15.96:0/1860959174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f829c103ef0 0x7f829c199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:23.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.666+0000 7f8299d9b640 1 -- 172.21.15.96:0/1860959174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f829c19f2b0 msgr2=0x7f829c1a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:23.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.666+0000 7f8299d9b640 1 --2- 172.21.15.96:0/1860959174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f829c19f2b0 0x7f829c1a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:23.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.666+0000 7f8299d9b640 1 -- 172.21.15.96:0/1860959174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f829c103ef0 msgr2=0x7f829c199df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:23.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.666+0000 7f8299d9b640 1 --2- 172.21.15.96:0/1860959174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f829c103ef0 0x7f829c199df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:23.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.666+0000 7f8299d9b640 1 -- 172.21.15.96:0/1860959174 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f829c1a1c90 con 0x7f829c108a20 2024-04-25T14:13:23.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.666+0000 7f8299d9b640 1 --2- 172.21.15.96:0/1860959174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f829c108a20 0x7f829c19a330 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f828c00c920 tx=0x7f828c00cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:23.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.667+0000 7f828b7fe640 1 -- 172.21.15.96:0/1860959174 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828c007bb0 con 0x7f829c108a20 2024-04-25T14:13:23.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.667+0000 7f828b7fe640 1 -- 172.21.15.96:0/1860959174 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f828c007d50 con 0x7f829c108a20 2024-04-25T14:13:23.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.667+0000 7f82a0a05640 1 -- 172.21.15.96:0/1860959174 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f829c1a1f80 con 0x7f829c108a20 2024-04-25T14:13:23.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.667+0000 7f828b7fe640 1 -- 172.21.15.96:0/1860959174 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828c018730 con 0x7f829c108a20 2024-04-25T14:13:23.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.667+0000 7f82a0a05640 1 -- 172.21.15.96:0/1860959174 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f829c1a2460 con 0x7f829c108a20 2024-04-25T14:13:23.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.668+0000 7f82a0a05640 1 -- 172.21.15.96:0/1860959174 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f829c107120 con 0x7f829c108a20 2024-04-25T14:13:23.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.670+0000 7f828b7fe640 1 -- 172.21.15.96:0/1860959174 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f828c00f450 con 0x7f829c108a20 2024-04-25T14:13:23.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.670+0000 7f828b7fe640 1 --2- 172.21.15.96:0/1860959174 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f82840afaf0 0x7f82840b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:23.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.671+0000 7f829a59c640 1 --2- 172.21.15.96:0/1860959174 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f82840afaf0 0x7f82840b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:23.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.671+0000 7f828b7fe640 1 -- 172.21.15.96:0/1860959174 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f828c0df020 con 0x7f829c108a20 2024-04-25T14:13:23.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.671+0000 7f829a59c640 1 --2- 172.21.15.96:0/1860959174 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f82840afaf0 0x7f82840b2000 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f829c06b3b0 tx=0x7f8290009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:23.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.674+0000 7f828b7fe640 1 -- 172.21.15.96:0/1860959174 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f828c014030 con 0x7f829c108a20 2024-04-25T14:13:23.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.853+0000 7f82a0a05640 1 -- 172.21.15.96:0/1860959174 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f829c063840 con 0x7f829c108a20 2024-04-25T14:13:23.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.853+0000 7f828b7fe640 1 -- 172.21.15.96:0/1860959174 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v78) ==== 100+0+11 (secure 0 0 0) 0x7f828c0a7c70 con 0x7f829c108a20 2024-04-25T14:13:23.853 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-04-25T14:13:23.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.855+0000 7f82a0a05640 1 -- 172.21.15.96:0/1860959174 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f82840afaf0 msgr2=0x7f82840b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:23.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.855+0000 7f82a0a05640 1 --2- 172.21.15.96:0/1860959174 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f82840afaf0 0x7f82840b2000 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f829c06b3b0 tx=0x7f8290009110 comp rx=0 tx=0).stop 2024-04-25T14:13:23.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.855+0000 7f82a0a05640 1 -- 172.21.15.96:0/1860959174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f829c108a20 msgr2=0x7f829c19a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:23.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.855+0000 7f82a0a05640 1 --2- 172.21.15.96:0/1860959174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f829c108a20 0x7f829c19a330 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f828c00c920 tx=0x7f828c00cd20 comp rx=0 tx=0).stop 2024-04-25T14:13:23.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.855+0000 7f82a0a05640 1 -- 172.21.15.96:0/1860959174 shutdown_connections 2024-04-25T14:13:23.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.855+0000 7f82a0a05640 1 --2- 172.21.15.96:0/1860959174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f829c19f2b0 0x7f829c1a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:23.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.855+0000 7f82a0a05640 1 --2- 172.21.15.96:0/1860959174 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f82840afaf0 0x7f82840b2000 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:23.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.855+0000 7f82a0a05640 1 --2- 172.21.15.96:0/1860959174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f829c108a20 0x7f829c19a330 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:23.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.855+0000 7f82a0a05640 1 --2- 172.21.15.96:0/1860959174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f829c103ef0 0x7f829c199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:23.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.855+0000 7f82a0a05640 1 -- 172.21.15.96:0/1860959174 >> 172.21.15.96:0/1860959174 conn(0x7f829c0fd350 msgr2=0x7f829c0ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:23.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.856+0000 7f82a0a05640 1 -- 172.21.15.96:0/1860959174 shutdown_connections 2024-04-25T14:13:23.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.856+0000 7f82a0a05640 1 -- 172.21.15.96:0/1860959174 wait complete. 2024-04-25T14:13:23.858 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:23 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/224515528' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:13:23.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:23 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/2709760329' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:13:23.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:23 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/2538282732' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:13:23.867 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:13:23.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:23 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/224515528' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:13:23.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:23 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/2709760329' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:13:23.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:23 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/2538282732' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:13:23.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:23 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/224515528' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:13:23.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:23 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/2709760329' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:13:23.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:23 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/2538282732' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:13:23.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.974+0000 7fadc337a640 1 -- 172.21.15.96:0/1615701749 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fadbc108900 msgr2=0x7fadbc10ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:23.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.974+0000 7fadc337a640 1 --2- 172.21.15.96:0/1615701749 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fadbc108900 0x7fadbc10ad20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fadb4009900 tx=0x7fadb402a410 comp rx=0 tx=0).stop 2024-04-25T14:13:23.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.974+0000 7fadc337a640 1 -- 172.21.15.96:0/1615701749 shutdown_connections 2024-04-25T14:13:23.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.974+0000 7fadc337a640 1 --2- 172.21.15.96:0/1615701749 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fadbc108900 0x7fadbc10ad20 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:23.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.974+0000 7fadc337a640 1 --2- 172.21.15.96:0/1615701749 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fadbc103c00 0x7fadbc104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:23.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.974+0000 7fadc337a640 1 --2- 172.21.15.96:0/1615701749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadbc1029c0 0x7fadbc1036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:23.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.974+0000 7fadc337a640 1 -- 172.21.15.96:0/1615701749 >> 172.21.15.96:0/1615701749 conn(0x7fadbc0fdbd0 msgr2=0x7fadbc100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:23.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.974+0000 7fadc337a640 1 -- 172.21.15.96:0/1615701749 shutdown_connections 2024-04-25T14:13:23.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.974+0000 7fadc337a640 1 -- 172.21.15.96:0/1615701749 wait complete. 2024-04-25T14:13:23.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.975+0000 7fadc337a640 1 Processor -- start 2024-04-25T14:13:23.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.975+0000 7fadc337a640 1 -- start start 2024-04-25T14:13:23.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.975+0000 7fadc337a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fadbc1029c0 0x7fadbc0ffba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:23.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.975+0000 7fadc337a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fadbc103c00 0x7fadbc0fe240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:23.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.975+0000 7fadc337a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadbc108900 0x7fadbc0fe780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:23.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.975+0000 7fadc337a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc10e540 con 0x7fadbc1029c0 2024-04-25T14:13:23.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.975+0000 7fadc337a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc10e3c0 con 0x7fadbc108900 2024-04-25T14:13:23.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.975+0000 7fadc337a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc10e6c0 con 0x7fadbc103c00 2024-04-25T14:13:23.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.975+0000 7fadc1116640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fadbc1029c0 0x7fadbc0ffba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:23.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.975+0000 7fadc1116640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fadbc1029c0 0x7fadbc0ffba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51468/0 (socket says 172.21.15.96:51468) 2024-04-25T14:13:23.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.975+0000 7fadc1116640 1 -- 172.21.15.96:0/2293878645 learned_addr learned my addr 172.21.15.96:0/2293878645 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:23.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.975+0000 7fadc1917640 1 --2- 172.21.15.96:0/2293878645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadbc108900 0x7fadbc0fe780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:23.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.975+0000 7fadc0915640 1 --2- 172.21.15.96:0/2293878645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fadbc103c00 0x7fadbc0fe240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:23.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.975+0000 7fadc1116640 1 -- 172.21.15.96:0/2293878645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fadbc103c00 msgr2=0x7fadbc0fe240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:23.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.975+0000 7fadc1116640 1 --2- 172.21.15.96:0/2293878645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fadbc103c00 0x7fadbc0fe240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:23.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.975+0000 7fadc1116640 1 -- 172.21.15.96:0/2293878645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadbc108900 msgr2=0x7fadbc0fe780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:23.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.975+0000 7fadc1116640 1 --2- 172.21.15.96:0/2293878645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadbc108900 0x7fadbc0fe780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:23.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.975+0000 7fadc1116640 1 -- 172.21.15.96:0/2293878645 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadbc0fee20 con 0x7fadbc1029c0 2024-04-25T14:13:23.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.975+0000 7fadc0915640 1 --2- 172.21.15.96:0/2293878645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fadbc103c00 0x7fadbc0fe240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:23.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.975+0000 7fadc1917640 1 --2- 172.21.15.96:0/2293878645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadbc108900 0x7fadbc0fe780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:23.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.976+0000 7fadc1116640 1 --2- 172.21.15.96:0/2293878645 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fadbc1029c0 0x7fadbc0ffba0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fadac00cce0 tx=0x7fadac00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:23.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.976+0000 7fadba7fc640 1 -- 172.21.15.96:0/2293878645 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac00f450 con 0x7fadbc1029c0 2024-04-25T14:13:23.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.976+0000 7fadba7fc640 1 -- 172.21.15.96:0/2293878645 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadac00faa0 con 0x7fadbc1029c0 2024-04-25T14:13:23.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.976+0000 7fadc337a640 1 -- 172.21.15.96:0/2293878645 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadbc1a42c0 con 0x7fadbc1029c0 2024-04-25T14:13:23.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.976+0000 7fadba7fc640 1 -- 172.21.15.96:0/2293878645 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac018b70 con 0x7fadbc1029c0 2024-04-25T14:13:23.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.976+0000 7fadc337a640 1 -- 172.21.15.96:0/2293878645 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadbc1a4750 con 0x7fadbc1029c0 2024-04-25T14:13:23.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.977+0000 7fadc337a640 1 -- 172.21.15.96:0/2293878645 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadbc1a4a10 con 0x7fadbc1029c0 2024-04-25T14:13:23.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.978+0000 7fadba7fc640 1 -- 172.21.15.96:0/2293878645 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fadac018d10 con 0x7fadbc1029c0 2024-04-25T14:13:23.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.979+0000 7fadba7fc640 1 --2- 172.21.15.96:0/2293878645 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fada40afaf0 0x7fada40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:23.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.979+0000 7fadc0915640 1 --2- 172.21.15.96:0/2293878645 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fada40afaf0 0x7fada40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:23.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.979+0000 7fadba7fc640 1 -- 172.21.15.96:0/2293878645 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fadac0df220 con 0x7fadbc1029c0 2024-04-25T14:13:23.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.980+0000 7fadc0915640 1 --2- 172.21.15.96:0/2293878645 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fada40afaf0 0x7fada40b2000 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fadb0006fd0 tx=0x7fadb0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:23.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:23.983+0000 7fadba7fc640 1 -- 172.21.15.96:0/2293878645 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fadac052ff0 con 0x7fadbc1029c0 2024-04-25T14:13:24.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.198+0000 7fadc337a640 1 -- 172.21.15.96:0/2293878645 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fadbc1a4be0 con 0x7fadbc1029c0 2024-04-25T14:13:24.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.198+0000 7fadba7fc640 1 -- 172.21.15.96:0/2293878645 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11927 (secure 0 0 0) 0x7fadac014030 con 0x7fadbc1029c0 2024-04-25T14:13:24.198 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:13:24.199 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/606227917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":606227917},{"type":"v1","addr":"172.21.15.96:6835","nonce":606227917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/3265300710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3265300710},{"type":"v1","addr":"172.21.15.96:6837","nonce":3265300710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T14:11:28.886961+0000","modified":"2024-04-25T14:11:35.532775+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24517,"mds_2":24478,"mds_3":24455,"mds_4":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/4080308509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":4080308509},{"type":"v1","addr":"172.21.15.96:6839","nonce":4080308509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/1606748971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":1606748971},{"type":"v1","addr":"172.21.15.96:6841","nonce":1606748971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6833/1995324831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1995324831},{"type":"v1","addr":"172.21.15.97:6833","nonce":1995324831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":4,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.151:6833/1382721608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1382721608},{"type":"v1","addr":"172.21.15.151:6833","nonce":1382721608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6835/3872301726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3872301726},{"type":"v1","addr":"172.21.15.97:6835","nonce":3872301726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.151:6835/1313823638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":1313823638},{"type":"v1","addr":"172.21.15.151:6835","nonce":1313823638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/2691348366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2691348366},{"type":"v1","addr":"172.21.15.151:6837","nonce":2691348366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3306780081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3306780081},{"type":"v1","addr":"172.21.15.97:6837","nonce":3306780081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.151:6839/245699981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":245699981},{"type":"v1","addr":"172.21.15.151:6839","nonce":245699981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6839/2850442269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":2850442269},{"type":"v1","addr":"172.21.15.97:6839","nonce":2850442269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24517,24536,24478,24455,24427]},"id":1}]} 2024-04-25T14:13:24.199 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 25 2024-04-25T14:13:24.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.200+0000 7fadc337a640 1 -- 172.21.15.96:0/2293878645 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fada40afaf0 msgr2=0x7fada40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:24.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.200+0000 7fadc337a640 1 --2- 172.21.15.96:0/2293878645 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fada40afaf0 0x7fada40b2000 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fadb0006fd0 tx=0x7fadb0008040 comp rx=0 tx=0).stop 2024-04-25T14:13:24.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.200+0000 7fadc337a640 1 -- 172.21.15.96:0/2293878645 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fadbc1029c0 msgr2=0x7fadbc0ffba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:24.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.200+0000 7fadc337a640 1 --2- 172.21.15.96:0/2293878645 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fadbc1029c0 0x7fadbc0ffba0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fadac00cce0 tx=0x7fadac00a620 comp rx=0 tx=0).stop 2024-04-25T14:13:24.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.201+0000 7fadc337a640 1 -- 172.21.15.96:0/2293878645 shutdown_connections 2024-04-25T14:13:24.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.201+0000 7fadc337a640 1 --2- 172.21.15.96:0/2293878645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadbc108900 0x7fadbc0fe780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:24.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.201+0000 7fadc337a640 1 --2- 172.21.15.96:0/2293878645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fadbc103c00 0x7fadbc0fe240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:24.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.201+0000 7fadc337a640 1 --2- 172.21.15.96:0/2293878645 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fada40afaf0 0x7fada40b2000 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:24.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.201+0000 7fadc337a640 1 --2- 172.21.15.96:0/2293878645 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fadbc1029c0 0x7fadbc0ffba0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:24.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.201+0000 7fadc337a640 1 -- 172.21.15.96:0/2293878645 >> 172.21.15.96:0/2293878645 conn(0x7fadbc0fdbd0 msgr2=0x7fadbc069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:24.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.201+0000 7fadc337a640 1 -- 172.21.15.96:0/2293878645 shutdown_connections 2024-04-25T14:13:24.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.201+0000 7fadc337a640 1 -- 172.21.15.96:0/2293878645 wait complete. 2024-04-25T14:13:24.212 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:13:24.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.320+0000 7f077faaa640 1 -- 172.21.15.96:0/3131212466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07781045d0 msgr2=0x7f07781069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:24.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.320+0000 7f077faaa640 1 --2- 172.21.15.96:0/3131212466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07781045d0 0x7f07781069f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f0774009900 tx=0x7f077402a410 comp rx=0 tx=0).stop 2024-04-25T14:13:24.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.320+0000 7f077faaa640 1 -- 172.21.15.96:0/3131212466 shutdown_connections 2024-04-25T14:13:24.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.320+0000 7f077faaa640 1 --2- 172.21.15.96:0/3131212466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0778109890 0x7f077810bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:24.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.320+0000 7f077faaa640 1 --2- 172.21.15.96:0/3131212466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0778106f30 0x7f0778109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:24.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.320+0000 7f077faaa640 1 --2- 172.21.15.96:0/3131212466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07781045d0 0x7f07781069f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:24.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.320+0000 7f077faaa640 1 -- 172.21.15.96:0/3131212466 >> 172.21.15.96:0/3131212466 conn(0x7f07780fdbd0 msgr2=0x7f0778100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:24.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.320+0000 7f077faaa640 1 -- 172.21.15.96:0/3131212466 shutdown_connections 2024-04-25T14:13:24.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.320+0000 7f077faaa640 1 -- 172.21.15.96:0/3131212466 wait complete. 2024-04-25T14:13:24.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.321+0000 7f077faaa640 1 Processor -- start 2024-04-25T14:13:24.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.321+0000 7f077faaa640 1 -- start start 2024-04-25T14:13:24.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.321+0000 7f077faaa640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07781045d0 0x7f0778103a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:24.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.321+0000 7f077faaa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0778106f30 0x7f07781020c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:24.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.321+0000 7f077faaa640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0778109890 0x7f0778102600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:24.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.321+0000 7f077faaa640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f077810f5c0 con 0x7f07781045d0 2024-04-25T14:13:24.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.321+0000 7f077faaa640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f077810f440 con 0x7f0778106f30 2024-04-25T14:13:24.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.321+0000 7f077faaa640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f077810f740 con 0x7f0778109890 2024-04-25T14:13:24.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.321+0000 7f077d846640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07781045d0 0x7f0778103a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:24.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.321+0000 7f077d846640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07781045d0 0x7f0778103a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51488/0 (socket says 172.21.15.96:51488) 2024-04-25T14:13:24.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.321+0000 7f077d846640 1 -- 172.21.15.96:0/1930490510 learned_addr learned my addr 172.21.15.96:0/1930490510 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:24.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.321+0000 7f077d045640 1 --2- 172.21.15.96:0/1930490510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0778106f30 0x7f07781020c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:24.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.321+0000 7f077e047640 1 --2- 172.21.15.96:0/1930490510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0778109890 0x7f0778102600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:24.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.321+0000 7f077d846640 1 -- 172.21.15.96:0/1930490510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0778109890 msgr2=0x7f0778102600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:24.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.321+0000 7f077d846640 1 --2- 172.21.15.96:0/1930490510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0778109890 0x7f0778102600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:24.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.321+0000 7f077d846640 1 -- 172.21.15.96:0/1930490510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0778106f30 msgr2=0x7f07781020c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:24.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.321+0000 7f077d846640 1 --2- 172.21.15.96:0/1930490510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0778106f30 0x7f07781020c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:24.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.321+0000 7f077d846640 1 -- 172.21.15.96:0/1930490510 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0778102ca0 con 0x7f07781045d0 2024-04-25T14:13:24.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.321+0000 7f077d045640 1 --2- 172.21.15.96:0/1930490510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0778106f30 0x7f07781020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:24.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.322+0000 7f077d846640 1 --2- 172.21.15.96:0/1930490510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07781045d0 0x7f0778103a20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0774006010 tx=0x7f0774031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:24.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.322+0000 7f076effd640 1 -- 172.21.15.96:0/1930490510 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077403f030 con 0x7f07781045d0 2024-04-25T14:13:24.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.322+0000 7f076effd640 1 -- 172.21.15.96:0/1930490510 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0774003aa0 con 0x7f07781045d0 2024-04-25T14:13:24.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.322+0000 7f076effd640 1 -- 172.21.15.96:0/1930490510 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0774035070 con 0x7f07781045d0 2024-04-25T14:13:24.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.322+0000 7f077faaa640 1 -- 172.21.15.96:0/1930490510 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07781a4350 con 0x7f07781045d0 2024-04-25T14:13:24.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.322+0000 7f077faaa640 1 -- 172.21.15.96:0/1930490510 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07781a4700 con 0x7f07781045d0 2024-04-25T14:13:24.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.323+0000 7f077faaa640 1 -- 172.21.15.96:0/1930490510 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07780ff6f0 con 0x7f07781045d0 2024-04-25T14:13:24.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.324+0000 7f076effd640 1 -- 172.21.15.96:0/1930490510 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f0774042030 con 0x7f07781045d0 2024-04-25T14:13:24.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.325+0000 7f076effd640 1 --2- 172.21.15.96:0/1930490510 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f07640afaf0 0x7f07640b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:24.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.325+0000 7f077d045640 1 --2- 172.21.15.96:0/1930490510 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f07640afaf0 0x7f07640b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:24.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.325+0000 7f076effd640 1 -- 172.21.15.96:0/1930490510 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f07740ff660 con 0x7f07781045d0 2024-04-25T14:13:24.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.325+0000 7f077d045640 1 --2- 172.21.15.96:0/1930490510 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f07640afaf0 0x7f07640b2000 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0768005fd0 tx=0x7f0768005f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:24.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.328+0000 7f076effd640 1 -- 172.21.15.96:0/1930490510 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f07740c8220 con 0x7f07781045d0 2024-04-25T14:13:24.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.510+0000 7f077faaa640 1 -- 172.21.15.96:0/1930490510 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f077802dd20 con 0x7f07781045d0 2024-04-25T14:13:24.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.512+0000 7f076effd640 1 -- 172.21.15.96:0/1930490510 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22616 (secure 0 0 0) 0x7f07740c7b20 con 0x7f07781045d0 2024-04-25T14:13:24.512 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:13:24.512 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":78,"fsid":"59334182-030c-11ef-bc93-c7b262605968","created":"2024-04-25T14:03:25.680745+0000","modified":"2024-04-25T14:12:22.496143+0000","last_up_change":"2024-04-25T14:10:35.894312+0000","last_in_change":"2024-04-25T14:10:15.583440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:06:47.402927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:11:27.434331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:11:27.854004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cf4ba3ac-142e-4e8e-96d4-410d77157eb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6803","nonce":3832506168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6805","nonce":3832506168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832506168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6807","nonce":3832506168}]},"public_addr":"172.21.15.96:6803/3832506168","cluster_addr":"172.21.15.96:6805/3832506168","heartbeat_back_addr":"172.21.15.96:6809/3832506168","heartbeat_front_addr":"172.21.15.96:6807/3832506168","state":["exists","up"]},{"osd":1,"uuid":"866e73b9-4a26-48ba-badf-704fe22f71cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6801","nonce":1083311494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6803","nonce":1083311494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6807","nonce":1083311494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6805","nonce":1083311494}]},"public_addr":"172.21.15.97:6801/1083311494","cluster_addr":"172.21.15.97:6803/1083311494","heartbeat_back_addr":"172.21.15.97:6807/1083311494","heartbeat_front_addr":"172.21.15.97:6805/1083311494","state":["exists","up"]},{"osd":2,"uuid":"45c79ecb-abc4-4dba-bae8-74804e4bc1bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6801","nonce":981902966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6803","nonce":981902966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6807","nonce":981902966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6805","nonce":981902966}]},"public_addr":"172.21.15.151:6801/981902966","cluster_addr":"172.21.15.151:6803/981902966","heartbeat_back_addr":"172.21.15.151:6807/981902966","heartbeat_front_addr":"172.21.15.151:6805/981902966","state":["exists","up"]},{"osd":3,"uuid":"9fc39468-a7ea-4ccf-b365-335ad2ea4c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6811","nonce":2274158748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6813","nonce":2274158748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6817","nonce":2274158748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6815","nonce":2274158748}]},"public_addr":"172.21.15.96:6811/2274158748","cluster_addr":"172.21.15.96:6813/2274158748","heartbeat_back_addr":"172.21.15.96:6817/2274158748","heartbeat_front_addr":"172.21.15.96:6815/2274158748","state":["exists","up"]},{"osd":4,"uuid":"616582f1-12d2-456b-b7b3-e5517800e8a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6809","nonce":3727031962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6811","nonce":3727031962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6815","nonce":3727031962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6813","nonce":3727031962}]},"public_addr":"172.21.15.97:6809/3727031962","cluster_addr":"172.21.15.97:6811/3727031962","heartbeat_back_addr":"172.21.15.97:6815/3727031962","heartbeat_front_addr":"172.21.15.97:6813/3727031962","state":["exists","up"]},{"osd":5,"uuid":"3d277784-ab0f-4c90-81a1-2a4034c52a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6809","nonce":223882546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6811","nonce":223882546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6815","nonce":223882546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6813","nonce":223882546}]},"public_addr":"172.21.15.151:6809/223882546","cluster_addr":"172.21.15.151:6811/223882546","heartbeat_back_addr":"172.21.15.151:6815/223882546","heartbeat_front_addr":"172.21.15.151:6813/223882546","state":["exists","up"]},{"osd":6,"uuid":"9a9ad585-cb1a-4239-9d18-055f59164049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6819","nonce":96197407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6821","nonce":96197407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6825","nonce":96197407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6823","nonce":96197407}]},"public_addr":"172.21.15.96:6819/96197407","cluster_addr":"172.21.15.96:6821/96197407","heartbeat_back_addr":"172.21.15.96:6825/96197407","heartbeat_front_addr":"172.21.15.96:6823/96197407","state":["exists","up"]},{"osd":7,"uuid":"303551c8-6af0-42f2-844f-45bf8db5baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6817","nonce":2247216400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6819","nonce":2247216400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6823","nonce":2247216400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6821","nonce":2247216400}]},"public_addr":"172.21.15.97:6817/2247216400","cluster_addr":"172.21.15.97:6819/2247216400","heartbeat_back_addr":"172.21.15.97:6823/2247216400","heartbeat_front_addr":"172.21.15.97:6821/2247216400","state":["exists","up"]},{"osd":8,"uuid":"81f6cc66-e4ca-430c-b336-b6678d968818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6817","nonce":2549822185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6819","nonce":2549822185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6823","nonce":2549822185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6821","nonce":2549822185}]},"public_addr":"172.21.15.151:6817/2549822185","cluster_addr":"172.21.15.151:6819/2549822185","heartbeat_back_addr":"172.21.15.151:6823/2549822185","heartbeat_front_addr":"172.21.15.151:6821/2549822185","state":["exists","up"]},{"osd":9,"uuid":"29d4da16-feab-4d1c-9d1d-de4f799bd724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6827","nonce":1099399643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6829","nonce":1099399643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6833","nonce":1099399643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6831","nonce":1099399643}]},"public_addr":"172.21.15.96:6827/1099399643","cluster_addr":"172.21.15.96:6829/1099399643","heartbeat_back_addr":"172.21.15.96:6833/1099399643","heartbeat_front_addr":"172.21.15.96:6831/1099399643","state":["exists","up"]},{"osd":10,"uuid":"197fee9b-eee4-4893-be1c-3d57f42250e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6825","nonce":155754520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6827","nonce":155754520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6831","nonce":155754520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6829","nonce":155754520}]},"public_addr":"172.21.15.97:6825/155754520","cluster_addr":"172.21.15.97:6827/155754520","heartbeat_back_addr":"172.21.15.97:6831/155754520","heartbeat_front_addr":"172.21.15.97:6829/155754520","state":["exists","up"]},{"osd":11,"uuid":"9c1d0a83-fc12-41a2-8eb0-80c4abdd2770","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6825","nonce":2027958746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6827","nonce":2027958746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6831","nonce":2027958746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6829","nonce":2027958746}]},"public_addr":"172.21.15.151:6825/2027958746","cluster_addr":"172.21.15.151:6827/2027958746","heartbeat_back_addr":"172.21.15.151:6831/2027958746","heartbeat_front_addr":"172.21.15.151:6829/2027958746","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:05:56.177390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:06:19.917663+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:06:43.608052+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:08.540382+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:33.474072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:58.242306+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:08:24.120442+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:08:49.661064+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:09:15.097271+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:09:41.904505+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:10:08.233252+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:10:33.712296+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/3839807484":"2024-04-26T14:04:21.307574+0000","172.21.15.96:0/1495237551":"2024-04-26T14:04:21.307574+0000","172.21.15.96:0/2793751450":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/129575834":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/2233075139":"2024-04-26T14:04:21.307574+0000","172.21.15.96:6801/2348397481":"2024-04-26T14:03:54.249537+0000","172.21.15.96:6801/433591062":"2024-04-26T14:04:21.307574+0000","172.21.15.96:6800/2348397481":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/3517674924":"2024-04-26T14:03:54.249537+0000","172.21.15.96:6800/433591062":"2024-04-26T14:04:21.307574+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:13:24.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.514+0000 7f077faaa640 1 -- 172.21.15.96:0/1930490510 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f07640afaf0 msgr2=0x7f07640b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:24.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.514+0000 7f077faaa640 1 --2- 172.21.15.96:0/1930490510 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f07640afaf0 0x7f07640b2000 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0768005fd0 tx=0x7f0768005f40 comp rx=0 tx=0).stop 2024-04-25T14:13:24.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.514+0000 7f077faaa640 1 -- 172.21.15.96:0/1930490510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07781045d0 msgr2=0x7f0778103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:24.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.514+0000 7f077faaa640 1 --2- 172.21.15.96:0/1930490510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07781045d0 0x7f0778103a20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0774006010 tx=0x7f0774031a50 comp rx=0 tx=0).stop 2024-04-25T14:13:24.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.514+0000 7f077faaa640 1 -- 172.21.15.96:0/1930490510 shutdown_connections 2024-04-25T14:13:24.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.514+0000 7f077faaa640 1 --2- 172.21.15.96:0/1930490510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0778109890 0x7f0778102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:24.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.514+0000 7f077faaa640 1 --2- 172.21.15.96:0/1930490510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0778106f30 0x7f07781020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:24.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.514+0000 7f077faaa640 1 --2- 172.21.15.96:0/1930490510 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f07640afaf0 0x7f07640b2000 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:24.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.514+0000 7f077faaa640 1 --2- 172.21.15.96:0/1930490510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07781045d0 0x7f0778103a20 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:24.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.514+0000 7f077faaa640 1 -- 172.21.15.96:0/1930490510 >> 172.21.15.96:0/1930490510 conn(0x7f07780fdbd0 msgr2=0x7f0778106640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:24.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.514+0000 7f077faaa640 1 -- 172.21.15.96:0/1930490510 shutdown_connections 2024-04-25T14:13:24.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.514+0000 7f077faaa640 1 -- 172.21.15.96:0/1930490510 wait complete. 2024-04-25T14:13:24.526 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:06:47.402927+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:11:27.434331+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:11:27.854004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:13:24.526 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:13:24.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.637+0000 7f5f00084640 1 -- 172.21.15.96:0/3700821036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ef8103af0 msgr2=0x7f5ef81047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:24.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.637+0000 7f5f00084640 1 --2- 172.21.15.96:0/3700821036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ef8103af0 0x7f5ef81047f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4009900 tx=0x7f5ef402a410 comp rx=0 tx=0).stop 2024-04-25T14:13:24.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.637+0000 7f5f00084640 1 -- 172.21.15.96:0/3700821036 shutdown_connections 2024-04-25T14:13:24.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.637+0000 7f5f00084640 1 --2- 172.21.15.96:0/3700821036 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ef8109a30 0x7f5ef810be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:24.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.637+0000 7f5f00084640 1 --2- 172.21.15.96:0/3700821036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ef8104d30 0x7f5ef8105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:24.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.637+0000 7f5f00084640 1 --2- 172.21.15.96:0/3700821036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ef8103af0 0x7f5ef81047f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:24.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.637+0000 7f5f00084640 1 -- 172.21.15.96:0/3700821036 >> 172.21.15.96:0/3700821036 conn(0x7f5ef80fece0 msgr2=0x7f5ef8101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:24.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.637+0000 7f5f00084640 1 -- 172.21.15.96:0/3700821036 shutdown_connections 2024-04-25T14:13:24.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.637+0000 7f5f00084640 1 -- 172.21.15.96:0/3700821036 wait complete. 2024-04-25T14:13:24.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.638+0000 7f5f00084640 1 Processor -- start 2024-04-25T14:13:24.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.638+0000 7f5f00084640 1 -- start start 2024-04-25T14:13:24.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.638+0000 7f5f00084640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ef8103af0 0x7f5ef819a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:24.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.638+0000 7f5f00084640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ef8104d30 0x7f5ef819a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:24.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.638+0000 7f5f00084640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ef8109a30 0x7f5ef81a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:24.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.638+0000 7f5f00084640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef810f5d0 con 0x7f5ef8103af0 2024-04-25T14:13:24.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.638+0000 7f5f00084640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef810f450 con 0x7f5ef8109a30 2024-04-25T14:13:24.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.638+0000 7f5f00084640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef810f750 con 0x7f5ef8104d30 2024-04-25T14:13:24.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.638+0000 7f5efde20640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ef8103af0 0x7f5ef819a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:24.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.638+0000 7f5efde20640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ef8103af0 0x7f5ef819a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51502/0 (socket says 172.21.15.96:51502) 2024-04-25T14:13:24.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.638+0000 7f5efde20640 1 -- 172.21.15.96:0/56093779 learned_addr learned my addr 172.21.15.96:0/56093779 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:24.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.638+0000 7f5efe621640 1 --2- 172.21.15.96:0/56093779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ef8109a30 0x7f5ef81a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:24.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.638+0000 7f5efd61f640 1 --2- 172.21.15.96:0/56093779 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ef8104d30 0x7f5ef819a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:24.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.639+0000 7f5efde20640 1 -- 172.21.15.96:0/56093779 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ef8104d30 msgr2=0x7f5ef819a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:24.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.639+0000 7f5efde20640 1 --2- 172.21.15.96:0/56093779 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ef8104d30 0x7f5ef819a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:24.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.639+0000 7f5efde20640 1 -- 172.21.15.96:0/56093779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ef8109a30 msgr2=0x7f5ef81a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:24.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.639+0000 7f5efde20640 1 --2- 172.21.15.96:0/56093779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ef8109a30 0x7f5ef81a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:24.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.639+0000 7f5efde20640 1 -- 172.21.15.96:0/56093779 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ef81a1d80 con 0x7f5ef8103af0 2024-04-25T14:13:24.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.639+0000 7f5efd61f640 1 --2- 172.21.15.96:0/56093779 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ef8104d30 0x7f5ef819a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:24.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.639+0000 7f5efe621640 1 --2- 172.21.15.96:0/56093779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ef8109a30 0x7f5ef81a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:24.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.639+0000 7f5efde20640 1 --2- 172.21.15.96:0/56093779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ef8103af0 0x7f5ef819a290 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4006010 tx=0x7f5ef4031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:24.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.639+0000 7f5eeeffd640 1 -- 172.21.15.96:0/56093779 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef403f030 con 0x7f5ef8103af0 2024-04-25T14:13:24.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.639+0000 7f5eeeffd640 1 -- 172.21.15.96:0/56093779 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ef4003aa0 con 0x7f5ef8103af0 2024-04-25T14:13:24.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.639+0000 7f5f00084640 1 -- 172.21.15.96:0/56093779 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ef81a2010 con 0x7f5ef8103af0 2024-04-25T14:13:24.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.639+0000 7f5eeeffd640 1 -- 172.21.15.96:0/56093779 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef4035070 con 0x7f5ef8103af0 2024-04-25T14:13:24.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.639+0000 7f5f00084640 1 -- 172.21.15.96:0/56093779 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ef81a2500 con 0x7f5ef8103af0 2024-04-25T14:13:24.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.640+0000 7f5f00084640 1 -- 172.21.15.96:0/56093779 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ef8108840 con 0x7f5ef8103af0 2024-04-25T14:13:24.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.641+0000 7f5eeeffd640 1 -- 172.21.15.96:0/56093779 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f5ef4042030 con 0x7f5ef8103af0 2024-04-25T14:13:24.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.642+0000 7f5eeeffd640 1 --2- 172.21.15.96:0/56093779 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f5ee40afaf0 0x7f5ee40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:24.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.642+0000 7f5efd61f640 1 --2- 172.21.15.96:0/56093779 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f5ee40afaf0 0x7f5ee40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:24.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.642+0000 7f5eeeffd640 1 -- 172.21.15.96:0/56093779 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f5ef40ff280 con 0x7f5ef8103af0 2024-04-25T14:13:24.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.642+0000 7f5efd61f640 1 --2- 172.21.15.96:0/56093779 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f5ee40afaf0 0x7f5ee40b2000 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8005fd0 tx=0x7f5ee8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:24.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.647+0000 7f5eeeffd640 1 -- 172.21.15.96:0/56093779 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5ef40c7e40 con 0x7f5ef8103af0 2024-04-25T14:13:24.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.822+0000 7f5f00084640 1 -- 172.21.15.96:0/56093779 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5ef80637a0 con 0x7f5ef8103af0 2024-04-25T14:13:24.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.823+0000 7f5eeeffd640 1 -- 172.21.15.96:0/56093779 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) ==== 93+0+10 (secure 0 0 0) 0x7f5ef40c7740 con 0x7f5ef8103af0 2024-04-25T14:13:24.823 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-04-25T14:13:24.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.825+0000 7f5f00084640 1 -- 172.21.15.96:0/56093779 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f5ee40afaf0 msgr2=0x7f5ee40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:24.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.825+0000 7f5f00084640 1 --2- 172.21.15.96:0/56093779 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f5ee40afaf0 0x7f5ee40b2000 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8005fd0 tx=0x7f5ee8005850 comp rx=0 tx=0).stop 2024-04-25T14:13:24.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.825+0000 7f5f00084640 1 -- 172.21.15.96:0/56093779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ef8103af0 msgr2=0x7f5ef819a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:24.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.825+0000 7f5f00084640 1 --2- 172.21.15.96:0/56093779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ef8103af0 0x7f5ef819a290 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4006010 tx=0x7f5ef4031a50 comp rx=0 tx=0).stop 2024-04-25T14:13:24.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.825+0000 7f5f00084640 1 -- 172.21.15.96:0/56093779 shutdown_connections 2024-04-25T14:13:24.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.825+0000 7f5f00084640 1 --2- 172.21.15.96:0/56093779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ef8109a30 0x7f5ef81a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:24.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.825+0000 7f5f00084640 1 --2- 172.21.15.96:0/56093779 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ef8104d30 0x7f5ef819a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:24.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.825+0000 7f5f00084640 1 --2- 172.21.15.96:0/56093779 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f5ee40afaf0 0x7f5ee40b2000 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:24.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.825+0000 7f5f00084640 1 --2- 172.21.15.96:0/56093779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ef8103af0 0x7f5ef819a290 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:24.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.825+0000 7f5f00084640 1 -- 172.21.15.96:0/56093779 >> 172.21.15.96:0/56093779 conn(0x7f5ef80fece0 msgr2=0x7f5ef8105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:24.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.825+0000 7f5f00084640 1 -- 172.21.15.96:0/56093779 shutdown_connections 2024-04-25T14:13:24.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.825+0000 7f5f00084640 1 -- 172.21.15.96:0/56093779 wait complete. 2024-04-25T14:13:24.835 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:24 smithi096 ceph-mon[29207]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T14:13:24.836 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:24 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/1860959174' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:13:24.836 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:24 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/2293878645' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:13:24.836 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:24 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/1930490510' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:13:24.837 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:13:24.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:24 smithi097 ceph-mon[32720]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T14:13:24.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:24 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/1860959174' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:13:24.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:24 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/2293878645' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:13:24.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:24 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/1930490510' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:13:24.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:24 smithi151 ceph-mon[32990]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T14:13:24.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:24 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/1860959174' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:13:24.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:24 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/2293878645' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:13:24.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:24 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/1930490510' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:13:24.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.946+0000 7f74ede27640 1 -- 172.21.15.96:0/1249028728 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74e8109a40 msgr2=0x7f74e810be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:24.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.946+0000 7f74ede27640 1 --2- 172.21.15.96:0/1249028728 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74e8109a40 0x7f74e810be60 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f74e0009ca0 tx=0x7f74e002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:13:24.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.946+0000 7f74ede27640 1 -- 172.21.15.96:0/1249028728 shutdown_connections 2024-04-25T14:13:24.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.946+0000 7f74ede27640 1 --2- 172.21.15.96:0/1249028728 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74e8109a40 0x7f74e810be60 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:24.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.946+0000 7f74ede27640 1 --2- 172.21.15.96:0/1249028728 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74e8104d40 0x7f74e8105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:24.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.946+0000 7f74ede27640 1 --2- 172.21.15.96:0/1249028728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74e8103b00 0x7f74e8104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:24.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.946+0000 7f74ede27640 1 -- 172.21.15.96:0/1249028728 >> 172.21.15.96:0/1249028728 conn(0x7f74e80fecf0 msgr2=0x7f74e81011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:24.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.947+0000 7f74ede27640 1 -- 172.21.15.96:0/1249028728 shutdown_connections 2024-04-25T14:13:24.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.947+0000 7f74ede27640 1 -- 172.21.15.96:0/1249028728 wait complete. 2024-04-25T14:13:24.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.947+0000 7f74ede27640 1 Processor -- start 2024-04-25T14:13:24.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.947+0000 7f74ede27640 1 -- start start 2024-04-25T14:13:24.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.947+0000 7f74ede27640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74e8103b00 0x7f74e819a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:24.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.948+0000 7f74ede27640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74e8104d40 0x7f74e819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:24.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.948+0000 7f74ede27640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74e8109a40 0x7f74e81a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:24.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.948+0000 7f74ede27640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f74e810f680 con 0x7f74e8104d40 2024-04-25T14:13:24.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.948+0000 7f74ede27640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f74e810f500 con 0x7f74e8103b00 2024-04-25T14:13:24.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.948+0000 7f74ede27640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f74e810f800 con 0x7f74e8109a40 2024-04-25T14:13:24.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.948+0000 7f74e6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74e8104d40 0x7f74e819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:24.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.948+0000 7f74e6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74e8104d40 0x7f74e819a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51522/0 (socket says 172.21.15.96:51522) 2024-04-25T14:13:24.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.948+0000 7f74e6ffd640 1 -- 172.21.15.96:0/1922511766 learned_addr learned my addr 172.21.15.96:0/1922511766 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:24.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.948+0000 7f74e77fe640 1 --2- 172.21.15.96:0/1922511766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74e8103b00 0x7f74e819a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:24.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.948+0000 7f74e6ffd640 1 -- 172.21.15.96:0/1922511766 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74e8109a40 msgr2=0x7f74e81a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:24.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.948+0000 7f74e6ffd640 1 --2- 172.21.15.96:0/1922511766 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74e8109a40 0x7f74e81a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:24.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.948+0000 7f74e6ffd640 1 -- 172.21.15.96:0/1922511766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74e8103b00 msgr2=0x7f74e819a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:24.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.948+0000 7f74e6ffd640 1 --2- 172.21.15.96:0/1922511766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74e8103b00 0x7f74e819a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:24.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.948+0000 7f74e6ffd640 1 -- 172.21.15.96:0/1922511766 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74e81a1c80 con 0x7f74e8104d40 2024-04-25T14:13:24.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.948+0000 7f74e77fe640 1 --2- 172.21.15.96:0/1922511766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74e8103b00 0x7f74e819a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:24.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.948+0000 7f74e6ffd640 1 --2- 172.21.15.96:0/1922511766 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74e8104d40 0x7f74e819a730 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f74dc00e9f0 tx=0x7f74dc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:24.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.948+0000 7f74e4ff9640 1 -- 172.21.15.96:0/1922511766 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74dc00cd80 con 0x7f74e8104d40 2024-04-25T14:13:24.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.948+0000 7f74e4ff9640 1 -- 172.21.15.96:0/1922511766 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74dc0054d0 con 0x7f74e8104d40 2024-04-25T14:13:24.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.948+0000 7f74ede27640 1 -- 172.21.15.96:0/1922511766 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74e81a1f70 con 0x7f74e8104d40 2024-04-25T14:13:24.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.948+0000 7f74e4ff9640 1 -- 172.21.15.96:0/1922511766 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74dc010690 con 0x7f74e8104d40 2024-04-25T14:13:24.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.948+0000 7f74ede27640 1 -- 172.21.15.96:0/1922511766 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74e81a23e0 con 0x7f74e8104d40 2024-04-25T14:13:24.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.949+0000 7f74ede27640 1 -- 172.21.15.96:0/1922511766 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74e8108850 con 0x7f74e8104d40 2024-04-25T14:13:24.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.951+0000 7f74e4ff9640 1 -- 172.21.15.96:0/1922511766 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f74dc004e90 con 0x7f74e8104d40 2024-04-25T14:13:24.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.951+0000 7f74e4ff9640 1 --2- 172.21.15.96:0/1922511766 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f74d00afb70 0x7f74d00b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:24.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.951+0000 7f74e77fe640 1 --2- 172.21.15.96:0/1922511766 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f74d00afb70 0x7f74d00b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:24.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.951+0000 7f74e4ff9640 1 -- 172.21.15.96:0/1922511766 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f74dc0df100 con 0x7f74e8104d40 2024-04-25T14:13:24.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.952+0000 7f74e77fe640 1 --2- 172.21.15.96:0/1922511766 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f74d00afb70 0x7f74d00b2080 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f74e8104700 tx=0x7f74d800a5e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:24.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:24.956+0000 7f74e4ff9640 1 -- 172.21.15.96:0/1922511766 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f74dc0a7cc0 con 0x7f74e8104d40 2024-04-25T14:13:25.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.135+0000 7f74ede27640 1 -- 172.21.15.96:0/1922511766 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f74e80637d0 con 0x7f74e8104d40 2024-04-25T14:13:25.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.135+0000 7f74e4ff9640 1 -- 172.21.15.96:0/1922511766 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v78) ==== 104+0+11 (secure 0 0 0) 0x7f74dc0a75c0 con 0x7f74e8104d40 2024-04-25T14:13:25.135 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-04-25T14:13:25.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.137+0000 7f74ede27640 1 -- 172.21.15.96:0/1922511766 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f74d00afb70 msgr2=0x7f74d00b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:25.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.137+0000 7f74ede27640 1 --2- 172.21.15.96:0/1922511766 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f74d00afb70 0x7f74d00b2080 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f74e8104700 tx=0x7f74d800a5e0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.137+0000 7f74ede27640 1 -- 172.21.15.96:0/1922511766 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74e8104d40 msgr2=0x7f74e819a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:25.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.137+0000 7f74ede27640 1 --2- 172.21.15.96:0/1922511766 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74e8104d40 0x7f74e819a730 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f74dc00e9f0 tx=0x7f74dc00edf0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.137+0000 7f74ede27640 1 -- 172.21.15.96:0/1922511766 shutdown_connections 2024-04-25T14:13:25.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.137+0000 7f74ede27640 1 --2- 172.21.15.96:0/1922511766 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74e8109a40 0x7f74e81a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.137+0000 7f74ede27640 1 --2- 172.21.15.96:0/1922511766 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f74d00afb70 0x7f74d00b2080 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.137+0000 7f74ede27640 1 --2- 172.21.15.96:0/1922511766 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74e8104d40 0x7f74e819a730 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.137+0000 7f74ede27640 1 --2- 172.21.15.96:0/1922511766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74e8103b00 0x7f74e819a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.137+0000 7f74ede27640 1 -- 172.21.15.96:0/1922511766 >> 172.21.15.96:0/1922511766 conn(0x7f74e80fecf0 msgr2=0x7f74e8105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:25.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.137+0000 7f74ede27640 1 -- 172.21.15.96:0/1922511766 shutdown_connections 2024-04-25T14:13:25.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.137+0000 7f74ede27640 1 -- 172.21.15.96:0/1922511766 wait complete. 2024-04-25T14:13:25.148 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:13:25.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.259+0000 7f48f126e640 1 -- 172.21.15.96:0/2791006041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48ec109900 msgr2=0x7f48ec10bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:25.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.259+0000 7f48f126e640 1 --2- 172.21.15.96:0/2791006041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48ec109900 0x7f48ec10bd20 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f48e4009900 tx=0x7f48e402a410 comp rx=0 tx=0).stop 2024-04-25T14:13:25.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.259+0000 7f48f126e640 1 -- 172.21.15.96:0/2791006041 shutdown_connections 2024-04-25T14:13:25.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.259+0000 7f48f126e640 1 --2- 172.21.15.96:0/2791006041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48ec109900 0x7f48ec10bd20 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.259+0000 7f48f126e640 1 --2- 172.21.15.96:0/2791006041 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48ec106fa0 0x7f48ec1093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.259+0000 7f48f126e640 1 --2- 172.21.15.96:0/2791006041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48ec104640 0x7f48ec106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.259+0000 7f48f126e640 1 -- 172.21.15.96:0/2791006041 >> 172.21.15.96:0/2791006041 conn(0x7f48ec0fdc40 msgr2=0x7f48ec1000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:25.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.259+0000 7f48f126e640 1 -- 172.21.15.96:0/2791006041 shutdown_connections 2024-04-25T14:13:25.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.259+0000 7f48f126e640 1 -- 172.21.15.96:0/2791006041 wait complete. 2024-04-25T14:13:25.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.259+0000 7f48f126e640 1 Processor -- start 2024-04-25T14:13:25.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.259+0000 7f48f126e640 1 -- start start 2024-04-25T14:13:25.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.260+0000 7f48f126e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48ec104640 0x7f48ec19a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:25.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.260+0000 7f48f126e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48ec106fa0 0x7f48ec19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:25.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.260+0000 7f48f126e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48ec109900 0x7f48ec1a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:25.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.260+0000 7f48f126e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f48ec10f5f0 con 0x7f48ec104640 2024-04-25T14:13:25.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.260+0000 7f48f126e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f48ec10f470 con 0x7f48ec106fa0 2024-04-25T14:13:25.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.260+0000 7f48f126e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f48ec10f770 con 0x7f48ec109900 2024-04-25T14:13:25.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.260+0000 7f48eaffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48ec104640 0x7f48ec19a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:25.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.260+0000 7f48eaffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48ec104640 0x7f48ec19a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51542/0 (socket says 172.21.15.96:51542) 2024-04-25T14:13:25.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.260+0000 7f48eaffd640 1 -- 172.21.15.96:0/2892399507 learned_addr learned my addr 172.21.15.96:0/2892399507 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:25.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.260+0000 7f48eaffd640 1 -- 172.21.15.96:0/2892399507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48ec109900 msgr2=0x7f48ec1a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:25.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.260+0000 7f48eaffd640 1 --2- 172.21.15.96:0/2892399507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48ec109900 0x7f48ec1a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.260+0000 7f48eaffd640 1 -- 172.21.15.96:0/2892399507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48ec106fa0 msgr2=0x7f48ec19a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:25.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.260+0000 7f48eaffd640 1 --2- 172.21.15.96:0/2892399507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48ec106fa0 0x7f48ec19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.260+0000 7f48eaffd640 1 -- 172.21.15.96:0/2892399507 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48ec1a1c50 con 0x7f48ec104640 2024-04-25T14:13:25.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.260+0000 7f48eaffd640 1 --2- 172.21.15.96:0/2892399507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48ec104640 0x7f48ec19a1c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f48dc00ccb0 tx=0x7f48dc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:25.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.260+0000 7f48e3fff640 1 -- 172.21.15.96:0/2892399507 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48dc00ce60 con 0x7f48ec104640 2024-04-25T14:13:25.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.260+0000 7f48e3fff640 1 -- 172.21.15.96:0/2892399507 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48dc00fb00 con 0x7f48ec104640 2024-04-25T14:13:25.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.260+0000 7f48f126e640 1 -- 172.21.15.96:0/2892399507 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48ec1a1f40 con 0x7f48ec104640 2024-04-25T14:13:25.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.260+0000 7f48e3fff640 1 -- 172.21.15.96:0/2892399507 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48dc018bb0 con 0x7f48ec104640 2024-04-25T14:13:25.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.260+0000 7f48f126e640 1 -- 172.21.15.96:0/2892399507 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48ec1a2400 con 0x7f48ec104640 2024-04-25T14:13:25.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.261+0000 7f48f126e640 1 -- 172.21.15.96:0/2892399507 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48ec0ff760 con 0x7f48ec104640 2024-04-25T14:13:25.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.263+0000 7f48e3fff640 1 -- 172.21.15.96:0/2892399507 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f48dc00f650 con 0x7f48ec104640 2024-04-25T14:13:25.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.264+0000 7f48e3fff640 1 --2- 172.21.15.96:0/2892399507 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f48d80afaf0 0x7f48d80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:25.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.264+0000 7f48ea7fc640 1 --2- 172.21.15.96:0/2892399507 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f48d80afaf0 0x7f48d80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:25.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.264+0000 7f48e3fff640 1 -- 172.21.15.96:0/2892399507 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f48dc0df440 con 0x7f48ec104640 2024-04-25T14:13:25.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.264+0000 7f48ea7fc640 1 --2- 172.21.15.96:0/2892399507 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f48d80afaf0 0x7f48d80b2000 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f48d4005fd0 tx=0x7f48d4005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:25.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.267+0000 7f48e3fff640 1 -- 172.21.15.96:0/2892399507 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f48dc014030 con 0x7f48ec104640 2024-04-25T14:13:25.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.444+0000 7f48f126e640 1 -- 172.21.15.96:0/2892399507 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f48ec02dd00 con 0x7f48ec104640 2024-04-25T14:13:25.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.444+0000 7f48e3fff640 1 -- 172.21.15.96:0/2892399507 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v78) ==== 100+0+11 (secure 0 0 0) 0x7f48dc0a8030 con 0x7f48ec104640 2024-04-25T14:13:25.444 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-04-25T14:13:25.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.446+0000 7f48f126e640 1 -- 172.21.15.96:0/2892399507 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f48d80afaf0 msgr2=0x7f48d80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:25.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.446+0000 7f48f126e640 1 --2- 172.21.15.96:0/2892399507 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f48d80afaf0 0x7f48d80b2000 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f48d4005fd0 tx=0x7f48d4005c70 comp rx=0 tx=0).stop 2024-04-25T14:13:25.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.446+0000 7f48f126e640 1 -- 172.21.15.96:0/2892399507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48ec104640 msgr2=0x7f48ec19a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:25.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.446+0000 7f48f126e640 1 --2- 172.21.15.96:0/2892399507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48ec104640 0x7f48ec19a1c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f48dc00ccb0 tx=0x7f48dc00a620 comp rx=0 tx=0).stop 2024-04-25T14:13:25.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.446+0000 7f48f126e640 1 -- 172.21.15.96:0/2892399507 shutdown_connections 2024-04-25T14:13:25.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.446+0000 7f48f126e640 1 --2- 172.21.15.96:0/2892399507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48ec109900 0x7f48ec1a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.446+0000 7f48f126e640 1 --2- 172.21.15.96:0/2892399507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48ec106fa0 0x7f48ec19a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.446+0000 7f48f126e640 1 --2- 172.21.15.96:0/2892399507 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f48d80afaf0 0x7f48d80b2000 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.446+0000 7f48f126e640 1 --2- 172.21.15.96:0/2892399507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48ec104640 0x7f48ec19a1c0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.446+0000 7f48f126e640 1 -- 172.21.15.96:0/2892399507 >> 172.21.15.96:0/2892399507 conn(0x7f48ec0fdc40 msgr2=0x7f48ec105500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:25.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.447+0000 7f48f126e640 1 -- 172.21.15.96:0/2892399507 shutdown_connections 2024-04-25T14:13:25.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.447+0000 7f48f126e640 1 -- 172.21.15.96:0/2892399507 wait complete. 2024-04-25T14:13:25.458 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:13:25.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.566+0000 7f1ff1196640 1 -- 172.21.15.96:0/2239007926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fec105080 msgr2=0x7f1fec105490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:25.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.566+0000 7f1ff1196640 1 --2- 172.21.15.96:0/2239007926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fec105080 0x7f1fec105490 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc009900 tx=0x7f1fdc02a410 comp rx=0 tx=0).stop 2024-04-25T14:13:25.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.566+0000 7f1ff1196640 1 -- 172.21.15.96:0/2239007926 shutdown_connections 2024-04-25T14:13:25.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.566+0000 7f1ff1196640 1 --2- 172.21.15.96:0/2239007926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fec102230 0x7f1fec10bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.566+0000 7f1ff1196640 1 --2- 172.21.15.96:0/2239007926 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fec101730 0x7f1fec101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.566+0000 7f1ff1196640 1 --2- 172.21.15.96:0/2239007926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fec105080 0x7f1fec105490 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.566+0000 7f1ff1196640 1 -- 172.21.15.96:0/2239007926 >> 172.21.15.96:0/2239007926 conn(0x7f1fec0fd330 msgr2=0x7f1fec0ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:25.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.566+0000 7f1ff1196640 1 -- 172.21.15.96:0/2239007926 shutdown_connections 2024-04-25T14:13:25.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.566+0000 7f1ff1196640 1 -- 172.21.15.96:0/2239007926 wait complete. 2024-04-25T14:13:25.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.566+0000 7f1ff1196640 1 Processor -- start 2024-04-25T14:13:25.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.567+0000 7f1ff1196640 1 -- start start 2024-04-25T14:13:25.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.567+0000 7f1ff1196640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fec101730 0x7f1fec19a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:25.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.567+0000 7f1ff1196640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fec102230 0x7f1fec19a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:25.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.567+0000 7f1ff1196640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fec105080 0x7f1fec1a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:25.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.567+0000 7f1ff1196640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec10f5f0 con 0x7f1fec105080 2024-04-25T14:13:25.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.567+0000 7f1ff1196640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec10f470 con 0x7f1fec102230 2024-04-25T14:13:25.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.567+0000 7f1ff1196640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec10f770 con 0x7f1fec101730 2024-04-25T14:13:25.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.567+0000 7f1feb59e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fec105080 0x7f1fec1a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:25.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.567+0000 7f1feb59e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fec105080 0x7f1fec1a17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51568/0 (socket says 172.21.15.96:51568) 2024-04-25T14:13:25.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.567+0000 7f1feb59e640 1 -- 172.21.15.96:0/3841485710 learned_addr learned my addr 172.21.15.96:0/3841485710 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:25.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.567+0000 7f1fea59c640 1 --2- 172.21.15.96:0/3841485710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fec102230 0x7f1fec19a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:25.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.567+0000 7f1fead9d640 1 --2- 172.21.15.96:0/3841485710 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fec101730 0x7f1fec19a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:25.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.567+0000 7f1feb59e640 1 -- 172.21.15.96:0/3841485710 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fec101730 msgr2=0x7f1fec19a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:25.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.567+0000 7f1feb59e640 1 --2- 172.21.15.96:0/3841485710 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fec101730 0x7f1fec19a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.567+0000 7f1feb59e640 1 -- 172.21.15.96:0/3841485710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fec102230 msgr2=0x7f1fec19a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:25.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.567+0000 7f1feb59e640 1 --2- 172.21.15.96:0/3841485710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fec102230 0x7f1fec19a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.567+0000 7f1feb59e640 1 -- 172.21.15.96:0/3841485710 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fec1a1e50 con 0x7f1fec105080 2024-04-25T14:13:25.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.568+0000 7f1feb59e640 1 --2- 172.21.15.96:0/3841485710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fec105080 0x7f1fec1a17e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f1fe400eaa0 tx=0x7f1fe400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:25.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.568+0000 7f1fd3fff640 1 -- 172.21.15.96:0/3841485710 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe400cdb0 con 0x7f1fec105080 2024-04-25T14:13:25.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.568+0000 7f1fd3fff640 1 -- 172.21.15.96:0/3841485710 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fe4005020 con 0x7f1fec105080 2024-04-25T14:13:25.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.568+0000 7f1ff1196640 1 -- 172.21.15.96:0/3841485710 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fec1a2140 con 0x7f1fec105080 2024-04-25T14:13:25.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.568+0000 7f1fd3fff640 1 -- 172.21.15.96:0/3841485710 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe40116d0 con 0x7f1fec105080 2024-04-25T14:13:25.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.568+0000 7f1ff1196640 1 -- 172.21.15.96:0/3841485710 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fec1a2640 con 0x7f1fec105080 2024-04-25T14:13:25.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.569+0000 7f1ff1196640 1 -- 172.21.15.96:0/3841485710 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fec108a10 con 0x7f1fec105080 2024-04-25T14:13:25.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.571+0000 7f1fd3fff640 1 -- 172.21.15.96:0/3841485710 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f1fe40051d0 con 0x7f1fec105080 2024-04-25T14:13:25.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.572+0000 7f1fd3fff640 1 --2- 172.21.15.96:0/3841485710 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1fd40afaf0 0x7f1fd40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:25.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.572+0000 7f1fead9d640 1 --2- 172.21.15.96:0/3841485710 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1fd40afaf0 0x7f1fd40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:25.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.572+0000 7f1fd3fff640 1 -- 172.21.15.96:0/3841485710 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f1fe40df060 con 0x7f1fec105080 2024-04-25T14:13:25.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.572+0000 7f1fead9d640 1 --2- 172.21.15.96:0/3841485710 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1fd40afaf0 0x7f1fd40b2000 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc005ab0 tx=0x7f1fdc031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:25.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.575+0000 7f1fd3fff640 1 -- 172.21.15.96:0/3841485710 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1fe4016030 con 0x7f1fec105080 2024-04-25T14:13:25.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.791+0000 7f1ff1196640 1 -- 172.21.15.96:0/3841485710 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1fec02db50 con 0x7f1fec105080 2024-04-25T14:13:25.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.792+0000 7f1fd3fff640 1 -- 172.21.15.96:0/3841485710 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11927 (secure 0 0 0) 0x7f1fe40a7610 con 0x7f1fec105080 2024-04-25T14:13:25.792 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:13:25.792 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/606227917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":606227917},{"type":"v1","addr":"172.21.15.96:6835","nonce":606227917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/3265300710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3265300710},{"type":"v1","addr":"172.21.15.96:6837","nonce":3265300710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T14:11:28.886961+0000","modified":"2024-04-25T14:11:35.532775+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24517,"mds_2":24478,"mds_3":24455,"mds_4":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/4080308509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":4080308509},{"type":"v1","addr":"172.21.15.96:6839","nonce":4080308509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/1606748971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":1606748971},{"type":"v1","addr":"172.21.15.96:6841","nonce":1606748971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6833/1995324831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1995324831},{"type":"v1","addr":"172.21.15.97:6833","nonce":1995324831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":4,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.151:6833/1382721608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1382721608},{"type":"v1","addr":"172.21.15.151:6833","nonce":1382721608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6835/3872301726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3872301726},{"type":"v1","addr":"172.21.15.97:6835","nonce":3872301726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.151:6835/1313823638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":1313823638},{"type":"v1","addr":"172.21.15.151:6835","nonce":1313823638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/2691348366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2691348366},{"type":"v1","addr":"172.21.15.151:6837","nonce":2691348366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3306780081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3306780081},{"type":"v1","addr":"172.21.15.97:6837","nonce":3306780081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.151:6839/245699981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":245699981},{"type":"v1","addr":"172.21.15.151:6839","nonce":245699981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6839/2850442269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":2850442269},{"type":"v1","addr":"172.21.15.97:6839","nonce":2850442269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24517,24536,24478,24455,24427]},"id":1}]} 2024-04-25T14:13:25.792 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 25 2024-04-25T14:13:25.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.794+0000 7f1ff1196640 1 -- 172.21.15.96:0/3841485710 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1fd40afaf0 msgr2=0x7f1fd40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:25.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.794+0000 7f1ff1196640 1 --2- 172.21.15.96:0/3841485710 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1fd40afaf0 0x7f1fd40b2000 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc005ab0 tx=0x7f1fdc031400 comp rx=0 tx=0).stop 2024-04-25T14:13:25.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.794+0000 7f1ff1196640 1 -- 172.21.15.96:0/3841485710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fec105080 msgr2=0x7f1fec1a17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:25.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.794+0000 7f1ff1196640 1 --2- 172.21.15.96:0/3841485710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fec105080 0x7f1fec1a17e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f1fe400eaa0 tx=0x7f1fe400ef60 comp rx=0 tx=0).stop 2024-04-25T14:13:25.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.794+0000 7f1ff1196640 1 -- 172.21.15.96:0/3841485710 shutdown_connections 2024-04-25T14:13:25.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.794+0000 7f1ff1196640 1 --2- 172.21.15.96:0/3841485710 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1fd40afaf0 0x7f1fd40b2000 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.794+0000 7f1ff1196640 1 --2- 172.21.15.96:0/3841485710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fec105080 0x7f1fec1a17e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.794+0000 7f1ff1196640 1 --2- 172.21.15.96:0/3841485710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fec102230 0x7f1fec19a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.794+0000 7f1ff1196640 1 --2- 172.21.15.96:0/3841485710 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fec101730 0x7f1fec19a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.794+0000 7f1ff1196640 1 -- 172.21.15.96:0/3841485710 >> 172.21.15.96:0/3841485710 conn(0x7f1fec0fd330 msgr2=0x7f1fec105be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:25.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.794+0000 7f1ff1196640 1 -- 172.21.15.96:0/3841485710 shutdown_connections 2024-04-25T14:13:25.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.794+0000 7f1ff1196640 1 -- 172.21.15.96:0/3841485710 wait complete. 2024-04-25T14:13:25.804 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:25 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/56093779' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:13:25.804 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:25 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/1922511766' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:13:25.804 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:25 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/2892399507' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:13:25.806 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:13:25.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.913+0000 7f03efbb2640 1 -- 172.21.15.96:0/1625213485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03e8103c00 msgr2=0x7f03e8104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:25.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.913+0000 7f03efbb2640 1 --2- 172.21.15.96:0/1625213485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03e8103c00 0x7f03e8104010 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f03e4009900 tx=0x7f03e402a410 comp rx=0 tx=0).stop 2024-04-25T14:13:25.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.913+0000 7f03efbb2640 1 -- 172.21.15.96:0/1625213485 shutdown_connections 2024-04-25T14:13:25.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.913+0000 7f03efbb2640 1 --2- 172.21.15.96:0/1625213485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03e8108900 0x7f03e810ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.913+0000 7f03efbb2640 1 --2- 172.21.15.96:0/1625213485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03e8103c00 0x7f03e8104010 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.913+0000 7f03efbb2640 1 --2- 172.21.15.96:0/1625213485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03e81029c0 0x7f03e81036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.913+0000 7f03efbb2640 1 -- 172.21.15.96:0/1625213485 >> 172.21.15.96:0/1625213485 conn(0x7f03e80fdbd0 msgr2=0x7f03e8100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:25.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.913+0000 7f03efbb2640 1 -- 172.21.15.96:0/1625213485 shutdown_connections 2024-04-25T14:13:25.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.913+0000 7f03efbb2640 1 -- 172.21.15.96:0/1625213485 wait complete. 2024-04-25T14:13:25.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.914+0000 7f03efbb2640 1 Processor -- start 2024-04-25T14:13:25.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.914+0000 7f03efbb2640 1 -- start start 2024-04-25T14:13:25.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.914+0000 7f03efbb2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03e81029c0 0x7f03e819a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:25.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.914+0000 7f03efbb2640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03e8103c00 0x7f03e819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:25.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.914+0000 7f03efbb2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03e8108900 0x7f03e81a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:25.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.914+0000 7f03efbb2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f03e810f5f0 con 0x7f03e8108900 2024-04-25T14:13:25.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.914+0000 7f03efbb2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f03e810f470 con 0x7f03e81029c0 2024-04-25T14:13:25.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.914+0000 7f03efbb2640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f03e810f770 con 0x7f03e8103c00 2024-04-25T14:13:25.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.914+0000 7f03ee14f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03e8108900 0x7f03e81a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:25.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.914+0000 7f03ee14f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03e8108900 0x7f03e81a1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51578/0 (socket says 172.21.15.96:51578) 2024-04-25T14:13:25.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.914+0000 7f03ee14f640 1 -- 172.21.15.96:0/983391090 learned_addr learned my addr 172.21.15.96:0/983391090 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:25.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.914+0000 7f03ed94e640 1 --2- 172.21.15.96:0/983391090 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03e81029c0 0x7f03e819a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:25.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.914+0000 7f03ed14d640 1 --2- 172.21.15.96:0/983391090 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03e8103c00 0x7f03e819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:25.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.914+0000 7f03ee14f640 1 -- 172.21.15.96:0/983391090 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03e8103c00 msgr2=0x7f03e819a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:25.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.914+0000 7f03ee14f640 1 --2- 172.21.15.96:0/983391090 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03e8103c00 0x7f03e819a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.914+0000 7f03ee14f640 1 -- 172.21.15.96:0/983391090 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03e81029c0 msgr2=0x7f03e819a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:25.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.914+0000 7f03ee14f640 1 --2- 172.21.15.96:0/983391090 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03e81029c0 0x7f03e819a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.914+0000 7f03ee14f640 1 -- 172.21.15.96:0/983391090 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03e81a1cc0 con 0x7f03e8108900 2024-04-25T14:13:25.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.914+0000 7f03ed94e640 1 --2- 172.21.15.96:0/983391090 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03e81029c0 0x7f03e819a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:25.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.914+0000 7f03ee14f640 1 --2- 172.21.15.96:0/983391090 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03e8108900 0x7f03e81a1700 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f03e000cce0 tx=0x7f03e000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:25.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.914+0000 7f03deffd640 1 -- 172.21.15.96:0/983391090 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e000f450 con 0x7f03e8108900 2024-04-25T14:13:25.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.915+0000 7f03deffd640 1 -- 172.21.15.96:0/983391090 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03e000faa0 con 0x7f03e8108900 2024-04-25T14:13:25.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.915+0000 7f03efbb2640 1 -- 172.21.15.96:0/983391090 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03e81a1fb0 con 0x7f03e8108900 2024-04-25T14:13:25.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.915+0000 7f03deffd640 1 -- 172.21.15.96:0/983391090 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e0018b70 con 0x7f03e8108900 2024-04-25T14:13:25.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.915+0000 7f03efbb2640 1 -- 172.21.15.96:0/983391090 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03e81a2410 con 0x7f03e8108900 2024-04-25T14:13:25.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.916+0000 7f03efbb2640 1 -- 172.21.15.96:0/983391090 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03e806afd0 con 0x7f03e8108900 2024-04-25T14:13:25.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.917+0000 7f03deffd640 1 -- 172.21.15.96:0/983391090 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f03e0018d10 con 0x7f03e8108900 2024-04-25T14:13:25.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.918+0000 7f03deffd640 1 --2- 172.21.15.96:0/983391090 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f03d40afbf0 0x7f03d40b2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:25.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.918+0000 7f03ed94e640 1 --2- 172.21.15.96:0/983391090 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f03d40afbf0 0x7f03d40b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:25.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.918+0000 7f03deffd640 1 -- 172.21.15.96:0/983391090 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f03e00df1e0 con 0x7f03e8108900 2024-04-25T14:13:25.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.918+0000 7f03ed94e640 1 --2- 172.21.15.96:0/983391090 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f03d40afbf0 0x7f03d40b2100 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f03e81035c0 tx=0x7f03d8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:25.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:25.921+0000 7f03deffd640 1 -- 172.21.15.96:0/983391090 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f03e806afd0 con 0x7f03e8108900 2024-04-25T14:13:25.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:25 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/56093779' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:13:25.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:25 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/1922511766' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:13:25.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:25 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/2892399507' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:13:25.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:25 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/56093779' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:13:25.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:25 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/1922511766' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:13:25.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:25 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/2892399507' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:13:26.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.098+0000 7f03efbb2640 1 -- 172.21.15.96:0/983391090 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f03e802dc10 con 0x7f03e8108900 2024-04-25T14:13:26.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.100+0000 7f03deffd640 1 -- 172.21.15.96:0/983391090 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22616 (secure 0 0 0) 0x7f03e0014030 con 0x7f03e8108900 2024-04-25T14:13:26.100 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:13:26.100 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":78,"fsid":"59334182-030c-11ef-bc93-c7b262605968","created":"2024-04-25T14:03:25.680745+0000","modified":"2024-04-25T14:12:22.496143+0000","last_up_change":"2024-04-25T14:10:35.894312+0000","last_in_change":"2024-04-25T14:10:15.583440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:06:47.402927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:11:27.434331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:11:27.854004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cf4ba3ac-142e-4e8e-96d4-410d77157eb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6803","nonce":3832506168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6805","nonce":3832506168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832506168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6807","nonce":3832506168}]},"public_addr":"172.21.15.96:6803/3832506168","cluster_addr":"172.21.15.96:6805/3832506168","heartbeat_back_addr":"172.21.15.96:6809/3832506168","heartbeat_front_addr":"172.21.15.96:6807/3832506168","state":["exists","up"]},{"osd":1,"uuid":"866e73b9-4a26-48ba-badf-704fe22f71cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6801","nonce":1083311494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6803","nonce":1083311494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6807","nonce":1083311494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6805","nonce":1083311494}]},"public_addr":"172.21.15.97:6801/1083311494","cluster_addr":"172.21.15.97:6803/1083311494","heartbeat_back_addr":"172.21.15.97:6807/1083311494","heartbeat_front_addr":"172.21.15.97:6805/1083311494","state":["exists","up"]},{"osd":2,"uuid":"45c79ecb-abc4-4dba-bae8-74804e4bc1bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6801","nonce":981902966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6803","nonce":981902966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6807","nonce":981902966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6805","nonce":981902966}]},"public_addr":"172.21.15.151:6801/981902966","cluster_addr":"172.21.15.151:6803/981902966","heartbeat_back_addr":"172.21.15.151:6807/981902966","heartbeat_front_addr":"172.21.15.151:6805/981902966","state":["exists","up"]},{"osd":3,"uuid":"9fc39468-a7ea-4ccf-b365-335ad2ea4c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6811","nonce":2274158748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6813","nonce":2274158748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6817","nonce":2274158748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6815","nonce":2274158748}]},"public_addr":"172.21.15.96:6811/2274158748","cluster_addr":"172.21.15.96:6813/2274158748","heartbeat_back_addr":"172.21.15.96:6817/2274158748","heartbeat_front_addr":"172.21.15.96:6815/2274158748","state":["exists","up"]},{"osd":4,"uuid":"616582f1-12d2-456b-b7b3-e5517800e8a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6809","nonce":3727031962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6811","nonce":3727031962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6815","nonce":3727031962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6813","nonce":3727031962}]},"public_addr":"172.21.15.97:6809/3727031962","cluster_addr":"172.21.15.97:6811/3727031962","heartbeat_back_addr":"172.21.15.97:6815/3727031962","heartbeat_front_addr":"172.21.15.97:6813/3727031962","state":["exists","up"]},{"osd":5,"uuid":"3d277784-ab0f-4c90-81a1-2a4034c52a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6809","nonce":223882546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6811","nonce":223882546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6815","nonce":223882546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6813","nonce":223882546}]},"public_addr":"172.21.15.151:6809/223882546","cluster_addr":"172.21.15.151:6811/223882546","heartbeat_back_addr":"172.21.15.151:6815/223882546","heartbeat_front_addr":"172.21.15.151:6813/223882546","state":["exists","up"]},{"osd":6,"uuid":"9a9ad585-cb1a-4239-9d18-055f59164049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6819","nonce":96197407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6821","nonce":96197407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6825","nonce":96197407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6823","nonce":96197407}]},"public_addr":"172.21.15.96:6819/96197407","cluster_addr":"172.21.15.96:6821/96197407","heartbeat_back_addr":"172.21.15.96:6825/96197407","heartbeat_front_addr":"172.21.15.96:6823/96197407","state":["exists","up"]},{"osd":7,"uuid":"303551c8-6af0-42f2-844f-45bf8db5baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6817","nonce":2247216400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6819","nonce":2247216400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6823","nonce":2247216400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6821","nonce":2247216400}]},"public_addr":"172.21.15.97:6817/2247216400","cluster_addr":"172.21.15.97:6819/2247216400","heartbeat_back_addr":"172.21.15.97:6823/2247216400","heartbeat_front_addr":"172.21.15.97:6821/2247216400","state":["exists","up"]},{"osd":8,"uuid":"81f6cc66-e4ca-430c-b336-b6678d968818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6817","nonce":2549822185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6819","nonce":2549822185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6823","nonce":2549822185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6821","nonce":2549822185}]},"public_addr":"172.21.15.151:6817/2549822185","cluster_addr":"172.21.15.151:6819/2549822185","heartbeat_back_addr":"172.21.15.151:6823/2549822185","heartbeat_front_addr":"172.21.15.151:6821/2549822185","state":["exists","up"]},{"osd":9,"uuid":"29d4da16-feab-4d1c-9d1d-de4f799bd724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6827","nonce":1099399643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6829","nonce":1099399643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6833","nonce":1099399643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6831","nonce":1099399643}]},"public_addr":"172.21.15.96:6827/1099399643","cluster_addr":"172.21.15.96:6829/1099399643","heartbeat_back_addr":"172.21.15.96:6833/1099399643","heartbeat_front_addr":"172.21.15.96:6831/1099399643","state":["exists","up"]},{"osd":10,"uuid":"197fee9b-eee4-4893-be1c-3d57f42250e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6825","nonce":155754520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6827","nonce":155754520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6831","nonce":155754520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6829","nonce":155754520}]},"public_addr":"172.21.15.97:6825/155754520","cluster_addr":"172.21.15.97:6827/155754520","heartbeat_back_addr":"172.21.15.97:6831/155754520","heartbeat_front_addr":"172.21.15.97:6829/155754520","state":["exists","up"]},{"osd":11,"uuid":"9c1d0a83-fc12-41a2-8eb0-80c4abdd2770","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6825","nonce":2027958746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6827","nonce":2027958746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6831","nonce":2027958746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6829","nonce":2027958746}]},"public_addr":"172.21.15.151:6825/2027958746","cluster_addr":"172.21.15.151:6827/2027958746","heartbeat_back_addr":"172.21.15.151:6831/2027958746","heartbeat_front_addr":"172.21.15.151:6829/2027958746","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:05:56.177390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:06:19.917663+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:06:43.608052+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:08.540382+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:33.474072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:58.242306+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:08:24.120442+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:08:49.661064+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:09:15.097271+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:09:41.904505+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:10:08.233252+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:10:33.712296+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/3839807484":"2024-04-26T14:04:21.307574+0000","172.21.15.96:0/1495237551":"2024-04-26T14:04:21.307574+0000","172.21.15.96:0/2793751450":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/129575834":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/2233075139":"2024-04-26T14:04:21.307574+0000","172.21.15.96:6801/2348397481":"2024-04-26T14:03:54.249537+0000","172.21.15.96:6801/433591062":"2024-04-26T14:04:21.307574+0000","172.21.15.96:6800/2348397481":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/3517674924":"2024-04-26T14:03:54.249537+0000","172.21.15.96:6800/433591062":"2024-04-26T14:04:21.307574+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:13:26.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.102+0000 7f03efbb2640 1 -- 172.21.15.96:0/983391090 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f03d40afbf0 msgr2=0x7f03d40b2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.102+0000 7f03efbb2640 1 --2- 172.21.15.96:0/983391090 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f03d40afbf0 0x7f03d40b2100 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f03e81035c0 tx=0x7f03d8005f90 comp rx=0 tx=0).stop 2024-04-25T14:13:26.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.102+0000 7f03efbb2640 1 -- 172.21.15.96:0/983391090 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03e8108900 msgr2=0x7f03e81a1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.102+0000 7f03efbb2640 1 --2- 172.21.15.96:0/983391090 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03e8108900 0x7f03e81a1700 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f03e000cce0 tx=0x7f03e000a620 comp rx=0 tx=0).stop 2024-04-25T14:13:26.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.102+0000 7f03efbb2640 1 -- 172.21.15.96:0/983391090 shutdown_connections 2024-04-25T14:13:26.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.102+0000 7f03efbb2640 1 --2- 172.21.15.96:0/983391090 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f03d40afbf0 0x7f03d40b2100 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.102+0000 7f03efbb2640 1 --2- 172.21.15.96:0/983391090 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03e8108900 0x7f03e81a1700 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.102+0000 7f03efbb2640 1 --2- 172.21.15.96:0/983391090 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03e8103c00 0x7f03e819a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.102+0000 7f03efbb2640 1 --2- 172.21.15.96:0/983391090 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03e81029c0 0x7f03e819a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.102+0000 7f03efbb2640 1 -- 172.21.15.96:0/983391090 >> 172.21.15.96:0/983391090 conn(0x7f03e80fdbd0 msgr2=0x7f03e81048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:26.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.102+0000 7f03efbb2640 1 -- 172.21.15.96:0/983391090 shutdown_connections 2024-04-25T14:13:26.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.102+0000 7f03efbb2640 1 -- 172.21.15.96:0/983391090 wait complete. 2024-04-25T14:13:26.114 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T14:11:28.886961+0000', 'modified': '2024-04-25T14:11:35.532775+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24536, 'mds_1': 24517, 'mds_2': 24478, 'mds_3': 24455, 'mds_4': 24427}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14640': {'gid': 14640, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.96:6839/4080308509', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6838', 'nonce': 4080308509}, {'type': 'v1', 'addr': '172.21.15.96:6839', 'nonce': 4080308509}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14652': {'gid': 14652, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.96:6841/1606748971', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6840', 'nonce': 1606748971}, {'type': 'v1', 'addr': '172.21.15.96:6841', 'nonce': 1606748971}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24419': {'gid': 24419, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.97:6833/1995324831', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6832', 'nonce': 1995324831}, {'type': 'v1', 'addr': '172.21.15.97:6833', 'nonce': 1995324831}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24427': {'gid': 24427, 'name': 'l', 'rank': 4, 'incarnation': 23, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.151:6833/1382721608', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6832', 'nonce': 1382721608}, {'type': 'v1', 'addr': '172.21.15.151:6833', 'nonce': 1382721608}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'e', 'rank': 3, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.97:6835/3872301726', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6834', 'nonce': 3872301726}, {'type': 'v1', 'addr': '172.21.15.97:6835', 'nonce': 3872301726}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'f', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.151:6835/1313823638', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6834', 'nonce': 1313823638}, {'type': 'v1', 'addr': '172.21.15.151:6835', 'nonce': 1313823638}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24490': {'gid': 24490, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.151:6837/2691348366', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6836', 'nonce': 2691348366}, {'type': 'v1', 'addr': '172.21.15.151:6837', 'nonce': 2691348366}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24497': {'gid': 24497, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.97:6837/3306780081', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6836', 'nonce': 3306780081}, {'type': 'v1', 'addr': '172.21.15.97:6837', 'nonce': 3306780081}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24517': {'gid': 24517, 'name': 'i', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.151:6839/245699981', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6838', 'nonce': 245699981}, {'type': 'v1', 'addr': '172.21.15.151:6839', 'nonce': 245699981}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24536': {'gid': 24536, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.97:6839/2850442269', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6838', 'nonce': 2850442269}, {'type': 'v1', 'addr': '172.21.15.97:6839', 'nonce': 2850442269}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24536, 'qdb_cluster': [24517, 24536, 24478, 24455, 24427]} 2024-04-25T14:13:26.114 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-25T14:13:26.115 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2024-04-25T14:13:26.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.223+0000 7f874fd6c640 1 -- 172.21.15.96:0/3525315285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8748103af0 msgr2=0x7f87481047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.223+0000 7f874fd6c640 1 --2- 172.21.15.96:0/3525315285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8748103af0 0x7f87481047f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8744009900 tx=0x7f874402a410 comp rx=0 tx=0).stop 2024-04-25T14:13:26.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.223+0000 7f874fd6c640 1 -- 172.21.15.96:0/3525315285 shutdown_connections 2024-04-25T14:13:26.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.223+0000 7f874fd6c640 1 --2- 172.21.15.96:0/3525315285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8748109a30 0x7f874810be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.223+0000 7f874fd6c640 1 --2- 172.21.15.96:0/3525315285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8748104d30 0x7f8748105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.223+0000 7f874fd6c640 1 --2- 172.21.15.96:0/3525315285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8748103af0 0x7f87481047f0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.223+0000 7f874fd6c640 1 -- 172.21.15.96:0/3525315285 >> 172.21.15.96:0/3525315285 conn(0x7f87480fece0 msgr2=0x7f8748101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:26.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.223+0000 7f874fd6c640 1 -- 172.21.15.96:0/3525315285 shutdown_connections 2024-04-25T14:13:26.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.223+0000 7f874fd6c640 1 -- 172.21.15.96:0/3525315285 wait complete. 2024-04-25T14:13:26.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.224+0000 7f874fd6c640 1 Processor -- start 2024-04-25T14:13:26.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.224+0000 7f874fd6c640 1 -- start start 2024-04-25T14:13:26.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.224+0000 7f874fd6c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8748103af0 0x7f874819a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.224+0000 7f874fd6c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8748104d30 0x7f874819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.224+0000 7f874fd6c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8748109a30 0x7f87481a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.224+0000 7f874fd6c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f874810f5f0 con 0x7f8748103af0 2024-04-25T14:13:26.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.224+0000 7f874fd6c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f874810f470 con 0x7f8748109a30 2024-04-25T14:13:26.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.224+0000 7f874fd6c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f874810f770 con 0x7f8748104d30 2024-04-25T14:13:26.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.224+0000 7f874db08640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8748103af0 0x7f874819a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.224+0000 7f874db08640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8748103af0 0x7f874819a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51592/0 (socket says 172.21.15.96:51592) 2024-04-25T14:13:26.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.224+0000 7f874db08640 1 -- 172.21.15.96:0/3686494576 learned_addr learned my addr 172.21.15.96:0/3686494576 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:26.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.224+0000 7f874e309640 1 --2- 172.21.15.96:0/3686494576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8748109a30 0x7f87481a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.224+0000 7f874d307640 1 --2- 172.21.15.96:0/3686494576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8748104d30 0x7f874819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.225+0000 7f874db08640 1 -- 172.21.15.96:0/3686494576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8748104d30 msgr2=0x7f874819a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.225+0000 7f874db08640 1 --2- 172.21.15.96:0/3686494576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8748104d30 0x7f874819a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.225+0000 7f874db08640 1 -- 172.21.15.96:0/3686494576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8748109a30 msgr2=0x7f87481a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.225+0000 7f874db08640 1 --2- 172.21.15.96:0/3686494576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8748109a30 0x7f87481a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.225+0000 7f874db08640 1 -- 172.21.15.96:0/3686494576 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87481a1d70 con 0x7f8748103af0 2024-04-25T14:13:26.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.225+0000 7f874d307640 1 --2- 172.21.15.96:0/3686494576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8748104d30 0x7f874819a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:26.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.225+0000 7f874db08640 1 --2- 172.21.15.96:0/3686494576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8748103af0 0x7f874819a230 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8744006010 tx=0x7f8744031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:26.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.225+0000 7f873effd640 1 -- 172.21.15.96:0/3686494576 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874403f030 con 0x7f8748103af0 2024-04-25T14:13:26.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.225+0000 7f873effd640 1 -- 172.21.15.96:0/3686494576 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8744003aa0 con 0x7f8748103af0 2024-04-25T14:13:26.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.225+0000 7f874fd6c640 1 -- 172.21.15.96:0/3686494576 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87481a2000 con 0x7f8748103af0 2024-04-25T14:13:26.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.225+0000 7f873effd640 1 -- 172.21.15.96:0/3686494576 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8744035070 con 0x7f8748103af0 2024-04-25T14:13:26.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.225+0000 7f874fd6c640 1 -- 172.21.15.96:0/3686494576 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87481a24f0 con 0x7f8748103af0 2024-04-25T14:13:26.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.227+0000 7f873effd640 1 -- 172.21.15.96:0/3686494576 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f8744042030 con 0x7f8748103af0 2024-04-25T14:13:26.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.228+0000 7f8756c19740 1 Processor -- start 2024-04-25T14:13:26.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.228+0000 7f8756c19740 1 -- start start 2024-04-25T14:13:26.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.228+0000 7f8756c19740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6d9007270 0x55c6d93a8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.228+0000 7f8756c19740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6d8fde3b0 0x55c6d93a88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.228+0000 7f8756c19740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6d8fe6590 0x55c6d93af7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.228+0000 7f8756c19740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c6d930d740 con 0x55c6d8fde3b0 2024-04-25T14:13:26.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.228+0000 7f8756c19740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c6d92a93a0 con 0x55c6d8fe6590 2024-04-25T14:13:26.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.228+0000 7f8756c19740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c6d9309700 con 0x55c6d9007270 2024-04-25T14:13:26.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.228+0000 7f874db08640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6d8fde3b0 0x55c6d93a88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.228+0000 7f874db08640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6d8fde3b0 0x55c6d93a88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51600/0 (socket says 172.21.15.96:51600) 2024-04-25T14:13:26.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.228+0000 7f873effd640 1 --2- 172.21.15.96:0/3686494576 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f87340afaf0 0x7f87340b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.228+0000 7f874db08640 1 -- 172.21.15.96:0/2736711941 learned_addr learned my addr 172.21.15.96:0/2736711941 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:26.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.228+0000 7f874e309640 1 --2- 172.21.15.96:0/2736711941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6d9007270 0x55c6d93a8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.228+0000 7f874e309640 1 --2- 172.21.15.96:0/3686494576 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f87340afaf0 0x7f87340b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.228+0000 7f874d307640 1 --2- 172.21.15.96:0/2736711941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6d8fe6590 0x55c6d93af7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.229+0000 7f873effd640 1 -- 172.21.15.96:0/3686494576 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f87440ff1d0 con 0x7f8748103af0 2024-04-25T14:13:26.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.229+0000 7f874db08640 1 -- 172.21.15.96:0/2736711941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6d9007270 msgr2=0x55c6d93a8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:26.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.229+0000 7f874db08640 1 --2- 172.21.15.96:0/2736711941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6d9007270 0x55c6d93a8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.229+0000 7f874db08640 1 -- 172.21.15.96:0/2736711941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6d8fe6590 msgr2=0x55c6d93af7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:26.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.229+0000 7f874db08640 1 --2- 172.21.15.96:0/2736711941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6d8fe6590 0x55c6d93af7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.229+0000 7f874db08640 1 -- 172.21.15.96:0/2736711941 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6d92c5750 con 0x55c6d8fde3b0 2024-04-25T14:13:26.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.229+0000 7f874e309640 1 --2- 172.21.15.96:0/3686494576 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f87340afaf0 0x7f87340b2000 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8740009fd0 tx=0x7f8740009b10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:26.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.229+0000 7f874db08640 1 --2- 172.21.15.96:0/2736711941 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6d8fde3b0 0x55c6d93a88f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f87440c4900 tx=0x7f8744003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d1045dbfa748787 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:26.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.229+0000 7f8733fff640 1 -- 172.21.15.96:0/2736711941 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87440fe030 con 0x55c6d8fde3b0 2024-04-25T14:13:26.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.229+0000 7f8733fff640 1 -- 172.21.15.96:0/2736711941 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87440c7d90 con 0x55c6d8fde3b0 2024-04-25T14:13:26.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.229+0000 7f8733fff640 1 -- 172.21.15.96:0/2736711941 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8744049070 con 0x55c6d8fde3b0 2024-04-25T14:13:26.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.230+0000 7f8756c19740 1 -- 172.21.15.96:0/2736711941 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6d8fde3b0 msgr2=0x55c6d93a88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.230+0000 7f8756c19740 1 --2- 172.21.15.96:0/2736711941 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6d8fde3b0 0x55c6d93a88f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f87440c4900 tx=0x7f8744003f90 comp rx=0 tx=0).stop 2024-04-25T14:13:26.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.230+0000 7f8756c19740 1 -- 172.21.15.96:0/2736711941 shutdown_connections 2024-04-25T14:13:26.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.230+0000 7f8756c19740 1 --2- 172.21.15.96:0/2736711941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6d8fe6590 0x55c6d93af7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.230+0000 7f8756c19740 1 --2- 172.21.15.96:0/2736711941 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6d8fde3b0 0x55c6d93a88f0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.230+0000 7f8756c19740 1 --2- 172.21.15.96:0/2736711941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6d9007270 0x55c6d93a8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.230+0000 7f8756c19740 1 -- 172.21.15.96:0/2736711941 >> 172.21.15.96:0/2736711941 conn(0x55c6d8ffb3c0 msgr2=0x55c6d93a3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:26.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.230+0000 7f8756c19740 1 -- 172.21.15.96:0/2736711941 shutdown_connections 2024-04-25T14:13:26.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.230+0000 7f8756c19740 1 -- 172.21.15.96:0/2736711941 wait complete. 2024-04-25T14:13:26.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.230+0000 7f8756c19740 1 -- start start 2024-04-25T14:13:26.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.230+0000 7f8756c19740 1 Processor -- start 2024-04-25T14:13:26.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.231+0000 7f8756c19740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6d8fde3b0 0x55c6d93b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.231+0000 7f8756c19740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6d8fe6590 0x55c6d93b2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.231+0000 7f8756c19740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6d9007270 0x55c6d93b9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.231+0000 7f8756c19740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c6d930d740 con 0x55c6d8fe6590 2024-04-25T14:13:26.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.231+0000 7f8756c19740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c6d9294fe0 con 0x55c6d8fde3b0 2024-04-25T14:13:26.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.231+0000 7f8756c19740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c6d93b0000 con 0x55c6d9007270 2024-04-25T14:13:26.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.231+0000 7f8756c19740 10 client.0 _ll_register_callbacks cb 0x55c6d92ab150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:13:26.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.231+0000 7f874d307640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6d8fe6590 0x55c6d93b2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.231+0000 7f874d307640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6d8fe6590 0x55c6d93b2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51602/0 (socket says 172.21.15.96:51602) 2024-04-25T14:13:26.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.231+0000 7f874d307640 1 -- 172.21.15.96:0/1288962496 learned_addr learned my addr 172.21.15.96:0/1288962496 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:26.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.231+0000 7f874d307640 1 -- 172.21.15.96:0/1288962496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6d9007270 msgr2=0x55c6d93b9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:26.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.231+0000 7f874db08640 1 --2- 172.21.15.96:0/1288962496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6d8fde3b0 0x55c6d93b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.231+0000 7f874e309640 1 --2- 172.21.15.96:0/1288962496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6d9007270 0x55c6d93b9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.231+0000 7f874d307640 1 --2- 172.21.15.96:0/1288962496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6d9007270 0x55c6d93b9360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.231+0000 7f874d307640 1 -- 172.21.15.96:0/1288962496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6d8fde3b0 msgr2=0x55c6d93b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:26.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.231+0000 7f874d307640 1 --2- 172.21.15.96:0/1288962496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6d8fde3b0 0x55c6d93b1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.231+0000 7f874d307640 1 -- 172.21.15.96:0/1288962496 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6d901f2d0 con 0x55c6d8fe6590 2024-04-25T14:13:26.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.231+0000 7f874db08640 1 --2- 172.21.15.96:0/1288962496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6d8fde3b0 0x55c6d93b1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:26.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.231+0000 7f874d307640 1 --2- 172.21.15.96:0/1288962496 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6d8fe6590 0x55c6d93b2400 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8738011a60 tx=0x7f8738011f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2edd83fee6c073a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:26.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.231+0000 7f87317fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:13:26.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.232+0000 7f87317fa640 1 -- 172.21.15.96:0/1288962496 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873801e060 con 0x55c6d8fe6590 2024-04-25T14:13:26.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.232+0000 7f87317fa640 1 -- 172.21.15.96:0/1288962496 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f87380193f0 con 0x55c6d8fe6590 2024-04-25T14:13:26.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.232+0000 7f8756c19740 1 -- 172.21.15.96:0/1288962496 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87440c7d90 con 0x55c6d8fe6590 2024-04-25T14:13:26.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.232+0000 7f87317fa640 1 -- 172.21.15.96:0/1288962496 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f873800fe70 con 0x55c6d8fe6590 2024-04-25T14:13:26.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.232+0000 7f8756c19740 10 client.15270 fetch_fsmap learned FSMap version 25 2024-04-25T14:13:26.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.232+0000 7f8756c19740 1 -- 172.21.15.96:0/1288962496 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c6d90e96b0 con 0x55c6d8fe6590 2024-04-25T14:13:26.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.232+0000 7f87317fa640 1 -- 172.21.15.96:0/1288962496 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87440c7d90 con 0x55c6d8fe6590 2024-04-25T14:13:26.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.232+0000 7f87317fa640 1 -- 172.21.15.96:0/1288962496 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f8738018030 con 0x55c6d8fe6590 2024-04-25T14:13:26.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.232+0000 7f8756c19740 10 client.15270 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T14:13:26.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.232+0000 7f8756c19740 10 client.15270 resolve_mds: resolved name 'l' to daemon mds.l 2024-04-25T14:13:26.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.232+0000 7f8756c19740 20 client.15270 populate_metadata read hostname 'smithi096' 2024-04-25T14:13:26.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.232+0000 7f8756c19740 1 --2- 172.21.15.96:0/1288962496 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x55c6d8fc5e40 0x55c6d93bfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.232+0000 7f8756c19740 4 client.15270 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:13:26.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.232+0000 7f8756c19740 1 -- 172.21.15.96:0/1288962496 --> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6d931b350 con 0x55c6d8fc5e40 2024-04-25T14:13:26.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.232+0000 7f874db08640 1 --2- 172.21.15.96:0/1288962496 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x55c6d8fc5e40 0x55c6d93bfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.233+0000 7f87317fa640 10 client.15270 ms_handle_connect on v2:172.21.15.151:6832/1382721608 2024-04-25T14:13:26.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.233+0000 7f874db08640 1 --2- 172.21.15.96:0/1288962496 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x55c6d8fc5e40 0x55c6d93bfcd0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f0da886cf2feffb3 server_cookie=1bb6bcef858189cf in_seq=0 out_seq=0 2024-04-25T14:13:26.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.234+0000 7f87317fa640 1 -- 172.21.15.96:0/1288962496 <== mds.4 v2:172.21.15.151:6832/1382721608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c6d931b350 con 0x55c6d8fc5e40 2024-04-25T14:13:26.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.234+0000 7f87317fa640 10 client.15270 handle_command_reply: tid=0 2024-04-25T14:13:26.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.234+0000 7f8756c19740 1 client.15270 shutdown 2024-04-25T14:13:26.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.234+0000 7f8756c19740 1 -- 172.21.15.96:0/1288962496 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6d8fe6590 msgr2=0x55c6d93b2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.234+0000 7f8756c19740 1 --2- 172.21.15.96:0/1288962496 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6d8fe6590 0x55c6d93b2400 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8738011a60 tx=0x7f8738011f20 comp rx=0 tx=0).stop 2024-04-25T14:13:26.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.234+0000 7f8756c19740 1 -- 172.21.15.96:0/1288962496 shutdown_connections 2024-04-25T14:13:26.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.234+0000 7f8756c19740 1 --2- 172.21.15.96:0/1288962496 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x55c6d8fc5e40 0x55c6d93bfcd0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.234+0000 7f8756c19740 1 --2- 172.21.15.96:0/1288962496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6d9007270 0x55c6d93b9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.234+0000 7f8756c19740 1 --2- 172.21.15.96:0/1288962496 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6d8fe6590 0x55c6d93b2400 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.234+0000 7f8756c19740 1 --2- 172.21.15.96:0/1288962496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6d8fde3b0 0x55c6d93b1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.234+0000 7f87317fa640 0 client.15270 ms_handle_reset on v2:172.21.15.151:6832/1382721608 2024-04-25T14:13:26.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.234+0000 7f8756c19740 1 -- 172.21.15.96:0/1288962496 >> 172.21.15.96:0/1288962496 conn(0x55c6d8ffb3c0 msgr2=0x55c6d93a2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:26.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.234+0000 7f8756c19740 1 -- 172.21.15.96:0/1288962496 shutdown_connections 2024-04-25T14:13:26.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.234+0000 7f8756c19740 1 -- 172.21.15.96:0/1288962496 wait complete. 2024-04-25T14:13:26.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.234+0000 7f8756c19740 20 client.15270 trim_cache size 0 max 16384 2024-04-25T14:13:26.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.252+0000 7f8756c19740 1 Processor -- start 2024-04-25T14:13:26.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.252+0000 7f8756c19740 1 -- start start 2024-04-25T14:13:26.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.253+0000 7f8756c19740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6d8fde3b0 0x55c6d939ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.253+0000 7f8756c19740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6d8fe6590 0x55c6d93b9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.253+0000 7f8756c19740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6d9007270 0x55c6d93b1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.253+0000 7f8756c19740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c6d93be010 con 0x55c6d8fde3b0 2024-04-25T14:13:26.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.253+0000 7f8756c19740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c6d93be6e0 con 0x55c6d9007270 2024-04-25T14:13:26.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.253+0000 7f8756c19740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c6d93bedb0 con 0x55c6d8fe6590 2024-04-25T14:13:26.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.253+0000 7f874db08640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6d8fde3b0 0x55c6d939ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.253+0000 7f874db08640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6d8fde3b0 0x55c6d939ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51606/0 (socket says 172.21.15.96:51606) 2024-04-25T14:13:26.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.253+0000 7f874db08640 1 -- 172.21.15.96:0/1705406628 learned_addr learned my addr 172.21.15.96:0/1705406628 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:26.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.253+0000 7f874e309640 1 --2- 172.21.15.96:0/1705406628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6d9007270 0x55c6d93b1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.253+0000 7f874db08640 1 -- 172.21.15.96:0/1705406628 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6d8fe6590 msgr2=0x55c6d93b9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:26.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.253+0000 7f874db08640 1 --2- 172.21.15.96:0/1705406628 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6d8fe6590 0x55c6d93b9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.253+0000 7f874db08640 1 -- 172.21.15.96:0/1705406628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6d9007270 msgr2=0x55c6d93b1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:26.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.253+0000 7f874db08640 1 --2- 172.21.15.96:0/1705406628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6d9007270 0x55c6d93b1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.253+0000 7f874db08640 1 -- 172.21.15.96:0/1705406628 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6d93ab740 con 0x55c6d8fde3b0 2024-04-25T14:13:26.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.253+0000 7f874e309640 1 --2- 172.21.15.96:0/1705406628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6d9007270 0x55c6d93b1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:26.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.253+0000 7f874db08640 1 --2- 172.21.15.96:0/1705406628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6d8fde3b0 0x55c6d939ea20 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f87440c7620 tx=0x7f874402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7057ce34e7e875cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:26.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.253+0000 7f87337fe640 1 -- 172.21.15.96:0/1705406628 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87440fe040 con 0x55c6d8fde3b0 2024-04-25T14:13:26.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.253+0000 7f87337fe640 1 -- 172.21.15.96:0/1705406628 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87440c7690 con 0x55c6d8fde3b0 2024-04-25T14:13:26.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.253+0000 7f87337fe640 1 -- 172.21.15.96:0/1705406628 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8744049040 con 0x55c6d8fde3b0 2024-04-25T14:13:26.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.254+0000 7f8756c19740 1 -- 172.21.15.96:0/1705406628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6d8fde3b0 msgr2=0x55c6d939ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.254+0000 7f8756c19740 1 --2- 172.21.15.96:0/1705406628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6d8fde3b0 0x55c6d939ea20 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f87440c7620 tx=0x7f874402a410 comp rx=0 tx=0).stop 2024-04-25T14:13:26.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.254+0000 7f8756c19740 1 -- 172.21.15.96:0/1705406628 shutdown_connections 2024-04-25T14:13:26.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.254+0000 7f8756c19740 1 --2- 172.21.15.96:0/1705406628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6d9007270 0x55c6d93b1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.254+0000 7f8756c19740 1 --2- 172.21.15.96:0/1705406628 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6d8fe6590 0x55c6d93b9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.254+0000 7f8756c19740 1 --2- 172.21.15.96:0/1705406628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6d8fde3b0 0x55c6d939ea20 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.254+0000 7f8756c19740 1 -- 172.21.15.96:0/1705406628 >> 172.21.15.96:0/1705406628 conn(0x55c6d8ffb3c0 msgr2=0x55c6d93ab970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:26.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.254+0000 7f8756c19740 1 -- 172.21.15.96:0/1705406628 shutdown_connections 2024-04-25T14:13:26.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.254+0000 7f8756c19740 1 -- 172.21.15.96:0/1705406628 wait complete. 2024-04-25T14:13:26.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.254+0000 7f8756c19740 1 -- start start 2024-04-25T14:13:26.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.254+0000 7f8756c19740 1 Processor -- start 2024-04-25T14:13:26.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.255+0000 7f8756c19740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6d8fde3b0 0x55c6d93b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.255+0000 7f8756c19740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6d8fe6590 0x55c6d93b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.255+0000 7f8756c19740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6d9007270 0x55c6d93b7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.255+0000 7f8756c19740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c6d93bfb50 con 0x55c6d8fde3b0 2024-04-25T14:13:26.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.255+0000 7f8756c19740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c6d93bf480 con 0x55c6d8fe6590 2024-04-25T14:13:26.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.255+0000 7f8756c19740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c6d93c0220 con 0x55c6d9007270 2024-04-25T14:13:26.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.255+0000 7f8756c19740 10 client.0 _ll_register_callbacks cb 0x7f873800c830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:13:26.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.255+0000 7f874db08640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6d8fde3b0 0x55c6d93b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.255+0000 7f874db08640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6d8fde3b0 0x55c6d93b47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51610/0 (socket says 172.21.15.96:51610) 2024-04-25T14:13:26.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.255+0000 7f874db08640 1 -- 172.21.15.96:0/1714612114 learned_addr learned my addr 172.21.15.96:0/1714612114 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:26.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.255+0000 7f874e309640 1 --2- 172.21.15.96:0/1714612114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6d9007270 0x55c6d93b7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.255+0000 7f874db08640 1 -- 172.21.15.96:0/1714612114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6d9007270 msgr2=0x55c6d93b7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:26.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.255+0000 7f874db08640 1 --2- 172.21.15.96:0/1714612114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6d9007270 0x55c6d93b7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.255+0000 7f874db08640 1 -- 172.21.15.96:0/1714612114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6d8fe6590 msgr2=0x55c6d93b4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:26.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.255+0000 7f874db08640 1 --2- 172.21.15.96:0/1714612114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6d8fe6590 0x55c6d93b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.255+0000 7f874db08640 1 -- 172.21.15.96:0/1714612114 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6d93ad0c0 con 0x55c6d8fde3b0 2024-04-25T14:13:26.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.255+0000 7f874e309640 1 --2- 172.21.15.96:0/1714612114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6d9007270 0x55c6d93b7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:26.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.255+0000 7f874db08640 1 --2- 172.21.15.96:0/1714612114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6d8fde3b0 0x55c6d93b47c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f87440fd020 tx=0x7f8744009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1a979a87442e2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:26.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.255+0000 7f87317fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:13:26.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.256+0000 7f87317fa640 1 -- 172.21.15.96:0/1714612114 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87440fe070 con 0x55c6d8fde3b0 2024-04-25T14:13:26.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.256+0000 7f87317fa640 1 -- 172.21.15.96:0/1714612114 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f8744003c50 con 0x55c6d8fde3b0 2024-04-25T14:13:26.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.256+0000 7f8756c19740 1 -- 172.21.15.96:0/1714612114 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87440c7690 con 0x55c6d8fde3b0 2024-04-25T14:13:26.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.256+0000 7f87317fa640 1 -- 172.21.15.96:0/1714612114 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8744049030 con 0x55c6d8fde3b0 2024-04-25T14:13:26.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.256+0000 7f8756c19740 10 client.15276 fetch_fsmap learned FSMap version 25 2024-04-25T14:13:26.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.256+0000 7f8756c19740 1 -- 172.21.15.96:0/1714612114 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c6d93b91d0 con 0x55c6d8fde3b0 2024-04-25T14:13:26.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.256+0000 7f87317fa640 1 -- 172.21.15.96:0/1714612114 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87440c7690 con 0x55c6d8fde3b0 2024-04-25T14:13:26.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.256+0000 7f87317fa640 1 -- 172.21.15.96:0/1714612114 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f87440c78f0 con 0x55c6d8fde3b0 2024-04-25T14:13:26.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.256+0000 7f8756c19740 10 client.15276 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T14:13:26.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.256+0000 7f8756c19740 10 client.15276 resolve_mds: resolved name 'l' to daemon mds.l 2024-04-25T14:13:26.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.256+0000 7f8756c19740 20 client.15276 populate_metadata read hostname 'smithi096' 2024-04-25T14:13:26.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.256+0000 7f8756c19740 1 --2- 172.21.15.96:0/1714612114 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x55c6d8f9b280 0x55c6d942ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.256+0000 7f8756c19740 4 client.15276 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T14:13:26.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.256+0000 7f8756c19740 1 -- 172.21.15.96:0/1714612114 --> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] -- command(tid 0: {"prefix": "status"}) -- 0x55c6d93151c0 con 0x55c6d8f9b280 2024-04-25T14:13:26.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.256+0000 7f874d307640 1 --2- 172.21.15.96:0/1714612114 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x55c6d8f9b280 0x55c6d942ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.257+0000 7f87317fa640 10 client.15276 ms_handle_connect on v2:172.21.15.151:6832/1382721608 2024-04-25T14:13:26.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.257+0000 7f874d307640 1 --2- 172.21.15.96:0/1714612114 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x55c6d8f9b280 0x55c6d942ba80 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b6b848fa1e9fcee1 server_cookie=e95839cb991798d0 in_seq=0 out_seq=0 2024-04-25T14:13:26.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.257+0000 7f87317fa640 1 -- 172.21.15.96:0/1714612114 <== mds.4 v2:172.21.15.151:6832/1382721608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55c6d93151c0 con 0x55c6d8f9b280 2024-04-25T14:13:26.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.257+0000 7f87317fa640 10 client.15276 handle_command_reply: tid=0 2024-04-25T14:13:26.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.257+0000 7f8756c19740 1 client.15276 shutdown 2024-04-25T14:13:26.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.257+0000 7f8756c19740 1 -- 172.21.15.96:0/1714612114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6d8fde3b0 msgr2=0x55c6d93b47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.257+0000 7f8756c19740 1 --2- 172.21.15.96:0/1714612114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6d8fde3b0 0x55c6d93b47c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f87440fd020 tx=0x7f8744009260 comp rx=0 tx=0).stop 2024-04-25T14:13:26.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.257+0000 7f8756c19740 1 -- 172.21.15.96:0/1714612114 shutdown_connections 2024-04-25T14:13:26.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.257+0000 7f8756c19740 1 --2- 172.21.15.96:0/1714612114 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x55c6d8f9b280 0x55c6d942ba80 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.257+0000 7f8756c19740 1 --2- 172.21.15.96:0/1714612114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6d9007270 0x55c6d93b7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.257+0000 7f87317fa640 0 client.15276 ms_handle_reset on v2:172.21.15.151:6832/1382721608 2024-04-25T14:13:26.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.257+0000 7f8756c19740 1 --2- 172.21.15.96:0/1714612114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6d8fe6590 0x55c6d93b4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.257+0000 7f8756c19740 1 --2- 172.21.15.96:0/1714612114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6d8fde3b0 0x55c6d93b47c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.257+0000 7f8756c19740 1 -- 172.21.15.96:0/1714612114 >> 172.21.15.96:0/1714612114 conn(0x55c6d8ffb3c0 msgr2=0x55c6d939ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:26.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.257+0000 7f8756c19740 1 -- 172.21.15.96:0/1714612114 shutdown_connections 2024-04-25T14:13:26.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.257+0000 7f8756c19740 1 -- 172.21.15.96:0/1714612114 wait complete. 2024-04-25T14:13:26.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.257+0000 7f8756c19740 20 client.15276 trim_cache size 0 max 16384 2024-04-25T14:13:26.264 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T14:13:26.265 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "59334182-030c-11ef-bc93-c7b262605968", 2024-04-25T14:13:26.265 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 4, 2024-04-25T14:13:26.265 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 24427, 2024-04-25T14:13:26.265 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-04-25T14:13:26.265 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-04-25T14:13:26.265 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-04-25T14:13:26.265 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 111.731046788, 2024-04-25T14:13:26.265 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 25, 2024-04-25T14:13:26.265 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 78, 2024-04-25T14:13:26.265 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 76, 2024-04-25T14:13:26.265 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 154.45499786400001 2024-04-25T14:13:26.265 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T14:13:26.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.258+0000 7f874fd6c640 1 -- 172.21.15.96:0/3686494576 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f87340afaf0 msgr2=0x7f87340b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.258+0000 7f874fd6c640 1 --2- 172.21.15.96:0/3686494576 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f87340afaf0 0x7f87340b2000 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8740009fd0 tx=0x7f8740009b10 comp rx=0 tx=0).stop 2024-04-25T14:13:26.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.258+0000 7f874fd6c640 1 -- 172.21.15.96:0/3686494576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8748103af0 msgr2=0x7f874819a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.258+0000 7f874fd6c640 1 --2- 172.21.15.96:0/3686494576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8748103af0 0x7f874819a230 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8744006010 tx=0x7f8744031a50 comp rx=0 tx=0).stop 2024-04-25T14:13:26.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.258+0000 7f874fd6c640 1 -- 172.21.15.96:0/3686494576 shutdown_connections 2024-04-25T14:13:26.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.258+0000 7f874fd6c640 1 --2- 172.21.15.96:0/3686494576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8748109a30 0x7f87481a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.258+0000 7f874fd6c640 1 --2- 172.21.15.96:0/3686494576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8748104d30 0x7f874819a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.258+0000 7f874fd6c640 1 --2- 172.21.15.96:0/3686494576 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f87340afaf0 0x7f87340b2000 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.258+0000 7f874fd6c640 1 --2- 172.21.15.96:0/3686494576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8748103af0 0x7f874819a230 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.258+0000 7f874fd6c640 1 -- 172.21.15.96:0/3686494576 >> 172.21.15.96:0/3686494576 conn(0x7f87480fece0 msgr2=0x7f8748105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:26.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.258+0000 7f874fd6c640 1 -- 172.21.15.96:0/3686494576 shutdown_connections 2024-04-25T14:13:26.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.258+0000 7f874fd6c640 1 -- 172.21.15.96:0/3686494576 wait complete. 2024-04-25T14:13:26.269 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-04-25T14:13:26.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.377+0000 7f98e1985640 1 -- 172.21.15.96:0/44770343 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98dc1029e0 msgr2=0x7f98dc1036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.377+0000 7f98e1985640 1 --2- 172.21.15.96:0/44770343 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98dc1029e0 0x7f98dc1036e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f98cc009900 tx=0x7f98cc02a410 comp rx=0 tx=0).stop 2024-04-25T14:13:26.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.378+0000 7f98e1985640 1 -- 172.21.15.96:0/44770343 shutdown_connections 2024-04-25T14:13:26.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.378+0000 7f98e1985640 1 --2- 172.21.15.96:0/44770343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98dc108920 0x7f98dc10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.378+0000 7f98e1985640 1 --2- 172.21.15.96:0/44770343 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98dc103c20 0x7f98dc104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.378+0000 7f98e1985640 1 --2- 172.21.15.96:0/44770343 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98dc1029e0 0x7f98dc1036e0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.378+0000 7f98e1985640 1 -- 172.21.15.96:0/44770343 >> 172.21.15.96:0/44770343 conn(0x7f98dc0fdbd0 msgr2=0x7f98dc100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:26.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.378+0000 7f98e1985640 1 -- 172.21.15.96:0/44770343 shutdown_connections 2024-04-25T14:13:26.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.378+0000 7f98e1985640 1 -- 172.21.15.96:0/44770343 wait complete. 2024-04-25T14:13:26.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.378+0000 7f98e1985640 1 Processor -- start 2024-04-25T14:13:26.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.378+0000 7f98e1985640 1 -- start start 2024-04-25T14:13:26.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.379+0000 7f98e1985640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98dc1029e0 0x7f98dc06a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.379+0000 7f98e1985640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98dc103c20 0x7f98dc19f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.379+0000 7f98e1985640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98dc108920 0x7f98dc1a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.379+0000 7f98e1985640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f98dc10e4e0 con 0x7f98dc1029e0 2024-04-25T14:13:26.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.379+0000 7f98e1985640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f98dc10e360 con 0x7f98dc103c20 2024-04-25T14:13:26.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.379+0000 7f98e1985640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f98dc10e660 con 0x7f98dc108920 2024-04-25T14:13:26.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.379+0000 7f98daffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98dc1029e0 0x7f98dc06a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.379+0000 7f98daffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98dc1029e0 0x7f98dc06a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51626/0 (socket says 172.21.15.96:51626) 2024-04-25T14:13:26.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.379+0000 7f98daffd640 1 -- 172.21.15.96:0/2216484798 learned_addr learned my addr 172.21.15.96:0/2216484798 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:26.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.379+0000 7f98da7fc640 1 --2- 172.21.15.96:0/2216484798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98dc103c20 0x7f98dc19f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.379+0000 7f98db7fe640 1 --2- 172.21.15.96:0/2216484798 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98dc108920 0x7f98dc1a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.379+0000 7f98daffd640 1 -- 172.21.15.96:0/2216484798 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98dc108920 msgr2=0x7f98dc1a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.379+0000 7f98daffd640 1 --2- 172.21.15.96:0/2216484798 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98dc108920 0x7f98dc1a1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.379+0000 7f98daffd640 1 -- 172.21.15.96:0/2216484798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98dc103c20 msgr2=0x7f98dc19f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.379+0000 7f98daffd640 1 --2- 172.21.15.96:0/2216484798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98dc103c20 0x7f98dc19f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.379+0000 7f98daffd640 1 -- 172.21.15.96:0/2216484798 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98dc1a2000 con 0x7f98dc1029e0 2024-04-25T14:13:26.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.379+0000 7f98da7fc640 1 --2- 172.21.15.96:0/2216484798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98dc103c20 0x7f98dc19f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:26.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.379+0000 7f98db7fe640 1 --2- 172.21.15.96:0/2216484798 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98dc108920 0x7f98dc1a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:26.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.379+0000 7f98daffd640 1 --2- 172.21.15.96:0/2216484798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98dc1029e0 0x7f98dc06a960 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f98cc006010 tx=0x7f98cc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:26.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.379+0000 7f98e0983640 1 -- 172.21.15.96:0/2216484798 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc03f030 con 0x7f98dc1029e0 2024-04-25T14:13:26.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.379+0000 7f98e0983640 1 -- 172.21.15.96:0/2216484798 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98cc003aa0 con 0x7f98dc1029e0 2024-04-25T14:13:26.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.379+0000 7f98e0983640 1 -- 172.21.15.96:0/2216484798 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc035070 con 0x7f98dc1029e0 2024-04-25T14:13:26.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.379+0000 7f98e1985640 1 -- 172.21.15.96:0/2216484798 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98dc1a2290 con 0x7f98dc1029e0 2024-04-25T14:13:26.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.379+0000 7f98e1985640 1 -- 172.21.15.96:0/2216484798 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98dc1a27d0 con 0x7f98dc1029e0 2024-04-25T14:13:26.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.382+0000 7f98e0983640 1 -- 172.21.15.96:0/2216484798 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f98cc042030 con 0x7f98dc1029e0 2024-04-25T14:13:26.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.382+0000 7f98e8831740 1 Processor -- start 2024-04-25T14:13:26.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.382+0000 7f98e8831740 1 -- start start 2024-04-25T14:13:26.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.382+0000 7f98e8831740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599429c8270 0x559942d6d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.382+0000 7f98e8831740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55994299f3b0 0x559942d6dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.382+0000 7f98e8831740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599429a7590 0x559942d74c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.382+0000 7f98e8831740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559942cce740 con 0x55994299f3b0 2024-04-25T14:13:26.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.382+0000 7f98e8831740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559942c6a3a0 con 0x5599429c8270 2024-04-25T14:13:26.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.382+0000 7f98e8831740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559942cca700 con 0x5599429a7590 2024-04-25T14:13:26.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.382+0000 7f98e0983640 1 --2- 172.21.15.96:0/2216484798 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f98c40afb70 0x7f98c40b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.382+0000 7f98daffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55994299f3b0 0x559942d6dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.382+0000 7f98daffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55994299f3b0 0x559942d6dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51632/0 (socket says 172.21.15.96:51632) 2024-04-25T14:13:26.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.382+0000 7f98daffd640 1 -- 172.21.15.96:0/1270712562 learned_addr learned my addr 172.21.15.96:0/1270712562 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:26.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.382+0000 7f98db7fe640 1 --2- 172.21.15.96:0/1270712562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599429c8270 0x559942d6d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.383+0000 7f98db7fe640 1 --2- 172.21.15.96:0/2216484798 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f98c40afb70 0x7f98c40b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.383+0000 7f98da7fc640 1 --2- 172.21.15.96:0/1270712562 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599429a7590 0x559942d74c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.383+0000 7f98e0983640 1 -- 172.21.15.96:0/2216484798 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f98cc0ff0f0 con 0x7f98dc1029e0 2024-04-25T14:13:26.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.383+0000 7f98daffd640 1 -- 172.21.15.96:0/1270712562 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599429a7590 msgr2=0x559942d74c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:26.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.383+0000 7f98daffd640 1 --2- 172.21.15.96:0/1270712562 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599429a7590 0x559942d74c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.383+0000 7f98daffd640 1 -- 172.21.15.96:0/1270712562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599429c8270 msgr2=0x559942d6d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:26.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.383+0000 7f98daffd640 1 --2- 172.21.15.96:0/1270712562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599429c8270 0x559942d6d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.383+0000 7f98daffd640 1 -- 172.21.15.96:0/1270712562 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559942c86750 con 0x55994299f3b0 2024-04-25T14:13:26.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.383+0000 7f98daffd640 1 --2- 172.21.15.96:0/1270712562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55994299f3b0 0x559942d6dd00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f98cc0c3d70 tx=0x7f98cc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de6c233d214e1ceb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:26.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.383+0000 7f98db7fe640 1 --2- 172.21.15.96:0/2216484798 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f98c40afb70 0x7f98c40b2080 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x559942d6e950 tx=0x7f98d4005f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:26.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.383+0000 7f98c1ffb640 1 -- 172.21.15.96:0/1270712562 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc049070 con 0x55994299f3b0 2024-04-25T14:13:26.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.383+0000 7f98c1ffb640 1 -- 172.21.15.96:0/1270712562 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98cc0c70c0 con 0x55994299f3b0 2024-04-25T14:13:26.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.383+0000 7f98c1ffb640 1 -- 172.21.15.96:0/1270712562 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc0c3840 con 0x55994299f3b0 2024-04-25T14:13:26.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.384+0000 7f98e8831740 1 -- 172.21.15.96:0/1270712562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55994299f3b0 msgr2=0x559942d6dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.384+0000 7f98e8831740 1 --2- 172.21.15.96:0/1270712562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55994299f3b0 0x559942d6dd00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f98cc0c3d70 tx=0x7f98cc009260 comp rx=0 tx=0).stop 2024-04-25T14:13:26.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.384+0000 7f98e8831740 1 -- 172.21.15.96:0/1270712562 shutdown_connections 2024-04-25T14:13:26.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.384+0000 7f98e8831740 1 --2- 172.21.15.96:0/1270712562 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599429a7590 0x559942d74c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.384+0000 7f98e8831740 1 --2- 172.21.15.96:0/1270712562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55994299f3b0 0x559942d6dd00 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.384+0000 7f98e8831740 1 --2- 172.21.15.96:0/1270712562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599429c8270 0x559942d6d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.384+0000 7f98e8831740 1 -- 172.21.15.96:0/1270712562 >> 172.21.15.96:0/1270712562 conn(0x5599429bc3c0 msgr2=0x559942d64d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:26.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.384+0000 7f98e8831740 1 -- 172.21.15.96:0/1270712562 shutdown_connections 2024-04-25T14:13:26.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.384+0000 7f98e8831740 1 -- 172.21.15.96:0/1270712562 wait complete. 2024-04-25T14:13:26.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.384+0000 7f98e8831740 1 -- start start 2024-04-25T14:13:26.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.385+0000 7f98e8831740 1 Processor -- start 2024-04-25T14:13:26.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.385+0000 7f98e8831740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55994299f3b0 0x559942d77340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.385+0000 7f98e8831740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599429a7590 0x559942d77880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.385+0000 7f98e8831740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599429c8270 0x559942d7e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.385+0000 7f98e8831740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559942cce740 con 0x5599429a7590 2024-04-25T14:13:26.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.385+0000 7f98e8831740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559942c55fe0 con 0x5599429c8270 2024-04-25T14:13:26.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.385+0000 7f98e8831740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559942d75410 con 0x55994299f3b0 2024-04-25T14:13:26.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.385+0000 7f98e8831740 10 client.0 _ll_register_callbacks cb 0x559942c6c150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:13:26.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.385+0000 7f98da7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599429a7590 0x559942d77880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.385+0000 7f98da7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599429a7590 0x559942d77880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51640/0 (socket says 172.21.15.96:51640) 2024-04-25T14:13:26.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.385+0000 7f98da7fc640 1 -- 172.21.15.96:0/57634683 learned_addr learned my addr 172.21.15.96:0/57634683 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:26.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.385+0000 7f98daffd640 1 --2- 172.21.15.96:0/57634683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55994299f3b0 0x559942d77340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.385+0000 7f98db7fe640 1 --2- 172.21.15.96:0/57634683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599429c8270 0x559942d7e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.385+0000 7f98da7fc640 1 -- 172.21.15.96:0/57634683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55994299f3b0 msgr2=0x559942d77340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:26.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.385+0000 7f98da7fc640 1 --2- 172.21.15.96:0/57634683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55994299f3b0 0x559942d77340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.385+0000 7f98da7fc640 1 -- 172.21.15.96:0/57634683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599429c8270 msgr2=0x559942d7e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:26.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.385+0000 7f98da7fc640 1 --2- 172.21.15.96:0/57634683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599429c8270 0x559942d7e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.385+0000 7f98da7fc640 1 -- 172.21.15.96:0/57634683 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599429e02d0 con 0x5599429a7590 2024-04-25T14:13:26.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.385+0000 7f98daffd640 1 --2- 172.21.15.96:0/57634683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55994299f3b0 0x559942d77340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:26.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.385+0000 7f98da7fc640 1 --2- 172.21.15.96:0/57634683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599429a7590 0x559942d77880 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f98d00136e0 tx=0x7f98d0013ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce89f5364dcba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:26.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.385+0000 7f98baffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:13:26.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.385+0000 7f98baffd640 1 -- 172.21.15.96:0/57634683 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d0011dd0 con 0x5599429a7590 2024-04-25T14:13:26.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.385+0000 7f98baffd640 1 -- 172.21.15.96:0/57634683 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f98d001b3e0 con 0x5599429a7590 2024-04-25T14:13:26.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.386+0000 7f98e8831740 1 -- 172.21.15.96:0/57634683 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98cc0c70c0 con 0x5599429a7590 2024-04-25T14:13:26.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.386+0000 7f98baffd640 1 -- 172.21.15.96:0/57634683 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f98d001b5e0 con 0x5599429a7590 2024-04-25T14:13:26.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.386+0000 7f98e8831740 10 client.15288 fetch_fsmap learned FSMap version 25 2024-04-25T14:13:26.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.386+0000 7f98e8831740 1 -- 172.21.15.96:0/57634683 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559942aaa6b0 con 0x5599429a7590 2024-04-25T14:13:26.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.386+0000 7f98baffd640 1 -- 172.21.15.96:0/57634683 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98cc0c70c0 con 0x5599429a7590 2024-04-25T14:13:26.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.386+0000 7f98baffd640 1 -- 172.21.15.96:0/57634683 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f98d001a070 con 0x5599429a7590 2024-04-25T14:13:26.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.386+0000 7f98e8831740 10 client.15288 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T14:13:26.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.386+0000 7f98e8831740 10 client.15288 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-25T14:13:26.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.386+0000 7f98e8831740 20 client.15288 populate_metadata read hostname 'smithi096' 2024-04-25T14:13:26.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.386+0000 7f98e8831740 1 --2- 172.21.15.96:0/57634683 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x559942986e40 0x559942d85100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.386+0000 7f98e8831740 4 client.15288 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:13:26.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.386+0000 7f98e8831740 1 -- 172.21.15.96:0/57634683 --> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559942cdc350 con 0x559942986e40 2024-04-25T14:13:26.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.386+0000 7f98daffd640 1 --2- 172.21.15.96:0/57634683 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x559942986e40 0x559942d85100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.387+0000 7f98baffd640 10 client.15288 ms_handle_connect on v2:172.21.15.97:6834/3872301726 2024-04-25T14:13:26.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.387+0000 7f98daffd640 1 --2- 172.21.15.96:0/57634683 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x559942986e40 0x559942d85100 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bdf456c0f108a6d3 server_cookie=a7d59d8c5360e7d0 in_seq=0 out_seq=0 2024-04-25T14:13:26.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.388+0000 7f98baffd640 1 -- 172.21.15.96:0/57634683 <== mds.3 v2:172.21.15.97:6834/3872301726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559942cdc350 con 0x559942986e40 2024-04-25T14:13:26.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.388+0000 7f98baffd640 10 client.15288 handle_command_reply: tid=0 2024-04-25T14:13:26.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.388+0000 7f98e8831740 1 client.15288 shutdown 2024-04-25T14:13:26.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.388+0000 7f98e8831740 1 -- 172.21.15.96:0/57634683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599429a7590 msgr2=0x559942d77880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.388+0000 7f98e8831740 1 --2- 172.21.15.96:0/57634683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599429a7590 0x559942d77880 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f98d00136e0 tx=0x7f98d0013ba0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.388+0000 7f98e8831740 1 -- 172.21.15.96:0/57634683 shutdown_connections 2024-04-25T14:13:26.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.388+0000 7f98e8831740 1 --2- 172.21.15.96:0/57634683 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x559942986e40 0x559942d85100 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.388+0000 7f98e8831740 1 --2- 172.21.15.96:0/57634683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599429c8270 0x559942d7e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.388+0000 7f98baffd640 0 client.15288 ms_handle_reset on v2:172.21.15.97:6834/3872301726 2024-04-25T14:13:26.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.388+0000 7f98e8831740 1 --2- 172.21.15.96:0/57634683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599429a7590 0x559942d77880 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.388+0000 7f98e8831740 1 --2- 172.21.15.96:0/57634683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55994299f3b0 0x559942d77340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.388+0000 7f98e8831740 1 -- 172.21.15.96:0/57634683 >> 172.21.15.96:0/57634683 conn(0x5599429bc3c0 msgr2=0x559942d63d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:26.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.388+0000 7f98e8831740 1 -- 172.21.15.96:0/57634683 shutdown_connections 2024-04-25T14:13:26.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.388+0000 7f98e8831740 1 -- 172.21.15.96:0/57634683 wait complete. 2024-04-25T14:13:26.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.388+0000 7f98e8831740 20 client.15288 trim_cache size 0 max 16384 2024-04-25T14:13:26.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.406+0000 7f98e8831740 1 Processor -- start 2024-04-25T14:13:26.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.406+0000 7f98e8831740 1 -- start start 2024-04-25T14:13:26.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.406+0000 7f98e8831740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55994299f3b0 0x559942d5fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.406+0000 7f98e8831740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599429a7590 0x559942d7e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.406+0000 7f98e8831740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599429c8270 0x559942d7ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.407+0000 7f98e8831740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559942d83440 con 0x55994299f3b0 2024-04-25T14:13:26.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.407+0000 7f98e8831740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559942d83b10 con 0x5599429a7590 2024-04-25T14:13:26.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.407+0000 7f98e8831740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559942d841e0 con 0x5599429c8270 2024-04-25T14:13:26.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.407+0000 7f98daffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55994299f3b0 0x559942d5fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.407+0000 7f98daffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55994299f3b0 0x559942d5fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51644/0 (socket says 172.21.15.96:51644) 2024-04-25T14:13:26.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.407+0000 7f98daffd640 1 -- 172.21.15.96:0/1421141089 learned_addr learned my addr 172.21.15.96:0/1421141089 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:26.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.407+0000 7f98db7fe640 1 --2- 172.21.15.96:0/1421141089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599429c8270 0x559942d7ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.407+0000 7f98daffd640 1 -- 172.21.15.96:0/1421141089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599429c8270 msgr2=0x559942d7ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:26.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.407+0000 7f98daffd640 1 --2- 172.21.15.96:0/1421141089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599429c8270 0x559942d7ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.407+0000 7f98daffd640 1 -- 172.21.15.96:0/1421141089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599429a7590 msgr2=0x559942d7e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:26.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.407+0000 7f98daffd640 1 --2- 172.21.15.96:0/1421141089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599429a7590 0x559942d7e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.407+0000 7f98daffd640 1 -- 172.21.15.96:0/1421141089 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559942d725b0 con 0x55994299f3b0 2024-04-25T14:13:26.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.407+0000 7f98db7fe640 1 --2- 172.21.15.96:0/1421141089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599429c8270 0x559942d7ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:26.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.407+0000 7f98daffd640 1 --2- 172.21.15.96:0/1421141089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55994299f3b0 0x559942d5fa20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f98cc0c3c40 tx=0x7f98cc0ea040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca814b3cb318adc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:26.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.407+0000 7f98c17fa640 1 -- 172.21.15.96:0/1421141089 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc049040 con 0x55994299f3b0 2024-04-25T14:13:26.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.407+0000 7f98c17fa640 1 -- 172.21.15.96:0/1421141089 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98cc0c7cb0 con 0x55994299f3b0 2024-04-25T14:13:26.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.407+0000 7f98c17fa640 1 -- 172.21.15.96:0/1421141089 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc0c3840 con 0x55994299f3b0 2024-04-25T14:13:26.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.408+0000 7f98e8831740 1 -- 172.21.15.96:0/1421141089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55994299f3b0 msgr2=0x559942d5fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.408+0000 7f98e8831740 1 --2- 172.21.15.96:0/1421141089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55994299f3b0 0x559942d5fa20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f98cc0c3c40 tx=0x7f98cc0ea040 comp rx=0 tx=0).stop 2024-04-25T14:13:26.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.408+0000 7f98e8831740 1 -- 172.21.15.96:0/1421141089 shutdown_connections 2024-04-25T14:13:26.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.408+0000 7f98e8831740 1 --2- 172.21.15.96:0/1421141089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599429c8270 0x559942d7ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.408+0000 7f98e8831740 1 --2- 172.21.15.96:0/1421141089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599429a7590 0x559942d7e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.408+0000 7f98e8831740 1 --2- 172.21.15.96:0/1421141089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55994299f3b0 0x559942d5fa20 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.408+0000 7f98e8831740 1 -- 172.21.15.96:0/1421141089 >> 172.21.15.96:0/1421141089 conn(0x5599429bc3c0 msgr2=0x559942d772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:26.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.408+0000 7f98e8831740 1 -- 172.21.15.96:0/1421141089 shutdown_connections 2024-04-25T14:13:26.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.408+0000 7f98e8831740 1 -- 172.21.15.96:0/1421141089 wait complete. 2024-04-25T14:13:26.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.408+0000 7f98e8831740 1 -- start start 2024-04-25T14:13:26.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.408+0000 7f98e8831740 1 Processor -- start 2024-04-25T14:13:26.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.409+0000 7f98e8831740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55994299f3b0 0x559942d7a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.409+0000 7f98e8831740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599429a7590 0x559942d7aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.409+0000 7f98e8831740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599429c8270 0x559942d7d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.409+0000 7f98e8831740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559942d84f80 con 0x55994299f3b0 2024-04-25T14:13:26.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.409+0000 7f98e8831740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559942d848b0 con 0x5599429a7590 2024-04-25T14:13:26.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.409+0000 7f98e8831740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559942d85650 con 0x5599429c8270 2024-04-25T14:13:26.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.409+0000 7f98e8831740 10 client.0 _ll_register_callbacks cb 0x7f98d0005850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:13:26.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.409+0000 7f98daffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55994299f3b0 0x559942d7a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.409+0000 7f98daffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55994299f3b0 0x559942d7a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51648/0 (socket says 172.21.15.96:51648) 2024-04-25T14:13:26.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.409+0000 7f98daffd640 1 -- 172.21.15.96:0/224123343 learned_addr learned my addr 172.21.15.96:0/224123343 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:26.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.409+0000 7f98db7fe640 1 --2- 172.21.15.96:0/224123343 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599429c8270 0x559942d7d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.409+0000 7f98da7fc640 1 --2- 172.21.15.96:0/224123343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599429a7590 0x559942d7aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.409+0000 7f98daffd640 1 -- 172.21.15.96:0/224123343 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599429c8270 msgr2=0x559942d7d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:26.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.409+0000 7f98daffd640 1 --2- 172.21.15.96:0/224123343 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599429c8270 0x559942d7d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.409+0000 7f98daffd640 1 -- 172.21.15.96:0/224123343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599429a7590 msgr2=0x559942d7aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:26.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.409+0000 7f98daffd640 1 --2- 172.21.15.96:0/224123343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599429a7590 0x559942d7aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.409+0000 7f98daffd640 1 -- 172.21.15.96:0/224123343 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559942d7df60 con 0x55994299f3b0 2024-04-25T14:13:26.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.409+0000 7f98db7fe640 1 --2- 172.21.15.96:0/224123343 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599429c8270 0x559942d7d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:26.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.409+0000 7f98da7fc640 1 --2- 172.21.15.96:0/224123343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599429a7590 0x559942d7aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:26.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.409+0000 7f98daffd640 1 --2- 172.21.15.96:0/224123343 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55994299f3b0 0x559942d7a520 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f98cc0c7770 tx=0x7f98cc031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d9c5017ac2bdf60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:26.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.409+0000 7f98baffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:13:26.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.409+0000 7f98baffd640 1 -- 172.21.15.96:0/224123343 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc049070 con 0x55994299f3b0 2024-04-25T14:13:26.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.409+0000 7f98baffd640 1 -- 172.21.15.96:0/224123343 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f98cc0c3840 con 0x55994299f3b0 2024-04-25T14:13:26.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.409+0000 7f98e8831740 1 -- 172.21.15.96:0/224123343 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98cc0c7cb0 con 0x55994299f3b0 2024-04-25T14:13:26.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.410+0000 7f98baffd640 1 -- 172.21.15.96:0/224123343 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f98cc0c7810 con 0x55994299f3b0 2024-04-25T14:13:26.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.410+0000 7f98e8831740 10 client.15294 fetch_fsmap learned FSMap version 25 2024-04-25T14:13:26.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.410+0000 7f98e8831740 1 -- 172.21.15.96:0/224123343 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559942d7f150 con 0x55994299f3b0 2024-04-25T14:13:26.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.410+0000 7f98baffd640 1 -- 172.21.15.96:0/224123343 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98cc0c7cb0 con 0x55994299f3b0 2024-04-25T14:13:26.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.410+0000 7f98baffd640 1 -- 172.21.15.96:0/224123343 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f98cc0cece0 con 0x55994299f3b0 2024-04-25T14:13:26.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.410+0000 7f98e8831740 10 client.15294 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T14:13:26.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.410+0000 7f98e8831740 10 client.15294 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-25T14:13:26.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.410+0000 7f98e8831740 20 client.15294 populate_metadata read hostname 'smithi096' 2024-04-25T14:13:26.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.410+0000 7f98e8831740 1 --2- 172.21.15.96:0/224123343 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x55994295c280 0x559942df1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.410+0000 7f98e8831740 4 client.15294 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T14:13:26.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.410+0000 7f98e8831740 1 -- 172.21.15.96:0/224123343 --> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] -- command(tid 0: {"prefix": "status"}) -- 0x559942cd61c0 con 0x55994295c280 2024-04-25T14:13:26.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.410+0000 7f98da7fc640 1 --2- 172.21.15.96:0/224123343 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x55994295c280 0x559942df1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.411+0000 7f98baffd640 10 client.15294 ms_handle_connect on v2:172.21.15.97:6834/3872301726 2024-04-25T14:13:26.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.411+0000 7f98da7fc640 1 --2- 172.21.15.96:0/224123343 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x55994295c280 0x559942df1140 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5ee6b123fb560859 server_cookie=eab05b4095b02503 in_seq=0 out_seq=0 2024-04-25T14:13:26.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.411+0000 7f98baffd640 1 -- 172.21.15.96:0/224123343 <== mds.3 v2:172.21.15.97:6834/3872301726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x559942cd61c0 con 0x55994295c280 2024-04-25T14:13:26.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.411+0000 7f98baffd640 10 client.15294 handle_command_reply: tid=0 2024-04-25T14:13:26.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.411+0000 7f98e8831740 1 client.15294 shutdown 2024-04-25T14:13:26.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.411+0000 7f98e8831740 1 -- 172.21.15.96:0/224123343 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55994299f3b0 msgr2=0x559942d7a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.411+0000 7f98e8831740 1 --2- 172.21.15.96:0/224123343 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55994299f3b0 0x559942d7a520 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f98cc0c7770 tx=0x7f98cc031400 comp rx=0 tx=0).stop 2024-04-25T14:13:26.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.411+0000 7f98e8831740 1 -- 172.21.15.96:0/224123343 shutdown_connections 2024-04-25T14:13:26.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.411+0000 7f98e8831740 1 --2- 172.21.15.96:0/224123343 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x55994295c280 0x559942df1140 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.411+0000 7f98e8831740 1 --2- 172.21.15.96:0/224123343 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599429c8270 0x559942d7d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.411+0000 7f98baffd640 0 client.15294 ms_handle_reset on v2:172.21.15.97:6834/3872301726 2024-04-25T14:13:26.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.411+0000 7f98e8831740 1 --2- 172.21.15.96:0/224123343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5599429a7590 0x559942d7aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.411+0000 7f98e8831740 1 --2- 172.21.15.96:0/224123343 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55994299f3b0 0x559942d7a520 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.411+0000 7f98e8831740 1 -- 172.21.15.96:0/224123343 >> 172.21.15.96:0/224123343 conn(0x5599429bc3c0 msgr2=0x559942d772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:26.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.411+0000 7f98e8831740 1 -- 172.21.15.96:0/224123343 shutdown_connections 2024-04-25T14:13:26.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.411+0000 7f98e8831740 1 -- 172.21.15.96:0/224123343 wait complete. 2024-04-25T14:13:26.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.411+0000 7f98e8831740 20 client.15294 trim_cache size 0 max 16384 2024-04-25T14:13:26.418 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T14:13:26.419 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "59334182-030c-11ef-bc93-c7b262605968", 2024-04-25T14:13:26.419 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 3, 2024-04-25T14:13:26.419 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 24455, 2024-04-25T14:13:26.419 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-04-25T14:13:26.419 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-04-25T14:13:26.419 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-04-25T14:13:26.419 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 112.89261458, 2024-04-25T14:13:26.419 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 25, 2024-04-25T14:13:26.419 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 78, 2024-04-25T14:13:26.419 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 76, 2024-04-25T14:13:26.419 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 143.06764502999999 2024-04-25T14:13:26.419 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T14:13:26.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.412+0000 7f98e1985640 1 -- 172.21.15.96:0/2216484798 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f98c40afb70 msgr2=0x7f98c40b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.412+0000 7f98e1985640 1 --2- 172.21.15.96:0/2216484798 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f98c40afb70 0x7f98c40b2080 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x559942d6e950 tx=0x7f98d4005f40 comp rx=0 tx=0).stop 2024-04-25T14:13:26.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.412+0000 7f98e1985640 1 -- 172.21.15.96:0/2216484798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98dc1029e0 msgr2=0x7f98dc06a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.412+0000 7f98e1985640 1 --2- 172.21.15.96:0/2216484798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98dc1029e0 0x7f98dc06a960 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f98cc006010 tx=0x7f98cc004270 comp rx=0 tx=0).stop 2024-04-25T14:13:26.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.412+0000 7f98e1985640 1 -- 172.21.15.96:0/2216484798 shutdown_connections 2024-04-25T14:13:26.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.412+0000 7f98e1985640 1 --2- 172.21.15.96:0/2216484798 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98dc108920 0x7f98dc1a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.412+0000 7f98e1985640 1 --2- 172.21.15.96:0/2216484798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98dc103c20 0x7f98dc19f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.412+0000 7f98e1985640 1 --2- 172.21.15.96:0/2216484798 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f98c40afb70 0x7f98c40b2080 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.412+0000 7f98e1985640 1 --2- 172.21.15.96:0/2216484798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98dc1029e0 0x7f98dc06a960 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.412+0000 7f98e1985640 1 -- 172.21.15.96:0/2216484798 >> 172.21.15.96:0/2216484798 conn(0x7f98dc0fdbd0 msgr2=0x7f98dc104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:26.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.412+0000 7f98e1985640 1 -- 172.21.15.96:0/2216484798 shutdown_connections 2024-04-25T14:13:26.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.412+0000 7f98e1985640 1 -- 172.21.15.96:0/2216484798 wait complete. 2024-04-25T14:13:26.423 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-04-25T14:13:26.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.533+0000 7f51b7661640 1 -- 172.21.15.96:0/1384437158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51b0106f30 msgr2=0x7f51b0109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.533+0000 7f51b7661640 1 --2- 172.21.15.96:0/1384437158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51b0106f30 0x7f51b0109350 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f51a0009900 tx=0x7f51a002a410 comp rx=0 tx=0).stop 2024-04-25T14:13:26.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.533+0000 7f51b7661640 1 -- 172.21.15.96:0/1384437158 shutdown_connections 2024-04-25T14:13:26.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.533+0000 7f51b7661640 1 --2- 172.21.15.96:0/1384437158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51b0109890 0x7f51b010bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.533+0000 7f51b7661640 1 --2- 172.21.15.96:0/1384437158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51b0106f30 0x7f51b0109350 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.533+0000 7f51b7661640 1 --2- 172.21.15.96:0/1384437158 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51b01045d0 0x7f51b01069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.533+0000 7f51b7661640 1 -- 172.21.15.96:0/1384437158 >> 172.21.15.96:0/1384437158 conn(0x7f51b00fdbd0 msgr2=0x7f51b0100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:26.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.533+0000 7f51b7661640 1 -- 172.21.15.96:0/1384437158 shutdown_connections 2024-04-25T14:13:26.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.533+0000 7f51b7661640 1 -- 172.21.15.96:0/1384437158 wait complete. 2024-04-25T14:13:26.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.533+0000 7f51b7661640 1 Processor -- start 2024-04-25T14:13:26.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.533+0000 7f51b7661640 1 -- start start 2024-04-25T14:13:26.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.534+0000 7f51b7661640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51b01045d0 0x7f51b019a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.534+0000 7f51b7661640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51b0106f30 0x7f51b019ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.534+0000 7f51b7661640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51b0109890 0x7f51b01a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.534+0000 7f51b7661640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f51b010f610 con 0x7f51b01045d0 2024-04-25T14:13:26.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.534+0000 7f51b7661640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f51b010f490 con 0x7f51b0109890 2024-04-25T14:13:26.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.534+0000 7f51b7661640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f51b010f790 con 0x7f51b0106f30 2024-04-25T14:13:26.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.534+0000 7f51b53fd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51b01045d0 0x7f51b019a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.534+0000 7f51b53fd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51b01045d0 0x7f51b019a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51658/0 (socket says 172.21.15.96:51658) 2024-04-25T14:13:26.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.534+0000 7f51b53fd640 1 -- 172.21.15.96:0/3673791463 learned_addr learned my addr 172.21.15.96:0/3673791463 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:26.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.534+0000 7f51b4bfc640 1 --2- 172.21.15.96:0/3673791463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51b0106f30 0x7f51b019ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.534+0000 7f51b53fd640 1 -- 172.21.15.96:0/3673791463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51b0106f30 msgr2=0x7f51b019ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.534+0000 7f51b53fd640 1 --2- 172.21.15.96:0/3673791463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51b0106f30 0x7f51b019ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.534+0000 7f51b53fd640 1 -- 172.21.15.96:0/3673791463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51b0109890 msgr2=0x7f51b01a1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.534+0000 7f51b53fd640 1 --2- 172.21.15.96:0/3673791463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51b0109890 0x7f51b01a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.534+0000 7f51b53fd640 1 -- 172.21.15.96:0/3673791463 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51b01a2110 con 0x7f51b01045d0 2024-04-25T14:13:26.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.534+0000 7f51b53fd640 1 --2- 172.21.15.96:0/3673791463 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51b01045d0 0x7f51b019a5d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f51ac00cce0 tx=0x7f51ac00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:26.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.534+0000 7f51a67fc640 1 -- 172.21.15.96:0/3673791463 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ac004b20 con 0x7f51b01045d0 2024-04-25T14:13:26.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.534+0000 7f51a67fc640 1 -- 172.21.15.96:0/3673791463 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51ac004cc0 con 0x7f51b01045d0 2024-04-25T14:13:26.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.534+0000 7f51b7661640 1 -- 172.21.15.96:0/3673791463 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51b01a2400 con 0x7f51b01045d0 2024-04-25T14:13:26.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.534+0000 7f51a67fc640 1 -- 172.21.15.96:0/3673791463 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ac00f6b0 con 0x7f51b01045d0 2024-04-25T14:13:26.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.534+0000 7f51b7661640 1 -- 172.21.15.96:0/3673791463 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51b01a2900 con 0x7f51b01045d0 2024-04-25T14:13:26.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.537+0000 7f51be5ef740 1 Processor -- start 2024-04-25T14:13:26.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.537+0000 7f51a67fc640 1 -- 172.21.15.96:0/3673791463 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f51ac00f950 con 0x7f51b01045d0 2024-04-25T14:13:26.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.537+0000 7f51be5ef740 1 -- start start 2024-04-25T14:13:26.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.537+0000 7f51be5ef740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a2944a4270 0x55a294849780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.537+0000 7f51be5ef740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a29447b3b0 0x55a294849d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.537+0000 7f51be5ef740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a294483590 0x55a294850c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.537+0000 7f51be5ef740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a2947aa740 con 0x55a294483590 2024-04-25T14:13:26.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.537+0000 7f51be5ef740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a2947463a0 con 0x55a29447b3b0 2024-04-25T14:13:26.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.537+0000 7f51be5ef740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a2947a6700 con 0x55a2944a4270 2024-04-25T14:13:26.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.537+0000 7f51b4bfc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a294483590 0x55a294850c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.538+0000 7f51b4bfc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a294483590 0x55a294850c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51670/0 (socket says 172.21.15.96:51670) 2024-04-25T14:13:26.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.538+0000 7f51b4bfc640 1 -- 172.21.15.96:0/1060292916 learned_addr learned my addr 172.21.15.96:0/1060292916 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:26.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.538+0000 7f51b5bfe640 1 --2- 172.21.15.96:0/1060292916 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a2944a4270 0x55a294849780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.538+0000 7f51b53fd640 1 --2- 172.21.15.96:0/1060292916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a29447b3b0 0x55a294849d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.538+0000 7f51a67fc640 1 --2- 172.21.15.96:0/3673791463 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f519c0afaf0 0x7f519c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.538+0000 7f51b4bfc640 1 -- 172.21.15.96:0/1060292916 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a2944a4270 msgr2=0x55a294849780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:26.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.538+0000 7f51b4bfc640 1 --2- 172.21.15.96:0/1060292916 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a2944a4270 0x55a294849780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.538+0000 7f51b4bfc640 1 -- 172.21.15.96:0/1060292916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a29447b3b0 msgr2=0x55a294849d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:26.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.538+0000 7f51b4bfc640 1 --2- 172.21.15.96:0/1060292916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a29447b3b0 0x55a294849d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.538+0000 7f51b4bfc640 1 -- 172.21.15.96:0/1060292916 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a294762750 con 0x55a294483590 2024-04-25T14:13:26.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.538+0000 7f51b5bfe640 1 --2- 172.21.15.96:0/3673791463 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f519c0afaf0 0x7f519c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.538+0000 7f51a67fc640 1 -- 172.21.15.96:0/3673791463 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f51ac0ded70 con 0x7f51b01045d0 2024-04-25T14:13:26.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.538+0000 7f51b4bfc640 1 --2- 172.21.15.96:0/1060292916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a294483590 0x55a294850c00 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f51a00096c0 tx=0x7f51a0032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=926634889e6f6aab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:26.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.538+0000 7f519b7fe640 1 -- 172.21.15.96:0/1060292916 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a003f070 con 0x55a294483590 2024-04-25T14:13:26.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.538+0000 7f519b7fe640 1 -- 172.21.15.96:0/1060292916 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51a00317a0 con 0x55a294483590 2024-04-25T14:13:26.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.538+0000 7f519b7fe640 1 -- 172.21.15.96:0/1060292916 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a0035070 con 0x55a294483590 2024-04-25T14:13:26.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.539+0000 7f51be5ef740 1 -- 172.21.15.96:0/1060292916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a294483590 msgr2=0x55a294850c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.539+0000 7f51b5bfe640 1 --2- 172.21.15.96:0/3673791463 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f519c0afaf0 0x7f519c0b2000 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f51a8005d00 tx=0x7f51a80046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:26.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.539+0000 7f51be5ef740 1 --2- 172.21.15.96:0/1060292916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a294483590 0x55a294850c00 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f51a00096c0 tx=0x7f51a0032ce0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.539+0000 7f51be5ef740 1 -- 172.21.15.96:0/1060292916 shutdown_connections 2024-04-25T14:13:26.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.539+0000 7f51be5ef740 1 --2- 172.21.15.96:0/1060292916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a294483590 0x55a294850c00 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.539+0000 7f51be5ef740 1 --2- 172.21.15.96:0/1060292916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a29447b3b0 0x55a294849d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.539+0000 7f51be5ef740 1 --2- 172.21.15.96:0/1060292916 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a2944a4270 0x55a294849780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.539+0000 7f51be5ef740 1 -- 172.21.15.96:0/1060292916 >> 172.21.15.96:0/1060292916 conn(0x55a2944983c0 msgr2=0x55a294840d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:26.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.539+0000 7f51be5ef740 1 -- 172.21.15.96:0/1060292916 shutdown_connections 2024-04-25T14:13:26.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.539+0000 7f51be5ef740 1 -- 172.21.15.96:0/1060292916 wait complete. 2024-04-25T14:13:26.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.540+0000 7f51be5ef740 1 -- start start 2024-04-25T14:13:26.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.540+0000 7f51be5ef740 1 Processor -- start 2024-04-25T14:13:26.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.540+0000 7f51be5ef740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a29447b3b0 0x55a294853610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.540+0000 7f51be5ef740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a294483590 0x55a294853b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.540+0000 7f51be5ef740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a2944a4270 0x55a29485aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.540+0000 7f51be5ef740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a2948515f0 con 0x55a294483590 2024-04-25T14:13:26.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.540+0000 7f51be5ef740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a294731fe0 con 0x55a29447b3b0 2024-04-25T14:13:26.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.540+0000 7f51be5ef740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a294851770 con 0x55a2944a4270 2024-04-25T14:13:26.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.540+0000 7f51be5ef740 10 client.0 _ll_register_callbacks cb 0x55a294748150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:13:26.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.540+0000 7f51b4bfc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a294483590 0x55a294853b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.540+0000 7f51b4bfc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a294483590 0x55a294853b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51682/0 (socket says 172.21.15.96:51682) 2024-04-25T14:13:26.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.540+0000 7f51b4bfc640 1 -- 172.21.15.96:0/1845199727 learned_addr learned my addr 172.21.15.96:0/1845199727 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:26.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.540+0000 7f51b53fd640 1 --2- 172.21.15.96:0/1845199727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a29447b3b0 0x55a294853610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.540+0000 7f51b5bfe640 1 --2- 172.21.15.96:0/1845199727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a2944a4270 0x55a29485aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.540+0000 7f51b4bfc640 1 -- 172.21.15.96:0/1845199727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a2944a4270 msgr2=0x55a29485aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:26.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.540+0000 7f51b4bfc640 1 --2- 172.21.15.96:0/1845199727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a2944a4270 0x55a29485aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.540+0000 7f51b4bfc640 1 -- 172.21.15.96:0/1845199727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a29447b3b0 msgr2=0x55a294853610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:26.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.540+0000 7f51b4bfc640 1 --2- 172.21.15.96:0/1845199727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a29447b3b0 0x55a294853610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.540+0000 7f51b4bfc640 1 -- 172.21.15.96:0/1845199727 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2944bc2d0 con 0x55a294483590 2024-04-25T14:13:26.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.541+0000 7f51b4bfc640 1 --2- 172.21.15.96:0/1845199727 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a294483590 0x55a294853b50 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f51a00045f0 tx=0x7f51a0031950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45dfb07c29f9c61d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:26.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.541+0000 7f5198ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:13:26.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.541+0000 7f5198ff9640 1 -- 172.21.15.96:0/1845199727 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a003f070 con 0x55a294483590 2024-04-25T14:13:26.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.541+0000 7f5198ff9640 1 -- 172.21.15.96:0/1845199727 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f51a0003800 con 0x55a294483590 2024-04-25T14:13:26.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.541+0000 7f51be5ef740 1 -- 172.21.15.96:0/1845199727 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51a00317a0 con 0x55a294483590 2024-04-25T14:13:26.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.541+0000 7f5198ff9640 1 -- 172.21.15.96:0/1845199727 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f51a0035040 con 0x55a294483590 2024-04-25T14:13:26.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.541+0000 7f51be5ef740 10 client.15306 fetch_fsmap learned FSMap version 25 2024-04-25T14:13:26.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.541+0000 7f51be5ef740 1 -- 172.21.15.96:0/1845199727 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a2945866b0 con 0x55a294483590 2024-04-25T14:13:26.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.541+0000 7f5198ff9640 1 -- 172.21.15.96:0/1845199727 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51a00317a0 con 0x55a294483590 2024-04-25T14:13:26.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.541+0000 7f5198ff9640 1 -- 172.21.15.96:0/1845199727 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f51a0044bc0 con 0x55a294483590 2024-04-25T14:13:26.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.541+0000 7f51be5ef740 10 client.15306 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T14:13:26.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.541+0000 7f51be5ef740 10 client.15306 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-25T14:13:26.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.541+0000 7f51be5ef740 20 client.15306 populate_metadata read hostname 'smithi096' 2024-04-25T14:13:26.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.541+0000 7f51be5ef740 1 --2- 172.21.15.96:0/1845199727 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x55a294462e40 0x55a294860df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.541+0000 7f51be5ef740 4 client.15306 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:13:26.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.541+0000 7f51be5ef740 1 -- 172.21.15.96:0/1845199727 --> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2947b8350 con 0x55a294462e40 2024-04-25T14:13:26.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.542+0000 7f51b53fd640 1 --2- 172.21.15.96:0/1845199727 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x55a294462e40 0x55a294860df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.542+0000 7f5198ff9640 10 client.15306 ms_handle_connect on v2:172.21.15.151:6834/1313823638 2024-04-25T14:13:26.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.542+0000 7f51b53fd640 1 --2- 172.21.15.96:0/1845199727 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x55a294462e40 0x55a294860df0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8154a391322ccb1e server_cookie=c5a60690975d327f in_seq=0 out_seq=0 2024-04-25T14:13:26.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.543+0000 7f5198ff9640 1 -- 172.21.15.96:0/1845199727 <== mds.2 v2:172.21.15.151:6834/1313823638 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a2947b8350 con 0x55a294462e40 2024-04-25T14:13:26.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.543+0000 7f5198ff9640 10 client.15306 handle_command_reply: tid=0 2024-04-25T14:13:26.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.543+0000 7f51be5ef740 1 client.15306 shutdown 2024-04-25T14:13:26.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.543+0000 7f51be5ef740 1 -- 172.21.15.96:0/1845199727 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a294483590 msgr2=0x55a294853b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.543+0000 7f51be5ef740 1 --2- 172.21.15.96:0/1845199727 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a294483590 0x55a294853b50 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f51a00045f0 tx=0x7f51a0031950 comp rx=0 tx=0).stop 2024-04-25T14:13:26.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.543+0000 7f51be5ef740 1 -- 172.21.15.96:0/1845199727 shutdown_connections 2024-04-25T14:13:26.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.543+0000 7f51be5ef740 1 --2- 172.21.15.96:0/1845199727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a2944a4270 0x55a29485aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.543+0000 7f51be5ef740 1 --2- 172.21.15.96:0/1845199727 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x55a294462e40 0x55a294860df0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.543+0000 7f51be5ef740 1 --2- 172.21.15.96:0/1845199727 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a294483590 0x55a294853b50 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.543+0000 7f5198ff9640 0 client.15306 ms_handle_reset on v2:172.21.15.151:6834/1313823638 2024-04-25T14:13:26.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.543+0000 7f51be5ef740 1 --2- 172.21.15.96:0/1845199727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a29447b3b0 0x55a294853610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.543+0000 7f51be5ef740 1 -- 172.21.15.96:0/1845199727 >> 172.21.15.96:0/1845199727 conn(0x55a2944983c0 msgr2=0x55a29483fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:26.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.543+0000 7f51be5ef740 1 -- 172.21.15.96:0/1845199727 shutdown_connections 2024-04-25T14:13:26.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.543+0000 7f51be5ef740 1 -- 172.21.15.96:0/1845199727 wait complete. 2024-04-25T14:13:26.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.543+0000 7f51be5ef740 20 client.15306 trim_cache size 0 max 16384 2024-04-25T14:13:26.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.562+0000 7f51be5ef740 1 Processor -- start 2024-04-25T14:13:26.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.562+0000 7f51be5ef740 1 -- start start 2024-04-25T14:13:26.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.562+0000 7f51be5ef740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a29447b3b0 0x55a2948406b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.562+0000 7f51be5ef740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a294483590 0x55a294840bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.562+0000 7f51be5ef740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a2944a4270 0x55a29485a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.562+0000 7f51be5ef740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a29485f130 con 0x55a29447b3b0 2024-04-25T14:13:26.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.562+0000 7f51be5ef740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a29485f800 con 0x55a2944a4270 2024-04-25T14:13:26.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.562+0000 7f51be5ef740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a29485fed0 con 0x55a294483590 2024-04-25T14:13:26.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.562+0000 7f51b53fd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a29447b3b0 0x55a2948406b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.562+0000 7f51b53fd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a29447b3b0 0x55a2948406b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51694/0 (socket says 172.21.15.96:51694) 2024-04-25T14:13:26.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.562+0000 7f51b53fd640 1 -- 172.21.15.96:0/2077059897 learned_addr learned my addr 172.21.15.96:0/2077059897 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:26.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.562+0000 7f51b53fd640 1 -- 172.21.15.96:0/2077059897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a294483590 msgr2=0x55a294840bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:26.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.562+0000 7f51b53fd640 1 --2- 172.21.15.96:0/2077059897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a294483590 0x55a294840bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.562+0000 7f51b53fd640 1 -- 172.21.15.96:0/2077059897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a2944a4270 msgr2=0x55a29485a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:26.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.562+0000 7f51b5bfe640 1 --2- 172.21.15.96:0/2077059897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a2944a4270 0x55a29485a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.562+0000 7f51b53fd640 1 --2- 172.21.15.96:0/2077059897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a2944a4270 0x55a29485a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.562+0000 7f51b53fd640 1 -- 172.21.15.96:0/2077059897 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a29484a580 con 0x55a29447b3b0 2024-04-25T14:13:26.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.562+0000 7f51b5bfe640 1 --2- 172.21.15.96:0/2077059897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a2944a4270 0x55a29485a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:26.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.563+0000 7f51b53fd640 1 --2- 172.21.15.96:0/2077059897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a29447b3b0 0x55a2948406b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f51ac0a6f30 tx=0x7f51ac005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a5d4f43de89125a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:26.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.563+0000 7f519affd640 1 -- 172.21.15.96:0/2077059897 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ac01d040 con 0x55a29447b3b0 2024-04-25T14:13:26.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.563+0000 7f519affd640 1 -- 172.21.15.96:0/2077059897 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51ac0a7230 con 0x55a29447b3b0 2024-04-25T14:13:26.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.563+0000 7f519affd640 1 -- 172.21.15.96:0/2077059897 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ac00fd80 con 0x55a29447b3b0 2024-04-25T14:13:26.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.563+0000 7f51be5ef740 1 -- 172.21.15.96:0/2077059897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a29447b3b0 msgr2=0x55a2948406b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.563+0000 7f51be5ef740 1 --2- 172.21.15.96:0/2077059897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a29447b3b0 0x55a2948406b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f51ac0a6f30 tx=0x7f51ac005870 comp rx=0 tx=0).stop 2024-04-25T14:13:26.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.563+0000 7f51be5ef740 1 -- 172.21.15.96:0/2077059897 shutdown_connections 2024-04-25T14:13:26.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.563+0000 7f51be5ef740 1 --2- 172.21.15.96:0/2077059897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a2944a4270 0x55a29485a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.563+0000 7f51be5ef740 1 --2- 172.21.15.96:0/2077059897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a294483590 0x55a294840bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.563+0000 7f51be5ef740 1 --2- 172.21.15.96:0/2077059897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a29447b3b0 0x55a2948406b0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.563+0000 7f51be5ef740 1 -- 172.21.15.96:0/2077059897 >> 172.21.15.96:0/2077059897 conn(0x55a2944983c0 msgr2=0x55a29484dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:26.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.564+0000 7f51be5ef740 1 -- 172.21.15.96:0/2077059897 shutdown_connections 2024-04-25T14:13:26.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.564+0000 7f51be5ef740 1 -- 172.21.15.96:0/2077059897 wait complete. 2024-04-25T14:13:26.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.564+0000 7f51be5ef740 1 -- start start 2024-04-25T14:13:26.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.564+0000 7f51be5ef740 1 Processor -- start 2024-04-25T14:13:26.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.564+0000 7f51be5ef740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a29447b3b0 0x55a294856c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.564+0000 7f51be5ef740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a294483590 0x55a2948571c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.564+0000 7f51be5ef740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2944a4270 0x55a29485a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.564+0000 7f51be5ef740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a294860c70 con 0x55a2944a4270 2024-04-25T14:13:26.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.564+0000 7f51be5ef740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a2948605a0 con 0x55a294483590 2024-04-25T14:13:26.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.564+0000 7f51be5ef740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a294861340 con 0x55a29447b3b0 2024-04-25T14:13:26.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.564+0000 7f51be5ef740 10 client.0 _ll_register_callbacks cb 0x7f51a0032bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:13:26.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.564+0000 7f51b5bfe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2944a4270 0x55a29485a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.564+0000 7f51b5bfe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2944a4270 0x55a29485a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51704/0 (socket says 172.21.15.96:51704) 2024-04-25T14:13:26.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.564+0000 7f51b5bfe640 1 -- 172.21.15.96:0/386239732 learned_addr learned my addr 172.21.15.96:0/386239732 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:26.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.565+0000 7f51b4bfc640 1 --2- 172.21.15.96:0/386239732 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a294483590 0x55a2948571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.565+0000 7f51b53fd640 1 --2- 172.21.15.96:0/386239732 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a29447b3b0 0x55a294856c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.565+0000 7f51b5bfe640 1 -- 172.21.15.96:0/386239732 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a29447b3b0 msgr2=0x55a294856c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:26.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.565+0000 7f51b5bfe640 1 --2- 172.21.15.96:0/386239732 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a29447b3b0 0x55a294856c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.565+0000 7f51b5bfe640 1 -- 172.21.15.96:0/386239732 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a294483590 msgr2=0x55a2948571c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:26.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.565+0000 7f51b5bfe640 1 --2- 172.21.15.96:0/386239732 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a294483590 0x55a2948571c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.565+0000 7f51b5bfe640 1 -- 172.21.15.96:0/386239732 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a29485a6c0 con 0x55a2944a4270 2024-04-25T14:13:26.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.565+0000 7f51b4bfc640 1 --2- 172.21.15.96:0/386239732 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a294483590 0x55a2948571c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:26.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.565+0000 7f51b53fd640 1 --2- 172.21.15.96:0/386239732 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a29447b3b0 0x55a294856c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:26.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.565+0000 7f51b5bfe640 1 --2- 172.21.15.96:0/386239732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2944a4270 0x55a29485a100 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f51a80117c0 tx=0x7f51a8005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e665ab3aa0062ce0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:26.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.565+0000 7f5198ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:13:26.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.565+0000 7f5198ff9640 1 -- 172.21.15.96:0/386239732 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a801eb60 con 0x55a2944a4270 2024-04-25T14:13:26.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.565+0000 7f5198ff9640 1 -- 172.21.15.96:0/386239732 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f51a801ed00 con 0x55a2944a4270 2024-04-25T14:13:26.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.565+0000 7f51be5ef740 1 -- 172.21.15.96:0/386239732 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51ac0a7230 con 0x55a2944a4270 2024-04-25T14:13:26.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.565+0000 7f5198ff9640 1 -- 172.21.15.96:0/386239732 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f51a8028470 con 0x55a2944a4270 2024-04-25T14:13:26.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.565+0000 7f51be5ef740 10 client.15312 fetch_fsmap learned FSMap version 25 2024-04-25T14:13:26.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.565+0000 7f51be5ef740 1 -- 172.21.15.96:0/386239732 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a2948c9e30 con 0x55a2944a4270 2024-04-25T14:13:26.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.565+0000 7f5198ff9640 1 -- 172.21.15.96:0/386239732 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51ac0a7230 con 0x55a2944a4270 2024-04-25T14:13:26.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.566+0000 7f5198ff9640 1 -- 172.21.15.96:0/386239732 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f51a800f040 con 0x55a2944a4270 2024-04-25T14:13:26.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.566+0000 7f51be5ef740 10 client.15312 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T14:13:26.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.566+0000 7f51be5ef740 10 client.15312 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-25T14:13:26.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.566+0000 7f51be5ef740 20 client.15312 populate_metadata read hostname 'smithi096' 2024-04-25T14:13:26.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.566+0000 7f51be5ef740 1 --2- 172.21.15.96:0/386239732 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x55a294438280 0x55a2948cd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.566+0000 7f51be5ef740 4 client.15312 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T14:13:26.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.566+0000 7f51be5ef740 1 -- 172.21.15.96:0/386239732 --> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] -- command(tid 0: {"prefix": "status"}) -- 0x55a2947b21c0 con 0x55a294438280 2024-04-25T14:13:26.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.566+0000 7f51b53fd640 1 --2- 172.21.15.96:0/386239732 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x55a294438280 0x55a2948cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.566+0000 7f5198ff9640 10 client.15312 ms_handle_connect on v2:172.21.15.151:6834/1313823638 2024-04-25T14:13:26.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.566+0000 7f51b53fd640 1 --2- 172.21.15.96:0/386239732 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x55a294438280 0x55a2948cd750 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=383d8783be227525 server_cookie=b05ea9e8ec521a74 in_seq=0 out_seq=0 2024-04-25T14:13:26.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.567+0000 7f5198ff9640 1 -- 172.21.15.96:0/386239732 <== mds.2 v2:172.21.15.151:6834/1313823638 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a2947b21c0 con 0x55a294438280 2024-04-25T14:13:26.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.567+0000 7f5198ff9640 10 client.15312 handle_command_reply: tid=0 2024-04-25T14:13:26.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.567+0000 7f51be5ef740 1 client.15312 shutdown 2024-04-25T14:13:26.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.567+0000 7f51be5ef740 1 -- 172.21.15.96:0/386239732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2944a4270 msgr2=0x55a29485a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.567+0000 7f51be5ef740 1 --2- 172.21.15.96:0/386239732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2944a4270 0x55a29485a100 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f51a80117c0 tx=0x7f51a8005c90 comp rx=0 tx=0).stop 2024-04-25T14:13:26.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.567+0000 7f51be5ef740 1 -- 172.21.15.96:0/386239732 shutdown_connections 2024-04-25T14:13:26.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.567+0000 7f51be5ef740 1 --2- 172.21.15.96:0/386239732 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x55a294438280 0x55a2948cd750 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.567+0000 7f51be5ef740 1 --2- 172.21.15.96:0/386239732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2944a4270 0x55a29485a100 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.567+0000 7f5198ff9640 0 client.15312 ms_handle_reset on v2:172.21.15.151:6834/1313823638 2024-04-25T14:13:26.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.567+0000 7f51be5ef740 1 --2- 172.21.15.96:0/386239732 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a294483590 0x55a2948571c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.567+0000 7f51be5ef740 1 --2- 172.21.15.96:0/386239732 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a29447b3b0 0x55a294856c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.567+0000 7f51be5ef740 1 -- 172.21.15.96:0/386239732 >> 172.21.15.96:0/386239732 conn(0x55a2944983c0 msgr2=0x55a2948406b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:26.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.567+0000 7f51be5ef740 1 -- 172.21.15.96:0/386239732 shutdown_connections 2024-04-25T14:13:26.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.567+0000 7f51be5ef740 1 -- 172.21.15.96:0/386239732 wait complete. 2024-04-25T14:13:26.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.567+0000 7f51be5ef740 20 client.15312 trim_cache size 0 max 16384 2024-04-25T14:13:26.574 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T14:13:26.574 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "59334182-030c-11ef-bc93-c7b262605968", 2024-04-25T14:13:26.574 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 2, 2024-04-25T14:13:26.574 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 24478, 2024-04-25T14:13:26.575 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-04-25T14:13:26.575 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-04-25T14:13:26.575 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-04-25T14:13:26.575 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 114.05415173199999, 2024-04-25T14:13:26.575 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 25, 2024-04-25T14:13:26.575 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 78, 2024-04-25T14:13:26.575 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 76, 2024-04-25T14:13:26.575 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 140.406753571 2024-04-25T14:13:26.575 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T14:13:26.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.567+0000 7f51b7661640 1 -- 172.21.15.96:0/3673791463 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f519c0afaf0 msgr2=0x7f519c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.567+0000 7f51b7661640 1 --2- 172.21.15.96:0/3673791463 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f519c0afaf0 0x7f519c0b2000 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f51a8005d00 tx=0x7f51a80046c0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.567+0000 7f51b7661640 1 -- 172.21.15.96:0/3673791463 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51b01045d0 msgr2=0x7f51b019a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.567+0000 7f51b7661640 1 --2- 172.21.15.96:0/3673791463 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51b01045d0 0x7f51b019a5d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f51ac00cce0 tx=0x7f51ac00a620 comp rx=0 tx=0).stop 2024-04-25T14:13:26.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.567+0000 7f51b7661640 1 -- 172.21.15.96:0/3673791463 shutdown_connections 2024-04-25T14:13:26.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.567+0000 7f51b7661640 1 --2- 172.21.15.96:0/3673791463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51b0109890 0x7f51b01a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.567+0000 7f51b7661640 1 --2- 172.21.15.96:0/3673791463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51b0106f30 0x7f51b019ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.567+0000 7f51b7661640 1 --2- 172.21.15.96:0/3673791463 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f519c0afaf0 0x7f519c0b2000 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.567+0000 7f51b7661640 1 --2- 172.21.15.96:0/3673791463 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51b01045d0 0x7f51b019a5d0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.567+0000 7f51b7661640 1 -- 172.21.15.96:0/3673791463 >> 172.21.15.96:0/3673791463 conn(0x7f51b00fdbd0 msgr2=0x7f51b0104b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:26.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.568+0000 7f51b7661640 1 -- 172.21.15.96:0/3673791463 shutdown_connections 2024-04-25T14:13:26.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.568+0000 7f51b7661640 1 -- 172.21.15.96:0/3673791463 wait complete. 2024-04-25T14:13:26.579 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-25T14:13:26.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.687+0000 7f43a4492640 1 -- 172.21.15.96:0/3710349866 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f439c102a00 msgr2=0x7f439c103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.687+0000 7f43a4492640 1 --2- 172.21.15.96:0/3710349866 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f439c102a00 0x7f439c103700 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4398009ca0 tx=0x7f439802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.687+0000 7f43a4492640 1 -- 172.21.15.96:0/3710349866 shutdown_connections 2024-04-25T14:13:26.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.687+0000 7f43a4492640 1 --2- 172.21.15.96:0/3710349866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f439c108940 0x7f439c10ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.687+0000 7f43a4492640 1 --2- 172.21.15.96:0/3710349866 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f439c103c40 0x7f439c104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.687+0000 7f43a4492640 1 --2- 172.21.15.96:0/3710349866 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f439c102a00 0x7f439c103700 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.687+0000 7f43a4492640 1 -- 172.21.15.96:0/3710349866 >> 172.21.15.96:0/3710349866 conn(0x7f439c0fdbf0 msgr2=0x7f439c1000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:26.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.687+0000 7f43a4492640 1 -- 172.21.15.96:0/3710349866 shutdown_connections 2024-04-25T14:13:26.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.687+0000 7f43a4492640 1 -- 172.21.15.96:0/3710349866 wait complete. 2024-04-25T14:13:26.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.688+0000 7f43a4492640 1 Processor -- start 2024-04-25T14:13:26.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.688+0000 7f43a4492640 1 -- start start 2024-04-25T14:13:26.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.688+0000 7f43a4492640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f439c102a00 0x7f439c075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.688+0000 7f43a4492640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f439c103c40 0x7f439c19eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.688+0000 7f43a4492640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f439c108940 0x7f439c1a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.688+0000 7f43a4492640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f439c10e500 con 0x7f439c103c40 2024-04-25T14:13:26.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.688+0000 7f43a4492640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f439c10e380 con 0x7f439c102a00 2024-04-25T14:13:26.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.688+0000 7f43a4492640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f439c10e680 con 0x7f439c108940 2024-04-25T14:13:26.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.688+0000 7f43a1a2d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f439c103c40 0x7f439c19eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.688+0000 7f43a1a2d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f439c103c40 0x7f439c19eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51714/0 (socket says 172.21.15.96:51714) 2024-04-25T14:13:26.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.688+0000 7f43a1a2d640 1 -- 172.21.15.96:0/3481389968 learned_addr learned my addr 172.21.15.96:0/3481389968 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:26.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.688+0000 7f43a222e640 1 --2- 172.21.15.96:0/3481389968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f439c102a00 0x7f439c075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.689+0000 7f43a2a2f640 1 --2- 172.21.15.96:0/3481389968 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f439c108940 0x7f439c1a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.689+0000 7f43a1a2d640 1 -- 172.21.15.96:0/3481389968 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f439c108940 msgr2=0x7f439c1a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.689+0000 7f43a1a2d640 1 --2- 172.21.15.96:0/3481389968 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f439c108940 0x7f439c1a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.689+0000 7f43a1a2d640 1 -- 172.21.15.96:0/3481389968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f439c102a00 msgr2=0x7f439c075860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.689+0000 7f43a1a2d640 1 --2- 172.21.15.96:0/3481389968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f439c102a00 0x7f439c075860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.689+0000 7f43a1a2d640 1 -- 172.21.15.96:0/3481389968 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f439c1a1cf0 con 0x7f439c103c40 2024-04-25T14:13:26.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.689+0000 7f43a2a2f640 1 --2- 172.21.15.96:0/3481389968 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f439c108940 0x7f439c1a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:26.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.689+0000 7f43a1a2d640 1 --2- 172.21.15.96:0/3481389968 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f439c103c40 0x7f439c19eff0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f438c00c970 tx=0x7f438c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:26.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.689+0000 7f43937fe640 1 -- 172.21.15.96:0/3481389968 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438c007bb0 con 0x7f439c103c40 2024-04-25T14:13:26.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.689+0000 7f43937fe640 1 -- 172.21.15.96:0/3481389968 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f438c007d50 con 0x7f439c103c40 2024-04-25T14:13:26.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.689+0000 7f43a4492640 1 -- 172.21.15.96:0/3481389968 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f439c1a1fe0 con 0x7f439c103c40 2024-04-25T14:13:26.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.689+0000 7f43937fe640 1 -- 172.21.15.96:0/3481389968 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438c00f450 con 0x7f439c103c40 2024-04-25T14:13:26.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.689+0000 7f43a4492640 1 -- 172.21.15.96:0/3481389968 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f439c1a23f0 con 0x7f439c103c40 2024-04-25T14:13:26.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.691+0000 7f43ab41c740 1 Processor -- start 2024-04-25T14:13:26.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.692+0000 7f43ab41c740 1 -- start start 2024-04-25T14:13:26.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.692+0000 7f43937fe640 1 -- 172.21.15.96:0/3481389968 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f438c004f20 con 0x7f439c103c40 2024-04-25T14:13:26.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.692+0000 7f43ab41c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56327260a270 0x5632729ab370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.692+0000 7f43ab41c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5632725e13b0 0x5632729ab8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.692+0000 7f43ab41c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632725e9590 0x5632729b27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.692+0000 7f43ab41c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563272910740 con 0x56327260a270 2024-04-25T14:13:26.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.692+0000 7f43ab41c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5632728ac3a0 con 0x5632725e9590 2024-04-25T14:13:26.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.692+0000 7f43ab41c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56327290c700 con 0x5632725e13b0 2024-04-25T14:13:26.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.692+0000 7f43a2a2f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56327260a270 0x5632729ab370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.692+0000 7f43a2a2f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56327260a270 0x5632729ab370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51730/0 (socket says 172.21.15.96:51730) 2024-04-25T14:13:26.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.692+0000 7f43a2a2f640 1 -- 172.21.15.96:0/2432319316 learned_addr learned my addr 172.21.15.96:0/2432319316 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:26.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.692+0000 7f43a222e640 1 --2- 172.21.15.96:0/2432319316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5632725e13b0 0x5632729ab8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.692+0000 7f43a1a2d640 1 --2- 172.21.15.96:0/2432319316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632725e9590 0x5632729b27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.692+0000 7f43a2a2f640 1 -- 172.21.15.96:0/2432319316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5632725e13b0 msgr2=0x5632729ab8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:26.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.692+0000 7f43a2a2f640 1 --2- 172.21.15.96:0/2432319316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5632725e13b0 0x5632729ab8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.692+0000 7f43a2a2f640 1 -- 172.21.15.96:0/2432319316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632725e9590 msgr2=0x5632729b27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:26.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.692+0000 7f43a2a2f640 1 --2- 172.21.15.96:0/2432319316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632725e9590 0x5632729b27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.692+0000 7f43a2a2f640 1 -- 172.21.15.96:0/2432319316 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632728c8750 con 0x56327260a270 2024-04-25T14:13:26.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.692+0000 7f43a1a2d640 1 --2- 172.21.15.96:0/2432319316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632725e9590 0x5632729b27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:26.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.693+0000 7f43937fe640 1 --2- 172.21.15.96:0/3481389968 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f43880afaf0 0x7f43880b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.693+0000 7f43a2a2f640 1 --2- 172.21.15.96:0/2432319316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56327260a270 0x5632729ab370 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f439400bb60 tx=0x7f439400d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf8db98d8507d40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:26.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.693+0000 7f43a222e640 1 --2- 172.21.15.96:0/3481389968 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f43880afaf0 0x7f43880b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.693+0000 7f43937fe640 1 -- 172.21.15.96:0/3481389968 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f438c0de9d0 con 0x7f439c103c40 2024-04-25T14:13:26.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.693+0000 7f4390bfe640 1 -- 172.21.15.96:0/2432319316 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439400be60 con 0x56327260a270 2024-04-25T14:13:26.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.693+0000 7f4390bfe640 1 -- 172.21.15.96:0/2432319316 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4394017ad0 con 0x56327260a270 2024-04-25T14:13:26.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.693+0000 7f4390bfe640 1 -- 172.21.15.96:0/2432319316 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439400fbe0 con 0x56327260a270 2024-04-25T14:13:26.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.694+0000 7f43ab41c740 1 -- 172.21.15.96:0/2432319316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56327260a270 msgr2=0x5632729ab370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.694+0000 7f43a222e640 1 --2- 172.21.15.96:0/3481389968 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f43880afaf0 0x7f43880b2000 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4398000f80 tx=0x7f439802f3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:26.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.694+0000 7f43ab41c740 1 --2- 172.21.15.96:0/2432319316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56327260a270 0x5632729ab370 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f439400bb60 tx=0x7f439400d710 comp rx=0 tx=0).stop 2024-04-25T14:13:26.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.694+0000 7f43ab41c740 1 -- 172.21.15.96:0/2432319316 shutdown_connections 2024-04-25T14:13:26.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.694+0000 7f43ab41c740 1 --2- 172.21.15.96:0/2432319316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632725e9590 0x5632729b27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.694+0000 7f43ab41c740 1 --2- 172.21.15.96:0/2432319316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5632725e13b0 0x5632729ab8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.694+0000 7f43ab41c740 1 --2- 172.21.15.96:0/2432319316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56327260a270 0x5632729ab370 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.694+0000 7f43ab41c740 1 -- 172.21.15.96:0/2432319316 >> 172.21.15.96:0/2432319316 conn(0x5632725fe3c0 msgr2=0x5632729a6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:26.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.694+0000 7f43ab41c740 1 -- 172.21.15.96:0/2432319316 shutdown_connections 2024-04-25T14:13:26.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.694+0000 7f43ab41c740 1 -- 172.21.15.96:0/2432319316 wait complete. 2024-04-25T14:13:26.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.694+0000 7f43ab41c740 1 -- start start 2024-04-25T14:13:26.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.694+0000 7f43ab41c740 1 Processor -- start 2024-04-25T14:13:26.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.694+0000 7f43ab41c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632725e13b0 0x5632729b52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.694+0000 7f43ab41c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5632725e9590 0x5632729b5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.694+0000 7f43ab41c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56327260a270 0x5632729bc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.695+0000 7f43ab41c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5632729b3240 con 0x5632725e13b0 2024-04-25T14:13:26.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.695+0000 7f43ab41c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563272897fe0 con 0x56327260a270 2024-04-25T14:13:26.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.695+0000 7f43ab41c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5632729b33c0 con 0x5632725e9590 2024-04-25T14:13:26.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.695+0000 7f43ab41c740 10 client.0 _ll_register_callbacks cb 0x5632728ae150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:13:26.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.695+0000 7f43a222e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632725e13b0 0x5632729b52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.695+0000 7f43a222e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632725e13b0 0x5632729b52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51738/0 (socket says 172.21.15.96:51738) 2024-04-25T14:13:26.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.695+0000 7f43a222e640 1 -- 172.21.15.96:0/291465706 learned_addr learned my addr 172.21.15.96:0/291465706 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:26.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.695+0000 7f43a1a2d640 1 --2- 172.21.15.96:0/291465706 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5632725e9590 0x5632729b5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.695+0000 7f43a222e640 1 -- 172.21.15.96:0/291465706 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5632725e9590 msgr2=0x5632729b5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:26.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.695+0000 7f43a222e640 1 --2- 172.21.15.96:0/291465706 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5632725e9590 0x5632729b5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.695+0000 7f43a222e640 1 -- 172.21.15.96:0/291465706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56327260a270 msgr2=0x5632729bc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:26.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.695+0000 7f43a222e640 1 --2- 172.21.15.96:0/291465706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56327260a270 0x5632729bc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.695+0000 7f43a222e640 1 -- 172.21.15.96:0/291465706 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632726222d0 con 0x5632725e13b0 2024-04-25T14:13:26.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.695+0000 7f43a1a2d640 1 --2- 172.21.15.96:0/291465706 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5632725e9590 0x5632729b5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:26.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.695+0000 7f43a222e640 1 --2- 172.21.15.96:0/291465706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632725e13b0 0x5632729b52f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f439c1022d0 tx=0x7f439802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c9fd4add00945c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:26.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.695+0000 7f4385ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:13:26.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.695+0000 7f4385ffb640 1 -- 172.21.15.96:0/291465706 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43980043d0 con 0x5632725e13b0 2024-04-25T14:13:26.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.695+0000 7f4385ffb640 1 -- 172.21.15.96:0/291465706 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f4398004570 con 0x5632725e13b0 2024-04-25T14:13:26.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.695+0000 7f43ab41c740 1 -- 172.21.15.96:0/291465706 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4394017ad0 con 0x5632725e13b0 2024-04-25T14:13:26.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.695+0000 7f4385ffb640 1 -- 172.21.15.96:0/291465706 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4398059070 con 0x5632725e13b0 2024-04-25T14:13:26.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.695+0000 7f43ab41c740 10 client.15324 fetch_fsmap learned FSMap version 25 2024-04-25T14:13:26.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.695+0000 7f43ab41c740 1 -- 172.21.15.96:0/291465706 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5632726ec6b0 con 0x5632725e13b0 2024-04-25T14:13:26.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.696+0000 7f4385ffb640 1 -- 172.21.15.96:0/291465706 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4394017ad0 con 0x5632725e13b0 2024-04-25T14:13:26.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.696+0000 7f4385ffb640 1 -- 172.21.15.96:0/291465706 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4398051020 con 0x5632725e13b0 2024-04-25T14:13:26.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.696+0000 7f43ab41c740 10 client.15324 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T14:13:26.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.696+0000 7f43ab41c740 10 client.15324 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T14:13:26.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.696+0000 7f43ab41c740 20 client.15324 populate_metadata read hostname 'smithi096' 2024-04-25T14:13:26.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.696+0000 7f43ab41c740 1 --2- 172.21.15.96:0/291465706 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x5632725c8e40 0x5632729c3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.696+0000 7f43ab41c740 4 client.15324 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:13:26.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.696+0000 7f43ab41c740 1 -- 172.21.15.96:0/291465706 --> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56327291e350 con 0x5632725c8e40 2024-04-25T14:13:26.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.696+0000 7f43a1a2d640 1 --2- 172.21.15.96:0/291465706 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x5632725c8e40 0x5632729c3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.697+0000 7f43a1a2d640 1 --2- 172.21.15.96:0/291465706 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x5632725c8e40 0x5632729c3020 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cb1f202c2479318f server_cookie=f6eeafda84ff7416 in_seq=0 out_seq=0 2024-04-25T14:13:26.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.697+0000 7f4385ffb640 10 client.15324 ms_handle_connect on v2:172.21.15.151:6838/245699981 2024-04-25T14:13:26.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.697+0000 7f4385ffb640 1 -- 172.21.15.96:0/291465706 <== mds.1 v2:172.21.15.151:6838/245699981 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56327291e350 con 0x5632725c8e40 2024-04-25T14:13:26.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.697+0000 7f4385ffb640 10 client.15324 handle_command_reply: tid=0 2024-04-25T14:13:26.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.698+0000 7f43ab41c740 1 client.15324 shutdown 2024-04-25T14:13:26.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.698+0000 7f43ab41c740 1 -- 172.21.15.96:0/291465706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632725e13b0 msgr2=0x5632729b52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.698+0000 7f43ab41c740 1 --2- 172.21.15.96:0/291465706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632725e13b0 0x5632729b52f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f439c1022d0 tx=0x7f439802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.698+0000 7f43ab41c740 1 -- 172.21.15.96:0/291465706 shutdown_connections 2024-04-25T14:13:26.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.698+0000 7f43ab41c740 1 --2- 172.21.15.96:0/291465706 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x5632725c8e40 0x5632729c3020 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.698+0000 7f43ab41c740 1 --2- 172.21.15.96:0/291465706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56327260a270 0x5632729bc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.698+0000 7f43ab41c740 1 --2- 172.21.15.96:0/291465706 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5632725e9590 0x5632729b5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.698+0000 7f43ab41c740 1 --2- 172.21.15.96:0/291465706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632725e13b0 0x5632729b52f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.698+0000 7f4385ffb640 0 client.15324 ms_handle_reset on v2:172.21.15.151:6838/245699981 2024-04-25T14:13:26.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.698+0000 7f43ab41c740 1 -- 172.21.15.96:0/291465706 >> 172.21.15.96:0/291465706 conn(0x5632725fe3c0 msgr2=0x5632729a5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:26.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.698+0000 7f43ab41c740 1 -- 172.21.15.96:0/291465706 shutdown_connections 2024-04-25T14:13:26.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.698+0000 7f43ab41c740 1 -- 172.21.15.96:0/291465706 wait complete. 2024-04-25T14:13:26.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.698+0000 7f43ab41c740 20 client.15324 trim_cache size 0 max 16384 2024-04-25T14:13:26.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.716+0000 7f43ab41c740 1 Processor -- start 2024-04-25T14:13:26.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.716+0000 7f43ab41c740 1 -- start start 2024-04-25T14:13:26.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.716+0000 7f43ab41c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5632725c8e40 0x5632729ae9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.716+0000 7f43ab41c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632725e13b0 0x5632729aa2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.716+0000 7f43ab41c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632725e9590 0x5632729aa7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.716+0000 7f43ab41c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5632729c1360 con 0x5632725e9590 2024-04-25T14:13:26.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.716+0000 7f43ab41c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5632729c1a30 con 0x5632725e13b0 2024-04-25T14:13:26.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.716+0000 7f43ab41c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5632729c2100 con 0x5632725c8e40 2024-04-25T14:13:26.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.717+0000 7f43a2a2f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632725e9590 0x5632729aa7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.717+0000 7f43a2a2f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632725e9590 0x5632729aa7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51744/0 (socket says 172.21.15.96:51744) 2024-04-25T14:13:26.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.717+0000 7f43a2a2f640 1 -- 172.21.15.96:0/691857660 learned_addr learned my addr 172.21.15.96:0/691857660 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:26.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.717+0000 7f43a222e640 1 --2- 172.21.15.96:0/691857660 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5632725c8e40 0x5632729ae9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.717+0000 7f43a1a2d640 1 --2- 172.21.15.96:0/691857660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632725e13b0 0x5632729aa2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.717+0000 7f43a2a2f640 1 -- 172.21.15.96:0/691857660 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5632725c8e40 msgr2=0x5632729ae9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:26.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.717+0000 7f43a2a2f640 1 --2- 172.21.15.96:0/691857660 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5632725c8e40 0x5632729ae9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.717+0000 7f43a2a2f640 1 -- 172.21.15.96:0/691857660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632725e13b0 msgr2=0x5632729aa2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:26.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.717+0000 7f43a2a2f640 1 --2- 172.21.15.96:0/691857660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632725e13b0 0x5632729aa2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.717+0000 7f43a2a2f640 1 -- 172.21.15.96:0/691857660 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632729b0160 con 0x5632725e9590 2024-04-25T14:13:26.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.717+0000 7f43a1a2d640 1 --2- 172.21.15.96:0/691857660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632725e13b0 0x5632729aa2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:26.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.717+0000 7f43a2a2f640 1 --2- 172.21.15.96:0/691857660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632725e9590 0x5632729aa7f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4394019040 tx=0x7f439400b310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cef42722c29d9df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:26.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.717+0000 7f4387fff640 1 -- 172.21.15.96:0/691857660 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4394026020 con 0x5632725e9590 2024-04-25T14:13:26.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.717+0000 7f4387fff640 1 -- 172.21.15.96:0/691857660 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f439400b750 con 0x5632725e9590 2024-04-25T14:13:26.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.717+0000 7f4387fff640 1 -- 172.21.15.96:0/691857660 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4394004e40 con 0x5632725e9590 2024-04-25T14:13:26.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.718+0000 7f43ab41c740 1 -- 172.21.15.96:0/691857660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632725e9590 msgr2=0x5632729aa7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.718+0000 7f43ab41c740 1 --2- 172.21.15.96:0/691857660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632725e9590 0x5632729aa7f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4394019040 tx=0x7f439400b310 comp rx=0 tx=0).stop 2024-04-25T14:13:26.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.718+0000 7f43ab41c740 1 -- 172.21.15.96:0/691857660 shutdown_connections 2024-04-25T14:13:26.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.718+0000 7f43ab41c740 1 --2- 172.21.15.96:0/691857660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632725e9590 0x5632729aa7f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.718+0000 7f43ab41c740 1 --2- 172.21.15.96:0/691857660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632725e13b0 0x5632729aa2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.718+0000 7f43ab41c740 1 --2- 172.21.15.96:0/691857660 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5632725c8e40 0x5632729ae9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.718+0000 7f43ab41c740 1 -- 172.21.15.96:0/691857660 >> 172.21.15.96:0/691857660 conn(0x5632725fe3c0 msgr2=0x5632729a9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:26.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.718+0000 7f43ab41c740 1 -- 172.21.15.96:0/691857660 shutdown_connections 2024-04-25T14:13:26.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.718+0000 7f43ab41c740 1 -- 172.21.15.96:0/691857660 wait complete. 2024-04-25T14:13:26.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.718+0000 7f43ab41c740 1 -- start start 2024-04-25T14:13:26.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.718+0000 7f43ab41c740 1 Processor -- start 2024-04-25T14:13:26.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.719+0000 7f43ab41c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632725c8e40 0x5632729bbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.719+0000 7f43ab41c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632725e13b0 0x563272a27e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.719+0000 7f43ab41c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5632725e9590 0x563272a2a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.719+0000 7f43ab41c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5632729c2ea0 con 0x5632725c8e40 2024-04-25T14:13:26.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.719+0000 7f43ab41c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5632729c27d0 con 0x5632725e13b0 2024-04-25T14:13:26.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.719+0000 7f43ab41c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5632729c3570 con 0x5632725e9590 2024-04-25T14:13:26.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.719+0000 7f43ab41c740 10 client.0 _ll_register_callbacks cb 0x7f4398009920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:13:26.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.719+0000 7f43a222e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632725c8e40 0x5632729bbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.719+0000 7f43a222e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632725c8e40 0x5632729bbe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51746/0 (socket says 172.21.15.96:51746) 2024-04-25T14:13:26.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.719+0000 7f43a222e640 1 -- 172.21.15.96:0/1258835820 learned_addr learned my addr 172.21.15.96:0/1258835820 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:26.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.719+0000 7f43a222e640 1 -- 172.21.15.96:0/1258835820 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5632725e9590 msgr2=0x563272a2a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:26.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.719+0000 7f43a222e640 1 --2- 172.21.15.96:0/1258835820 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5632725e9590 0x563272a2a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.719+0000 7f43a222e640 1 -- 172.21.15.96:0/1258835820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632725e13b0 msgr2=0x563272a27e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:26.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.719+0000 7f43a222e640 1 --2- 172.21.15.96:0/1258835820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632725e13b0 0x563272a27e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.719+0000 7f43a222e640 1 -- 172.21.15.96:0/1258835820 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563272a2ac40 con 0x5632725c8e40 2024-04-25T14:13:26.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.719+0000 7f43a222e640 1 --2- 172.21.15.96:0/1258835820 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632725c8e40 0x5632729bbe00 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4398030760 tx=0x7f4398004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4278130f28777a1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:26.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.719+0000 7f4385ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:13:26.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.719+0000 7f4385ffb640 1 -- 172.21.15.96:0/1258835820 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4398030be0 con 0x5632725c8e40 2024-04-25T14:13:26.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.719+0000 7f4385ffb640 1 -- 172.21.15.96:0/1258835820 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f4398030d80 con 0x5632725c8e40 2024-04-25T14:13:26.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.719+0000 7f43ab41c740 1 -- 172.21.15.96:0/1258835820 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f439400b750 con 0x5632725c8e40 2024-04-25T14:13:26.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.719+0000 7f4385ffb640 1 -- 172.21.15.96:0/1258835820 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f439802f660 con 0x5632725c8e40 2024-04-25T14:13:26.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.719+0000 7f43ab41c740 10 client.15330 fetch_fsmap learned FSMap version 25 2024-04-25T14:13:26.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.719+0000 7f43ab41c740 1 -- 172.21.15.96:0/1258835820 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563272a2bf30 con 0x5632725c8e40 2024-04-25T14:13:26.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.720+0000 7f4385ffb640 1 -- 172.21.15.96:0/1258835820 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f439400b750 con 0x5632725c8e40 2024-04-25T14:13:26.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.720+0000 7f4385ffb640 1 -- 172.21.15.96:0/1258835820 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4398051070 con 0x5632725c8e40 2024-04-25T14:13:26.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.720+0000 7f43ab41c740 10 client.15330 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T14:13:26.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.720+0000 7f43ab41c740 10 client.15330 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T14:13:26.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.720+0000 7f43ab41c740 20 client.15330 populate_metadata read hostname 'smithi096' 2024-04-25T14:13:26.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.720+0000 7f43ab41c740 1 --2- 172.21.15.96:0/1258835820 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x56327260a270 0x563272a2f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.720+0000 7f43ab41c740 4 client.15330 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T14:13:26.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.720+0000 7f43ab41c740 1 -- 172.21.15.96:0/1258835820 --> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] -- command(tid 0: {"prefix": "status"}) -- 0x5632729181c0 con 0x56327260a270 2024-04-25T14:13:26.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.720+0000 7f43a1a2d640 1 --2- 172.21.15.96:0/1258835820 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x56327260a270 0x563272a2f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.721+0000 7f4385ffb640 10 client.15330 ms_handle_connect on v2:172.21.15.151:6838/245699981 2024-04-25T14:13:26.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.721+0000 7f43a1a2d640 1 --2- 172.21.15.96:0/1258835820 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x56327260a270 0x563272a2f990 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5385067636c3de06 server_cookie=e232ef757345d0fa in_seq=0 out_seq=0 2024-04-25T14:13:26.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.721+0000 7f4385ffb640 1 -- 172.21.15.96:0/1258835820 <== mds.1 v2:172.21.15.151:6838/245699981 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5632729181c0 con 0x56327260a270 2024-04-25T14:13:26.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.721+0000 7f4385ffb640 10 client.15330 handle_command_reply: tid=0 2024-04-25T14:13:26.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.721+0000 7f43ab41c740 1 client.15330 shutdown 2024-04-25T14:13:26.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.721+0000 7f43ab41c740 1 -- 172.21.15.96:0/1258835820 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632725c8e40 msgr2=0x5632729bbe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.721+0000 7f43ab41c740 1 --2- 172.21.15.96:0/1258835820 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632725c8e40 0x5632729bbe00 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4398030760 tx=0x7f4398004740 comp rx=0 tx=0).stop 2024-04-25T14:13:26.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.721+0000 7f43ab41c740 1 -- 172.21.15.96:0/1258835820 shutdown_connections 2024-04-25T14:13:26.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.721+0000 7f43ab41c740 1 --2- 172.21.15.96:0/1258835820 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x56327260a270 0x563272a2f990 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.721+0000 7f43ab41c740 1 --2- 172.21.15.96:0/1258835820 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5632725e9590 0x563272a2a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.721+0000 7f4385ffb640 0 client.15330 ms_handle_reset on v2:172.21.15.151:6838/245699981 2024-04-25T14:13:26.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.721+0000 7f43ab41c740 1 --2- 172.21.15.96:0/1258835820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632725e13b0 0x563272a27e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.721+0000 7f43ab41c740 1 --2- 172.21.15.96:0/1258835820 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632725c8e40 0x5632729bbe00 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.721+0000 7f43ab41c740 1 -- 172.21.15.96:0/1258835820 >> 172.21.15.96:0/1258835820 conn(0x5632725fe3c0 msgr2=0x5632729a9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:26.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.721+0000 7f43ab41c740 1 -- 172.21.15.96:0/1258835820 shutdown_connections 2024-04-25T14:13:26.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.721+0000 7f43ab41c740 1 -- 172.21.15.96:0/1258835820 wait complete. 2024-04-25T14:13:26.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.721+0000 7f43ab41c740 20 client.15330 trim_cache size 0 max 16384 2024-04-25T14:13:26.732 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T14:13:26.732 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "59334182-030c-11ef-bc93-c7b262605968", 2024-04-25T14:13:26.732 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 1, 2024-04-25T14:13:26.732 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 24517, 2024-04-25T14:13:26.732 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-04-25T14:13:26.732 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-04-25T14:13:26.732 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-04-25T14:13:26.733 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 115.144251041, 2024-04-25T14:13:26.733 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 25, 2024-04-25T14:13:26.733 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 78, 2024-04-25T14:13:26.733 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 76, 2024-04-25T14:13:26.733 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 131.82646327699999 2024-04-25T14:13:26.733 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T14:13:26.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.722+0000 7f43a4492640 1 -- 172.21.15.96:0/3481389968 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f43880afaf0 msgr2=0x7f43880b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.722+0000 7f43a4492640 1 --2- 172.21.15.96:0/3481389968 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f43880afaf0 0x7f43880b2000 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4398000f80 tx=0x7f439802f3b0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.722+0000 7f43a4492640 1 -- 172.21.15.96:0/3481389968 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f439c103c40 msgr2=0x7f439c19eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.722+0000 7f43a4492640 1 --2- 172.21.15.96:0/3481389968 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f439c103c40 0x7f439c19eff0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f438c00c970 tx=0x7f438c00ce30 comp rx=0 tx=0).stop 2024-04-25T14:13:26.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.722+0000 7f43a4492640 1 -- 172.21.15.96:0/3481389968 shutdown_connections 2024-04-25T14:13:26.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.722+0000 7f43a4492640 1 --2- 172.21.15.96:0/3481389968 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f439c108940 0x7f439c1a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.722+0000 7f43a4492640 1 --2- 172.21.15.96:0/3481389968 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f43880afaf0 0x7f43880b2000 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.722+0000 7f43a4492640 1 --2- 172.21.15.96:0/3481389968 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f439c103c40 0x7f439c19eff0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.722+0000 7f43a4492640 1 --2- 172.21.15.96:0/3481389968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f439c102a00 0x7f439c075860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.722+0000 7f43a4492640 1 -- 172.21.15.96:0/3481389968 >> 172.21.15.96:0/3481389968 conn(0x7f439c0fdbf0 msgr2=0x7f439c104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:26.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.722+0000 7f43a4492640 1 -- 172.21.15.96:0/3481389968 shutdown_connections 2024-04-25T14:13:26.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.722+0000 7f43a4492640 1 -- 172.21.15.96:0/3481389968 wait complete. 2024-04-25T14:13:26.735 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:26 smithi096 ceph-mon[29207]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T14:13:26.735 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:26 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/3841485710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:13:26.735 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:26 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/983391090' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:13:26.736 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T14:13:26.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.844+0000 7f000bd06640 1 -- 172.21.15.96:0/1124971448 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0004109a40 msgr2=0x7f000410be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.844+0000 7f000bd06640 1 --2- 172.21.15.96:0/1124971448 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0004109a40 0x7f000410be60 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7efffc009ca0 tx=0x7efffc02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.844+0000 7f000bd06640 1 -- 172.21.15.96:0/1124971448 shutdown_connections 2024-04-25T14:13:26.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.844+0000 7f000bd06640 1 --2- 172.21.15.96:0/1124971448 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0004109a40 0x7f000410be60 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.844+0000 7f000bd06640 1 --2- 172.21.15.96:0/1124971448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0004104d40 0x7f0004105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.844+0000 7f000bd06640 1 --2- 172.21.15.96:0/1124971448 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0004103b00 0x7f0004104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.844+0000 7f000bd06640 1 -- 172.21.15.96:0/1124971448 >> 172.21.15.96:0/1124971448 conn(0x7f00040fecf0 msgr2=0x7f00041011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:26.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.844+0000 7f000bd06640 1 -- 172.21.15.96:0/1124971448 shutdown_connections 2024-04-25T14:13:26.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.844+0000 7f000bd06640 1 -- 172.21.15.96:0/1124971448 wait complete. 2024-04-25T14:13:26.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.844+0000 7f000bd06640 1 Processor -- start 2024-04-25T14:13:26.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.845+0000 7f000bd06640 1 -- start start 2024-04-25T14:13:26.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.845+0000 7f000bd06640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0004103b00 0x7f000419a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.845+0000 7f000bd06640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0004104d40 0x7f000419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.845+0000 7f000bd06640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0004109a40 0x7f00041a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.845+0000 7f000bd06640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f000410f630 con 0x7f0004104d40 2024-04-25T14:13:26.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.845+0000 7f000bd06640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f000410f4b0 con 0x7f0004103b00 2024-04-25T14:13:26.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.845+0000 7f000bd06640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f000410f7b0 con 0x7f0004109a40 2024-04-25T14:13:26.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.845+0000 7f00092a1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0004104d40 0x7f000419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.845+0000 7f00092a1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0004104d40 0x7f000419a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51766/0 (socket says 172.21.15.96:51766) 2024-04-25T14:13:26.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.845+0000 7f00092a1640 1 -- 172.21.15.96:0/4064372940 learned_addr learned my addr 172.21.15.96:0/4064372940 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:26.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.845+0000 7f0009aa2640 1 --2- 172.21.15.96:0/4064372940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0004103b00 0x7f000419a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.845+0000 7f00092a1640 1 -- 172.21.15.96:0/4064372940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0004109a40 msgr2=0x7f00041a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.845+0000 7f00092a1640 1 --2- 172.21.15.96:0/4064372940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0004109a40 0x7f00041a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.845+0000 7f00092a1640 1 -- 172.21.15.96:0/4064372940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0004103b00 msgr2=0x7f000419a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.845+0000 7f00092a1640 1 --2- 172.21.15.96:0/4064372940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0004103b00 0x7f000419a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.845+0000 7f00092a1640 1 -- 172.21.15.96:0/4064372940 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00041a1d10 con 0x7f0004104d40 2024-04-25T14:13:26.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.845+0000 7f0009aa2640 1 --2- 172.21.15.96:0/4064372940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0004103b00 0x7f000419a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:26.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.845+0000 7f00092a1640 1 --2- 172.21.15.96:0/4064372940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0004104d40 0x7f000419a710 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7efff400d950 tx=0x7efff400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:26.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.845+0000 7efffaffd640 1 -- 172.21.15.96:0/4064372940 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff4017070 con 0x7f0004104d40 2024-04-25T14:13:26.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.845+0000 7efffaffd640 1 -- 172.21.15.96:0/4064372940 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efff4009c30 con 0x7f0004104d40 2024-04-25T14:13:26.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.845+0000 7f000bd06640 1 -- 172.21.15.96:0/4064372940 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00041a1fa0 con 0x7f0004104d40 2024-04-25T14:13:26.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.845+0000 7efffaffd640 1 -- 172.21.15.96:0/4064372940 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff400f620 con 0x7f0004104d40 2024-04-25T14:13:26.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.846+0000 7f000bd06640 1 -- 172.21.15.96:0/4064372940 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00041a2490 con 0x7f0004104d40 2024-04-25T14:13:26.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.848+0000 7efffaffd640 1 -- 172.21.15.96:0/4064372940 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7efff4010460 con 0x7f0004104d40 2024-04-25T14:13:26.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.848+0000 7f0012bb9740 1 Processor -- start 2024-04-25T14:13:26.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.849+0000 7f0012bb9740 1 -- start start 2024-04-25T14:13:26.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.849+0000 7f0012bb9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564100ef2270 0x564101293370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.849+0000 7f0012bb9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564100ec93b0 0x5641012938f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.849+0000 7f0012bb9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564100ed1590 0x56410129a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.849+0000 7f0012bb9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5641011f8740 con 0x564100ed1590 2024-04-25T14:13:26.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.849+0000 7f0012bb9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5641011943a0 con 0x564100ec93b0 2024-04-25T14:13:26.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.849+0000 7f0012bb9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5641011f4700 con 0x564100ef2270 2024-04-25T14:13:26.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.849+0000 7f00092a1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564100ed1590 0x56410129a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.849+0000 7f00092a1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564100ed1590 0x56410129a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51768/0 (socket says 172.21.15.96:51768) 2024-04-25T14:13:26.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.849+0000 7f00092a1640 1 -- 172.21.15.96:0/4099781163 learned_addr learned my addr 172.21.15.96:0/4099781163 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:26.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.849+0000 7f0009aa2640 1 --2- 172.21.15.96:0/4099781163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564100ec93b0 0x5641012938f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.849+0000 7efffaffd640 1 --2- 172.21.15.96:0/4064372940 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7efff00afaf0 0x7efff00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.849+0000 7f000a2a3640 1 --2- 172.21.15.96:0/4099781163 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564100ef2270 0x564101293370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.849+0000 7f00092a1640 1 -- 172.21.15.96:0/4099781163 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564100ef2270 msgr2=0x564101293370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:26.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.849+0000 7f00092a1640 1 --2- 172.21.15.96:0/4099781163 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564100ef2270 0x564101293370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.849+0000 7f00092a1640 1 -- 172.21.15.96:0/4099781163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564100ec93b0 msgr2=0x5641012938f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:26.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.849+0000 7f00092a1640 1 --2- 172.21.15.96:0/4099781163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564100ec93b0 0x5641012938f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.849+0000 7f00092a1640 1 -- 172.21.15.96:0/4099781163 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641011b0750 con 0x564100ed1590 2024-04-25T14:13:26.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.849+0000 7f000a2a3640 1 --2- 172.21.15.96:0/4099781163 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564100ef2270 0x564101293370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:26.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.849+0000 7f000a2a3640 1 --2- 172.21.15.96:0/4064372940 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7efff00afaf0 0x7efff00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.849+0000 7efffaffd640 1 -- 172.21.15.96:0/4064372940 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7efff40de960 con 0x7f0004104d40 2024-04-25T14:13:26.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.850+0000 7f00092a1640 1 --2- 172.21.15.96:0/4099781163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564100ed1590 0x56410129a7f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7efff404abf0 tx=0x7efff4022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cce12e47f3c99b75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:26.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.850+0000 7effeffff640 1 -- 172.21.15.96:0/4099781163 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff40a6bf0 con 0x564100ed1590 2024-04-25T14:13:26.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.850+0000 7f000a2a3640 1 --2- 172.21.15.96:0/4064372940 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7efff00afaf0 0x7efff00b2000 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7efffc02ade0 tx=0x7efffc0093f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:26.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.850+0000 7effeffff640 1 -- 172.21.15.96:0/4099781163 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efff40a6930 con 0x564100ed1590 2024-04-25T14:13:26.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.850+0000 7effeffff640 1 -- 172.21.15.96:0/4099781163 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff40a6590 con 0x564100ed1590 2024-04-25T14:13:26.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.851+0000 7f0012bb9740 1 -- 172.21.15.96:0/4099781163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564100ed1590 msgr2=0x56410129a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.851+0000 7f0012bb9740 1 --2- 172.21.15.96:0/4099781163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564100ed1590 0x56410129a7f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7efff404abf0 tx=0x7efff4022000 comp rx=0 tx=0).stop 2024-04-25T14:13:26.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.851+0000 7f0012bb9740 1 -- 172.21.15.96:0/4099781163 shutdown_connections 2024-04-25T14:13:26.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.851+0000 7f0012bb9740 1 --2- 172.21.15.96:0/4099781163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564100ed1590 0x56410129a7f0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.851+0000 7f0012bb9740 1 --2- 172.21.15.96:0/4099781163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564100ec93b0 0x5641012938f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.851+0000 7f0012bb9740 1 --2- 172.21.15.96:0/4099781163 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564100ef2270 0x564101293370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.851+0000 7f0012bb9740 1 -- 172.21.15.96:0/4099781163 >> 172.21.15.96:0/4099781163 conn(0x564100ee63c0 msgr2=0x56410128ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:26.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.851+0000 7f0012bb9740 1 -- 172.21.15.96:0/4099781163 shutdown_connections 2024-04-25T14:13:26.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.851+0000 7f0012bb9740 1 -- 172.21.15.96:0/4099781163 wait complete. 2024-04-25T14:13:26.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.851+0000 7f0012bb9740 1 -- start start 2024-04-25T14:13:26.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.851+0000 7f0012bb9740 1 Processor -- start 2024-04-25T14:13:26.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.852+0000 7f0012bb9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564100ec93b0 0x56410129d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.852+0000 7f0012bb9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564100ed1590 0x56410129d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.852+0000 7f0012bb9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564100ef2270 0x5641012a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.852+0000 7f0012bb9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56410129b1e0 con 0x564100ef2270 2024-04-25T14:13:26.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.852+0000 7f0012bb9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56410117ffe0 con 0x564100ec93b0 2024-04-25T14:13:26.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.852+0000 7f0012bb9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56410129b360 con 0x564100ed1590 2024-04-25T14:13:26.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.852+0000 7f0012bb9740 10 client.0 _ll_register_callbacks cb 0x564101196150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:13:26.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.852+0000 7f000a2a3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564100ef2270 0x5641012a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.852+0000 7f000a2a3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564100ef2270 0x5641012a46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51784/0 (socket says 172.21.15.96:51784) 2024-04-25T14:13:26.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.852+0000 7f000a2a3640 1 -- 172.21.15.96:0/2809994775 learned_addr learned my addr 172.21.15.96:0/2809994775 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:26.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.852+0000 7f0009aa2640 1 --2- 172.21.15.96:0/2809994775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564100ec93b0 0x56410129d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.852+0000 7f000a2a3640 1 -- 172.21.15.96:0/2809994775 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564100ed1590 msgr2=0x56410129d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:26.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.852+0000 7f000a2a3640 1 --2- 172.21.15.96:0/2809994775 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564100ed1590 0x56410129d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.852+0000 7f000a2a3640 1 -- 172.21.15.96:0/2809994775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564100ec93b0 msgr2=0x56410129d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:26.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.852+0000 7f000a2a3640 1 --2- 172.21.15.96:0/2809994775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564100ec93b0 0x56410129d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.852+0000 7f000a2a3640 1 -- 172.21.15.96:0/2809994775 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564100f0a2d0 con 0x564100ef2270 2024-04-25T14:13:26.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.852+0000 7f0009aa2640 1 --2- 172.21.15.96:0/2809994775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564100ec93b0 0x56410129d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:26.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.852+0000 7f000a2a3640 1 --2- 172.21.15.96:0/2809994775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564100ef2270 0x5641012a46a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7efffc02f430 tx=0x7efffc0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a433373a2c135a62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:26.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.852+0000 7effed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:13:26.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.852+0000 7effed7fa640 1 -- 172.21.15.96:0/2809994775 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efffc04b070 con 0x564100ef2270 2024-04-25T14:13:26.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.852+0000 7effed7fa640 1 -- 172.21.15.96:0/2809994775 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7efffc030a90 con 0x564100ef2270 2024-04-25T14:13:26.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.852+0000 7f0012bb9740 1 -- 172.21.15.96:0/2809994775 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efff40a6930 con 0x564100ef2270 2024-04-25T14:13:26.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.852+0000 7effed7fa640 1 -- 172.21.15.96:0/2809994775 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efffc056070 con 0x564100ef2270 2024-04-25T14:13:26.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.853+0000 7f0012bb9740 10 client.15342 fetch_fsmap learned FSMap version 25 2024-04-25T14:13:26.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.853+0000 7f0012bb9740 1 -- 172.21.15.96:0/2809994775 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564100fd46b0 con 0x564100ef2270 2024-04-25T14:13:26.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.853+0000 7effed7fa640 1 -- 172.21.15.96:0/2809994775 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efff40a6930 con 0x564100ef2270 2024-04-25T14:13:26.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.853+0000 7effed7fa640 1 -- 172.21.15.96:0/2809994775 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7efffc047420 con 0x564100ef2270 2024-04-25T14:13:26.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.853+0000 7f0012bb9740 10 client.15342 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T14:13:26.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.853+0000 7f0012bb9740 10 client.15342 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T14:13:26.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.853+0000 7f0012bb9740 20 client.15342 populate_metadata read hostname 'smithi096' 2024-04-25T14:13:26.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.853+0000 7f0012bb9740 1 --2- 172.21.15.96:0/2809994775 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x564100eb0e40 0x5641012aaf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.853+0000 7f0012bb9740 4 client.15342 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:13:26.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.853+0000 7f0012bb9740 1 -- 172.21.15.96:0/2809994775 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564101206350 con 0x564100eb0e40 2024-04-25T14:13:26.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.853+0000 7f0009aa2640 1 --2- 172.21.15.96:0/2809994775 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x564100eb0e40 0x5641012aaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.854+0000 7effed7fa640 10 client.15342 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:13:26.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.854+0000 7f0009aa2640 1 --2- 172.21.15.96:0/2809994775 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x564100eb0e40 0x5641012aaf20 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d144dc503aa80645 server_cookie=d352d610f97ffd2b in_seq=0 out_seq=0 2024-04-25T14:13:26.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.855+0000 7effed7fa640 1 -- 172.21.15.96:0/2809994775 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564101206350 con 0x564100eb0e40 2024-04-25T14:13:26.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.855+0000 7effed7fa640 10 client.15342 handle_command_reply: tid=0 2024-04-25T14:13:26.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.855+0000 7f0012bb9740 1 client.15342 shutdown 2024-04-25T14:13:26.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.855+0000 7f0012bb9740 1 -- 172.21.15.96:0/2809994775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564100ef2270 msgr2=0x5641012a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.855+0000 7f0012bb9740 1 --2- 172.21.15.96:0/2809994775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564100ef2270 0x5641012a46a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7efffc02f430 tx=0x7efffc0095c0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.855+0000 7f0012bb9740 1 -- 172.21.15.96:0/2809994775 shutdown_connections 2024-04-25T14:13:26.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.855+0000 7f0012bb9740 1 --2- 172.21.15.96:0/2809994775 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x564100eb0e40 0x5641012aaf20 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.855+0000 7f0012bb9740 1 --2- 172.21.15.96:0/2809994775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564100ef2270 0x5641012a46a0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.855+0000 7f0012bb9740 1 --2- 172.21.15.96:0/2809994775 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564100ed1590 0x56410129d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.855+0000 7f0012bb9740 1 --2- 172.21.15.96:0/2809994775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564100ec93b0 0x56410129d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.855+0000 7effed7fa640 0 client.15342 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:13:26.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.855+0000 7f0012bb9740 1 -- 172.21.15.96:0/2809994775 >> 172.21.15.96:0/2809994775 conn(0x564100ee63c0 msgr2=0x56410128dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:26.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.855+0000 7f0012bb9740 1 -- 172.21.15.96:0/2809994775 shutdown_connections 2024-04-25T14:13:26.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.855+0000 7f0012bb9740 1 -- 172.21.15.96:0/2809994775 wait complete. 2024-04-25T14:13:26.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.855+0000 7f0012bb9740 20 client.15342 trim_cache size 0 max 16384 2024-04-25T14:13:26.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.873+0000 7f0012bb9740 1 Processor -- start 2024-04-25T14:13:26.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.873+0000 7f0012bb9740 1 -- start start 2024-04-25T14:13:26.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.874+0000 7f0012bb9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564100ec93b0 0x56410129b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.874+0000 7f0012bb9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564100ed1590 0x56410129bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.874+0000 7f0012bb9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564100ef2270 0x56410128e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.874+0000 7f0012bb9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5641012a9260 con 0x564100ef2270 2024-04-25T14:13:26.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.874+0000 7f0012bb9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5641012a9930 con 0x564100ec93b0 2024-04-25T14:13:26.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.874+0000 7f0012bb9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5641012aa000 con 0x564100ed1590 2024-04-25T14:13:26.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.874+0000 7f000a2a3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564100ef2270 0x56410128e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.874+0000 7f000a2a3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564100ef2270 0x56410128e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51800/0 (socket says 172.21.15.96:51800) 2024-04-25T14:13:26.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.874+0000 7f000a2a3640 1 -- 172.21.15.96:0/88041779 learned_addr learned my addr 172.21.15.96:0/88041779 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:26.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.874+0000 7f00092a1640 1 --2- 172.21.15.96:0/88041779 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564100ed1590 0x56410129bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.874+0000 7f000a2a3640 1 -- 172.21.15.96:0/88041779 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564100ed1590 msgr2=0x56410129bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:26.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.874+0000 7f000a2a3640 1 --2- 172.21.15.96:0/88041779 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564100ed1590 0x56410129bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.874+0000 7f000a2a3640 1 -- 172.21.15.96:0/88041779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564100ec93b0 msgr2=0x56410129b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:26.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.874+0000 7f0009aa2640 1 --2- 172.21.15.96:0/88041779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564100ec93b0 0x56410129b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.874+0000 7f000a2a3640 1 --2- 172.21.15.96:0/88041779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564100ec93b0 0x56410129b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.874+0000 7f000a2a3640 1 -- 172.21.15.96:0/88041779 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564101292f20 con 0x564100ef2270 2024-04-25T14:13:26.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.874+0000 7f0009aa2640 1 --2- 172.21.15.96:0/88041779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564100ec93b0 0x56410129b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:26.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.874+0000 7f000a2a3640 1 --2- 172.21.15.96:0/88041779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564100ef2270 0x56410128e6f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7efffc047840 tx=0x7efffc048fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c75f38f15bd4204 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:26.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.874+0000 7effef7fe640 1 -- 172.21.15.96:0/88041779 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efffc04b040 con 0x564100ef2270 2024-04-25T14:13:26.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.874+0000 7effef7fe640 1 -- 172.21.15.96:0/88041779 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efffc036040 con 0x564100ef2270 2024-04-25T14:13:26.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.874+0000 7effef7fe640 1 -- 172.21.15.96:0/88041779 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efffc041070 con 0x564100ef2270 2024-04-25T14:13:26.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.875+0000 7f0012bb9740 1 -- 172.21.15.96:0/88041779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564100ef2270 msgr2=0x56410128e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.875+0000 7f0012bb9740 1 --2- 172.21.15.96:0/88041779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564100ef2270 0x56410128e6f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7efffc047840 tx=0x7efffc048fc0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.875+0000 7f0012bb9740 1 -- 172.21.15.96:0/88041779 shutdown_connections 2024-04-25T14:13:26.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.875+0000 7f0012bb9740 1 --2- 172.21.15.96:0/88041779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564100ef2270 0x56410128e6f0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.875+0000 7f0012bb9740 1 --2- 172.21.15.96:0/88041779 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564100ed1590 0x56410129bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.875+0000 7f0012bb9740 1 --2- 172.21.15.96:0/88041779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564100ec93b0 0x56410129b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.875+0000 7f0012bb9740 1 -- 172.21.15.96:0/88041779 >> 172.21.15.96:0/88041779 conn(0x564100ee63c0 msgr2=0x5641012976e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:26.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.875+0000 7f0012bb9740 1 -- 172.21.15.96:0/88041779 shutdown_connections 2024-04-25T14:13:26.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.875+0000 7f0012bb9740 1 -- 172.21.15.96:0/88041779 wait complete. 2024-04-25T14:13:26.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.875+0000 7f0012bb9740 1 -- start start 2024-04-25T14:13:26.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.875+0000 7f0012bb9740 1 Processor -- start 2024-04-25T14:13:26.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.876+0000 7f0012bb9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564100ec93b0 0x56410129f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.876+0000 7f0012bb9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564100ed1590 0x56410129fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.876+0000 7f0012bb9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564100ef2270 0x5641012a2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.876+0000 7f0012bb9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5641012aada0 con 0x564100ec93b0 2024-04-25T14:13:26.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.876+0000 7f0012bb9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5641012aa6d0 con 0x564100ef2270 2024-04-25T14:13:26.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.876+0000 7f0012bb9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5641012ab470 con 0x564100ed1590 2024-04-25T14:13:26.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.876+0000 7f0012bb9740 10 client.0 _ll_register_callbacks cb 0x7efffc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:13:26.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.876+0000 7f0009aa2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564100ec93b0 0x56410129f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.876+0000 7f0009aa2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564100ec93b0 0x56410129f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51812/0 (socket says 172.21.15.96:51812) 2024-04-25T14:13:26.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.876+0000 7f0009aa2640 1 -- 172.21.15.96:0/4268061506 learned_addr learned my addr 172.21.15.96:0/4268061506 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:26.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.876+0000 7f00092a1640 1 --2- 172.21.15.96:0/4268061506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564100ed1590 0x56410129fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.876+0000 7f000a2a3640 1 --2- 172.21.15.96:0/4268061506 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564100ef2270 0x5641012a2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.876+0000 7f0009aa2640 1 -- 172.21.15.96:0/4268061506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564100ed1590 msgr2=0x56410129fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:26.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.876+0000 7f0009aa2640 1 --2- 172.21.15.96:0/4268061506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564100ed1590 0x56410129fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.876+0000 7f0009aa2640 1 -- 172.21.15.96:0/4268061506 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564100ef2270 msgr2=0x5641012a2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:26.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.876+0000 7f0009aa2640 1 --2- 172.21.15.96:0/4268061506 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564100ef2270 0x5641012a2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.876+0000 7f0009aa2640 1 -- 172.21.15.96:0/4268061506 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641012a3290 con 0x564100ec93b0 2024-04-25T14:13:26.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.876+0000 7f00092a1640 1 --2- 172.21.15.96:0/4268061506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564100ed1590 0x56410129fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:26.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.876+0000 7f000a2a3640 1 --2- 172.21.15.96:0/4268061506 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564100ef2270 0x5641012a2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:26.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.876+0000 7f0009aa2640 1 --2- 172.21.15.96:0/4268061506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564100ec93b0 0x56410129f850 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f0000005f00 tx=0x7f00000118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f235c5a487b219a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:26.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.876+0000 7effed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:13:26.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.876+0000 7effed7fa640 1 -- 172.21.15.96:0/4268061506 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000001d420 con 0x564100ec93b0 2024-04-25T14:13:26.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.876+0000 7effed7fa640 1 -- 172.21.15.96:0/4268061506 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f000001d5c0 con 0x564100ec93b0 2024-04-25T14:13:26.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.876+0000 7f0012bb9740 1 -- 172.21.15.96:0/4268061506 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efffc036040 con 0x564100ec93b0 2024-04-25T14:13:26.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.877+0000 7effed7fa640 1 -- 172.21.15.96:0/4268061506 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f000001d7c0 con 0x564100ec93b0 2024-04-25T14:13:26.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.877+0000 7f0012bb9740 10 client.15348 fetch_fsmap learned FSMap version 25 2024-04-25T14:13:26.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.877+0000 7f0012bb9740 1 -- 172.21.15.96:0/4268061506 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5641013140c0 con 0x564100ec93b0 2024-04-25T14:13:26.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.877+0000 7effed7fa640 1 -- 172.21.15.96:0/4268061506 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efffc036040 con 0x564100ec93b0 2024-04-25T14:13:26.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.877+0000 7effed7fa640 1 -- 172.21.15.96:0/4268061506 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f0000010490 con 0x564100ec93b0 2024-04-25T14:13:26.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.877+0000 7f0012bb9740 10 client.15348 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T14:13:26.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.877+0000 7f0012bb9740 10 client.15348 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T14:13:26.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.877+0000 7f0012bb9740 20 client.15348 populate_metadata read hostname 'smithi096' 2024-04-25T14:13:26.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.877+0000 7f0012bb9740 1 --2- 172.21.15.96:0/4268061506 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x564100e86280 0x564101317a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.877+0000 7f0012bb9740 4 client.15348 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T14:13:26.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.877+0000 7f0012bb9740 1 -- 172.21.15.96:0/4268061506 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "status"}) -- 0x5641012001c0 con 0x564100e86280 2024-04-25T14:13:26.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.877+0000 7f00092a1640 1 --2- 172.21.15.96:0/4268061506 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x564100e86280 0x564101317a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.878+0000 7effed7fa640 10 client.15348 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:13:26.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.878+0000 7f00092a1640 1 --2- 172.21.15.96:0/4268061506 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x564100e86280 0x564101317a00 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d5dc99c7b49c9c server_cookie=f2759efbc65eb3af in_seq=0 out_seq=0 2024-04-25T14:13:26.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.878+0000 7effed7fa640 1 -- 172.21.15.96:0/4268061506 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5641012001c0 con 0x564100e86280 2024-04-25T14:13:26.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.878+0000 7effed7fa640 10 client.15348 handle_command_reply: tid=0 2024-04-25T14:13:26.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.878+0000 7f0012bb9740 1 client.15348 shutdown 2024-04-25T14:13:26.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.878+0000 7f0012bb9740 1 -- 172.21.15.96:0/4268061506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564100ec93b0 msgr2=0x56410129f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.878+0000 7f0012bb9740 1 --2- 172.21.15.96:0/4268061506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564100ec93b0 0x56410129f850 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f0000005f00 tx=0x7f00000118d0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.878+0000 7f0012bb9740 1 -- 172.21.15.96:0/4268061506 shutdown_connections 2024-04-25T14:13:26.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.878+0000 7f0012bb9740 1 --2- 172.21.15.96:0/4268061506 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564100ef2270 0x5641012a2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.878+0000 7f0012bb9740 1 --2- 172.21.15.96:0/4268061506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564100ed1590 0x56410129fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.878+0000 7f0012bb9740 1 --2- 172.21.15.96:0/4268061506 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x564100e86280 0x564101317a00 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.878+0000 7f0012bb9740 1 --2- 172.21.15.96:0/4268061506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564100ec93b0 0x56410129f850 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.878+0000 7effed7fa640 0 client.15348 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:13:26.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.878+0000 7f0012bb9740 1 -- 172.21.15.96:0/4268061506 >> 172.21.15.96:0/4268061506 conn(0x564100ee63c0 msgr2=0x56410129b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:26.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.878+0000 7f0012bb9740 1 -- 172.21.15.96:0/4268061506 shutdown_connections 2024-04-25T14:13:26.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.878+0000 7f0012bb9740 1 -- 172.21.15.96:0/4268061506 wait complete. 2024-04-25T14:13:26.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.878+0000 7f0012bb9740 20 client.15348 trim_cache size 0 max 16384 2024-04-25T14:13:26.885 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T14:13:26.885 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "59334182-030c-11ef-bc93-c7b262605968", 2024-04-25T14:13:26.885 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 0, 2024-04-25T14:13:26.885 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 24536, 2024-04-25T14:13:26.885 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-04-25T14:13:26.885 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-04-25T14:13:26.886 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-04-25T14:13:26.886 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 117.33507052900001, 2024-04-25T14:13:26.886 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 25, 2024-04-25T14:13:26.886 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 78, 2024-04-25T14:13:26.886 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 76, 2024-04-25T14:13:26.886 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 123.07288779300001 2024-04-25T14:13:26.886 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T14:13:26.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.879+0000 7f000bd06640 1 -- 172.21.15.96:0/4064372940 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7efff00afaf0 msgr2=0x7efff00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.879+0000 7f000bd06640 1 --2- 172.21.15.96:0/4064372940 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7efff00afaf0 0x7efff00b2000 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7efffc02ade0 tx=0x7efffc0093f0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.879+0000 7f000bd06640 1 -- 172.21.15.96:0/4064372940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0004104d40 msgr2=0x7f000419a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.879+0000 7f000bd06640 1 --2- 172.21.15.96:0/4064372940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0004104d40 0x7f000419a710 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7efff400d950 tx=0x7efff400de10 comp rx=0 tx=0).stop 2024-04-25T14:13:26.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.879+0000 7f000bd06640 1 -- 172.21.15.96:0/4064372940 shutdown_connections 2024-04-25T14:13:26.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.879+0000 7f000bd06640 1 --2- 172.21.15.96:0/4064372940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0004109a40 0x7f00041a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.879+0000 7f000bd06640 1 --2- 172.21.15.96:0/4064372940 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7efff00afaf0 0x7efff00b2000 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.879+0000 7f000bd06640 1 --2- 172.21.15.96:0/4064372940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0004104d40 0x7f000419a710 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.879+0000 7f000bd06640 1 --2- 172.21.15.96:0/4064372940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0004103b00 0x7f000419a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.879+0000 7f000bd06640 1 -- 172.21.15.96:0/4064372940 >> 172.21.15.96:0/4064372940 conn(0x7f00040fecf0 msgr2=0x7f0004105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:26.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.879+0000 7f000bd06640 1 -- 172.21.15.96:0/4064372940 shutdown_connections 2024-04-25T14:13:26.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:26.879+0000 7f000bd06640 1 -- 172.21.15.96:0/4064372940 wait complete. 2024-04-25T14:13:26.890 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T14:13:26.900 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T14:13:26.900 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T14:13:26.900 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T14:13:26.900 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T14:13:26.955 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T14:13:26.955 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-25T14:13:26.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:13:26.956 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list 2024-04-25T14:13:26.957 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:26 smithi151 ceph-mon[32990]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T14:13:26.957 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:26 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/3841485710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:13:26.957 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:26 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/983391090' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:13:26.957 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:26 smithi097 ceph-mon[32720]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T14:13:26.957 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:26 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/3841485710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:13:26.957 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:26 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/983391090' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:13:26.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:13:26.972 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link delete ceph-brx 2024-04-25T14:13:27.037 INFO:teuthology.orchestra.run.smithi096.stderr:Cannot find device "ceph-brx" 2024-04-25T14:13:27.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:13:27.038 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-04-25T14:13:27.039 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T14:13:27.039 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T14:13:27.039 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-25T14:13:27.316 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:13:28.774 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:28 smithi096 ceph-mon[29207]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-04-25T14:13:28.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:28 smithi097 ceph-mon[32720]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-04-25T14:13:28.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:28 smithi151 ceph-mon[32990]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-04-25T14:13:29.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.175+0000 7f708db9a700 1 -- 172.21.15.96:0/335047117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f70881033b0 msgr2=0x7f70881037c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:29.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.175+0000 7f708db9a700 1 --2- 172.21.15.96:0/335047117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f70881033b0 0x7f70881037c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f70780099c0 tx=0x7f7078009cd0 comp rx=0 tx=0).stop 2024-04-25T14:13:29.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.175+0000 7f708db9a700 1 -- 172.21.15.96:0/335047117 shutdown_connections 2024-04-25T14:13:29.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.175+0000 7f708db9a700 1 --2- 172.21.15.96:0/335047117 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7088108630 0x7f708810ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:29.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.175+0000 7f708db9a700 1 --2- 172.21.15.96:0/335047117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f70881033b0 0x7f70881037c0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:29.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.175+0000 7f708db9a700 1 --2- 172.21.15.96:0/335047117 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7088107ce0 0x7f70881080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:29.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.175+0000 7f708db9a700 1 -- 172.21.15.96:0/335047117 >> 172.21.15.96:0/335047117 conn(0x7f70880fee00 msgr2=0x7f7088101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:29.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.175+0000 7f708db9a700 1 -- 172.21.15.96:0/335047117 shutdown_connections 2024-04-25T14:13:29.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.175+0000 7f708db9a700 1 -- 172.21.15.96:0/335047117 wait complete. 2024-04-25T14:13:29.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.176+0000 7f708db9a700 1 Processor -- start 2024-04-25T14:13:29.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.176+0000 7f708db9a700 1 -- start start 2024-04-25T14:13:29.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.176+0000 7f708db9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70881033b0 0x7f7088199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:29.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.176+0000 7f708db9a700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7088107ce0 0x7f7088199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:29.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.176+0000 7f708db9a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7088108630 0x7f708819a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:29.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.176+0000 7f708db9a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7088113260 con 0x7f7088108630 2024-04-25T14:13:29.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.176+0000 7f708db9a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f70881130e0 con 0x7f70881033b0 2024-04-25T14:13:29.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.176+0000 7f708db9a700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f70881133e0 con 0x7f7088107ce0 2024-04-25T14:13:29.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.177+0000 7f7087fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7088108630 0x7f708819a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:29.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.177+0000 7f7087fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7088108630 0x7f708819a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51828/0 (socket says 172.21.15.96:51828) 2024-04-25T14:13:29.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.177+0000 7f7087fff700 1 -- 172.21.15.96:0/2604527194 learned_addr learned my addr 172.21.15.96:0/2604527194 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:29.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.177+0000 7f70877fe700 1 --2- 172.21.15.96:0/2604527194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70881033b0 0x7f7088199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:29.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.177+0000 7f707effd700 1 --2- 172.21.15.96:0/2604527194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7088107ce0 0x7f7088199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:29.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.177+0000 7f7087fff700 1 -- 172.21.15.96:0/2604527194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7088107ce0 msgr2=0x7f7088199eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:29.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.177+0000 7f7087fff700 1 --2- 172.21.15.96:0/2604527194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7088107ce0 0x7f7088199eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:29.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.177+0000 7f7087fff700 1 -- 172.21.15.96:0/2604527194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70881033b0 msgr2=0x7f7088199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:29.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.177+0000 7f7087fff700 1 --2- 172.21.15.96:0/2604527194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70881033b0 0x7f7088199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:29.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.177+0000 7f7087fff700 1 -- 172.21.15.96:0/2604527194 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70881a8ff0 con 0x7f7088108630 2024-04-25T14:13:29.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.177+0000 7f7087fff700 1 --2- 172.21.15.96:0/2604527194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7088108630 0x7f708819a3f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7080007f10 tx=0x7f708000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:29.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.177+0000 7f70857fa700 1 -- 172.21.15.96:0/2604527194 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7080004bb0 con 0x7f7088108630 2024-04-25T14:13:29.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.177+0000 7f708db9a700 1 -- 172.21.15.96:0/2604527194 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70881a92e0 con 0x7f7088108630 2024-04-25T14:13:29.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.177+0000 7f70857fa700 1 -- 172.21.15.96:0/2604527194 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7080004d50 con 0x7f7088108630 2024-04-25T14:13:29.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.177+0000 7f70857fa700 1 -- 172.21.15.96:0/2604527194 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7080005a30 con 0x7f7088108630 2024-04-25T14:13:29.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.177+0000 7f708db9a700 1 -- 172.21.15.96:0/2604527194 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70881a97d0 con 0x7f7088108630 2024-04-25T14:13:29.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.179+0000 7f708db9a700 1 -- 172.21.15.96:0/2604527194 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7088104860 con 0x7f7088108630 2024-04-25T14:13:29.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.181+0000 7f70857fa700 1 -- 172.21.15.96:0/2604527194 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f708001a020 con 0x7f7088108630 2024-04-25T14:13:29.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.181+0000 7f70857fa700 1 --2- 172.21.15.96:0/2604527194 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f70740afb80 0x7f70740b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:29.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.181+0000 7f70877fe700 1 --2- 172.21.15.96:0/2604527194 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f70740afb80 0x7f70740b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:29.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.181+0000 7f70857fa700 1 -- 172.21.15.96:0/2604527194 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f70800de310 con 0x7f7088108630 2024-04-25T14:13:29.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.182+0000 7f70877fe700 1 --2- 172.21.15.96:0/2604527194 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f70740afb80 0x7f70740b2090 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7078009990 tx=0x7f707801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:29.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.187+0000 7f70857fa700 1 -- 172.21.15.96:0/2604527194 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7080010070 con 0x7f7088108630 2024-04-25T14:13:29.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.439+0000 7f708db9a700 1 -- 172.21.15.96:0/2604527194 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f708806c500 con 0x7f7088108630 2024-04-25T14:13:29.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.439+0000 7f70857fa700 1 -- 172.21.15.96:0/2604527194 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+73 (secure 0 0 0) 0x7f70800a6ed0 con 0x7f7088108630 2024-04-25T14:13:29.440 INFO:teuthology.orchestra.run.smithi096.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T14:13:29.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.442+0000 7f708db9a700 1 -- 172.21.15.96:0/2604527194 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f70740afb80 msgr2=0x7f70740b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:29.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.442+0000 7f708db9a700 1 --2- 172.21.15.96:0/2604527194 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f70740afb80 0x7f70740b2090 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7078009990 tx=0x7f707801a040 comp rx=0 tx=0).stop 2024-04-25T14:13:29.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.442+0000 7f708db9a700 1 -- 172.21.15.96:0/2604527194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7088108630 msgr2=0x7f708819a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:29.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.442+0000 7f708db9a700 1 --2- 172.21.15.96:0/2604527194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7088108630 0x7f708819a3f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7080007f10 tx=0x7f708000d3b0 comp rx=0 tx=0).stop 2024-04-25T14:13:29.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.442+0000 7f708db9a700 1 -- 172.21.15.96:0/2604527194 shutdown_connections 2024-04-25T14:13:29.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.442+0000 7f708db9a700 1 --2- 172.21.15.96:0/2604527194 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f70740afb80 0x7f70740b2090 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:29.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.442+0000 7f708db9a700 1 --2- 172.21.15.96:0/2604527194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7088108630 0x7f708819a3f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:29.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.442+0000 7f708db9a700 1 --2- 172.21.15.96:0/2604527194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70881033b0 0x7f7088199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:29.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.442+0000 7f708db9a700 1 --2- 172.21.15.96:0/2604527194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7088107ce0 0x7f7088199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:29.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.442+0000 7f708db9a700 1 -- 172.21.15.96:0/2604527194 >> 172.21.15.96:0/2604527194 conn(0x7f70880fee00 msgr2=0x7f7088109590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:29.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.442+0000 7f708db9a700 1 -- 172.21.15.96:0/2604527194 shutdown_connections 2024-04-25T14:13:29.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:29.442+0000 7f708db9a700 1 -- 172.21.15.96:0/2604527194 wait complete. 2024-04-25T14:13:29.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:29 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/2604527194' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T14:13:29.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:29 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/2604527194' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T14:13:29.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:29 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/2604527194' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T14:13:30.151 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T14:13:30.152 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T14:13:30.152 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi096.front.sepia.ceph.com 2024-04-25T14:13:30.152 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T14:13:30.152 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T14:13:30.152 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T14:13:30.152 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T14:13:30.152 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T14:13:30.152 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T14:13:30.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:13:30.153 DEBUG:teuthology.orchestra.run.smithi096:> ip addr 2024-04-25T14:13:30.168 INFO:teuthology.orchestra.run.smithi096.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T14:13:30.168 INFO:teuthology.orchestra.run.smithi096.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T14:13:30.168 INFO:teuthology.orchestra.run.smithi096.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T14:13:30.168 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:13:30.168 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 ::1/128 scope host 2024-04-25T14:13:30.168 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:13:30.168 INFO:teuthology.orchestra.run.smithi096.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T14:13:30.168 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:d8:d9:bc brd ff:ff:ff:ff:ff:ff 2024-04-25T14:13:30.168 INFO:teuthology.orchestra.run.smithi096.stdout: altname enp7s0f0 2024-04-25T14:13:30.169 INFO:teuthology.orchestra.run.smithi096.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T14:13:30.169 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:d8:d9:bd brd ff:ff:ff:ff:ff:ff 2024-04-25T14:13:30.169 INFO:teuthology.orchestra.run.smithi096.stdout: altname enp7s0f1 2024-04-25T14:13:30.169 INFO:teuthology.orchestra.run.smithi096.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T14:13:30.169 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:8f:cc:0c brd ff:ff:ff:ff:ff:ff 2024-04-25T14:13:30.169 INFO:teuthology.orchestra.run.smithi096.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T14:13:30.169 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:8f:cc:0d brd ff:ff:ff:ff:ff:ff 2024-04-25T14:13:30.169 INFO:teuthology.orchestra.run.smithi096.stdout: inet 172.21.15.96/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T14:13:30.169 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft 42093sec preferred_lft 42093sec 2024-04-25T14:13:30.169 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::90f7:1abc:dbc0:da5a/64 scope link noprefixroute 2024-04-25T14:13:30.169 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:13:30.170 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T14:13:30.170 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:13:30.170 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-04-25T14:13:30.170 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link add name ceph-brx type bridge 2024-04-25T14:13:30.170 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip addr flush dev ceph-brx 2024-04-25T14:13:30.170 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link set ceph-brx up 2024-04-25T14:13:30.170 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T14:13:30.170 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-04-25T14:13:30.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:13:30.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:13:30.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:13:30.355 DEBUG:teuthology.orchestra.run.smithi096:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T14:13:30.379 INFO:teuthology.orchestra.run.smithi096.stdout:1 2024-04-25T14:13:30.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:13:30.381 DEBUG:teuthology.orchestra.run.smithi096:> route 2024-04-25T14:13:30.449 INFO:teuthology.orchestra.run.smithi096.stdout:Kernel IP routing table 2024-04-25T14:13:30.449 INFO:teuthology.orchestra.run.smithi096.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T14:13:30.449 INFO:teuthology.orchestra.run.smithi096.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T14:13:30.449 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T14:13:30.449 INFO:teuthology.orchestra.run.smithi096.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T14:13:30.449 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:13:30.449 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-04-25T14:13:30.450 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T14:13:30.450 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T14:13:30.450 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T14:13:30.450 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-04-25T14:13:30.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:13:30.640 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:30 smithi096 ceph-mon[29207]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T14:13:30.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:13:30.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:13:30.727 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list 2024-04-25T14:13:30.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:13:30.747 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list-id 2024-04-25T14:13:30.796 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:13:30.797 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-04-25T14:13:30.797 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T14:13:30.797 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T14:13:30.797 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-04-25T14:13:30.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:13:30.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:13:30.912 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T14:13:30.912 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:13:30.912 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-04-25T14:13:30.912 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T14:13:30.913 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T14:13:30.913 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T14:13:30.913 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T14:13:30.913 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T14:13:30.913 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-04-25T14:13:30.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:30 smithi097 ceph-mon[32720]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T14:13:30.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:30 smithi151 ceph-mon[32990]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T14:13:30.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:13:31.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:13:31.054 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:13:31.054 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-04-25T14:13:31.054 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link set brx.0 up 2024-04-25T14:13:31.054 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T14:13:31.054 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-04-25T14:13:31.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:13:31.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:13:31.129 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:13:31.129 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T14:13:31.143 INFO:teuthology.orchestra.run.smithi096.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T14:13:31.143 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:13:31.144 DEBUG:teuthology.orchestra.run.smithi096:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T14:13:31.194 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-25T14:13:31.195 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-25T14:13:31.195 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/358c30dc-ec4f-409a-a4ed-b22ff1971e3f 2024-04-25T14:13:31.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:13:31.195 DEBUG:teuthology.orchestra.run.smithi096:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/358c30dc-ec4f-409a-a4ed-b22ff1971e3f /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-04-25T14:13:31.291 INFO:teuthology.orchestra.run.smithi096.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-04-25T14:13:31.291 INFO:teuthology.orchestra.run.smithi096.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2024-04-25T14:13:31.291 INFO:teuthology.orchestra.run.smithi096.stdout:invalid new device string format 2024-04-25T14:13:31.407 INFO:teuthology.orchestra.run.smithi096.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-04-25T14:13:31.408 INFO:teuthology.orchestra.run.smithi096.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2024-04-25T14:13:31.408 INFO:teuthology.orchestra.run.smithi096.stdout:invalid new device string format 2024-04-25T14:13:31.408 INFO:teuthology.orchestra.run.smithi096.stdout:mount.ceph: resolved to: "172.21.15.96:3300,172.21.15.97:3300,172.21.15.151:3300" 2024-04-25T14:13:31.408 INFO:teuthology.orchestra.run.smithi096.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.96:3300,172.21.15.97:3300,172.21.15.151:3300:/volumes/qa/sv_1/358c30dc-ec4f-409a-a4ed-b22ff1971e3f 2024-04-25T14:13:31.408 INFO:teuthology.orchestra.run.smithi096.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync,key=0,fsid=59334182-030c-11ef-bc93-c7b262605968" will pass to kernel 2024-04-25T14:13:31.409 INFO:teuthology.orchestra.run.smithi096.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T14:13:31.409 INFO:teuthology.orchestra.run.smithi096.stdout: You just mounted a file system that supports labels which does not 2024-04-25T14:13:31.409 INFO:teuthology.orchestra.run.smithi096.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T14:13:31.409 INFO:teuthology.orchestra.run.smithi096.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T14:13:31.409 INFO:teuthology.orchestra.run.smithi096.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T14:13:31.411 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T14:13:31.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:13:31.411 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T14:13:31.436 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T14:13:31.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T14:13:31.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:13:31.532 INFO:teuthology.orchestra.run.smithi096.stdout:client24977 2024-04-25T14:13:31.533 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T14:13:31.533 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:13:31.549 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T14:13:31.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T14:13:31.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:13:31.647 INFO:teuthology.orchestra.run.smithi096.stdout:client24977 2024-04-25T14:13:31.647 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T14:13:31.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T14:13:31.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:13:31.703 INFO:teuthology.orchestra.run.smithi096.stdout:client24977 2024-04-25T14:13:31.703 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/59334182-030c-11ef-bc93-c7b262605968.client24977/status') 2024-04-25T14:13:31.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:13:31.759 INFO:teuthology.orchestra.run.smithi096.stdout:instance: client.24977 (3)172.21.15.96:0/1030745016 2024-04-25T14:13:31.759 INFO:teuthology.orchestra.run.smithi096.stdout:blocklisted: false 2024-04-25T14:13:31.759 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records in 2024-04-25T14:13:31.760 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records out 2024-04-25T14:13:31.760 INFO:teuthology.orchestra.run.smithi096.stderr:71 bytes copied, 3.0341e-05 s, 2.3 MB/s 2024-04-25T14:13:31.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:13:31.770 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T14:13:31.782 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T14:13:31.783 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi096.front.sepia.ceph.com 2024-04-25T14:13:31.783 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-04-25T14:13:31.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31.891+0000 7f4805beb640 1 -- 172.21.15.96:0/4177754979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4800103c40 msgr2=0x7f4800104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:31.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31.891+0000 7f4805beb640 1 --2- 172.21.15.96:0/4177754979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4800103c40 0x7f4800104050 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f47f4009900 tx=0x7f47f402a410 comp rx=0 tx=0).stop 2024-04-25T14:13:31.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31.891+0000 7f4805beb640 1 -- 172.21.15.96:0/4177754979 shutdown_connections 2024-04-25T14:13:31.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31.891+0000 7f4805beb640 1 --2- 172.21.15.96:0/4177754979 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4800108940 0x7f480010ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:31.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31.891+0000 7f4805beb640 1 --2- 172.21.15.96:0/4177754979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4800103c40 0x7f4800104050 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:31.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31.891+0000 7f4805beb640 1 --2- 172.21.15.96:0/4177754979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4800102a00 0x7f4800103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:31.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31.891+0000 7f4805beb640 1 -- 172.21.15.96:0/4177754979 >> 172.21.15.96:0/4177754979 conn(0x7f48000fdbd0 msgr2=0x7f48001000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:31.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31.891+0000 7f4805beb640 1 -- 172.21.15.96:0/4177754979 shutdown_connections 2024-04-25T14:13:31.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31.891+0000 7f4805beb640 1 -- 172.21.15.96:0/4177754979 wait complete. 2024-04-25T14:13:31.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31.892+0000 7f4805beb640 1 Processor -- start 2024-04-25T14:13:31.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31.892+0000 7f4805beb640 1 -- start start 2024-04-25T14:13:31.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31.892+0000 7f4805beb640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4800102a00 0x7f4800199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:31.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31.892+0000 7f4805beb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4800103c40 0x7f4800199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:31.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31.892+0000 7f4805beb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4800108940 0x7f48001a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:31.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31.892+0000 7f4805beb640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f480010e530 con 0x7f4800108940 2024-04-25T14:13:31.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31.892+0000 7f4805beb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f480010e3b0 con 0x7f4800103c40 2024-04-25T14:13:31.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31.892+0000 7f4805beb640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f480010e6b0 con 0x7f4800102a00 2024-04-25T14:13:31.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31.892+0000 7f47fffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4800108940 0x7f48001a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:31.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31.892+0000 7f47fffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4800108940 0x7f48001a0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40382/0 (socket says 172.21.15.96:40382) 2024-04-25T14:13:31.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31.892+0000 7f47fffff640 1 -- 172.21.15.96:0/938525689 learned_addr learned my addr 172.21.15.96:0/938525689 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:31.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31.892+0000 7f47feffd640 1 --2- 172.21.15.96:0/938525689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4800103c40 0x7f4800199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:31.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31.893+0000 7f47ff7fe640 1 --2- 172.21.15.96:0/938525689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4800102a00 0x7f4800199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:31.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31.893+0000 7f47fffff640 1 -- 172.21.15.96:0/938525689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4800102a00 msgr2=0x7f4800199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:31.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31.893+0000 7f47fffff640 1 --2- 172.21.15.96:0/938525689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4800102a00 0x7f4800199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:31.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31.893+0000 7f47fffff640 1 -- 172.21.15.96:0/938525689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4800103c40 msgr2=0x7f4800199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:31.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31.893+0000 7f47fffff640 1 --2- 172.21.15.96:0/938525689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4800103c40 0x7f4800199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:31.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31.893+0000 7f47fffff640 1 -- 172.21.15.96:0/938525689 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48001a0c70 con 0x7f4800108940 2024-04-25T14:13:31.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31.893+0000 7f47fffff640 1 --2- 172.21.15.96:0/938525689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4800108940 0x7f48001a0600 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f47f800cad0 tx=0x7f47f800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:31.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31.893+0000 7f47fcff9640 1 -- 172.21.15.96:0/938525689 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f800ae20 con 0x7f4800108940 2024-04-25T14:13:31.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31.893+0000 7f47fcff9640 1 -- 172.21.15.96:0/938525689 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47f8005020 con 0x7f4800108940 2024-04-25T14:13:31.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31.893+0000 7f4805beb640 1 -- 172.21.15.96:0/938525689 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48001a0f60 con 0x7f4800108940 2024-04-25T14:13:31.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31.893+0000 7f47fcff9640 1 -- 172.21.15.96:0/938525689 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f800f6b0 con 0x7f4800108940 2024-04-25T14:13:31.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31.893+0000 7f4805beb640 1 -- 172.21.15.96:0/938525689 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48001a13c0 con 0x7f4800108940 2024-04-25T14:13:31.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31.894+0000 7f4805beb640 1 -- 172.21.15.96:0/938525689 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4800050740 con 0x7f4800108940 2024-04-25T14:13:31.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31.896+0000 7f47fcff9640 1 -- 172.21.15.96:0/938525689 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f47f80051d0 con 0x7f4800108940 2024-04-25T14:13:31.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31.896+0000 7f47fcff9640 1 --2- 172.21.15.96:0/938525689 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f47e80afb60 0x7f47e80b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:31.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31.896+0000 7f47ff7fe640 1 --2- 172.21.15.96:0/938525689 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f47e80afb60 0x7f47e80b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:31.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31.897+0000 7f47fcff9640 1 -- 172.21.15.96:0/938525689 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f47f80defc0 con 0x7f4800108940 2024-04-25T14:13:31.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31.897+0000 7f47ff7fe640 1 --2- 172.21.15.96:0/938525689 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f47e80afb60 0x7f47e80b2070 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f47f0005d00 tx=0x7f47f0005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:31.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:31.899+0000 7f47fcff9640 1 -- 172.21.15.96:0/938525689 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f47f8014030 con 0x7f4800108940 2024-04-25T14:13:31.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:31 smithi097 ceph-mon[32720]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T14:13:31.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:31 smithi151 ceph-mon[32990]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T14:13:32.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:31 smithi096 ceph-mon[29207]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T14:13:32.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.118+0000 7f4805beb640 1 -- 172.21.15.96:0/938525689 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f4800107750 con 0x7f4800108940 2024-04-25T14:13:32.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.646+0000 7f47fcff9640 1 -- 172.21.15.96:0/938525689 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v26) ==== 117+0+0 (secure 0 0 0) 0x7f47f80a7c00 con 0x7f4800108940 2024-04-25T14:13:32.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.649+0000 7f4805beb640 1 -- 172.21.15.96:0/938525689 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f47e80afb60 msgr2=0x7f47e80b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:32.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.649+0000 7f4805beb640 1 --2- 172.21.15.96:0/938525689 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f47e80afb60 0x7f47e80b2070 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f47f0005d00 tx=0x7f47f0005c50 comp rx=0 tx=0).stop 2024-04-25T14:13:32.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.649+0000 7f4805beb640 1 -- 172.21.15.96:0/938525689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4800108940 msgr2=0x7f48001a0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:32.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.649+0000 7f4805beb640 1 --2- 172.21.15.96:0/938525689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4800108940 0x7f48001a0600 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f47f800cad0 tx=0x7f47f800cf90 comp rx=0 tx=0).stop 2024-04-25T14:13:32.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.649+0000 7f4805beb640 1 -- 172.21.15.96:0/938525689 shutdown_connections 2024-04-25T14:13:32.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.649+0000 7f4805beb640 1 --2- 172.21.15.96:0/938525689 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f47e80afb60 0x7f47e80b2070 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:32.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.649+0000 7f4805beb640 1 --2- 172.21.15.96:0/938525689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4800108940 0x7f48001a0600 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:32.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.649+0000 7f4805beb640 1 --2- 172.21.15.96:0/938525689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4800103c40 0x7f4800199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:32.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.649+0000 7f4805beb640 1 --2- 172.21.15.96:0/938525689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4800102a00 0x7f4800199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:32.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.649+0000 7f4805beb640 1 -- 172.21.15.96:0/938525689 >> 172.21.15.96:0/938525689 conn(0x7f48000fdbd0 msgr2=0x7f4800104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:32.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.649+0000 7f4805beb640 1 -- 172.21.15.96:0/938525689 shutdown_connections 2024-04-25T14:13:32.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.650+0000 7f4805beb640 1 -- 172.21.15.96:0/938525689 wait complete. 2024-04-25T14:13:32.663 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-25T14:13:32.673 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-04-25T14:13:32.733 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:13:32.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.850+0000 7f4b4359e640 1 -- 172.21.15.96:0/31005488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b44101c70 msgr2=0x7f4b44102080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:32.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.850+0000 7f4b4359e640 1 --2- 172.21.15.96:0/31005488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b44101c70 0x7f4b44102080 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4b34009900 tx=0x7f4b3402a410 comp rx=0 tx=0).stop 2024-04-25T14:13:32.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.850+0000 7f4b4359e640 1 -- 172.21.15.96:0/31005488 shutdown_connections 2024-04-25T14:13:32.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.850+0000 7f4b4359e640 1 --2- 172.21.15.96:0/31005488 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b441095c0 0x7f4b4410bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:32.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.850+0000 7f4b4359e640 1 --2- 172.21.15.96:0/31005488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b44106a50 0x7f4b44108f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:32.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.850+0000 7f4b4359e640 1 --2- 172.21.15.96:0/31005488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b44101c70 0x7f4b44102080 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:32.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.850+0000 7f4b4359e640 1 -- 172.21.15.96:0/31005488 >> 172.21.15.96:0/31005488 conn(0x7f4b440fd1a0 msgr2=0x7f4b440ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:32.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.850+0000 7f4b4359e640 1 -- 172.21.15.96:0/31005488 shutdown_connections 2024-04-25T14:13:32.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.850+0000 7f4b4359e640 1 -- 172.21.15.96:0/31005488 wait complete. 2024-04-25T14:13:32.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.851+0000 7f4b4359e640 1 Processor -- start 2024-04-25T14:13:32.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.851+0000 7f4b4359e640 1 -- start start 2024-04-25T14:13:32.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.851+0000 7f4b4359e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b44101c70 0x7f4b4419a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:32.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.851+0000 7f4b4359e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b44106a50 0x7f4b4419a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:32.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.851+0000 7f4b4359e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b441095c0 0x7f4b441a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:32.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.851+0000 7f4b4359e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4410f400 con 0x7f4b44106a50 2024-04-25T14:13:32.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.851+0000 7f4b4359e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4410f280 con 0x7f4b441095c0 2024-04-25T14:13:32.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.851+0000 7f4b4359e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4410f580 con 0x7f4b44101c70 2024-04-25T14:13:32.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.851+0000 7f4b3bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b44106a50 0x7f4b4419a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:32.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.851+0000 7f4b3bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b44106a50 0x7f4b4419a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40398/0 (socket says 172.21.15.96:40398) 2024-04-25T14:13:32.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.851+0000 7f4b3bfff640 1 -- 172.21.15.96:0/2515948312 learned_addr learned my addr 172.21.15.96:0/2515948312 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:32.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.852+0000 7f4b42d9d640 1 --2- 172.21.15.96:0/2515948312 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b441095c0 0x7f4b441a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:32.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.852+0000 7f4b4259c640 1 --2- 172.21.15.96:0/2515948312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b44101c70 0x7f4b4419a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:32.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.852+0000 7f4b3bfff640 1 -- 172.21.15.96:0/2515948312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b44101c70 msgr2=0x7f4b4419a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:32.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.852+0000 7f4b3bfff640 1 --2- 172.21.15.96:0/2515948312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b44101c70 0x7f4b4419a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:32.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.852+0000 7f4b3bfff640 1 -- 172.21.15.96:0/2515948312 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b441095c0 msgr2=0x7f4b441a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:32.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.852+0000 7f4b3bfff640 1 --2- 172.21.15.96:0/2515948312 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b441095c0 0x7f4b441a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:32.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.852+0000 7f4b3bfff640 1 -- 172.21.15.96:0/2515948312 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b441a1ad0 con 0x7f4b44106a50 2024-04-25T14:13:32.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.852+0000 7f4b3bfff640 1 --2- 172.21.15.96:0/2515948312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b44106a50 0x7f4b4419a580 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c00e9f0 tx=0x7f4b2c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:32.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.852+0000 7f4b3b7fe640 1 -- 172.21.15.96:0/2515948312 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c00cd80 con 0x7f4b44106a50 2024-04-25T14:13:32.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.852+0000 7f4b3b7fe640 1 -- 172.21.15.96:0/2515948312 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b2c0108e0 con 0x7f4b44106a50 2024-04-25T14:13:32.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.852+0000 7f4b4359e640 1 -- 172.21.15.96:0/2515948312 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b441a1dc0 con 0x7f4b44106a50 2024-04-25T14:13:32.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.852+0000 7f4b3b7fe640 1 -- 172.21.15.96:0/2515948312 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c018980 con 0x7f4b44106a50 2024-04-25T14:13:32.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.852+0000 7f4b4359e640 1 -- 172.21.15.96:0/2515948312 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b441a2280 con 0x7f4b44106a50 2024-04-25T14:13:32.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.853+0000 7f4b4359e640 1 -- 172.21.15.96:0/2515948312 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b440fecc0 con 0x7f4b44106a50 2024-04-25T14:13:32.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.856+0000 7f4b3b7fe640 1 -- 172.21.15.96:0/2515948312 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f4b2c010430 con 0x7f4b44106a50 2024-04-25T14:13:32.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.856+0000 7f4b3b7fe640 1 --2- 172.21.15.96:0/2515948312 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f4b300afb60 0x7f4b300b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:32.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.856+0000 7f4b4259c640 1 --2- 172.21.15.96:0/2515948312 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f4b300afb60 0x7f4b300b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:32.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.856+0000 7f4b3b7fe640 1 -- 172.21.15.96:0/2515948312 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f4b2c0df380 con 0x7f4b44106a50 2024-04-25T14:13:32.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.857+0000 7f4b4259c640 1 --2- 172.21.15.96:0/2515948312 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f4b300afb60 0x7f4b300b2070 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4b34009230 tx=0x7f4b34031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:32.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:32.860+0000 7f4b3b7fe640 1 -- 172.21.15.96:0/2515948312 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4b2c0a78f0 con 0x7f4b44106a50 2024-04-25T14:13:32.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:32 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/938525689' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-25T14:13:32.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:32 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/938525689' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-25T14:13:33.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.037+0000 7f4b4359e640 1 -- 172.21.15.96:0/2515948312 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4b440637d0 con 0x7f4b44106a50 2024-04-25T14:13:33.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.039+0000 7f4b3b7fe640 1 -- 172.21.15.96:0/2515948312 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22616 (secure 0 0 0) 0x7f4b2c0abbf0 con 0x7f4b44106a50 2024-04-25T14:13:33.039 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:13:33.039 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":78,"fsid":"59334182-030c-11ef-bc93-c7b262605968","created":"2024-04-25T14:03:25.680745+0000","modified":"2024-04-25T14:12:22.496143+0000","last_up_change":"2024-04-25T14:10:35.894312+0000","last_in_change":"2024-04-25T14:10:15.583440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:06:47.402927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:11:27.434331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:11:27.854004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cf4ba3ac-142e-4e8e-96d4-410d77157eb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6803","nonce":3832506168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6805","nonce":3832506168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832506168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6807","nonce":3832506168}]},"public_addr":"172.21.15.96:6803/3832506168","cluster_addr":"172.21.15.96:6805/3832506168","heartbeat_back_addr":"172.21.15.96:6809/3832506168","heartbeat_front_addr":"172.21.15.96:6807/3832506168","state":["exists","up"]},{"osd":1,"uuid":"866e73b9-4a26-48ba-badf-704fe22f71cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6801","nonce":1083311494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6803","nonce":1083311494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6807","nonce":1083311494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6805","nonce":1083311494}]},"public_addr":"172.21.15.97:6801/1083311494","cluster_addr":"172.21.15.97:6803/1083311494","heartbeat_back_addr":"172.21.15.97:6807/1083311494","heartbeat_front_addr":"172.21.15.97:6805/1083311494","state":["exists","up"]},{"osd":2,"uuid":"45c79ecb-abc4-4dba-bae8-74804e4bc1bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6801","nonce":981902966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6803","nonce":981902966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6807","nonce":981902966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6805","nonce":981902966}]},"public_addr":"172.21.15.151:6801/981902966","cluster_addr":"172.21.15.151:6803/981902966","heartbeat_back_addr":"172.21.15.151:6807/981902966","heartbeat_front_addr":"172.21.15.151:6805/981902966","state":["exists","up"]},{"osd":3,"uuid":"9fc39468-a7ea-4ccf-b365-335ad2ea4c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6811","nonce":2274158748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6813","nonce":2274158748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6817","nonce":2274158748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6815","nonce":2274158748}]},"public_addr":"172.21.15.96:6811/2274158748","cluster_addr":"172.21.15.96:6813/2274158748","heartbeat_back_addr":"172.21.15.96:6817/2274158748","heartbeat_front_addr":"172.21.15.96:6815/2274158748","state":["exists","up"]},{"osd":4,"uuid":"616582f1-12d2-456b-b7b3-e5517800e8a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6809","nonce":3727031962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6811","nonce":3727031962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6815","nonce":3727031962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6813","nonce":3727031962}]},"public_addr":"172.21.15.97:6809/3727031962","cluster_addr":"172.21.15.97:6811/3727031962","heartbeat_back_addr":"172.21.15.97:6815/3727031962","heartbeat_front_addr":"172.21.15.97:6813/3727031962","state":["exists","up"]},{"osd":5,"uuid":"3d277784-ab0f-4c90-81a1-2a4034c52a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6809","nonce":223882546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6811","nonce":223882546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6815","nonce":223882546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6813","nonce":223882546}]},"public_addr":"172.21.15.151:6809/223882546","cluster_addr":"172.21.15.151:6811/223882546","heartbeat_back_addr":"172.21.15.151:6815/223882546","heartbeat_front_addr":"172.21.15.151:6813/223882546","state":["exists","up"]},{"osd":6,"uuid":"9a9ad585-cb1a-4239-9d18-055f59164049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6819","nonce":96197407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6821","nonce":96197407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6825","nonce":96197407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6823","nonce":96197407}]},"public_addr":"172.21.15.96:6819/96197407","cluster_addr":"172.21.15.96:6821/96197407","heartbeat_back_addr":"172.21.15.96:6825/96197407","heartbeat_front_addr":"172.21.15.96:6823/96197407","state":["exists","up"]},{"osd":7,"uuid":"303551c8-6af0-42f2-844f-45bf8db5baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6817","nonce":2247216400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6819","nonce":2247216400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6823","nonce":2247216400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6821","nonce":2247216400}]},"public_addr":"172.21.15.97:6817/2247216400","cluster_addr":"172.21.15.97:6819/2247216400","heartbeat_back_addr":"172.21.15.97:6823/2247216400","heartbeat_front_addr":"172.21.15.97:6821/2247216400","state":["exists","up"]},{"osd":8,"uuid":"81f6cc66-e4ca-430c-b336-b6678d968818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6817","nonce":2549822185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6819","nonce":2549822185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6823","nonce":2549822185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6821","nonce":2549822185}]},"public_addr":"172.21.15.151:6817/2549822185","cluster_addr":"172.21.15.151:6819/2549822185","heartbeat_back_addr":"172.21.15.151:6823/2549822185","heartbeat_front_addr":"172.21.15.151:6821/2549822185","state":["exists","up"]},{"osd":9,"uuid":"29d4da16-feab-4d1c-9d1d-de4f799bd724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6827","nonce":1099399643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6829","nonce":1099399643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6833","nonce":1099399643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6831","nonce":1099399643}]},"public_addr":"172.21.15.96:6827/1099399643","cluster_addr":"172.21.15.96:6829/1099399643","heartbeat_back_addr":"172.21.15.96:6833/1099399643","heartbeat_front_addr":"172.21.15.96:6831/1099399643","state":["exists","up"]},{"osd":10,"uuid":"197fee9b-eee4-4893-be1c-3d57f42250e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6825","nonce":155754520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6827","nonce":155754520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6831","nonce":155754520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6829","nonce":155754520}]},"public_addr":"172.21.15.97:6825/155754520","cluster_addr":"172.21.15.97:6827/155754520","heartbeat_back_addr":"172.21.15.97:6831/155754520","heartbeat_front_addr":"172.21.15.97:6829/155754520","state":["exists","up"]},{"osd":11,"uuid":"9c1d0a83-fc12-41a2-8eb0-80c4abdd2770","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6825","nonce":2027958746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6827","nonce":2027958746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6831","nonce":2027958746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6829","nonce":2027958746}]},"public_addr":"172.21.15.151:6825/2027958746","cluster_addr":"172.21.15.151:6827/2027958746","heartbeat_back_addr":"172.21.15.151:6831/2027958746","heartbeat_front_addr":"172.21.15.151:6829/2027958746","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:05:56.177390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:06:19.917663+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:06:43.608052+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:08.540382+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:33.474072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:58.242306+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:08:24.120442+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:08:49.661064+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:09:15.097271+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:09:41.904505+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:10:08.233252+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:10:33.712296+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/3839807484":"2024-04-26T14:04:21.307574+0000","172.21.15.96:0/1495237551":"2024-04-26T14:04:21.307574+0000","172.21.15.96:0/2793751450":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/129575834":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/2233075139":"2024-04-26T14:04:21.307574+0000","172.21.15.96:6801/2348397481":"2024-04-26T14:03:54.249537+0000","172.21.15.96:6801/433591062":"2024-04-26T14:04:21.307574+0000","172.21.15.96:6800/2348397481":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/3517674924":"2024-04-26T14:03:54.249537+0000","172.21.15.96:6800/433591062":"2024-04-26T14:04:21.307574+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:13:33.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.041+0000 7f4b4359e640 1 -- 172.21.15.96:0/2515948312 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f4b300afb60 msgr2=0x7f4b300b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:33.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.041+0000 7f4b4359e640 1 --2- 172.21.15.96:0/2515948312 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f4b300afb60 0x7f4b300b2070 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4b34009230 tx=0x7f4b34031400 comp rx=0 tx=0).stop 2024-04-25T14:13:33.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.041+0000 7f4b4359e640 1 -- 172.21.15.96:0/2515948312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b44106a50 msgr2=0x7f4b4419a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:33.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.041+0000 7f4b4359e640 1 --2- 172.21.15.96:0/2515948312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b44106a50 0x7f4b4419a580 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c00e9f0 tx=0x7f4b2c00eeb0 comp rx=0 tx=0).stop 2024-04-25T14:13:33.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.041+0000 7f4b4359e640 1 -- 172.21.15.96:0/2515948312 shutdown_connections 2024-04-25T14:13:33.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.041+0000 7f4b4359e640 1 --2- 172.21.15.96:0/2515948312 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b441095c0 0x7f4b441a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:33.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.041+0000 7f4b4359e640 1 --2- 172.21.15.96:0/2515948312 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f4b300afb60 0x7f4b300b2070 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:33.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.041+0000 7f4b4359e640 1 --2- 172.21.15.96:0/2515948312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b44106a50 0x7f4b4419a580 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:33.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.041+0000 7f4b4359e640 1 --2- 172.21.15.96:0/2515948312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b44101c70 0x7f4b4419a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:33.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.041+0000 7f4b4359e640 1 -- 172.21.15.96:0/2515948312 >> 172.21.15.96:0/2515948312 conn(0x7f4b440fd1a0 msgr2=0x7f4b4410a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:33.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.041+0000 7f4b4359e640 1 -- 172.21.15.96:0/2515948312 shutdown_connections 2024-04-25T14:13:33.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.041+0000 7f4b4359e640 1 -- 172.21.15.96:0/2515948312 wait complete. 2024-04-25T14:13:33.051 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:32 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/938525689' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-25T14:13:33.053 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:06:47.402927+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:11:27.434331+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:11:27.854004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:13:33.053 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:13:33.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.165+0000 7f889c084640 1 -- 172.21.15.96:0/3891840086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8894108920 msgr2=0x7f889410ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:33.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.165+0000 7f889c084640 1 --2- 172.21.15.96:0/3891840086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8894108920 0x7f889410ad40 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f888c009ca0 tx=0x7f888c02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:13:33.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.165+0000 7f889c084640 1 -- 172.21.15.96:0/3891840086 shutdown_connections 2024-04-25T14:13:33.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.165+0000 7f889c084640 1 --2- 172.21.15.96:0/3891840086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8894108920 0x7f889410ad40 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:33.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.165+0000 7f889c084640 1 --2- 172.21.15.96:0/3891840086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8894103c20 0x7f8894104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:33.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.165+0000 7f889c084640 1 --2- 172.21.15.96:0/3891840086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88941029e0 0x7f88941036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:33.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.165+0000 7f889c084640 1 -- 172.21.15.96:0/3891840086 >> 172.21.15.96:0/3891840086 conn(0x7f88940fdbd0 msgr2=0x7f8894100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:33.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.165+0000 7f889c084640 1 -- 172.21.15.96:0/3891840086 shutdown_connections 2024-04-25T14:13:33.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.165+0000 7f889c084640 1 -- 172.21.15.96:0/3891840086 wait complete. 2024-04-25T14:13:33.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.165+0000 7f889c084640 1 Processor -- start 2024-04-25T14:13:33.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.166+0000 7f889c084640 1 -- start start 2024-04-25T14:13:33.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.166+0000 7f889c084640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88941029e0 0x7f889419a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:33.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.166+0000 7f889c084640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8894103c20 0x7f889419a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:33.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.166+0000 7f889c084640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8894108920 0x7f88941a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:33.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.166+0000 7f889c084640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f889410f610 con 0x7f8894103c20 2024-04-25T14:13:33.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.166+0000 7f889c084640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f889410f490 con 0x7f88941029e0 2024-04-25T14:13:33.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.166+0000 7f889c084640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f889410f790 con 0x7f8894108920 2024-04-25T14:13:33.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.166+0000 7f889961f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8894103c20 0x7f889419a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:33.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.166+0000 7f8899e20640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88941029e0 0x7f889419a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:33.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.166+0000 7f889961f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8894103c20 0x7f889419a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40420/0 (socket says 172.21.15.96:40420) 2024-04-25T14:13:33.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.166+0000 7f889961f640 1 -- 172.21.15.96:0/1245420034 learned_addr learned my addr 172.21.15.96:0/1245420034 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:33.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.166+0000 7f889a621640 1 --2- 172.21.15.96:0/1245420034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8894108920 0x7f88941a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:33.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.166+0000 7f889961f640 1 -- 172.21.15.96:0/1245420034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8894108920 msgr2=0x7f88941a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:33.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.166+0000 7f889961f640 1 --2- 172.21.15.96:0/1245420034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8894108920 0x7f88941a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:33.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.166+0000 7f889961f640 1 -- 172.21.15.96:0/1245420034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88941029e0 msgr2=0x7f889419a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:33.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.166+0000 7f889961f640 1 --2- 172.21.15.96:0/1245420034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88941029e0 0x7f889419a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:33.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.166+0000 7f889961f640 1 -- 172.21.15.96:0/1245420034 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88941a1d40 con 0x7f8894103c20 2024-04-25T14:13:33.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.166+0000 7f889a621640 1 --2- 172.21.15.96:0/1245420034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8894108920 0x7f88941a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:33.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.166+0000 7f8899e20640 1 --2- 172.21.15.96:0/1245420034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88941029e0 0x7f889419a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:13:33.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.166+0000 7f889961f640 1 --2- 172.21.15.96:0/1245420034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8894103c20 0x7f889419a740 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f889000e9f0 tx=0x7f889000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:33.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.167+0000 7f888affd640 1 -- 172.21.15.96:0/1245420034 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889000cd80 con 0x7f8894103c20 2024-04-25T14:13:33.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.167+0000 7f888affd640 1 -- 172.21.15.96:0/1245420034 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88900108e0 con 0x7f8894103c20 2024-04-25T14:13:33.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.167+0000 7f889c084640 1 -- 172.21.15.96:0/1245420034 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88941a1fd0 con 0x7f8894103c20 2024-04-25T14:13:33.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.167+0000 7f888affd640 1 -- 172.21.15.96:0/1245420034 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8890018980 con 0x7f8894103c20 2024-04-25T14:13:33.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.167+0000 7f889c084640 1 -- 172.21.15.96:0/1245420034 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88941a2510 con 0x7f8894103c20 2024-04-25T14:13:33.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.168+0000 7f889c084640 1 -- 172.21.15.96:0/1245420034 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8894107730 con 0x7f8894103c20 2024-04-25T14:13:33.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.169+0000 7f888affd640 1 -- 172.21.15.96:0/1245420034 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f8890010430 con 0x7f8894103c20 2024-04-25T14:13:33.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.170+0000 7f888affd640 1 --2- 172.21.15.96:0/1245420034 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f88800afb70 0x7f88800b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:33.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.170+0000 7f8899e20640 1 --2- 172.21.15.96:0/1245420034 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f88800afb70 0x7f88800b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:33.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.170+0000 7f888affd640 1 -- 172.21.15.96:0/1245420034 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f88900df430 con 0x7f8894103c20 2024-04-25T14:13:33.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.170+0000 7f8899e20640 1 --2- 172.21.15.96:0/1245420034 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f88800afb70 0x7f88800b2080 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f88941035e0 tx=0x7f888400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:33.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.173+0000 7f888affd640 1 -- 172.21.15.96:0/1245420034 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f88900a7ff0 con 0x7f8894103c20 2024-04-25T14:13:33.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.350+0000 7f889c084640 1 -- 172.21.15.96:0/1245420034 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f88940637a0 con 0x7f8894103c20 2024-04-25T14:13:33.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.351+0000 7f888affd640 1 -- 172.21.15.96:0/1245420034 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) ==== 93+0+10 (secure 0 0 0) 0x7f88900b0070 con 0x7f8894103c20 2024-04-25T14:13:33.351 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-04-25T14:13:33.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.353+0000 7f889c084640 1 -- 172.21.15.96:0/1245420034 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f88800afb70 msgr2=0x7f88800b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:33.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.353+0000 7f889c084640 1 --2- 172.21.15.96:0/1245420034 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f88800afb70 0x7f88800b2080 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f88941035e0 tx=0x7f888400b390 comp rx=0 tx=0).stop 2024-04-25T14:13:33.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.353+0000 7f889c084640 1 -- 172.21.15.96:0/1245420034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8894103c20 msgr2=0x7f889419a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:33.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.353+0000 7f889c084640 1 --2- 172.21.15.96:0/1245420034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8894103c20 0x7f889419a740 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f889000e9f0 tx=0x7f889000eeb0 comp rx=0 tx=0).stop 2024-04-25T14:13:33.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.353+0000 7f889c084640 1 -- 172.21.15.96:0/1245420034 shutdown_connections 2024-04-25T14:13:33.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.353+0000 7f889c084640 1 --2- 172.21.15.96:0/1245420034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8894108920 0x7f88941a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:33.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.353+0000 7f889c084640 1 --2- 172.21.15.96:0/1245420034 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f88800afb70 0x7f88800b2080 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:33.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.353+0000 7f889c084640 1 --2- 172.21.15.96:0/1245420034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8894103c20 0x7f889419a740 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:33.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.353+0000 7f889c084640 1 --2- 172.21.15.96:0/1245420034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88941029e0 0x7f889419a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:33.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.353+0000 7f889c084640 1 -- 172.21.15.96:0/1245420034 >> 172.21.15.96:0/1245420034 conn(0x7f88940fdbd0 msgr2=0x7f8894104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:33.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.353+0000 7f889c084640 1 -- 172.21.15.96:0/1245420034 shutdown_connections 2024-04-25T14:13:33.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.353+0000 7f889c084640 1 -- 172.21.15.96:0/1245420034 wait complete. 2024-04-25T14:13:33.364 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:13:33.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.475+0000 7f82bc772640 1 -- 172.21.15.96:0/795366385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82b41050a0 msgr2=0x7f82b41054b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:33.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.475+0000 7f82bc772640 1 --2- 172.21.15.96:0/795366385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82b41050a0 0x7f82b41054b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f82b0009900 tx=0x7f82b002a410 comp rx=0 tx=0).stop 2024-04-25T14:13:33.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.475+0000 7f82bc772640 1 -- 172.21.15.96:0/795366385 shutdown_connections 2024-04-25T14:13:33.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.475+0000 7f82bc772640 1 --2- 172.21.15.96:0/795366385 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82b4102250 0x7f82b410bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:33.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.475+0000 7f82bc772640 1 --2- 172.21.15.96:0/795366385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82b4101750 0x7f82b4101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:33.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.475+0000 7f82bc772640 1 --2- 172.21.15.96:0/795366385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82b41050a0 0x7f82b41054b0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:33.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.475+0000 7f82bc772640 1 -- 172.21.15.96:0/795366385 >> 172.21.15.96:0/795366385 conn(0x7f82b40fd350 msgr2=0x7f82b40ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:33.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.475+0000 7f82bc772640 1 -- 172.21.15.96:0/795366385 shutdown_connections 2024-04-25T14:13:33.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.475+0000 7f82bc772640 1 -- 172.21.15.96:0/795366385 wait complete. 2024-04-25T14:13:33.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.475+0000 7f82bc772640 1 Processor -- start 2024-04-25T14:13:33.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.476+0000 7f82bc772640 1 -- start start 2024-04-25T14:13:33.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.476+0000 7f82bc772640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82b4101750 0x7f82b419a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:33.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.476+0000 7f82bc772640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82b4102250 0x7f82b419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:33.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.476+0000 7f82bc772640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82b41050a0 0x7f82b41a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:33.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.476+0000 7f82bc772640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f82b410f610 con 0x7f82b41050a0 2024-04-25T14:13:33.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.476+0000 7f82bc772640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f82b410f490 con 0x7f82b4101750 2024-04-25T14:13:33.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.476+0000 7f82bc772640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f82b410f790 con 0x7f82b4102250 2024-04-25T14:13:33.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.476+0000 7f82ba50e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82b4101750 0x7f82b419a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:33.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.476+0000 7f82bad0f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82b41050a0 0x7f82b41a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:33.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.476+0000 7f82ba50e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82b4101750 0x7f82b419a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:57176/0 (socket says 172.21.15.96:57176) 2024-04-25T14:13:33.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.476+0000 7f82ba50e640 1 -- 172.21.15.96:0/1815522462 learned_addr learned my addr 172.21.15.96:0/1815522462 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:33.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.476+0000 7f82b9d0d640 1 --2- 172.21.15.96:0/1815522462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82b4102250 0x7f82b419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:33.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.476+0000 7f82bad0f640 1 -- 172.21.15.96:0/1815522462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82b4102250 msgr2=0x7f82b419a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:33.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.476+0000 7f82bad0f640 1 --2- 172.21.15.96:0/1815522462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82b4102250 0x7f82b419a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:33.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.476+0000 7f82bad0f640 1 -- 172.21.15.96:0/1815522462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82b4101750 msgr2=0x7f82b419a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:33.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.476+0000 7f82bad0f640 1 --2- 172.21.15.96:0/1815522462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82b4101750 0x7f82b419a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:33.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.476+0000 7f82bad0f640 1 -- 172.21.15.96:0/1815522462 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82b41a1d30 con 0x7f82b41050a0 2024-04-25T14:13:33.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.476+0000 7f82ba50e640 1 --2- 172.21.15.96:0/1815522462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82b4101750 0x7f82b419a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:13:33.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.476+0000 7f82bad0f640 1 --2- 172.21.15.96:0/1815522462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82b41050a0 0x7f82b41a16c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f82ac00e940 tx=0x7f82ac00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:33.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.477+0000 7f82ab7fe640 1 -- 172.21.15.96:0/1815522462 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ac004d20 con 0x7f82b41050a0 2024-04-25T14:13:33.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.477+0000 7f82ab7fe640 1 -- 172.21.15.96:0/1815522462 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82ac005020 con 0x7f82b41050a0 2024-04-25T14:13:33.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.477+0000 7f82bc772640 1 -- 172.21.15.96:0/1815522462 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82b41a1fc0 con 0x7f82b41050a0 2024-04-25T14:13:33.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.477+0000 7f82ab7fe640 1 -- 172.21.15.96:0/1815522462 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ac0116d0 con 0x7f82b41050a0 2024-04-25T14:13:33.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.477+0000 7f82bc772640 1 -- 172.21.15.96:0/1815522462 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82b41a2480 con 0x7f82b41050a0 2024-04-25T14:13:33.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.478+0000 7f82bc772640 1 -- 172.21.15.96:0/1815522462 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82b4108a30 con 0x7f82b41050a0 2024-04-25T14:13:33.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.479+0000 7f82ab7fe640 1 -- 172.21.15.96:0/1815522462 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f82ac005460 con 0x7f82b41050a0 2024-04-25T14:13:33.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.479+0000 7f82ab7fe640 1 --2- 172.21.15.96:0/1815522462 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f82a00afaf0 0x7f82a00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:33.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.480+0000 7f82ba50e640 1 --2- 172.21.15.96:0/1815522462 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f82a00afaf0 0x7f82a00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:33.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.480+0000 7f82ab7fe640 1 -- 172.21.15.96:0/1815522462 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f82ac0def90 con 0x7f82b41050a0 2024-04-25T14:13:33.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.480+0000 7f82ba50e640 1 --2- 172.21.15.96:0/1815522462 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f82a00afaf0 0x7f82a00b2000 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f82b0031500 tx=0x7f82b00046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:33.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.483+0000 7f82ab7fe640 1 -- 172.21.15.96:0/1815522462 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f82ac016030 con 0x7f82b41050a0 2024-04-25T14:13:33.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.663+0000 7f82bc772640 1 -- 172.21.15.96:0/1815522462 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f82b40637a0 con 0x7f82b41050a0 2024-04-25T14:13:33.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.663+0000 7f82ab7fe640 1 -- 172.21.15.96:0/1815522462 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v78) ==== 104+0+11 (secure 0 0 0) 0x7f82ac0a7500 con 0x7f82b41050a0 2024-04-25T14:13:33.663 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-04-25T14:13:33.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.665+0000 7f82bc772640 1 -- 172.21.15.96:0/1815522462 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f82a00afaf0 msgr2=0x7f82a00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:33.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.665+0000 7f82bc772640 1 --2- 172.21.15.96:0/1815522462 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f82a00afaf0 0x7f82a00b2000 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f82b0031500 tx=0x7f82b00046c0 comp rx=0 tx=0).stop 2024-04-25T14:13:33.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.665+0000 7f82bc772640 1 -- 172.21.15.96:0/1815522462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82b41050a0 msgr2=0x7f82b41a16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:33.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.665+0000 7f82bc772640 1 --2- 172.21.15.96:0/1815522462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82b41050a0 0x7f82b41a16c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f82ac00e940 tx=0x7f82ac00ee00 comp rx=0 tx=0).stop 2024-04-25T14:13:33.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.665+0000 7f82bc772640 1 -- 172.21.15.96:0/1815522462 shutdown_connections 2024-04-25T14:13:33.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.665+0000 7f82bc772640 1 --2- 172.21.15.96:0/1815522462 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f82a00afaf0 0x7f82a00b2000 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:33.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.665+0000 7f82bc772640 1 --2- 172.21.15.96:0/1815522462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82b41050a0 0x7f82b41a16c0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:33.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.665+0000 7f82bc772640 1 --2- 172.21.15.96:0/1815522462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82b4102250 0x7f82b419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:33.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.665+0000 7f82bc772640 1 --2- 172.21.15.96:0/1815522462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82b4101750 0x7f82b419a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:33.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.665+0000 7f82bc772640 1 -- 172.21.15.96:0/1815522462 >> 172.21.15.96:0/1815522462 conn(0x7f82b40fd350 msgr2=0x7f82b4105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:33.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.665+0000 7f82bc772640 1 -- 172.21.15.96:0/1815522462 shutdown_connections 2024-04-25T14:13:33.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.665+0000 7f82bc772640 1 -- 172.21.15.96:0/1815522462 wait complete. 2024-04-25T14:13:33.676 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:13:33.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.787+0000 7f121bc70640 1 -- 172.21.15.96:0/3757159822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1214103af0 msgr2=0x7f12141047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:33.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.787+0000 7f121bc70640 1 --2- 172.21.15.96:0/3757159822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1214103af0 0x7f12141047f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1210009900 tx=0x7f121002a410 comp rx=0 tx=0).stop 2024-04-25T14:13:33.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.787+0000 7f121bc70640 1 -- 172.21.15.96:0/3757159822 shutdown_connections 2024-04-25T14:13:33.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.787+0000 7f121bc70640 1 --2- 172.21.15.96:0/3757159822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1214109a30 0x7f121410be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:33.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.787+0000 7f121bc70640 1 --2- 172.21.15.96:0/3757159822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1214104d30 0x7f1214105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:33.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.787+0000 7f121bc70640 1 --2- 172.21.15.96:0/3757159822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1214103af0 0x7f12141047f0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:33.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.787+0000 7f121bc70640 1 -- 172.21.15.96:0/3757159822 >> 172.21.15.96:0/3757159822 conn(0x7f12140fece0 msgr2=0x7f1214101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:33.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.787+0000 7f121bc70640 1 -- 172.21.15.96:0/3757159822 shutdown_connections 2024-04-25T14:13:33.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.787+0000 7f121bc70640 1 -- 172.21.15.96:0/3757159822 wait complete. 2024-04-25T14:13:33.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.788+0000 7f121bc70640 1 Processor -- start 2024-04-25T14:13:33.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.788+0000 7f121bc70640 1 -- start start 2024-04-25T14:13:33.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.788+0000 7f121bc70640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1214103af0 0x7f1214077960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:33.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.788+0000 7f121bc70640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1214104d30 0x7f1214076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:33.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.788+0000 7f121bc70640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1214109a30 0x7f1214076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:33.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.788+0000 7f121bc70640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f121410f5f0 con 0x7f1214104d30 2024-04-25T14:13:33.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.788+0000 7f121bc70640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f121410f470 con 0x7f1214103af0 2024-04-25T14:13:33.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.788+0000 7f121bc70640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f121410f770 con 0x7f1214109a30 2024-04-25T14:13:33.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.788+0000 7f121920b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1214104d30 0x7f1214076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:33.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.788+0000 7f121920b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1214104d30 0x7f1214076000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40452/0 (socket says 172.21.15.96:40452) 2024-04-25T14:13:33.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.788+0000 7f121920b640 1 -- 172.21.15.96:0/3759940074 learned_addr learned my addr 172.21.15.96:0/3759940074 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:33.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.788+0000 7f1219a0c640 1 --2- 172.21.15.96:0/3759940074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1214103af0 0x7f1214077960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:33.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.788+0000 7f121a20d640 1 --2- 172.21.15.96:0/3759940074 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1214109a30 0x7f1214076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:33.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.789+0000 7f121920b640 1 -- 172.21.15.96:0/3759940074 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1214109a30 msgr2=0x7f1214076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:33.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.789+0000 7f121920b640 1 --2- 172.21.15.96:0/3759940074 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1214109a30 0x7f1214076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:33.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.789+0000 7f121920b640 1 -- 172.21.15.96:0/3759940074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1214103af0 msgr2=0x7f1214077960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:33.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.789+0000 7f121920b640 1 --2- 172.21.15.96:0/3759940074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1214103af0 0x7f1214077960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:33.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.789+0000 7f121920b640 1 -- 172.21.15.96:0/3759940074 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1214076be0 con 0x7f1214104d30 2024-04-25T14:13:33.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.789+0000 7f121a20d640 1 --2- 172.21.15.96:0/3759940074 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1214109a30 0x7f1214076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:33.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.789+0000 7f121920b640 1 --2- 172.21.15.96:0/3759940074 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1214104d30 0x7f1214076000 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f120400d770 tx=0x7f120400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:33.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.789+0000 7f120affd640 1 -- 172.21.15.96:0/3759940074 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204017070 con 0x7f1214104d30 2024-04-25T14:13:33.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.789+0000 7f120affd640 1 -- 172.21.15.96:0/3759940074 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12040054d0 con 0x7f1214104d30 2024-04-25T14:13:33.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.789+0000 7f121bc70640 1 -- 172.21.15.96:0/3759940074 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12141a4310 con 0x7f1214104d30 2024-04-25T14:13:33.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.789+0000 7f120affd640 1 -- 172.21.15.96:0/3759940074 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204004c30 con 0x7f1214104d30 2024-04-25T14:13:33.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.789+0000 7f121bc70640 1 -- 172.21.15.96:0/3759940074 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12141a4720 con 0x7f1214104d30 2024-04-25T14:13:33.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.790+0000 7f121bc70640 1 -- 172.21.15.96:0/3759940074 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1214108840 con 0x7f1214104d30 2024-04-25T14:13:33.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.791+0000 7f120affd640 1 -- 172.21.15.96:0/3759940074 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f1204004dd0 con 0x7f1214104d30 2024-04-25T14:13:33.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.792+0000 7f120affd640 1 --2- 172.21.15.96:0/3759940074 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f12000afaf0 0x7f12000b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:33.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.792+0000 7f120affd640 1 -- 172.21.15.96:0/3759940074 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f12040deb10 con 0x7f1214104d30 2024-04-25T14:13:33.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.792+0000 7f1219a0c640 1 --2- 172.21.15.96:0/3759940074 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f12000afaf0 0x7f12000b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:33.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.793+0000 7f1219a0c640 1 --2- 172.21.15.96:0/3759940074 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f12000afaf0 0x7f12000b2000 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1210005ab0 tx=0x7f1210031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:33.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.796+0000 7f120affd640 1 -- 172.21.15.96:0/3759940074 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f12040a76d0 con 0x7f1214104d30 2024-04-25T14:13:33.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:33 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/938525689' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-25T14:13:33.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:33 smithi097 ceph-mon[32720]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T14:13:33.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:33 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/2515948312' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:13:33.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:33 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/1245420034' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:13:33.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:33 smithi097 ceph-mon[32720]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T14:13:33.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:33 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/938525689' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-25T14:13:33.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:33 smithi151 ceph-mon[32990]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T14:13:33.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:33 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/2515948312' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:13:33.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:33 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/1245420034' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:13:33.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:33 smithi151 ceph-mon[32990]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T14:13:33.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.975+0000 7f121bc70640 1 -- 172.21.15.96:0/3759940074 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f12140637d0 con 0x7f1214104d30 2024-04-25T14:13:33.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.976+0000 7f120affd640 1 -- 172.21.15.96:0/3759940074 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v78) ==== 100+0+11 (secure 0 0 0) 0x7f12040a6fd0 con 0x7f1214104d30 2024-04-25T14:13:33.976 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-04-25T14:13:33.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.978+0000 7f121bc70640 1 -- 172.21.15.96:0/3759940074 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f12000afaf0 msgr2=0x7f12000b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:33.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.978+0000 7f121bc70640 1 --2- 172.21.15.96:0/3759940074 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f12000afaf0 0x7f12000b2000 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1210005ab0 tx=0x7f1210031400 comp rx=0 tx=0).stop 2024-04-25T14:13:33.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.978+0000 7f121bc70640 1 -- 172.21.15.96:0/3759940074 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1214104d30 msgr2=0x7f1214076000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:33.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.978+0000 7f121bc70640 1 --2- 172.21.15.96:0/3759940074 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1214104d30 0x7f1214076000 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f120400d770 tx=0x7f120400dc30 comp rx=0 tx=0).stop 2024-04-25T14:13:33.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.978+0000 7f121bc70640 1 -- 172.21.15.96:0/3759940074 shutdown_connections 2024-04-25T14:13:33.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.978+0000 7f121bc70640 1 --2- 172.21.15.96:0/3759940074 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1214109a30 0x7f1214076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:33.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.978+0000 7f121bc70640 1 --2- 172.21.15.96:0/3759940074 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f12000afaf0 0x7f12000b2000 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:33.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.978+0000 7f121bc70640 1 --2- 172.21.15.96:0/3759940074 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1214104d30 0x7f1214076000 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:33.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.978+0000 7f121bc70640 1 --2- 172.21.15.96:0/3759940074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1214103af0 0x7f1214077960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:33.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.978+0000 7f121bc70640 1 -- 172.21.15.96:0/3759940074 >> 172.21.15.96:0/3759940074 conn(0x7f12140fece0 msgr2=0x7f1214105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:33.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.978+0000 7f121bc70640 1 -- 172.21.15.96:0/3759940074 shutdown_connections 2024-04-25T14:13:33.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:33.978+0000 7f121bc70640 1 -- 172.21.15.96:0/3759940074 wait complete. 2024-04-25T14:13:33.988 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:33 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/938525689' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-25T14:13:33.988 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:33 smithi096 ceph-mon[29207]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T14:13:33.988 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:33 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/2515948312' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:13:33.988 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:33 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/1245420034' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:13:33.988 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:33 smithi096 ceph-mon[29207]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T14:13:33.989 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:13:34.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.098+0000 7f187cb84640 1 -- 172.21.15.96:0/1070448085 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18781029e0 msgr2=0x7f18781036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:34.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.098+0000 7f187cb84640 1 --2- 172.21.15.96:0/1070448085 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18781029e0 0x7f18781036e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1860009900 tx=0x7f186002a410 comp rx=0 tx=0).stop 2024-04-25T14:13:34.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.098+0000 7f187cb84640 1 -- 172.21.15.96:0/1070448085 shutdown_connections 2024-04-25T14:13:34.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.098+0000 7f187cb84640 1 --2- 172.21.15.96:0/1070448085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1878108920 0x7f187810ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.098+0000 7f187cb84640 1 --2- 172.21.15.96:0/1070448085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1878103c20 0x7f1878104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.098+0000 7f187cb84640 1 --2- 172.21.15.96:0/1070448085 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18781029e0 0x7f18781036e0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.098+0000 7f187cb84640 1 -- 172.21.15.96:0/1070448085 >> 172.21.15.96:0/1070448085 conn(0x7f18780fdbd0 msgr2=0x7f1878100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:34.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.098+0000 7f187cb84640 1 -- 172.21.15.96:0/1070448085 shutdown_connections 2024-04-25T14:13:34.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.098+0000 7f187cb84640 1 -- 172.21.15.96:0/1070448085 wait complete. 2024-04-25T14:13:34.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.099+0000 7f187cb84640 1 Processor -- start 2024-04-25T14:13:34.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.099+0000 7f187cb84640 1 -- start start 2024-04-25T14:13:34.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.099+0000 7f187cb84640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18781029e0 0x7f1878199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:34.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.099+0000 7f187cb84640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1878103c20 0x7f18781996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:34.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.099+0000 7f187cb84640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1878108920 0x7f18781a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:34.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.099+0000 7f187cb84640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f187810e4e0 con 0x7f1878103c20 2024-04-25T14:13:34.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.099+0000 7f187cb84640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f187810e360 con 0x7f1878108920 2024-04-25T14:13:34.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.099+0000 7f187cb84640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f187810e660 con 0x7f18781029e0 2024-04-25T14:13:34.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.099+0000 7f1875d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1878103c20 0x7f18781996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:34.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.099+0000 7f1875d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1878103c20 0x7f18781996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40458/0 (socket says 172.21.15.96:40458) 2024-04-25T14:13:34.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.099+0000 7f1875d9b640 1 -- 172.21.15.96:0/3395186268 learned_addr learned my addr 172.21.15.96:0/3395186268 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:34.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.100+0000 7f187659c640 1 --2- 172.21.15.96:0/3395186268 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18781029e0 0x7f1878199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:34.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.100+0000 7f1876d9d640 1 --2- 172.21.15.96:0/3395186268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1878108920 0x7f18781a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:34.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.100+0000 7f1875d9b640 1 -- 172.21.15.96:0/3395186268 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18781029e0 msgr2=0x7f1878199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:34.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.100+0000 7f1875d9b640 1 --2- 172.21.15.96:0/3395186268 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18781029e0 0x7f1878199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.100+0000 7f1875d9b640 1 -- 172.21.15.96:0/3395186268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1878108920 msgr2=0x7f18781a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:34.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.100+0000 7f1875d9b640 1 --2- 172.21.15.96:0/3395186268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1878108920 0x7f18781a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.100+0000 7f1875d9b640 1 -- 172.21.15.96:0/3395186268 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18781a0c10 con 0x7f1878103c20 2024-04-25T14:13:34.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.100+0000 7f1875d9b640 1 --2- 172.21.15.96:0/3395186268 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1878103c20 0x7f18781996c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f186800c920 tx=0x7f186800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:34.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.100+0000 7f186f7fe640 1 -- 172.21.15.96:0/3395186268 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1868007c70 con 0x7f1878103c20 2024-04-25T14:13:34.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.100+0000 7f186f7fe640 1 -- 172.21.15.96:0/3395186268 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1868007e10 con 0x7f1878103c20 2024-04-25T14:13:34.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.100+0000 7f187cb84640 1 -- 172.21.15.96:0/3395186268 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18781a0f00 con 0x7f1878103c20 2024-04-25T14:13:34.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.100+0000 7f186f7fe640 1 -- 172.21.15.96:0/3395186268 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1868018820 con 0x7f1878103c20 2024-04-25T14:13:34.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.100+0000 7f187cb84640 1 -- 172.21.15.96:0/3395186268 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18781a1360 con 0x7f1878103c20 2024-04-25T14:13:34.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.101+0000 7f187cb84640 1 -- 172.21.15.96:0/3395186268 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1878107730 con 0x7f1878103c20 2024-04-25T14:13:34.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.102+0000 7f186f7fe640 1 -- 172.21.15.96:0/3395186268 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f186800f450 con 0x7f1878103c20 2024-04-25T14:13:34.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.103+0000 7f186f7fe640 1 --2- 172.21.15.96:0/3395186268 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f18640afaf0 0x7f18640b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:34.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.103+0000 7f187659c640 1 --2- 172.21.15.96:0/3395186268 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f18640afaf0 0x7f18640b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:34.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.103+0000 7f186f7fe640 1 -- 172.21.15.96:0/3395186268 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f18680def50 con 0x7f1878103c20 2024-04-25T14:13:34.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.104+0000 7f187659c640 1 --2- 172.21.15.96:0/3395186268 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f18640afaf0 0x7f18640b2000 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1860005ab0 tx=0x7f1860031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:34.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.107+0000 7f186f7fe640 1 -- 172.21.15.96:0/3395186268 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1868014030 con 0x7f1878103c20 2024-04-25T14:13:34.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.284+0000 7f187cb84640 1 -- 172.21.15.96:0/3395186268 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f187802dd20 con 0x7f1878103c20 2024-04-25T14:13:34.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.286+0000 7f186f7fe640 1 -- 172.21.15.96:0/3395186268 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22616 (secure 0 0 0) 0x7f18680a7ba0 con 0x7f1878103c20 2024-04-25T14:13:34.287 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:13:34.287 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":78,"fsid":"59334182-030c-11ef-bc93-c7b262605968","created":"2024-04-25T14:03:25.680745+0000","modified":"2024-04-25T14:12:22.496143+0000","last_up_change":"2024-04-25T14:10:35.894312+0000","last_in_change":"2024-04-25T14:10:15.583440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:06:47.402927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:11:27.434331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:11:27.854004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cf4ba3ac-142e-4e8e-96d4-410d77157eb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6803","nonce":3832506168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6805","nonce":3832506168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832506168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6807","nonce":3832506168}]},"public_addr":"172.21.15.96:6803/3832506168","cluster_addr":"172.21.15.96:6805/3832506168","heartbeat_back_addr":"172.21.15.96:6809/3832506168","heartbeat_front_addr":"172.21.15.96:6807/3832506168","state":["exists","up"]},{"osd":1,"uuid":"866e73b9-4a26-48ba-badf-704fe22f71cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6801","nonce":1083311494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6803","nonce":1083311494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6807","nonce":1083311494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6805","nonce":1083311494}]},"public_addr":"172.21.15.97:6801/1083311494","cluster_addr":"172.21.15.97:6803/1083311494","heartbeat_back_addr":"172.21.15.97:6807/1083311494","heartbeat_front_addr":"172.21.15.97:6805/1083311494","state":["exists","up"]},{"osd":2,"uuid":"45c79ecb-abc4-4dba-bae8-74804e4bc1bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6801","nonce":981902966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6803","nonce":981902966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6807","nonce":981902966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6805","nonce":981902966}]},"public_addr":"172.21.15.151:6801/981902966","cluster_addr":"172.21.15.151:6803/981902966","heartbeat_back_addr":"172.21.15.151:6807/981902966","heartbeat_front_addr":"172.21.15.151:6805/981902966","state":["exists","up"]},{"osd":3,"uuid":"9fc39468-a7ea-4ccf-b365-335ad2ea4c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6811","nonce":2274158748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6813","nonce":2274158748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6817","nonce":2274158748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6815","nonce":2274158748}]},"public_addr":"172.21.15.96:6811/2274158748","cluster_addr":"172.21.15.96:6813/2274158748","heartbeat_back_addr":"172.21.15.96:6817/2274158748","heartbeat_front_addr":"172.21.15.96:6815/2274158748","state":["exists","up"]},{"osd":4,"uuid":"616582f1-12d2-456b-b7b3-e5517800e8a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6809","nonce":3727031962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6811","nonce":3727031962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6815","nonce":3727031962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6813","nonce":3727031962}]},"public_addr":"172.21.15.97:6809/3727031962","cluster_addr":"172.21.15.97:6811/3727031962","heartbeat_back_addr":"172.21.15.97:6815/3727031962","heartbeat_front_addr":"172.21.15.97:6813/3727031962","state":["exists","up"]},{"osd":5,"uuid":"3d277784-ab0f-4c90-81a1-2a4034c52a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6809","nonce":223882546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6811","nonce":223882546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6815","nonce":223882546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6813","nonce":223882546}]},"public_addr":"172.21.15.151:6809/223882546","cluster_addr":"172.21.15.151:6811/223882546","heartbeat_back_addr":"172.21.15.151:6815/223882546","heartbeat_front_addr":"172.21.15.151:6813/223882546","state":["exists","up"]},{"osd":6,"uuid":"9a9ad585-cb1a-4239-9d18-055f59164049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6819","nonce":96197407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6821","nonce":96197407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6825","nonce":96197407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6823","nonce":96197407}]},"public_addr":"172.21.15.96:6819/96197407","cluster_addr":"172.21.15.96:6821/96197407","heartbeat_back_addr":"172.21.15.96:6825/96197407","heartbeat_front_addr":"172.21.15.96:6823/96197407","state":["exists","up"]},{"osd":7,"uuid":"303551c8-6af0-42f2-844f-45bf8db5baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6817","nonce":2247216400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6819","nonce":2247216400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6823","nonce":2247216400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6821","nonce":2247216400}]},"public_addr":"172.21.15.97:6817/2247216400","cluster_addr":"172.21.15.97:6819/2247216400","heartbeat_back_addr":"172.21.15.97:6823/2247216400","heartbeat_front_addr":"172.21.15.97:6821/2247216400","state":["exists","up"]},{"osd":8,"uuid":"81f6cc66-e4ca-430c-b336-b6678d968818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6817","nonce":2549822185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6819","nonce":2549822185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6823","nonce":2549822185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6821","nonce":2549822185}]},"public_addr":"172.21.15.151:6817/2549822185","cluster_addr":"172.21.15.151:6819/2549822185","heartbeat_back_addr":"172.21.15.151:6823/2549822185","heartbeat_front_addr":"172.21.15.151:6821/2549822185","state":["exists","up"]},{"osd":9,"uuid":"29d4da16-feab-4d1c-9d1d-de4f799bd724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6827","nonce":1099399643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6829","nonce":1099399643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6833","nonce":1099399643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6831","nonce":1099399643}]},"public_addr":"172.21.15.96:6827/1099399643","cluster_addr":"172.21.15.96:6829/1099399643","heartbeat_back_addr":"172.21.15.96:6833/1099399643","heartbeat_front_addr":"172.21.15.96:6831/1099399643","state":["exists","up"]},{"osd":10,"uuid":"197fee9b-eee4-4893-be1c-3d57f42250e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6825","nonce":155754520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6827","nonce":155754520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6831","nonce":155754520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6829","nonce":155754520}]},"public_addr":"172.21.15.97:6825/155754520","cluster_addr":"172.21.15.97:6827/155754520","heartbeat_back_addr":"172.21.15.97:6831/155754520","heartbeat_front_addr":"172.21.15.97:6829/155754520","state":["exists","up"]},{"osd":11,"uuid":"9c1d0a83-fc12-41a2-8eb0-80c4abdd2770","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6825","nonce":2027958746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6827","nonce":2027958746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6831","nonce":2027958746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6829","nonce":2027958746}]},"public_addr":"172.21.15.151:6825/2027958746","cluster_addr":"172.21.15.151:6827/2027958746","heartbeat_back_addr":"172.21.15.151:6831/2027958746","heartbeat_front_addr":"172.21.15.151:6829/2027958746","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:05:56.177390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:06:19.917663+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:06:43.608052+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:08.540382+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:33.474072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:58.242306+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:08:24.120442+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:08:49.661064+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:09:15.097271+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:09:41.904505+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:10:08.233252+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:10:33.712296+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/3839807484":"2024-04-26T14:04:21.307574+0000","172.21.15.96:0/1495237551":"2024-04-26T14:04:21.307574+0000","172.21.15.96:0/2793751450":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/129575834":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/2233075139":"2024-04-26T14:04:21.307574+0000","172.21.15.96:6801/2348397481":"2024-04-26T14:03:54.249537+0000","172.21.15.96:6801/433591062":"2024-04-26T14:04:21.307574+0000","172.21.15.96:6800/2348397481":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/3517674924":"2024-04-26T14:03:54.249537+0000","172.21.15.96:6800/433591062":"2024-04-26T14:04:21.307574+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:13:34.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.288+0000 7f187cb84640 1 -- 172.21.15.96:0/3395186268 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f18640afaf0 msgr2=0x7f18640b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:34.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.288+0000 7f187cb84640 1 --2- 172.21.15.96:0/3395186268 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f18640afaf0 0x7f18640b2000 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1860005ab0 tx=0x7f1860031400 comp rx=0 tx=0).stop 2024-04-25T14:13:34.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.288+0000 7f187cb84640 1 -- 172.21.15.96:0/3395186268 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1878103c20 msgr2=0x7f18781996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:34.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.288+0000 7f187cb84640 1 --2- 172.21.15.96:0/3395186268 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1878103c20 0x7f18781996c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f186800c920 tx=0x7f186800cde0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.288+0000 7f187cb84640 1 -- 172.21.15.96:0/3395186268 shutdown_connections 2024-04-25T14:13:34.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.288+0000 7f187cb84640 1 --2- 172.21.15.96:0/3395186268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1878108920 0x7f18781a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.288+0000 7f187cb84640 1 --2- 172.21.15.96:0/3395186268 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f18640afaf0 0x7f18640b2000 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.288+0000 7f187cb84640 1 --2- 172.21.15.96:0/3395186268 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1878103c20 0x7f18781996c0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.288+0000 7f187cb84640 1 --2- 172.21.15.96:0/3395186268 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18781029e0 0x7f1878199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.288+0000 7f187cb84640 1 -- 172.21.15.96:0/3395186268 >> 172.21.15.96:0/3395186268 conn(0x7f18780fdbd0 msgr2=0x7f1878104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:34.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.288+0000 7f187cb84640 1 -- 172.21.15.96:0/3395186268 shutdown_connections 2024-04-25T14:13:34.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.288+0000 7f187cb84640 1 -- 172.21.15.96:0/3395186268 wait complete. 2024-04-25T14:13:34.301 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:06:47.402927+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:11:27.434331+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:11:27.854004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:13:34.301 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:13:34.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.413+0000 7f947ddf3640 1 -- 172.21.15.96:0/3477090173 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94781029e0 msgr2=0x7f94781036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:34.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.413+0000 7f947ddf3640 1 --2- 172.21.15.96:0/3477090173 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94781029e0 0x7f94781036e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f9468009ca0 tx=0x7f946802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.413+0000 7f947ddf3640 1 -- 172.21.15.96:0/3477090173 shutdown_connections 2024-04-25T14:13:34.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.413+0000 7f947ddf3640 1 --2- 172.21.15.96:0/3477090173 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9478108920 0x7f947810ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.413+0000 7f947ddf3640 1 --2- 172.21.15.96:0/3477090173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9478103c20 0x7f9478104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.413+0000 7f947ddf3640 1 --2- 172.21.15.96:0/3477090173 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94781029e0 0x7f94781036e0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.413+0000 7f947ddf3640 1 -- 172.21.15.96:0/3477090173 >> 172.21.15.96:0/3477090173 conn(0x7f94780fdbd0 msgr2=0x7f9478100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:34.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.413+0000 7f947ddf3640 1 -- 172.21.15.96:0/3477090173 shutdown_connections 2024-04-25T14:13:34.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.413+0000 7f947ddf3640 1 -- 172.21.15.96:0/3477090173 wait complete. 2024-04-25T14:13:34.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.413+0000 7f947ddf3640 1 Processor -- start 2024-04-25T14:13:34.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.414+0000 7f947ddf3640 1 -- start start 2024-04-25T14:13:34.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.414+0000 7f947ddf3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94781029e0 0x7f947806a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:34.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.414+0000 7f947ddf3640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9478103c20 0x7f947819f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:34.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.414+0000 7f947ddf3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9478108920 0x7f94781a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:34.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.414+0000 7f947ddf3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f947810e4e0 con 0x7f94781029e0 2024-04-25T14:13:34.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.414+0000 7f947ddf3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f947810e360 con 0x7f9478108920 2024-04-25T14:13:34.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.414+0000 7f947ddf3640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f947810e660 con 0x7f9478103c20 2024-04-25T14:13:34.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.414+0000 7f94777fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94781029e0 0x7f947806a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:34.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.414+0000 7f94777fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94781029e0 0x7f947806a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40476/0 (socket says 172.21.15.96:40476) 2024-04-25T14:13:34.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.414+0000 7f94777fe640 1 -- 172.21.15.96:0/2489342176 learned_addr learned my addr 172.21.15.96:0/2489342176 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:34.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.414+0000 7f9477fff640 1 --2- 172.21.15.96:0/2489342176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9478108920 0x7f94781a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:34.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.414+0000 7f9476ffd640 1 --2- 172.21.15.96:0/2489342176 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9478103c20 0x7f947819f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:34.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.414+0000 7f94777fe640 1 -- 172.21.15.96:0/2489342176 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9478103c20 msgr2=0x7f947819f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:34.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.414+0000 7f94777fe640 1 --2- 172.21.15.96:0/2489342176 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9478103c20 0x7f947819f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.414+0000 7f94777fe640 1 -- 172.21.15.96:0/2489342176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9478108920 msgr2=0x7f94781a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:34.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.414+0000 7f94777fe640 1 --2- 172.21.15.96:0/2489342176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9478108920 0x7f94781a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.414+0000 7f94777fe640 1 -- 172.21.15.96:0/2489342176 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94781a2080 con 0x7f94781029e0 2024-04-25T14:13:34.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.414+0000 7f94777fe640 1 --2- 172.21.15.96:0/2489342176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94781029e0 0x7f947806a900 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f946802ade0 tx=0x7f94680316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:34.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.415+0000 7f9474ff9640 1 -- 172.21.15.96:0/2489342176 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9468003b70 con 0x7f94781029e0 2024-04-25T14:13:34.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.415+0000 7f9474ff9640 1 -- 172.21.15.96:0/2489342176 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9468003d10 con 0x7f94781029e0 2024-04-25T14:13:34.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.415+0000 7f947ddf3640 1 -- 172.21.15.96:0/2489342176 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94781a2310 con 0x7f94781029e0 2024-04-25T14:13:34.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.415+0000 7f9474ff9640 1 -- 172.21.15.96:0/2489342176 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946802fb90 con 0x7f94781029e0 2024-04-25T14:13:34.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.415+0000 7f947ddf3640 1 -- 172.21.15.96:0/2489342176 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94781a28b0 con 0x7f94781029e0 2024-04-25T14:13:34.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.416+0000 7f947ddf3640 1 -- 172.21.15.96:0/2489342176 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9478107730 con 0x7f94781029e0 2024-04-25T14:13:34.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.417+0000 7f9474ff9640 1 -- 172.21.15.96:0/2489342176 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f946802fd30 con 0x7f94781029e0 2024-04-25T14:13:34.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.418+0000 7f9474ff9640 1 --2- 172.21.15.96:0/2489342176 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f94640afb70 0x7f94640b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:34.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.418+0000 7f9476ffd640 1 --2- 172.21.15.96:0/2489342176 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f94640afb70 0x7f94640b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:34.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.418+0000 7f9474ff9640 1 -- 172.21.15.96:0/2489342176 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f94681002b0 con 0x7f94781029e0 2024-04-25T14:13:34.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.418+0000 7f9476ffd640 1 --2- 172.21.15.96:0/2489342176 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f94640afb70 0x7f94640b2080 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f947806a2a0 tx=0x7f9460005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:34.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.421+0000 7f9474ff9640 1 -- 172.21.15.96:0/2489342176 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f94680c8e70 con 0x7f94781029e0 2024-04-25T14:13:34.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.601+0000 7f947ddf3640 1 -- 172.21.15.96:0/2489342176 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f947806c2a0 con 0x7f94781029e0 2024-04-25T14:13:34.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.601+0000 7f9474ff9640 1 -- 172.21.15.96:0/2489342176 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) ==== 93+0+10 (secure 0 0 0) 0x7f94680c8770 con 0x7f94781029e0 2024-04-25T14:13:34.602 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-04-25T14:13:34.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.603+0000 7f947ddf3640 1 -- 172.21.15.96:0/2489342176 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f94640afb70 msgr2=0x7f94640b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:34.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.603+0000 7f947ddf3640 1 --2- 172.21.15.96:0/2489342176 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f94640afb70 0x7f94640b2080 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f947806a2a0 tx=0x7f9460005f90 comp rx=0 tx=0).stop 2024-04-25T14:13:34.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.603+0000 7f947ddf3640 1 -- 172.21.15.96:0/2489342176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94781029e0 msgr2=0x7f947806a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:34.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.603+0000 7f947ddf3640 1 --2- 172.21.15.96:0/2489342176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94781029e0 0x7f947806a900 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f946802ade0 tx=0x7f94680316e0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.603+0000 7f947ddf3640 1 -- 172.21.15.96:0/2489342176 shutdown_connections 2024-04-25T14:13:34.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.603+0000 7f947ddf3640 1 --2- 172.21.15.96:0/2489342176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9478108920 0x7f94781a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.603+0000 7f947ddf3640 1 --2- 172.21.15.96:0/2489342176 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9478103c20 0x7f947819f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.603+0000 7f947ddf3640 1 --2- 172.21.15.96:0/2489342176 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f94640afb70 0x7f94640b2080 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.603+0000 7f947ddf3640 1 --2- 172.21.15.96:0/2489342176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94781029e0 0x7f947806a900 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.603+0000 7f947ddf3640 1 -- 172.21.15.96:0/2489342176 >> 172.21.15.96:0/2489342176 conn(0x7f94780fdbd0 msgr2=0x7f9478104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:34.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.604+0000 7f947ddf3640 1 -- 172.21.15.96:0/2489342176 shutdown_connections 2024-04-25T14:13:34.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.604+0000 7f947ddf3640 1 -- 172.21.15.96:0/2489342176 wait complete. 2024-04-25T14:13:34.615 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:13:34.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.727+0000 7f23f9ff3640 1 -- 172.21.15.96:0/1339420747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23f4104470 msgr2=0x7f23f410abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:34.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.727+0000 7f23f9ff3640 1 --2- 172.21.15.96:0/1339420747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23f4104470 0x7f23f410abb0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f23ec009ca0 tx=0x7f23ec02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.728+0000 7f23f9ff3640 1 -- 172.21.15.96:0/1339420747 shutdown_connections 2024-04-25T14:13:34.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.728+0000 7f23f9ff3640 1 --2- 172.21.15.96:0/1339420747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23f4104470 0x7f23f410abb0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.728+0000 7f23f9ff3640 1 --2- 172.21.15.96:0/1339420747 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23f4103930 0x7f23f4103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.728+0000 7f23f9ff3640 1 --2- 172.21.15.96:0/1339420747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23f4102f50 0x7f23f4103360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.728+0000 7f23f9ff3640 1 -- 172.21.15.96:0/1339420747 >> 172.21.15.96:0/1339420747 conn(0x7f23f40fd350 msgr2=0x7f23f40ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:34.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.728+0000 7f23f9ff3640 1 -- 172.21.15.96:0/1339420747 shutdown_connections 2024-04-25T14:13:34.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.728+0000 7f23f9ff3640 1 -- 172.21.15.96:0/1339420747 wait complete. 2024-04-25T14:13:34.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.728+0000 7f23f9ff3640 1 Processor -- start 2024-04-25T14:13:34.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.728+0000 7f23f9ff3640 1 -- start start 2024-04-25T14:13:34.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.728+0000 7f23f9ff3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23f4102f50 0x7f23f419a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:34.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.728+0000 7f23f9ff3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23f4103930 0x7f23f419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:34.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.728+0000 7f23f9ff3640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23f4104470 0x7f23f41a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:34.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.728+0000 7f23f9ff3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f23f410e590 con 0x7f23f4103930 2024-04-25T14:13:34.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.728+0000 7f23f9ff3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f23f410e410 con 0x7f23f4102f50 2024-04-25T14:13:34.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.728+0000 7f23f9ff3640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f23f410e710 con 0x7f23f4104470 2024-04-25T14:13:34.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.728+0000 7f23ebfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23f4103930 0x7f23f419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:34.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.729+0000 7f23ebfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23f4103930 0x7f23f419a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40496/0 (socket says 172.21.15.96:40496) 2024-04-25T14:13:34.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.729+0000 7f23ebfff640 1 -- 172.21.15.96:0/577196138 learned_addr learned my addr 172.21.15.96:0/577196138 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:34.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.729+0000 7f23f37fe640 1 --2- 172.21.15.96:0/577196138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23f4102f50 0x7f23f419a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:34.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.729+0000 7f23f3fff640 1 --2- 172.21.15.96:0/577196138 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23f4104470 0x7f23f41a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:34.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.729+0000 7f23ebfff640 1 -- 172.21.15.96:0/577196138 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23f4104470 msgr2=0x7f23f41a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:34.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.729+0000 7f23ebfff640 1 --2- 172.21.15.96:0/577196138 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23f4104470 0x7f23f41a1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.729+0000 7f23ebfff640 1 -- 172.21.15.96:0/577196138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23f4102f50 msgr2=0x7f23f419a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:34.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.729+0000 7f23ebfff640 1 --2- 172.21.15.96:0/577196138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23f4102f50 0x7f23f419a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.729+0000 7f23ebfff640 1 -- 172.21.15.96:0/577196138 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23f41a1d00 con 0x7f23f4103930 2024-04-25T14:13:34.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.729+0000 7f23ebfff640 1 --2- 172.21.15.96:0/577196138 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23f4103930 0x7f23f419a700 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f23dc00d770 tx=0x7f23dc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:34.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.729+0000 7f23f17fa640 1 -- 172.21.15.96:0/577196138 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23dc017070 con 0x7f23f4103930 2024-04-25T14:13:34.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.729+0000 7f23f17fa640 1 -- 172.21.15.96:0/577196138 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23dc0054d0 con 0x7f23f4103930 2024-04-25T14:13:34.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.729+0000 7f23f9ff3640 1 -- 172.21.15.96:0/577196138 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23f41a1ff0 con 0x7f23f4103930 2024-04-25T14:13:34.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.729+0000 7f23f17fa640 1 -- 172.21.15.96:0/577196138 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23dc004c30 con 0x7f23f4103930 2024-04-25T14:13:34.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.729+0000 7f23f9ff3640 1 -- 172.21.15.96:0/577196138 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23f41a2400 con 0x7f23f4103930 2024-04-25T14:13:34.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.730+0000 7f23f9ff3640 1 -- 172.21.15.96:0/577196138 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23f41a2660 con 0x7f23f4103930 2024-04-25T14:13:34.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.731+0000 7f23f17fa640 1 -- 172.21.15.96:0/577196138 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f23dc004dd0 con 0x7f23f4103930 2024-04-25T14:13:34.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.732+0000 7f23f17fa640 1 --2- 172.21.15.96:0/577196138 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f23e00afaf0 0x7f23e00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:34.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.732+0000 7f23f37fe640 1 --2- 172.21.15.96:0/577196138 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f23e00afaf0 0x7f23e00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:34.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.732+0000 7f23f17fa640 1 -- 172.21.15.96:0/577196138 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f23dc0deb70 con 0x7f23f4103930 2024-04-25T14:13:34.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.733+0000 7f23f37fe640 1 --2- 172.21.15.96:0/577196138 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f23e00afaf0 0x7f23e00b2000 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f23f406b500 tx=0x7f23e4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:34.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.736+0000 7f23f17fa640 1 -- 172.21.15.96:0/577196138 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f23dc0a7730 con 0x7f23f4103930 2024-04-25T14:13:34.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.913+0000 7f23f9ff3640 1 -- 172.21.15.96:0/577196138 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f23f41a29f0 con 0x7f23f4103930 2024-04-25T14:13:34.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.914+0000 7f23f17fa640 1 -- 172.21.15.96:0/577196138 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v78) ==== 104+0+11 (secure 0 0 0) 0x7f23dc0a7030 con 0x7f23f4103930 2024-04-25T14:13:34.936 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-04-25T14:13:34.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.916+0000 7f23f9ff3640 1 -- 172.21.15.96:0/577196138 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f23e00afaf0 msgr2=0x7f23e00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:34.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.916+0000 7f23f9ff3640 1 --2- 172.21.15.96:0/577196138 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f23e00afaf0 0x7f23e00b2000 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f23f406b500 tx=0x7f23e4005f90 comp rx=0 tx=0).stop 2024-04-25T14:13:34.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.916+0000 7f23f9ff3640 1 -- 172.21.15.96:0/577196138 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23f4103930 msgr2=0x7f23f419a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:34.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.916+0000 7f23f9ff3640 1 --2- 172.21.15.96:0/577196138 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23f4103930 0x7f23f419a700 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f23dc00d770 tx=0x7f23dc00dc30 comp rx=0 tx=0).stop 2024-04-25T14:13:34.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.916+0000 7f23f9ff3640 1 -- 172.21.15.96:0/577196138 shutdown_connections 2024-04-25T14:13:34.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.916+0000 7f23f9ff3640 1 --2- 172.21.15.96:0/577196138 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23f4104470 0x7f23f41a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.916+0000 7f23f9ff3640 1 --2- 172.21.15.96:0/577196138 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f23e00afaf0 0x7f23e00b2000 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.916+0000 7f23f9ff3640 1 --2- 172.21.15.96:0/577196138 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23f4103930 0x7f23f419a700 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.916+0000 7f23f9ff3640 1 --2- 172.21.15.96:0/577196138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23f4102f50 0x7f23f419a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.916+0000 7f23f9ff3640 1 -- 172.21.15.96:0/577196138 >> 172.21.15.96:0/577196138 conn(0x7f23f40fd350 msgr2=0x7f23f40fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:34.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.916+0000 7f23f9ff3640 1 -- 172.21.15.96:0/577196138 shutdown_connections 2024-04-25T14:13:34.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:34.916+0000 7f23f9ff3640 1 -- 172.21.15.96:0/577196138 wait complete. 2024-04-25T14:13:34.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:34 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/1815522462' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:13:34.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:34 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/3759940074' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:13:34.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:34 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/3395186268' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:13:34.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:34 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/2489342176' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:13:34.939 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:13:34.940 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:34 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/1815522462' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:13:34.940 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:34 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/3759940074' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:13:34.940 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:34 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/3395186268' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:13:34.940 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:34 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/2489342176' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:13:34.940 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:34 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/1815522462' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:13:34.940 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:34 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/3759940074' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:13:34.940 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:34 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/3395186268' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:13:34.941 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:34 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/2489342176' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:13:35.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.051+0000 7f608794d640 1 -- 172.21.15.96:0/1093724235 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60801029e0 msgr2=0x7f60801036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:35.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.051+0000 7f608794d640 1 --2- 172.21.15.96:0/1093724235 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60801029e0 0x7f60801036e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f607c009ca0 tx=0x7f607c02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:13:35.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.051+0000 7f608794d640 1 -- 172.21.15.96:0/1093724235 shutdown_connections 2024-04-25T14:13:35.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.051+0000 7f608794d640 1 --2- 172.21.15.96:0/1093724235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6080108920 0x7f608010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:35.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.051+0000 7f608794d640 1 --2- 172.21.15.96:0/1093724235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6080103c20 0x7f6080104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:35.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.051+0000 7f608794d640 1 --2- 172.21.15.96:0/1093724235 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60801029e0 0x7f60801036e0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:35.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.051+0000 7f608794d640 1 -- 172.21.15.96:0/1093724235 >> 172.21.15.96:0/1093724235 conn(0x7f60800fdbd0 msgr2=0x7f6080100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:35.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.051+0000 7f608794d640 1 -- 172.21.15.96:0/1093724235 shutdown_connections 2024-04-25T14:13:35.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.051+0000 7f608794d640 1 -- 172.21.15.96:0/1093724235 wait complete. 2024-04-25T14:13:35.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.051+0000 7f608794d640 1 Processor -- start 2024-04-25T14:13:35.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.052+0000 7f608794d640 1 -- start start 2024-04-25T14:13:35.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.052+0000 7f608794d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60801029e0 0x7f6080199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:35.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.052+0000 7f608794d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6080103c20 0x7f6080199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:35.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.052+0000 7f608794d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6080108920 0x7f60801a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:35.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.052+0000 7f608794d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f608010e4e0 con 0x7f6080108920 2024-04-25T14:13:35.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.052+0000 7f608794d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f608010e360 con 0x7f6080103c20 2024-04-25T14:13:35.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.052+0000 7f608794d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f608010e660 con 0x7f60801029e0 2024-04-25T14:13:35.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.052+0000 7f6085eea640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6080108920 0x7f60801a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:35.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.052+0000 7f6085eea640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6080108920 0x7f60801a08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40518/0 (socket says 172.21.15.96:40518) 2024-04-25T14:13:35.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.052+0000 7f6085eea640 1 -- 172.21.15.96:0/2484256442 learned_addr learned my addr 172.21.15.96:0/2484256442 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:35.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.052+0000 7f60856e9640 1 --2- 172.21.15.96:0/2484256442 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60801029e0 0x7f6080199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:35.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.052+0000 7f6084ee8640 1 --2- 172.21.15.96:0/2484256442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6080103c20 0x7f6080199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:35.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.052+0000 7f6085eea640 1 -- 172.21.15.96:0/2484256442 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60801029e0 msgr2=0x7f6080199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:35.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.052+0000 7f6085eea640 1 --2- 172.21.15.96:0/2484256442 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60801029e0 0x7f6080199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:35.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.052+0000 7f6085eea640 1 -- 172.21.15.96:0/2484256442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6080103c20 msgr2=0x7f6080199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:35.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.052+0000 7f6085eea640 1 --2- 172.21.15.96:0/2484256442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6080103c20 0x7f6080199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:35.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.052+0000 7f6085eea640 1 -- 172.21.15.96:0/2484256442 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60801a0f50 con 0x7f6080108920 2024-04-25T14:13:35.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.052+0000 7f6085eea640 1 --2- 172.21.15.96:0/2484256442 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6080108920 0x7f60801a08e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f607800d750 tx=0x7f607800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:35.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.052+0000 7f60767fc640 1 -- 172.21.15.96:0/2484256442 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6078017070 con 0x7f6080108920 2024-04-25T14:13:35.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.052+0000 7f60767fc640 1 -- 172.21.15.96:0/2484256442 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60780054d0 con 0x7f6080108920 2024-04-25T14:13:35.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.052+0000 7f608794d640 1 -- 172.21.15.96:0/2484256442 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60801a1240 con 0x7f6080108920 2024-04-25T14:13:35.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.052+0000 7f60767fc640 1 -- 172.21.15.96:0/2484256442 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6078004c60 con 0x7f6080108920 2024-04-25T14:13:35.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.053+0000 7f608794d640 1 -- 172.21.15.96:0/2484256442 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60801a17e0 con 0x7f6080108920 2024-04-25T14:13:35.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.053+0000 7f608794d640 1 -- 172.21.15.96:0/2484256442 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6080107730 con 0x7f6080108920 2024-04-25T14:13:35.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.056+0000 7f60767fc640 1 -- 172.21.15.96:0/2484256442 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f6078009bd0 con 0x7f6080108920 2024-04-25T14:13:35.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.056+0000 7f60767fc640 1 --2- 172.21.15.96:0/2484256442 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f606c0afb70 0x7f606c0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:35.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.056+0000 7f60856e9640 1 --2- 172.21.15.96:0/2484256442 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f606c0afb70 0x7f606c0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:35.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.056+0000 7f60767fc640 1 -- 172.21.15.96:0/2484256442 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f60780dea00 con 0x7f6080108920 2024-04-25T14:13:35.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.057+0000 7f60856e9640 1 --2- 172.21.15.96:0/2484256442 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f606c0afb70 0x7f606c0b2080 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f607c000f80 tx=0x7f607c004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:35.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.060+0000 7f60767fc640 1 -- 172.21.15.96:0/2484256442 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f60780a7640 con 0x7f6080108920 2024-04-25T14:13:35.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.237+0000 7f608794d640 1 -- 172.21.15.96:0/2484256442 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f608002dd30 con 0x7f6080108920 2024-04-25T14:13:35.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.238+0000 7f60767fc640 1 -- 172.21.15.96:0/2484256442 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v78) ==== 100+0+11 (secure 0 0 0) 0x7f60780a6f40 con 0x7f6080108920 2024-04-25T14:13:35.239 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-04-25T14:13:35.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.240+0000 7f608794d640 1 -- 172.21.15.96:0/2484256442 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f606c0afb70 msgr2=0x7f606c0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:35.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.240+0000 7f608794d640 1 --2- 172.21.15.96:0/2484256442 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f606c0afb70 0x7f606c0b2080 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f607c000f80 tx=0x7f607c004a40 comp rx=0 tx=0).stop 2024-04-25T14:13:35.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.240+0000 7f608794d640 1 -- 172.21.15.96:0/2484256442 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6080108920 msgr2=0x7f60801a08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:35.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.240+0000 7f608794d640 1 --2- 172.21.15.96:0/2484256442 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6080108920 0x7f60801a08e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f607800d750 tx=0x7f607800dc10 comp rx=0 tx=0).stop 2024-04-25T14:13:35.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.240+0000 7f608794d640 1 -- 172.21.15.96:0/2484256442 shutdown_connections 2024-04-25T14:13:35.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.240+0000 7f608794d640 1 --2- 172.21.15.96:0/2484256442 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f606c0afb70 0x7f606c0b2080 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:35.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.240+0000 7f608794d640 1 --2- 172.21.15.96:0/2484256442 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6080108920 0x7f60801a08e0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:35.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.240+0000 7f608794d640 1 --2- 172.21.15.96:0/2484256442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6080103c20 0x7f6080199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:35.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.240+0000 7f608794d640 1 --2- 172.21.15.96:0/2484256442 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60801029e0 0x7f6080199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:35.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.240+0000 7f608794d640 1 -- 172.21.15.96:0/2484256442 >> 172.21.15.96:0/2484256442 conn(0x7f60800fdbd0 msgr2=0x7f6080104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:35.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.240+0000 7f608794d640 1 -- 172.21.15.96:0/2484256442 shutdown_connections 2024-04-25T14:13:35.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.240+0000 7f608794d640 1 -- 172.21.15.96:0/2484256442 wait complete. 2024-04-25T14:13:35.251 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-04-25T14:13:35.251 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:13:35.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.360+0000 7fd24469f640 1 -- 172.21.15.96:0/616220779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd23c104d40 msgr2=0x7fd23c105150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:35.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.360+0000 7fd24469f640 1 --2- 172.21.15.96:0/616220779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd23c104d40 0x7fd23c105150 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd22c009900 tx=0x7fd22c02a410 comp rx=0 tx=0).stop 2024-04-25T14:13:35.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.361+0000 7fd24469f640 1 -- 172.21.15.96:0/616220779 shutdown_connections 2024-04-25T14:13:35.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.361+0000 7fd24469f640 1 --2- 172.21.15.96:0/616220779 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd23c109a40 0x7fd23c10be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:35.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.361+0000 7fd24469f640 1 --2- 172.21.15.96:0/616220779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd23c104d40 0x7fd23c105150 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:35.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.361+0000 7fd24469f640 1 --2- 172.21.15.96:0/616220779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd23c103b00 0x7fd23c104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:35.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.361+0000 7fd24469f640 1 -- 172.21.15.96:0/616220779 >> 172.21.15.96:0/616220779 conn(0x7fd23c0fed30 msgr2=0x7fd23c1011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:35.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.361+0000 7fd24469f640 1 -- 172.21.15.96:0/616220779 shutdown_connections 2024-04-25T14:13:35.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.361+0000 7fd24469f640 1 -- 172.21.15.96:0/616220779 wait complete. 2024-04-25T14:13:35.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.362+0000 7fd24469f640 1 Processor -- start 2024-04-25T14:13:35.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.362+0000 7fd24469f640 1 -- start start 2024-04-25T14:13:35.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.362+0000 7fd24469f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd23c103b00 0x7fd23c076850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:35.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.362+0000 7fd24469f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd23c104d40 0x7fd23c074ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:35.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.362+0000 7fd24469f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd23c109a40 0x7fd23c075430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:35.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.362+0000 7fd24469f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd23c10f630 con 0x7fd23c104d40 2024-04-25T14:13:35.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.362+0000 7fd24469f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd23c10f4b0 con 0x7fd23c103b00 2024-04-25T14:13:35.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.362+0000 7fd24469f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd23c10f7b0 con 0x7fd23c109a40 2024-04-25T14:13:35.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.362+0000 7fd241c3a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd23c104d40 0x7fd23c074ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:35.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.362+0000 7fd241c3a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd23c104d40 0x7fd23c074ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40538/0 (socket says 172.21.15.96:40538) 2024-04-25T14:13:35.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.362+0000 7fd241c3a640 1 -- 172.21.15.96:0/1462640069 learned_addr learned my addr 172.21.15.96:0/1462640069 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:35.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.362+0000 7fd242c3c640 1 --2- 172.21.15.96:0/1462640069 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd23c109a40 0x7fd23c075430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:35.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.362+0000 7fd24243b640 1 --2- 172.21.15.96:0/1462640069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd23c103b00 0x7fd23c076850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:35.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.362+0000 7fd241c3a640 1 -- 172.21.15.96:0/1462640069 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd23c109a40 msgr2=0x7fd23c075430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:35.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.362+0000 7fd241c3a640 1 --2- 172.21.15.96:0/1462640069 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd23c109a40 0x7fd23c075430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:35.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.362+0000 7fd241c3a640 1 -- 172.21.15.96:0/1462640069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd23c103b00 msgr2=0x7fd23c076850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:35.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.362+0000 7fd241c3a640 1 --2- 172.21.15.96:0/1462640069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd23c103b00 0x7fd23c076850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:35.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.362+0000 7fd241c3a640 1 -- 172.21.15.96:0/1462640069 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd23c075ad0 con 0x7fd23c104d40 2024-04-25T14:13:35.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.362+0000 7fd242c3c640 1 --2- 172.21.15.96:0/1462640069 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd23c109a40 0x7fd23c075430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:35.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.362+0000 7fd24243b640 1 --2- 172.21.15.96:0/1462640069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd23c103b00 0x7fd23c076850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:35.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.363+0000 7fd241c3a640 1 --2- 172.21.15.96:0/1462640069 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd23c104d40 0x7fd23c074ef0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd22c006010 tx=0x7fd22c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:35.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.363+0000 7fd2337fe640 1 -- 172.21.15.96:0/1462640069 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd22c03f030 con 0x7fd23c104d40 2024-04-25T14:13:35.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.363+0000 7fd24469f640 1 -- 172.21.15.96:0/1462640069 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd23c1a42a0 con 0x7fd23c104d40 2024-04-25T14:13:35.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.363+0000 7fd2337fe640 1 -- 172.21.15.96:0/1462640069 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd22c003aa0 con 0x7fd23c104d40 2024-04-25T14:13:35.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.363+0000 7fd2337fe640 1 -- 172.21.15.96:0/1462640069 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd22c035070 con 0x7fd23c104d40 2024-04-25T14:13:35.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.363+0000 7fd24469f640 1 -- 172.21.15.96:0/1462640069 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd23c1a4650 con 0x7fd23c104d40 2024-04-25T14:13:35.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.364+0000 7fd24469f640 1 -- 172.21.15.96:0/1462640069 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd23c108850 con 0x7fd23c104d40 2024-04-25T14:13:35.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.365+0000 7fd2337fe640 1 -- 172.21.15.96:0/1462640069 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fd22c042030 con 0x7fd23c104d40 2024-04-25T14:13:35.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.366+0000 7fd2337fe640 1 --2- 172.21.15.96:0/1462640069 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd2280afaf0 0x7fd2280b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:35.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.366+0000 7fd24243b640 1 --2- 172.21.15.96:0/1462640069 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd2280afaf0 0x7fd2280b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:35.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.366+0000 7fd2337fe640 1 -- 172.21.15.96:0/1462640069 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fd22c0ff960 con 0x7fd23c104d40 2024-04-25T14:13:35.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.366+0000 7fd24243b640 1 --2- 172.21.15.96:0/1462640069 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd2280afaf0 0x7fd2280b2000 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd23c104700 tx=0x7fd23800a5e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:35.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.369+0000 7fd2337fe640 1 -- 172.21.15.96:0/1462640069 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd22c0c8520 con 0x7fd23c104d40 2024-04-25T14:13:35.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.586+0000 7fd24469f640 1 -- 172.21.15.96:0/1462640069 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd23c075d60 con 0x7fd23c104d40 2024-04-25T14:13:35.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.587+0000 7fd2337fe640 1 -- 172.21.15.96:0/1462640069 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11927 (secure 0 0 0) 0x7fd22c0c7e20 con 0x7fd23c104d40 2024-04-25T14:13:35.586 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:13:35.587 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/606227917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":606227917},{"type":"v1","addr":"172.21.15.96:6835","nonce":606227917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/3265300710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3265300710},{"type":"v1","addr":"172.21.15.96:6837","nonce":3265300710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T14:11:28.886961+0000","modified":"2024-04-25T14:13:32.120297+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24517,"mds_2":24478,"mds_3":24455,"mds_4":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/4080308509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":4080308509},{"type":"v1","addr":"172.21.15.96:6839","nonce":4080308509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/1606748971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":1606748971},{"type":"v1","addr":"172.21.15.96:6841","nonce":1606748971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6833/1995324831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1995324831},{"type":"v1","addr":"172.21.15.97:6833","nonce":1995324831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":4,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.151:6833/1382721608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1382721608},{"type":"v1","addr":"172.21.15.151:6833","nonce":1382721608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6835/3872301726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3872301726},{"type":"v1","addr":"172.21.15.97:6835","nonce":3872301726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.151:6835/1313823638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":1313823638},{"type":"v1","addr":"172.21.15.151:6835","nonce":1313823638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/2691348366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2691348366},{"type":"v1","addr":"172.21.15.151:6837","nonce":2691348366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3306780081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3306780081},{"type":"v1","addr":"172.21.15.97:6837","nonce":3306780081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.151:6839/245699981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":245699981},{"type":"v1","addr":"172.21.15.151:6839","nonce":245699981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6839/2850442269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":2850442269},{"type":"v1","addr":"172.21.15.97:6839","nonce":2850442269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24427,24455,24478,24517,24536]},"id":1}]} 2024-04-25T14:13:35.587 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 26 2024-04-25T14:13:35.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.588+0000 7fd24469f640 1 -- 172.21.15.96:0/1462640069 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd2280afaf0 msgr2=0x7fd2280b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:35.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.588+0000 7fd24469f640 1 --2- 172.21.15.96:0/1462640069 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd2280afaf0 0x7fd2280b2000 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd23c104700 tx=0x7fd23800a5e0 comp rx=0 tx=0).stop 2024-04-25T14:13:35.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.588+0000 7fd24469f640 1 -- 172.21.15.96:0/1462640069 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd23c104d40 msgr2=0x7fd23c074ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:35.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.588+0000 7fd24469f640 1 --2- 172.21.15.96:0/1462640069 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd23c104d40 0x7fd23c074ef0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd22c006010 tx=0x7fd22c031a50 comp rx=0 tx=0).stop 2024-04-25T14:13:35.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.589+0000 7fd24469f640 1 -- 172.21.15.96:0/1462640069 shutdown_connections 2024-04-25T14:13:35.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.589+0000 7fd24469f640 1 --2- 172.21.15.96:0/1462640069 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd23c109a40 0x7fd23c075430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:35.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.589+0000 7fd24469f640 1 --2- 172.21.15.96:0/1462640069 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd2280afaf0 0x7fd2280b2000 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:35.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.589+0000 7fd24469f640 1 --2- 172.21.15.96:0/1462640069 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd23c104d40 0x7fd23c074ef0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:35.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.589+0000 7fd24469f640 1 --2- 172.21.15.96:0/1462640069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd23c103b00 0x7fd23c076850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:35.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.589+0000 7fd24469f640 1 -- 172.21.15.96:0/1462640069 >> 172.21.15.96:0/1462640069 conn(0x7fd23c0fed30 msgr2=0x7fd23c105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:35.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.589+0000 7fd24469f640 1 -- 172.21.15.96:0/1462640069 shutdown_connections 2024-04-25T14:13:35.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.589+0000 7fd24469f640 1 -- 172.21.15.96:0/1462640069 wait complete. 2024-04-25T14:13:35.600 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-04-25T14:13:35.600 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-04-25T14:13:35.600 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:13:35.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.709+0000 7ffb35d7d640 1 -- 172.21.15.96:0/558239900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb301033c0 msgr2=0x7ffb301037d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:35.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.709+0000 7ffb35d7d640 1 --2- 172.21.15.96:0/558239900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb301033c0 0x7ffb301037d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ffb24009900 tx=0x7ffb2402a410 comp rx=0 tx=0).stop 2024-04-25T14:13:35.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.709+0000 7ffb35d7d640 1 -- 172.21.15.96:0/558239900 shutdown_connections 2024-04-25T14:13:35.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.709+0000 7ffb35d7d640 1 --2- 172.21.15.96:0/558239900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb301080c0 0x7ffb3010a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:35.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.709+0000 7ffb35d7d640 1 --2- 172.21.15.96:0/558239900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb301033c0 0x7ffb301037d0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:35.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.709+0000 7ffb35d7d640 1 --2- 172.21.15.96:0/558239900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb30102180 0x7ffb30102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:35.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.709+0000 7ffb35d7d640 1 -- 172.21.15.96:0/558239900 >> 172.21.15.96:0/558239900 conn(0x7ffb300fd370 msgr2=0x7ffb300ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:35.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.709+0000 7ffb35d7d640 1 -- 172.21.15.96:0/558239900 shutdown_connections 2024-04-25T14:13:35.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.709+0000 7ffb35d7d640 1 -- 172.21.15.96:0/558239900 wait complete. 2024-04-25T14:13:35.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.710+0000 7ffb35d7d640 1 Processor -- start 2024-04-25T14:13:35.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.710+0000 7ffb35d7d640 1 -- start start 2024-04-25T14:13:35.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.710+0000 7ffb35d7d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb30102180 0x7ffb3019a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:35.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.710+0000 7ffb35d7d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb301033c0 0x7ffb3019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:35.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.710+0000 7ffb35d7d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb301080c0 0x7ffb301a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:35.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.710+0000 7ffb35d7d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffb3010dca0 con 0x7ffb301033c0 2024-04-25T14:13:35.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.710+0000 7ffb35d7d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ffb3010db20 con 0x7ffb301080c0 2024-04-25T14:13:35.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.710+0000 7ffb35d7d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffb3010de20 con 0x7ffb30102180 2024-04-25T14:13:35.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.710+0000 7ffb2effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb301033c0 0x7ffb3019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:35.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.711+0000 7ffb2effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb301033c0 0x7ffb3019a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40556/0 (socket says 172.21.15.96:40556) 2024-04-25T14:13:35.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.711+0000 7ffb2effd640 1 -- 172.21.15.96:0/4129930221 learned_addr learned my addr 172.21.15.96:0/4129930221 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:35.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.711+0000 7ffb2f7fe640 1 --2- 172.21.15.96:0/4129930221 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb30102180 0x7ffb3019a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:35.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.711+0000 7ffb2ffff640 1 --2- 172.21.15.96:0/4129930221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb301080c0 0x7ffb301a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:35.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.711+0000 7ffb2effd640 1 -- 172.21.15.96:0/4129930221 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb30102180 msgr2=0x7ffb3019a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:35.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.711+0000 7ffb2effd640 1 --2- 172.21.15.96:0/4129930221 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb30102180 0x7ffb3019a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:35.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.711+0000 7ffb2effd640 1 -- 172.21.15.96:0/4129930221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb301080c0 msgr2=0x7ffb301a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:35.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.711+0000 7ffb2effd640 1 --2- 172.21.15.96:0/4129930221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb301080c0 0x7ffb301a16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:35.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.711+0000 7ffb2effd640 1 -- 172.21.15.96:0/4129930221 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb301a1d10 con 0x7ffb301033c0 2024-04-25T14:13:35.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.711+0000 7ffb2effd640 1 --2- 172.21.15.96:0/4129930221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb301033c0 0x7ffb3019a710 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ffb24006010 tx=0x7ffb24004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:35.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.711+0000 7ffb2cff9640 1 -- 172.21.15.96:0/4129930221 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2403f030 con 0x7ffb301033c0 2024-04-25T14:13:35.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.711+0000 7ffb2cff9640 1 -- 172.21.15.96:0/4129930221 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb24003cb0 con 0x7ffb301033c0 2024-04-25T14:13:35.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.711+0000 7ffb2cff9640 1 -- 172.21.15.96:0/4129930221 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb24035070 con 0x7ffb301033c0 2024-04-25T14:13:35.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.711+0000 7ffb35d7d640 1 -- 172.21.15.96:0/4129930221 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb301a1fa0 con 0x7ffb301033c0 2024-04-25T14:13:35.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.711+0000 7ffb35d7d640 1 -- 172.21.15.96:0/4129930221 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb301a23b0 con 0x7ffb301033c0 2024-04-25T14:13:35.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.712+0000 7ffb35d7d640 1 -- 172.21.15.96:0/4129930221 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb301a2610 con 0x7ffb301033c0 2024-04-25T14:13:35.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.714+0000 7ffb2cff9640 1 -- 172.21.15.96:0/4129930221 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7ffb24042030 con 0x7ffb301033c0 2024-04-25T14:13:35.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.714+0000 7ffb2cff9640 1 --2- 172.21.15.96:0/4129930221 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ffb180afaf0 0x7ffb180b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:35.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.715+0000 7ffb2f7fe640 1 --2- 172.21.15.96:0/4129930221 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ffb180afaf0 0x7ffb180b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:35.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.715+0000 7ffb2cff9640 1 -- 172.21.15.96:0/4129930221 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7ffb240ff220 con 0x7ffb301033c0 2024-04-25T14:13:35.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.715+0000 7ffb2f7fe640 1 --2- 172.21.15.96:0/4129930221 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ffb180afaf0 0x7ffb180b2000 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ffb30102d80 tx=0x7ffb2000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:35.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.719+0000 7ffb2cff9640 1 -- 172.21.15.96:0/4129930221 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffb24003e60 con 0x7ffb301033c0 2024-04-25T14:13:35.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:35 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/577196138' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:13:35.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:35 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/2484256442' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:13:35.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:35 smithi096 ceph-mon[29207]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T14:13:35.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:35 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/1462640069' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:13:35.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.896+0000 7ffb35d7d640 1 -- 172.21.15.96:0/4129930221 --> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ffb301070c0 con 0x7ffb180afaf0 2024-04-25T14:13:35.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.904+0000 7ffb2cff9640 1 -- 172.21.15.96:0/4129930221 <== mgr.14152 v2:172.21.15.96:6800/536332425 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259165 (secure 0 0 0) 0x7ffb301070c0 con 0x7ffb180afaf0 2024-04-25T14:13:35.907 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:13:35.908 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-04-25T14:13:35.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.907+0000 7ffb35d7d640 1 -- 172.21.15.96:0/4129930221 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ffb180afaf0 msgr2=0x7ffb180b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:35.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.907+0000 7ffb35d7d640 1 --2- 172.21.15.96:0/4129930221 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ffb180afaf0 0x7ffb180b2000 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ffb30102d80 tx=0x7ffb2000b390 comp rx=0 tx=0).stop 2024-04-25T14:13:35.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.907+0000 7ffb35d7d640 1 -- 172.21.15.96:0/4129930221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb301033c0 msgr2=0x7ffb3019a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:35.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.907+0000 7ffb35d7d640 1 --2- 172.21.15.96:0/4129930221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb301033c0 0x7ffb3019a710 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ffb24006010 tx=0x7ffb24004340 comp rx=0 tx=0).stop 2024-04-25T14:13:35.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.907+0000 7ffb35d7d640 1 -- 172.21.15.96:0/4129930221 shutdown_connections 2024-04-25T14:13:35.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.907+0000 7ffb35d7d640 1 --2- 172.21.15.96:0/4129930221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb301080c0 0x7ffb301a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:35.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.907+0000 7ffb35d7d640 1 --2- 172.21.15.96:0/4129930221 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ffb180afaf0 0x7ffb180b2000 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:35.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.907+0000 7ffb35d7d640 1 --2- 172.21.15.96:0/4129930221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb301033c0 0x7ffb3019a710 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:35.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.907+0000 7ffb35d7d640 1 --2- 172.21.15.96:0/4129930221 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb30102180 0x7ffb3019a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:35.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.907+0000 7ffb35d7d640 1 -- 172.21.15.96:0/4129930221 >> 172.21.15.96:0/4129930221 conn(0x7ffb300fd370 msgr2=0x7ffb3006acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:35.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.907+0000 7ffb35d7d640 1 -- 172.21.15.96:0/4129930221 shutdown_connections 2024-04-25T14:13:35.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:35.907+0000 7ffb35d7d640 1 -- 172.21.15.96:0/4129930221 wait complete. 2024-04-25T14:13:35.919 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2024-04-25T14:13:35.432428+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657731,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1285,"num_read_kb":743,"num_write":372,"num_write_kb":2291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":2952580,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":808,"ondisk_log_size":808,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359604,"kb_used_data":10452,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124582220,"statfs":{"total":1151940427776,"available":1151572193280,"internally_reserved":0,"allocated":10702848,"data_stored":6822781,"data_compressed":182373,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":105,"num_read_kb":55,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003170"},"pg_stats":[{"pgid":"3.3f","version":"78'8","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.448766+0000","last_change":"2024-04-25T14:11:29.569615+0000","last_active":"2024-04-25T14:12:51.448766+0000","last_peered":"2024-04-25T14:12:51.448766+0000","last_clean":"2024-04-25T14:12:51.448766+0000","last_became_active":"2024-04-25T14:11:29.569237+0000","last_became_peered":"2024-04-25T14:11:29.569237+0000","last_unstale":"2024-04-25T14:12:51.448766+0000","last_undegraded":"2024-04-25T14:12:51.448766+0000","last_fullsized":"2024-04-25T14:12:51.448766+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:24.367533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"78'4","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.440396+0000","last_change":"2024-04-25T14:11:29.571820+0000","last_active":"2024-04-25T14:12:51.440396+0000","last_peered":"2024-04-25T14:12:51.440396+0000","last_clean":"2024-04-25T14:12:51.440396+0000","last_became_active":"2024-04-25T14:11:29.571456+0000","last_became_peered":"2024-04-25T14:11:29.571456+0000","last_unstale":"2024-04-25T14:12:51.440396+0000","last_undegraded":"2024-04-25T14:12:51.440396+0000","last_fullsized":"2024-04-25T14:12:51.440396+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:33.442301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"78'7","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.456139+0000","last_change":"2024-04-25T14:11:29.579302+0000","last_active":"2024-04-25T14:12:51.456139+0000","last_peered":"2024-04-25T14:12:51.456139+0000","last_clean":"2024-04-25T14:12:51.456139+0000","last_became_active":"2024-04-25T14:11:29.577734+0000","last_became_peered":"2024-04-25T14:11:29.577734+0000","last_unstale":"2024-04-25T14:12:51.456139+0000","last_undegraded":"2024-04-25T14:12:51.456139+0000","last_fullsized":"2024-04-25T14:12:51.456139+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:59.147554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"78'8","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.442356+0000","last_change":"2024-04-25T14:11:29.574942+0000","last_active":"2024-04-25T14:12:51.442356+0000","last_peered":"2024-04-25T14:12:51.442356+0000","last_clean":"2024-04-25T14:12:51.442356+0000","last_became_active":"2024-04-25T14:11:29.573774+0000","last_became_peered":"2024-04-25T14:11:29.573774+0000","last_unstale":"2024-04-25T14:12:51.442356+0000","last_undegraded":"2024-04-25T14:12:51.442356+0000","last_fullsized":"2024-04-25T14:12:51.442356+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:10.626265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"78'4","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.426798+0000","last_change":"2024-04-25T14:11:29.557199+0000","last_active":"2024-04-25T14:12:51.426798+0000","last_peered":"2024-04-25T14:12:51.426798+0000","last_clean":"2024-04-25T14:12:51.426798+0000","last_became_active":"2024-04-25T14:11:29.556836+0000","last_became_peered":"2024-04-25T14:11:29.556836+0000","last_unstale":"2024-04-25T14:12:51.426798+0000","last_undegraded":"2024-04-25T14:12:51.426798+0000","last_fullsized":"2024-04-25T14:12:51.426798+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:04:45.882935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"78'6","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.434655+0000","last_change":"2024-04-25T14:11:29.557988+0000","last_active":"2024-04-25T14:12:51.434655+0000","last_peered":"2024-04-25T14:12:51.434655+0000","last_clean":"2024-04-25T14:12:51.434655+0000","last_became_active":"2024-04-25T14:11:29.557622+0000","last_became_peered":"2024-04-25T14:11:29.557622+0000","last_unstale":"2024-04-25T14:12:51.434655+0000","last_undegraded":"2024-04-25T14:12:51.434655+0000","last_fullsized":"2024-04-25T14:12:51.434655+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:28:37.578368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"78'5","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.452293+0000","last_change":"2024-04-25T14:11:29.565049+0000","last_active":"2024-04-25T14:12:51.452293+0000","last_peered":"2024-04-25T14:12:51.452293+0000","last_clean":"2024-04-25T14:12:51.452293+0000","last_became_active":"2024-04-25T14:11:29.564688+0000","last_became_peered":"2024-04-25T14:11:29.564688+0000","last_unstale":"2024-04-25T14:12:51.452293+0000","last_undegraded":"2024-04-25T14:12:51.452293+0000","last_fullsized":"2024-04-25T14:12:51.452293+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:48.128778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"78'9","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.432114+0000","last_change":"2024-04-25T14:11:29.565879+0000","last_active":"2024-04-25T14:12:51.432114+0000","last_peered":"2024-04-25T14:12:51.432114+0000","last_clean":"2024-04-25T14:12:51.432114+0000","last_became_active":"2024-04-25T14:11:29.565525+0000","last_became_peered":"2024-04-25T14:11:29.565525+0000","last_unstale":"2024-04-25T14:12:51.432114+0000","last_undegraded":"2024-04-25T14:12:51.432114+0000","last_fullsized":"2024-04-25T14:12:51.432114+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:06.956334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"78'11","reported_seq":34,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.450891+0000","last_change":"2024-04-25T14:12:22.517776+0000","last_active":"2024-04-25T14:12:51.450891+0000","last_peered":"2024-04-25T14:12:51.450891+0000","last_clean":"2024-04-25T14:12:51.450891+0000","last_became_active":"2024-04-25T14:12:22.517232+0000","last_became_peered":"2024-04-25T14:12:22.517232+0000","last_unstale":"2024-04-25T14:12:51.450891+0000","last_undegraded":"2024-04-25T14:12:51.450891+0000","last_fullsized":"2024-04-25T14:12:51.450891+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:51.912445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,11],"acting":[1,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"78'16","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.438962+0000","last_change":"2024-04-25T14:11:29.561841+0000","last_active":"2024-04-25T14:12:51.438962+0000","last_peered":"2024-04-25T14:12:51.438962+0000","last_clean":"2024-04-25T14:12:51.438962+0000","last_became_active":"2024-04-25T14:11:29.560145+0000","last_became_peered":"2024-04-25T14:11:29.560145+0000","last_unstale":"2024-04-25T14:12:51.438962+0000","last_undegraded":"2024-04-25T14:12:51.438962+0000","last_fullsized":"2024-04-25T14:12:51.438962+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:11.654692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"78'6","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.459530+0000","last_change":"2024-04-25T14:11:29.570954+0000","last_active":"2024-04-25T14:12:51.459530+0000","last_peered":"2024-04-25T14:12:51.459530+0000","last_clean":"2024-04-25T14:12:51.459530+0000","last_became_active":"2024-04-25T14:11:29.570368+0000","last_became_peered":"2024-04-25T14:11:29.570368+0000","last_unstale":"2024-04-25T14:12:51.459530+0000","last_undegraded":"2024-04-25T14:12:51.459530+0000","last_fullsized":"2024-04-25T14:12:51.459530+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:50.153695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"78'7","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.443244+0000","last_change":"2024-04-25T14:11:29.571911+0000","last_active":"2024-04-25T14:12:51.443244+0000","last_peered":"2024-04-25T14:12:51.443244+0000","last_clean":"2024-04-25T14:12:51.443244+0000","last_became_active":"2024-04-25T14:11:29.568557+0000","last_became_peered":"2024-04-25T14:11:29.568557+0000","last_unstale":"2024-04-25T14:12:51.443244+0000","last_undegraded":"2024-04-25T14:12:51.443244+0000","last_fullsized":"2024-04-25T14:12:51.443244+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:24.221527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"78'10","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.426972+0000","last_change":"2024-04-25T14:11:29.566826+0000","last_active":"2024-04-25T14:12:51.426972+0000","last_peered":"2024-04-25T14:12:51.426972+0000","last_clean":"2024-04-25T14:12:51.426972+0000","last_became_active":"2024-04-25T14:11:29.565825+0000","last_became_peered":"2024-04-25T14:11:29.565825+0000","last_unstale":"2024-04-25T14:12:51.426972+0000","last_undegraded":"2024-04-25T14:12:51.426972+0000","last_fullsized":"2024-04-25T14:12:51.426972+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:46.711809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"78'7","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.424340+0000","last_change":"2024-04-25T14:11:29.577359+0000","last_active":"2024-04-25T14:12:51.424340+0000","last_peered":"2024-04-25T14:12:51.424340+0000","last_clean":"2024-04-25T14:12:51.424340+0000","last_became_active":"2024-04-25T14:11:29.574643+0000","last_became_peered":"2024-04-25T14:11:29.574643+0000","last_unstale":"2024-04-25T14:12:51.424340+0000","last_undegraded":"2024-04-25T14:12:51.424340+0000","last_fullsized":"2024-04-25T14:12:51.424340+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:07.582154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"78'7","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.432124+0000","last_change":"2024-04-25T14:11:29.583606+0000","last_active":"2024-04-25T14:12:51.432124+0000","last_peered":"2024-04-25T14:12:51.432124+0000","last_clean":"2024-04-25T14:12:51.432124+0000","last_became_active":"2024-04-25T14:11:29.582091+0000","last_became_peered":"2024-04-25T14:11:29.582091+0000","last_unstale":"2024-04-25T14:12:51.432124+0000","last_undegraded":"2024-04-25T14:12:51.432124+0000","last_fullsized":"2024-04-25T14:12:51.432124+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:58.711591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"78'9","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.458160+0000","last_change":"2024-04-25T14:11:29.571223+0000","last_active":"2024-04-25T14:12:51.458160+0000","last_peered":"2024-04-25T14:12:51.458160+0000","last_clean":"2024-04-25T14:12:51.458160+0000","last_became_active":"2024-04-25T14:11:29.570856+0000","last_became_peered":"2024-04-25T14:11:29.570856+0000","last_unstale":"2024-04-25T14:12:51.458160+0000","last_undegraded":"2024-04-25T14:12:51.458160+0000","last_fullsized":"2024-04-25T14:12:51.458160+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:53.058088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"78'8","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.449162+0000","last_change":"2024-04-25T14:11:29.574467+0000","last_active":"2024-04-25T14:12:51.449162+0000","last_peered":"2024-04-25T14:12:51.449162+0000","last_clean":"2024-04-25T14:12:51.449162+0000","last_became_active":"2024-04-25T14:11:29.573878+0000","last_became_peered":"2024-04-25T14:11:29.573878+0000","last_unstale":"2024-04-25T14:12:51.449162+0000","last_undegraded":"2024-04-25T14:12:51.449162+0000","last_fullsized":"2024-04-25T14:12:51.449162+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:22:02.087606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"78'10","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.454589+0000","last_change":"2024-04-25T14:11:29.561612+0000","last_active":"2024-04-25T14:12:51.454589+0000","last_peered":"2024-04-25T14:12:51.454589+0000","last_clean":"2024-04-25T14:12:51.454589+0000","last_became_active":"2024-04-25T14:11:29.561211+0000","last_became_peered":"2024-04-25T14:11:29.561211+0000","last_unstale":"2024-04-25T14:12:51.454589+0000","last_undegraded":"2024-04-25T14:12:51.454589+0000","last_fullsized":"2024-04-25T14:12:51.454589+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:29.195093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"78'7","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.430634+0000","last_change":"2024-04-25T14:11:29.575231+0000","last_active":"2024-04-25T14:12:51.430634+0000","last_peered":"2024-04-25T14:12:51.430634+0000","last_clean":"2024-04-25T14:12:51.430634+0000","last_became_active":"2024-04-25T14:11:29.573928+0000","last_became_peered":"2024-04-25T14:11:29.573928+0000","last_unstale":"2024-04-25T14:12:51.430634+0000","last_undegraded":"2024-04-25T14:12:51.430634+0000","last_fullsized":"2024-04-25T14:12:51.430634+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:10.025584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"76'13","reported_seq":172,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.613466+0000","last_change":"2024-04-25T14:12:22.527280+0000","last_active":"2024-04-25T14:13:33.613466+0000","last_peered":"2024-04-25T14:13:33.613466+0000","last_clean":"2024-04-25T14:13:33.613466+0000","last_became_active":"2024-04-25T14:12:22.515420+0000","last_became_peered":"2024-04-25T14:12:22.515420+0000","last_unstale":"2024-04-25T14:13:33.613466+0000","last_undegraded":"2024-04-25T14:13:33.613466+0000","last_fullsized":"2024-04-25T14:13:33.613466+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:53.882212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"78'5","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.427842+0000","last_change":"2024-04-25T14:11:29.576855+0000","last_active":"2024-04-25T14:12:51.427842+0000","last_peered":"2024-04-25T14:12:51.427842+0000","last_clean":"2024-04-25T14:12:51.427842+0000","last_became_active":"2024-04-25T14:11:29.574566+0000","last_became_peered":"2024-04-25T14:11:29.574566+0000","last_unstale":"2024-04-25T14:12:51.427842+0000","last_undegraded":"2024-04-25T14:12:51.427842+0000","last_fullsized":"2024-04-25T14:12:51.427842+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:31.325583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"76'2","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514493+0000","last_change":"2024-04-25T14:11:28.552763+0000","last_active":"2024-04-25T14:12:22.514493+0000","last_peered":"2024-04-25T14:12:22.514493+0000","last_clean":"2024-04-25T14:12:22.514493+0000","last_became_active":"2024-04-25T14:11:28.552403+0000","last_became_peered":"2024-04-25T14:11:28.552403+0000","last_unstale":"2024-04-25T14:12:22.514493+0000","last_undegraded":"2024-04-25T14:12:22.514493+0000","last_fullsized":"2024-04-25T14:12:22.514493+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:31.473779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"78'6","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.456271+0000","last_change":"2024-04-25T14:11:29.571019+0000","last_active":"2024-04-25T14:12:51.456271+0000","last_peered":"2024-04-25T14:12:51.456271+0000","last_clean":"2024-04-25T14:12:51.456271+0000","last_became_active":"2024-04-25T14:11:29.570633+0000","last_became_peered":"2024-04-25T14:11:29.570633+0000","last_unstale":"2024-04-25T14:12:51.456271+0000","last_undegraded":"2024-04-25T14:12:51.456271+0000","last_fullsized":"2024-04-25T14:12:51.456271+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:38:32.664318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"76'12","reported_seq":32,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514608+0000","last_change":"2024-04-25T14:11:28.545189+0000","last_active":"2024-04-25T14:12:22.514608+0000","last_peered":"2024-04-25T14:12:22.514608+0000","last_clean":"2024-04-25T14:12:22.514608+0000","last_became_active":"2024-04-25T14:11:28.544857+0000","last_became_peered":"2024-04-25T14:11:28.544857+0000","last_unstale":"2024-04-25T14:12:22.514608+0000","last_undegraded":"2024-04-25T14:12:22.514608+0000","last_fullsized":"2024-04-25T14:12:22.514608+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:58.464226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"78'9","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.455643+0000","last_change":"2024-04-25T14:11:29.576696+0000","last_active":"2024-04-25T14:12:51.455643+0000","last_peered":"2024-04-25T14:12:51.455643+0000","last_clean":"2024-04-25T14:12:51.455643+0000","last_became_active":"2024-04-25T14:11:29.576337+0000","last_became_peered":"2024-04-25T14:11:29.576337+0000","last_unstale":"2024-04-25T14:12:51.455643+0000","last_undegraded":"2024-04-25T14:12:51.455643+0000","last_fullsized":"2024-04-25T14:12:51.455643+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:31:30.410434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"76'5","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514457+0000","last_change":"2024-04-25T14:11:28.542466+0000","last_active":"2024-04-25T14:12:22.514457+0000","last_peered":"2024-04-25T14:12:22.514457+0000","last_clean":"2024-04-25T14:12:22.514457+0000","last_became_active":"2024-04-25T14:11:28.542092+0000","last_became_peered":"2024-04-25T14:11:28.542092+0000","last_unstale":"2024-04-25T14:12:22.514457+0000","last_undegraded":"2024-04-25T14:12:22.514457+0000","last_fullsized":"2024-04-25T14:12:22.514457+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:33.810111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"78'15","reported_seq":32,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.441623+0000","last_change":"2024-04-25T14:11:29.567842+0000","last_active":"2024-04-25T14:12:51.441623+0000","last_peered":"2024-04-25T14:12:51.441623+0000","last_clean":"2024-04-25T14:12:51.441623+0000","last_became_active":"2024-04-25T14:11:29.566269+0000","last_became_peered":"2024-04-25T14:11:29.566269+0000","last_unstale":"2024-04-25T14:12:51.441623+0000","last_undegraded":"2024-04-25T14:12:51.441623+0000","last_fullsized":"2024-04-25T14:12:51.441623+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:31.009318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"78'7","reported_seq":149,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:13:29.686280+0000","last_change":"2024-04-25T14:11:28.546215+0000","last_active":"2024-04-25T14:13:29.686280+0000","last_peered":"2024-04-25T14:13:29.686280+0000","last_clean":"2024-04-25T14:13:29.686280+0000","last_became_active":"2024-04-25T14:11:28.545828+0000","last_became_peered":"2024-04-25T14:11:28.545828+0000","last_unstale":"2024-04-25T14:13:29.686280+0000","last_undegraded":"2024-04-25T14:13:29.686280+0000","last_fullsized":"2024-04-25T14:13:29.686280+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:27.152582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":123,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"78'7","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.422934+0000","last_change":"2024-04-25T14:11:29.559566+0000","last_active":"2024-04-25T14:12:51.422934+0000","last_peered":"2024-04-25T14:12:51.422934+0000","last_clean":"2024-04-25T14:12:51.422934+0000","last_became_active":"2024-04-25T14:11:29.558560+0000","last_became_peered":"2024-04-25T14:11:29.558560+0000","last_unstale":"2024-04-25T14:12:51.422934+0000","last_undegraded":"2024-04-25T14:12:51.422934+0000","last_fullsized":"2024-04-25T14:12:51.422934+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:23.677099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"76'7","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.515784+0000","last_change":"2024-04-25T14:11:28.544020+0000","last_active":"2024-04-25T14:12:22.515784+0000","last_peered":"2024-04-25T14:12:22.515784+0000","last_clean":"2024-04-25T14:12:22.515784+0000","last_became_active":"2024-04-25T14:11:28.543598+0000","last_became_peered":"2024-04-25T14:11:28.543598+0000","last_unstale":"2024-04-25T14:12:22.515784+0000","last_undegraded":"2024-04-25T14:12:22.515784+0000","last_fullsized":"2024-04-25T14:12:22.515784+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:49:06.543876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"78'8","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.438175+0000","last_change":"2024-04-25T14:11:29.576122+0000","last_active":"2024-04-25T14:12:51.438175+0000","last_peered":"2024-04-25T14:12:51.438175+0000","last_clean":"2024-04-25T14:12:51.438175+0000","last_became_active":"2024-04-25T14:11:29.573668+0000","last_became_peered":"2024-04-25T14:11:29.573668+0000","last_unstale":"2024-04-25T14:12:51.438175+0000","last_undegraded":"2024-04-25T14:12:51.438175+0000","last_fullsized":"2024-04-25T14:12:51.438175+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:06.998876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"76'3","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.505016+0000","last_change":"2024-04-25T14:11:28.541554+0000","last_active":"2024-04-25T14:12:22.505016+0000","last_peered":"2024-04-25T14:12:22.505016+0000","last_clean":"2024-04-25T14:12:22.505016+0000","last_became_active":"2024-04-25T14:11:28.541062+0000","last_became_peered":"2024-04-25T14:11:28.541062+0000","last_unstale":"2024-04-25T14:12:22.505016+0000","last_undegraded":"2024-04-25T14:12:22.505016+0000","last_fullsized":"2024-04-25T14:12:22.505016+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:16.719820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"78'8","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.433513+0000","last_change":"2024-04-25T14:11:29.564550+0000","last_active":"2024-04-25T14:12:51.433513+0000","last_peered":"2024-04-25T14:12:51.433513+0000","last_clean":"2024-04-25T14:12:51.433513+0000","last_became_active":"2024-04-25T14:11:29.564193+0000","last_became_peered":"2024-04-25T14:11:29.564193+0000","last_unstale":"2024-04-25T14:12:51.433513+0000","last_undegraded":"2024-04-25T14:12:51.433513+0000","last_fullsized":"2024-04-25T14:12:51.433513+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:59.357327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"76'3","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.508406+0000","last_change":"2024-04-25T14:11:28.540504+0000","last_active":"2024-04-25T14:12:22.508406+0000","last_peered":"2024-04-25T14:12:22.508406+0000","last_clean":"2024-04-25T14:12:22.508406+0000","last_became_active":"2024-04-25T14:11:28.540150+0000","last_became_peered":"2024-04-25T14:11:28.540150+0000","last_unstale":"2024-04-25T14:12:22.508406+0000","last_undegraded":"2024-04-25T14:12:22.508406+0000","last_fullsized":"2024-04-25T14:12:22.508406+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:35.331489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"78'11","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.430603+0000","last_change":"2024-04-25T14:11:29.561226+0000","last_active":"2024-04-25T14:12:51.430603+0000","last_peered":"2024-04-25T14:12:51.430603+0000","last_clean":"2024-04-25T14:12:51.430603+0000","last_became_active":"2024-04-25T14:11:29.560847+0000","last_became_peered":"2024-04-25T14:11:29.560847+0000","last_unstale":"2024-04-25T14:12:51.430603+0000","last_undegraded":"2024-04-25T14:12:51.430603+0000","last_fullsized":"2024-04-25T14:12:51.430603+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:05.299388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'5","reported_seq":259,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:13:30.687104+0000","last_change":"2024-04-25T14:11:28.548580+0000","last_active":"2024-04-25T14:13:30.687104+0000","last_peered":"2024-04-25T14:13:30.687104+0000","last_clean":"2024-04-25T14:13:30.687104+0000","last_became_active":"2024-04-25T14:11:28.548250+0000","last_became_peered":"2024-04-25T14:11:28.548250+0000","last_unstale":"2024-04-25T14:13:30.687104+0000","last_undegraded":"2024-04-25T14:13:30.687104+0000","last_fullsized":"2024-04-25T14:13:30.687104+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:01.166285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":235,"num_read_kb":235,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"78'10","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.455188+0000","last_change":"2024-04-25T14:11:29.575983+0000","last_active":"2024-04-25T14:12:51.455188+0000","last_peered":"2024-04-25T14:12:51.455188+0000","last_clean":"2024-04-25T14:12:51.455188+0000","last_became_active":"2024-04-25T14:11:29.575147+0000","last_became_peered":"2024-04-25T14:11:29.575147+0000","last_unstale":"2024-04-25T14:12:51.455188+0000","last_undegraded":"2024-04-25T14:12:51.455188+0000","last_fullsized":"2024-04-25T14:12:51.455188+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:40:45.366736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"76'4","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.530419+0000","last_change":"2024-04-25T14:12:22.530419+0000","last_active":"2024-04-25T14:12:22.530419+0000","last_peered":"2024-04-25T14:12:22.530419+0000","last_clean":"2024-04-25T14:12:22.530419+0000","last_became_active":"2024-04-25T14:12:22.514999+0000","last_became_peered":"2024-04-25T14:12:22.514999+0000","last_unstale":"2024-04-25T14:12:22.530419+0000","last_undegraded":"2024-04-25T14:12:22.530419+0000","last_fullsized":"2024-04-25T14:12:22.530419+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:02.075270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"78'9","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.424732+0000","last_change":"2024-04-25T14:11:29.581424+0000","last_active":"2024-04-25T14:12:51.424732+0000","last_peered":"2024-04-25T14:12:51.424732+0000","last_clean":"2024-04-25T14:12:51.424732+0000","last_became_active":"2024-04-25T14:11:29.581086+0000","last_became_peered":"2024-04-25T14:11:29.581086+0000","last_unstale":"2024-04-25T14:12:51.424732+0000","last_undegraded":"2024-04-25T14:12:51.424732+0000","last_fullsized":"2024-04-25T14:12:51.424732+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:30.468728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"76'5","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.508977+0000","last_change":"2024-04-25T14:11:28.540771+0000","last_active":"2024-04-25T14:12:22.508977+0000","last_peered":"2024-04-25T14:12:22.508977+0000","last_clean":"2024-04-25T14:12:22.508977+0000","last_became_active":"2024-04-25T14:11:28.540442+0000","last_became_peered":"2024-04-25T14:11:28.540442+0000","last_unstale":"2024-04-25T14:12:22.508977+0000","last_undegraded":"2024-04-25T14:12:22.508977+0000","last_fullsized":"2024-04-25T14:12:22.508977+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:52.479423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"78'11","reported_seq":34,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.458541+0000","last_change":"2024-04-25T14:12:22.517721+0000","last_active":"2024-04-25T14:12:51.458541+0000","last_peered":"2024-04-25T14:12:51.458541+0000","last_clean":"2024-04-25T14:12:51.458541+0000","last_became_active":"2024-04-25T14:12:22.517232+0000","last_became_peered":"2024-04-25T14:12:22.517232+0000","last_unstale":"2024-04-25T14:12:51.458541+0000","last_undegraded":"2024-04-25T14:12:51.458541+0000","last_fullsized":"2024-04-25T14:12:51.458541+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:46.150626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"78'32","reported_seq":297,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:13:30.686843+0000","last_change":"2024-04-25T14:11:28.543093+0000","last_active":"2024-04-25T14:13:30.686843+0000","last_peered":"2024-04-25T14:13:30.686843+0000","last_clean":"2024-04-25T14:13:30.686843+0000","last_became_active":"2024-04-25T14:11:28.542407+0000","last_became_peered":"2024-04-25T14:11:28.542407+0000","last_unstale":"2024-04-25T14:13:30.686843+0000","last_undegraded":"2024-04-25T14:13:30.686843+0000","last_fullsized":"2024-04-25T14:13:30.686843+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:43.767375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59643,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":246,"num_read_kb":181,"num_write":34,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"78'6","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.455852+0000","last_change":"2024-04-25T14:11:29.571896+0000","last_active":"2024-04-25T14:12:51.455852+0000","last_peered":"2024-04-25T14:12:51.455852+0000","last_clean":"2024-04-25T14:12:51.455852+0000","last_became_active":"2024-04-25T14:11:29.571537+0000","last_became_peered":"2024-04-25T14:11:29.571537+0000","last_unstale":"2024-04-25T14:12:51.455852+0000","last_undegraded":"2024-04-25T14:12:51.455852+0000","last_fullsized":"2024-04-25T14:12:51.455852+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:32:07.914841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"76'9","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.530152+0000","last_change":"2024-04-25T14:12:22.529692+0000","last_active":"2024-04-25T14:12:22.530152+0000","last_peered":"2024-04-25T14:12:22.530152+0000","last_clean":"2024-04-25T14:12:22.530152+0000","last_became_active":"2024-04-25T14:12:22.517225+0000","last_became_peered":"2024-04-25T14:12:22.517225+0000","last_unstale":"2024-04-25T14:12:22.530152+0000","last_undegraded":"2024-04-25T14:12:22.530152+0000","last_fullsized":"2024-04-25T14:12:22.530152+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:36.610551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"78'9","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.458904+0000","last_change":"2024-04-25T14:11:29.563877+0000","last_active":"2024-04-25T14:12:51.458904+0000","last_peered":"2024-04-25T14:12:51.458904+0000","last_clean":"2024-04-25T14:12:51.458904+0000","last_became_active":"2024-04-25T14:11:29.563535+0000","last_became_peered":"2024-04-25T14:11:29.563535+0000","last_unstale":"2024-04-25T14:12:51.458904+0000","last_undegraded":"2024-04-25T14:12:51.458904+0000","last_fullsized":"2024-04-25T14:12:51.458904+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:07:38.805863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"76'5","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.508526+0000","last_change":"2024-04-25T14:11:28.543787+0000","last_active":"2024-04-25T14:12:22.508526+0000","last_peered":"2024-04-25T14:12:22.508526+0000","last_clean":"2024-04-25T14:12:22.508526+0000","last_became_active":"2024-04-25T14:11:28.540816+0000","last_became_peered":"2024-04-25T14:11:28.540816+0000","last_unstale":"2024-04-25T14:12:22.508526+0000","last_undegraded":"2024-04-25T14:12:22.508526+0000","last_fullsized":"2024-04-25T14:12:22.508526+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:18.776280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"78'15","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.462197+0000","last_change":"2024-04-25T14:12:22.514028+0000","last_active":"2024-04-25T14:12:51.462197+0000","last_peered":"2024-04-25T14:12:51.462197+0000","last_clean":"2024-04-25T14:12:51.462197+0000","last_became_active":"2024-04-25T14:12:22.513536+0000","last_became_peered":"2024-04-25T14:12:22.513536+0000","last_unstale":"2024-04-25T14:12:51.462197+0000","last_undegraded":"2024-04-25T14:12:51.462197+0000","last_fullsized":"2024-04-25T14:12:51.462197+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:31.921284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'7","reported_seq":145,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:13:29.702488+0000","last_change":"2024-04-25T14:11:28.537450+0000","last_active":"2024-04-25T14:13:29.702488+0000","last_peered":"2024-04-25T14:13:29.702488+0000","last_clean":"2024-04-25T14:13:29.702488+0000","last_became_active":"2024-04-25T14:11:28.537088+0000","last_became_peered":"2024-04-25T14:11:28.537088+0000","last_unstale":"2024-04-25T14:13:29.702488+0000","last_undegraded":"2024-04-25T14:13:29.702488+0000","last_fullsized":"2024-04-25T14:13:29.702488+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:05.779706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":120,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"78'6","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.451486+0000","last_change":"2024-04-25T14:11:29.572789+0000","last_active":"2024-04-25T14:12:51.451486+0000","last_peered":"2024-04-25T14:12:51.451486+0000","last_clean":"2024-04-25T14:12:51.451486+0000","last_became_active":"2024-04-25T14:11:29.572201+0000","last_became_peered":"2024-04-25T14:11:29.572201+0000","last_unstale":"2024-04-25T14:12:51.451486+0000","last_undegraded":"2024-04-25T14:12:51.451486+0000","last_fullsized":"2024-04-25T14:12:51.451486+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:39:58.237070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"76'17","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514338+0000","last_change":"2024-04-25T14:11:28.540577+0000","last_active":"2024-04-25T14:12:22.514338+0000","last_peered":"2024-04-25T14:12:22.514338+0000","last_clean":"2024-04-25T14:12:22.514338+0000","last_became_active":"2024-04-25T14:11:28.540212+0000","last_became_peered":"2024-04-25T14:11:28.540212+0000","last_unstale":"2024-04-25T14:12:22.514338+0000","last_undegraded":"2024-04-25T14:12:22.514338+0000","last_fullsized":"2024-04-25T14:12:22.514338+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:31.422332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"78'6","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.437799+0000","last_change":"2024-04-25T14:11:29.574952+0000","last_active":"2024-04-25T14:12:51.437799+0000","last_peered":"2024-04-25T14:12:51.437799+0000","last_clean":"2024-04-25T14:12:51.437799+0000","last_became_active":"2024-04-25T14:11:29.573677+0000","last_became_peered":"2024-04-25T14:11:29.573677+0000","last_unstale":"2024-04-25T14:12:51.437799+0000","last_undegraded":"2024-04-25T14:12:51.437799+0000","last_fullsized":"2024-04-25T14:12:51.437799+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:35.896103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"76'2","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514622+0000","last_change":"2024-04-25T14:11:28.541356+0000","last_active":"2024-04-25T14:12:22.514622+0000","last_peered":"2024-04-25T14:12:22.514622+0000","last_clean":"2024-04-25T14:12:22.514622+0000","last_became_active":"2024-04-25T14:11:28.540976+0000","last_became_peered":"2024-04-25T14:11:28.540976+0000","last_unstale":"2024-04-25T14:12:22.514622+0000","last_undegraded":"2024-04-25T14:12:22.514622+0000","last_fullsized":"2024-04-25T14:12:22.514622+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:32:50.599900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"78'5","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.442075+0000","last_change":"2024-04-25T14:11:29.554317+0000","last_active":"2024-04-25T14:12:51.442075+0000","last_peered":"2024-04-25T14:12:51.442075+0000","last_clean":"2024-04-25T14:12:51.442075+0000","last_became_active":"2024-04-25T14:11:29.553935+0000","last_became_peered":"2024-04-25T14:11:29.553935+0000","last_unstale":"2024-04-25T14:12:51.442075+0000","last_undegraded":"2024-04-25T14:12:51.442075+0000","last_fullsized":"2024-04-25T14:12:51.442075+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:01.405964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"76'1","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.509102+0000","last_change":"2024-04-25T14:11:28.546892+0000","last_active":"2024-04-25T14:12:22.509102+0000","last_peered":"2024-04-25T14:12:22.509102+0000","last_clean":"2024-04-25T14:12:22.509102+0000","last_became_active":"2024-04-25T14:11:28.546566+0000","last_became_peered":"2024-04-25T14:11:28.546566+0000","last_unstale":"2024-04-25T14:12:22.509102+0000","last_undegraded":"2024-04-25T14:12:22.509102+0000","last_fullsized":"2024-04-25T14:12:22.509102+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:05.239635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'5","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.431262+0000","last_change":"2024-04-25T14:11:29.565083+0000","last_active":"2024-04-25T14:12:51.431262+0000","last_peered":"2024-04-25T14:12:51.431262+0000","last_clean":"2024-04-25T14:12:51.431262+0000","last_became_active":"2024-04-25T14:11:29.564062+0000","last_became_peered":"2024-04-25T14:11:29.564062+0000","last_unstale":"2024-04-25T14:12:51.431262+0000","last_undegraded":"2024-04-25T14:12:51.431262+0000","last_fullsized":"2024-04-25T14:12:51.431262+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:07:10.664113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"76'5","reported_seq":140,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:13:29.608884+0000","last_change":"2024-04-25T14:11:28.538649+0000","last_active":"2024-04-25T14:13:29.608884+0000","last_peered":"2024-04-25T14:13:29.608884+0000","last_clean":"2024-04-25T14:13:29.608884+0000","last_became_active":"2024-04-25T14:11:28.538353+0000","last_became_peered":"2024-04-25T14:11:28.538353+0000","last_unstale":"2024-04-25T14:13:29.608884+0000","last_undegraded":"2024-04-25T14:13:29.608884+0000","last_fullsized":"2024-04-25T14:13:29.608884+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:06.513039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"78'7","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.435524+0000","last_change":"2024-04-25T14:11:29.576263+0000","last_active":"2024-04-25T14:12:51.435524+0000","last_peered":"2024-04-25T14:12:51.435524+0000","last_clean":"2024-04-25T14:12:51.435524+0000","last_became_active":"2024-04-25T14:11:29.575877+0000","last_became_peered":"2024-04-25T14:11:29.575877+0000","last_unstale":"2024-04-25T14:12:51.435524+0000","last_undegraded":"2024-04-25T14:12:51.435524+0000","last_fullsized":"2024-04-25T14:12:51.435524+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:33:37.639537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"78'8","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.438492+0000","last_change":"2024-04-25T14:11:29.568220+0000","last_active":"2024-04-25T14:12:51.438492+0000","last_peered":"2024-04-25T14:12:51.438492+0000","last_clean":"2024-04-25T14:12:51.438492+0000","last_became_active":"2024-04-25T14:11:29.567741+0000","last_became_peered":"2024-04-25T14:11:29.567741+0000","last_unstale":"2024-04-25T14:12:51.438492+0000","last_undegraded":"2024-04-25T14:12:51.438492+0000","last_fullsized":"2024-04-25T14:12:51.438492+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:05.134466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"78'6","reported_seq":143,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:13:30.703173+0000","last_change":"2024-04-25T14:11:28.537503+0000","last_active":"2024-04-25T14:13:30.703173+0000","last_peered":"2024-04-25T14:13:30.703173+0000","last_clean":"2024-04-25T14:13:30.703173+0000","last_became_active":"2024-04-25T14:11:28.537169+0000","last_became_peered":"2024-04-25T14:11:28.537169+0000","last_unstale":"2024-04-25T14:13:30.703173+0000","last_undegraded":"2024-04-25T14:13:30.703173+0000","last_fullsized":"2024-04-25T14:13:30.703173+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:09.752988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"78'8","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.454448+0000","last_change":"2024-04-25T14:12:22.516273+0000","last_active":"2024-04-25T14:12:51.454448+0000","last_peered":"2024-04-25T14:12:51.454448+0000","last_clean":"2024-04-25T14:12:51.454448+0000","last_became_active":"2024-04-25T14:12:22.515737+0000","last_became_peered":"2024-04-25T14:12:22.515737+0000","last_unstale":"2024-04-25T14:12:51.454448+0000","last_undegraded":"2024-04-25T14:12:51.454448+0000","last_fullsized":"2024-04-25T14:12:51.454448+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:36.241587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"76'6","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.512277+0000","last_change":"2024-04-25T14:11:28.538216+0000","last_active":"2024-04-25T14:12:22.512277+0000","last_peered":"2024-04-25T14:12:22.512277+0000","last_clean":"2024-04-25T14:12:22.512277+0000","last_became_active":"2024-04-25T14:11:28.537772+0000","last_became_peered":"2024-04-25T14:11:28.537772+0000","last_unstale":"2024-04-25T14:12:22.512277+0000","last_undegraded":"2024-04-25T14:12:22.512277+0000","last_fullsized":"2024-04-25T14:12:22.512277+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:55.460966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":94,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.509039+0000","last_change":"2024-04-25T14:10:12.194216+0000","last_active":"2024-04-25T14:12:22.509039+0000","last_peered":"2024-04-25T14:12:22.509039+0000","last_clean":"2024-04-25T14:12:22.509039+0000","last_became_active":"2024-04-25T14:10:12.189616+0000","last_became_peered":"2024-04-25T14:10:12.189616+0000","last_unstale":"2024-04-25T14:12:22.509039+0000","last_undegraded":"2024-04-25T14:12:22.509039+0000","last_fullsized":"2024-04-25T14:12:22.509039+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:47.553264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:47.553264+0000","last_clean_scrub_stamp":"2024-04-25T14:06:47.553264+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:47.576058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"78'5","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.433764+0000","last_change":"2024-04-25T14:11:29.574985+0000","last_active":"2024-04-25T14:12:51.433764+0000","last_peered":"2024-04-25T14:12:51.433764+0000","last_clean":"2024-04-25T14:12:51.433764+0000","last_became_active":"2024-04-25T14:11:29.574629+0000","last_became_peered":"2024-04-25T14:11:29.574629+0000","last_unstale":"2024-04-25T14:12:51.433764+0000","last_undegraded":"2024-04-25T14:12:51.433764+0000","last_fullsized":"2024-04-25T14:12:51.433764+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:01.899889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"76'4","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.520398+0000","last_change":"2024-04-25T14:12:22.519851+0000","last_active":"2024-04-25T14:12:22.520398+0000","last_peered":"2024-04-25T14:12:22.520398+0000","last_clean":"2024-04-25T14:12:22.520398+0000","last_became_active":"2024-04-25T14:12:22.515791+0000","last_became_peered":"2024-04-25T14:12:22.515791+0000","last_unstale":"2024-04-25T14:12:22.520398+0000","last_undegraded":"2024-04-25T14:12:22.520398+0000","last_fullsized":"2024-04-25T14:12:22.520398+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:47:01.242259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"76'4","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514464+0000","last_change":"2024-04-25T14:11:28.544269+0000","last_active":"2024-04-25T14:12:22.514464+0000","last_peered":"2024-04-25T14:12:22.514464+0000","last_clean":"2024-04-25T14:12:22.514464+0000","last_became_active":"2024-04-25T14:11:28.543897+0000","last_became_peered":"2024-04-25T14:11:28.543897+0000","last_unstale":"2024-04-25T14:12:22.514464+0000","last_undegraded":"2024-04-25T14:12:22.514464+0000","last_fullsized":"2024-04-25T14:12:22.514464+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:37.668187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"78'9","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.431345+0000","last_change":"2024-04-25T14:11:29.566428+0000","last_active":"2024-04-25T14:12:51.431345+0000","last_peered":"2024-04-25T14:12:51.431345+0000","last_clean":"2024-04-25T14:12:51.431345+0000","last_became_active":"2024-04-25T14:11:29.562598+0000","last_became_peered":"2024-04-25T14:11:29.562598+0000","last_unstale":"2024-04-25T14:12:51.431345+0000","last_undegraded":"2024-04-25T14:12:51.431345+0000","last_fullsized":"2024-04-25T14:12:51.431345+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:46.611359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"78'14","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.561086+0000","last_change":"2024-04-25T14:11:28.542571+0000","last_active":"2024-04-25T14:12:51.561086+0000","last_peered":"2024-04-25T14:12:51.561086+0000","last_clean":"2024-04-25T14:12:51.561086+0000","last_became_active":"2024-04-25T14:11:28.542272+0000","last_became_peered":"2024-04-25T14:11:28.542272+0000","last_unstale":"2024-04-25T14:12:51.561086+0000","last_undegraded":"2024-04-25T14:12:51.561086+0000","last_fullsized":"2024-04-25T14:12:51.561086+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:41:05.128068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"78'5","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.458486+0000","last_change":"2024-04-25T14:11:29.578565+0000","last_active":"2024-04-25T14:12:51.458486+0000","last_peered":"2024-04-25T14:12:51.458486+0000","last_clean":"2024-04-25T14:12:51.458486+0000","last_became_active":"2024-04-25T14:11:29.576782+0000","last_became_peered":"2024-04-25T14:11:29.576782+0000","last_unstale":"2024-04-25T14:12:51.458486+0000","last_undegraded":"2024-04-25T14:12:51.458486+0000","last_fullsized":"2024-04-25T14:12:51.458486+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:31.634602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"76'7","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.511746+0000","last_change":"2024-04-25T14:11:28.543838+0000","last_active":"2024-04-25T14:12:22.511746+0000","last_peered":"2024-04-25T14:12:22.511746+0000","last_clean":"2024-04-25T14:12:22.511746+0000","last_became_active":"2024-04-25T14:11:28.543511+0000","last_became_peered":"2024-04-25T14:11:28.543511+0000","last_unstale":"2024-04-25T14:12:22.511746+0000","last_undegraded":"2024-04-25T14:12:22.511746+0000","last_fullsized":"2024-04-25T14:12:22.511746+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:34.440234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"78'7","reported_seq":30,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.426925+0000","last_change":"2024-04-25T14:12:22.516416+0000","last_active":"2024-04-25T14:12:51.426925+0000","last_peered":"2024-04-25T14:12:51.426925+0000","last_clean":"2024-04-25T14:12:51.426925+0000","last_became_active":"2024-04-25T14:12:22.515928+0000","last_became_peered":"2024-04-25T14:12:22.515928+0000","last_unstale":"2024-04-25T14:12:51.426925+0000","last_undegraded":"2024-04-25T14:12:51.426925+0000","last_fullsized":"2024-04-25T14:12:51.426925+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:26.989293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"76'2","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.512719+0000","last_change":"2024-04-25T14:11:28.539443+0000","last_active":"2024-04-25T14:12:22.512719+0000","last_peered":"2024-04-25T14:12:22.512719+0000","last_clean":"2024-04-25T14:12:22.512719+0000","last_became_active":"2024-04-25T14:11:28.539113+0000","last_became_peered":"2024-04-25T14:11:28.539113+0000","last_unstale":"2024-04-25T14:12:22.512719+0000","last_undegraded":"2024-04-25T14:12:22.512719+0000","last_fullsized":"2024-04-25T14:12:22.512719+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:36.664972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"78'2","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.415389+0000","last_change":"2024-04-25T14:11:29.575451+0000","last_active":"2024-04-25T14:12:51.415389+0000","last_peered":"2024-04-25T14:12:51.415389+0000","last_clean":"2024-04-25T14:12:51.415389+0000","last_became_active":"2024-04-25T14:11:29.574983+0000","last_became_peered":"2024-04-25T14:11:29.574983+0000","last_unstale":"2024-04-25T14:12:51.415389+0000","last_undegraded":"2024-04-25T14:12:51.415389+0000","last_fullsized":"2024-04-25T14:12:51.415389+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:53.317608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"76'7","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514234+0000","last_change":"2024-04-25T14:11:28.546394+0000","last_active":"2024-04-25T14:12:22.514234+0000","last_peered":"2024-04-25T14:12:22.514234+0000","last_clean":"2024-04-25T14:12:22.514234+0000","last_became_active":"2024-04-25T14:11:28.546064+0000","last_became_peered":"2024-04-25T14:11:28.546064+0000","last_unstale":"2024-04-25T14:12:22.514234+0000","last_undegraded":"2024-04-25T14:12:22.514234+0000","last_fullsized":"2024-04-25T14:12:22.514234+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:15.301492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"78'9","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.439570+0000","last_change":"2024-04-25T14:11:29.569584+0000","last_active":"2024-04-25T14:12:51.439570+0000","last_peered":"2024-04-25T14:12:51.439570+0000","last_clean":"2024-04-25T14:12:51.439570+0000","last_became_active":"2024-04-25T14:11:29.569217+0000","last_became_peered":"2024-04-25T14:11:29.569217+0000","last_unstale":"2024-04-25T14:12:51.439570+0000","last_undegraded":"2024-04-25T14:12:51.439570+0000","last_fullsized":"2024-04-25T14:12:51.439570+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:05.708277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"76'3","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.512047+0000","last_change":"2024-04-25T14:11:28.544399+0000","last_active":"2024-04-25T14:12:22.512047+0000","last_peered":"2024-04-25T14:12:22.512047+0000","last_clean":"2024-04-25T14:12:22.512047+0000","last_became_active":"2024-04-25T14:11:28.544098+0000","last_became_peered":"2024-04-25T14:11:28.544098+0000","last_unstale":"2024-04-25T14:12:22.512047+0000","last_undegraded":"2024-04-25T14:12:22.512047+0000","last_fullsized":"2024-04-25T14:12:22.512047+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:05.148386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"78'8","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.430478+0000","last_change":"2024-04-25T14:12:22.517727+0000","last_active":"2024-04-25T14:12:51.430478+0000","last_peered":"2024-04-25T14:12:51.430478+0000","last_clean":"2024-04-25T14:12:51.430478+0000","last_became_active":"2024-04-25T14:12:22.517364+0000","last_became_peered":"2024-04-25T14:12:22.517364+0000","last_unstale":"2024-04-25T14:12:51.430478+0000","last_undegraded":"2024-04-25T14:12:51.430478+0000","last_fullsized":"2024-04-25T14:12:51.430478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:30:11.023308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"76'4","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514211+0000","last_change":"2024-04-25T14:11:28.542488+0000","last_active":"2024-04-25T14:12:22.514211+0000","last_peered":"2024-04-25T14:12:22.514211+0000","last_clean":"2024-04-25T14:12:22.514211+0000","last_became_active":"2024-04-25T14:11:28.542171+0000","last_became_peered":"2024-04-25T14:11:28.542171+0000","last_unstale":"2024-04-25T14:12:22.514211+0000","last_undegraded":"2024-04-25T14:12:22.514211+0000","last_fullsized":"2024-04-25T14:12:22.514211+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:48.201762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"78'5","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.459752+0000","last_change":"2024-04-25T14:11:29.573805+0000","last_active":"2024-04-25T14:12:51.459752+0000","last_peered":"2024-04-25T14:12:51.459752+0000","last_clean":"2024-04-25T14:12:51.459752+0000","last_became_active":"2024-04-25T14:11:29.573420+0000","last_became_peered":"2024-04-25T14:11:29.573420+0000","last_unstale":"2024-04-25T14:12:51.459752+0000","last_undegraded":"2024-04-25T14:12:51.459752+0000","last_fullsized":"2024-04-25T14:12:51.459752+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:18.721977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"76'5","reported_seq":141,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:13:28.701588+0000","last_change":"2024-04-25T14:11:28.552863+0000","last_active":"2024-04-25T14:13:28.701588+0000","last_peered":"2024-04-25T14:13:28.701588+0000","last_clean":"2024-04-25T14:13:28.701588+0000","last_became_active":"2024-04-25T14:11:28.552461+0000","last_became_peered":"2024-04-25T14:11:28.552461+0000","last_unstale":"2024-04-25T14:13:28.701588+0000","last_undegraded":"2024-04-25T14:13:28.701588+0000","last_fullsized":"2024-04-25T14:13:28.701588+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:46.264473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":117,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"78'7","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.459450+0000","last_change":"2024-04-25T14:11:29.571268+0000","last_active":"2024-04-25T14:12:51.459450+0000","last_peered":"2024-04-25T14:12:51.459450+0000","last_clean":"2024-04-25T14:12:51.459450+0000","last_became_active":"2024-04-25T14:11:29.570876+0000","last_became_peered":"2024-04-25T14:11:29.570876+0000","last_unstale":"2024-04-25T14:12:51.459450+0000","last_undegraded":"2024-04-25T14:12:51.459450+0000","last_fullsized":"2024-04-25T14:12:51.459450+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:15.093635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"76'3","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.508764+0000","last_change":"2024-04-25T14:11:28.543343+0000","last_active":"2024-04-25T14:12:22.508764+0000","last_peered":"2024-04-25T14:12:22.508764+0000","last_clean":"2024-04-25T14:12:22.508764+0000","last_became_active":"2024-04-25T14:11:28.540790+0000","last_became_peered":"2024-04-25T14:11:28.540790+0000","last_unstale":"2024-04-25T14:12:22.508764+0000","last_undegraded":"2024-04-25T14:12:22.508764+0000","last_fullsized":"2024-04-25T14:12:22.508764+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:20.498016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'7","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.458962+0000","last_change":"2024-04-25T14:11:29.566161+0000","last_active":"2024-04-25T14:12:51.458962+0000","last_peered":"2024-04-25T14:12:51.458962+0000","last_clean":"2024-04-25T14:12:51.458962+0000","last_became_active":"2024-04-25T14:11:29.564197+0000","last_became_peered":"2024-04-25T14:11:29.564197+0000","last_unstale":"2024-04-25T14:12:51.458962+0000","last_undegraded":"2024-04-25T14:12:51.458962+0000","last_fullsized":"2024-04-25T14:12:51.458962+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:35.291546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"76'5","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.505214+0000","last_change":"2024-04-25T14:11:28.543479+0000","last_active":"2024-04-25T14:12:22.505214+0000","last_peered":"2024-04-25T14:12:22.505214+0000","last_clean":"2024-04-25T14:12:22.505214+0000","last_became_active":"2024-04-25T14:11:28.543167+0000","last_became_peered":"2024-04-25T14:11:28.543167+0000","last_unstale":"2024-04-25T14:12:22.505214+0000","last_undegraded":"2024-04-25T14:12:22.505214+0000","last_fullsized":"2024-04-25T14:12:22.505214+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:54:53.572809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"78'9","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.455482+0000","last_change":"2024-04-25T14:11:29.572548+0000","last_active":"2024-04-25T14:12:51.455482+0000","last_peered":"2024-04-25T14:12:51.455482+0000","last_clean":"2024-04-25T14:12:51.455482+0000","last_became_active":"2024-04-25T14:11:29.572167+0000","last_became_peered":"2024-04-25T14:11:29.572167+0000","last_unstale":"2024-04-25T14:12:51.455482+0000","last_undegraded":"2024-04-25T14:12:51.455482+0000","last_fullsized":"2024-04-25T14:12:51.455482+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:38.423331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"78'9","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.440678+0000","last_change":"2024-04-25T14:11:29.560956+0000","last_active":"2024-04-25T14:12:51.440678+0000","last_peered":"2024-04-25T14:12:51.440678+0000","last_clean":"2024-04-25T14:12:51.440678+0000","last_became_active":"2024-04-25T14:11:29.560519+0000","last_became_peered":"2024-04-25T14:11:29.560519+0000","last_unstale":"2024-04-25T14:12:51.440678+0000","last_undegraded":"2024-04-25T14:12:51.440678+0000","last_fullsized":"2024-04-25T14:12:51.440678+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:53:47.677923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"78'12","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.435167+0000","last_change":"2024-04-25T14:11:29.574372+0000","last_active":"2024-04-25T14:12:51.435167+0000","last_peered":"2024-04-25T14:12:51.435167+0000","last_clean":"2024-04-25T14:12:51.435167+0000","last_became_active":"2024-04-25T14:11:29.573995+0000","last_became_peered":"2024-04-25T14:11:29.573995+0000","last_unstale":"2024-04-25T14:12:51.435167+0000","last_undegraded":"2024-04-25T14:12:51.435167+0000","last_fullsized":"2024-04-25T14:12:51.435167+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:12.381512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"78'8","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.432226+0000","last_change":"2024-04-25T14:11:29.578324+0000","last_active":"2024-04-25T14:12:51.432226+0000","last_peered":"2024-04-25T14:12:51.432226+0000","last_clean":"2024-04-25T14:12:51.432226+0000","last_became_active":"2024-04-25T14:11:29.577917+0000","last_became_peered":"2024-04-25T14:11:29.577917+0000","last_unstale":"2024-04-25T14:12:51.432226+0000","last_undegraded":"2024-04-25T14:12:51.432226+0000","last_fullsized":"2024-04-25T14:12:51.432226+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:20:27.070720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"78'7","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.432298+0000","last_change":"2024-04-25T14:11:29.578879+0000","last_active":"2024-04-25T14:12:51.432298+0000","last_peered":"2024-04-25T14:12:51.432298+0000","last_clean":"2024-04-25T14:12:51.432298+0000","last_became_active":"2024-04-25T14:11:29.578551+0000","last_became_peered":"2024-04-25T14:11:29.578551+0000","last_unstale":"2024-04-25T14:12:51.432298+0000","last_undegraded":"2024-04-25T14:12:51.432298+0000","last_fullsized":"2024-04-25T14:12:51.432298+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:41.083794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"78'4","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.437936+0000","last_change":"2024-04-25T14:11:29.574054+0000","last_active":"2024-04-25T14:12:51.437936+0000","last_peered":"2024-04-25T14:12:51.437936+0000","last_clean":"2024-04-25T14:12:51.437936+0000","last_became_active":"2024-04-25T14:11:29.573700+0000","last_became_peered":"2024-04-25T14:11:29.573700+0000","last_unstale":"2024-04-25T14:12:51.437936+0000","last_undegraded":"2024-04-25T14:12:51.437936+0000","last_fullsized":"2024-04-25T14:12:51.437936+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:23.304580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"78'6","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.440195+0000","last_change":"2024-04-25T14:11:29.553768+0000","last_active":"2024-04-25T14:12:51.440195+0000","last_peered":"2024-04-25T14:12:51.440195+0000","last_clean":"2024-04-25T14:12:51.440195+0000","last_became_active":"2024-04-25T14:11:29.553376+0000","last_became_peered":"2024-04-25T14:11:29.553376+0000","last_unstale":"2024-04-25T14:12:51.440195+0000","last_undegraded":"2024-04-25T14:12:51.440195+0000","last_fullsized":"2024-04-25T14:12:51.440195+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:46.688368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"78'15","reported_seq":32,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.440586+0000","last_change":"2024-04-25T14:11:29.563122+0000","last_active":"2024-04-25T14:12:51.440586+0000","last_peered":"2024-04-25T14:12:51.440586+0000","last_clean":"2024-04-25T14:12:51.440586+0000","last_became_active":"2024-04-25T14:11:29.562717+0000","last_became_peered":"2024-04-25T14:11:29.562717+0000","last_unstale":"2024-04-25T14:12:51.440586+0000","last_undegraded":"2024-04-25T14:12:51.440586+0000","last_fullsized":"2024-04-25T14:12:51.440586+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:37.585352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"78'2","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.436426+0000","last_change":"2024-04-25T14:11:29.576487+0000","last_active":"2024-04-25T14:12:51.436426+0000","last_peered":"2024-04-25T14:12:51.436426+0000","last_clean":"2024-04-25T14:12:51.436426+0000","last_became_active":"2024-04-25T14:11:29.575070+0000","last_became_peered":"2024-04-25T14:11:29.575070+0000","last_unstale":"2024-04-25T14:12:51.436426+0000","last_undegraded":"2024-04-25T14:12:51.436426+0000","last_fullsized":"2024-04-25T14:12:51.436426+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:13.803505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"78'6","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.427743+0000","last_change":"2024-04-25T14:11:29.568733+0000","last_active":"2024-04-25T14:12:51.427743+0000","last_peered":"2024-04-25T14:12:51.427743+0000","last_clean":"2024-04-25T14:12:51.427743+0000","last_became_active":"2024-04-25T14:11:29.567665+0000","last_became_peered":"2024-04-25T14:11:29.567665+0000","last_unstale":"2024-04-25T14:12:51.427743+0000","last_undegraded":"2024-04-25T14:12:51.427743+0000","last_fullsized":"2024-04-25T14:12:51.427743+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:57.956199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"78'8","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.433324+0000","last_change":"2024-04-25T14:11:29.574648+0000","last_active":"2024-04-25T14:12:51.433324+0000","last_peered":"2024-04-25T14:12:51.433324+0000","last_clean":"2024-04-25T14:12:51.433324+0000","last_became_active":"2024-04-25T14:11:29.574247+0000","last_became_peered":"2024-04-25T14:11:29.574247+0000","last_unstale":"2024-04-25T14:12:51.433324+0000","last_undegraded":"2024-04-25T14:12:51.433324+0000","last_fullsized":"2024-04-25T14:12:51.433324+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:57.110017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"78'10","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.458289+0000","last_change":"2024-04-25T14:11:29.573633+0000","last_active":"2024-04-25T14:12:51.458289+0000","last_peered":"2024-04-25T14:12:51.458289+0000","last_clean":"2024-04-25T14:12:51.458289+0000","last_became_active":"2024-04-25T14:11:29.573303+0000","last_became_peered":"2024-04-25T14:11:29.573303+0000","last_unstale":"2024-04-25T14:12:51.458289+0000","last_undegraded":"2024-04-25T14:12:51.458289+0000","last_fullsized":"2024-04-25T14:12:51.458289+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:11.458117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"78'11","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.460396+0000","last_change":"2024-04-25T14:11:29.569136+0000","last_active":"2024-04-25T14:12:51.460396+0000","last_peered":"2024-04-25T14:12:51.460396+0000","last_clean":"2024-04-25T14:12:51.460396+0000","last_became_active":"2024-04-25T14:11:29.568801+0000","last_became_peered":"2024-04-25T14:11:29.568801+0000","last_unstale":"2024-04-25T14:12:51.460396+0000","last_undegraded":"2024-04-25T14:12:51.460396+0000","last_fullsized":"2024-04-25T14:12:51.460396+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:30.969081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"78'8","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.442382+0000","last_change":"2024-04-25T14:11:29.566754+0000","last_active":"2024-04-25T14:12:51.442382+0000","last_peered":"2024-04-25T14:12:51.442382+0000","last_clean":"2024-04-25T14:12:51.442382+0000","last_became_active":"2024-04-25T14:11:29.566368+0000","last_became_peered":"2024-04-25T14:11:29.566368+0000","last_unstale":"2024-04-25T14:12:51.442382+0000","last_undegraded":"2024-04-25T14:12:51.442382+0000","last_fullsized":"2024-04-25T14:12:51.442382+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:37.057718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281277,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67129,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1209,"num_read_kb":679,"num_write":247,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":740,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":202003,"data_compressed":5550,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678055,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":851968,"data_stored":405341,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874284,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":1056477,"data_compressed":43412,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037809,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":868352,"data_stored":405203,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234039,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32012,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713140,"statfs":{"total":95995035648,"available":95962255360,"internally_reserved":0,"allocated":856064,"data_stored":404909,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430267,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":917504,"data_stored":999080,"data_compressed":41562,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593793,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":851968,"data_stored":404920,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":465784,"data_compressed":7643,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":872448,"data_stored":406259,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":466160,"data_compressed":7643,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149776,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":868352,"data_stored":404745,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953546,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":860160,"data_stored":405985,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790022,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":997918,"data_compressed":41562,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":61703,"data_compressed":1850,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":62079,"data_compressed":1850,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":61911,"data_compressed":1850,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:13:35.923 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:13:35.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:35 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/577196138' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:13:35.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:35 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/2484256442' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:13:35.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:35 smithi097 ceph-mon[32720]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T14:13:35.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:35 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/1462640069' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:13:35.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:35 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/577196138' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:13:35.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:35 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/2484256442' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:13:35.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:35 smithi151 ceph-mon[32990]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T14:13:35.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:35 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/1462640069' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:13:36.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.067+0000 7f77a8e1a640 1 -- 172.21.15.96:0/2625486107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77a4108900 msgr2=0x7f77a410ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:36.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.067+0000 7f77a8e1a640 1 --2- 172.21.15.96:0/2625486107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77a4108900 0x7f77a410ad20 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7794009900 tx=0x7f779402a410 comp rx=0 tx=0).stop 2024-04-25T14:13:36.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.067+0000 7f77a8e1a640 1 -- 172.21.15.96:0/2625486107 shutdown_connections 2024-04-25T14:13:36.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.067+0000 7f77a8e1a640 1 --2- 172.21.15.96:0/2625486107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77a4108900 0x7f77a410ad20 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:36.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.067+0000 7f77a8e1a640 1 --2- 172.21.15.96:0/2625486107 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77a4103c00 0x7f77a4104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:36.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.067+0000 7f77a8e1a640 1 --2- 172.21.15.96:0/2625486107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77a41029c0 0x7f77a41036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:36.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.067+0000 7f77a8e1a640 1 -- 172.21.15.96:0/2625486107 >> 172.21.15.96:0/2625486107 conn(0x7f77a40fdbd0 msgr2=0x7f77a4100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:36.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.067+0000 7f77a8e1a640 1 -- 172.21.15.96:0/2625486107 shutdown_connections 2024-04-25T14:13:36.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.067+0000 7f77a8e1a640 1 -- 172.21.15.96:0/2625486107 wait complete. 2024-04-25T14:13:36.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.068+0000 7f77a8e1a640 1 Processor -- start 2024-04-25T14:13:36.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.068+0000 7f77a8e1a640 1 -- start start 2024-04-25T14:13:36.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.068+0000 7f77a8e1a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77a41029c0 0x7f77a419a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:36.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.068+0000 7f77a8e1a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77a4103c00 0x7f77a419a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:36.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.068+0000 7f77a8e1a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77a4108900 0x7f77a41a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:36.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.068+0000 7f77a8e1a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f77a410f5e0 con 0x7f77a4108900 2024-04-25T14:13:36.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.068+0000 7f77a8e1a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f77a410f460 con 0x7f77a41029c0 2024-04-25T14:13:36.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.068+0000 7f77a8e1a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f77a410f760 con 0x7f77a4103c00 2024-04-25T14:13:36.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.068+0000 7f77a2d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77a4108900 0x7f77a41a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:36.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.068+0000 7f77a2d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77a4108900 0x7f77a41a1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40568/0 (socket says 172.21.15.96:40568) 2024-04-25T14:13:36.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.068+0000 7f77a2d9d640 1 -- 172.21.15.96:0/3765560834 learned_addr learned my addr 172.21.15.96:0/3765560834 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:36.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.068+0000 7f77a259c640 1 --2- 172.21.15.96:0/3765560834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77a41029c0 0x7f77a419a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:36.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.068+0000 7f77a1d9b640 1 --2- 172.21.15.96:0/3765560834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77a4103c00 0x7f77a419a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:36.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.069+0000 7f77a2d9d640 1 -- 172.21.15.96:0/3765560834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77a4103c00 msgr2=0x7f77a419a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:36.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.069+0000 7f77a2d9d640 1 --2- 172.21.15.96:0/3765560834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77a4103c00 0x7f77a419a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:36.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.069+0000 7f77a2d9d640 1 -- 172.21.15.96:0/3765560834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77a41029c0 msgr2=0x7f77a419a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:36.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.069+0000 7f77a2d9d640 1 --2- 172.21.15.96:0/3765560834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77a41029c0 0x7f77a419a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:36.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.069+0000 7f77a2d9d640 1 -- 172.21.15.96:0/3765560834 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a41a1c20 con 0x7f77a4108900 2024-04-25T14:13:36.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.069+0000 7f77a2d9d640 1 --2- 172.21.15.96:0/3765560834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77a4108900 0x7f77a41a1660 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7794009230 tx=0x7f77940042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:36.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.069+0000 7f77937fe640 1 -- 172.21.15.96:0/3765560834 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779403f030 con 0x7f77a4108900 2024-04-25T14:13:36.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.069+0000 7f77937fe640 1 -- 172.21.15.96:0/3765560834 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7794031910 con 0x7f77a4108900 2024-04-25T14:13:36.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.069+0000 7f77937fe640 1 -- 172.21.15.96:0/3765560834 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7794035070 con 0x7f77a4108900 2024-04-25T14:13:36.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.069+0000 7f77a8e1a640 1 -- 172.21.15.96:0/3765560834 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77a41a1eb0 con 0x7f77a4108900 2024-04-25T14:13:36.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.069+0000 7f77a8e1a640 1 -- 172.21.15.96:0/3765560834 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77a41a22c0 con 0x7f77a4108900 2024-04-25T14:13:36.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.070+0000 7f77a8e1a640 1 -- 172.21.15.96:0/3765560834 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77a4107710 con 0x7f77a4108900 2024-04-25T14:13:36.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.071+0000 7f77937fe640 1 -- 172.21.15.96:0/3765560834 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f7794042030 con 0x7f77a4108900 2024-04-25T14:13:36.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.072+0000 7f77937fe640 1 --2- 172.21.15.96:0/3765560834 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f778c0afaf0 0x7f778c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:36.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.072+0000 7f77a259c640 1 --2- 172.21.15.96:0/3765560834 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f778c0afaf0 0x7f778c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:36.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.072+0000 7f77937fe640 1 -- 172.21.15.96:0/3765560834 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f77940ff740 con 0x7f77a4108900 2024-04-25T14:13:36.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.072+0000 7f77a259c640 1 --2- 172.21.15.96:0/3765560834 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f778c0afaf0 0x7f778c0b2000 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f77a41035c0 tx=0x7f779c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:36.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.075+0000 7f77937fe640 1 -- 172.21.15.96:0/3765560834 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f77940c8300 con 0x7f77a4108900 2024-04-25T14:13:36.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.251+0000 7f77a8e1a640 1 -- 172.21.15.96:0/3765560834 --> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f77a4062fe0 con 0x7f778c0afaf0 2024-04-25T14:13:36.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.258+0000 7f77937fe640 1 -- 172.21.15.96:0/3765560834 <== mgr.14152 v2:172.21.15.96:6800/536332425 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259165 (secure 0 0 0) 0x7f77a4062fe0 con 0x7f778c0afaf0 2024-04-25T14:13:36.263 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:13:36.264 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-04-25T14:13:36.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.261+0000 7f77a8e1a640 1 -- 172.21.15.96:0/3765560834 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f778c0afaf0 msgr2=0x7f778c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:36.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.261+0000 7f77a8e1a640 1 --2- 172.21.15.96:0/3765560834 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f778c0afaf0 0x7f778c0b2000 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f77a41035c0 tx=0x7f779c005f90 comp rx=0 tx=0).stop 2024-04-25T14:13:36.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.261+0000 7f77a8e1a640 1 -- 172.21.15.96:0/3765560834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77a4108900 msgr2=0x7f77a41a1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:36.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.261+0000 7f77a8e1a640 1 --2- 172.21.15.96:0/3765560834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77a4108900 0x7f77a41a1660 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7794009230 tx=0x7f77940042f0 comp rx=0 tx=0).stop 2024-04-25T14:13:36.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.261+0000 7f77a8e1a640 1 -- 172.21.15.96:0/3765560834 shutdown_connections 2024-04-25T14:13:36.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.261+0000 7f77a8e1a640 1 --2- 172.21.15.96:0/3765560834 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f778c0afaf0 0x7f778c0b2000 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:36.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.261+0000 7f77a8e1a640 1 --2- 172.21.15.96:0/3765560834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77a4108900 0x7f77a41a1660 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:36.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.261+0000 7f77a8e1a640 1 --2- 172.21.15.96:0/3765560834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77a4103c00 0x7f77a419a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:36.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.261+0000 7f77a8e1a640 1 --2- 172.21.15.96:0/3765560834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77a41029c0 0x7f77a419a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:36.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.261+0000 7f77a8e1a640 1 -- 172.21.15.96:0/3765560834 >> 172.21.15.96:0/3765560834 conn(0x7f77a40fdbd0 msgr2=0x7f77a41048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:36.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.261+0000 7f77a8e1a640 1 -- 172.21.15.96:0/3765560834 shutdown_connections 2024-04-25T14:13:36.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.261+0000 7f77a8e1a640 1 -- 172.21.15.96:0/3765560834 wait complete. 2024-04-25T14:13:36.272 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2024-04-25T14:13:35.432428+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657731,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1285,"num_read_kb":743,"num_write":372,"num_write_kb":2291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":2952580,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":808,"ondisk_log_size":808,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359604,"kb_used_data":10452,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124582220,"statfs":{"total":1151940427776,"available":1151572193280,"internally_reserved":0,"allocated":10702848,"data_stored":6822781,"data_compressed":182373,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":105,"num_read_kb":55,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003170"},"pg_stats":[{"pgid":"3.3f","version":"78'8","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.448766+0000","last_change":"2024-04-25T14:11:29.569615+0000","last_active":"2024-04-25T14:12:51.448766+0000","last_peered":"2024-04-25T14:12:51.448766+0000","last_clean":"2024-04-25T14:12:51.448766+0000","last_became_active":"2024-04-25T14:11:29.569237+0000","last_became_peered":"2024-04-25T14:11:29.569237+0000","last_unstale":"2024-04-25T14:12:51.448766+0000","last_undegraded":"2024-04-25T14:12:51.448766+0000","last_fullsized":"2024-04-25T14:12:51.448766+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:24.367533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"78'4","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.440396+0000","last_change":"2024-04-25T14:11:29.571820+0000","last_active":"2024-04-25T14:12:51.440396+0000","last_peered":"2024-04-25T14:12:51.440396+0000","last_clean":"2024-04-25T14:12:51.440396+0000","last_became_active":"2024-04-25T14:11:29.571456+0000","last_became_peered":"2024-04-25T14:11:29.571456+0000","last_unstale":"2024-04-25T14:12:51.440396+0000","last_undegraded":"2024-04-25T14:12:51.440396+0000","last_fullsized":"2024-04-25T14:12:51.440396+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:33.442301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"78'7","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.456139+0000","last_change":"2024-04-25T14:11:29.579302+0000","last_active":"2024-04-25T14:12:51.456139+0000","last_peered":"2024-04-25T14:12:51.456139+0000","last_clean":"2024-04-25T14:12:51.456139+0000","last_became_active":"2024-04-25T14:11:29.577734+0000","last_became_peered":"2024-04-25T14:11:29.577734+0000","last_unstale":"2024-04-25T14:12:51.456139+0000","last_undegraded":"2024-04-25T14:12:51.456139+0000","last_fullsized":"2024-04-25T14:12:51.456139+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:59.147554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"78'8","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.442356+0000","last_change":"2024-04-25T14:11:29.574942+0000","last_active":"2024-04-25T14:12:51.442356+0000","last_peered":"2024-04-25T14:12:51.442356+0000","last_clean":"2024-04-25T14:12:51.442356+0000","last_became_active":"2024-04-25T14:11:29.573774+0000","last_became_peered":"2024-04-25T14:11:29.573774+0000","last_unstale":"2024-04-25T14:12:51.442356+0000","last_undegraded":"2024-04-25T14:12:51.442356+0000","last_fullsized":"2024-04-25T14:12:51.442356+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:10.626265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"78'4","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.426798+0000","last_change":"2024-04-25T14:11:29.557199+0000","last_active":"2024-04-25T14:12:51.426798+0000","last_peered":"2024-04-25T14:12:51.426798+0000","last_clean":"2024-04-25T14:12:51.426798+0000","last_became_active":"2024-04-25T14:11:29.556836+0000","last_became_peered":"2024-04-25T14:11:29.556836+0000","last_unstale":"2024-04-25T14:12:51.426798+0000","last_undegraded":"2024-04-25T14:12:51.426798+0000","last_fullsized":"2024-04-25T14:12:51.426798+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:04:45.882935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"78'6","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.434655+0000","last_change":"2024-04-25T14:11:29.557988+0000","last_active":"2024-04-25T14:12:51.434655+0000","last_peered":"2024-04-25T14:12:51.434655+0000","last_clean":"2024-04-25T14:12:51.434655+0000","last_became_active":"2024-04-25T14:11:29.557622+0000","last_became_peered":"2024-04-25T14:11:29.557622+0000","last_unstale":"2024-04-25T14:12:51.434655+0000","last_undegraded":"2024-04-25T14:12:51.434655+0000","last_fullsized":"2024-04-25T14:12:51.434655+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:28:37.578368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"78'5","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.452293+0000","last_change":"2024-04-25T14:11:29.565049+0000","last_active":"2024-04-25T14:12:51.452293+0000","last_peered":"2024-04-25T14:12:51.452293+0000","last_clean":"2024-04-25T14:12:51.452293+0000","last_became_active":"2024-04-25T14:11:29.564688+0000","last_became_peered":"2024-04-25T14:11:29.564688+0000","last_unstale":"2024-04-25T14:12:51.452293+0000","last_undegraded":"2024-04-25T14:12:51.452293+0000","last_fullsized":"2024-04-25T14:12:51.452293+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:48.128778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"78'9","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.432114+0000","last_change":"2024-04-25T14:11:29.565879+0000","last_active":"2024-04-25T14:12:51.432114+0000","last_peered":"2024-04-25T14:12:51.432114+0000","last_clean":"2024-04-25T14:12:51.432114+0000","last_became_active":"2024-04-25T14:11:29.565525+0000","last_became_peered":"2024-04-25T14:11:29.565525+0000","last_unstale":"2024-04-25T14:12:51.432114+0000","last_undegraded":"2024-04-25T14:12:51.432114+0000","last_fullsized":"2024-04-25T14:12:51.432114+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:06.956334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"78'11","reported_seq":34,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.450891+0000","last_change":"2024-04-25T14:12:22.517776+0000","last_active":"2024-04-25T14:12:51.450891+0000","last_peered":"2024-04-25T14:12:51.450891+0000","last_clean":"2024-04-25T14:12:51.450891+0000","last_became_active":"2024-04-25T14:12:22.517232+0000","last_became_peered":"2024-04-25T14:12:22.517232+0000","last_unstale":"2024-04-25T14:12:51.450891+0000","last_undegraded":"2024-04-25T14:12:51.450891+0000","last_fullsized":"2024-04-25T14:12:51.450891+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:51.912445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,11],"acting":[1,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"78'16","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.438962+0000","last_change":"2024-04-25T14:11:29.561841+0000","last_active":"2024-04-25T14:12:51.438962+0000","last_peered":"2024-04-25T14:12:51.438962+0000","last_clean":"2024-04-25T14:12:51.438962+0000","last_became_active":"2024-04-25T14:11:29.560145+0000","last_became_peered":"2024-04-25T14:11:29.560145+0000","last_unstale":"2024-04-25T14:12:51.438962+0000","last_undegraded":"2024-04-25T14:12:51.438962+0000","last_fullsized":"2024-04-25T14:12:51.438962+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:11.654692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"78'6","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.459530+0000","last_change":"2024-04-25T14:11:29.570954+0000","last_active":"2024-04-25T14:12:51.459530+0000","last_peered":"2024-04-25T14:12:51.459530+0000","last_clean":"2024-04-25T14:12:51.459530+0000","last_became_active":"2024-04-25T14:11:29.570368+0000","last_became_peered":"2024-04-25T14:11:29.570368+0000","last_unstale":"2024-04-25T14:12:51.459530+0000","last_undegraded":"2024-04-25T14:12:51.459530+0000","last_fullsized":"2024-04-25T14:12:51.459530+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:50.153695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"78'7","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.443244+0000","last_change":"2024-04-25T14:11:29.571911+0000","last_active":"2024-04-25T14:12:51.443244+0000","last_peered":"2024-04-25T14:12:51.443244+0000","last_clean":"2024-04-25T14:12:51.443244+0000","last_became_active":"2024-04-25T14:11:29.568557+0000","last_became_peered":"2024-04-25T14:11:29.568557+0000","last_unstale":"2024-04-25T14:12:51.443244+0000","last_undegraded":"2024-04-25T14:12:51.443244+0000","last_fullsized":"2024-04-25T14:12:51.443244+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:24.221527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"78'10","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.426972+0000","last_change":"2024-04-25T14:11:29.566826+0000","last_active":"2024-04-25T14:12:51.426972+0000","last_peered":"2024-04-25T14:12:51.426972+0000","last_clean":"2024-04-25T14:12:51.426972+0000","last_became_active":"2024-04-25T14:11:29.565825+0000","last_became_peered":"2024-04-25T14:11:29.565825+0000","last_unstale":"2024-04-25T14:12:51.426972+0000","last_undegraded":"2024-04-25T14:12:51.426972+0000","last_fullsized":"2024-04-25T14:12:51.426972+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:46.711809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"78'7","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.424340+0000","last_change":"2024-04-25T14:11:29.577359+0000","last_active":"2024-04-25T14:12:51.424340+0000","last_peered":"2024-04-25T14:12:51.424340+0000","last_clean":"2024-04-25T14:12:51.424340+0000","last_became_active":"2024-04-25T14:11:29.574643+0000","last_became_peered":"2024-04-25T14:11:29.574643+0000","last_unstale":"2024-04-25T14:12:51.424340+0000","last_undegraded":"2024-04-25T14:12:51.424340+0000","last_fullsized":"2024-04-25T14:12:51.424340+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:07.582154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"78'7","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.432124+0000","last_change":"2024-04-25T14:11:29.583606+0000","last_active":"2024-04-25T14:12:51.432124+0000","last_peered":"2024-04-25T14:12:51.432124+0000","last_clean":"2024-04-25T14:12:51.432124+0000","last_became_active":"2024-04-25T14:11:29.582091+0000","last_became_peered":"2024-04-25T14:11:29.582091+0000","last_unstale":"2024-04-25T14:12:51.432124+0000","last_undegraded":"2024-04-25T14:12:51.432124+0000","last_fullsized":"2024-04-25T14:12:51.432124+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:58.711591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"78'9","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.458160+0000","last_change":"2024-04-25T14:11:29.571223+0000","last_active":"2024-04-25T14:12:51.458160+0000","last_peered":"2024-04-25T14:12:51.458160+0000","last_clean":"2024-04-25T14:12:51.458160+0000","last_became_active":"2024-04-25T14:11:29.570856+0000","last_became_peered":"2024-04-25T14:11:29.570856+0000","last_unstale":"2024-04-25T14:12:51.458160+0000","last_undegraded":"2024-04-25T14:12:51.458160+0000","last_fullsized":"2024-04-25T14:12:51.458160+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:53.058088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"78'8","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.449162+0000","last_change":"2024-04-25T14:11:29.574467+0000","last_active":"2024-04-25T14:12:51.449162+0000","last_peered":"2024-04-25T14:12:51.449162+0000","last_clean":"2024-04-25T14:12:51.449162+0000","last_became_active":"2024-04-25T14:11:29.573878+0000","last_became_peered":"2024-04-25T14:11:29.573878+0000","last_unstale":"2024-04-25T14:12:51.449162+0000","last_undegraded":"2024-04-25T14:12:51.449162+0000","last_fullsized":"2024-04-25T14:12:51.449162+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:22:02.087606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"78'10","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.454589+0000","last_change":"2024-04-25T14:11:29.561612+0000","last_active":"2024-04-25T14:12:51.454589+0000","last_peered":"2024-04-25T14:12:51.454589+0000","last_clean":"2024-04-25T14:12:51.454589+0000","last_became_active":"2024-04-25T14:11:29.561211+0000","last_became_peered":"2024-04-25T14:11:29.561211+0000","last_unstale":"2024-04-25T14:12:51.454589+0000","last_undegraded":"2024-04-25T14:12:51.454589+0000","last_fullsized":"2024-04-25T14:12:51.454589+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:29.195093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"78'7","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.430634+0000","last_change":"2024-04-25T14:11:29.575231+0000","last_active":"2024-04-25T14:12:51.430634+0000","last_peered":"2024-04-25T14:12:51.430634+0000","last_clean":"2024-04-25T14:12:51.430634+0000","last_became_active":"2024-04-25T14:11:29.573928+0000","last_became_peered":"2024-04-25T14:11:29.573928+0000","last_unstale":"2024-04-25T14:12:51.430634+0000","last_undegraded":"2024-04-25T14:12:51.430634+0000","last_fullsized":"2024-04-25T14:12:51.430634+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:10.025584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"76'13","reported_seq":172,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.613466+0000","last_change":"2024-04-25T14:12:22.527280+0000","last_active":"2024-04-25T14:13:33.613466+0000","last_peered":"2024-04-25T14:13:33.613466+0000","last_clean":"2024-04-25T14:13:33.613466+0000","last_became_active":"2024-04-25T14:12:22.515420+0000","last_became_peered":"2024-04-25T14:12:22.515420+0000","last_unstale":"2024-04-25T14:13:33.613466+0000","last_undegraded":"2024-04-25T14:13:33.613466+0000","last_fullsized":"2024-04-25T14:13:33.613466+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:53.882212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"78'5","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.427842+0000","last_change":"2024-04-25T14:11:29.576855+0000","last_active":"2024-04-25T14:12:51.427842+0000","last_peered":"2024-04-25T14:12:51.427842+0000","last_clean":"2024-04-25T14:12:51.427842+0000","last_became_active":"2024-04-25T14:11:29.574566+0000","last_became_peered":"2024-04-25T14:11:29.574566+0000","last_unstale":"2024-04-25T14:12:51.427842+0000","last_undegraded":"2024-04-25T14:12:51.427842+0000","last_fullsized":"2024-04-25T14:12:51.427842+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:31.325583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"76'2","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514493+0000","last_change":"2024-04-25T14:11:28.552763+0000","last_active":"2024-04-25T14:12:22.514493+0000","last_peered":"2024-04-25T14:12:22.514493+0000","last_clean":"2024-04-25T14:12:22.514493+0000","last_became_active":"2024-04-25T14:11:28.552403+0000","last_became_peered":"2024-04-25T14:11:28.552403+0000","last_unstale":"2024-04-25T14:12:22.514493+0000","last_undegraded":"2024-04-25T14:12:22.514493+0000","last_fullsized":"2024-04-25T14:12:22.514493+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:31.473779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"78'6","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.456271+0000","last_change":"2024-04-25T14:11:29.571019+0000","last_active":"2024-04-25T14:12:51.456271+0000","last_peered":"2024-04-25T14:12:51.456271+0000","last_clean":"2024-04-25T14:12:51.456271+0000","last_became_active":"2024-04-25T14:11:29.570633+0000","last_became_peered":"2024-04-25T14:11:29.570633+0000","last_unstale":"2024-04-25T14:12:51.456271+0000","last_undegraded":"2024-04-25T14:12:51.456271+0000","last_fullsized":"2024-04-25T14:12:51.456271+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:38:32.664318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"76'12","reported_seq":32,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514608+0000","last_change":"2024-04-25T14:11:28.545189+0000","last_active":"2024-04-25T14:12:22.514608+0000","last_peered":"2024-04-25T14:12:22.514608+0000","last_clean":"2024-04-25T14:12:22.514608+0000","last_became_active":"2024-04-25T14:11:28.544857+0000","last_became_peered":"2024-04-25T14:11:28.544857+0000","last_unstale":"2024-04-25T14:12:22.514608+0000","last_undegraded":"2024-04-25T14:12:22.514608+0000","last_fullsized":"2024-04-25T14:12:22.514608+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:58.464226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"78'9","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.455643+0000","last_change":"2024-04-25T14:11:29.576696+0000","last_active":"2024-04-25T14:12:51.455643+0000","last_peered":"2024-04-25T14:12:51.455643+0000","last_clean":"2024-04-25T14:12:51.455643+0000","last_became_active":"2024-04-25T14:11:29.576337+0000","last_became_peered":"2024-04-25T14:11:29.576337+0000","last_unstale":"2024-04-25T14:12:51.455643+0000","last_undegraded":"2024-04-25T14:12:51.455643+0000","last_fullsized":"2024-04-25T14:12:51.455643+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:31:30.410434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"76'5","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514457+0000","last_change":"2024-04-25T14:11:28.542466+0000","last_active":"2024-04-25T14:12:22.514457+0000","last_peered":"2024-04-25T14:12:22.514457+0000","last_clean":"2024-04-25T14:12:22.514457+0000","last_became_active":"2024-04-25T14:11:28.542092+0000","last_became_peered":"2024-04-25T14:11:28.542092+0000","last_unstale":"2024-04-25T14:12:22.514457+0000","last_undegraded":"2024-04-25T14:12:22.514457+0000","last_fullsized":"2024-04-25T14:12:22.514457+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:33.810111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"78'15","reported_seq":32,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.441623+0000","last_change":"2024-04-25T14:11:29.567842+0000","last_active":"2024-04-25T14:12:51.441623+0000","last_peered":"2024-04-25T14:12:51.441623+0000","last_clean":"2024-04-25T14:12:51.441623+0000","last_became_active":"2024-04-25T14:11:29.566269+0000","last_became_peered":"2024-04-25T14:11:29.566269+0000","last_unstale":"2024-04-25T14:12:51.441623+0000","last_undegraded":"2024-04-25T14:12:51.441623+0000","last_fullsized":"2024-04-25T14:12:51.441623+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:31.009318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"78'7","reported_seq":149,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:13:29.686280+0000","last_change":"2024-04-25T14:11:28.546215+0000","last_active":"2024-04-25T14:13:29.686280+0000","last_peered":"2024-04-25T14:13:29.686280+0000","last_clean":"2024-04-25T14:13:29.686280+0000","last_became_active":"2024-04-25T14:11:28.545828+0000","last_became_peered":"2024-04-25T14:11:28.545828+0000","last_unstale":"2024-04-25T14:13:29.686280+0000","last_undegraded":"2024-04-25T14:13:29.686280+0000","last_fullsized":"2024-04-25T14:13:29.686280+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:27.152582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":123,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"78'7","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.422934+0000","last_change":"2024-04-25T14:11:29.559566+0000","last_active":"2024-04-25T14:12:51.422934+0000","last_peered":"2024-04-25T14:12:51.422934+0000","last_clean":"2024-04-25T14:12:51.422934+0000","last_became_active":"2024-04-25T14:11:29.558560+0000","last_became_peered":"2024-04-25T14:11:29.558560+0000","last_unstale":"2024-04-25T14:12:51.422934+0000","last_undegraded":"2024-04-25T14:12:51.422934+0000","last_fullsized":"2024-04-25T14:12:51.422934+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:23.677099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"76'7","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.515784+0000","last_change":"2024-04-25T14:11:28.544020+0000","last_active":"2024-04-25T14:12:22.515784+0000","last_peered":"2024-04-25T14:12:22.515784+0000","last_clean":"2024-04-25T14:12:22.515784+0000","last_became_active":"2024-04-25T14:11:28.543598+0000","last_became_peered":"2024-04-25T14:11:28.543598+0000","last_unstale":"2024-04-25T14:12:22.515784+0000","last_undegraded":"2024-04-25T14:12:22.515784+0000","last_fullsized":"2024-04-25T14:12:22.515784+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:49:06.543876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"78'8","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.438175+0000","last_change":"2024-04-25T14:11:29.576122+0000","last_active":"2024-04-25T14:12:51.438175+0000","last_peered":"2024-04-25T14:12:51.438175+0000","last_clean":"2024-04-25T14:12:51.438175+0000","last_became_active":"2024-04-25T14:11:29.573668+0000","last_became_peered":"2024-04-25T14:11:29.573668+0000","last_unstale":"2024-04-25T14:12:51.438175+0000","last_undegraded":"2024-04-25T14:12:51.438175+0000","last_fullsized":"2024-04-25T14:12:51.438175+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:06.998876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"76'3","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.505016+0000","last_change":"2024-04-25T14:11:28.541554+0000","last_active":"2024-04-25T14:12:22.505016+0000","last_peered":"2024-04-25T14:12:22.505016+0000","last_clean":"2024-04-25T14:12:22.505016+0000","last_became_active":"2024-04-25T14:11:28.541062+0000","last_became_peered":"2024-04-25T14:11:28.541062+0000","last_unstale":"2024-04-25T14:12:22.505016+0000","last_undegraded":"2024-04-25T14:12:22.505016+0000","last_fullsized":"2024-04-25T14:12:22.505016+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:16.719820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"78'8","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.433513+0000","last_change":"2024-04-25T14:11:29.564550+0000","last_active":"2024-04-25T14:12:51.433513+0000","last_peered":"2024-04-25T14:12:51.433513+0000","last_clean":"2024-04-25T14:12:51.433513+0000","last_became_active":"2024-04-25T14:11:29.564193+0000","last_became_peered":"2024-04-25T14:11:29.564193+0000","last_unstale":"2024-04-25T14:12:51.433513+0000","last_undegraded":"2024-04-25T14:12:51.433513+0000","last_fullsized":"2024-04-25T14:12:51.433513+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:59.357327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"76'3","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.508406+0000","last_change":"2024-04-25T14:11:28.540504+0000","last_active":"2024-04-25T14:12:22.508406+0000","last_peered":"2024-04-25T14:12:22.508406+0000","last_clean":"2024-04-25T14:12:22.508406+0000","last_became_active":"2024-04-25T14:11:28.540150+0000","last_became_peered":"2024-04-25T14:11:28.540150+0000","last_unstale":"2024-04-25T14:12:22.508406+0000","last_undegraded":"2024-04-25T14:12:22.508406+0000","last_fullsized":"2024-04-25T14:12:22.508406+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:35.331489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"78'11","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.430603+0000","last_change":"2024-04-25T14:11:29.561226+0000","last_active":"2024-04-25T14:12:51.430603+0000","last_peered":"2024-04-25T14:12:51.430603+0000","last_clean":"2024-04-25T14:12:51.430603+0000","last_became_active":"2024-04-25T14:11:29.560847+0000","last_became_peered":"2024-04-25T14:11:29.560847+0000","last_unstale":"2024-04-25T14:12:51.430603+0000","last_undegraded":"2024-04-25T14:12:51.430603+0000","last_fullsized":"2024-04-25T14:12:51.430603+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:05.299388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'5","reported_seq":259,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:13:30.687104+0000","last_change":"2024-04-25T14:11:28.548580+0000","last_active":"2024-04-25T14:13:30.687104+0000","last_peered":"2024-04-25T14:13:30.687104+0000","last_clean":"2024-04-25T14:13:30.687104+0000","last_became_active":"2024-04-25T14:11:28.548250+0000","last_became_peered":"2024-04-25T14:11:28.548250+0000","last_unstale":"2024-04-25T14:13:30.687104+0000","last_undegraded":"2024-04-25T14:13:30.687104+0000","last_fullsized":"2024-04-25T14:13:30.687104+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:01.166285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":235,"num_read_kb":235,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"78'10","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.455188+0000","last_change":"2024-04-25T14:11:29.575983+0000","last_active":"2024-04-25T14:12:51.455188+0000","last_peered":"2024-04-25T14:12:51.455188+0000","last_clean":"2024-04-25T14:12:51.455188+0000","last_became_active":"2024-04-25T14:11:29.575147+0000","last_became_peered":"2024-04-25T14:11:29.575147+0000","last_unstale":"2024-04-25T14:12:51.455188+0000","last_undegraded":"2024-04-25T14:12:51.455188+0000","last_fullsized":"2024-04-25T14:12:51.455188+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:40:45.366736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"76'4","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.530419+0000","last_change":"2024-04-25T14:12:22.530419+0000","last_active":"2024-04-25T14:12:22.530419+0000","last_peered":"2024-04-25T14:12:22.530419+0000","last_clean":"2024-04-25T14:12:22.530419+0000","last_became_active":"2024-04-25T14:12:22.514999+0000","last_became_peered":"2024-04-25T14:12:22.514999+0000","last_unstale":"2024-04-25T14:12:22.530419+0000","last_undegraded":"2024-04-25T14:12:22.530419+0000","last_fullsized":"2024-04-25T14:12:22.530419+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:02.075270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"78'9","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.424732+0000","last_change":"2024-04-25T14:11:29.581424+0000","last_active":"2024-04-25T14:12:51.424732+0000","last_peered":"2024-04-25T14:12:51.424732+0000","last_clean":"2024-04-25T14:12:51.424732+0000","last_became_active":"2024-04-25T14:11:29.581086+0000","last_became_peered":"2024-04-25T14:11:29.581086+0000","last_unstale":"2024-04-25T14:12:51.424732+0000","last_undegraded":"2024-04-25T14:12:51.424732+0000","last_fullsized":"2024-04-25T14:12:51.424732+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:30.468728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"76'5","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.508977+0000","last_change":"2024-04-25T14:11:28.540771+0000","last_active":"2024-04-25T14:12:22.508977+0000","last_peered":"2024-04-25T14:12:22.508977+0000","last_clean":"2024-04-25T14:12:22.508977+0000","last_became_active":"2024-04-25T14:11:28.540442+0000","last_became_peered":"2024-04-25T14:11:28.540442+0000","last_unstale":"2024-04-25T14:12:22.508977+0000","last_undegraded":"2024-04-25T14:12:22.508977+0000","last_fullsized":"2024-04-25T14:12:22.508977+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:52.479423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"78'11","reported_seq":34,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.458541+0000","last_change":"2024-04-25T14:12:22.517721+0000","last_active":"2024-04-25T14:12:51.458541+0000","last_peered":"2024-04-25T14:12:51.458541+0000","last_clean":"2024-04-25T14:12:51.458541+0000","last_became_active":"2024-04-25T14:12:22.517232+0000","last_became_peered":"2024-04-25T14:12:22.517232+0000","last_unstale":"2024-04-25T14:12:51.458541+0000","last_undegraded":"2024-04-25T14:12:51.458541+0000","last_fullsized":"2024-04-25T14:12:51.458541+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:46.150626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"78'32","reported_seq":297,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:13:30.686843+0000","last_change":"2024-04-25T14:11:28.543093+0000","last_active":"2024-04-25T14:13:30.686843+0000","last_peered":"2024-04-25T14:13:30.686843+0000","last_clean":"2024-04-25T14:13:30.686843+0000","last_became_active":"2024-04-25T14:11:28.542407+0000","last_became_peered":"2024-04-25T14:11:28.542407+0000","last_unstale":"2024-04-25T14:13:30.686843+0000","last_undegraded":"2024-04-25T14:13:30.686843+0000","last_fullsized":"2024-04-25T14:13:30.686843+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:43.767375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59643,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":246,"num_read_kb":181,"num_write":34,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"78'6","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.455852+0000","last_change":"2024-04-25T14:11:29.571896+0000","last_active":"2024-04-25T14:12:51.455852+0000","last_peered":"2024-04-25T14:12:51.455852+0000","last_clean":"2024-04-25T14:12:51.455852+0000","last_became_active":"2024-04-25T14:11:29.571537+0000","last_became_peered":"2024-04-25T14:11:29.571537+0000","last_unstale":"2024-04-25T14:12:51.455852+0000","last_undegraded":"2024-04-25T14:12:51.455852+0000","last_fullsized":"2024-04-25T14:12:51.455852+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:32:07.914841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"76'9","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.530152+0000","last_change":"2024-04-25T14:12:22.529692+0000","last_active":"2024-04-25T14:12:22.530152+0000","last_peered":"2024-04-25T14:12:22.530152+0000","last_clean":"2024-04-25T14:12:22.530152+0000","last_became_active":"2024-04-25T14:12:22.517225+0000","last_became_peered":"2024-04-25T14:12:22.517225+0000","last_unstale":"2024-04-25T14:12:22.530152+0000","last_undegraded":"2024-04-25T14:12:22.530152+0000","last_fullsized":"2024-04-25T14:12:22.530152+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:36.610551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"78'9","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.458904+0000","last_change":"2024-04-25T14:11:29.563877+0000","last_active":"2024-04-25T14:12:51.458904+0000","last_peered":"2024-04-25T14:12:51.458904+0000","last_clean":"2024-04-25T14:12:51.458904+0000","last_became_active":"2024-04-25T14:11:29.563535+0000","last_became_peered":"2024-04-25T14:11:29.563535+0000","last_unstale":"2024-04-25T14:12:51.458904+0000","last_undegraded":"2024-04-25T14:12:51.458904+0000","last_fullsized":"2024-04-25T14:12:51.458904+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:07:38.805863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"76'5","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.508526+0000","last_change":"2024-04-25T14:11:28.543787+0000","last_active":"2024-04-25T14:12:22.508526+0000","last_peered":"2024-04-25T14:12:22.508526+0000","last_clean":"2024-04-25T14:12:22.508526+0000","last_became_active":"2024-04-25T14:11:28.540816+0000","last_became_peered":"2024-04-25T14:11:28.540816+0000","last_unstale":"2024-04-25T14:12:22.508526+0000","last_undegraded":"2024-04-25T14:12:22.508526+0000","last_fullsized":"2024-04-25T14:12:22.508526+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:18.776280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"78'15","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.462197+0000","last_change":"2024-04-25T14:12:22.514028+0000","last_active":"2024-04-25T14:12:51.462197+0000","last_peered":"2024-04-25T14:12:51.462197+0000","last_clean":"2024-04-25T14:12:51.462197+0000","last_became_active":"2024-04-25T14:12:22.513536+0000","last_became_peered":"2024-04-25T14:12:22.513536+0000","last_unstale":"2024-04-25T14:12:51.462197+0000","last_undegraded":"2024-04-25T14:12:51.462197+0000","last_fullsized":"2024-04-25T14:12:51.462197+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:31.921284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'7","reported_seq":145,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:13:29.702488+0000","last_change":"2024-04-25T14:11:28.537450+0000","last_active":"2024-04-25T14:13:29.702488+0000","last_peered":"2024-04-25T14:13:29.702488+0000","last_clean":"2024-04-25T14:13:29.702488+0000","last_became_active":"2024-04-25T14:11:28.537088+0000","last_became_peered":"2024-04-25T14:11:28.537088+0000","last_unstale":"2024-04-25T14:13:29.702488+0000","last_undegraded":"2024-04-25T14:13:29.702488+0000","last_fullsized":"2024-04-25T14:13:29.702488+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:05.779706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":120,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"78'6","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.451486+0000","last_change":"2024-04-25T14:11:29.572789+0000","last_active":"2024-04-25T14:12:51.451486+0000","last_peered":"2024-04-25T14:12:51.451486+0000","last_clean":"2024-04-25T14:12:51.451486+0000","last_became_active":"2024-04-25T14:11:29.572201+0000","last_became_peered":"2024-04-25T14:11:29.572201+0000","last_unstale":"2024-04-25T14:12:51.451486+0000","last_undegraded":"2024-04-25T14:12:51.451486+0000","last_fullsized":"2024-04-25T14:12:51.451486+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:39:58.237070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"76'17","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514338+0000","last_change":"2024-04-25T14:11:28.540577+0000","last_active":"2024-04-25T14:12:22.514338+0000","last_peered":"2024-04-25T14:12:22.514338+0000","last_clean":"2024-04-25T14:12:22.514338+0000","last_became_active":"2024-04-25T14:11:28.540212+0000","last_became_peered":"2024-04-25T14:11:28.540212+0000","last_unstale":"2024-04-25T14:12:22.514338+0000","last_undegraded":"2024-04-25T14:12:22.514338+0000","last_fullsized":"2024-04-25T14:12:22.514338+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:31.422332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"78'6","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.437799+0000","last_change":"2024-04-25T14:11:29.574952+0000","last_active":"2024-04-25T14:12:51.437799+0000","last_peered":"2024-04-25T14:12:51.437799+0000","last_clean":"2024-04-25T14:12:51.437799+0000","last_became_active":"2024-04-25T14:11:29.573677+0000","last_became_peered":"2024-04-25T14:11:29.573677+0000","last_unstale":"2024-04-25T14:12:51.437799+0000","last_undegraded":"2024-04-25T14:12:51.437799+0000","last_fullsized":"2024-04-25T14:12:51.437799+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:35.896103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"76'2","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514622+0000","last_change":"2024-04-25T14:11:28.541356+0000","last_active":"2024-04-25T14:12:22.514622+0000","last_peered":"2024-04-25T14:12:22.514622+0000","last_clean":"2024-04-25T14:12:22.514622+0000","last_became_active":"2024-04-25T14:11:28.540976+0000","last_became_peered":"2024-04-25T14:11:28.540976+0000","last_unstale":"2024-04-25T14:12:22.514622+0000","last_undegraded":"2024-04-25T14:12:22.514622+0000","last_fullsized":"2024-04-25T14:12:22.514622+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:32:50.599900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"78'5","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.442075+0000","last_change":"2024-04-25T14:11:29.554317+0000","last_active":"2024-04-25T14:12:51.442075+0000","last_peered":"2024-04-25T14:12:51.442075+0000","last_clean":"2024-04-25T14:12:51.442075+0000","last_became_active":"2024-04-25T14:11:29.553935+0000","last_became_peered":"2024-04-25T14:11:29.553935+0000","last_unstale":"2024-04-25T14:12:51.442075+0000","last_undegraded":"2024-04-25T14:12:51.442075+0000","last_fullsized":"2024-04-25T14:12:51.442075+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:01.405964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"76'1","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.509102+0000","last_change":"2024-04-25T14:11:28.546892+0000","last_active":"2024-04-25T14:12:22.509102+0000","last_peered":"2024-04-25T14:12:22.509102+0000","last_clean":"2024-04-25T14:12:22.509102+0000","last_became_active":"2024-04-25T14:11:28.546566+0000","last_became_peered":"2024-04-25T14:11:28.546566+0000","last_unstale":"2024-04-25T14:12:22.509102+0000","last_undegraded":"2024-04-25T14:12:22.509102+0000","last_fullsized":"2024-04-25T14:12:22.509102+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:05.239635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'5","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.431262+0000","last_change":"2024-04-25T14:11:29.565083+0000","last_active":"2024-04-25T14:12:51.431262+0000","last_peered":"2024-04-25T14:12:51.431262+0000","last_clean":"2024-04-25T14:12:51.431262+0000","last_became_active":"2024-04-25T14:11:29.564062+0000","last_became_peered":"2024-04-25T14:11:29.564062+0000","last_unstale":"2024-04-25T14:12:51.431262+0000","last_undegraded":"2024-04-25T14:12:51.431262+0000","last_fullsized":"2024-04-25T14:12:51.431262+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:07:10.664113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"76'5","reported_seq":140,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:13:29.608884+0000","last_change":"2024-04-25T14:11:28.538649+0000","last_active":"2024-04-25T14:13:29.608884+0000","last_peered":"2024-04-25T14:13:29.608884+0000","last_clean":"2024-04-25T14:13:29.608884+0000","last_became_active":"2024-04-25T14:11:28.538353+0000","last_became_peered":"2024-04-25T14:11:28.538353+0000","last_unstale":"2024-04-25T14:13:29.608884+0000","last_undegraded":"2024-04-25T14:13:29.608884+0000","last_fullsized":"2024-04-25T14:13:29.608884+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:06.513039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"78'7","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.435524+0000","last_change":"2024-04-25T14:11:29.576263+0000","last_active":"2024-04-25T14:12:51.435524+0000","last_peered":"2024-04-25T14:12:51.435524+0000","last_clean":"2024-04-25T14:12:51.435524+0000","last_became_active":"2024-04-25T14:11:29.575877+0000","last_became_peered":"2024-04-25T14:11:29.575877+0000","last_unstale":"2024-04-25T14:12:51.435524+0000","last_undegraded":"2024-04-25T14:12:51.435524+0000","last_fullsized":"2024-04-25T14:12:51.435524+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:33:37.639537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"78'8","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.438492+0000","last_change":"2024-04-25T14:11:29.568220+0000","last_active":"2024-04-25T14:12:51.438492+0000","last_peered":"2024-04-25T14:12:51.438492+0000","last_clean":"2024-04-25T14:12:51.438492+0000","last_became_active":"2024-04-25T14:11:29.567741+0000","last_became_peered":"2024-04-25T14:11:29.567741+0000","last_unstale":"2024-04-25T14:12:51.438492+0000","last_undegraded":"2024-04-25T14:12:51.438492+0000","last_fullsized":"2024-04-25T14:12:51.438492+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:05.134466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"78'6","reported_seq":143,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:13:30.703173+0000","last_change":"2024-04-25T14:11:28.537503+0000","last_active":"2024-04-25T14:13:30.703173+0000","last_peered":"2024-04-25T14:13:30.703173+0000","last_clean":"2024-04-25T14:13:30.703173+0000","last_became_active":"2024-04-25T14:11:28.537169+0000","last_became_peered":"2024-04-25T14:11:28.537169+0000","last_unstale":"2024-04-25T14:13:30.703173+0000","last_undegraded":"2024-04-25T14:13:30.703173+0000","last_fullsized":"2024-04-25T14:13:30.703173+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:09.752988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"78'8","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.454448+0000","last_change":"2024-04-25T14:12:22.516273+0000","last_active":"2024-04-25T14:12:51.454448+0000","last_peered":"2024-04-25T14:12:51.454448+0000","last_clean":"2024-04-25T14:12:51.454448+0000","last_became_active":"2024-04-25T14:12:22.515737+0000","last_became_peered":"2024-04-25T14:12:22.515737+0000","last_unstale":"2024-04-25T14:12:51.454448+0000","last_undegraded":"2024-04-25T14:12:51.454448+0000","last_fullsized":"2024-04-25T14:12:51.454448+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:36.241587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"76'6","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.512277+0000","last_change":"2024-04-25T14:11:28.538216+0000","last_active":"2024-04-25T14:12:22.512277+0000","last_peered":"2024-04-25T14:12:22.512277+0000","last_clean":"2024-04-25T14:12:22.512277+0000","last_became_active":"2024-04-25T14:11:28.537772+0000","last_became_peered":"2024-04-25T14:11:28.537772+0000","last_unstale":"2024-04-25T14:12:22.512277+0000","last_undegraded":"2024-04-25T14:12:22.512277+0000","last_fullsized":"2024-04-25T14:12:22.512277+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:55.460966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":94,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.509039+0000","last_change":"2024-04-25T14:10:12.194216+0000","last_active":"2024-04-25T14:12:22.509039+0000","last_peered":"2024-04-25T14:12:22.509039+0000","last_clean":"2024-04-25T14:12:22.509039+0000","last_became_active":"2024-04-25T14:10:12.189616+0000","last_became_peered":"2024-04-25T14:10:12.189616+0000","last_unstale":"2024-04-25T14:12:22.509039+0000","last_undegraded":"2024-04-25T14:12:22.509039+0000","last_fullsized":"2024-04-25T14:12:22.509039+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:47.553264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:47.553264+0000","last_clean_scrub_stamp":"2024-04-25T14:06:47.553264+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:47.576058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"78'5","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.433764+0000","last_change":"2024-04-25T14:11:29.574985+0000","last_active":"2024-04-25T14:12:51.433764+0000","last_peered":"2024-04-25T14:12:51.433764+0000","last_clean":"2024-04-25T14:12:51.433764+0000","last_became_active":"2024-04-25T14:11:29.574629+0000","last_became_peered":"2024-04-25T14:11:29.574629+0000","last_unstale":"2024-04-25T14:12:51.433764+0000","last_undegraded":"2024-04-25T14:12:51.433764+0000","last_fullsized":"2024-04-25T14:12:51.433764+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:01.899889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"76'4","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.520398+0000","last_change":"2024-04-25T14:12:22.519851+0000","last_active":"2024-04-25T14:12:22.520398+0000","last_peered":"2024-04-25T14:12:22.520398+0000","last_clean":"2024-04-25T14:12:22.520398+0000","last_became_active":"2024-04-25T14:12:22.515791+0000","last_became_peered":"2024-04-25T14:12:22.515791+0000","last_unstale":"2024-04-25T14:12:22.520398+0000","last_undegraded":"2024-04-25T14:12:22.520398+0000","last_fullsized":"2024-04-25T14:12:22.520398+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:47:01.242259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"76'4","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514464+0000","last_change":"2024-04-25T14:11:28.544269+0000","last_active":"2024-04-25T14:12:22.514464+0000","last_peered":"2024-04-25T14:12:22.514464+0000","last_clean":"2024-04-25T14:12:22.514464+0000","last_became_active":"2024-04-25T14:11:28.543897+0000","last_became_peered":"2024-04-25T14:11:28.543897+0000","last_unstale":"2024-04-25T14:12:22.514464+0000","last_undegraded":"2024-04-25T14:12:22.514464+0000","last_fullsized":"2024-04-25T14:12:22.514464+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:37.668187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"78'9","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.431345+0000","last_change":"2024-04-25T14:11:29.566428+0000","last_active":"2024-04-25T14:12:51.431345+0000","last_peered":"2024-04-25T14:12:51.431345+0000","last_clean":"2024-04-25T14:12:51.431345+0000","last_became_active":"2024-04-25T14:11:29.562598+0000","last_became_peered":"2024-04-25T14:11:29.562598+0000","last_unstale":"2024-04-25T14:12:51.431345+0000","last_undegraded":"2024-04-25T14:12:51.431345+0000","last_fullsized":"2024-04-25T14:12:51.431345+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:46.611359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"78'14","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.561086+0000","last_change":"2024-04-25T14:11:28.542571+0000","last_active":"2024-04-25T14:12:51.561086+0000","last_peered":"2024-04-25T14:12:51.561086+0000","last_clean":"2024-04-25T14:12:51.561086+0000","last_became_active":"2024-04-25T14:11:28.542272+0000","last_became_peered":"2024-04-25T14:11:28.542272+0000","last_unstale":"2024-04-25T14:12:51.561086+0000","last_undegraded":"2024-04-25T14:12:51.561086+0000","last_fullsized":"2024-04-25T14:12:51.561086+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:41:05.128068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"78'5","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.458486+0000","last_change":"2024-04-25T14:11:29.578565+0000","last_active":"2024-04-25T14:12:51.458486+0000","last_peered":"2024-04-25T14:12:51.458486+0000","last_clean":"2024-04-25T14:12:51.458486+0000","last_became_active":"2024-04-25T14:11:29.576782+0000","last_became_peered":"2024-04-25T14:11:29.576782+0000","last_unstale":"2024-04-25T14:12:51.458486+0000","last_undegraded":"2024-04-25T14:12:51.458486+0000","last_fullsized":"2024-04-25T14:12:51.458486+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:31.634602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"76'7","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.511746+0000","last_change":"2024-04-25T14:11:28.543838+0000","last_active":"2024-04-25T14:12:22.511746+0000","last_peered":"2024-04-25T14:12:22.511746+0000","last_clean":"2024-04-25T14:12:22.511746+0000","last_became_active":"2024-04-25T14:11:28.543511+0000","last_became_peered":"2024-04-25T14:11:28.543511+0000","last_unstale":"2024-04-25T14:12:22.511746+0000","last_undegraded":"2024-04-25T14:12:22.511746+0000","last_fullsized":"2024-04-25T14:12:22.511746+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:34.440234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"78'7","reported_seq":30,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.426925+0000","last_change":"2024-04-25T14:12:22.516416+0000","last_active":"2024-04-25T14:12:51.426925+0000","last_peered":"2024-04-25T14:12:51.426925+0000","last_clean":"2024-04-25T14:12:51.426925+0000","last_became_active":"2024-04-25T14:12:22.515928+0000","last_became_peered":"2024-04-25T14:12:22.515928+0000","last_unstale":"2024-04-25T14:12:51.426925+0000","last_undegraded":"2024-04-25T14:12:51.426925+0000","last_fullsized":"2024-04-25T14:12:51.426925+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:26.989293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"76'2","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.512719+0000","last_change":"2024-04-25T14:11:28.539443+0000","last_active":"2024-04-25T14:12:22.512719+0000","last_peered":"2024-04-25T14:12:22.512719+0000","last_clean":"2024-04-25T14:12:22.512719+0000","last_became_active":"2024-04-25T14:11:28.539113+0000","last_became_peered":"2024-04-25T14:11:28.539113+0000","last_unstale":"2024-04-25T14:12:22.512719+0000","last_undegraded":"2024-04-25T14:12:22.512719+0000","last_fullsized":"2024-04-25T14:12:22.512719+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:36.664972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"78'2","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.415389+0000","last_change":"2024-04-25T14:11:29.575451+0000","last_active":"2024-04-25T14:12:51.415389+0000","last_peered":"2024-04-25T14:12:51.415389+0000","last_clean":"2024-04-25T14:12:51.415389+0000","last_became_active":"2024-04-25T14:11:29.574983+0000","last_became_peered":"2024-04-25T14:11:29.574983+0000","last_unstale":"2024-04-25T14:12:51.415389+0000","last_undegraded":"2024-04-25T14:12:51.415389+0000","last_fullsized":"2024-04-25T14:12:51.415389+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:53.317608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"76'7","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514234+0000","last_change":"2024-04-25T14:11:28.546394+0000","last_active":"2024-04-25T14:12:22.514234+0000","last_peered":"2024-04-25T14:12:22.514234+0000","last_clean":"2024-04-25T14:12:22.514234+0000","last_became_active":"2024-04-25T14:11:28.546064+0000","last_became_peered":"2024-04-25T14:11:28.546064+0000","last_unstale":"2024-04-25T14:12:22.514234+0000","last_undegraded":"2024-04-25T14:12:22.514234+0000","last_fullsized":"2024-04-25T14:12:22.514234+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:15.301492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"78'9","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.439570+0000","last_change":"2024-04-25T14:11:29.569584+0000","last_active":"2024-04-25T14:12:51.439570+0000","last_peered":"2024-04-25T14:12:51.439570+0000","last_clean":"2024-04-25T14:12:51.439570+0000","last_became_active":"2024-04-25T14:11:29.569217+0000","last_became_peered":"2024-04-25T14:11:29.569217+0000","last_unstale":"2024-04-25T14:12:51.439570+0000","last_undegraded":"2024-04-25T14:12:51.439570+0000","last_fullsized":"2024-04-25T14:12:51.439570+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:05.708277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"76'3","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.512047+0000","last_change":"2024-04-25T14:11:28.544399+0000","last_active":"2024-04-25T14:12:22.512047+0000","last_peered":"2024-04-25T14:12:22.512047+0000","last_clean":"2024-04-25T14:12:22.512047+0000","last_became_active":"2024-04-25T14:11:28.544098+0000","last_became_peered":"2024-04-25T14:11:28.544098+0000","last_unstale":"2024-04-25T14:12:22.512047+0000","last_undegraded":"2024-04-25T14:12:22.512047+0000","last_fullsized":"2024-04-25T14:12:22.512047+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:05.148386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"78'8","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.430478+0000","last_change":"2024-04-25T14:12:22.517727+0000","last_active":"2024-04-25T14:12:51.430478+0000","last_peered":"2024-04-25T14:12:51.430478+0000","last_clean":"2024-04-25T14:12:51.430478+0000","last_became_active":"2024-04-25T14:12:22.517364+0000","last_became_peered":"2024-04-25T14:12:22.517364+0000","last_unstale":"2024-04-25T14:12:51.430478+0000","last_undegraded":"2024-04-25T14:12:51.430478+0000","last_fullsized":"2024-04-25T14:12:51.430478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:30:11.023308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"76'4","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514211+0000","last_change":"2024-04-25T14:11:28.542488+0000","last_active":"2024-04-25T14:12:22.514211+0000","last_peered":"2024-04-25T14:12:22.514211+0000","last_clean":"2024-04-25T14:12:22.514211+0000","last_became_active":"2024-04-25T14:11:28.542171+0000","last_became_peered":"2024-04-25T14:11:28.542171+0000","last_unstale":"2024-04-25T14:12:22.514211+0000","last_undegraded":"2024-04-25T14:12:22.514211+0000","last_fullsized":"2024-04-25T14:12:22.514211+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:48.201762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"78'5","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.459752+0000","last_change":"2024-04-25T14:11:29.573805+0000","last_active":"2024-04-25T14:12:51.459752+0000","last_peered":"2024-04-25T14:12:51.459752+0000","last_clean":"2024-04-25T14:12:51.459752+0000","last_became_active":"2024-04-25T14:11:29.573420+0000","last_became_peered":"2024-04-25T14:11:29.573420+0000","last_unstale":"2024-04-25T14:12:51.459752+0000","last_undegraded":"2024-04-25T14:12:51.459752+0000","last_fullsized":"2024-04-25T14:12:51.459752+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:18.721977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"76'5","reported_seq":141,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:13:28.701588+0000","last_change":"2024-04-25T14:11:28.552863+0000","last_active":"2024-04-25T14:13:28.701588+0000","last_peered":"2024-04-25T14:13:28.701588+0000","last_clean":"2024-04-25T14:13:28.701588+0000","last_became_active":"2024-04-25T14:11:28.552461+0000","last_became_peered":"2024-04-25T14:11:28.552461+0000","last_unstale":"2024-04-25T14:13:28.701588+0000","last_undegraded":"2024-04-25T14:13:28.701588+0000","last_fullsized":"2024-04-25T14:13:28.701588+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:46.264473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":117,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"78'7","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.459450+0000","last_change":"2024-04-25T14:11:29.571268+0000","last_active":"2024-04-25T14:12:51.459450+0000","last_peered":"2024-04-25T14:12:51.459450+0000","last_clean":"2024-04-25T14:12:51.459450+0000","last_became_active":"2024-04-25T14:11:29.570876+0000","last_became_peered":"2024-04-25T14:11:29.570876+0000","last_unstale":"2024-04-25T14:12:51.459450+0000","last_undegraded":"2024-04-25T14:12:51.459450+0000","last_fullsized":"2024-04-25T14:12:51.459450+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:15.093635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"76'3","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.508764+0000","last_change":"2024-04-25T14:11:28.543343+0000","last_active":"2024-04-25T14:12:22.508764+0000","last_peered":"2024-04-25T14:12:22.508764+0000","last_clean":"2024-04-25T14:12:22.508764+0000","last_became_active":"2024-04-25T14:11:28.540790+0000","last_became_peered":"2024-04-25T14:11:28.540790+0000","last_unstale":"2024-04-25T14:12:22.508764+0000","last_undegraded":"2024-04-25T14:12:22.508764+0000","last_fullsized":"2024-04-25T14:12:22.508764+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:20.498016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'7","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.458962+0000","last_change":"2024-04-25T14:11:29.566161+0000","last_active":"2024-04-25T14:12:51.458962+0000","last_peered":"2024-04-25T14:12:51.458962+0000","last_clean":"2024-04-25T14:12:51.458962+0000","last_became_active":"2024-04-25T14:11:29.564197+0000","last_became_peered":"2024-04-25T14:11:29.564197+0000","last_unstale":"2024-04-25T14:12:51.458962+0000","last_undegraded":"2024-04-25T14:12:51.458962+0000","last_fullsized":"2024-04-25T14:12:51.458962+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:35.291546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"76'5","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.505214+0000","last_change":"2024-04-25T14:11:28.543479+0000","last_active":"2024-04-25T14:12:22.505214+0000","last_peered":"2024-04-25T14:12:22.505214+0000","last_clean":"2024-04-25T14:12:22.505214+0000","last_became_active":"2024-04-25T14:11:28.543167+0000","last_became_peered":"2024-04-25T14:11:28.543167+0000","last_unstale":"2024-04-25T14:12:22.505214+0000","last_undegraded":"2024-04-25T14:12:22.505214+0000","last_fullsized":"2024-04-25T14:12:22.505214+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:54:53.572809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"78'9","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.455482+0000","last_change":"2024-04-25T14:11:29.572548+0000","last_active":"2024-04-25T14:12:51.455482+0000","last_peered":"2024-04-25T14:12:51.455482+0000","last_clean":"2024-04-25T14:12:51.455482+0000","last_became_active":"2024-04-25T14:11:29.572167+0000","last_became_peered":"2024-04-25T14:11:29.572167+0000","last_unstale":"2024-04-25T14:12:51.455482+0000","last_undegraded":"2024-04-25T14:12:51.455482+0000","last_fullsized":"2024-04-25T14:12:51.455482+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:38.423331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"78'9","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.440678+0000","last_change":"2024-04-25T14:11:29.560956+0000","last_active":"2024-04-25T14:12:51.440678+0000","last_peered":"2024-04-25T14:12:51.440678+0000","last_clean":"2024-04-25T14:12:51.440678+0000","last_became_active":"2024-04-25T14:11:29.560519+0000","last_became_peered":"2024-04-25T14:11:29.560519+0000","last_unstale":"2024-04-25T14:12:51.440678+0000","last_undegraded":"2024-04-25T14:12:51.440678+0000","last_fullsized":"2024-04-25T14:12:51.440678+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:53:47.677923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"78'12","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.435167+0000","last_change":"2024-04-25T14:11:29.574372+0000","last_active":"2024-04-25T14:12:51.435167+0000","last_peered":"2024-04-25T14:12:51.435167+0000","last_clean":"2024-04-25T14:12:51.435167+0000","last_became_active":"2024-04-25T14:11:29.573995+0000","last_became_peered":"2024-04-25T14:11:29.573995+0000","last_unstale":"2024-04-25T14:12:51.435167+0000","last_undegraded":"2024-04-25T14:12:51.435167+0000","last_fullsized":"2024-04-25T14:12:51.435167+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:12.381512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"78'8","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.432226+0000","last_change":"2024-04-25T14:11:29.578324+0000","last_active":"2024-04-25T14:12:51.432226+0000","last_peered":"2024-04-25T14:12:51.432226+0000","last_clean":"2024-04-25T14:12:51.432226+0000","last_became_active":"2024-04-25T14:11:29.577917+0000","last_became_peered":"2024-04-25T14:11:29.577917+0000","last_unstale":"2024-04-25T14:12:51.432226+0000","last_undegraded":"2024-04-25T14:12:51.432226+0000","last_fullsized":"2024-04-25T14:12:51.432226+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:20:27.070720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"78'7","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.432298+0000","last_change":"2024-04-25T14:11:29.578879+0000","last_active":"2024-04-25T14:12:51.432298+0000","last_peered":"2024-04-25T14:12:51.432298+0000","last_clean":"2024-04-25T14:12:51.432298+0000","last_became_active":"2024-04-25T14:11:29.578551+0000","last_became_peered":"2024-04-25T14:11:29.578551+0000","last_unstale":"2024-04-25T14:12:51.432298+0000","last_undegraded":"2024-04-25T14:12:51.432298+0000","last_fullsized":"2024-04-25T14:12:51.432298+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:41.083794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"78'4","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.437936+0000","last_change":"2024-04-25T14:11:29.574054+0000","last_active":"2024-04-25T14:12:51.437936+0000","last_peered":"2024-04-25T14:12:51.437936+0000","last_clean":"2024-04-25T14:12:51.437936+0000","last_became_active":"2024-04-25T14:11:29.573700+0000","last_became_peered":"2024-04-25T14:11:29.573700+0000","last_unstale":"2024-04-25T14:12:51.437936+0000","last_undegraded":"2024-04-25T14:12:51.437936+0000","last_fullsized":"2024-04-25T14:12:51.437936+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:23.304580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"78'6","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.440195+0000","last_change":"2024-04-25T14:11:29.553768+0000","last_active":"2024-04-25T14:12:51.440195+0000","last_peered":"2024-04-25T14:12:51.440195+0000","last_clean":"2024-04-25T14:12:51.440195+0000","last_became_active":"2024-04-25T14:11:29.553376+0000","last_became_peered":"2024-04-25T14:11:29.553376+0000","last_unstale":"2024-04-25T14:12:51.440195+0000","last_undegraded":"2024-04-25T14:12:51.440195+0000","last_fullsized":"2024-04-25T14:12:51.440195+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:46.688368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"78'15","reported_seq":32,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.440586+0000","last_change":"2024-04-25T14:11:29.563122+0000","last_active":"2024-04-25T14:12:51.440586+0000","last_peered":"2024-04-25T14:12:51.440586+0000","last_clean":"2024-04-25T14:12:51.440586+0000","last_became_active":"2024-04-25T14:11:29.562717+0000","last_became_peered":"2024-04-25T14:11:29.562717+0000","last_unstale":"2024-04-25T14:12:51.440586+0000","last_undegraded":"2024-04-25T14:12:51.440586+0000","last_fullsized":"2024-04-25T14:12:51.440586+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:37.585352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"78'2","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.436426+0000","last_change":"2024-04-25T14:11:29.576487+0000","last_active":"2024-04-25T14:12:51.436426+0000","last_peered":"2024-04-25T14:12:51.436426+0000","last_clean":"2024-04-25T14:12:51.436426+0000","last_became_active":"2024-04-25T14:11:29.575070+0000","last_became_peered":"2024-04-25T14:11:29.575070+0000","last_unstale":"2024-04-25T14:12:51.436426+0000","last_undegraded":"2024-04-25T14:12:51.436426+0000","last_fullsized":"2024-04-25T14:12:51.436426+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:13.803505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"78'6","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.427743+0000","last_change":"2024-04-25T14:11:29.568733+0000","last_active":"2024-04-25T14:12:51.427743+0000","last_peered":"2024-04-25T14:12:51.427743+0000","last_clean":"2024-04-25T14:12:51.427743+0000","last_became_active":"2024-04-25T14:11:29.567665+0000","last_became_peered":"2024-04-25T14:11:29.567665+0000","last_unstale":"2024-04-25T14:12:51.427743+0000","last_undegraded":"2024-04-25T14:12:51.427743+0000","last_fullsized":"2024-04-25T14:12:51.427743+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:57.956199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"78'8","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.433324+0000","last_change":"2024-04-25T14:11:29.574648+0000","last_active":"2024-04-25T14:12:51.433324+0000","last_peered":"2024-04-25T14:12:51.433324+0000","last_clean":"2024-04-25T14:12:51.433324+0000","last_became_active":"2024-04-25T14:11:29.574247+0000","last_became_peered":"2024-04-25T14:11:29.574247+0000","last_unstale":"2024-04-25T14:12:51.433324+0000","last_undegraded":"2024-04-25T14:12:51.433324+0000","last_fullsized":"2024-04-25T14:12:51.433324+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:57.110017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"78'10","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.458289+0000","last_change":"2024-04-25T14:11:29.573633+0000","last_active":"2024-04-25T14:12:51.458289+0000","last_peered":"2024-04-25T14:12:51.458289+0000","last_clean":"2024-04-25T14:12:51.458289+0000","last_became_active":"2024-04-25T14:11:29.573303+0000","last_became_peered":"2024-04-25T14:11:29.573303+0000","last_unstale":"2024-04-25T14:12:51.458289+0000","last_undegraded":"2024-04-25T14:12:51.458289+0000","last_fullsized":"2024-04-25T14:12:51.458289+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:11.458117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"78'11","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.460396+0000","last_change":"2024-04-25T14:11:29.569136+0000","last_active":"2024-04-25T14:12:51.460396+0000","last_peered":"2024-04-25T14:12:51.460396+0000","last_clean":"2024-04-25T14:12:51.460396+0000","last_became_active":"2024-04-25T14:11:29.568801+0000","last_became_peered":"2024-04-25T14:11:29.568801+0000","last_unstale":"2024-04-25T14:12:51.460396+0000","last_undegraded":"2024-04-25T14:12:51.460396+0000","last_fullsized":"2024-04-25T14:12:51.460396+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:30.969081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"78'8","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.442382+0000","last_change":"2024-04-25T14:11:29.566754+0000","last_active":"2024-04-25T14:12:51.442382+0000","last_peered":"2024-04-25T14:12:51.442382+0000","last_clean":"2024-04-25T14:12:51.442382+0000","last_became_active":"2024-04-25T14:11:29.566368+0000","last_became_peered":"2024-04-25T14:11:29.566368+0000","last_unstale":"2024-04-25T14:12:51.442382+0000","last_undegraded":"2024-04-25T14:12:51.442382+0000","last_fullsized":"2024-04-25T14:12:51.442382+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:37.057718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281277,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67129,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1209,"num_read_kb":679,"num_write":247,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":740,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":202003,"data_compressed":5550,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678055,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":851968,"data_stored":405341,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874284,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":1056477,"data_compressed":43412,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037809,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":868352,"data_stored":405203,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234039,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32012,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713140,"statfs":{"total":95995035648,"available":95962255360,"internally_reserved":0,"allocated":856064,"data_stored":404909,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430267,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":917504,"data_stored":999080,"data_compressed":41562,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593793,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":851968,"data_stored":404920,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":465784,"data_compressed":7643,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":872448,"data_stored":406259,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":466160,"data_compressed":7643,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149776,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":868352,"data_stored":404745,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953546,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":860160,"data_stored":405985,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790022,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":997918,"data_compressed":41562,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":61703,"data_compressed":1850,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":62079,"data_compressed":1850,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":61911,"data_compressed":1850,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:13:36.276 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-04-25T14:13:36.276 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:13:36.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.421+0000 7f3232127640 1 -- 172.21.15.96:0/813678581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f322c108900 msgr2=0x7f322c10ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:36.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.421+0000 7f3232127640 1 --2- 172.21.15.96:0/813678581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f322c108900 0x7f322c10ad20 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3224009ca0 tx=0x7f322402a8c0 comp rx=0 tx=0).stop 2024-04-25T14:13:36.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.421+0000 7f3232127640 1 -- 172.21.15.96:0/813678581 shutdown_connections 2024-04-25T14:13:36.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.421+0000 7f3232127640 1 --2- 172.21.15.96:0/813678581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f322c108900 0x7f322c10ad20 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:36.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.421+0000 7f3232127640 1 --2- 172.21.15.96:0/813678581 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f322c103c00 0x7f322c104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:36.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.421+0000 7f3232127640 1 --2- 172.21.15.96:0/813678581 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f322c1029c0 0x7f322c1036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:36.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.421+0000 7f3232127640 1 -- 172.21.15.96:0/813678581 >> 172.21.15.96:0/813678581 conn(0x7f322c0fdbd0 msgr2=0x7f322c100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:36.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.421+0000 7f3232127640 1 -- 172.21.15.96:0/813678581 shutdown_connections 2024-04-25T14:13:36.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.421+0000 7f3232127640 1 -- 172.21.15.96:0/813678581 wait complete. 2024-04-25T14:13:36.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.421+0000 7f3232127640 1 Processor -- start 2024-04-25T14:13:36.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.422+0000 7f3232127640 1 -- start start 2024-04-25T14:13:36.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.422+0000 7f3232127640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f322c1029c0 0x7f322c1990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:36.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.422+0000 7f3232127640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f322c103c00 0x7f322c1995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:36.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.422+0000 7f3232127640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f322c108900 0x7f322c1a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:36.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.422+0000 7f3232127640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f322c10e4d0 con 0x7f322c108900 2024-04-25T14:13:36.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.422+0000 7f3232127640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f322c10e350 con 0x7f322c1029c0 2024-04-25T14:13:36.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.422+0000 7f3232127640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f322c10e650 con 0x7f322c103c00 2024-04-25T14:13:36.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.422+0000 7f322bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f322c108900 0x7f322c1a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:36.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.422+0000 7f322bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f322c108900 0x7f322c1a0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40582/0 (socket says 172.21.15.96:40582) 2024-04-25T14:13:36.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.422+0000 7f322bfff640 1 -- 172.21.15.96:0/3825157868 learned_addr learned my addr 172.21.15.96:0/3825157868 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:36.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.422+0000 7f322b7fe640 1 --2- 172.21.15.96:0/3825157868 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f322c1029c0 0x7f322c1990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:36.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.422+0000 7f322affd640 1 --2- 172.21.15.96:0/3825157868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f322c103c00 0x7f322c1995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:36.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.422+0000 7f322bfff640 1 -- 172.21.15.96:0/3825157868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f322c103c00 msgr2=0x7f322c1995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:36.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.422+0000 7f322bfff640 1 --2- 172.21.15.96:0/3825157868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f322c103c00 0x7f322c1995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:36.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.422+0000 7f322bfff640 1 -- 172.21.15.96:0/3825157868 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f322c1029c0 msgr2=0x7f322c1990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:36.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.422+0000 7f322bfff640 1 --2- 172.21.15.96:0/3825157868 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f322c1029c0 0x7f322c1990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:36.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.422+0000 7f322bfff640 1 -- 172.21.15.96:0/3825157868 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f322c1a0be0 con 0x7f322c108900 2024-04-25T14:13:36.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.422+0000 7f322bfff640 1 --2- 172.21.15.96:0/3825157868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f322c108900 0x7f322c1a0570 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f32240097b0 tx=0x7f3224031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:36.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.423+0000 7f3228ff9640 1 -- 172.21.15.96:0/3825157868 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3224003bf0 con 0x7f322c108900 2024-04-25T14:13:36.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.423+0000 7f3228ff9640 1 -- 172.21.15.96:0/3825157868 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3224003d90 con 0x7f322c108900 2024-04-25T14:13:36.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.423+0000 7f3232127640 1 -- 172.21.15.96:0/3825157868 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f322c1a0e70 con 0x7f322c108900 2024-04-25T14:13:36.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.423+0000 7f3228ff9640 1 -- 172.21.15.96:0/3825157868 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322402fc30 con 0x7f322c108900 2024-04-25T14:13:36.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.423+0000 7f3232127640 1 -- 172.21.15.96:0/3825157868 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f322c1a1330 con 0x7f322c108900 2024-04-25T14:13:36.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.424+0000 7f3232127640 1 -- 172.21.15.96:0/3825157868 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f322c107710 con 0x7f322c108900 2024-04-25T14:13:36.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.425+0000 7f3228ff9640 1 -- 172.21.15.96:0/3825157868 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f3224004040 con 0x7f322c108900 2024-04-25T14:13:36.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.426+0000 7f3228ff9640 1 --2- 172.21.15.96:0/3825157868 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f32140afb70 0x7f32140b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:36.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.426+0000 7f322b7fe640 1 --2- 172.21.15.96:0/3825157868 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f32140afb70 0x7f32140b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:36.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.426+0000 7f3228ff9640 1 -- 172.21.15.96:0/3825157868 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f3224100370 con 0x7f322c108900 2024-04-25T14:13:36.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.426+0000 7f322b7fe640 1 --2- 172.21.15.96:0/3825157868 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f32140afb70 0x7f32140b2080 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f322c1035c0 tx=0x7f321c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:36.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.429+0000 7f3228ff9640 1 -- 172.21.15.96:0/3825157868 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f32240c8f30 con 0x7f322c108900 2024-04-25T14:13:36.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.609+0000 7f3232127640 1 -- 172.21.15.96:0/3825157868 --> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f322c062fe0 con 0x7f32140afb70 2024-04-25T14:13:36.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.616+0000 7f3228ff9640 1 -- 172.21.15.96:0/3825157868 <== mgr.14152 v2:172.21.15.96:6800/536332425 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259165 (secure 0 0 0) 0x7f322c062fe0 con 0x7f32140afb70 2024-04-25T14:13:36.619 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:13:36.620 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-04-25T14:13:36.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.620+0000 7f3232127640 1 -- 172.21.15.96:0/3825157868 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f32140afb70 msgr2=0x7f32140b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:36.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.620+0000 7f3232127640 1 --2- 172.21.15.96:0/3825157868 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f32140afb70 0x7f32140b2080 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f322c1035c0 tx=0x7f321c005f90 comp rx=0 tx=0).stop 2024-04-25T14:13:36.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.620+0000 7f3232127640 1 -- 172.21.15.96:0/3825157868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f322c108900 msgr2=0x7f322c1a0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:36.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.620+0000 7f3232127640 1 --2- 172.21.15.96:0/3825157868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f322c108900 0x7f322c1a0570 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f32240097b0 tx=0x7f3224031840 comp rx=0 tx=0).stop 2024-04-25T14:13:36.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.620+0000 7f3232127640 1 -- 172.21.15.96:0/3825157868 shutdown_connections 2024-04-25T14:13:36.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.620+0000 7f3232127640 1 --2- 172.21.15.96:0/3825157868 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f32140afb70 0x7f32140b2080 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:36.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.620+0000 7f3232127640 1 --2- 172.21.15.96:0/3825157868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f322c108900 0x7f322c1a0570 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:36.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.620+0000 7f3232127640 1 --2- 172.21.15.96:0/3825157868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f322c103c00 0x7f322c1995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:36.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.620+0000 7f3232127640 1 --2- 172.21.15.96:0/3825157868 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f322c1029c0 0x7f322c1990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:36.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.620+0000 7f3232127640 1 -- 172.21.15.96:0/3825157868 >> 172.21.15.96:0/3825157868 conn(0x7f322c0fdbd0 msgr2=0x7f322c1048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:36.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.620+0000 7f3232127640 1 -- 172.21.15.96:0/3825157868 shutdown_connections 2024-04-25T14:13:36.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.620+0000 7f3232127640 1 -- 172.21.15.96:0/3825157868 wait complete. 2024-04-25T14:13:36.631 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2024-04-25T14:13:35.432428+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657731,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1285,"num_read_kb":743,"num_write":372,"num_write_kb":2291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":2952580,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":808,"ondisk_log_size":808,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359604,"kb_used_data":10452,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124582220,"statfs":{"total":1151940427776,"available":1151572193280,"internally_reserved":0,"allocated":10702848,"data_stored":6822781,"data_compressed":182373,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":105,"num_read_kb":55,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003170"},"pg_stats":[{"pgid":"3.3f","version":"78'8","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.448766+0000","last_change":"2024-04-25T14:11:29.569615+0000","last_active":"2024-04-25T14:12:51.448766+0000","last_peered":"2024-04-25T14:12:51.448766+0000","last_clean":"2024-04-25T14:12:51.448766+0000","last_became_active":"2024-04-25T14:11:29.569237+0000","last_became_peered":"2024-04-25T14:11:29.569237+0000","last_unstale":"2024-04-25T14:12:51.448766+0000","last_undegraded":"2024-04-25T14:12:51.448766+0000","last_fullsized":"2024-04-25T14:12:51.448766+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:24.367533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"78'4","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.440396+0000","last_change":"2024-04-25T14:11:29.571820+0000","last_active":"2024-04-25T14:12:51.440396+0000","last_peered":"2024-04-25T14:12:51.440396+0000","last_clean":"2024-04-25T14:12:51.440396+0000","last_became_active":"2024-04-25T14:11:29.571456+0000","last_became_peered":"2024-04-25T14:11:29.571456+0000","last_unstale":"2024-04-25T14:12:51.440396+0000","last_undegraded":"2024-04-25T14:12:51.440396+0000","last_fullsized":"2024-04-25T14:12:51.440396+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:33.442301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"78'7","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.456139+0000","last_change":"2024-04-25T14:11:29.579302+0000","last_active":"2024-04-25T14:12:51.456139+0000","last_peered":"2024-04-25T14:12:51.456139+0000","last_clean":"2024-04-25T14:12:51.456139+0000","last_became_active":"2024-04-25T14:11:29.577734+0000","last_became_peered":"2024-04-25T14:11:29.577734+0000","last_unstale":"2024-04-25T14:12:51.456139+0000","last_undegraded":"2024-04-25T14:12:51.456139+0000","last_fullsized":"2024-04-25T14:12:51.456139+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:59.147554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"78'8","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.442356+0000","last_change":"2024-04-25T14:11:29.574942+0000","last_active":"2024-04-25T14:12:51.442356+0000","last_peered":"2024-04-25T14:12:51.442356+0000","last_clean":"2024-04-25T14:12:51.442356+0000","last_became_active":"2024-04-25T14:11:29.573774+0000","last_became_peered":"2024-04-25T14:11:29.573774+0000","last_unstale":"2024-04-25T14:12:51.442356+0000","last_undegraded":"2024-04-25T14:12:51.442356+0000","last_fullsized":"2024-04-25T14:12:51.442356+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:10.626265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"78'4","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.426798+0000","last_change":"2024-04-25T14:11:29.557199+0000","last_active":"2024-04-25T14:12:51.426798+0000","last_peered":"2024-04-25T14:12:51.426798+0000","last_clean":"2024-04-25T14:12:51.426798+0000","last_became_active":"2024-04-25T14:11:29.556836+0000","last_became_peered":"2024-04-25T14:11:29.556836+0000","last_unstale":"2024-04-25T14:12:51.426798+0000","last_undegraded":"2024-04-25T14:12:51.426798+0000","last_fullsized":"2024-04-25T14:12:51.426798+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:04:45.882935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"78'6","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.434655+0000","last_change":"2024-04-25T14:11:29.557988+0000","last_active":"2024-04-25T14:12:51.434655+0000","last_peered":"2024-04-25T14:12:51.434655+0000","last_clean":"2024-04-25T14:12:51.434655+0000","last_became_active":"2024-04-25T14:11:29.557622+0000","last_became_peered":"2024-04-25T14:11:29.557622+0000","last_unstale":"2024-04-25T14:12:51.434655+0000","last_undegraded":"2024-04-25T14:12:51.434655+0000","last_fullsized":"2024-04-25T14:12:51.434655+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:28:37.578368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"78'5","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.452293+0000","last_change":"2024-04-25T14:11:29.565049+0000","last_active":"2024-04-25T14:12:51.452293+0000","last_peered":"2024-04-25T14:12:51.452293+0000","last_clean":"2024-04-25T14:12:51.452293+0000","last_became_active":"2024-04-25T14:11:29.564688+0000","last_became_peered":"2024-04-25T14:11:29.564688+0000","last_unstale":"2024-04-25T14:12:51.452293+0000","last_undegraded":"2024-04-25T14:12:51.452293+0000","last_fullsized":"2024-04-25T14:12:51.452293+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:48.128778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"78'9","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.432114+0000","last_change":"2024-04-25T14:11:29.565879+0000","last_active":"2024-04-25T14:12:51.432114+0000","last_peered":"2024-04-25T14:12:51.432114+0000","last_clean":"2024-04-25T14:12:51.432114+0000","last_became_active":"2024-04-25T14:11:29.565525+0000","last_became_peered":"2024-04-25T14:11:29.565525+0000","last_unstale":"2024-04-25T14:12:51.432114+0000","last_undegraded":"2024-04-25T14:12:51.432114+0000","last_fullsized":"2024-04-25T14:12:51.432114+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:06.956334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"78'11","reported_seq":34,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.450891+0000","last_change":"2024-04-25T14:12:22.517776+0000","last_active":"2024-04-25T14:12:51.450891+0000","last_peered":"2024-04-25T14:12:51.450891+0000","last_clean":"2024-04-25T14:12:51.450891+0000","last_became_active":"2024-04-25T14:12:22.517232+0000","last_became_peered":"2024-04-25T14:12:22.517232+0000","last_unstale":"2024-04-25T14:12:51.450891+0000","last_undegraded":"2024-04-25T14:12:51.450891+0000","last_fullsized":"2024-04-25T14:12:51.450891+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:51.912445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,11],"acting":[1,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"78'16","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.438962+0000","last_change":"2024-04-25T14:11:29.561841+0000","last_active":"2024-04-25T14:12:51.438962+0000","last_peered":"2024-04-25T14:12:51.438962+0000","last_clean":"2024-04-25T14:12:51.438962+0000","last_became_active":"2024-04-25T14:11:29.560145+0000","last_became_peered":"2024-04-25T14:11:29.560145+0000","last_unstale":"2024-04-25T14:12:51.438962+0000","last_undegraded":"2024-04-25T14:12:51.438962+0000","last_fullsized":"2024-04-25T14:12:51.438962+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:11.654692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"78'6","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.459530+0000","last_change":"2024-04-25T14:11:29.570954+0000","last_active":"2024-04-25T14:12:51.459530+0000","last_peered":"2024-04-25T14:12:51.459530+0000","last_clean":"2024-04-25T14:12:51.459530+0000","last_became_active":"2024-04-25T14:11:29.570368+0000","last_became_peered":"2024-04-25T14:11:29.570368+0000","last_unstale":"2024-04-25T14:12:51.459530+0000","last_undegraded":"2024-04-25T14:12:51.459530+0000","last_fullsized":"2024-04-25T14:12:51.459530+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:50.153695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"78'7","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.443244+0000","last_change":"2024-04-25T14:11:29.571911+0000","last_active":"2024-04-25T14:12:51.443244+0000","last_peered":"2024-04-25T14:12:51.443244+0000","last_clean":"2024-04-25T14:12:51.443244+0000","last_became_active":"2024-04-25T14:11:29.568557+0000","last_became_peered":"2024-04-25T14:11:29.568557+0000","last_unstale":"2024-04-25T14:12:51.443244+0000","last_undegraded":"2024-04-25T14:12:51.443244+0000","last_fullsized":"2024-04-25T14:12:51.443244+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:24.221527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"78'10","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.426972+0000","last_change":"2024-04-25T14:11:29.566826+0000","last_active":"2024-04-25T14:12:51.426972+0000","last_peered":"2024-04-25T14:12:51.426972+0000","last_clean":"2024-04-25T14:12:51.426972+0000","last_became_active":"2024-04-25T14:11:29.565825+0000","last_became_peered":"2024-04-25T14:11:29.565825+0000","last_unstale":"2024-04-25T14:12:51.426972+0000","last_undegraded":"2024-04-25T14:12:51.426972+0000","last_fullsized":"2024-04-25T14:12:51.426972+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:46.711809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"78'7","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.424340+0000","last_change":"2024-04-25T14:11:29.577359+0000","last_active":"2024-04-25T14:12:51.424340+0000","last_peered":"2024-04-25T14:12:51.424340+0000","last_clean":"2024-04-25T14:12:51.424340+0000","last_became_active":"2024-04-25T14:11:29.574643+0000","last_became_peered":"2024-04-25T14:11:29.574643+0000","last_unstale":"2024-04-25T14:12:51.424340+0000","last_undegraded":"2024-04-25T14:12:51.424340+0000","last_fullsized":"2024-04-25T14:12:51.424340+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:07.582154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"78'7","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.432124+0000","last_change":"2024-04-25T14:11:29.583606+0000","last_active":"2024-04-25T14:12:51.432124+0000","last_peered":"2024-04-25T14:12:51.432124+0000","last_clean":"2024-04-25T14:12:51.432124+0000","last_became_active":"2024-04-25T14:11:29.582091+0000","last_became_peered":"2024-04-25T14:11:29.582091+0000","last_unstale":"2024-04-25T14:12:51.432124+0000","last_undegraded":"2024-04-25T14:12:51.432124+0000","last_fullsized":"2024-04-25T14:12:51.432124+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:58.711591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"78'9","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.458160+0000","last_change":"2024-04-25T14:11:29.571223+0000","last_active":"2024-04-25T14:12:51.458160+0000","last_peered":"2024-04-25T14:12:51.458160+0000","last_clean":"2024-04-25T14:12:51.458160+0000","last_became_active":"2024-04-25T14:11:29.570856+0000","last_became_peered":"2024-04-25T14:11:29.570856+0000","last_unstale":"2024-04-25T14:12:51.458160+0000","last_undegraded":"2024-04-25T14:12:51.458160+0000","last_fullsized":"2024-04-25T14:12:51.458160+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:53.058088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"78'8","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.449162+0000","last_change":"2024-04-25T14:11:29.574467+0000","last_active":"2024-04-25T14:12:51.449162+0000","last_peered":"2024-04-25T14:12:51.449162+0000","last_clean":"2024-04-25T14:12:51.449162+0000","last_became_active":"2024-04-25T14:11:29.573878+0000","last_became_peered":"2024-04-25T14:11:29.573878+0000","last_unstale":"2024-04-25T14:12:51.449162+0000","last_undegraded":"2024-04-25T14:12:51.449162+0000","last_fullsized":"2024-04-25T14:12:51.449162+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:22:02.087606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"78'10","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.454589+0000","last_change":"2024-04-25T14:11:29.561612+0000","last_active":"2024-04-25T14:12:51.454589+0000","last_peered":"2024-04-25T14:12:51.454589+0000","last_clean":"2024-04-25T14:12:51.454589+0000","last_became_active":"2024-04-25T14:11:29.561211+0000","last_became_peered":"2024-04-25T14:11:29.561211+0000","last_unstale":"2024-04-25T14:12:51.454589+0000","last_undegraded":"2024-04-25T14:12:51.454589+0000","last_fullsized":"2024-04-25T14:12:51.454589+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:29.195093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"78'7","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.430634+0000","last_change":"2024-04-25T14:11:29.575231+0000","last_active":"2024-04-25T14:12:51.430634+0000","last_peered":"2024-04-25T14:12:51.430634+0000","last_clean":"2024-04-25T14:12:51.430634+0000","last_became_active":"2024-04-25T14:11:29.573928+0000","last_became_peered":"2024-04-25T14:11:29.573928+0000","last_unstale":"2024-04-25T14:12:51.430634+0000","last_undegraded":"2024-04-25T14:12:51.430634+0000","last_fullsized":"2024-04-25T14:12:51.430634+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:10.025584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"76'13","reported_seq":172,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.613466+0000","last_change":"2024-04-25T14:12:22.527280+0000","last_active":"2024-04-25T14:13:33.613466+0000","last_peered":"2024-04-25T14:13:33.613466+0000","last_clean":"2024-04-25T14:13:33.613466+0000","last_became_active":"2024-04-25T14:12:22.515420+0000","last_became_peered":"2024-04-25T14:12:22.515420+0000","last_unstale":"2024-04-25T14:13:33.613466+0000","last_undegraded":"2024-04-25T14:13:33.613466+0000","last_fullsized":"2024-04-25T14:13:33.613466+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:53.882212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"78'5","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.427842+0000","last_change":"2024-04-25T14:11:29.576855+0000","last_active":"2024-04-25T14:12:51.427842+0000","last_peered":"2024-04-25T14:12:51.427842+0000","last_clean":"2024-04-25T14:12:51.427842+0000","last_became_active":"2024-04-25T14:11:29.574566+0000","last_became_peered":"2024-04-25T14:11:29.574566+0000","last_unstale":"2024-04-25T14:12:51.427842+0000","last_undegraded":"2024-04-25T14:12:51.427842+0000","last_fullsized":"2024-04-25T14:12:51.427842+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:31.325583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"76'2","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514493+0000","last_change":"2024-04-25T14:11:28.552763+0000","last_active":"2024-04-25T14:12:22.514493+0000","last_peered":"2024-04-25T14:12:22.514493+0000","last_clean":"2024-04-25T14:12:22.514493+0000","last_became_active":"2024-04-25T14:11:28.552403+0000","last_became_peered":"2024-04-25T14:11:28.552403+0000","last_unstale":"2024-04-25T14:12:22.514493+0000","last_undegraded":"2024-04-25T14:12:22.514493+0000","last_fullsized":"2024-04-25T14:12:22.514493+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:31.473779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"78'6","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.456271+0000","last_change":"2024-04-25T14:11:29.571019+0000","last_active":"2024-04-25T14:12:51.456271+0000","last_peered":"2024-04-25T14:12:51.456271+0000","last_clean":"2024-04-25T14:12:51.456271+0000","last_became_active":"2024-04-25T14:11:29.570633+0000","last_became_peered":"2024-04-25T14:11:29.570633+0000","last_unstale":"2024-04-25T14:12:51.456271+0000","last_undegraded":"2024-04-25T14:12:51.456271+0000","last_fullsized":"2024-04-25T14:12:51.456271+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:38:32.664318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"76'12","reported_seq":32,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514608+0000","last_change":"2024-04-25T14:11:28.545189+0000","last_active":"2024-04-25T14:12:22.514608+0000","last_peered":"2024-04-25T14:12:22.514608+0000","last_clean":"2024-04-25T14:12:22.514608+0000","last_became_active":"2024-04-25T14:11:28.544857+0000","last_became_peered":"2024-04-25T14:11:28.544857+0000","last_unstale":"2024-04-25T14:12:22.514608+0000","last_undegraded":"2024-04-25T14:12:22.514608+0000","last_fullsized":"2024-04-25T14:12:22.514608+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:58.464226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"78'9","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.455643+0000","last_change":"2024-04-25T14:11:29.576696+0000","last_active":"2024-04-25T14:12:51.455643+0000","last_peered":"2024-04-25T14:12:51.455643+0000","last_clean":"2024-04-25T14:12:51.455643+0000","last_became_active":"2024-04-25T14:11:29.576337+0000","last_became_peered":"2024-04-25T14:11:29.576337+0000","last_unstale":"2024-04-25T14:12:51.455643+0000","last_undegraded":"2024-04-25T14:12:51.455643+0000","last_fullsized":"2024-04-25T14:12:51.455643+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:31:30.410434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"76'5","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514457+0000","last_change":"2024-04-25T14:11:28.542466+0000","last_active":"2024-04-25T14:12:22.514457+0000","last_peered":"2024-04-25T14:12:22.514457+0000","last_clean":"2024-04-25T14:12:22.514457+0000","last_became_active":"2024-04-25T14:11:28.542092+0000","last_became_peered":"2024-04-25T14:11:28.542092+0000","last_unstale":"2024-04-25T14:12:22.514457+0000","last_undegraded":"2024-04-25T14:12:22.514457+0000","last_fullsized":"2024-04-25T14:12:22.514457+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:33.810111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"78'15","reported_seq":32,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.441623+0000","last_change":"2024-04-25T14:11:29.567842+0000","last_active":"2024-04-25T14:12:51.441623+0000","last_peered":"2024-04-25T14:12:51.441623+0000","last_clean":"2024-04-25T14:12:51.441623+0000","last_became_active":"2024-04-25T14:11:29.566269+0000","last_became_peered":"2024-04-25T14:11:29.566269+0000","last_unstale":"2024-04-25T14:12:51.441623+0000","last_undegraded":"2024-04-25T14:12:51.441623+0000","last_fullsized":"2024-04-25T14:12:51.441623+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:31.009318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"78'7","reported_seq":149,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:13:29.686280+0000","last_change":"2024-04-25T14:11:28.546215+0000","last_active":"2024-04-25T14:13:29.686280+0000","last_peered":"2024-04-25T14:13:29.686280+0000","last_clean":"2024-04-25T14:13:29.686280+0000","last_became_active":"2024-04-25T14:11:28.545828+0000","last_became_peered":"2024-04-25T14:11:28.545828+0000","last_unstale":"2024-04-25T14:13:29.686280+0000","last_undegraded":"2024-04-25T14:13:29.686280+0000","last_fullsized":"2024-04-25T14:13:29.686280+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:27.152582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":123,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"78'7","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.422934+0000","last_change":"2024-04-25T14:11:29.559566+0000","last_active":"2024-04-25T14:12:51.422934+0000","last_peered":"2024-04-25T14:12:51.422934+0000","last_clean":"2024-04-25T14:12:51.422934+0000","last_became_active":"2024-04-25T14:11:29.558560+0000","last_became_peered":"2024-04-25T14:11:29.558560+0000","last_unstale":"2024-04-25T14:12:51.422934+0000","last_undegraded":"2024-04-25T14:12:51.422934+0000","last_fullsized":"2024-04-25T14:12:51.422934+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:23.677099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"76'7","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.515784+0000","last_change":"2024-04-25T14:11:28.544020+0000","last_active":"2024-04-25T14:12:22.515784+0000","last_peered":"2024-04-25T14:12:22.515784+0000","last_clean":"2024-04-25T14:12:22.515784+0000","last_became_active":"2024-04-25T14:11:28.543598+0000","last_became_peered":"2024-04-25T14:11:28.543598+0000","last_unstale":"2024-04-25T14:12:22.515784+0000","last_undegraded":"2024-04-25T14:12:22.515784+0000","last_fullsized":"2024-04-25T14:12:22.515784+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:49:06.543876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"78'8","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.438175+0000","last_change":"2024-04-25T14:11:29.576122+0000","last_active":"2024-04-25T14:12:51.438175+0000","last_peered":"2024-04-25T14:12:51.438175+0000","last_clean":"2024-04-25T14:12:51.438175+0000","last_became_active":"2024-04-25T14:11:29.573668+0000","last_became_peered":"2024-04-25T14:11:29.573668+0000","last_unstale":"2024-04-25T14:12:51.438175+0000","last_undegraded":"2024-04-25T14:12:51.438175+0000","last_fullsized":"2024-04-25T14:12:51.438175+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:06.998876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"76'3","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.505016+0000","last_change":"2024-04-25T14:11:28.541554+0000","last_active":"2024-04-25T14:12:22.505016+0000","last_peered":"2024-04-25T14:12:22.505016+0000","last_clean":"2024-04-25T14:12:22.505016+0000","last_became_active":"2024-04-25T14:11:28.541062+0000","last_became_peered":"2024-04-25T14:11:28.541062+0000","last_unstale":"2024-04-25T14:12:22.505016+0000","last_undegraded":"2024-04-25T14:12:22.505016+0000","last_fullsized":"2024-04-25T14:12:22.505016+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:16.719820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"78'8","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.433513+0000","last_change":"2024-04-25T14:11:29.564550+0000","last_active":"2024-04-25T14:12:51.433513+0000","last_peered":"2024-04-25T14:12:51.433513+0000","last_clean":"2024-04-25T14:12:51.433513+0000","last_became_active":"2024-04-25T14:11:29.564193+0000","last_became_peered":"2024-04-25T14:11:29.564193+0000","last_unstale":"2024-04-25T14:12:51.433513+0000","last_undegraded":"2024-04-25T14:12:51.433513+0000","last_fullsized":"2024-04-25T14:12:51.433513+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:59.357327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"76'3","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.508406+0000","last_change":"2024-04-25T14:11:28.540504+0000","last_active":"2024-04-25T14:12:22.508406+0000","last_peered":"2024-04-25T14:12:22.508406+0000","last_clean":"2024-04-25T14:12:22.508406+0000","last_became_active":"2024-04-25T14:11:28.540150+0000","last_became_peered":"2024-04-25T14:11:28.540150+0000","last_unstale":"2024-04-25T14:12:22.508406+0000","last_undegraded":"2024-04-25T14:12:22.508406+0000","last_fullsized":"2024-04-25T14:12:22.508406+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:35.331489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"78'11","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.430603+0000","last_change":"2024-04-25T14:11:29.561226+0000","last_active":"2024-04-25T14:12:51.430603+0000","last_peered":"2024-04-25T14:12:51.430603+0000","last_clean":"2024-04-25T14:12:51.430603+0000","last_became_active":"2024-04-25T14:11:29.560847+0000","last_became_peered":"2024-04-25T14:11:29.560847+0000","last_unstale":"2024-04-25T14:12:51.430603+0000","last_undegraded":"2024-04-25T14:12:51.430603+0000","last_fullsized":"2024-04-25T14:12:51.430603+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:05.299388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'5","reported_seq":259,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:13:30.687104+0000","last_change":"2024-04-25T14:11:28.548580+0000","last_active":"2024-04-25T14:13:30.687104+0000","last_peered":"2024-04-25T14:13:30.687104+0000","last_clean":"2024-04-25T14:13:30.687104+0000","last_became_active":"2024-04-25T14:11:28.548250+0000","last_became_peered":"2024-04-25T14:11:28.548250+0000","last_unstale":"2024-04-25T14:13:30.687104+0000","last_undegraded":"2024-04-25T14:13:30.687104+0000","last_fullsized":"2024-04-25T14:13:30.687104+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:01.166285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":235,"num_read_kb":235,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"78'10","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.455188+0000","last_change":"2024-04-25T14:11:29.575983+0000","last_active":"2024-04-25T14:12:51.455188+0000","last_peered":"2024-04-25T14:12:51.455188+0000","last_clean":"2024-04-25T14:12:51.455188+0000","last_became_active":"2024-04-25T14:11:29.575147+0000","last_became_peered":"2024-04-25T14:11:29.575147+0000","last_unstale":"2024-04-25T14:12:51.455188+0000","last_undegraded":"2024-04-25T14:12:51.455188+0000","last_fullsized":"2024-04-25T14:12:51.455188+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:40:45.366736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"76'4","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.530419+0000","last_change":"2024-04-25T14:12:22.530419+0000","last_active":"2024-04-25T14:12:22.530419+0000","last_peered":"2024-04-25T14:12:22.530419+0000","last_clean":"2024-04-25T14:12:22.530419+0000","last_became_active":"2024-04-25T14:12:22.514999+0000","last_became_peered":"2024-04-25T14:12:22.514999+0000","last_unstale":"2024-04-25T14:12:22.530419+0000","last_undegraded":"2024-04-25T14:12:22.530419+0000","last_fullsized":"2024-04-25T14:12:22.530419+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:02.075270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"78'9","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.424732+0000","last_change":"2024-04-25T14:11:29.581424+0000","last_active":"2024-04-25T14:12:51.424732+0000","last_peered":"2024-04-25T14:12:51.424732+0000","last_clean":"2024-04-25T14:12:51.424732+0000","last_became_active":"2024-04-25T14:11:29.581086+0000","last_became_peered":"2024-04-25T14:11:29.581086+0000","last_unstale":"2024-04-25T14:12:51.424732+0000","last_undegraded":"2024-04-25T14:12:51.424732+0000","last_fullsized":"2024-04-25T14:12:51.424732+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:30.468728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"76'5","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.508977+0000","last_change":"2024-04-25T14:11:28.540771+0000","last_active":"2024-04-25T14:12:22.508977+0000","last_peered":"2024-04-25T14:12:22.508977+0000","last_clean":"2024-04-25T14:12:22.508977+0000","last_became_active":"2024-04-25T14:11:28.540442+0000","last_became_peered":"2024-04-25T14:11:28.540442+0000","last_unstale":"2024-04-25T14:12:22.508977+0000","last_undegraded":"2024-04-25T14:12:22.508977+0000","last_fullsized":"2024-04-25T14:12:22.508977+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:52.479423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"78'11","reported_seq":34,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.458541+0000","last_change":"2024-04-25T14:12:22.517721+0000","last_active":"2024-04-25T14:12:51.458541+0000","last_peered":"2024-04-25T14:12:51.458541+0000","last_clean":"2024-04-25T14:12:51.458541+0000","last_became_active":"2024-04-25T14:12:22.517232+0000","last_became_peered":"2024-04-25T14:12:22.517232+0000","last_unstale":"2024-04-25T14:12:51.458541+0000","last_undegraded":"2024-04-25T14:12:51.458541+0000","last_fullsized":"2024-04-25T14:12:51.458541+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:46.150626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"78'32","reported_seq":297,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:13:30.686843+0000","last_change":"2024-04-25T14:11:28.543093+0000","last_active":"2024-04-25T14:13:30.686843+0000","last_peered":"2024-04-25T14:13:30.686843+0000","last_clean":"2024-04-25T14:13:30.686843+0000","last_became_active":"2024-04-25T14:11:28.542407+0000","last_became_peered":"2024-04-25T14:11:28.542407+0000","last_unstale":"2024-04-25T14:13:30.686843+0000","last_undegraded":"2024-04-25T14:13:30.686843+0000","last_fullsized":"2024-04-25T14:13:30.686843+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:43.767375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59643,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":246,"num_read_kb":181,"num_write":34,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"78'6","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.455852+0000","last_change":"2024-04-25T14:11:29.571896+0000","last_active":"2024-04-25T14:12:51.455852+0000","last_peered":"2024-04-25T14:12:51.455852+0000","last_clean":"2024-04-25T14:12:51.455852+0000","last_became_active":"2024-04-25T14:11:29.571537+0000","last_became_peered":"2024-04-25T14:11:29.571537+0000","last_unstale":"2024-04-25T14:12:51.455852+0000","last_undegraded":"2024-04-25T14:12:51.455852+0000","last_fullsized":"2024-04-25T14:12:51.455852+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:32:07.914841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"76'9","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.530152+0000","last_change":"2024-04-25T14:12:22.529692+0000","last_active":"2024-04-25T14:12:22.530152+0000","last_peered":"2024-04-25T14:12:22.530152+0000","last_clean":"2024-04-25T14:12:22.530152+0000","last_became_active":"2024-04-25T14:12:22.517225+0000","last_became_peered":"2024-04-25T14:12:22.517225+0000","last_unstale":"2024-04-25T14:12:22.530152+0000","last_undegraded":"2024-04-25T14:12:22.530152+0000","last_fullsized":"2024-04-25T14:12:22.530152+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:36.610551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"78'9","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.458904+0000","last_change":"2024-04-25T14:11:29.563877+0000","last_active":"2024-04-25T14:12:51.458904+0000","last_peered":"2024-04-25T14:12:51.458904+0000","last_clean":"2024-04-25T14:12:51.458904+0000","last_became_active":"2024-04-25T14:11:29.563535+0000","last_became_peered":"2024-04-25T14:11:29.563535+0000","last_unstale":"2024-04-25T14:12:51.458904+0000","last_undegraded":"2024-04-25T14:12:51.458904+0000","last_fullsized":"2024-04-25T14:12:51.458904+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:07:38.805863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"76'5","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.508526+0000","last_change":"2024-04-25T14:11:28.543787+0000","last_active":"2024-04-25T14:12:22.508526+0000","last_peered":"2024-04-25T14:12:22.508526+0000","last_clean":"2024-04-25T14:12:22.508526+0000","last_became_active":"2024-04-25T14:11:28.540816+0000","last_became_peered":"2024-04-25T14:11:28.540816+0000","last_unstale":"2024-04-25T14:12:22.508526+0000","last_undegraded":"2024-04-25T14:12:22.508526+0000","last_fullsized":"2024-04-25T14:12:22.508526+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:18.776280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"78'15","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.462197+0000","last_change":"2024-04-25T14:12:22.514028+0000","last_active":"2024-04-25T14:12:51.462197+0000","last_peered":"2024-04-25T14:12:51.462197+0000","last_clean":"2024-04-25T14:12:51.462197+0000","last_became_active":"2024-04-25T14:12:22.513536+0000","last_became_peered":"2024-04-25T14:12:22.513536+0000","last_unstale":"2024-04-25T14:12:51.462197+0000","last_undegraded":"2024-04-25T14:12:51.462197+0000","last_fullsized":"2024-04-25T14:12:51.462197+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:31.921284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'7","reported_seq":145,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:13:29.702488+0000","last_change":"2024-04-25T14:11:28.537450+0000","last_active":"2024-04-25T14:13:29.702488+0000","last_peered":"2024-04-25T14:13:29.702488+0000","last_clean":"2024-04-25T14:13:29.702488+0000","last_became_active":"2024-04-25T14:11:28.537088+0000","last_became_peered":"2024-04-25T14:11:28.537088+0000","last_unstale":"2024-04-25T14:13:29.702488+0000","last_undegraded":"2024-04-25T14:13:29.702488+0000","last_fullsized":"2024-04-25T14:13:29.702488+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:05.779706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":120,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"78'6","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.451486+0000","last_change":"2024-04-25T14:11:29.572789+0000","last_active":"2024-04-25T14:12:51.451486+0000","last_peered":"2024-04-25T14:12:51.451486+0000","last_clean":"2024-04-25T14:12:51.451486+0000","last_became_active":"2024-04-25T14:11:29.572201+0000","last_became_peered":"2024-04-25T14:11:29.572201+0000","last_unstale":"2024-04-25T14:12:51.451486+0000","last_undegraded":"2024-04-25T14:12:51.451486+0000","last_fullsized":"2024-04-25T14:12:51.451486+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:39:58.237070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"76'17","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514338+0000","last_change":"2024-04-25T14:11:28.540577+0000","last_active":"2024-04-25T14:12:22.514338+0000","last_peered":"2024-04-25T14:12:22.514338+0000","last_clean":"2024-04-25T14:12:22.514338+0000","last_became_active":"2024-04-25T14:11:28.540212+0000","last_became_peered":"2024-04-25T14:11:28.540212+0000","last_unstale":"2024-04-25T14:12:22.514338+0000","last_undegraded":"2024-04-25T14:12:22.514338+0000","last_fullsized":"2024-04-25T14:12:22.514338+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:31.422332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"78'6","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.437799+0000","last_change":"2024-04-25T14:11:29.574952+0000","last_active":"2024-04-25T14:12:51.437799+0000","last_peered":"2024-04-25T14:12:51.437799+0000","last_clean":"2024-04-25T14:12:51.437799+0000","last_became_active":"2024-04-25T14:11:29.573677+0000","last_became_peered":"2024-04-25T14:11:29.573677+0000","last_unstale":"2024-04-25T14:12:51.437799+0000","last_undegraded":"2024-04-25T14:12:51.437799+0000","last_fullsized":"2024-04-25T14:12:51.437799+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:35.896103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"76'2","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514622+0000","last_change":"2024-04-25T14:11:28.541356+0000","last_active":"2024-04-25T14:12:22.514622+0000","last_peered":"2024-04-25T14:12:22.514622+0000","last_clean":"2024-04-25T14:12:22.514622+0000","last_became_active":"2024-04-25T14:11:28.540976+0000","last_became_peered":"2024-04-25T14:11:28.540976+0000","last_unstale":"2024-04-25T14:12:22.514622+0000","last_undegraded":"2024-04-25T14:12:22.514622+0000","last_fullsized":"2024-04-25T14:12:22.514622+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:32:50.599900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"78'5","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.442075+0000","last_change":"2024-04-25T14:11:29.554317+0000","last_active":"2024-04-25T14:12:51.442075+0000","last_peered":"2024-04-25T14:12:51.442075+0000","last_clean":"2024-04-25T14:12:51.442075+0000","last_became_active":"2024-04-25T14:11:29.553935+0000","last_became_peered":"2024-04-25T14:11:29.553935+0000","last_unstale":"2024-04-25T14:12:51.442075+0000","last_undegraded":"2024-04-25T14:12:51.442075+0000","last_fullsized":"2024-04-25T14:12:51.442075+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:01.405964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"76'1","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.509102+0000","last_change":"2024-04-25T14:11:28.546892+0000","last_active":"2024-04-25T14:12:22.509102+0000","last_peered":"2024-04-25T14:12:22.509102+0000","last_clean":"2024-04-25T14:12:22.509102+0000","last_became_active":"2024-04-25T14:11:28.546566+0000","last_became_peered":"2024-04-25T14:11:28.546566+0000","last_unstale":"2024-04-25T14:12:22.509102+0000","last_undegraded":"2024-04-25T14:12:22.509102+0000","last_fullsized":"2024-04-25T14:12:22.509102+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:05.239635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'5","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.431262+0000","last_change":"2024-04-25T14:11:29.565083+0000","last_active":"2024-04-25T14:12:51.431262+0000","last_peered":"2024-04-25T14:12:51.431262+0000","last_clean":"2024-04-25T14:12:51.431262+0000","last_became_active":"2024-04-25T14:11:29.564062+0000","last_became_peered":"2024-04-25T14:11:29.564062+0000","last_unstale":"2024-04-25T14:12:51.431262+0000","last_undegraded":"2024-04-25T14:12:51.431262+0000","last_fullsized":"2024-04-25T14:12:51.431262+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:07:10.664113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"76'5","reported_seq":140,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:13:29.608884+0000","last_change":"2024-04-25T14:11:28.538649+0000","last_active":"2024-04-25T14:13:29.608884+0000","last_peered":"2024-04-25T14:13:29.608884+0000","last_clean":"2024-04-25T14:13:29.608884+0000","last_became_active":"2024-04-25T14:11:28.538353+0000","last_became_peered":"2024-04-25T14:11:28.538353+0000","last_unstale":"2024-04-25T14:13:29.608884+0000","last_undegraded":"2024-04-25T14:13:29.608884+0000","last_fullsized":"2024-04-25T14:13:29.608884+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:06.513039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"78'7","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.435524+0000","last_change":"2024-04-25T14:11:29.576263+0000","last_active":"2024-04-25T14:12:51.435524+0000","last_peered":"2024-04-25T14:12:51.435524+0000","last_clean":"2024-04-25T14:12:51.435524+0000","last_became_active":"2024-04-25T14:11:29.575877+0000","last_became_peered":"2024-04-25T14:11:29.575877+0000","last_unstale":"2024-04-25T14:12:51.435524+0000","last_undegraded":"2024-04-25T14:12:51.435524+0000","last_fullsized":"2024-04-25T14:12:51.435524+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:33:37.639537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"78'8","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.438492+0000","last_change":"2024-04-25T14:11:29.568220+0000","last_active":"2024-04-25T14:12:51.438492+0000","last_peered":"2024-04-25T14:12:51.438492+0000","last_clean":"2024-04-25T14:12:51.438492+0000","last_became_active":"2024-04-25T14:11:29.567741+0000","last_became_peered":"2024-04-25T14:11:29.567741+0000","last_unstale":"2024-04-25T14:12:51.438492+0000","last_undegraded":"2024-04-25T14:12:51.438492+0000","last_fullsized":"2024-04-25T14:12:51.438492+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:05.134466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"78'6","reported_seq":143,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:13:30.703173+0000","last_change":"2024-04-25T14:11:28.537503+0000","last_active":"2024-04-25T14:13:30.703173+0000","last_peered":"2024-04-25T14:13:30.703173+0000","last_clean":"2024-04-25T14:13:30.703173+0000","last_became_active":"2024-04-25T14:11:28.537169+0000","last_became_peered":"2024-04-25T14:11:28.537169+0000","last_unstale":"2024-04-25T14:13:30.703173+0000","last_undegraded":"2024-04-25T14:13:30.703173+0000","last_fullsized":"2024-04-25T14:13:30.703173+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:09.752988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"78'8","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.454448+0000","last_change":"2024-04-25T14:12:22.516273+0000","last_active":"2024-04-25T14:12:51.454448+0000","last_peered":"2024-04-25T14:12:51.454448+0000","last_clean":"2024-04-25T14:12:51.454448+0000","last_became_active":"2024-04-25T14:12:22.515737+0000","last_became_peered":"2024-04-25T14:12:22.515737+0000","last_unstale":"2024-04-25T14:12:51.454448+0000","last_undegraded":"2024-04-25T14:12:51.454448+0000","last_fullsized":"2024-04-25T14:12:51.454448+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:36.241587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"76'6","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.512277+0000","last_change":"2024-04-25T14:11:28.538216+0000","last_active":"2024-04-25T14:12:22.512277+0000","last_peered":"2024-04-25T14:12:22.512277+0000","last_clean":"2024-04-25T14:12:22.512277+0000","last_became_active":"2024-04-25T14:11:28.537772+0000","last_became_peered":"2024-04-25T14:11:28.537772+0000","last_unstale":"2024-04-25T14:12:22.512277+0000","last_undegraded":"2024-04-25T14:12:22.512277+0000","last_fullsized":"2024-04-25T14:12:22.512277+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:55.460966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":94,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.509039+0000","last_change":"2024-04-25T14:10:12.194216+0000","last_active":"2024-04-25T14:12:22.509039+0000","last_peered":"2024-04-25T14:12:22.509039+0000","last_clean":"2024-04-25T14:12:22.509039+0000","last_became_active":"2024-04-25T14:10:12.189616+0000","last_became_peered":"2024-04-25T14:10:12.189616+0000","last_unstale":"2024-04-25T14:12:22.509039+0000","last_undegraded":"2024-04-25T14:12:22.509039+0000","last_fullsized":"2024-04-25T14:12:22.509039+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:47.553264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:47.553264+0000","last_clean_scrub_stamp":"2024-04-25T14:06:47.553264+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:47.576058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"78'5","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.433764+0000","last_change":"2024-04-25T14:11:29.574985+0000","last_active":"2024-04-25T14:12:51.433764+0000","last_peered":"2024-04-25T14:12:51.433764+0000","last_clean":"2024-04-25T14:12:51.433764+0000","last_became_active":"2024-04-25T14:11:29.574629+0000","last_became_peered":"2024-04-25T14:11:29.574629+0000","last_unstale":"2024-04-25T14:12:51.433764+0000","last_undegraded":"2024-04-25T14:12:51.433764+0000","last_fullsized":"2024-04-25T14:12:51.433764+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:01.899889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"76'4","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.520398+0000","last_change":"2024-04-25T14:12:22.519851+0000","last_active":"2024-04-25T14:12:22.520398+0000","last_peered":"2024-04-25T14:12:22.520398+0000","last_clean":"2024-04-25T14:12:22.520398+0000","last_became_active":"2024-04-25T14:12:22.515791+0000","last_became_peered":"2024-04-25T14:12:22.515791+0000","last_unstale":"2024-04-25T14:12:22.520398+0000","last_undegraded":"2024-04-25T14:12:22.520398+0000","last_fullsized":"2024-04-25T14:12:22.520398+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:47:01.242259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"76'4","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514464+0000","last_change":"2024-04-25T14:11:28.544269+0000","last_active":"2024-04-25T14:12:22.514464+0000","last_peered":"2024-04-25T14:12:22.514464+0000","last_clean":"2024-04-25T14:12:22.514464+0000","last_became_active":"2024-04-25T14:11:28.543897+0000","last_became_peered":"2024-04-25T14:11:28.543897+0000","last_unstale":"2024-04-25T14:12:22.514464+0000","last_undegraded":"2024-04-25T14:12:22.514464+0000","last_fullsized":"2024-04-25T14:12:22.514464+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:37.668187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"78'9","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.431345+0000","last_change":"2024-04-25T14:11:29.566428+0000","last_active":"2024-04-25T14:12:51.431345+0000","last_peered":"2024-04-25T14:12:51.431345+0000","last_clean":"2024-04-25T14:12:51.431345+0000","last_became_active":"2024-04-25T14:11:29.562598+0000","last_became_peered":"2024-04-25T14:11:29.562598+0000","last_unstale":"2024-04-25T14:12:51.431345+0000","last_undegraded":"2024-04-25T14:12:51.431345+0000","last_fullsized":"2024-04-25T14:12:51.431345+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:46.611359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"78'14","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.561086+0000","last_change":"2024-04-25T14:11:28.542571+0000","last_active":"2024-04-25T14:12:51.561086+0000","last_peered":"2024-04-25T14:12:51.561086+0000","last_clean":"2024-04-25T14:12:51.561086+0000","last_became_active":"2024-04-25T14:11:28.542272+0000","last_became_peered":"2024-04-25T14:11:28.542272+0000","last_unstale":"2024-04-25T14:12:51.561086+0000","last_undegraded":"2024-04-25T14:12:51.561086+0000","last_fullsized":"2024-04-25T14:12:51.561086+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:41:05.128068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"78'5","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.458486+0000","last_change":"2024-04-25T14:11:29.578565+0000","last_active":"2024-04-25T14:12:51.458486+0000","last_peered":"2024-04-25T14:12:51.458486+0000","last_clean":"2024-04-25T14:12:51.458486+0000","last_became_active":"2024-04-25T14:11:29.576782+0000","last_became_peered":"2024-04-25T14:11:29.576782+0000","last_unstale":"2024-04-25T14:12:51.458486+0000","last_undegraded":"2024-04-25T14:12:51.458486+0000","last_fullsized":"2024-04-25T14:12:51.458486+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:31.634602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"76'7","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.511746+0000","last_change":"2024-04-25T14:11:28.543838+0000","last_active":"2024-04-25T14:12:22.511746+0000","last_peered":"2024-04-25T14:12:22.511746+0000","last_clean":"2024-04-25T14:12:22.511746+0000","last_became_active":"2024-04-25T14:11:28.543511+0000","last_became_peered":"2024-04-25T14:11:28.543511+0000","last_unstale":"2024-04-25T14:12:22.511746+0000","last_undegraded":"2024-04-25T14:12:22.511746+0000","last_fullsized":"2024-04-25T14:12:22.511746+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:34.440234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"78'7","reported_seq":30,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.426925+0000","last_change":"2024-04-25T14:12:22.516416+0000","last_active":"2024-04-25T14:12:51.426925+0000","last_peered":"2024-04-25T14:12:51.426925+0000","last_clean":"2024-04-25T14:12:51.426925+0000","last_became_active":"2024-04-25T14:12:22.515928+0000","last_became_peered":"2024-04-25T14:12:22.515928+0000","last_unstale":"2024-04-25T14:12:51.426925+0000","last_undegraded":"2024-04-25T14:12:51.426925+0000","last_fullsized":"2024-04-25T14:12:51.426925+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:26.989293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"76'2","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.512719+0000","last_change":"2024-04-25T14:11:28.539443+0000","last_active":"2024-04-25T14:12:22.512719+0000","last_peered":"2024-04-25T14:12:22.512719+0000","last_clean":"2024-04-25T14:12:22.512719+0000","last_became_active":"2024-04-25T14:11:28.539113+0000","last_became_peered":"2024-04-25T14:11:28.539113+0000","last_unstale":"2024-04-25T14:12:22.512719+0000","last_undegraded":"2024-04-25T14:12:22.512719+0000","last_fullsized":"2024-04-25T14:12:22.512719+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:36.664972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"78'2","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.415389+0000","last_change":"2024-04-25T14:11:29.575451+0000","last_active":"2024-04-25T14:12:51.415389+0000","last_peered":"2024-04-25T14:12:51.415389+0000","last_clean":"2024-04-25T14:12:51.415389+0000","last_became_active":"2024-04-25T14:11:29.574983+0000","last_became_peered":"2024-04-25T14:11:29.574983+0000","last_unstale":"2024-04-25T14:12:51.415389+0000","last_undegraded":"2024-04-25T14:12:51.415389+0000","last_fullsized":"2024-04-25T14:12:51.415389+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:53.317608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"76'7","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514234+0000","last_change":"2024-04-25T14:11:28.546394+0000","last_active":"2024-04-25T14:12:22.514234+0000","last_peered":"2024-04-25T14:12:22.514234+0000","last_clean":"2024-04-25T14:12:22.514234+0000","last_became_active":"2024-04-25T14:11:28.546064+0000","last_became_peered":"2024-04-25T14:11:28.546064+0000","last_unstale":"2024-04-25T14:12:22.514234+0000","last_undegraded":"2024-04-25T14:12:22.514234+0000","last_fullsized":"2024-04-25T14:12:22.514234+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:15.301492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"78'9","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.439570+0000","last_change":"2024-04-25T14:11:29.569584+0000","last_active":"2024-04-25T14:12:51.439570+0000","last_peered":"2024-04-25T14:12:51.439570+0000","last_clean":"2024-04-25T14:12:51.439570+0000","last_became_active":"2024-04-25T14:11:29.569217+0000","last_became_peered":"2024-04-25T14:11:29.569217+0000","last_unstale":"2024-04-25T14:12:51.439570+0000","last_undegraded":"2024-04-25T14:12:51.439570+0000","last_fullsized":"2024-04-25T14:12:51.439570+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:05.708277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"76'3","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.512047+0000","last_change":"2024-04-25T14:11:28.544399+0000","last_active":"2024-04-25T14:12:22.512047+0000","last_peered":"2024-04-25T14:12:22.512047+0000","last_clean":"2024-04-25T14:12:22.512047+0000","last_became_active":"2024-04-25T14:11:28.544098+0000","last_became_peered":"2024-04-25T14:11:28.544098+0000","last_unstale":"2024-04-25T14:12:22.512047+0000","last_undegraded":"2024-04-25T14:12:22.512047+0000","last_fullsized":"2024-04-25T14:12:22.512047+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:05.148386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"78'8","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.430478+0000","last_change":"2024-04-25T14:12:22.517727+0000","last_active":"2024-04-25T14:12:51.430478+0000","last_peered":"2024-04-25T14:12:51.430478+0000","last_clean":"2024-04-25T14:12:51.430478+0000","last_became_active":"2024-04-25T14:12:22.517364+0000","last_became_peered":"2024-04-25T14:12:22.517364+0000","last_unstale":"2024-04-25T14:12:51.430478+0000","last_undegraded":"2024-04-25T14:12:51.430478+0000","last_fullsized":"2024-04-25T14:12:51.430478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:30:11.023308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"76'4","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.514211+0000","last_change":"2024-04-25T14:11:28.542488+0000","last_active":"2024-04-25T14:12:22.514211+0000","last_peered":"2024-04-25T14:12:22.514211+0000","last_clean":"2024-04-25T14:12:22.514211+0000","last_became_active":"2024-04-25T14:11:28.542171+0000","last_became_peered":"2024-04-25T14:11:28.542171+0000","last_unstale":"2024-04-25T14:12:22.514211+0000","last_undegraded":"2024-04-25T14:12:22.514211+0000","last_fullsized":"2024-04-25T14:12:22.514211+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:48.201762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"78'5","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.459752+0000","last_change":"2024-04-25T14:11:29.573805+0000","last_active":"2024-04-25T14:12:51.459752+0000","last_peered":"2024-04-25T14:12:51.459752+0000","last_clean":"2024-04-25T14:12:51.459752+0000","last_became_active":"2024-04-25T14:11:29.573420+0000","last_became_peered":"2024-04-25T14:11:29.573420+0000","last_unstale":"2024-04-25T14:12:51.459752+0000","last_undegraded":"2024-04-25T14:12:51.459752+0000","last_fullsized":"2024-04-25T14:12:51.459752+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:18.721977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"76'5","reported_seq":141,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:13:28.701588+0000","last_change":"2024-04-25T14:11:28.552863+0000","last_active":"2024-04-25T14:13:28.701588+0000","last_peered":"2024-04-25T14:13:28.701588+0000","last_clean":"2024-04-25T14:13:28.701588+0000","last_became_active":"2024-04-25T14:11:28.552461+0000","last_became_peered":"2024-04-25T14:11:28.552461+0000","last_unstale":"2024-04-25T14:13:28.701588+0000","last_undegraded":"2024-04-25T14:13:28.701588+0000","last_fullsized":"2024-04-25T14:13:28.701588+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:46.264473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":117,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"78'7","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.459450+0000","last_change":"2024-04-25T14:11:29.571268+0000","last_active":"2024-04-25T14:12:51.459450+0000","last_peered":"2024-04-25T14:12:51.459450+0000","last_clean":"2024-04-25T14:12:51.459450+0000","last_became_active":"2024-04-25T14:11:29.570876+0000","last_became_peered":"2024-04-25T14:11:29.570876+0000","last_unstale":"2024-04-25T14:12:51.459450+0000","last_undegraded":"2024-04-25T14:12:51.459450+0000","last_fullsized":"2024-04-25T14:12:51.459450+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:15.093635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"76'3","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.508764+0000","last_change":"2024-04-25T14:11:28.543343+0000","last_active":"2024-04-25T14:12:22.508764+0000","last_peered":"2024-04-25T14:12:22.508764+0000","last_clean":"2024-04-25T14:12:22.508764+0000","last_became_active":"2024-04-25T14:11:28.540790+0000","last_became_peered":"2024-04-25T14:11:28.540790+0000","last_unstale":"2024-04-25T14:12:22.508764+0000","last_undegraded":"2024-04-25T14:12:22.508764+0000","last_fullsized":"2024-04-25T14:12:22.508764+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:20.498016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'7","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.458962+0000","last_change":"2024-04-25T14:11:29.566161+0000","last_active":"2024-04-25T14:12:51.458962+0000","last_peered":"2024-04-25T14:12:51.458962+0000","last_clean":"2024-04-25T14:12:51.458962+0000","last_became_active":"2024-04-25T14:11:29.564197+0000","last_became_peered":"2024-04-25T14:11:29.564197+0000","last_unstale":"2024-04-25T14:12:51.458962+0000","last_undegraded":"2024-04-25T14:12:51.458962+0000","last_fullsized":"2024-04-25T14:12:51.458962+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:35.291546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"76'5","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:22.505214+0000","last_change":"2024-04-25T14:11:28.543479+0000","last_active":"2024-04-25T14:12:22.505214+0000","last_peered":"2024-04-25T14:12:22.505214+0000","last_clean":"2024-04-25T14:12:22.505214+0000","last_became_active":"2024-04-25T14:11:28.543167+0000","last_became_peered":"2024-04-25T14:11:28.543167+0000","last_unstale":"2024-04-25T14:12:22.505214+0000","last_undegraded":"2024-04-25T14:12:22.505214+0000","last_fullsized":"2024-04-25T14:12:22.505214+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:27.513595+0000","last_clean_scrub_stamp":"2024-04-25T14:11:27.513595+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:54:53.572809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"78'9","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.455482+0000","last_change":"2024-04-25T14:11:29.572548+0000","last_active":"2024-04-25T14:12:51.455482+0000","last_peered":"2024-04-25T14:12:51.455482+0000","last_clean":"2024-04-25T14:12:51.455482+0000","last_became_active":"2024-04-25T14:11:29.572167+0000","last_became_peered":"2024-04-25T14:11:29.572167+0000","last_unstale":"2024-04-25T14:12:51.455482+0000","last_undegraded":"2024-04-25T14:12:51.455482+0000","last_fullsized":"2024-04-25T14:12:51.455482+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:38.423331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"78'9","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.440678+0000","last_change":"2024-04-25T14:11:29.560956+0000","last_active":"2024-04-25T14:12:51.440678+0000","last_peered":"2024-04-25T14:12:51.440678+0000","last_clean":"2024-04-25T14:12:51.440678+0000","last_became_active":"2024-04-25T14:11:29.560519+0000","last_became_peered":"2024-04-25T14:11:29.560519+0000","last_unstale":"2024-04-25T14:12:51.440678+0000","last_undegraded":"2024-04-25T14:12:51.440678+0000","last_fullsized":"2024-04-25T14:12:51.440678+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:53:47.677923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"78'12","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.435167+0000","last_change":"2024-04-25T14:11:29.574372+0000","last_active":"2024-04-25T14:12:51.435167+0000","last_peered":"2024-04-25T14:12:51.435167+0000","last_clean":"2024-04-25T14:12:51.435167+0000","last_became_active":"2024-04-25T14:11:29.573995+0000","last_became_peered":"2024-04-25T14:11:29.573995+0000","last_unstale":"2024-04-25T14:12:51.435167+0000","last_undegraded":"2024-04-25T14:12:51.435167+0000","last_fullsized":"2024-04-25T14:12:51.435167+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:12.381512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"78'8","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.432226+0000","last_change":"2024-04-25T14:11:29.578324+0000","last_active":"2024-04-25T14:12:51.432226+0000","last_peered":"2024-04-25T14:12:51.432226+0000","last_clean":"2024-04-25T14:12:51.432226+0000","last_became_active":"2024-04-25T14:11:29.577917+0000","last_became_peered":"2024-04-25T14:11:29.577917+0000","last_unstale":"2024-04-25T14:12:51.432226+0000","last_undegraded":"2024-04-25T14:12:51.432226+0000","last_fullsized":"2024-04-25T14:12:51.432226+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:20:27.070720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"78'7","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.432298+0000","last_change":"2024-04-25T14:11:29.578879+0000","last_active":"2024-04-25T14:12:51.432298+0000","last_peered":"2024-04-25T14:12:51.432298+0000","last_clean":"2024-04-25T14:12:51.432298+0000","last_became_active":"2024-04-25T14:11:29.578551+0000","last_became_peered":"2024-04-25T14:11:29.578551+0000","last_unstale":"2024-04-25T14:12:51.432298+0000","last_undegraded":"2024-04-25T14:12:51.432298+0000","last_fullsized":"2024-04-25T14:12:51.432298+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:41.083794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"78'4","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.437936+0000","last_change":"2024-04-25T14:11:29.574054+0000","last_active":"2024-04-25T14:12:51.437936+0000","last_peered":"2024-04-25T14:12:51.437936+0000","last_clean":"2024-04-25T14:12:51.437936+0000","last_became_active":"2024-04-25T14:11:29.573700+0000","last_became_peered":"2024-04-25T14:11:29.573700+0000","last_unstale":"2024-04-25T14:12:51.437936+0000","last_undegraded":"2024-04-25T14:12:51.437936+0000","last_fullsized":"2024-04-25T14:12:51.437936+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:23.304580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"78'6","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.440195+0000","last_change":"2024-04-25T14:11:29.553768+0000","last_active":"2024-04-25T14:12:51.440195+0000","last_peered":"2024-04-25T14:12:51.440195+0000","last_clean":"2024-04-25T14:12:51.440195+0000","last_became_active":"2024-04-25T14:11:29.553376+0000","last_became_peered":"2024-04-25T14:11:29.553376+0000","last_unstale":"2024-04-25T14:12:51.440195+0000","last_undegraded":"2024-04-25T14:12:51.440195+0000","last_fullsized":"2024-04-25T14:12:51.440195+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:46.688368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"78'15","reported_seq":32,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.440586+0000","last_change":"2024-04-25T14:11:29.563122+0000","last_active":"2024-04-25T14:12:51.440586+0000","last_peered":"2024-04-25T14:12:51.440586+0000","last_clean":"2024-04-25T14:12:51.440586+0000","last_became_active":"2024-04-25T14:11:29.562717+0000","last_became_peered":"2024-04-25T14:11:29.562717+0000","last_unstale":"2024-04-25T14:12:51.440586+0000","last_undegraded":"2024-04-25T14:12:51.440586+0000","last_fullsized":"2024-04-25T14:12:51.440586+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:37.585352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"78'2","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.436426+0000","last_change":"2024-04-25T14:11:29.576487+0000","last_active":"2024-04-25T14:12:51.436426+0000","last_peered":"2024-04-25T14:12:51.436426+0000","last_clean":"2024-04-25T14:12:51.436426+0000","last_became_active":"2024-04-25T14:11:29.575070+0000","last_became_peered":"2024-04-25T14:11:29.575070+0000","last_unstale":"2024-04-25T14:12:51.436426+0000","last_undegraded":"2024-04-25T14:12:51.436426+0000","last_fullsized":"2024-04-25T14:12:51.436426+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:13.803505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"78'6","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.427743+0000","last_change":"2024-04-25T14:11:29.568733+0000","last_active":"2024-04-25T14:12:51.427743+0000","last_peered":"2024-04-25T14:12:51.427743+0000","last_clean":"2024-04-25T14:12:51.427743+0000","last_became_active":"2024-04-25T14:11:29.567665+0000","last_became_peered":"2024-04-25T14:11:29.567665+0000","last_unstale":"2024-04-25T14:12:51.427743+0000","last_undegraded":"2024-04-25T14:12:51.427743+0000","last_fullsized":"2024-04-25T14:12:51.427743+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:57.956199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"78'8","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.433324+0000","last_change":"2024-04-25T14:11:29.574648+0000","last_active":"2024-04-25T14:12:51.433324+0000","last_peered":"2024-04-25T14:12:51.433324+0000","last_clean":"2024-04-25T14:12:51.433324+0000","last_became_active":"2024-04-25T14:11:29.574247+0000","last_became_peered":"2024-04-25T14:11:29.574247+0000","last_unstale":"2024-04-25T14:12:51.433324+0000","last_undegraded":"2024-04-25T14:12:51.433324+0000","last_fullsized":"2024-04-25T14:12:51.433324+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:57.110017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"78'10","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.458289+0000","last_change":"2024-04-25T14:11:29.573633+0000","last_active":"2024-04-25T14:12:51.458289+0000","last_peered":"2024-04-25T14:12:51.458289+0000","last_clean":"2024-04-25T14:12:51.458289+0000","last_became_active":"2024-04-25T14:11:29.573303+0000","last_became_peered":"2024-04-25T14:11:29.573303+0000","last_unstale":"2024-04-25T14:12:51.458289+0000","last_undegraded":"2024-04-25T14:12:51.458289+0000","last_fullsized":"2024-04-25T14:12:51.458289+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:11.458117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"78'11","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.460396+0000","last_change":"2024-04-25T14:11:29.569136+0000","last_active":"2024-04-25T14:12:51.460396+0000","last_peered":"2024-04-25T14:12:51.460396+0000","last_clean":"2024-04-25T14:12:51.460396+0000","last_became_active":"2024-04-25T14:11:29.568801+0000","last_became_peered":"2024-04-25T14:11:29.568801+0000","last_unstale":"2024-04-25T14:12:51.460396+0000","last_undegraded":"2024-04-25T14:12:51.460396+0000","last_fullsized":"2024-04-25T14:12:51.460396+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:30.969081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"78'8","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T14:12:51.442382+0000","last_change":"2024-04-25T14:11:29.566754+0000","last_active":"2024-04-25T14:12:51.442382+0000","last_peered":"2024-04-25T14:12:51.442382+0000","last_clean":"2024-04-25T14:12:51.442382+0000","last_became_active":"2024-04-25T14:11:29.566368+0000","last_became_peered":"2024-04-25T14:11:29.566368+0000","last_unstale":"2024-04-25T14:12:51.442382+0000","last_undegraded":"2024-04-25T14:12:51.442382+0000","last_fullsized":"2024-04-25T14:12:51.442382+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:28.522260+0000","last_clean_scrub_stamp":"2024-04-25T14:11:28.522260+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:37.057718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281277,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67129,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1209,"num_read_kb":679,"num_write":247,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":740,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":202003,"data_compressed":5550,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678055,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":851968,"data_stored":405341,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874284,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":1056477,"data_compressed":43412,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037809,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":868352,"data_stored":405203,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234039,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32012,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713140,"statfs":{"total":95995035648,"available":95962255360,"internally_reserved":0,"allocated":856064,"data_stored":404909,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430267,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":917504,"data_stored":999080,"data_compressed":41562,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593793,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":851968,"data_stored":404920,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":465784,"data_compressed":7643,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":872448,"data_stored":406259,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":466160,"data_compressed":7643,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149776,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":868352,"data_stored":404745,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953546,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":860160,"data_stored":405985,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790022,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":997918,"data_compressed":41562,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":61703,"data_compressed":1850,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":62079,"data_compressed":1850,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":61911,"data_compressed":1850,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:13:36.635 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:13:36.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.778+0000 7fb662172640 1 -- 172.21.15.96:0/3249393889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb65c108900 msgr2=0x7fb65c10ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:36.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.778+0000 7fb662172640 1 --2- 172.21.15.96:0/3249393889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb65c108900 0x7fb65c10ad20 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb654009900 tx=0x7fb65402a410 comp rx=0 tx=0).stop 2024-04-25T14:13:36.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.778+0000 7fb662172640 1 -- 172.21.15.96:0/3249393889 shutdown_connections 2024-04-25T14:13:36.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.778+0000 7fb662172640 1 --2- 172.21.15.96:0/3249393889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb65c108900 0x7fb65c10ad20 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:36.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.778+0000 7fb662172640 1 --2- 172.21.15.96:0/3249393889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb65c103c00 0x7fb65c104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:36.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.778+0000 7fb662172640 1 --2- 172.21.15.96:0/3249393889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb65c1029c0 0x7fb65c1036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:36.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.778+0000 7fb662172640 1 -- 172.21.15.96:0/3249393889 >> 172.21.15.96:0/3249393889 conn(0x7fb65c0fdbd0 msgr2=0x7fb65c100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:36.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.778+0000 7fb662172640 1 -- 172.21.15.96:0/3249393889 shutdown_connections 2024-04-25T14:13:36.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.778+0000 7fb662172640 1 -- 172.21.15.96:0/3249393889 wait complete. 2024-04-25T14:13:36.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.778+0000 7fb662172640 1 Processor -- start 2024-04-25T14:13:36.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.779+0000 7fb662172640 1 -- start start 2024-04-25T14:13:36.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.779+0000 7fb662172640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb65c1029c0 0x7fb65c06a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:36.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.779+0000 7fb662172640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb65c103c00 0x7fb65c19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:36.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.779+0000 7fb662172640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb65c108900 0x7fb65c1a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:36.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.779+0000 7fb662172640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb65c10e4d0 con 0x7fb65c103c00 2024-04-25T14:13:36.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.779+0000 7fb662172640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb65c10e350 con 0x7fb65c108900 2024-04-25T14:13:36.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.779+0000 7fb662172640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb65c10e650 con 0x7fb65c1029c0 2024-04-25T14:13:36.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.779+0000 7fb65affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb65c103c00 0x7fb65c19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:36.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.779+0000 7fb65affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb65c103c00 0x7fb65c19ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40602/0 (socket says 172.21.15.96:40602) 2024-04-25T14:13:36.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.779+0000 7fb65affd640 1 -- 172.21.15.96:0/219767468 learned_addr learned my addr 172.21.15.96:0/219767468 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:36.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.779+0000 7fb65bfff640 1 --2- 172.21.15.96:0/219767468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb65c108900 0x7fb65c1a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:36.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.779+0000 7fb65b7fe640 1 --2- 172.21.15.96:0/219767468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb65c1029c0 0x7fb65c06a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:36.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.779+0000 7fb65affd640 1 -- 172.21.15.96:0/219767468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb65c1029c0 msgr2=0x7fb65c06a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:36.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.779+0000 7fb65affd640 1 --2- 172.21.15.96:0/219767468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb65c1029c0 0x7fb65c06a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:36.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.779+0000 7fb65affd640 1 -- 172.21.15.96:0/219767468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb65c108900 msgr2=0x7fb65c1a14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:36.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.779+0000 7fb65affd640 1 --2- 172.21.15.96:0/219767468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb65c108900 0x7fb65c1a14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:36.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.779+0000 7fb65affd640 1 -- 172.21.15.96:0/219767468 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb65c1a1cc0 con 0x7fb65c103c00 2024-04-25T14:13:36.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.780+0000 7fb65affd640 1 --2- 172.21.15.96:0/219767468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb65c103c00 0x7fb65c19ef50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb65000d950 tx=0x7fb65000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:36.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.780+0000 7fb658ff9640 1 -- 172.21.15.96:0/219767468 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb650017070 con 0x7fb65c103c00 2024-04-25T14:13:36.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.780+0000 7fb658ff9640 1 -- 172.21.15.96:0/219767468 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb650009c30 con 0x7fb65c103c00 2024-04-25T14:13:36.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.780+0000 7fb662172640 1 -- 172.21.15.96:0/219767468 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb65c1a1fb0 con 0x7fb65c103c00 2024-04-25T14:13:36.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.780+0000 7fb658ff9640 1 -- 172.21.15.96:0/219767468 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65000f620 con 0x7fb65c103c00 2024-04-25T14:13:36.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.780+0000 7fb662172640 1 -- 172.21.15.96:0/219767468 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb65c1a2420 con 0x7fb65c103c00 2024-04-25T14:13:36.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.781+0000 7fb662172640 1 -- 172.21.15.96:0/219767468 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb65c107710 con 0x7fb65c103c00 2024-04-25T14:13:36.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.782+0000 7fb658ff9640 1 -- 172.21.15.96:0/219767468 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fb650010460 con 0x7fb65c103c00 2024-04-25T14:13:36.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.783+0000 7fb658ff9640 1 --2- 172.21.15.96:0/219767468 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb6440afaf0 0x7fb6440b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:36.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.783+0000 7fb65b7fe640 1 --2- 172.21.15.96:0/219767468 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb6440afaf0 0x7fb6440b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:36.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.783+0000 7fb658ff9640 1 -- 172.21.15.96:0/219767468 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fb6500de820 con 0x7fb65c103c00 2024-04-25T14:13:36.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.784+0000 7fb65b7fe640 1 --2- 172.21.15.96:0/219767468 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb6440afaf0 0x7fb6440b2000 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb65c1035c0 tx=0x7fb64c009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:36.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.787+0000 7fb658ff9640 1 -- 172.21.15.96:0/219767468 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb6500a73e0 con 0x7fb65c103c00 2024-04-25T14:13:36.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:36 smithi097 ceph-mon[32720]: from='client.15420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:13:36.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:36 smithi097 ceph-mon[32720]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:13:36.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:36 smithi151 ceph-mon[32990]: from='client.15420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:13:36.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:36 smithi151 ceph-mon[32990]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:13:36.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.964+0000 7fb662172640 1 -- 172.21.15.96:0/219767468 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb65c02ddc0 con 0x7fb65c103c00 2024-04-25T14:13:36.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.967+0000 7fb658ff9640 1 -- 172.21.15.96:0/219767468 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22616 (secure 0 0 0) 0x7fb650009de0 con 0x7fb65c103c00 2024-04-25T14:13:36.966 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:13:36.967 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":78,"fsid":"59334182-030c-11ef-bc93-c7b262605968","created":"2024-04-25T14:03:25.680745+0000","modified":"2024-04-25T14:12:22.496143+0000","last_up_change":"2024-04-25T14:10:35.894312+0000","last_in_change":"2024-04-25T14:10:15.583440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:06:47.402927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:11:27.434331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:11:27.854004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cf4ba3ac-142e-4e8e-96d4-410d77157eb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6803","nonce":3832506168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6805","nonce":3832506168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832506168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6807","nonce":3832506168}]},"public_addr":"172.21.15.96:6803/3832506168","cluster_addr":"172.21.15.96:6805/3832506168","heartbeat_back_addr":"172.21.15.96:6809/3832506168","heartbeat_front_addr":"172.21.15.96:6807/3832506168","state":["exists","up"]},{"osd":1,"uuid":"866e73b9-4a26-48ba-badf-704fe22f71cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6801","nonce":1083311494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6803","nonce":1083311494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6807","nonce":1083311494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6805","nonce":1083311494}]},"public_addr":"172.21.15.97:6801/1083311494","cluster_addr":"172.21.15.97:6803/1083311494","heartbeat_back_addr":"172.21.15.97:6807/1083311494","heartbeat_front_addr":"172.21.15.97:6805/1083311494","state":["exists","up"]},{"osd":2,"uuid":"45c79ecb-abc4-4dba-bae8-74804e4bc1bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6801","nonce":981902966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6803","nonce":981902966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6807","nonce":981902966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6805","nonce":981902966}]},"public_addr":"172.21.15.151:6801/981902966","cluster_addr":"172.21.15.151:6803/981902966","heartbeat_back_addr":"172.21.15.151:6807/981902966","heartbeat_front_addr":"172.21.15.151:6805/981902966","state":["exists","up"]},{"osd":3,"uuid":"9fc39468-a7ea-4ccf-b365-335ad2ea4c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6811","nonce":2274158748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6813","nonce":2274158748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6817","nonce":2274158748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6815","nonce":2274158748}]},"public_addr":"172.21.15.96:6811/2274158748","cluster_addr":"172.21.15.96:6813/2274158748","heartbeat_back_addr":"172.21.15.96:6817/2274158748","heartbeat_front_addr":"172.21.15.96:6815/2274158748","state":["exists","up"]},{"osd":4,"uuid":"616582f1-12d2-456b-b7b3-e5517800e8a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6809","nonce":3727031962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6811","nonce":3727031962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6815","nonce":3727031962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6813","nonce":3727031962}]},"public_addr":"172.21.15.97:6809/3727031962","cluster_addr":"172.21.15.97:6811/3727031962","heartbeat_back_addr":"172.21.15.97:6815/3727031962","heartbeat_front_addr":"172.21.15.97:6813/3727031962","state":["exists","up"]},{"osd":5,"uuid":"3d277784-ab0f-4c90-81a1-2a4034c52a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6809","nonce":223882546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6811","nonce":223882546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6815","nonce":223882546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6813","nonce":223882546}]},"public_addr":"172.21.15.151:6809/223882546","cluster_addr":"172.21.15.151:6811/223882546","heartbeat_back_addr":"172.21.15.151:6815/223882546","heartbeat_front_addr":"172.21.15.151:6813/223882546","state":["exists","up"]},{"osd":6,"uuid":"9a9ad585-cb1a-4239-9d18-055f59164049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6819","nonce":96197407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6821","nonce":96197407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6825","nonce":96197407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6823","nonce":96197407}]},"public_addr":"172.21.15.96:6819/96197407","cluster_addr":"172.21.15.96:6821/96197407","heartbeat_back_addr":"172.21.15.96:6825/96197407","heartbeat_front_addr":"172.21.15.96:6823/96197407","state":["exists","up"]},{"osd":7,"uuid":"303551c8-6af0-42f2-844f-45bf8db5baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6817","nonce":2247216400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6819","nonce":2247216400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6823","nonce":2247216400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6821","nonce":2247216400}]},"public_addr":"172.21.15.97:6817/2247216400","cluster_addr":"172.21.15.97:6819/2247216400","heartbeat_back_addr":"172.21.15.97:6823/2247216400","heartbeat_front_addr":"172.21.15.97:6821/2247216400","state":["exists","up"]},{"osd":8,"uuid":"81f6cc66-e4ca-430c-b336-b6678d968818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6817","nonce":2549822185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6819","nonce":2549822185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6823","nonce":2549822185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6821","nonce":2549822185}]},"public_addr":"172.21.15.151:6817/2549822185","cluster_addr":"172.21.15.151:6819/2549822185","heartbeat_back_addr":"172.21.15.151:6823/2549822185","heartbeat_front_addr":"172.21.15.151:6821/2549822185","state":["exists","up"]},{"osd":9,"uuid":"29d4da16-feab-4d1c-9d1d-de4f799bd724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6827","nonce":1099399643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6829","nonce":1099399643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6833","nonce":1099399643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6831","nonce":1099399643}]},"public_addr":"172.21.15.96:6827/1099399643","cluster_addr":"172.21.15.96:6829/1099399643","heartbeat_back_addr":"172.21.15.96:6833/1099399643","heartbeat_front_addr":"172.21.15.96:6831/1099399643","state":["exists","up"]},{"osd":10,"uuid":"197fee9b-eee4-4893-be1c-3d57f42250e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6825","nonce":155754520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6827","nonce":155754520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6831","nonce":155754520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6829","nonce":155754520}]},"public_addr":"172.21.15.97:6825/155754520","cluster_addr":"172.21.15.97:6827/155754520","heartbeat_back_addr":"172.21.15.97:6831/155754520","heartbeat_front_addr":"172.21.15.97:6829/155754520","state":["exists","up"]},{"osd":11,"uuid":"9c1d0a83-fc12-41a2-8eb0-80c4abdd2770","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6825","nonce":2027958746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6827","nonce":2027958746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6831","nonce":2027958746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6829","nonce":2027958746}]},"public_addr":"172.21.15.151:6825/2027958746","cluster_addr":"172.21.15.151:6827/2027958746","heartbeat_back_addr":"172.21.15.151:6831/2027958746","heartbeat_front_addr":"172.21.15.151:6829/2027958746","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:05:56.177390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:06:19.917663+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:06:43.608052+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:08.540382+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:33.474072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:58.242306+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:08:24.120442+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:08:49.661064+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:09:15.097271+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:09:41.904505+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:10:08.233252+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:10:33.712296+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/3839807484":"2024-04-26T14:04:21.307574+0000","172.21.15.96:0/1495237551":"2024-04-26T14:04:21.307574+0000","172.21.15.96:0/2793751450":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/129575834":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/2233075139":"2024-04-26T14:04:21.307574+0000","172.21.15.96:6801/2348397481":"2024-04-26T14:03:54.249537+0000","172.21.15.96:6801/433591062":"2024-04-26T14:04:21.307574+0000","172.21.15.96:6800/2348397481":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/3517674924":"2024-04-26T14:03:54.249537+0000","172.21.15.96:6800/433591062":"2024-04-26T14:04:21.307574+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:13:36.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.969+0000 7fb662172640 1 -- 172.21.15.96:0/219767468 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb6440afaf0 msgr2=0x7fb6440b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:36.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.969+0000 7fb662172640 1 --2- 172.21.15.96:0/219767468 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb6440afaf0 0x7fb6440b2000 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb65c1035c0 tx=0x7fb64c009870 comp rx=0 tx=0).stop 2024-04-25T14:13:36.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.969+0000 7fb662172640 1 -- 172.21.15.96:0/219767468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb65c103c00 msgr2=0x7fb65c19ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:36.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.969+0000 7fb662172640 1 --2- 172.21.15.96:0/219767468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb65c103c00 0x7fb65c19ef50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb65000d950 tx=0x7fb65000de10 comp rx=0 tx=0).stop 2024-04-25T14:13:36.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.969+0000 7fb662172640 1 -- 172.21.15.96:0/219767468 shutdown_connections 2024-04-25T14:13:36.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.969+0000 7fb662172640 1 --2- 172.21.15.96:0/219767468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb65c108900 0x7fb65c1a14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:36.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.969+0000 7fb662172640 1 --2- 172.21.15.96:0/219767468 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb6440afaf0 0x7fb6440b2000 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:36.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.969+0000 7fb662172640 1 --2- 172.21.15.96:0/219767468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb65c103c00 0x7fb65c19ef50 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:36.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.969+0000 7fb662172640 1 --2- 172.21.15.96:0/219767468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb65c1029c0 0x7fb65c06a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:36.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.969+0000 7fb662172640 1 -- 172.21.15.96:0/219767468 >> 172.21.15.96:0/219767468 conn(0x7fb65c0fdbd0 msgr2=0x7fb65c1048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:36.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.969+0000 7fb662172640 1 -- 172.21.15.96:0/219767468 shutdown_connections 2024-04-25T14:13:36.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:36.969+0000 7fb662172640 1 -- 172.21.15.96:0/219767468 wait complete. 2024-04-25T14:13:36.979 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:36 smithi096 ceph-mon[29207]: from='client.15420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:13:36.979 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:36 smithi096 ceph-mon[29207]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:13:36.980 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:06:47.402927+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:11:27.434331+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:11:27.854004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:13:36.981 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:13:37.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.092+0000 7f715fc61640 1 -- 172.21.15.96:0/3222459482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7158103ae0 msgr2=0x7f71581047e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:37.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.092+0000 7f715fc61640 1 --2- 172.21.15.96:0/3222459482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7158103ae0 0x7f71581047e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7154009900 tx=0x7f715402a410 comp rx=0 tx=0).stop 2024-04-25T14:13:37.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.092+0000 7f715fc61640 1 -- 172.21.15.96:0/3222459482 shutdown_connections 2024-04-25T14:13:37.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.092+0000 7f715fc61640 1 --2- 172.21.15.96:0/3222459482 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7158109a20 0x7f715810be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.092+0000 7f715fc61640 1 --2- 172.21.15.96:0/3222459482 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7158104d20 0x7f7158105130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.092+0000 7f715fc61640 1 --2- 172.21.15.96:0/3222459482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7158103ae0 0x7f71581047e0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.092+0000 7f715fc61640 1 -- 172.21.15.96:0/3222459482 >> 172.21.15.96:0/3222459482 conn(0x7f71580fecd0 msgr2=0x7f7158101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:37.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.092+0000 7f715fc61640 1 -- 172.21.15.96:0/3222459482 shutdown_connections 2024-04-25T14:13:37.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.092+0000 7f715fc61640 1 -- 172.21.15.96:0/3222459482 wait complete. 2024-04-25T14:13:37.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.093+0000 7f715fc61640 1 Processor -- start 2024-04-25T14:13:37.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.093+0000 7f715fc61640 1 -- start start 2024-04-25T14:13:37.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.093+0000 7f715fc61640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7158103ae0 0x7f715819a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:37.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.093+0000 7f715fc61640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7158104d20 0x7f715819a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:37.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.093+0000 7f715fc61640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7158109a20 0x7f71581a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:37.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.093+0000 7f715fc61640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f715810f5c0 con 0x7f7158109a20 2024-04-25T14:13:37.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.093+0000 7f715fc61640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f715810f440 con 0x7f7158103ae0 2024-04-25T14:13:37.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.093+0000 7f715fc61640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f715810f740 con 0x7f7158104d20 2024-04-25T14:13:37.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.093+0000 7f715e1fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7158109a20 0x7f71581a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:37.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.093+0000 7f715e1fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7158109a20 0x7f71581a16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40620/0 (socket says 172.21.15.96:40620) 2024-04-25T14:13:37.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.093+0000 7f715e1fe640 1 -- 172.21.15.96:0/289112696 learned_addr learned my addr 172.21.15.96:0/289112696 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:37.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.093+0000 7f715d9fd640 1 --2- 172.21.15.96:0/289112696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7158103ae0 0x7f715819a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:37.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.093+0000 7f715e1fe640 1 -- 172.21.15.96:0/289112696 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7158104d20 msgr2=0x7f715819a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:37.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.093+0000 7f715d1fc640 1 --2- 172.21.15.96:0/289112696 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7158104d20 0x7f715819a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:37.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.093+0000 7f715e1fe640 1 --2- 172.21.15.96:0/289112696 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7158104d20 0x7f715819a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.093+0000 7f715e1fe640 1 -- 172.21.15.96:0/289112696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7158103ae0 msgr2=0x7f715819a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:37.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.093+0000 7f715e1fe640 1 --2- 172.21.15.96:0/289112696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7158103ae0 0x7f715819a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.093+0000 7f715e1fe640 1 -- 172.21.15.96:0/289112696 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71581a1cb0 con 0x7f7158109a20 2024-04-25T14:13:37.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.093+0000 7f715d9fd640 1 --2- 172.21.15.96:0/289112696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7158103ae0 0x7f715819a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:37.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.094+0000 7f715e1fe640 1 --2- 172.21.15.96:0/289112696 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7158109a20 0x7f71581a16f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f715000eaa0 tx=0x7f715000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:37.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.094+0000 7f714effd640 1 -- 172.21.15.96:0/289112696 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715000cdb0 con 0x7f7158109a20 2024-04-25T14:13:37.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.094+0000 7f714effd640 1 -- 172.21.15.96:0/289112696 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7150005020 con 0x7f7158109a20 2024-04-25T14:13:37.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.094+0000 7f715fc61640 1 -- 172.21.15.96:0/289112696 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71581a1fa0 con 0x7f7158109a20 2024-04-25T14:13:37.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.094+0000 7f714effd640 1 -- 172.21.15.96:0/289112696 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71500116d0 con 0x7f7158109a20 2024-04-25T14:13:37.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.094+0000 7f715fc61640 1 -- 172.21.15.96:0/289112696 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71581a23b0 con 0x7f7158109a20 2024-04-25T14:13:37.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.095+0000 7f715fc61640 1 -- 172.21.15.96:0/289112696 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7158069310 con 0x7f7158109a20 2024-04-25T14:13:37.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.096+0000 7f714effd640 1 -- 172.21.15.96:0/289112696 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f71500051d0 con 0x7f7158109a20 2024-04-25T14:13:37.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.097+0000 7f714effd640 1 --2- 172.21.15.96:0/289112696 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f71440afaf0 0x7f71440b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:37.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.097+0000 7f715d9fd640 1 --2- 172.21.15.96:0/289112696 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f71440afaf0 0x7f71440b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:37.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.097+0000 7f714effd640 1 -- 172.21.15.96:0/289112696 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f71500dee90 con 0x7f7158109a20 2024-04-25T14:13:37.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.097+0000 7f715d9fd640 1 --2- 172.21.15.96:0/289112696 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f71440afaf0 0x7f71440b2000 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7154005f70 tx=0x7f7154000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:37.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.101+0000 7f714effd640 1 -- 172.21.15.96:0/289112696 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7158069310 con 0x7f7158109a20 2024-04-25T14:13:37.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.281+0000 7f715fc61640 1 -- 172.21.15.96:0/289112696 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f71580637a0 con 0x7f7158109a20 2024-04-25T14:13:37.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.282+0000 7f714effd640 1 -- 172.21.15.96:0/289112696 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) ==== 93+0+10 (secure 0 0 0) 0x7f7150016030 con 0x7f7158109a20 2024-04-25T14:13:37.282 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-04-25T14:13:37.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.284+0000 7f715fc61640 1 -- 172.21.15.96:0/289112696 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f71440afaf0 msgr2=0x7f71440b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:37.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.284+0000 7f715fc61640 1 --2- 172.21.15.96:0/289112696 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f71440afaf0 0x7f71440b2000 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7154005f70 tx=0x7f7154000bc0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.284+0000 7f715fc61640 1 -- 172.21.15.96:0/289112696 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7158109a20 msgr2=0x7f71581a16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:37.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.284+0000 7f715fc61640 1 --2- 172.21.15.96:0/289112696 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7158109a20 0x7f71581a16f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f715000eaa0 tx=0x7f715000ef60 comp rx=0 tx=0).stop 2024-04-25T14:13:37.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.284+0000 7f715fc61640 1 -- 172.21.15.96:0/289112696 shutdown_connections 2024-04-25T14:13:37.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.284+0000 7f715fc61640 1 --2- 172.21.15.96:0/289112696 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f71440afaf0 0x7f71440b2000 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.284+0000 7f715fc61640 1 --2- 172.21.15.96:0/289112696 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7158109a20 0x7f71581a16f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.284+0000 7f715fc61640 1 --2- 172.21.15.96:0/289112696 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7158104d20 0x7f715819a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.284+0000 7f715fc61640 1 --2- 172.21.15.96:0/289112696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7158103ae0 0x7f715819a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.284+0000 7f715fc61640 1 -- 172.21.15.96:0/289112696 >> 172.21.15.96:0/289112696 conn(0x7f71580fecd0 msgr2=0x7f7158105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:37.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.284+0000 7f715fc61640 1 -- 172.21.15.96:0/289112696 shutdown_connections 2024-04-25T14:13:37.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.284+0000 7f715fc61640 1 -- 172.21.15.96:0/289112696 wait complete. 2024-04-25T14:13:37.294 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:13:37.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.406+0000 7f9126957640 1 -- 172.21.15.96:0/3064390067 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91201050b0 msgr2=0x7f91201054c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:37.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.406+0000 7f9126957640 1 --2- 172.21.15.96:0/3064390067 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91201050b0 0x7f91201054c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9110009900 tx=0x7f911002a410 comp rx=0 tx=0).stop 2024-04-25T14:13:37.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.406+0000 7f9126957640 1 -- 172.21.15.96:0/3064390067 shutdown_connections 2024-04-25T14:13:37.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.406+0000 7f9126957640 1 --2- 172.21.15.96:0/3064390067 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9120102260 0x7f912010bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.406+0000 7f9126957640 1 --2- 172.21.15.96:0/3064390067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9120101760 0x7f9120101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.406+0000 7f9126957640 1 --2- 172.21.15.96:0/3064390067 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91201050b0 0x7f91201054c0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.406+0000 7f9126957640 1 -- 172.21.15.96:0/3064390067 >> 172.21.15.96:0/3064390067 conn(0x7f91200fd380 msgr2=0x7f91200ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:37.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.406+0000 7f9126957640 1 -- 172.21.15.96:0/3064390067 shutdown_connections 2024-04-25T14:13:37.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.406+0000 7f9126957640 1 -- 172.21.15.96:0/3064390067 wait complete. 2024-04-25T14:13:37.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.407+0000 7f9126957640 1 Processor -- start 2024-04-25T14:13:37.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.407+0000 7f9126957640 1 -- start start 2024-04-25T14:13:37.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.407+0000 7f9126957640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9120101760 0x7f912019a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:37.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.407+0000 7f9126957640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9120102260 0x7f912019a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:37.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.407+0000 7f9126957640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91201050b0 0x7f91201a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:37.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.407+0000 7f9126957640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f912010f620 con 0x7f9120101760 2024-04-25T14:13:37.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.407+0000 7f9126957640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f912010f4a0 con 0x7f91201050b0 2024-04-25T14:13:37.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.407+0000 7f9126957640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f912010f7a0 con 0x7f9120102260 2024-04-25T14:13:37.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.407+0000 7f911ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9120101760 0x7f912019a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:37.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.407+0000 7f911ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9120101760 0x7f912019a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40630/0 (socket says 172.21.15.96:40630) 2024-04-25T14:13:37.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.407+0000 7f911ffff640 1 -- 172.21.15.96:0/705061720 learned_addr learned my addr 172.21.15.96:0/705061720 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:37.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.407+0000 7f911f7fe640 1 --2- 172.21.15.96:0/705061720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9120102260 0x7f912019a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:37.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.408+0000 7f9124ef4640 1 --2- 172.21.15.96:0/705061720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91201050b0 0x7f91201a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:37.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.408+0000 7f911ffff640 1 -- 172.21.15.96:0/705061720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9120102260 msgr2=0x7f912019a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:37.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.408+0000 7f911ffff640 1 --2- 172.21.15.96:0/705061720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9120102260 0x7f912019a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.408+0000 7f911ffff640 1 -- 172.21.15.96:0/705061720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91201050b0 msgr2=0x7f91201a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:37.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.408+0000 7f911ffff640 1 --2- 172.21.15.96:0/705061720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91201050b0 0x7f91201a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.408+0000 7f911ffff640 1 -- 172.21.15.96:0/705061720 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91201a1da0 con 0x7f9120101760 2024-04-25T14:13:37.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.408+0000 7f911ffff640 1 --2- 172.21.15.96:0/705061720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9120101760 0x7f912019a260 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9110006010 tx=0x7f9110031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:37.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.408+0000 7f911d7fa640 1 -- 172.21.15.96:0/705061720 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911003f030 con 0x7f9120101760 2024-04-25T14:13:37.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.408+0000 7f911d7fa640 1 -- 172.21.15.96:0/705061720 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9110003aa0 con 0x7f9120101760 2024-04-25T14:13:37.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.408+0000 7f9126957640 1 -- 172.21.15.96:0/705061720 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91201a2030 con 0x7f9120101760 2024-04-25T14:13:37.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.408+0000 7f911d7fa640 1 -- 172.21.15.96:0/705061720 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9110035070 con 0x7f9120101760 2024-04-25T14:13:37.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.408+0000 7f9126957640 1 -- 172.21.15.96:0/705061720 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91201a2520 con 0x7f9120101760 2024-04-25T14:13:37.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.409+0000 7f9126957640 1 -- 172.21.15.96:0/705061720 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9120108a40 con 0x7f9120101760 2024-04-25T14:13:37.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.410+0000 7f911d7fa640 1 -- 172.21.15.96:0/705061720 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f9110042030 con 0x7f9120101760 2024-04-25T14:13:37.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.411+0000 7f911d7fa640 1 --2- 172.21.15.96:0/705061720 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f91080afaf0 0x7f91080b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:37.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.411+0000 7f911f7fe640 1 --2- 172.21.15.96:0/705061720 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f91080afaf0 0x7f91080b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:37.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.411+0000 7f911d7fa640 1 -- 172.21.15.96:0/705061720 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f91100ff610 con 0x7f9120101760 2024-04-25T14:13:37.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.412+0000 7f911f7fe640 1 --2- 172.21.15.96:0/705061720 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f91080afaf0 0x7f91080b2000 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f912019b300 tx=0x7f911400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:37.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.414+0000 7f911d7fa640 1 -- 172.21.15.96:0/705061720 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f91100c81d0 con 0x7f9120101760 2024-04-25T14:13:37.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.599+0000 7f9126957640 1 -- 172.21.15.96:0/705061720 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f91200637a0 con 0x7f9120101760 2024-04-25T14:13:37.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.600+0000 7f911d7fa640 1 -- 172.21.15.96:0/705061720 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v78) ==== 104+0+11 (secure 0 0 0) 0x7f91100d0250 con 0x7f9120101760 2024-04-25T14:13:37.599 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-04-25T14:13:37.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.602+0000 7f9126957640 1 -- 172.21.15.96:0/705061720 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f91080afaf0 msgr2=0x7f91080b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:37.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.602+0000 7f9126957640 1 --2- 172.21.15.96:0/705061720 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f91080afaf0 0x7f91080b2000 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f912019b300 tx=0x7f911400b410 comp rx=0 tx=0).stop 2024-04-25T14:13:37.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.602+0000 7f9126957640 1 -- 172.21.15.96:0/705061720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9120101760 msgr2=0x7f912019a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:37.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.602+0000 7f9126957640 1 --2- 172.21.15.96:0/705061720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9120101760 0x7f912019a260 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9110006010 tx=0x7f9110031a50 comp rx=0 tx=0).stop 2024-04-25T14:13:37.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.602+0000 7f9126957640 1 -- 172.21.15.96:0/705061720 shutdown_connections 2024-04-25T14:13:37.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.602+0000 7f9126957640 1 --2- 172.21.15.96:0/705061720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91201050b0 0x7f91201a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.602+0000 7f9126957640 1 --2- 172.21.15.96:0/705061720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9120102260 0x7f912019a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.602+0000 7f9126957640 1 --2- 172.21.15.96:0/705061720 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f91080afaf0 0x7f91080b2000 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.602+0000 7f9126957640 1 --2- 172.21.15.96:0/705061720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9120101760 0x7f912019a260 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.602+0000 7f9126957640 1 -- 172.21.15.96:0/705061720 >> 172.21.15.96:0/705061720 conn(0x7f91200fd380 msgr2=0x7f9120105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:37.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.602+0000 7f9126957640 1 -- 172.21.15.96:0/705061720 shutdown_connections 2024-04-25T14:13:37.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.602+0000 7f9126957640 1 -- 172.21.15.96:0/705061720 wait complete. 2024-04-25T14:13:37.613 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:13:37.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.727+0000 7fb4da401640 1 -- 172.21.15.96:0/2423121595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4d41080d0 msgr2=0x7fb4d410a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:37.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.727+0000 7fb4da401640 1 --2- 172.21.15.96:0/2423121595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4d41080d0 0x7fb4d410a4f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc009ca0 tx=0x7fb4cc02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.727+0000 7fb4da401640 1 -- 172.21.15.96:0/2423121595 shutdown_connections 2024-04-25T14:13:37.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.727+0000 7fb4da401640 1 --2- 172.21.15.96:0/2423121595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4d41080d0 0x7fb4d410a4f0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.727+0000 7fb4da401640 1 --2- 172.21.15.96:0/2423121595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4d41033d0 0x7fb4d41037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.727+0000 7fb4da401640 1 --2- 172.21.15.96:0/2423121595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4d4102190 0x7fb4d4102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.727+0000 7fb4da401640 1 -- 172.21.15.96:0/2423121595 >> 172.21.15.96:0/2423121595 conn(0x7fb4d40fd380 msgr2=0x7fb4d40ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:37.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.727+0000 7fb4da401640 1 -- 172.21.15.96:0/2423121595 shutdown_connections 2024-04-25T14:13:37.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.727+0000 7fb4da401640 1 -- 172.21.15.96:0/2423121595 wait complete. 2024-04-25T14:13:37.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.727+0000 7fb4da401640 1 Processor -- start 2024-04-25T14:13:37.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.727+0000 7fb4da401640 1 -- start start 2024-04-25T14:13:37.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.728+0000 7fb4da401640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4d4102190 0x7fb4d4199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:37.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.728+0000 7fb4da401640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4d41033d0 0x7fb4d4199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:37.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.728+0000 7fb4da401640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4d41080d0 0x7fb4d41a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:37.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.728+0000 7fb4da401640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d410e530 con 0x7fb4d41080d0 2024-04-25T14:13:37.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.728+0000 7fb4da401640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d410e3b0 con 0x7fb4d41033d0 2024-04-25T14:13:37.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.728+0000 7fb4da401640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d410e6b0 con 0x7fb4d4102190 2024-04-25T14:13:37.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.728+0000 7fb4d899e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4d41080d0 0x7fb4d41a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:37.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.728+0000 7fb4d899e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4d41080d0 0x7fb4d41a05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40654/0 (socket says 172.21.15.96:40654) 2024-04-25T14:13:37.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.728+0000 7fb4d899e640 1 -- 172.21.15.96:0/3186044197 learned_addr learned my addr 172.21.15.96:0/3186044197 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:37.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.728+0000 7fb4d3fff640 1 --2- 172.21.15.96:0/3186044197 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4d4102190 0x7fb4d4199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:37.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.728+0000 7fb4d899e640 1 -- 172.21.15.96:0/3186044197 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4d4102190 msgr2=0x7fb4d4199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:37.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.728+0000 7fb4d899e640 1 --2- 172.21.15.96:0/3186044197 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4d4102190 0x7fb4d4199100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.728+0000 7fb4d37fe640 1 --2- 172.21.15.96:0/3186044197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4d41033d0 0x7fb4d4199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:37.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.728+0000 7fb4d899e640 1 -- 172.21.15.96:0/3186044197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4d41033d0 msgr2=0x7fb4d4199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:37.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.728+0000 7fb4d899e640 1 --2- 172.21.15.96:0/3186044197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4d41033d0 0x7fb4d4199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.728+0000 7fb4d899e640 1 -- 172.21.15.96:0/3186044197 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d41a0c40 con 0x7fb4d41080d0 2024-04-25T14:13:37.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.728+0000 7fb4d3fff640 1 --2- 172.21.15.96:0/3186044197 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4d4102190 0x7fb4d4199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:37.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.728+0000 7fb4d37fe640 1 --2- 172.21.15.96:0/3186044197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4d41033d0 0x7fb4d4199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:37.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.728+0000 7fb4d899e640 1 --2- 172.21.15.96:0/3186044197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4d41080d0 0x7fb4d41a05d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc0097b0 tx=0x7fb4cc031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:37.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.729+0000 7fb4d17fa640 1 -- 172.21.15.96:0/3186044197 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc003bf0 con 0x7fb4d41080d0 2024-04-25T14:13:37.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.729+0000 7fb4d17fa640 1 -- 172.21.15.96:0/3186044197 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4cc003d90 con 0x7fb4d41080d0 2024-04-25T14:13:37.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.729+0000 7fb4da401640 1 -- 172.21.15.96:0/3186044197 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4d41a0ed0 con 0x7fb4d41080d0 2024-04-25T14:13:37.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.729+0000 7fb4d17fa640 1 -- 172.21.15.96:0/3186044197 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc02fc30 con 0x7fb4d41080d0 2024-04-25T14:13:37.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.729+0000 7fb4da401640 1 -- 172.21.15.96:0/3186044197 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4d41a1330 con 0x7fb4d41080d0 2024-04-25T14:13:37.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.730+0000 7fb4da401640 1 -- 172.21.15.96:0/3186044197 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4d406a160 con 0x7fb4d41080d0 2024-04-25T14:13:37.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.731+0000 7fb4d17fa640 1 -- 172.21.15.96:0/3186044197 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fb4cc004040 con 0x7fb4d41080d0 2024-04-25T14:13:37.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.732+0000 7fb4d17fa640 1 --2- 172.21.15.96:0/3186044197 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb4bc0afaf0 0x7fb4bc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:37.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.732+0000 7fb4d3fff640 1 --2- 172.21.15.96:0/3186044197 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb4bc0afaf0 0x7fb4bc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:37.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.732+0000 7fb4d17fa640 1 -- 172.21.15.96:0/3186044197 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fb4cc1003f0 con 0x7fb4d41080d0 2024-04-25T14:13:37.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.732+0000 7fb4d3fff640 1 --2- 172.21.15.96:0/3186044197 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb4bc0afaf0 0x7fb4bc0b2000 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4102d90 tx=0x7fb4c4005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:37.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.736+0000 7fb4d17fa640 1 -- 172.21.15.96:0/3186044197 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb4cc0c8fb0 con 0x7fb4d41080d0 2024-04-25T14:13:37.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.914+0000 7fb4da401640 1 -- 172.21.15.96:0/3186044197 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb4d40637a0 con 0x7fb4d41080d0 2024-04-25T14:13:37.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.914+0000 7fb4d17fa640 1 -- 172.21.15.96:0/3186044197 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v78) ==== 100+0+11 (secure 0 0 0) 0x7fb4cc0c88b0 con 0x7fb4d41080d0 2024-04-25T14:13:37.914 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-04-25T14:13:37.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.916+0000 7fb4da401640 1 -- 172.21.15.96:0/3186044197 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb4bc0afaf0 msgr2=0x7fb4bc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:37.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.916+0000 7fb4da401640 1 --2- 172.21.15.96:0/3186044197 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb4bc0afaf0 0x7fb4bc0b2000 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4102d90 tx=0x7fb4c4005e70 comp rx=0 tx=0).stop 2024-04-25T14:13:37.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.916+0000 7fb4da401640 1 -- 172.21.15.96:0/3186044197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4d41080d0 msgr2=0x7fb4d41a05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:37.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.916+0000 7fb4da401640 1 --2- 172.21.15.96:0/3186044197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4d41080d0 0x7fb4d41a05d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc0097b0 tx=0x7fb4cc031840 comp rx=0 tx=0).stop 2024-04-25T14:13:37.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.916+0000 7fb4da401640 1 -- 172.21.15.96:0/3186044197 shutdown_connections 2024-04-25T14:13:37.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.916+0000 7fb4da401640 1 --2- 172.21.15.96:0/3186044197 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb4bc0afaf0 0x7fb4bc0b2000 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.916+0000 7fb4da401640 1 --2- 172.21.15.96:0/3186044197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4d41080d0 0x7fb4d41a05d0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.916+0000 7fb4da401640 1 --2- 172.21.15.96:0/3186044197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4d41033d0 0x7fb4d4199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.916+0000 7fb4da401640 1 --2- 172.21.15.96:0/3186044197 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4d4102190 0x7fb4d4199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.916+0000 7fb4da401640 1 -- 172.21.15.96:0/3186044197 >> 172.21.15.96:0/3186044197 conn(0x7fb4d40fd380 msgr2=0x7fb4d41040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:37.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.916+0000 7fb4da401640 1 -- 172.21.15.96:0/3186044197 shutdown_connections 2024-04-25T14:13:37.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:37.916+0000 7fb4da401640 1 -- 172.21.15.96:0/3186044197 wait complete. 2024-04-25T14:13:37.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:37 smithi097 ceph-mon[32720]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:13:37.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:37 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/219767468' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:13:37.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:37 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/289112696' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:13:37.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:37 smithi097 ceph-mon[32720]: pgmap v346: 97 pgs: 97 active+clean; 648 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 938 B/s wr, 12 op/s 2024-04-25T14:13:37.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:37 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/705061720' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:13:37.925 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:37 smithi096 ceph-mon[29207]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:13:37.925 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:37 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/219767468' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:13:37.925 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:37 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/289112696' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:13:37.925 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:37 smithi096 ceph-mon[29207]: pgmap v346: 97 pgs: 97 active+clean; 648 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 938 B/s wr, 12 op/s 2024-04-25T14:13:37.926 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:37 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/705061720' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:13:37.927 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:13:37.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:37 smithi151 ceph-mon[32990]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:13:37.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:37 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/219767468' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:13:37.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:37 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/289112696' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:13:37.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:37 smithi151 ceph-mon[32990]: pgmap v346: 97 pgs: 97 active+clean; 648 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 938 B/s wr, 12 op/s 2024-04-25T14:13:37.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:37 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/705061720' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:13:38.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.035+0000 7fbb497ce640 1 -- 172.21.15.96:0/455648574 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb44108920 msgr2=0x7fbb4410ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:38.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.035+0000 7fbb497ce640 1 --2- 172.21.15.96:0/455648574 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb44108920 0x7fbb4410ad40 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c009ca0 tx=0x7fbb3c02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.035+0000 7fbb497ce640 1 -- 172.21.15.96:0/455648574 shutdown_connections 2024-04-25T14:13:38.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.035+0000 7fbb497ce640 1 --2- 172.21.15.96:0/455648574 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb44108920 0x7fbb4410ad40 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.035+0000 7fbb497ce640 1 --2- 172.21.15.96:0/455648574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb44103c20 0x7fbb44104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.035+0000 7fbb497ce640 1 --2- 172.21.15.96:0/455648574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb441029e0 0x7fbb441036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.035+0000 7fbb497ce640 1 -- 172.21.15.96:0/455648574 >> 172.21.15.96:0/455648574 conn(0x7fbb440fdbd0 msgr2=0x7fbb44100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:38.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.035+0000 7fbb497ce640 1 -- 172.21.15.96:0/455648574 shutdown_connections 2024-04-25T14:13:38.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.035+0000 7fbb497ce640 1 -- 172.21.15.96:0/455648574 wait complete. 2024-04-25T14:13:38.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.036+0000 7fbb497ce640 1 Processor -- start 2024-04-25T14:13:38.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.036+0000 7fbb497ce640 1 -- start start 2024-04-25T14:13:38.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.036+0000 7fbb497ce640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb441029e0 0x7fbb4406bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.036+0000 7fbb497ce640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb44103c20 0x7fbb4419f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.036+0000 7fbb497ce640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb44108920 0x7fbb441a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.036+0000 7fbb497ce640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4410e4f0 con 0x7fbb441029e0 2024-04-25T14:13:38.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.036+0000 7fbb497ce640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4410e370 con 0x7fbb44103c20 2024-04-25T14:13:38.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.036+0000 7fbb497ce640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4410e670 con 0x7fbb44108920 2024-04-25T14:13:38.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.036+0000 7fbb42ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb441029e0 0x7fbb4406bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.036+0000 7fbb42ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb441029e0 0x7fbb4406bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40670/0 (socket says 172.21.15.96:40670) 2024-04-25T14:13:38.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.036+0000 7fbb42ffd640 1 -- 172.21.15.96:0/19164621 learned_addr learned my addr 172.21.15.96:0/19164621 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:38.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.036+0000 7fbb3bfff640 1 --2- 172.21.15.96:0/19164621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb44103c20 0x7fbb4419f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.036+0000 7fbb437fe640 1 --2- 172.21.15.96:0/19164621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb44108920 0x7fbb441a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.036+0000 7fbb42ffd640 1 -- 172.21.15.96:0/19164621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb44108920 msgr2=0x7fbb441a1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:38.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.036+0000 7fbb42ffd640 1 --2- 172.21.15.96:0/19164621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb44108920 0x7fbb441a1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.037+0000 7fbb42ffd640 1 -- 172.21.15.96:0/19164621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb44103c20 msgr2=0x7fbb4419f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:38.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.037+0000 7fbb42ffd640 1 --2- 172.21.15.96:0/19164621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb44103c20 0x7fbb4419f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.037+0000 7fbb42ffd640 1 -- 172.21.15.96:0/19164621 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb441a1fb0 con 0x7fbb441029e0 2024-04-25T14:13:38.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.037+0000 7fbb3bfff640 1 --2- 172.21.15.96:0/19164621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb44103c20 0x7fbb4419f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:38.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.037+0000 7fbb437fe640 1 --2- 172.21.15.96:0/19164621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb44108920 0x7fbb441a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:38.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.037+0000 7fbb42ffd640 1 --2- 172.21.15.96:0/19164621 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb441029e0 0x7fbb4406bc60 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fbb34005fd0 tx=0x7fbb3400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:38.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.037+0000 7fbb40ff9640 1 -- 172.21.15.96:0/19164621 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb34017070 con 0x7fbb441029e0 2024-04-25T14:13:38.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.037+0000 7fbb40ff9640 1 -- 172.21.15.96:0/19164621 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb34009c30 con 0x7fbb441029e0 2024-04-25T14:13:38.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.037+0000 7fbb497ce640 1 -- 172.21.15.96:0/19164621 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb441a22a0 con 0x7fbb441029e0 2024-04-25T14:13:38.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.037+0000 7fbb40ff9640 1 -- 172.21.15.96:0/19164621 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb34003d10 con 0x7fbb441029e0 2024-04-25T14:13:38.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.037+0000 7fbb497ce640 1 -- 172.21.15.96:0/19164621 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb441a27f0 con 0x7fbb441029e0 2024-04-25T14:13:38.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.038+0000 7fbb497ce640 1 -- 172.21.15.96:0/19164621 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb44107730 con 0x7fbb441029e0 2024-04-25T14:13:38.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.039+0000 7fbb40ff9640 1 -- 172.21.15.96:0/19164621 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fbb34003eb0 con 0x7fbb441029e0 2024-04-25T14:13:38.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.040+0000 7fbb40ff9640 1 --2- 172.21.15.96:0/19164621 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fbb300afaf0 0x7fbb300b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.040+0000 7fbb3bfff640 1 --2- 172.21.15.96:0/19164621 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fbb300afaf0 0x7fbb300b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.040+0000 7fbb40ff9640 1 -- 172.21.15.96:0/19164621 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fbb340df190 con 0x7fbb441029e0 2024-04-25T14:13:38.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.041+0000 7fbb3bfff640 1 --2- 172.21.15.96:0/19164621 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fbb300afaf0 0x7fbb300b2000 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c005e00 tx=0x7fbb2c005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:38.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.044+0000 7fbb40ff9640 1 -- 172.21.15.96:0/19164621 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbb34020030 con 0x7fbb441029e0 2024-04-25T14:13:38.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.258+0000 7fbb497ce640 1 -- 172.21.15.96:0/19164621 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb44069b70 con 0x7fbb441029e0 2024-04-25T14:13:38.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.258+0000 7fbb40ff9640 1 -- 172.21.15.96:0/19164621 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11927 (secure 0 0 0) 0x7fbb34009de0 con 0x7fbb441029e0 2024-04-25T14:13:38.257 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:13:38.257 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/606227917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":606227917},{"type":"v1","addr":"172.21.15.96:6835","nonce":606227917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/3265300710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3265300710},{"type":"v1","addr":"172.21.15.96:6837","nonce":3265300710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T14:11:28.886961+0000","modified":"2024-04-25T14:13:32.120297+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24517,"mds_2":24478,"mds_3":24455,"mds_4":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/4080308509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":4080308509},{"type":"v1","addr":"172.21.15.96:6839","nonce":4080308509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/1606748971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":1606748971},{"type":"v1","addr":"172.21.15.96:6841","nonce":1606748971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6833/1995324831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1995324831},{"type":"v1","addr":"172.21.15.97:6833","nonce":1995324831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":4,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.151:6833/1382721608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1382721608},{"type":"v1","addr":"172.21.15.151:6833","nonce":1382721608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6835/3872301726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3872301726},{"type":"v1","addr":"172.21.15.97:6835","nonce":3872301726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.151:6835/1313823638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":1313823638},{"type":"v1","addr":"172.21.15.151:6835","nonce":1313823638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/2691348366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2691348366},{"type":"v1","addr":"172.21.15.151:6837","nonce":2691348366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3306780081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3306780081},{"type":"v1","addr":"172.21.15.97:6837","nonce":3306780081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.151:6839/245699981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":245699981},{"type":"v1","addr":"172.21.15.151:6839","nonce":245699981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6839/2850442269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":2850442269},{"type":"v1","addr":"172.21.15.97:6839","nonce":2850442269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24427,24455,24478,24517,24536]},"id":1}]} 2024-04-25T14:13:38.258 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 26 2024-04-25T14:13:38.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.260+0000 7fbb497ce640 1 -- 172.21.15.96:0/19164621 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fbb300afaf0 msgr2=0x7fbb300b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:38.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.260+0000 7fbb497ce640 1 --2- 172.21.15.96:0/19164621 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fbb300afaf0 0x7fbb300b2000 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c005e00 tx=0x7fbb2c005c70 comp rx=0 tx=0).stop 2024-04-25T14:13:38.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.260+0000 7fbb497ce640 1 -- 172.21.15.96:0/19164621 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb441029e0 msgr2=0x7fbb4406bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:38.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.260+0000 7fbb497ce640 1 --2- 172.21.15.96:0/19164621 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb441029e0 0x7fbb4406bc60 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fbb34005fd0 tx=0x7fbb3400dde0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.260+0000 7fbb497ce640 1 -- 172.21.15.96:0/19164621 shutdown_connections 2024-04-25T14:13:38.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.260+0000 7fbb497ce640 1 --2- 172.21.15.96:0/19164621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb44108920 0x7fbb441a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.260+0000 7fbb497ce640 1 --2- 172.21.15.96:0/19164621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb44103c20 0x7fbb4419f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.260+0000 7fbb497ce640 1 --2- 172.21.15.96:0/19164621 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fbb300afaf0 0x7fbb300b2000 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.260+0000 7fbb497ce640 1 --2- 172.21.15.96:0/19164621 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb441029e0 0x7fbb4406bc60 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.260+0000 7fbb497ce640 1 -- 172.21.15.96:0/19164621 >> 172.21.15.96:0/19164621 conn(0x7fbb440fdbd0 msgr2=0x7fbb44104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:38.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.261+0000 7fbb497ce640 1 -- 172.21.15.96:0/19164621 shutdown_connections 2024-04-25T14:13:38.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.261+0000 7fbb497ce640 1 -- 172.21.15.96:0/19164621 wait complete. 2024-04-25T14:13:38.271 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:13:38.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.379+0000 7f7670c51640 1 -- 172.21.15.96:0/3266234168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f766c103c40 msgr2=0x7f766c104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:38.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.379+0000 7f7670c51640 1 --2- 172.21.15.96:0/3266234168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f766c103c40 0x7f766c104050 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7654009900 tx=0x7f765402a410 comp rx=0 tx=0).stop 2024-04-25T14:13:38.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.379+0000 7f7670c51640 1 -- 172.21.15.96:0/3266234168 shutdown_connections 2024-04-25T14:13:38.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.379+0000 7f7670c51640 1 --2- 172.21.15.96:0/3266234168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f766c108940 0x7f766c10ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.379+0000 7f7670c51640 1 --2- 172.21.15.96:0/3266234168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f766c103c40 0x7f766c104050 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.379+0000 7f7670c51640 1 --2- 172.21.15.96:0/3266234168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f766c102a00 0x7f766c103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.379+0000 7f7670c51640 1 -- 172.21.15.96:0/3266234168 >> 172.21.15.96:0/3266234168 conn(0x7f766c0fdbf0 msgr2=0x7f766c1000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:38.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.379+0000 7f7670c51640 1 -- 172.21.15.96:0/3266234168 shutdown_connections 2024-04-25T14:13:38.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.379+0000 7f7670c51640 1 -- 172.21.15.96:0/3266234168 wait complete. 2024-04-25T14:13:38.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.380+0000 7f7670c51640 1 Processor -- start 2024-04-25T14:13:38.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.380+0000 7f7670c51640 1 -- start start 2024-04-25T14:13:38.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.380+0000 7f7670c51640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f766c102a00 0x7f766c1991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.380+0000 7f7670c51640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f766c103c40 0x7f766c1996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.380+0000 7f7670c51640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f766c108940 0x7f766c1a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.380+0000 7f7670c51640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f766c10e580 con 0x7f766c103c40 2024-04-25T14:13:38.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.380+0000 7f7670c51640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f766c10e400 con 0x7f766c108940 2024-04-25T14:13:38.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.380+0000 7f7670c51640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f766c10e700 con 0x7f766c102a00 2024-04-25T14:13:38.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.380+0000 7f7663fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f766c103c40 0x7f766c1996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.380+0000 7f7663fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f766c103c40 0x7f766c1996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40694/0 (socket says 172.21.15.96:40694) 2024-04-25T14:13:38.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.380+0000 7f7663fff640 1 -- 172.21.15.96:0/3836224282 learned_addr learned my addr 172.21.15.96:0/3836224282 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:38.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.381+0000 7f766a59c640 1 --2- 172.21.15.96:0/3836224282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f766c102a00 0x7f766c1991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.381+0000 7f766ad9d640 1 --2- 172.21.15.96:0/3836224282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f766c108940 0x7f766c1a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.381+0000 7f7663fff640 1 -- 172.21.15.96:0/3836224282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f766c102a00 msgr2=0x7f766c1991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:38.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.381+0000 7f7663fff640 1 --2- 172.21.15.96:0/3836224282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f766c102a00 0x7f766c1991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.381+0000 7f7663fff640 1 -- 172.21.15.96:0/3836224282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f766c108940 msgr2=0x7f766c1a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:38.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.381+0000 7f7663fff640 1 --2- 172.21.15.96:0/3836224282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f766c108940 0x7f766c1a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.381+0000 7f7663fff640 1 -- 172.21.15.96:0/3836224282 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f766c1a0c80 con 0x7f766c103c40 2024-04-25T14:13:38.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.381+0000 7f7663fff640 1 --2- 172.21.15.96:0/3836224282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f766c103c40 0x7f766c1996f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7654006010 tx=0x7f7654004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:38.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.381+0000 7f76637fe640 1 -- 172.21.15.96:0/3836224282 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765403f030 con 0x7f766c103c40 2024-04-25T14:13:38.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.381+0000 7f76637fe640 1 -- 172.21.15.96:0/3836224282 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7654003cb0 con 0x7f766c103c40 2024-04-25T14:13:38.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.381+0000 7f7670c51640 1 -- 172.21.15.96:0/3836224282 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f766c1a0f10 con 0x7f766c103c40 2024-04-25T14:13:38.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.381+0000 7f76637fe640 1 -- 172.21.15.96:0/3836224282 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7654035070 con 0x7f766c103c40 2024-04-25T14:13:38.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.381+0000 7f7670c51640 1 -- 172.21.15.96:0/3836224282 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f766c1a1320 con 0x7f766c103c40 2024-04-25T14:13:38.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.382+0000 7f7670c51640 1 -- 172.21.15.96:0/3836224282 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f766c107750 con 0x7f766c103c40 2024-04-25T14:13:38.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.383+0000 7f76637fe640 1 -- 172.21.15.96:0/3836224282 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f7654042030 con 0x7f766c103c40 2024-04-25T14:13:38.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.384+0000 7f76637fe640 1 --2- 172.21.15.96:0/3836224282 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f76580afaf0 0x7f76580b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.384+0000 7f766a59c640 1 --2- 172.21.15.96:0/3836224282 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f76580afaf0 0x7f76580b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.384+0000 7f76637fe640 1 -- 172.21.15.96:0/3836224282 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f76540ffa00 con 0x7f766c103c40 2024-04-25T14:13:38.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.384+0000 7f766a59c640 1 --2- 172.21.15.96:0/3836224282 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f76580afaf0 0x7f76580b2000 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f765c005d00 tx=0x7f765c005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:38.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.388+0000 7f76637fe640 1 -- 172.21.15.96:0/3836224282 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f76540c85c0 con 0x7f766c103c40 2024-04-25T14:13:38.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.570+0000 7f7670c51640 1 -- 172.21.15.96:0/3836224282 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f766c02dd20 con 0x7f766c103c40 2024-04-25T14:13:38.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.571+0000 7f76637fe640 1 -- 172.21.15.96:0/3836224282 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22616 (secure 0 0 0) 0x7f7654003e60 con 0x7f766c103c40 2024-04-25T14:13:38.570 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:13:38.571 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":78,"fsid":"59334182-030c-11ef-bc93-c7b262605968","created":"2024-04-25T14:03:25.680745+0000","modified":"2024-04-25T14:12:22.496143+0000","last_up_change":"2024-04-25T14:10:35.894312+0000","last_in_change":"2024-04-25T14:10:15.583440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:06:47.402927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:11:27.434331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:11:27.854004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cf4ba3ac-142e-4e8e-96d4-410d77157eb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6803","nonce":3832506168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6805","nonce":3832506168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832506168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6807","nonce":3832506168}]},"public_addr":"172.21.15.96:6803/3832506168","cluster_addr":"172.21.15.96:6805/3832506168","heartbeat_back_addr":"172.21.15.96:6809/3832506168","heartbeat_front_addr":"172.21.15.96:6807/3832506168","state":["exists","up"]},{"osd":1,"uuid":"866e73b9-4a26-48ba-badf-704fe22f71cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6801","nonce":1083311494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6803","nonce":1083311494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6807","nonce":1083311494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6805","nonce":1083311494}]},"public_addr":"172.21.15.97:6801/1083311494","cluster_addr":"172.21.15.97:6803/1083311494","heartbeat_back_addr":"172.21.15.97:6807/1083311494","heartbeat_front_addr":"172.21.15.97:6805/1083311494","state":["exists","up"]},{"osd":2,"uuid":"45c79ecb-abc4-4dba-bae8-74804e4bc1bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6801","nonce":981902966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6803","nonce":981902966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6807","nonce":981902966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6805","nonce":981902966}]},"public_addr":"172.21.15.151:6801/981902966","cluster_addr":"172.21.15.151:6803/981902966","heartbeat_back_addr":"172.21.15.151:6807/981902966","heartbeat_front_addr":"172.21.15.151:6805/981902966","state":["exists","up"]},{"osd":3,"uuid":"9fc39468-a7ea-4ccf-b365-335ad2ea4c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6811","nonce":2274158748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6813","nonce":2274158748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6817","nonce":2274158748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6815","nonce":2274158748}]},"public_addr":"172.21.15.96:6811/2274158748","cluster_addr":"172.21.15.96:6813/2274158748","heartbeat_back_addr":"172.21.15.96:6817/2274158748","heartbeat_front_addr":"172.21.15.96:6815/2274158748","state":["exists","up"]},{"osd":4,"uuid":"616582f1-12d2-456b-b7b3-e5517800e8a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6809","nonce":3727031962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6811","nonce":3727031962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6815","nonce":3727031962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6813","nonce":3727031962}]},"public_addr":"172.21.15.97:6809/3727031962","cluster_addr":"172.21.15.97:6811/3727031962","heartbeat_back_addr":"172.21.15.97:6815/3727031962","heartbeat_front_addr":"172.21.15.97:6813/3727031962","state":["exists","up"]},{"osd":5,"uuid":"3d277784-ab0f-4c90-81a1-2a4034c52a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6809","nonce":223882546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6811","nonce":223882546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6815","nonce":223882546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6813","nonce":223882546}]},"public_addr":"172.21.15.151:6809/223882546","cluster_addr":"172.21.15.151:6811/223882546","heartbeat_back_addr":"172.21.15.151:6815/223882546","heartbeat_front_addr":"172.21.15.151:6813/223882546","state":["exists","up"]},{"osd":6,"uuid":"9a9ad585-cb1a-4239-9d18-055f59164049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6819","nonce":96197407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6821","nonce":96197407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6825","nonce":96197407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6823","nonce":96197407}]},"public_addr":"172.21.15.96:6819/96197407","cluster_addr":"172.21.15.96:6821/96197407","heartbeat_back_addr":"172.21.15.96:6825/96197407","heartbeat_front_addr":"172.21.15.96:6823/96197407","state":["exists","up"]},{"osd":7,"uuid":"303551c8-6af0-42f2-844f-45bf8db5baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6817","nonce":2247216400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6819","nonce":2247216400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6823","nonce":2247216400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6821","nonce":2247216400}]},"public_addr":"172.21.15.97:6817/2247216400","cluster_addr":"172.21.15.97:6819/2247216400","heartbeat_back_addr":"172.21.15.97:6823/2247216400","heartbeat_front_addr":"172.21.15.97:6821/2247216400","state":["exists","up"]},{"osd":8,"uuid":"81f6cc66-e4ca-430c-b336-b6678d968818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6817","nonce":2549822185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6819","nonce":2549822185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6823","nonce":2549822185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6821","nonce":2549822185}]},"public_addr":"172.21.15.151:6817/2549822185","cluster_addr":"172.21.15.151:6819/2549822185","heartbeat_back_addr":"172.21.15.151:6823/2549822185","heartbeat_front_addr":"172.21.15.151:6821/2549822185","state":["exists","up"]},{"osd":9,"uuid":"29d4da16-feab-4d1c-9d1d-de4f799bd724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6827","nonce":1099399643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6829","nonce":1099399643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6833","nonce":1099399643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6831","nonce":1099399643}]},"public_addr":"172.21.15.96:6827/1099399643","cluster_addr":"172.21.15.96:6829/1099399643","heartbeat_back_addr":"172.21.15.96:6833/1099399643","heartbeat_front_addr":"172.21.15.96:6831/1099399643","state":["exists","up"]},{"osd":10,"uuid":"197fee9b-eee4-4893-be1c-3d57f42250e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6825","nonce":155754520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6827","nonce":155754520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6831","nonce":155754520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6829","nonce":155754520}]},"public_addr":"172.21.15.97:6825/155754520","cluster_addr":"172.21.15.97:6827/155754520","heartbeat_back_addr":"172.21.15.97:6831/155754520","heartbeat_front_addr":"172.21.15.97:6829/155754520","state":["exists","up"]},{"osd":11,"uuid":"9c1d0a83-fc12-41a2-8eb0-80c4abdd2770","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6825","nonce":2027958746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6827","nonce":2027958746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6831","nonce":2027958746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6829","nonce":2027958746}]},"public_addr":"172.21.15.151:6825/2027958746","cluster_addr":"172.21.15.151:6827/2027958746","heartbeat_back_addr":"172.21.15.151:6831/2027958746","heartbeat_front_addr":"172.21.15.151:6829/2027958746","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:05:56.177390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:06:19.917663+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:06:43.608052+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:08.540382+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:33.474072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:58.242306+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:08:24.120442+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:08:49.661064+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:09:15.097271+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:09:41.904505+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:10:08.233252+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:10:33.712296+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/3839807484":"2024-04-26T14:04:21.307574+0000","172.21.15.96:0/1495237551":"2024-04-26T14:04:21.307574+0000","172.21.15.96:0/2793751450":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/129575834":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/2233075139":"2024-04-26T14:04:21.307574+0000","172.21.15.96:6801/2348397481":"2024-04-26T14:03:54.249537+0000","172.21.15.96:6801/433591062":"2024-04-26T14:04:21.307574+0000","172.21.15.96:6800/2348397481":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/3517674924":"2024-04-26T14:03:54.249537+0000","172.21.15.96:6800/433591062":"2024-04-26T14:04:21.307574+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:13:38.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.573+0000 7f7670c51640 1 -- 172.21.15.96:0/3836224282 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f76580afaf0 msgr2=0x7f76580b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:38.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.573+0000 7f7670c51640 1 --2- 172.21.15.96:0/3836224282 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f76580afaf0 0x7f76580b2000 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f765c005d00 tx=0x7f765c005c50 comp rx=0 tx=0).stop 2024-04-25T14:13:38.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.573+0000 7f7670c51640 1 -- 172.21.15.96:0/3836224282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f766c103c40 msgr2=0x7f766c1996f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:38.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.573+0000 7f7670c51640 1 --2- 172.21.15.96:0/3836224282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f766c103c40 0x7f766c1996f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7654006010 tx=0x7f7654004340 comp rx=0 tx=0).stop 2024-04-25T14:13:38.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.573+0000 7f7670c51640 1 -- 172.21.15.96:0/3836224282 shutdown_connections 2024-04-25T14:13:38.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.573+0000 7f7670c51640 1 --2- 172.21.15.96:0/3836224282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f766c108940 0x7f766c1a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.573+0000 7f7670c51640 1 --2- 172.21.15.96:0/3836224282 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f76580afaf0 0x7f76580b2000 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.573+0000 7f7670c51640 1 --2- 172.21.15.96:0/3836224282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f766c103c40 0x7f766c1996f0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.573+0000 7f7670c51640 1 --2- 172.21.15.96:0/3836224282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f766c102a00 0x7f766c1991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.573+0000 7f7670c51640 1 -- 172.21.15.96:0/3836224282 >> 172.21.15.96:0/3836224282 conn(0x7f766c0fdbf0 msgr2=0x7f766c104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:38.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.573+0000 7f7670c51640 1 -- 172.21.15.96:0/3836224282 shutdown_connections 2024-04-25T14:13:38.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.573+0000 7f7670c51640 1 -- 172.21.15.96:0/3836224282 wait complete. 2024-04-25T14:13:38.585 DEBUG:tasks.fwd_scrub:Yielding 2024-04-25T14:13:38.585 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-25T14:13:38.599 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi096.front.sepia.ceph.com 2024-04-25T14:13:38.599 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-25T14:13:38.599 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-04-25T14:13:38.599 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T14:13:38.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:13:38.600 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T14:13:38.602 INFO:teuthology.run_tasks:Running task workunit... 2024-04-25T14:13:38.635 INFO:tasks.workunit:Pulling workunits from ref 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T14:13:38.636 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-25T14:13:38.636 INFO:tasks.workunit:timeout=3h 2024-04-25T14:13:38.636 INFO:tasks.workunit:cleanup=True 2024-04-25T14:13:38.636 DEBUG:teuthology.orchestra.run.smithi096:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-25T14:13:38.652 INFO:teuthology.orchestra.run.smithi096.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-25T14:13:38.652 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-25T14:13:38.652 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 73h/115d Inode: 1099511628283 Links: 2 2024-04-25T14:13:38.652 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:13:38.652 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-25T14:13:38.652 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-04-25 14:11:35.014987963 +0000 2024-04-25T14:13:38.652 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-04-25 14:11:35.014987963 +0000 2024-04-25T14:13:38.652 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-04-25 14:13:31.433223388 +0000 2024-04-25T14:13:38.652 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-04-25 14:11:35.014987963 +0000 2024-04-25T14:13:38.653 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-25T14:13:38.653 DEBUG:teuthology.orchestra.run.smithi096:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-25T14:13:38.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.716+0000 7ff46650d640 1 -- 172.21.15.96:0/3230044408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4601029e0 msgr2=0x7ff4601036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:38.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.716+0000 7ff46650d640 1 --2- 172.21.15.96:0/3230044408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4601029e0 0x7ff4601036e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff450009900 tx=0x7ff45002a3f0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.716+0000 7ff46650d640 1 -- 172.21.15.96:0/3230044408 shutdown_connections 2024-04-25T14:13:38.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.716+0000 7ff46650d640 1 --2- 172.21.15.96:0/3230044408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff460108920 0x7ff46010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.716+0000 7ff46650d640 1 --2- 172.21.15.96:0/3230044408 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff460103c20 0x7ff460104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.716+0000 7ff46650d640 1 --2- 172.21.15.96:0/3230044408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4601029e0 0x7ff4601036e0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.716+0000 7ff46650d640 1 -- 172.21.15.96:0/3230044408 >> 172.21.15.96:0/3230044408 conn(0x7ff4600fdbd0 msgr2=0x7ff460100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:38.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.716+0000 7ff46650d640 1 -- 172.21.15.96:0/3230044408 shutdown_connections 2024-04-25T14:13:38.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.716+0000 7ff46650d640 1 -- 172.21.15.96:0/3230044408 wait complete. 2024-04-25T14:13:38.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.717+0000 7ff46650d640 1 Processor -- start 2024-04-25T14:13:38.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.717+0000 7ff46650d640 1 -- start start 2024-04-25T14:13:38.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.717+0000 7ff46650d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4601029e0 0x7ff46006a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.717+0000 7ff46650d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff460103c20 0x7ff46019f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.717+0000 7ff46650d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff460108920 0x7ff4601a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.717+0000 7ff46650d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff46010e4e0 con 0x7ff460108920 2024-04-25T14:13:38.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.717+0000 7ff46650d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff46010e360 con 0x7ff4601029e0 2024-04-25T14:13:38.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.717+0000 7ff46650d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff46010e660 con 0x7ff460103c20 2024-04-25T14:13:38.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.717+0000 7ff464aaa640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff460108920 0x7ff4601a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.717+0000 7ff464aaa640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff460108920 0x7ff4601a1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40710/0 (socket says 172.21.15.96:40710) 2024-04-25T14:13:38.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.717+0000 7ff464aaa640 1 -- 172.21.15.96:0/1216040073 learned_addr learned my addr 172.21.15.96:0/1216040073 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:38.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.717+0000 7ff45f7fe640 1 --2- 172.21.15.96:0/1216040073 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff460103c20 0x7ff46019f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.718+0000 7ff464aaa640 1 -- 172.21.15.96:0/1216040073 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff460103c20 msgr2=0x7ff46019f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:38.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.718+0000 7ff464aaa640 1 --2- 172.21.15.96:0/1216040073 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff460103c20 0x7ff46019f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.718+0000 7ff464aaa640 1 -- 172.21.15.96:0/1216040073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4601029e0 msgr2=0x7ff46006a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:38.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.718+0000 7ff464aaa640 1 --2- 172.21.15.96:0/1216040073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4601029e0 0x7ff46006a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.718+0000 7ff464aaa640 1 -- 172.21.15.96:0/1216040073 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff460199220 con 0x7ff460108920 2024-04-25T14:13:38.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.718+0000 7ff464aaa640 1 --2- 172.21.15.96:0/1216040073 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff460108920 0x7ff4601a1870 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff458007440 tx=0x7ff458007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:38.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.718+0000 7ff45d7fa640 1 -- 172.21.15.96:0/1216040073 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff458012d30 con 0x7ff460108920 2024-04-25T14:13:38.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.718+0000 7ff45d7fa640 1 -- 172.21.15.96:0/1216040073 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4580140d0 con 0x7ff460108920 2024-04-25T14:13:38.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.718+0000 7ff46650d640 1 -- 172.21.15.96:0/1216040073 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4601994b0 con 0x7ff460108920 2024-04-25T14:13:38.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.718+0000 7ff45d7fa640 1 -- 172.21.15.96:0/1216040073 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff458005280 con 0x7ff460108920 2024-04-25T14:13:38.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.718+0000 7ff46650d640 1 -- 172.21.15.96:0/1216040073 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff460199a00 con 0x7ff460108920 2024-04-25T14:13:38.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.721+0000 7ff45d7fa640 1 -- 172.21.15.96:0/1216040073 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7ff458014580 con 0x7ff460108920 2024-04-25T14:13:38.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.721+0000 7ff46d3c3740 1 Processor -- start 2024-04-25T14:13:38.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.721+0000 7ff46d3c3740 1 -- start start 2024-04-25T14:13:38.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.721+0000 7ff46d3c3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56433f3e8410 0x56433f7894a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.721+0000 7ff46d3c3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433f3bf550 0x56433f789a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.721+0000 7ff46d3c3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56433f3c7730 0x56433f790920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.721+0000 7ff46d3c3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56433f6eeb00 con 0x56433f3bf550 2024-04-25T14:13:38.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.721+0000 7ff46d3c3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56433f68a920 con 0x56433f3c7730 2024-04-25T14:13:38.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.721+0000 7ff46d3c3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56433f6eaac0 con 0x56433f3e8410 2024-04-25T14:13:38.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.721+0000 7ff45ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433f3bf550 0x56433f789a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.721+0000 7ff45ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433f3bf550 0x56433f789a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40726/0 (socket says 172.21.15.96:40726) 2024-04-25T14:13:38.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.721+0000 7ff45ffff640 1 -- 172.21.15.96:0/3293611900 learned_addr learned my addr 172.21.15.96:0/3293611900 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:38.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.721+0000 7ff45f7fe640 1 --2- 172.21.15.96:0/3293611900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56433f3c7730 0x56433f790920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.721+0000 7ff45d7fa640 1 --2- 172.21.15.96:0/1216040073 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ff4480afb70 0x7ff4480b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.721+0000 7ff45f7fe640 1 --2- 172.21.15.96:0/3293611900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56433f3e8410 0x56433f7894a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.722+0000 7ff464aaa640 1 --2- 172.21.15.96:0/1216040073 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ff4480afb70 0x7ff4480b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.722+0000 7ff45ffff640 1 -- 172.21.15.96:0/3293611900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56433f3e8410 msgr2=0x56433f7894a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:38.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.722+0000 7ff45ffff640 1 --2- 172.21.15.96:0/3293611900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56433f3e8410 0x56433f7894a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.722+0000 7ff45ffff640 1 -- 172.21.15.96:0/3293611900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56433f3c7730 msgr2=0x56433f790920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:38.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.722+0000 7ff45ffff640 1 --2- 172.21.15.96:0/3293611900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56433f3c7730 0x56433f790920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.722+0000 7ff45ffff640 1 -- 172.21.15.96:0/3293611900 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56433f4c87e0 con 0x56433f3bf550 2024-04-25T14:13:38.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.722+0000 7ff45f7fe640 1 --2- 172.21.15.96:0/3293611900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56433f3e8410 0x56433f7894a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:38.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.722+0000 7ff45d7fa640 1 -- 172.21.15.96:0/1216040073 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7ff4580e6780 con 0x7ff460108920 2024-04-25T14:13:38.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.722+0000 7ff45ffff640 1 --2- 172.21.15.96:0/3293611900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433f3bf550 0x56433f789a20 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff45002a910 tx=0x7ff450000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51c4bfa550f270d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:38.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.722+0000 7ff464aaa640 1 --2- 172.21.15.96:0/1216040073 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ff4480afb70 0x7ff4480b2080 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff4580abeb0 tx=0x7ff45807c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:38.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.722+0000 7ff4467fc640 1 -- 172.21.15.96:0/3293611900 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45003e030 con 0x56433f3bf550 2024-04-25T14:13:38.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.722+0000 7ff4467fc640 1 -- 172.21.15.96:0/3293611900 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff450032d60 con 0x56433f3bf550 2024-04-25T14:13:38.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.722+0000 7ff4467fc640 1 -- 172.21.15.96:0/3293611900 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff450035040 con 0x56433f3bf550 2024-04-25T14:13:38.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.723+0000 7ff46d3c3740 1 -- 172.21.15.96:0/3293611900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433f3bf550 msgr2=0x56433f789a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:38.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.723+0000 7ff46d3c3740 1 --2- 172.21.15.96:0/3293611900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433f3bf550 0x56433f789a20 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff45002a910 tx=0x7ff450000bc0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.723+0000 7ff46d3c3740 1 -- 172.21.15.96:0/3293611900 shutdown_connections 2024-04-25T14:13:38.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.723+0000 7ff46d3c3740 1 --2- 172.21.15.96:0/3293611900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56433f3c7730 0x56433f790920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.723+0000 7ff46d3c3740 1 --2- 172.21.15.96:0/3293611900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433f3bf550 0x56433f789a20 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.723+0000 7ff46d3c3740 1 --2- 172.21.15.96:0/3293611900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56433f3e8410 0x56433f7894a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.723+0000 7ff46d3c3740 1 -- 172.21.15.96:0/3293611900 >> 172.21.15.96:0/3293611900 conn(0x56433f3dc560 msgr2=0x56433f784e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:38.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.723+0000 7ff46d3c3740 1 -- 172.21.15.96:0/3293611900 shutdown_connections 2024-04-25T14:13:38.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.723+0000 7ff46d3c3740 1 -- 172.21.15.96:0/3293611900 wait complete. 2024-04-25T14:13:38.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.723+0000 7ff46d3c3740 1 -- start start 2024-04-25T14:13:38.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.724+0000 7ff46d3c3740 1 Processor -- start 2024-04-25T14:13:38.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.724+0000 7ff46d3c3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56433f3bf550 0x56433f793200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.724+0000 7ff46d3c3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56433f3c7730 0x56433f793740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.724+0000 7ff46d3c3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433f3e8410 0x56433f79a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.724+0000 7ff46d3c3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56433f791310 con 0x56433f3e8410 2024-04-25T14:13:38.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.724+0000 7ff46d3c3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56433f6885d0 con 0x56433f3bf550 2024-04-25T14:13:38.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.724+0000 7ff46d3c3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56433f791490 con 0x56433f3c7730 2024-04-25T14:13:38.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.724+0000 7ff46d3c3740 10 client.0 _ll_register_callbacks cb 0x56433f676080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:13:38.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.724+0000 7ff45f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433f3e8410 0x56433f79a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.724+0000 7ff45f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433f3e8410 0x56433f79a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40736/0 (socket says 172.21.15.96:40736) 2024-04-25T14:13:38.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.724+0000 7ff45f7fe640 1 -- 172.21.15.96:0/2805517846 learned_addr learned my addr 172.21.15.96:0/2805517846 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:38.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.724+0000 7ff45ffff640 1 --2- 172.21.15.96:0/2805517846 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56433f3c7730 0x56433f793740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.724+0000 7ff45f7fe640 1 --2- 172.21.15.96:0/2805517846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56433f3bf550 0x56433f793200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.724+0000 7ff45f7fe640 1 -- 172.21.15.96:0/2805517846 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56433f3c7730 msgr2=0x56433f793740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:38.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.724+0000 7ff45f7fe640 1 --2- 172.21.15.96:0/2805517846 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56433f3c7730 0x56433f793740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.724+0000 7ff45f7fe640 1 -- 172.21.15.96:0/2805517846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56433f3bf550 msgr2=0x56433f793200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:38.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.724+0000 7ff45f7fe640 1 --2- 172.21.15.96:0/2805517846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56433f3bf550 0x56433f793200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.724+0000 7ff45f7fe640 1 -- 172.21.15.96:0/2805517846 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56433f528f70 con 0x56433f3e8410 2024-04-25T14:13:38.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.724+0000 7ff45ffff640 1 --2- 172.21.15.96:0/2805517846 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56433f3c7730 0x56433f793740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:13:38.729 DEBUG:teuthology.orchestra.run.smithi096:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T14:13:38.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.725+0000 7ff45f7fe640 1 --2- 172.21.15.96:0/2805517846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433f3e8410 0x56433f79a6a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff454011bf0 tx=0x7ff454004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa6acda40a1e3068 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:38.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.725+0000 7ff43f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:13:38.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.725+0000 7ff43f7fe640 1 -- 172.21.15.96:0/2805517846 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45400fc50 con 0x56433f3e8410 2024-04-25T14:13:38.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.725+0000 7ff43f7fe640 1 -- 172.21.15.96:0/2805517846 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7ff45400fdf0 con 0x56433f3e8410 2024-04-25T14:13:38.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.725+0000 7ff46d3c3740 1 -- 172.21.15.96:0/2805517846 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff450032d60 con 0x56433f3e8410 2024-04-25T14:13:38.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.725+0000 7ff43f7fe640 1 -- 172.21.15.96:0/2805517846 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff454011050 con 0x56433f3e8410 2024-04-25T14:13:38.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.725+0000 7ff46d3c3740 10 client.15480 fetch_fsmap learned FSMap version 26 2024-04-25T14:13:38.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.725+0000 7ff46d3c3740 1 -- 172.21.15.96:0/2805517846 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56433f3c0170 con 0x56433f3e8410 2024-04-25T14:13:38.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.725+0000 7ff43f7fe640 1 -- 172.21.15.96:0/2805517846 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff450032d60 con 0x56433f3e8410 2024-04-25T14:13:38.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.725+0000 7ff43f7fe640 1 -- 172.21.15.96:0/2805517846 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff454003990 con 0x56433f3e8410 2024-04-25T14:13:38.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.725+0000 7ff46d3c3740 10 client.15480 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:13:38.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.725+0000 7ff46d3c3740 10 client.15480 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:13:38.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.725+0000 7ff46d3c3740 20 client.15480 populate_metadata read hostname 'smithi096' 2024-04-25T14:13:38.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.725+0000 7ff46d3c3740 1 --2- 172.21.15.96:0/2805517846 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x56433f3a71f0 0x56433f7a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.725+0000 7ff46d3c3740 4 client.15480 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:13:38.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.725+0000 7ff46d3c3740 1 -- 172.21.15.96:0/2805517846 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56433f6fc710 con 0x56433f3a71f0 2024-04-25T14:13:38.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.726+0000 7ff45ffff640 1 --2- 172.21.15.96:0/2805517846 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x56433f3a71f0 0x56433f7a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.726+0000 7ff43f7fe640 10 client.15480 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:13:38.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.726+0000 7ff45ffff640 1 --2- 172.21.15.96:0/2805517846 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x56433f3a71f0 0x56433f7a0bd0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebec8c3e3e2a56f server_cookie=75c4642f0a0ac3a1 in_seq=0 out_seq=0 2024-04-25T14:13:38.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.727+0000 7ff43f7fe640 1 -- 172.21.15.96:0/2805517846 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56433f6fc710 con 0x56433f3a71f0 2024-04-25T14:13:38.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.727+0000 7ff43f7fe640 10 client.15480 handle_command_reply: tid=0 2024-04-25T14:13:38.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.727+0000 7ff46d3c3740 1 client.15480 shutdown 2024-04-25T14:13:38.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.727+0000 7ff46d3c3740 1 -- 172.21.15.96:0/2805517846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433f3e8410 msgr2=0x56433f79a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:38.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.727+0000 7ff46d3c3740 1 --2- 172.21.15.96:0/2805517846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433f3e8410 0x56433f79a6a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff454011bf0 tx=0x7ff454004730 comp rx=0 tx=0).stop 2024-04-25T14:13:38.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.727+0000 7ff46d3c3740 1 -- 172.21.15.96:0/2805517846 shutdown_connections 2024-04-25T14:13:38.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.727+0000 7ff46d3c3740 1 --2- 172.21.15.96:0/2805517846 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x56433f3a71f0 0x56433f7a0bd0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.727+0000 7ff46d3c3740 1 --2- 172.21.15.96:0/2805517846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433f3e8410 0x56433f79a6a0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.727+0000 7ff46d3c3740 1 --2- 172.21.15.96:0/2805517846 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56433f3c7730 0x56433f793740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.727+0000 7ff46d3c3740 1 --2- 172.21.15.96:0/2805517846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56433f3bf550 0x56433f793200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.727+0000 7ff43f7fe640 0 client.15480 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:13:38.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.727+0000 7ff46d3c3740 1 -- 172.21.15.96:0/2805517846 >> 172.21.15.96:0/2805517846 conn(0x56433f3dc560 msgr2=0x56433f783ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:38.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.727+0000 7ff46d3c3740 1 -- 172.21.15.96:0/2805517846 shutdown_connections 2024-04-25T14:13:38.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.727+0000 7ff46d3c3740 1 -- 172.21.15.96:0/2805517846 wait complete. 2024-04-25T14:13:38.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.727+0000 7ff46d3c3740 20 client.15480 trim_cache size 0 max 16384 2024-04-25T14:13:38.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.745+0000 7ff46d3c3740 1 Processor -- start 2024-04-25T14:13:38.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.745+0000 7ff46d3c3740 1 -- start start 2024-04-25T14:13:38.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.745+0000 7ff46d3c3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56433f3bf550 0x56433f7a5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.745+0000 7ff46d3c3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433f3c7730 0x56433f7a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.745+0000 7ff46d3c3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56433f3e8410 0x56433f7a5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.746+0000 7ff46d3c3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff4601a9030 con 0x56433f3c7730 2024-04-25T14:13:38.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.746+0000 7ff46d3c3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff4601aab70 con 0x56433f3e8410 2024-04-25T14:13:38.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.746+0000 7ff46d3c3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff4601a9700 con 0x56433f3bf550 2024-04-25T14:13:38.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.746+0000 7ff45ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433f3c7730 0x56433f7a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.746+0000 7ff45ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433f3c7730 0x56433f7a58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40744/0 (socket says 172.21.15.96:40744) 2024-04-25T14:13:38.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.746+0000 7ff45ffff640 1 -- 172.21.15.96:0/1372429036 learned_addr learned my addr 172.21.15.96:0/1372429036 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:38.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.746+0000 7ff45f7fe640 1 --2- 172.21.15.96:0/1372429036 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56433f3bf550 0x56433f7a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.746+0000 7ff45f7fe640 1 --2- 172.21.15.96:0/1372429036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56433f3e8410 0x56433f7a5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.746+0000 7ff45ffff640 1 -- 172.21.15.96:0/1372429036 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56433f3bf550 msgr2=0x56433f7a5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:38.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.746+0000 7ff45ffff640 1 --2- 172.21.15.96:0/1372429036 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56433f3bf550 0x56433f7a5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.746+0000 7ff45ffff640 1 -- 172.21.15.96:0/1372429036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56433f3e8410 msgr2=0x56433f7a5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:38.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.746+0000 7ff45ffff640 1 --2- 172.21.15.96:0/1372429036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56433f3e8410 0x56433f7a5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.746+0000 7ff45ffff640 1 -- 172.21.15.96:0/1372429036 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56433f791250 con 0x56433f3c7730 2024-04-25T14:13:38.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.746+0000 7ff45f7fe640 1 --2- 172.21.15.96:0/1372429036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56433f3e8410 0x56433f7a5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:38.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.746+0000 7ff45ffff640 1 --2- 172.21.15.96:0/1372429036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433f3c7730 0x56433f7a58c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff450009250 tx=0x7ff450009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b232cd3181bef99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:38.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.746+0000 7ff445ffb640 1 -- 172.21.15.96:0/1372429036 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45004f020 con 0x56433f3c7730 2024-04-25T14:13:38.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.746+0000 7ff445ffb640 1 -- 172.21.15.96:0/1372429036 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4500042d0 con 0x56433f3c7730 2024-04-25T14:13:38.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.746+0000 7ff445ffb640 1 -- 172.21.15.96:0/1372429036 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45003e040 con 0x56433f3c7730 2024-04-25T14:13:38.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.747+0000 7ff46d3c3740 1 -- 172.21.15.96:0/1372429036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433f3c7730 msgr2=0x56433f7a58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:38.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.747+0000 7ff46d3c3740 1 --2- 172.21.15.96:0/1372429036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433f3c7730 0x56433f7a58c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff450009250 tx=0x7ff450009680 comp rx=0 tx=0).stop 2024-04-25T14:13:38.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.747+0000 7ff46d3c3740 1 -- 172.21.15.96:0/1372429036 shutdown_connections 2024-04-25T14:13:38.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.747+0000 7ff46d3c3740 1 --2- 172.21.15.96:0/1372429036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56433f3e8410 0x56433f7a5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.747+0000 7ff46d3c3740 1 --2- 172.21.15.96:0/1372429036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433f3c7730 0x56433f7a58c0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.747+0000 7ff46d3c3740 1 --2- 172.21.15.96:0/1372429036 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56433f3bf550 0x56433f7a5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.747+0000 7ff46d3c3740 1 -- 172.21.15.96:0/1372429036 >> 172.21.15.96:0/1372429036 conn(0x56433f3dc560 msgr2=0x56433f78d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:38.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.747+0000 7ff46d3c3740 1 -- 172.21.15.96:0/1372429036 shutdown_connections 2024-04-25T14:13:38.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.747+0000 7ff46d3c3740 1 -- 172.21.15.96:0/1372429036 wait complete. 2024-04-25T14:13:38.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.748+0000 7ff46d3c3740 1 -- start start 2024-04-25T14:13:38.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.748+0000 7ff46d3c3740 1 Processor -- start 2024-04-25T14:13:38.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.748+0000 7ff46d3c3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433f3bf550 0x56433f797a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.748+0000 7ff46d3c3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56433f3c7730 0x56433f797f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.748+0000 7ff46d3c3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56433f3e8410 0x56433f798eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.748+0000 7ff46d3c3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff4601aa4a0 con 0x56433f3bf550 2024-04-25T14:13:38.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.748+0000 7ff46d3c3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff4601a9dd0 con 0x56433f3e8410 2024-04-25T14:13:38.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.748+0000 7ff46d3c3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff4601a7010 con 0x56433f3c7730 2024-04-25T14:13:38.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.748+0000 7ff46d3c3740 10 client.0 _ll_register_callbacks cb 0x7ff454011a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:13:38.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.748+0000 7ff45f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433f3bf550 0x56433f797a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.748+0000 7ff45f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433f3bf550 0x56433f797a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40758/0 (socket says 172.21.15.96:40758) 2024-04-25T14:13:38.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.748+0000 7ff45f7fe640 1 -- 172.21.15.96:0/2964699184 learned_addr learned my addr 172.21.15.96:0/2964699184 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:38.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.748+0000 7ff45ffff640 1 --2- 172.21.15.96:0/2964699184 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56433f3c7730 0x56433f797f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.748+0000 7ff45f7fe640 1 --2- 172.21.15.96:0/2964699184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56433f3e8410 0x56433f798eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.749+0000 7ff45f7fe640 1 -- 172.21.15.96:0/2964699184 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56433f3c7730 msgr2=0x56433f797f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:38.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.749+0000 7ff45f7fe640 1 --2- 172.21.15.96:0/2964699184 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56433f3c7730 0x56433f797f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.749+0000 7ff45f7fe640 1 -- 172.21.15.96:0/2964699184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56433f3e8410 msgr2=0x56433f798eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:38.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.749+0000 7ff45f7fe640 1 --2- 172.21.15.96:0/2964699184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56433f3e8410 0x56433f798eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.749+0000 7ff45f7fe640 1 -- 172.21.15.96:0/2964699184 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56433f799470 con 0x56433f3bf550 2024-04-25T14:13:38.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.749+0000 7ff45f7fe640 1 --2- 172.21.15.96:0/2964699184 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433f3bf550 0x56433f797a40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff454011bc0 tx=0x7ff454004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e960c968fd0eb4e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:38.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.749+0000 7ff43f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:13:38.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.749+0000 7ff43f7fe640 1 -- 172.21.15.96:0/2964699184 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff454005d30 con 0x56433f3bf550 2024-04-25T14:13:38.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.749+0000 7ff43f7fe640 1 -- 172.21.15.96:0/2964699184 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7ff4540040f0 con 0x56433f3bf550 2024-04-25T14:13:38.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.749+0000 7ff46d3c3740 1 -- 172.21.15.96:0/2964699184 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4500042d0 con 0x56433f3bf550 2024-04-25T14:13:38.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.749+0000 7ff43f7fe640 1 -- 172.21.15.96:0/2964699184 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff454041050 con 0x56433f3bf550 2024-04-25T14:13:38.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.749+0000 7ff46d3c3740 10 client.15486 fetch_fsmap learned FSMap version 26 2024-04-25T14:13:38.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.749+0000 7ff46d3c3740 1 -- 172.21.15.96:0/2964699184 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56433f809390 con 0x56433f3bf550 2024-04-25T14:13:38.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.749+0000 7ff43f7fe640 1 -- 172.21.15.96:0/2964699184 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4500042d0 con 0x56433f3bf550 2024-04-25T14:13:38.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.750+0000 7ff43f7fe640 1 -- 172.21.15.96:0/2964699184 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff45402b620 con 0x56433f3bf550 2024-04-25T14:13:38.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.750+0000 7ff46d3c3740 10 client.15486 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:13:38.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.750+0000 7ff46d3c3740 10 client.15486 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:13:38.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.750+0000 7ff46d3c3740 20 client.15486 populate_metadata read hostname 'smithi096' 2024-04-25T14:13:38.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.750+0000 7ff46d3c3740 1 --2- 172.21.15.96:0/2964699184 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x56433f37c630 0x56433f80bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.750+0000 7ff46d3c3740 4 client.15486 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T14:13:38.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.750+0000 7ff46d3c3740 1 -- 172.21.15.96:0/2964699184 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56433f6f6580 con 0x56433f37c630 2024-04-25T14:13:38.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.750+0000 7ff45ffff640 1 --2- 172.21.15.96:0/2964699184 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x56433f37c630 0x56433f80bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.750+0000 7ff45ffff640 1 --2- 172.21.15.96:0/2964699184 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x56433f37c630 0x56433f80bcb0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f95d7890e9d755d server_cookie=f28f8e95937c133 in_seq=0 out_seq=0 2024-04-25T14:13:38.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.750+0000 7ff43f7fe640 10 client.15486 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:13:38.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.751+0000 7ff43f7fe640 1 -- 172.21.15.96:0/2964699184 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56433f6f6580 con 0x56433f37c630 2024-04-25T14:13:38.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.751+0000 7ff43f7fe640 10 client.15486 handle_command_reply: tid=0 2024-04-25T14:13:38.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.751+0000 7ff46d3c3740 1 client.15486 shutdown 2024-04-25T14:13:38.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.752+0000 7ff46d3c3740 1 -- 172.21.15.96:0/2964699184 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433f3bf550 msgr2=0x56433f797a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:38.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.752+0000 7ff46d3c3740 1 --2- 172.21.15.96:0/2964699184 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433f3bf550 0x56433f797a40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff454011bc0 tx=0x7ff454004730 comp rx=0 tx=0).stop 2024-04-25T14:13:38.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.752+0000 7ff46d3c3740 1 -- 172.21.15.96:0/2964699184 shutdown_connections 2024-04-25T14:13:38.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.752+0000 7ff46d3c3740 1 --2- 172.21.15.96:0/2964699184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56433f3e8410 0x56433f798eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.752+0000 7ff46d3c3740 1 --2- 172.21.15.96:0/2964699184 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56433f3c7730 0x56433f797f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.752+0000 7ff46d3c3740 1 --2- 172.21.15.96:0/2964699184 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x56433f37c630 0x56433f80bcb0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.752+0000 7ff46d3c3740 1 --2- 172.21.15.96:0/2964699184 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433f3bf550 0x56433f797a40 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.752+0000 7ff46d3c3740 1 -- 172.21.15.96:0/2964699184 >> 172.21.15.96:0/2964699184 conn(0x56433f3dc560 msgr2=0x56433f787c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:38.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.752+0000 7ff43f7fe640 0 client.15486 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:13:38.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.752+0000 7ff46d3c3740 1 -- 172.21.15.96:0/2964699184 shutdown_connections 2024-04-25T14:13:38.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.752+0000 7ff46d3c3740 1 -- 172.21.15.96:0/2964699184 wait complete. 2024-04-25T14:13:38.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.752+0000 7ff46d3c3740 20 client.15486 trim_cache size 0 max 16384 2024-04-25T14:13:38.757 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T14:13:38.757 INFO:teuthology.orchestra.run.smithi096.stdout: "return_code": 0, 2024-04-25T14:13:38.757 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_tag": "37a9c54e-98a0-4e20-9f73-8dd63f5a0d7e", 2024-04-25T14:13:38.757 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": "asynchronous" 2024-04-25T14:13:38.757 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T14:13:38.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.753+0000 7ff46650d640 1 -- 172.21.15.96:0/1216040073 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ff4480afb70 msgr2=0x7ff4480b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:38.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.753+0000 7ff46650d640 1 --2- 172.21.15.96:0/1216040073 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ff4480afb70 0x7ff4480b2080 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff4580abeb0 tx=0x7ff45807c040 comp rx=0 tx=0).stop 2024-04-25T14:13:38.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.753+0000 7ff46650d640 1 -- 172.21.15.96:0/1216040073 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff460108920 msgr2=0x7ff4601a1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:38.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.753+0000 7ff46650d640 1 --2- 172.21.15.96:0/1216040073 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff460108920 0x7ff4601a1870 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff458007440 tx=0x7ff458007900 comp rx=0 tx=0).stop 2024-04-25T14:13:38.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.753+0000 7ff46650d640 1 -- 172.21.15.96:0/1216040073 shutdown_connections 2024-04-25T14:13:38.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.753+0000 7ff46650d640 1 --2- 172.21.15.96:0/1216040073 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ff4480afb70 0x7ff4480b2080 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.753+0000 7ff46650d640 1 --2- 172.21.15.96:0/1216040073 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff460108920 0x7ff4601a1870 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.753+0000 7ff46650d640 1 --2- 172.21.15.96:0/1216040073 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff460103c20 0x7ff46019f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.753+0000 7ff46650d640 1 --2- 172.21.15.96:0/1216040073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4601029e0 0x7ff46006a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.753+0000 7ff46650d640 1 -- 172.21.15.96:0/1216040073 >> 172.21.15.96:0/1216040073 conn(0x7ff4600fdbd0 msgr2=0x7ff460104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:38.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.753+0000 7ff46650d640 1 -- 172.21.15.96:0/1216040073 shutdown_connections 2024-04-25T14:13:38.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.753+0000 7ff46650d640 1 -- 172.21.15.96:0/1216040073 wait complete. 2024-04-25T14:13:38.759 INFO:tasks.workunit.client.0.smithi096.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-25T14:13:38.763 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:13:38.763 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:13:38.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:38 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/3186044197' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:13:38.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:38 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/19164621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:13:38.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:38 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/3836224282' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:13:38.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.920+0000 7f43497ce640 1 -- 172.21.15.96:0/2229057631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43441029e0 msgr2=0x7f43441036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:38.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.920+0000 7f43497ce640 1 --2- 172.21.15.96:0/2229057631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43441029e0 0x7f43441036e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4334009900 tx=0x7f433402a410 comp rx=0 tx=0).stop 2024-04-25T14:13:38.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.920+0000 7f43497ce640 1 -- 172.21.15.96:0/2229057631 shutdown_connections 2024-04-25T14:13:38.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.920+0000 7f43497ce640 1 --2- 172.21.15.96:0/2229057631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4344108920 0x7f434410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.920+0000 7f43497ce640 1 --2- 172.21.15.96:0/2229057631 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4344103c20 0x7f4344104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.920+0000 7f43497ce640 1 --2- 172.21.15.96:0/2229057631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43441029e0 0x7f43441036e0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.920+0000 7f43497ce640 1 -- 172.21.15.96:0/2229057631 >> 172.21.15.96:0/2229057631 conn(0x7f43440fdbd0 msgr2=0x7f4344100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:38.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.920+0000 7f43497ce640 1 -- 172.21.15.96:0/2229057631 shutdown_connections 2024-04-25T14:13:38.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.920+0000 7f43497ce640 1 -- 172.21.15.96:0/2229057631 wait complete. 2024-04-25T14:13:38.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.921+0000 7f43497ce640 1 Processor -- start 2024-04-25T14:13:38.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.921+0000 7f43497ce640 1 -- start start 2024-04-25T14:13:38.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.921+0000 7f43497ce640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43441029e0 0x7f4344075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.921+0000 7f43497ce640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4344103c20 0x7f434419f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.921+0000 7f43497ce640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4344108920 0x7f43441a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.921+0000 7f43497ce640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f434410e4e0 con 0x7f4344103c20 2024-04-25T14:13:38.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.921+0000 7f43497ce640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f434410e360 con 0x7f43441029e0 2024-04-25T14:13:38.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.921+0000 7f43497ce640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f434410e660 con 0x7f4344108920 2024-04-25T14:13:38.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.921+0000 7f433bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4344103c20 0x7f434419f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.921+0000 7f433bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4344103c20 0x7f434419f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40766/0 (socket says 172.21.15.96:40766) 2024-04-25T14:13:38.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.921+0000 7f433bfff640 1 -- 172.21.15.96:0/1899616738 learned_addr learned my addr 172.21.15.96:0/1899616738 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:38.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.921+0000 7f4342ffd640 1 --2- 172.21.15.96:0/1899616738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43441029e0 0x7f4344075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.921+0000 7f433bfff640 1 -- 172.21.15.96:0/1899616738 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4344108920 msgr2=0x7f43441a15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:38.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.921+0000 7f433bfff640 1 --2- 172.21.15.96:0/1899616738 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4344108920 0x7f43441a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.921+0000 7f433bfff640 1 -- 172.21.15.96:0/1899616738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43441029e0 msgr2=0x7f4344075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:38.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.921+0000 7f433bfff640 1 --2- 172.21.15.96:0/1899616738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43441029e0 0x7f4344075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.921+0000 7f433bfff640 1 -- 172.21.15.96:0/1899616738 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43441a1dc0 con 0x7f4344103c20 2024-04-25T14:13:38.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.921+0000 7f433bfff640 1 --2- 172.21.15.96:0/1899616738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4344103c20 0x7f434419f050 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f432c00d770 tx=0x7f432c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:38.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.921+0000 7f4340ff9640 1 -- 172.21.15.96:0/1899616738 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432c017070 con 0x7f4344103c20 2024-04-25T14:13:38.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.922+0000 7f43497ce640 1 -- 172.21.15.96:0/1899616738 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43441a20b0 con 0x7f4344103c20 2024-04-25T14:13:38.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.922+0000 7f43497ce640 1 -- 172.21.15.96:0/1899616738 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43441a2600 con 0x7f4344103c20 2024-04-25T14:13:38.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.922+0000 7f4340ff9640 1 -- 172.21.15.96:0/1899616738 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f432c0054d0 con 0x7f4344103c20 2024-04-25T14:13:38.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.922+0000 7f4340ff9640 1 -- 172.21.15.96:0/1899616738 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432c020060 con 0x7f4344103c20 2024-04-25T14:13:38.925 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:38 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/3186044197' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:13:38.925 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:38 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/19164621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:13:38.925 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:38 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/3836224282' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:13:38.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.924+0000 7f4340ff9640 1 -- 172.21.15.96:0/1899616738 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f432c009c30 con 0x7f4344103c20 2024-04-25T14:13:38.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.924+0000 7f4350682740 1 Processor -- start 2024-04-25T14:13:38.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.924+0000 7f4350682740 1 -- start start 2024-04-25T14:13:38.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.924+0000 7f4350682740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56374baa5390 0x56374be46a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.924+0000 7f4350682740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56374ba7c4d0 0x56374be46fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.925+0000 7f4350682740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374ba846b0 0x56374be47c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.925+0000 7f4350682740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56374bdabd80 con 0x56374ba846b0 2024-04-25T14:13:38.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.925+0000 7f4350682740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56374bd658b0 con 0x56374ba7c4d0 2024-04-25T14:13:38.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.925+0000 7f4350682740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56374bda7d40 con 0x56374baa5390 2024-04-25T14:13:38.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.925+0000 7f433bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374ba846b0 0x56374be47c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.925+0000 7f433bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374ba846b0 0x56374be47c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40782/0 (socket says 172.21.15.96:40782) 2024-04-25T14:13:38.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.925+0000 7f433bfff640 1 -- 172.21.15.96:0/1450749552 learned_addr learned my addr 172.21.15.96:0/1450749552 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:38.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.925+0000 7f43437fe640 1 --2- 172.21.15.96:0/1450749552 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56374baa5390 0x56374be46a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.925+0000 7f4342ffd640 1 --2- 172.21.15.96:0/1450749552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56374ba7c4d0 0x56374be46fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.925+0000 7f4340ff9640 1 --2- 172.21.15.96:0/1899616738 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f43300afaf0 0x7f43300b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.925+0000 7f43437fe640 1 -- 172.21.15.96:0/1450749552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56374ba7c4d0 msgr2=0x56374be46fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:38.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.925+0000 7f43437fe640 1 --2- 172.21.15.96:0/1450749552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56374ba7c4d0 0x56374be46fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.925+0000 7f43437fe640 1 -- 172.21.15.96:0/1450749552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374ba846b0 msgr2=0x56374be47c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:38.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.925+0000 7f43437fe640 1 --2- 172.21.15.96:0/1450749552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374ba846b0 0x56374be47c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.925+0000 7f43437fe640 1 -- 172.21.15.96:0/1450749552 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56374bd36a40 con 0x56374baa5390 2024-04-25T14:13:38.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.925+0000 7f433bfff640 1 --2- 172.21.15.96:0/1450749552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374ba846b0 0x56374be47c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:13:38.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.925+0000 7f4340ff9640 1 -- 172.21.15.96:0/1899616738 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f432c0de700 con 0x7f4344103c20 2024-04-25T14:13:38.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.925+0000 7f43437fe640 1 --2- 172.21.15.96:0/1899616738 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f43300afaf0 0x7f43300b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.925+0000 7f43437fe640 1 --2- 172.21.15.96:0/1450749552 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56374baa5390 0x56374be46a60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f433c009e40 tx=0x7f433c00e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b032062487ad5de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:38.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.926+0000 7f43397fa640 1 -- 172.21.15.96:0/1450749552 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433c003dd0 con 0x56374baa5390 2024-04-25T14:13:38.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.926+0000 7f43437fe640 1 --2- 172.21.15.96:0/1899616738 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f43300afaf0 0x7f43300b2000 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f433c01f000 tx=0x7f433c020000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:38.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.926+0000 7f43397fa640 1 -- 172.21.15.96:0/1450749552 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f433c011910 con 0x56374baa5390 2024-04-25T14:13:38.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.926+0000 7f43397fa640 1 -- 172.21.15.96:0/1450749552 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433c01a650 con 0x56374baa5390 2024-04-25T14:13:38.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.926+0000 7f4350682740 1 -- 172.21.15.96:0/1450749552 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56374baa5390 msgr2=0x56374be46a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:38.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.926+0000 7f4350682740 1 --2- 172.21.15.96:0/1450749552 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56374baa5390 0x56374be46a60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f433c009e40 tx=0x7f433c00e6c0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.927+0000 7f4350682740 1 -- 172.21.15.96:0/1450749552 shutdown_connections 2024-04-25T14:13:38.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.927+0000 7f4350682740 1 --2- 172.21.15.96:0/1450749552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374ba846b0 0x56374be47c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.927+0000 7f4350682740 1 --2- 172.21.15.96:0/1450749552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56374ba7c4d0 0x56374be46fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.927+0000 7f4350682740 1 --2- 172.21.15.96:0/1450749552 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56374baa5390 0x56374be46a60 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.927+0000 7f4350682740 1 -- 172.21.15.96:0/1450749552 >> 172.21.15.96:0/1450749552 conn(0x56374ba994e0 msgr2=0x56374be42270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:38.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.927+0000 7f4350682740 1 -- 172.21.15.96:0/1450749552 shutdown_connections 2024-04-25T14:13:38.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.927+0000 7f4350682740 1 -- 172.21.15.96:0/1450749552 wait complete. 2024-04-25T14:13:38.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.927+0000 7f4350682740 1 -- start start 2024-04-25T14:13:38.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.927+0000 7f4350682740 1 Processor -- start 2024-04-25T14:13:38.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.927+0000 7f4350682740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374ba7c4d0 0x56374be50910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.927+0000 7f4350682740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56374ba846b0 0x56374be50e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.927+0000 7f4350682740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56374baa5390 0x56374be57db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.927+0000 7f4350682740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56374be48780 con 0x56374ba7c4d0 2024-04-25T14:13:38.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.927+0000 7f4350682740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56374bd32d10 con 0x56374ba846b0 2024-04-25T14:13:38.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.927+0000 7f4350682740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56374be48900 con 0x56374baa5390 2024-04-25T14:13:38.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.927+0000 7f4350682740 10 client.0 _ll_register_callbacks cb 0x56374bd5f210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:13:38.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.927+0000 7f4342ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374ba7c4d0 0x56374be50910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.927+0000 7f4342ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374ba7c4d0 0x56374be50910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40796/0 (socket says 172.21.15.96:40796) 2024-04-25T14:13:38.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.927+0000 7f4342ffd640 1 -- 172.21.15.96:0/2902542276 learned_addr learned my addr 172.21.15.96:0/2902542276 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:38.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.928+0000 7f433bfff640 1 --2- 172.21.15.96:0/2902542276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56374ba846b0 0x56374be50e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.928+0000 7f43437fe640 1 --2- 172.21.15.96:0/2902542276 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56374baa5390 0x56374be57db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.928+0000 7f4342ffd640 1 -- 172.21.15.96:0/2902542276 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56374baa5390 msgr2=0x56374be57db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:38.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.928+0000 7f4342ffd640 1 --2- 172.21.15.96:0/2902542276 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56374baa5390 0x56374be57db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.928+0000 7f4342ffd640 1 -- 172.21.15.96:0/2902542276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56374ba846b0 msgr2=0x56374be50e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:38.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.928+0000 7f4342ffd640 1 --2- 172.21.15.96:0/2902542276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56374ba846b0 0x56374be50e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.928+0000 7f4342ffd640 1 -- 172.21.15.96:0/2902542276 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56374bb877d0 con 0x56374ba7c4d0 2024-04-25T14:13:38.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.928+0000 7f43437fe640 1 --2- 172.21.15.96:0/2902542276 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56374baa5390 0x56374be57db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:38.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.928+0000 7f4342ffd640 1 --2- 172.21.15.96:0/2902542276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374ba7c4d0 0x56374be50910 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f4334005940 tx=0x7f43340317d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc9134296fae77d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:38.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.928+0000 7f432a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:13:38.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.928+0000 7f432a7fc640 1 -- 172.21.15.96:0/2902542276 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4334035040 con 0x56374ba7c4d0 2024-04-25T14:13:38.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.928+0000 7f432a7fc640 1 -- 172.21.15.96:0/2902542276 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f4334004240 con 0x56374ba7c4d0 2024-04-25T14:13:38.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.928+0000 7f4350682740 1 -- 172.21.15.96:0/2902542276 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f433c011910 con 0x56374ba7c4d0 2024-04-25T14:13:38.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.928+0000 7f432a7fc640 1 -- 172.21.15.96:0/2902542276 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4334004440 con 0x56374ba7c4d0 2024-04-25T14:13:38.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.928+0000 7f4350682740 10 client.15498 fetch_fsmap learned FSMap version 26 2024-04-25T14:13:38.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.928+0000 7f4350682740 1 -- 172.21.15.96:0/2902542276 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56374babd3f0 con 0x56374ba7c4d0 2024-04-25T14:13:38.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.928+0000 7f432a7fc640 1 -- 172.21.15.96:0/2902542276 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f433c011910 con 0x56374ba7c4d0 2024-04-25T14:13:38.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.929+0000 7f432a7fc640 1 -- 172.21.15.96:0/2902542276 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4334004aa0 con 0x56374ba7c4d0 2024-04-25T14:13:38.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.929+0000 7f4350682740 10 client.15498 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:13:38.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.929+0000 7f4350682740 10 client.15498 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:13:38.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.929+0000 7f4350682740 20 client.15498 populate_metadata read hostname 'smithi096' 2024-04-25T14:13:38.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.929+0000 7f4350682740 1 --2- 172.21.15.96:0/2902542276 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x56374ba63f60 0x56374be5e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.929+0000 7f4350682740 4 client.15498 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:13:38.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.929+0000 7f4350682740 1 -- 172.21.15.96:0/2902542276 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56374bdb9990 con 0x56374ba63f60 2024-04-25T14:13:38.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.929+0000 7f433bfff640 1 --2- 172.21.15.96:0/2902542276 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x56374ba63f60 0x56374be5e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.930+0000 7f432a7fc640 10 client.15498 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:13:38.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.930+0000 7f433bfff640 1 --2- 172.21.15.96:0/2902542276 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x56374ba63f60 0x56374be5e2f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d53be7c34b0e05a server_cookie=1bb2b21572c8d8c7 in_seq=0 out_seq=0 2024-04-25T14:13:38.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.930+0000 7f432a7fc640 1 -- 172.21.15.96:0/2902542276 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56374bdb9990 con 0x56374ba63f60 2024-04-25T14:13:38.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.930+0000 7f432a7fc640 10 client.15498 handle_command_reply: tid=0 2024-04-25T14:13:38.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.930+0000 7f4350682740 1 client.15498 shutdown 2024-04-25T14:13:38.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.930+0000 7f4350682740 1 -- 172.21.15.96:0/2902542276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374ba7c4d0 msgr2=0x56374be50910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:38.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.930+0000 7f4350682740 1 --2- 172.21.15.96:0/2902542276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374ba7c4d0 0x56374be50910 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f4334005940 tx=0x7f43340317d0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.930+0000 7f4350682740 1 -- 172.21.15.96:0/2902542276 shutdown_connections 2024-04-25T14:13:38.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.930+0000 7f4350682740 1 --2- 172.21.15.96:0/2902542276 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56374baa5390 0x56374be57db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.930+0000 7f4350682740 1 --2- 172.21.15.96:0/2902542276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56374ba846b0 0x56374be50e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.930+0000 7f4350682740 1 --2- 172.21.15.96:0/2902542276 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x56374ba63f60 0x56374be5e2f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.930+0000 7f4350682740 1 --2- 172.21.15.96:0/2902542276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374ba7c4d0 0x56374be50910 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.930+0000 7f432a7fc640 0 client.15498 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:13:38.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.930+0000 7f4350682740 1 -- 172.21.15.96:0/2902542276 >> 172.21.15.96:0/2902542276 conn(0x56374ba994e0 msgr2=0x56374be45180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:38.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.931+0000 7f4350682740 1 -- 172.21.15.96:0/2902542276 shutdown_connections 2024-04-25T14:13:38.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.931+0000 7f4350682740 1 -- 172.21.15.96:0/2902542276 wait complete. 2024-04-25T14:13:38.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.931+0000 7f4350682740 20 client.15498 trim_cache size 0 max 16384 2024-04-25T14:13:38.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:38 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/3186044197' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:13:38.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:38 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/19164621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:13:38.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:38 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/3836224282' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:13:38.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.952+0000 7f4350682740 1 Processor -- start 2024-04-25T14:13:38.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.952+0000 7f4350682740 1 -- start start 2024-04-25T14:13:38.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.952+0000 7f4350682740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56374ba7c4d0 0x56374be4b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.952+0000 7f4350682740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374ba846b0 0x56374be42310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.952+0000 7f4350682740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56374baa5390 0x56374be4f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.952+0000 7f4350682740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f432400e530 con 0x56374ba846b0 2024-04-25T14:13:38.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.952+0000 7f4350682740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4324010070 con 0x56374ba7c4d0 2024-04-25T14:13:38.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.952+0000 7f4350682740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f432400ec00 con 0x56374baa5390 2024-04-25T14:13:38.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.952+0000 7f433bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374ba846b0 0x56374be42310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.952+0000 7f433bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374ba846b0 0x56374be42310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40812/0 (socket says 172.21.15.96:40812) 2024-04-25T14:13:38.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.952+0000 7f433bfff640 1 -- 172.21.15.96:0/135743620 learned_addr learned my addr 172.21.15.96:0/135743620 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:38.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.952+0000 7f433bfff640 1 -- 172.21.15.96:0/135743620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56374baa5390 msgr2=0x56374be4f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:38.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.952+0000 7f433bfff640 1 --2- 172.21.15.96:0/135743620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56374baa5390 0x56374be4f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.952+0000 7f433bfff640 1 -- 172.21.15.96:0/135743620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56374ba7c4d0 msgr2=0x56374be4b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:38.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.952+0000 7f433bfff640 1 --2- 172.21.15.96:0/135743620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56374ba7c4d0 0x56374be4b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.952+0000 7f433bfff640 1 -- 172.21.15.96:0/135743620 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56374ba7d0f0 con 0x56374ba846b0 2024-04-25T14:13:38.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.953+0000 7f433bfff640 1 --2- 172.21.15.96:0/135743620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374ba846b0 0x56374be42310 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f432c0057a0 tx=0x7f432c0e0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19449c4569ce1ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:38.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.953+0000 7f4338ff9640 1 -- 172.21.15.96:0/135743620 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432c0a6510 con 0x56374ba846b0 2024-04-25T14:13:38.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.953+0000 7f4338ff9640 1 -- 172.21.15.96:0/135743620 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f432c0af360 con 0x56374ba846b0 2024-04-25T14:13:38.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.953+0000 7f4338ff9640 1 -- 172.21.15.96:0/135743620 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432c0a6330 con 0x56374ba846b0 2024-04-25T14:13:38.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.954+0000 7f4350682740 1 -- 172.21.15.96:0/135743620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374ba846b0 msgr2=0x56374be42310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:38.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.954+0000 7f4350682740 1 --2- 172.21.15.96:0/135743620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374ba846b0 0x56374be42310 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f432c0057a0 tx=0x7f432c0e0040 comp rx=0 tx=0).stop 2024-04-25T14:13:38.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.954+0000 7f4350682740 1 -- 172.21.15.96:0/135743620 shutdown_connections 2024-04-25T14:13:38.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.954+0000 7f4350682740 1 --2- 172.21.15.96:0/135743620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56374baa5390 0x56374be4f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.954+0000 7f4350682740 1 --2- 172.21.15.96:0/135743620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374ba846b0 0x56374be42310 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.954+0000 7f4350682740 1 --2- 172.21.15.96:0/135743620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56374ba7c4d0 0x56374be4b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.954+0000 7f4350682740 1 -- 172.21.15.96:0/135743620 >> 172.21.15.96:0/135743620 conn(0x56374ba994e0 msgr2=0x56374be4ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:38.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.954+0000 7f4350682740 1 -- 172.21.15.96:0/135743620 shutdown_connections 2024-04-25T14:13:38.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.954+0000 7f4350682740 1 -- 172.21.15.96:0/135743620 wait complete. 2024-04-25T14:13:38.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.954+0000 7f4350682740 1 -- start start 2024-04-25T14:13:38.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.954+0000 7f4350682740 1 Processor -- start 2024-04-25T14:13:38.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.954+0000 7f4350682740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56374ba7c4d0 0x56374be54ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.954+0000 7f4350682740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56374ba846b0 0x56374be55010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.954+0000 7f4350682740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374baa5390 0x56374be55e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.954+0000 7f4350682740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f432400f9a0 con 0x56374baa5390 2024-04-25T14:13:38.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.954+0000 7f4350682740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f432400f2d0 con 0x56374ba846b0 2024-04-25T14:13:38.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.954+0000 7f4350682740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f432400c510 con 0x56374ba7c4d0 2024-04-25T14:13:38.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.954+0000 7f4350682740 10 client.0 _ll_register_callbacks cb 0x7f4334005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:13:38.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.954+0000 7f43437fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374baa5390 0x56374be55e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.954+0000 7f43437fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374baa5390 0x56374be55e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40820/0 (socket says 172.21.15.96:40820) 2024-04-25T14:13:38.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.954+0000 7f43437fe640 1 -- 172.21.15.96:0/4060805777 learned_addr learned my addr 172.21.15.96:0/4060805777 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:38.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.955+0000 7f4342ffd640 1 --2- 172.21.15.96:0/4060805777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56374ba7c4d0 0x56374be54ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.955+0000 7f433bfff640 1 --2- 172.21.15.96:0/4060805777 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56374ba846b0 0x56374be55010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.955+0000 7f43437fe640 1 -- 172.21.15.96:0/4060805777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56374ba7c4d0 msgr2=0x56374be54ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:38.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.955+0000 7f43437fe640 1 --2- 172.21.15.96:0/4060805777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56374ba7c4d0 0x56374be54ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.955+0000 7f43437fe640 1 -- 172.21.15.96:0/4060805777 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56374ba846b0 msgr2=0x56374be55010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:38.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.955+0000 7f43437fe640 1 --2- 172.21.15.96:0/4060805777 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56374ba846b0 0x56374be55010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.955+0000 7f43437fe640 1 -- 172.21.15.96:0/4060805777 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56374bd6c4b0 con 0x56374baa5390 2024-04-25T14:13:38.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.955+0000 7f433bfff640 1 --2- 172.21.15.96:0/4060805777 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56374ba846b0 0x56374be55010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:38.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.955+0000 7f4342ffd640 1 --2- 172.21.15.96:0/4060805777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56374ba7c4d0 0x56374be54ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:38.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.955+0000 7f43437fe640 1 --2- 172.21.15.96:0/4060805777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374baa5390 0x56374be55e80 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f433c009c70 tx=0x7f433c025ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9dfa6e46d0fd6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:38.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.955+0000 7f432a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:13:38.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.955+0000 7f432a7fc640 1 -- 172.21.15.96:0/4060805777 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433c033020 con 0x56374baa5390 2024-04-25T14:13:38.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.955+0000 7f432a7fc640 1 -- 172.21.15.96:0/4060805777 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f433c025d90 con 0x56374baa5390 2024-04-25T14:13:38.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.955+0000 7f4350682740 1 -- 172.21.15.96:0/4060805777 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56374bdb3800 con 0x56374baa5390 2024-04-25T14:13:38.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.955+0000 7f432a7fc640 1 -- 172.21.15.96:0/4060805777 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f433c009970 con 0x56374baa5390 2024-04-25T14:13:38.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.955+0000 7f4350682740 10 client.15504 fetch_fsmap learned FSMap version 26 2024-04-25T14:13:38.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.955+0000 7f4350682740 1 -- 172.21.15.96:0/4060805777 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56374be57530 con 0x56374baa5390 2024-04-25T14:13:38.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.955+0000 7f432a7fc640 1 -- 172.21.15.96:0/4060805777 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56374bdb3800 con 0x56374baa5390 2024-04-25T14:13:38.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.955+0000 7f432a7fc640 1 -- 172.21.15.96:0/4060805777 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f433c00f040 con 0x56374baa5390 2024-04-25T14:13:38.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.956+0000 7f4350682740 10 client.15504 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:13:38.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.956+0000 7f4350682740 10 client.15504 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:13:38.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.956+0000 7f4350682740 20 client.15504 populate_metadata read hostname 'smithi096' 2024-04-25T14:13:38.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.956+0000 7f4350682740 1 --2- 172.21.15.96:0/4060805777 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x56374ba393a0 0x56374bec8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.956+0000 7f4350682740 4 client.15504 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:13:38.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.956+0000 7f4350682740 1 -- 172.21.15.96:0/4060805777 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56374bd2e4a0 con 0x56374ba393a0 2024-04-25T14:13:38.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.956+0000 7f4342ffd640 1 --2- 172.21.15.96:0/4060805777 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x56374ba393a0 0x56374bec8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.957+0000 7f432a7fc640 10 client.15504 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:13:38.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.957+0000 7f4342ffd640 1 --2- 172.21.15.96:0/4060805777 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x56374ba393a0 0x56374bec8a40 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9df07ef5b87f02a9 server_cookie=6cc7811508bac6e5 in_seq=0 out_seq=0 2024-04-25T14:13:38.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.957+0000 7f432a7fc640 1 -- 172.21.15.96:0/4060805777 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56374bd2e4a0 con 0x56374ba393a0 2024-04-25T14:13:38.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.957+0000 7f432a7fc640 10 client.15504 handle_command_reply: tid=0 2024-04-25T14:13:38.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.957+0000 7f4350682740 1 client.15504 shutdown 2024-04-25T14:13:38.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.957+0000 7f4350682740 1 -- 172.21.15.96:0/4060805777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374baa5390 msgr2=0x56374be55e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:38.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.957+0000 7f4350682740 1 --2- 172.21.15.96:0/4060805777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374baa5390 0x56374be55e80 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f433c009c70 tx=0x7f433c025ba0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.957+0000 7f4350682740 1 -- 172.21.15.96:0/4060805777 shutdown_connections 2024-04-25T14:13:38.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.957+0000 7f4350682740 1 --2- 172.21.15.96:0/4060805777 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x56374ba393a0 0x56374bec8a40 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.957+0000 7f4350682740 1 --2- 172.21.15.96:0/4060805777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374baa5390 0x56374be55e80 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.957+0000 7f4350682740 1 --2- 172.21.15.96:0/4060805777 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56374ba846b0 0x56374be55010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.957+0000 7f432a7fc640 0 client.15504 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:13:38.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.957+0000 7f4350682740 1 --2- 172.21.15.96:0/4060805777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56374ba7c4d0 0x56374be54ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.957+0000 7f4350682740 1 -- 172.21.15.96:0/4060805777 >> 172.21.15.96:0/4060805777 conn(0x56374ba994e0 msgr2=0x56374be50c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:38.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.957+0000 7f4350682740 1 -- 172.21.15.96:0/4060805777 shutdown_connections 2024-04-25T14:13:38.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.957+0000 7f4350682740 1 -- 172.21.15.96:0/4060805777 wait complete. 2024-04-25T14:13:38.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.957+0000 7f4350682740 20 client.15504 trim_cache size 0 max 16384 2024-04-25T14:13:38.962 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T14:13:38.962 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "no active scrubs running", 2024-04-25T14:13:38.963 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": {} 2024-04-25T14:13:38.963 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T14:13:38.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.958+0000 7f43497ce640 1 -- 172.21.15.96:0/1899616738 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f43300afaf0 msgr2=0x7f43300b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:38.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.958+0000 7f43497ce640 1 --2- 172.21.15.96:0/1899616738 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f43300afaf0 0x7f43300b2000 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f433c01f000 tx=0x7f433c020000 comp rx=0 tx=0).stop 2024-04-25T14:13:38.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.958+0000 7f43497ce640 1 -- 172.21.15.96:0/1899616738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4344103c20 msgr2=0x7f434419f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:38.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.958+0000 7f43497ce640 1 --2- 172.21.15.96:0/1899616738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4344103c20 0x7f434419f050 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f432c00d770 tx=0x7f432c00dc30 comp rx=0 tx=0).stop 2024-04-25T14:13:38.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.958+0000 7f43497ce640 1 -- 172.21.15.96:0/1899616738 shutdown_connections 2024-04-25T14:13:38.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.958+0000 7f43497ce640 1 --2- 172.21.15.96:0/1899616738 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4344108920 0x7f43441a15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.958+0000 7f43497ce640 1 --2- 172.21.15.96:0/1899616738 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f43300afaf0 0x7f43300b2000 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.958+0000 7f43497ce640 1 --2- 172.21.15.96:0/1899616738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4344103c20 0x7f434419f050 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.958+0000 7f43497ce640 1 --2- 172.21.15.96:0/1899616738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43441029e0 0x7f4344075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.958+0000 7f43497ce640 1 -- 172.21.15.96:0/1899616738 >> 172.21.15.96:0/1899616738 conn(0x7f43440fdbd0 msgr2=0x7f4344104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:38.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.958+0000 7f43497ce640 1 -- 172.21.15.96:0/1899616738 shutdown_connections 2024-04-25T14:13:38.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:38.958+0000 7f43497ce640 1 -- 172.21.15.96:0/1899616738 wait complete. 2024-04-25T14:13:38.968 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T14:13:38.968 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:13:39.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.078+0000 7f149d5f3640 1 -- 172.21.15.96:0/1125002059 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1498103c00 msgr2=0x7f1498104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.078+0000 7f149d5f3640 1 --2- 172.21.15.96:0/1125002059 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1498103c00 0x7f1498104010 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f1480009900 tx=0x7f148002a410 comp rx=0 tx=0).stop 2024-04-25T14:13:39.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.078+0000 7f149d5f3640 1 -- 172.21.15.96:0/1125002059 shutdown_connections 2024-04-25T14:13:39.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.078+0000 7f149d5f3640 1 --2- 172.21.15.96:0/1125002059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1498108900 0x7f149810ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.078+0000 7f149d5f3640 1 --2- 172.21.15.96:0/1125002059 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1498103c00 0x7f1498104010 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.078+0000 7f149d5f3640 1 --2- 172.21.15.96:0/1125002059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14981029c0 0x7f14981036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.078+0000 7f149d5f3640 1 -- 172.21.15.96:0/1125002059 >> 172.21.15.96:0/1125002059 conn(0x7f14980fdbd0 msgr2=0x7f1498100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:39.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.078+0000 7f149d5f3640 1 -- 172.21.15.96:0/1125002059 shutdown_connections 2024-04-25T14:13:39.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.078+0000 7f149d5f3640 1 -- 172.21.15.96:0/1125002059 wait complete. 2024-04-25T14:13:39.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.078+0000 7f149d5f3640 1 Processor -- start 2024-04-25T14:13:39.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.079+0000 7f149d5f3640 1 -- start start 2024-04-25T14:13:39.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.079+0000 7f149d5f3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14981029c0 0x7f1498199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.079+0000 7f149d5f3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1498103c00 0x7f1498199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.079+0000 7f149d5f3640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1498108900 0x7f14981a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.079+0000 7f149d5f3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f149810e4c0 con 0x7f1498103c00 2024-04-25T14:13:39.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.079+0000 7f149d5f3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f149810e340 con 0x7f14981029c0 2024-04-25T14:13:39.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.079+0000 7f149d5f3640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f149810e640 con 0x7f1498108900 2024-04-25T14:13:39.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.079+0000 7f14967fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1498103c00 0x7f1498199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.079+0000 7f14967fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1498103c00 0x7f1498199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40838/0 (socket says 172.21.15.96:40838) 2024-04-25T14:13:39.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.079+0000 7f14967fc640 1 -- 172.21.15.96:0/3232318215 learned_addr learned my addr 172.21.15.96:0/3232318215 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:39.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.079+0000 7f14977fe640 1 --2- 172.21.15.96:0/3232318215 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1498108900 0x7f14981a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.079+0000 7f1496ffd640 1 --2- 172.21.15.96:0/3232318215 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14981029c0 0x7f1498199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.079+0000 7f14967fc640 1 -- 172.21.15.96:0/3232318215 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1498108900 msgr2=0x7f14981a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.079+0000 7f14967fc640 1 --2- 172.21.15.96:0/3232318215 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1498108900 0x7f14981a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.079+0000 7f14967fc640 1 -- 172.21.15.96:0/3232318215 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14981029c0 msgr2=0x7f1498199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.079+0000 7f14967fc640 1 --2- 172.21.15.96:0/3232318215 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14981029c0 0x7f1498199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.079+0000 7f14967fc640 1 -- 172.21.15.96:0/3232318215 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14981a0bb0 con 0x7f1498103c00 2024-04-25T14:13:39.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.079+0000 7f14977fe640 1 --2- 172.21.15.96:0/3232318215 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1498108900 0x7f14981a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:39.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.079+0000 7f1496ffd640 1 --2- 172.21.15.96:0/3232318215 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14981029c0 0x7f1498199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:39.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.079+0000 7f14967fc640 1 --2- 172.21.15.96:0/3232318215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1498103c00 0x7f1498199660 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f1480006010 tx=0x7f14800043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:39.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.080+0000 7f148ffff640 1 -- 172.21.15.96:0/3232318215 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f148003f030 con 0x7f1498103c00 2024-04-25T14:13:39.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.080+0000 7f148ffff640 1 -- 172.21.15.96:0/3232318215 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1480003cb0 con 0x7f1498103c00 2024-04-25T14:13:39.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.080+0000 7f149d5f3640 1 -- 172.21.15.96:0/3232318215 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14981a0e40 con 0x7f1498103c00 2024-04-25T14:13:39.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.080+0000 7f148ffff640 1 -- 172.21.15.96:0/3232318215 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1480035070 con 0x7f1498103c00 2024-04-25T14:13:39.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.080+0000 7f149d5f3640 1 -- 172.21.15.96:0/3232318215 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14981a12a0 con 0x7f1498103c00 2024-04-25T14:13:39.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.081+0000 7f149d5f3640 1 -- 172.21.15.96:0/3232318215 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1498107710 con 0x7f1498103c00 2024-04-25T14:13:39.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.083+0000 7f148ffff640 1 -- 172.21.15.96:0/3232318215 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f1480042030 con 0x7f1498103c00 2024-04-25T14:13:39.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.083+0000 7f148ffff640 1 --2- 172.21.15.96:0/3232318215 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f14840afb70 0x7f14840b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.084+0000 7f1496ffd640 1 --2- 172.21.15.96:0/3232318215 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f14840afb70 0x7f14840b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.084+0000 7f148ffff640 1 -- 172.21.15.96:0/3232318215 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f14800ffa50 con 0x7f1498103c00 2024-04-25T14:13:39.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.084+0000 7f1496ffd640 1 --2- 172.21.15.96:0/3232318215 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f14840afb70 0x7f14840b2080 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f14981035c0 tx=0x7f149000a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:39.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.087+0000 7f148ffff640 1 -- 172.21.15.96:0/3232318215 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f14800c8610 con 0x7f1498103c00 2024-04-25T14:13:39.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.356+0000 7f149d5f3640 1 -- 172.21.15.96:0/3232318215 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f149802db50 con 0x7f1498103c00 2024-04-25T14:13:39.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.357+0000 7f148ffff640 1 -- 172.21.15.96:0/3232318215 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11927 (secure 0 0 0) 0x7f14800c7f10 con 0x7f1498103c00 2024-04-25T14:13:39.356 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:13:39.356 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/606227917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":606227917},{"type":"v1","addr":"172.21.15.96:6835","nonce":606227917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/3265300710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3265300710},{"type":"v1","addr":"172.21.15.96:6837","nonce":3265300710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T14:11:28.886961+0000","modified":"2024-04-25T14:13:32.120297+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24517,"mds_2":24478,"mds_3":24455,"mds_4":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/4080308509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":4080308509},{"type":"v1","addr":"172.21.15.96:6839","nonce":4080308509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/1606748971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":1606748971},{"type":"v1","addr":"172.21.15.96:6841","nonce":1606748971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6833/1995324831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1995324831},{"type":"v1","addr":"172.21.15.97:6833","nonce":1995324831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":4,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.151:6833/1382721608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1382721608},{"type":"v1","addr":"172.21.15.151:6833","nonce":1382721608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6835/3872301726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3872301726},{"type":"v1","addr":"172.21.15.97:6835","nonce":3872301726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.151:6835/1313823638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":1313823638},{"type":"v1","addr":"172.21.15.151:6835","nonce":1313823638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/2691348366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2691348366},{"type":"v1","addr":"172.21.15.151:6837","nonce":2691348366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3306780081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3306780081},{"type":"v1","addr":"172.21.15.97:6837","nonce":3306780081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.151:6839/245699981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":245699981},{"type":"v1","addr":"172.21.15.151:6839","nonce":245699981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6839/2850442269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":2850442269},{"type":"v1","addr":"172.21.15.97:6839","nonce":2850442269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24427,24455,24478,24517,24536]},"id":1}]} 2024-04-25T14:13:39.356 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 26 2024-04-25T14:13:39.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.359+0000 7f149d5f3640 1 -- 172.21.15.96:0/3232318215 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f14840afb70 msgr2=0x7f14840b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.359+0000 7f149d5f3640 1 --2- 172.21.15.96:0/3232318215 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f14840afb70 0x7f14840b2080 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f14981035c0 tx=0x7f149000a560 comp rx=0 tx=0).stop 2024-04-25T14:13:39.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.359+0000 7f149d5f3640 1 -- 172.21.15.96:0/3232318215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1498103c00 msgr2=0x7f1498199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.359+0000 7f149d5f3640 1 --2- 172.21.15.96:0/3232318215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1498103c00 0x7f1498199660 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f1480006010 tx=0x7f14800043b0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.359+0000 7f149d5f3640 1 -- 172.21.15.96:0/3232318215 shutdown_connections 2024-04-25T14:13:39.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.359+0000 7f149d5f3640 1 --2- 172.21.15.96:0/3232318215 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1498108900 0x7f14981a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.359+0000 7f149d5f3640 1 --2- 172.21.15.96:0/3232318215 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f14840afb70 0x7f14840b2080 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.359+0000 7f149d5f3640 1 --2- 172.21.15.96:0/3232318215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1498103c00 0x7f1498199660 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.359+0000 7f149d5f3640 1 --2- 172.21.15.96:0/3232318215 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14981029c0 0x7f1498199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.359+0000 7f149d5f3640 1 -- 172.21.15.96:0/3232318215 >> 172.21.15.96:0/3232318215 conn(0x7f14980fdbd0 msgr2=0x7f14981048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:39.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.359+0000 7f149d5f3640 1 -- 172.21.15.96:0/3232318215 shutdown_connections 2024-04-25T14:13:39.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.359+0000 7f149d5f3640 1 -- 172.21.15.96:0/3232318215 wait complete. 2024-04-25T14:13:39.370 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T14:13:39.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.480+0000 7ff6d2ed1640 1 -- 172.21.15.96:0/2519931060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6cc1029e0 msgr2=0x7ff6cc1036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.480+0000 7ff6d2ed1640 1 --2- 172.21.15.96:0/2519931060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6cc1029e0 0x7ff6cc1036e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0009900 tx=0x7ff6c002a410 comp rx=0 tx=0).stop 2024-04-25T14:13:39.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.480+0000 7ff6d2ed1640 1 -- 172.21.15.96:0/2519931060 shutdown_connections 2024-04-25T14:13:39.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.480+0000 7ff6d2ed1640 1 --2- 172.21.15.96:0/2519931060 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6cc108920 0x7ff6cc10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.480+0000 7ff6d2ed1640 1 --2- 172.21.15.96:0/2519931060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6cc103c20 0x7ff6cc104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.480+0000 7ff6d2ed1640 1 --2- 172.21.15.96:0/2519931060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6cc1029e0 0x7ff6cc1036e0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.480+0000 7ff6d2ed1640 1 -- 172.21.15.96:0/2519931060 >> 172.21.15.96:0/2519931060 conn(0x7ff6cc0fdbd0 msgr2=0x7ff6cc100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:39.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.480+0000 7ff6d2ed1640 1 -- 172.21.15.96:0/2519931060 shutdown_connections 2024-04-25T14:13:39.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.480+0000 7ff6d2ed1640 1 -- 172.21.15.96:0/2519931060 wait complete. 2024-04-25T14:13:39.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.481+0000 7ff6d2ed1640 1 Processor -- start 2024-04-25T14:13:39.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.481+0000 7ff6d2ed1640 1 -- start start 2024-04-25T14:13:39.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.481+0000 7ff6d2ed1640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6cc1029e0 0x7ff6cc199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.481+0000 7ff6d2ed1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6cc103c20 0x7ff6cc1997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.481+0000 7ff6d2ed1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6cc108920 0x7ff6cc1a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.481+0000 7ff6d2ed1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff6cc10e4e0 con 0x7ff6cc108920 2024-04-25T14:13:39.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.481+0000 7ff6d2ed1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff6cc10e360 con 0x7ff6cc103c20 2024-04-25T14:13:39.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.481+0000 7ff6d2ed1640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff6cc10e660 con 0x7ff6cc1029e0 2024-04-25T14:13:39.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.481+0000 7ff6d146e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6cc108920 0x7ff6cc1a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.481+0000 7ff6d146e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6cc108920 0x7ff6cc1a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40850/0 (socket says 172.21.15.96:40850) 2024-04-25T14:13:39.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.481+0000 7ff6d146e640 1 -- 172.21.15.96:0/4127672094 learned_addr learned my addr 172.21.15.96:0/4127672094 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:39.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.481+0000 7ff6d0c6d640 1 --2- 172.21.15.96:0/4127672094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6cc1029e0 0x7ff6cc199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.481+0000 7ff6d146e640 1 -- 172.21.15.96:0/4127672094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6cc1029e0 msgr2=0x7ff6cc199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.481+0000 7ff6d146e640 1 --2- 172.21.15.96:0/4127672094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6cc1029e0 0x7ff6cc199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.481+0000 7ff6d146e640 1 -- 172.21.15.96:0/4127672094 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6cc103c20 msgr2=0x7ff6cc1997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.481+0000 7ff6d146e640 1 --2- 172.21.15.96:0/4127672094 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6cc103c20 0x7ff6cc1997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.481+0000 7ff6d146e640 1 -- 172.21.15.96:0/4127672094 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6cc1a0c40 con 0x7ff6cc108920 2024-04-25T14:13:39.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.481+0000 7ff6d146e640 1 --2- 172.21.15.96:0/4127672094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6cc108920 0x7ff6cc1a0680 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff6c400eca0 tx=0x7ff6c400b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:39.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.481+0000 7ff6c9ffb640 1 -- 172.21.15.96:0/4127672094 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4004b20 con 0x7ff6cc108920 2024-04-25T14:13:39.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.482+0000 7ff6c9ffb640 1 -- 172.21.15.96:0/4127672094 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6c4004cc0 con 0x7ff6cc108920 2024-04-25T14:13:39.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.482+0000 7ff6c9ffb640 1 -- 172.21.15.96:0/4127672094 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4017670 con 0x7ff6cc108920 2024-04-25T14:13:39.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.482+0000 7ff6d2ed1640 1 -- 172.21.15.96:0/4127672094 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6cc074e30 con 0x7ff6cc108920 2024-04-25T14:13:39.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.482+0000 7ff6d2ed1640 1 -- 172.21.15.96:0/4127672094 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6cc075380 con 0x7ff6cc108920 2024-04-25T14:13:39.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.485+0000 7ff6d9d7c740 1 Processor -- start 2024-04-25T14:13:39.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.485+0000 7ff6c9ffb640 1 -- 172.21.15.96:0/4127672094 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7ff6c4005020 con 0x7ff6cc108920 2024-04-25T14:13:39.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.485+0000 7ff6d9d7c740 1 -- start start 2024-04-25T14:13:39.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.485+0000 7ff6d9d7c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583d2d17390 0x5583d30b9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.485+0000 7ff6d9d7c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583d2cee4d0 0x5583d30b9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.485+0000 7ff6d9d7c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d2cf66b0 0x5583d30baa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.485+0000 7ff6d9d7c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5583d301dd80 con 0x5583d2cf66b0 2024-04-25T14:13:39.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.485+0000 7ff6d9d7c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5583d2fd7a20 con 0x5583d2cee4d0 2024-04-25T14:13:39.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.485+0000 7ff6d9d7c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5583d3019d40 con 0x5583d2d17390 2024-04-25T14:13:39.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.485+0000 7ff6cbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d2cf66b0 0x5583d30baa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.485+0000 7ff6cbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d2cf66b0 0x5583d30baa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40856/0 (socket says 172.21.15.96:40856) 2024-04-25T14:13:39.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.485+0000 7ff6cbfff640 1 -- 172.21.15.96:0/1270914853 learned_addr learned my addr 172.21.15.96:0/1270914853 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:39.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.485+0000 7ff6d0c6d640 1 --2- 172.21.15.96:0/1270914853 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583d2cee4d0 0x5583d30b9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.485+0000 7ff6c9ffb640 1 --2- 172.21.15.96:0/4127672094 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ff6bc0afaf0 0x7ff6bc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.485+0000 7ff6cbfff640 1 --2- 172.21.15.96:0/1270914853 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583d2d17390 0x5583d30b9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.486+0000 7ff6d146e640 1 --2- 172.21.15.96:0/4127672094 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ff6bc0afaf0 0x7ff6bc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.486+0000 7ff6c9ffb640 1 -- 172.21.15.96:0/4127672094 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7ff6c40e0800 con 0x7ff6cc108920 2024-04-25T14:13:39.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.486+0000 7ff6cbfff640 1 -- 172.21.15.96:0/1270914853 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583d2d17390 msgr2=0x5583d30b9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:39.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.486+0000 7ff6cbfff640 1 --2- 172.21.15.96:0/1270914853 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583d2d17390 0x5583d30b9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.486+0000 7ff6cbfff640 1 -- 172.21.15.96:0/1270914853 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583d2cee4d0 msgr2=0x5583d30b9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:39.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.486+0000 7ff6cbfff640 1 --2- 172.21.15.96:0/1270914853 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583d2cee4d0 0x5583d30b9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.486+0000 7ff6cbfff640 1 -- 172.21.15.96:0/1270914853 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583d2fe2c30 con 0x5583d2cf66b0 2024-04-25T14:13:39.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.486+0000 7ff6cbfff640 1 --2- 172.21.15.96:0/1270914853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d2cf66b0 0x5583d30baa20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8005fd0 tx=0x7ff6b800c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af8ab546b8e10d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:39.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.486+0000 7ff6d146e640 1 --2- 172.21.15.96:0/4127672094 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ff6bc0afaf0 0x7ff6bc0b2000 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff6c40a4d40 tx=0x7ff6c4090040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:39.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.486+0000 7ff6b6ffd640 1 -- 172.21.15.96:0/1270914853 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b8004220 con 0x5583d2cf66b0 2024-04-25T14:13:39.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.486+0000 7ff6b6ffd640 1 -- 172.21.15.96:0/1270914853 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6b80043c0 con 0x5583d2cf66b0 2024-04-25T14:13:39.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.486+0000 7ff6b6ffd640 1 -- 172.21.15.96:0/1270914853 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b8009990 con 0x5583d2cf66b0 2024-04-25T14:13:39.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.487+0000 7ff6d9d7c740 1 -- 172.21.15.96:0/1270914853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d2cf66b0 msgr2=0x5583d30baa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.487+0000 7ff6d9d7c740 1 --2- 172.21.15.96:0/1270914853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d2cf66b0 0x5583d30baa20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8005fd0 tx=0x7ff6b800c310 comp rx=0 tx=0).stop 2024-04-25T14:13:39.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.487+0000 7ff6d9d7c740 1 -- 172.21.15.96:0/1270914853 shutdown_connections 2024-04-25T14:13:39.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.487+0000 7ff6d9d7c740 1 --2- 172.21.15.96:0/1270914853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d2cf66b0 0x5583d30baa20 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.487+0000 7ff6d9d7c740 1 --2- 172.21.15.96:0/1270914853 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583d2cee4d0 0x5583d30b9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.487+0000 7ff6d9d7c740 1 --2- 172.21.15.96:0/1270914853 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583d2d17390 0x5583d30b9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.487+0000 7ff6d9d7c740 1 -- 172.21.15.96:0/1270914853 >> 172.21.15.96:0/1270914853 conn(0x5583d2d0b4e0 msgr2=0x5583d30b5020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:39.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.487+0000 7ff6d9d7c740 1 -- 172.21.15.96:0/1270914853 shutdown_connections 2024-04-25T14:13:39.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.487+0000 7ff6d9d7c740 1 -- 172.21.15.96:0/1270914853 wait complete. 2024-04-25T14:13:39.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.487+0000 7ff6d9d7c740 1 -- start start 2024-04-25T14:13:39.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.488+0000 7ff6d9d7c740 1 Processor -- start 2024-04-25T14:13:39.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.488+0000 7ff6d9d7c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d2cee4d0 0x5583d30c3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.488+0000 7ff6d9d7c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583d2cf66b0 0x5583d30c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.488+0000 7ff6d9d7c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583d2d17390 0x5583d30ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.488+0000 7ff6d9d7c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5583d30bb530 con 0x5583d2cee4d0 2024-04-25T14:13:39.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.488+0000 7ff6d9d7c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5583d2fa4dc0 con 0x5583d2cf66b0 2024-04-25T14:13:39.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.488+0000 7ff6d9d7c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5583d30bb6b0 con 0x5583d2d17390 2024-04-25T14:13:39.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.488+0000 7ff6d9d7c740 10 client.0 _ll_register_callbacks cb 0x5583d2fc0600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:13:39.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.488+0000 7ff6cbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d2cee4d0 0x5583d30c3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.488+0000 7ff6cbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d2cee4d0 0x5583d30c3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40870/0 (socket says 172.21.15.96:40870) 2024-04-25T14:13:39.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.488+0000 7ff6cbfff640 1 -- 172.21.15.96:0/910528791 learned_addr learned my addr 172.21.15.96:0/910528791 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:39.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.488+0000 7ff6d0c6d640 1 --2- 172.21.15.96:0/910528791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583d2cf66b0 0x5583d30c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.488+0000 7ff6cbfff640 1 -- 172.21.15.96:0/910528791 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583d2d17390 msgr2=0x5583d30ca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:39.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.488+0000 7ff6cbfff640 1 --2- 172.21.15.96:0/910528791 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583d2d17390 0x5583d30ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.488+0000 7ff6cbfff640 1 -- 172.21.15.96:0/910528791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583d2cf66b0 msgr2=0x5583d30c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:39.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.488+0000 7ff6cbfff640 1 --2- 172.21.15.96:0/910528791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583d2cf66b0 0x5583d30c3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.488+0000 7ff6cbfff640 1 -- 172.21.15.96:0/910528791 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583d2d2f3f0 con 0x5583d2cee4d0 2024-04-25T14:13:39.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.489+0000 7ff6cbfff640 1 --2- 172.21.15.96:0/910528791 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d2cee4d0 0x5583d30c3510 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8004610 tx=0x7ff6b8005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=393bb02f9669fa3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:39.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.489+0000 7ff6abfff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:13:39.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.489+0000 7ff6abfff640 1 -- 172.21.15.96:0/910528791 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b8023020 con 0x5583d2cee4d0 2024-04-25T14:13:39.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.489+0000 7ff6abfff640 1 -- 172.21.15.96:0/910528791 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7ff6b8009b40 con 0x5583d2cee4d0 2024-04-25T14:13:39.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.489+0000 7ff6d9d7c740 1 -- 172.21.15.96:0/910528791 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6b80043c0 con 0x5583d2cee4d0 2024-04-25T14:13:39.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.489+0000 7ff6abfff640 1 -- 172.21.15.96:0/910528791 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff6b802e050 con 0x5583d2cee4d0 2024-04-25T14:13:39.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.489+0000 7ff6d9d7c740 10 client.15522 fetch_fsmap learned FSMap version 26 2024-04-25T14:13:39.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.489+0000 7ff6d9d7c740 1 -- 172.21.15.96:0/910528791 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5583d2df97d0 con 0x5583d2cee4d0 2024-04-25T14:13:39.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.489+0000 7ff6abfff640 1 -- 172.21.15.96:0/910528791 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6b80043c0 con 0x5583d2cee4d0 2024-04-25T14:13:39.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.489+0000 7ff6abfff640 1 -- 172.21.15.96:0/910528791 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff6b800f040 con 0x5583d2cee4d0 2024-04-25T14:13:39.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.489+0000 7ff6d9d7c740 10 client.15522 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:13:39.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.489+0000 7ff6d9d7c740 10 client.15522 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-25T14:13:39.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.489+0000 7ff6d9d7c740 20 client.15522 populate_metadata read hostname 'smithi096' 2024-04-25T14:13:39.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.489+0000 7ff6d9d7c740 1 --2- 172.21.15.96:0/910528791 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x5583d2cd5f60 0x5583d30d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.489+0000 7ff6d9d7c740 4 client.15522 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:13:39.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.489+0000 7ff6d9d7c740 1 -- 172.21.15.96:0/910528791 --> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583d302b990 con 0x5583d2cd5f60 2024-04-25T14:13:39.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.490+0000 7ff6d0c6d640 1 --2- 172.21.15.96:0/910528791 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x5583d2cd5f60 0x5583d30d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.490+0000 7ff6abfff640 10 client.15522 ms_handle_connect on v2:172.21.15.151:6832/1382721608 2024-04-25T14:13:39.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.490+0000 7ff6d0c6d640 1 --2- 172.21.15.96:0/910528791 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x5583d2cd5f60 0x5583d30d0f10 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=238a61c71609c047 server_cookie=9826f0e2e888323b in_seq=0 out_seq=0 2024-04-25T14:13:39.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.491+0000 7ff6abfff640 1 -- 172.21.15.96:0/910528791 <== mds.4 v2:172.21.15.151:6832/1382721608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5583d302b990 con 0x5583d2cd5f60 2024-04-25T14:13:39.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.491+0000 7ff6abfff640 10 client.15522 handle_command_reply: tid=0 2024-04-25T14:13:39.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.491+0000 7ff6d9d7c740 1 client.15522 shutdown 2024-04-25T14:13:39.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.491+0000 7ff6d9d7c740 1 -- 172.21.15.96:0/910528791 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d2cee4d0 msgr2=0x5583d30c3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.491+0000 7ff6d9d7c740 1 --2- 172.21.15.96:0/910528791 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d2cee4d0 0x5583d30c3510 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8004610 tx=0x7ff6b8005bc0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.491+0000 7ff6d9d7c740 1 -- 172.21.15.96:0/910528791 shutdown_connections 2024-04-25T14:13:39.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.491+0000 7ff6d9d7c740 1 --2- 172.21.15.96:0/910528791 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x5583d2cd5f60 0x5583d30d0f10 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.491+0000 7ff6d9d7c740 1 --2- 172.21.15.96:0/910528791 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583d2d17390 0x5583d30ca9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.491+0000 7ff6abfff640 0 client.15522 ms_handle_reset on v2:172.21.15.151:6832/1382721608 2024-04-25T14:13:39.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.491+0000 7ff6d9d7c740 1 --2- 172.21.15.96:0/910528791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583d2cf66b0 0x5583d30c3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.491+0000 7ff6d9d7c740 1 --2- 172.21.15.96:0/910528791 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d2cee4d0 0x5583d30c3510 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.491+0000 7ff6d9d7c740 1 -- 172.21.15.96:0/910528791 >> 172.21.15.96:0/910528791 conn(0x5583d2d0b4e0 msgr2=0x5583d30b7f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:39.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.491+0000 7ff6d9d7c740 1 -- 172.21.15.96:0/910528791 shutdown_connections 2024-04-25T14:13:39.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.491+0000 7ff6d9d7c740 1 -- 172.21.15.96:0/910528791 wait complete. 2024-04-25T14:13:39.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.491+0000 7ff6d9d7c740 20 client.15522 trim_cache size 0 max 16384 2024-04-25T14:13:39.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.505+0000 7ff6d9d7c740 1 Processor -- start 2024-04-25T14:13:39.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.505+0000 7ff6d9d7c740 1 -- start start 2024-04-25T14:13:39.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.505+0000 7ff6d9d7c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d2cd5f60 0x5583d30b4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.505+0000 7ff6d9d7c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583d2cee4d0 0x5583d30ca940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.505+0000 7ff6d9d7c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583d2cf66b0 0x5583d30b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.505+0000 7ff6d9d7c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff6cc1a91e0 con 0x5583d2cd5f60 2024-04-25T14:13:39.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.505+0000 7ff6d9d7c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff6cc1aad20 con 0x5583d2cf66b0 2024-04-25T14:13:39.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.505+0000 7ff6d9d7c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff6cc1a98b0 con 0x5583d2cee4d0 2024-04-25T14:13:39.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.505+0000 7ff6cbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d2cd5f60 0x5583d30b4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.505+0000 7ff6cbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d2cd5f60 0x5583d30b4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40876/0 (socket says 172.21.15.96:40876) 2024-04-25T14:13:39.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.505+0000 7ff6cbfff640 1 -- 172.21.15.96:0/3727610171 learned_addr learned my addr 172.21.15.96:0/3727610171 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:39.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.505+0000 7ff6cbfff640 1 -- 172.21.15.96:0/3727610171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583d2cee4d0 msgr2=0x5583d30ca940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:39.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.505+0000 7ff6cbfff640 1 --2- 172.21.15.96:0/3727610171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583d2cee4d0 0x5583d30ca940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.505+0000 7ff6cbfff640 1 -- 172.21.15.96:0/3727610171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583d2cf66b0 msgr2=0x5583d30b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:39.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.505+0000 7ff6cbfff640 1 --2- 172.21.15.96:0/3727610171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583d2cf66b0 0x5583d30b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.505+0000 7ff6cbfff640 1 -- 172.21.15.96:0/3727610171 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583d30b93d0 con 0x5583d2cd5f60 2024-04-25T14:13:39.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.505+0000 7ff6cbfff640 1 --2- 172.21.15.96:0/3727610171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d2cd5f60 0x5583d30b4a50 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff6b801dad0 tx=0x7ff6b8013730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c594c5c6406659 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:39.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.505+0000 7ff6b67fc640 1 -- 172.21.15.96:0/3727610171 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b8023040 con 0x5583d2cd5f60 2024-04-25T14:13:39.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.506+0000 7ff6b67fc640 1 -- 172.21.15.96:0/3727610171 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6b801d460 con 0x5583d2cd5f60 2024-04-25T14:13:39.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.506+0000 7ff6b67fc640 1 -- 172.21.15.96:0/3727610171 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b80035f0 con 0x5583d2cd5f60 2024-04-25T14:13:39.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.506+0000 7ff6d9d7c740 1 -- 172.21.15.96:0/3727610171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d2cd5f60 msgr2=0x5583d30b4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.506+0000 7ff6d9d7c740 1 --2- 172.21.15.96:0/3727610171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d2cd5f60 0x5583d30b4a50 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff6b801dad0 tx=0x7ff6b8013730 comp rx=0 tx=0).stop 2024-04-25T14:13:39.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.506+0000 7ff6d9d7c740 1 -- 172.21.15.96:0/3727610171 shutdown_connections 2024-04-25T14:13:39.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.506+0000 7ff6d9d7c740 1 --2- 172.21.15.96:0/3727610171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583d2cf66b0 0x5583d30b4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.506+0000 7ff6d9d7c740 1 --2- 172.21.15.96:0/3727610171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583d2cee4d0 0x5583d30ca940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.506+0000 7ff6d9d7c740 1 --2- 172.21.15.96:0/3727610171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d2cd5f60 0x5583d30b4a50 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.506+0000 7ff6d9d7c740 1 -- 172.21.15.96:0/3727610171 >> 172.21.15.96:0/3727610171 conn(0x5583d2d0b4e0 msgr2=0x5583d30be270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:39.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.506+0000 7ff6d9d7c740 1 -- 172.21.15.96:0/3727610171 shutdown_connections 2024-04-25T14:13:39.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.506+0000 7ff6d9d7c740 1 -- 172.21.15.96:0/3727610171 wait complete. 2024-04-25T14:13:39.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.507+0000 7ff6d9d7c740 1 -- start start 2024-04-25T14:13:39.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.507+0000 7ff6d9d7c740 1 Processor -- start 2024-04-25T14:13:39.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.507+0000 7ff6d9d7c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d2cd5f60 0x5583d30ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.507+0000 7ff6d9d7c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583d2cee4d0 0x5583d30c7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.507+0000 7ff6d9d7c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583d2cf66b0 0x5583d30c7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.507+0000 7ff6d9d7c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff6cc1aa650 con 0x5583d2cd5f60 2024-04-25T14:13:39.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.507+0000 7ff6d9d7c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff6cc1a9f80 con 0x5583d2cf66b0 2024-04-25T14:13:39.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.507+0000 7ff6d9d7c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff6cc1a71c0 con 0x5583d2cee4d0 2024-04-25T14:13:39.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.507+0000 7ff6d9d7c740 10 client.0 _ll_register_callbacks cb 0x7ff6c40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:13:39.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.507+0000 7ff6cbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d2cd5f60 0x5583d30ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.507+0000 7ff6cbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d2cd5f60 0x5583d30ca8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40892/0 (socket says 172.21.15.96:40892) 2024-04-25T14:13:39.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.507+0000 7ff6cbfff640 1 -- 172.21.15.96:0/4266407164 learned_addr learned my addr 172.21.15.96:0/4266407164 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:39.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.507+0000 7ff6cbfff640 1 --2- 172.21.15.96:0/4266407164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583d2cf66b0 0x5583d30c7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.507+0000 7ff6d0c6d640 1 --2- 172.21.15.96:0/4266407164 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583d2cee4d0 0x5583d30c7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.508+0000 7ff6cbfff640 1 -- 172.21.15.96:0/4266407164 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583d2cee4d0 msgr2=0x5583d30c7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:39.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.508+0000 7ff6cbfff640 1 --2- 172.21.15.96:0/4266407164 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583d2cee4d0 0x5583d30c7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.508+0000 7ff6cbfff640 1 -- 172.21.15.96:0/4266407164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583d2cf66b0 msgr2=0x5583d30c7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:39.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.508+0000 7ff6cbfff640 1 --2- 172.21.15.96:0/4266407164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583d2cf66b0 0x5583d30c7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.508+0000 7ff6cbfff640 1 -- 172.21.15.96:0/4266407164 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583d3137180 con 0x5583d2cd5f60 2024-04-25T14:13:39.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.508+0000 7ff6cbfff640 1 --2- 172.21.15.96:0/4266407164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d2cd5f60 0x5583d30ca8c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff6b800cc80 tx=0x7ff6b801d820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4e1ce4e35da42dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:39.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.508+0000 7ff6abfff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:13:39.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.508+0000 7ff6abfff640 1 -- 172.21.15.96:0/4266407164 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b8009d50 con 0x5583d2cd5f60 2024-04-25T14:13:39.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.508+0000 7ff6abfff640 1 -- 172.21.15.96:0/4266407164 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7ff6b80035f0 con 0x5583d2cd5f60 2024-04-25T14:13:39.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.508+0000 7ff6d9d7c740 1 -- 172.21.15.96:0/4266407164 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5583d3025800 con 0x5583d2cd5f60 2024-04-25T14:13:39.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.508+0000 7ff6abfff640 1 -- 172.21.15.96:0/4266407164 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff6b80037f0 con 0x5583d2cd5f60 2024-04-25T14:13:39.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.508+0000 7ff6d9d7c740 10 client.15528 fetch_fsmap learned FSMap version 26 2024-04-25T14:13:39.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.508+0000 7ff6d9d7c740 1 -- 172.21.15.96:0/4266407164 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5583d31383c0 con 0x5583d2cd5f60 2024-04-25T14:13:39.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.508+0000 7ff6abfff640 1 -- 172.21.15.96:0/4266407164 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5583d3025800 con 0x5583d2cd5f60 2024-04-25T14:13:39.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.508+0000 7ff6abfff640 1 -- 172.21.15.96:0/4266407164 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff6b800c040 con 0x5583d2cd5f60 2024-04-25T14:13:39.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.508+0000 7ff6d9d7c740 10 client.15528 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:13:39.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.508+0000 7ff6d9d7c740 10 client.15528 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-25T14:13:39.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.508+0000 7ff6d9d7c740 20 client.15528 populate_metadata read hostname 'smithi096' 2024-04-25T14:13:39.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.508+0000 7ff6d9d7c740 1 --2- 172.21.15.96:0/4266407164 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x5583d2d17390 0x5583d313acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.508+0000 7ff6d9d7c740 4 client.15528 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:13:39.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.508+0000 7ff6d9d7c740 1 -- 172.21.15.96:0/4266407164 --> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5583d2fa0610 con 0x5583d2d17390 2024-04-25T14:13:39.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.509+0000 7ff6d0c6d640 1 --2- 172.21.15.96:0/4266407164 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x5583d2d17390 0x5583d313acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.509+0000 7ff6abfff640 10 client.15528 ms_handle_connect on v2:172.21.15.151:6832/1382721608 2024-04-25T14:13:39.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.509+0000 7ff6d0c6d640 1 --2- 172.21.15.96:0/4266407164 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x5583d2d17390 0x5583d313acc0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5ad20bb13cb49991 server_cookie=fa9a1b2b4c76f044 in_seq=0 out_seq=0 2024-04-25T14:13:39.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.509+0000 7ff6abfff640 1 -- 172.21.15.96:0/4266407164 <== mds.4 v2:172.21.15.151:6832/1382721608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5583d2fa0610 con 0x5583d2d17390 2024-04-25T14:13:39.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.509+0000 7ff6abfff640 10 client.15528 handle_command_reply: tid=0 2024-04-25T14:13:39.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.509+0000 7ff6d9d7c740 1 client.15528 shutdown 2024-04-25T14:13:39.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.510+0000 7ff6d9d7c740 1 -- 172.21.15.96:0/4266407164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d2cd5f60 msgr2=0x5583d30ca8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.510+0000 7ff6d9d7c740 1 --2- 172.21.15.96:0/4266407164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d2cd5f60 0x5583d30ca8c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff6b800cc80 tx=0x7ff6b801d820 comp rx=0 tx=0).stop 2024-04-25T14:13:39.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.510+0000 7ff6d9d7c740 1 -- 172.21.15.96:0/4266407164 shutdown_connections 2024-04-25T14:13:39.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.510+0000 7ff6d9d7c740 1 --2- 172.21.15.96:0/4266407164 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x5583d2d17390 0x5583d313acc0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.510+0000 7ff6d9d7c740 1 --2- 172.21.15.96:0/4266407164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583d2cf66b0 0x5583d30c7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.510+0000 7ff6abfff640 0 client.15528 ms_handle_reset on v2:172.21.15.151:6832/1382721608 2024-04-25T14:13:39.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.510+0000 7ff6d9d7c740 1 --2- 172.21.15.96:0/4266407164 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583d2cee4d0 0x5583d30c7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.510+0000 7ff6d9d7c740 1 --2- 172.21.15.96:0/4266407164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d2cd5f60 0x5583d30ca8c0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.510+0000 7ff6d9d7c740 1 -- 172.21.15.96:0/4266407164 >> 172.21.15.96:0/4266407164 conn(0x5583d2d0b4e0 msgr2=0x5583d30c30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:39.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.510+0000 7ff6d9d7c740 1 -- 172.21.15.96:0/4266407164 shutdown_connections 2024-04-25T14:13:39.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.510+0000 7ff6d9d7c740 1 -- 172.21.15.96:0/4266407164 wait complete. 2024-04-25T14:13:39.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.510+0000 7ff6d9d7c740 20 client.15528 trim_cache size 0 max 16384 2024-04-25T14:13:39.515 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-04-25T14:13:39.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.510+0000 7ff6d2ed1640 1 -- 172.21.15.96:0/4127672094 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ff6bc0afaf0 msgr2=0x7ff6bc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.510+0000 7ff6d2ed1640 1 --2- 172.21.15.96:0/4127672094 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ff6bc0afaf0 0x7ff6bc0b2000 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff6c40a4d40 tx=0x7ff6c4090040 comp rx=0 tx=0).stop 2024-04-25T14:13:39.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.510+0000 7ff6d2ed1640 1 -- 172.21.15.96:0/4127672094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6cc108920 msgr2=0x7ff6cc1a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.510+0000 7ff6d2ed1640 1 --2- 172.21.15.96:0/4127672094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6cc108920 0x7ff6cc1a0680 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff6c400eca0 tx=0x7ff6c400b690 comp rx=0 tx=0).stop 2024-04-25T14:13:39.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.510+0000 7ff6d2ed1640 1 -- 172.21.15.96:0/4127672094 shutdown_connections 2024-04-25T14:13:39.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.510+0000 7ff6d2ed1640 1 --2- 172.21.15.96:0/4127672094 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ff6bc0afaf0 0x7ff6bc0b2000 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.510+0000 7ff6d2ed1640 1 --2- 172.21.15.96:0/4127672094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6cc108920 0x7ff6cc1a0680 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.510+0000 7ff6d2ed1640 1 --2- 172.21.15.96:0/4127672094 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6cc103c20 0x7ff6cc1997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.510+0000 7ff6d2ed1640 1 --2- 172.21.15.96:0/4127672094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6cc1029e0 0x7ff6cc199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.510+0000 7ff6d2ed1640 1 -- 172.21.15.96:0/4127672094 >> 172.21.15.96:0/4127672094 conn(0x7ff6cc0fdbd0 msgr2=0x7ff6cc104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:39.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.511+0000 7ff6d2ed1640 1 -- 172.21.15.96:0/4127672094 shutdown_connections 2024-04-25T14:13:39.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.511+0000 7ff6d2ed1640 1 -- 172.21.15.96:0/4127672094 wait complete. 2024-04-25T14:13:39.520 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T14:13:39.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.632+0000 7f17b2b5e640 1 -- 172.21.15.96:0/1738177758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17ac104d30 msgr2=0x7f17ac105140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.632+0000 7f17b2b5e640 1 --2- 172.21.15.96:0/1738177758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17ac104d30 0x7f17ac105140 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1798009900 tx=0x7f179802a410 comp rx=0 tx=0).stop 2024-04-25T14:13:39.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.632+0000 7f17b2b5e640 1 -- 172.21.15.96:0/1738177758 shutdown_connections 2024-04-25T14:13:39.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.632+0000 7f17b2b5e640 1 --2- 172.21.15.96:0/1738177758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17ac109a30 0x7f17ac10be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.632+0000 7f17b2b5e640 1 --2- 172.21.15.96:0/1738177758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17ac104d30 0x7f17ac105140 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.632+0000 7f17b2b5e640 1 --2- 172.21.15.96:0/1738177758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17ac103af0 0x7f17ac1047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.632+0000 7f17b2b5e640 1 -- 172.21.15.96:0/1738177758 >> 172.21.15.96:0/1738177758 conn(0x7f17ac0fece0 msgr2=0x7f17ac101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:39.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.632+0000 7f17b2b5e640 1 -- 172.21.15.96:0/1738177758 shutdown_connections 2024-04-25T14:13:39.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.632+0000 7f17b2b5e640 1 -- 172.21.15.96:0/1738177758 wait complete. 2024-04-25T14:13:39.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.633+0000 7f17b2b5e640 1 Processor -- start 2024-04-25T14:13:39.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.633+0000 7f17b2b5e640 1 -- start start 2024-04-25T14:13:39.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.633+0000 7f17b2b5e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17ac103af0 0x7f17ac19a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.633+0000 7f17b2b5e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17ac104d30 0x7f17ac19a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.633+0000 7f17b2b5e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17ac109a30 0x7f17ac1a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.633+0000 7f17b2b5e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f17ac10f610 con 0x7f17ac109a30 2024-04-25T14:13:39.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.633+0000 7f17b2b5e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f17ac10f490 con 0x7f17ac103af0 2024-04-25T14:13:39.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.633+0000 7f17b2b5e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f17ac10f790 con 0x7f17ac104d30 2024-04-25T14:13:39.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.633+0000 7f17b10fb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17ac109a30 0x7f17ac1a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.633+0000 7f17b10fb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17ac109a30 0x7f17ac1a16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40920/0 (socket says 172.21.15.96:40920) 2024-04-25T14:13:39.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.633+0000 7f17b10fb640 1 -- 172.21.15.96:0/4151656367 learned_addr learned my addr 172.21.15.96:0/4151656367 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:39.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.633+0000 7f17b08fa640 1 --2- 172.21.15.96:0/4151656367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17ac103af0 0x7f17ac19a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.633+0000 7f17abfff640 1 --2- 172.21.15.96:0/4151656367 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17ac104d30 0x7f17ac19a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.634+0000 7f17b10fb640 1 -- 172.21.15.96:0/4151656367 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17ac104d30 msgr2=0x7f17ac19a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.634+0000 7f17b10fb640 1 --2- 172.21.15.96:0/4151656367 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17ac104d30 0x7f17ac19a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.634+0000 7f17b10fb640 1 -- 172.21.15.96:0/4151656367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17ac103af0 msgr2=0x7f17ac19a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.634+0000 7f17b10fb640 1 --2- 172.21.15.96:0/4151656367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17ac103af0 0x7f17ac19a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.634+0000 7f17b10fb640 1 -- 172.21.15.96:0/4151656367 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17ac1a1d50 con 0x7f17ac109a30 2024-04-25T14:13:39.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.634+0000 7f17b10fb640 1 --2- 172.21.15.96:0/4151656367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17ac109a30 0x7f17ac1a16e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f17a4009fd0 tx=0x7f17a400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:39.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.634+0000 7f17a9ffb640 1 -- 172.21.15.96:0/4151656367 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a4017070 con 0x7f17ac109a30 2024-04-25T14:13:39.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.634+0000 7f17a9ffb640 1 -- 172.21.15.96:0/4151656367 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17a4003e20 con 0x7f17ac109a30 2024-04-25T14:13:39.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.634+0000 7f17b2b5e640 1 -- 172.21.15.96:0/4151656367 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17ac1a1fe0 con 0x7f17ac109a30 2024-04-25T14:13:39.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.634+0000 7f17a9ffb640 1 -- 172.21.15.96:0/4151656367 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a4010460 con 0x7f17ac109a30 2024-04-25T14:13:39.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.634+0000 7f17b2b5e640 1 -- 172.21.15.96:0/4151656367 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17ac1a2450 con 0x7f17ac109a30 2024-04-25T14:13:39.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.637+0000 7f17b9a0f740 1 Processor -- start 2024-04-25T14:13:39.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.637+0000 7f17a9ffb640 1 -- 172.21.15.96:0/4151656367 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f17a4003970 con 0x7f17ac109a30 2024-04-25T14:13:39.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.637+0000 7f17b9a0f740 1 -- start start 2024-04-25T14:13:39.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.638+0000 7f17b9a0f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f73c43390 0x556f73fe9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.638+0000 7f17b9a0f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f73c1a4d0 0x556f73fea1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.638+0000 7f17b9a0f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f73c226b0 0x556f73feae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.638+0000 7f17b9a0f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556f73f49d80 con 0x556f73c1a4d0 2024-04-25T14:13:39.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.638+0000 7f17b9a0f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556f73f03a20 con 0x556f73c43390 2024-04-25T14:13:39.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.638+0000 7f17b9a0f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556f73f45d40 con 0x556f73c226b0 2024-04-25T14:13:39.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.638+0000 7f17b08fa640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f73c1a4d0 0x556f73fea1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.638+0000 7f17b08fa640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f73c1a4d0 0x556f73fea1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40928/0 (socket says 172.21.15.96:40928) 2024-04-25T14:13:39.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.638+0000 7f17b08fa640 1 -- 172.21.15.96:0/996916061 learned_addr learned my addr 172.21.15.96:0/996916061 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:39.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.638+0000 7f17abfff640 1 --2- 172.21.15.96:0/996916061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f73c43390 0x556f73fe9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.638+0000 7f17abfff640 1 --2- 172.21.15.96:0/996916061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f73c226b0 0x556f73feae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.638+0000 7f17b08fa640 1 -- 172.21.15.96:0/996916061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f73c226b0 msgr2=0x556f73feae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:39.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.638+0000 7f17b08fa640 1 --2- 172.21.15.96:0/996916061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f73c226b0 0x556f73feae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.638+0000 7f17b08fa640 1 -- 172.21.15.96:0/996916061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f73c43390 msgr2=0x556f73fe9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:39.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.638+0000 7f17b08fa640 1 --2- 172.21.15.96:0/996916061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f73c43390 0x556f73fe9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.638+0000 7f17b08fa640 1 -- 172.21.15.96:0/996916061 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f73f0ec30 con 0x556f73c1a4d0 2024-04-25T14:13:39.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.638+0000 7f17abfff640 1 --2- 172.21.15.96:0/996916061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f73c43390 0x556f73fe9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:39.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.638+0000 7f17a9ffb640 1 --2- 172.21.15.96:0/4151656367 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f179c0afaf0 0x7f179c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.638+0000 7f17abfff640 1 --2- 172.21.15.96:0/4151656367 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f179c0afaf0 0x7f179c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.638+0000 7f17b08fa640 1 --2- 172.21.15.96:0/996916061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f73c1a4d0 0x556f73fea1a0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f17a000bc70 tx=0x7f17a00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a7b400b9968bf6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:39.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.638+0000 7f17a9ffb640 1 -- 172.21.15.96:0/4151656367 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f17a40de8f0 con 0x7f17ac109a30 2024-04-25T14:13:39.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.638+0000 7f1796ffd640 1 -- 172.21.15.96:0/996916061 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a001c680 con 0x556f73c1a4d0 2024-04-25T14:13:39.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.639+0000 7f1796ffd640 1 -- 172.21.15.96:0/996916061 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17a000be20 con 0x556f73c1a4d0 2024-04-25T14:13:39.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.639+0000 7f1796ffd640 1 -- 172.21.15.96:0/996916061 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a001a660 con 0x556f73c1a4d0 2024-04-25T14:13:39.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.639+0000 7f17b9a0f740 1 -- 172.21.15.96:0/996916061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f73c1a4d0 msgr2=0x556f73fea1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.639+0000 7f17abfff640 1 --2- 172.21.15.96:0/4151656367 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f179c0afaf0 0x7f179c0b2000 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1798009230 tx=0x7f1798000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:39.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.639+0000 7f17b9a0f740 1 --2- 172.21.15.96:0/996916061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f73c1a4d0 0x556f73fea1a0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f17a000bc70 tx=0x7f17a00107a0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.639+0000 7f17b9a0f740 1 -- 172.21.15.96:0/996916061 shutdown_connections 2024-04-25T14:13:39.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.639+0000 7f17b9a0f740 1 --2- 172.21.15.96:0/996916061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f73c226b0 0x556f73feae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.639+0000 7f17b9a0f740 1 --2- 172.21.15.96:0/996916061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f73c1a4d0 0x556f73fea1a0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.639+0000 7f17b9a0f740 1 --2- 172.21.15.96:0/996916061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f73c43390 0x556f73fe9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.639+0000 7f17b9a0f740 1 -- 172.21.15.96:0/996916061 >> 172.21.15.96:0/996916061 conn(0x556f73c374e0 msgr2=0x556f73fe1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:39.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.639+0000 7f17b9a0f740 1 -- 172.21.15.96:0/996916061 shutdown_connections 2024-04-25T14:13:39.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.639+0000 7f17b9a0f740 1 -- 172.21.15.96:0/996916061 wait complete. 2024-04-25T14:13:39.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.640+0000 7f17b9a0f740 1 -- start start 2024-04-25T14:13:39.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.640+0000 7f17b9a0f740 1 Processor -- start 2024-04-25T14:13:39.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.640+0000 7f17b9a0f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f73c1a4d0 0x556f73ff39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.640+0000 7f17b9a0f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f73c226b0 0x556f73ff3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.640+0000 7f17b9a0f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f73c43390 0x556f73ffae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.640+0000 7f17b9a0f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556f73feb940 con 0x556f73c43390 2024-04-25T14:13:39.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.640+0000 7f17b9a0f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556f73ed0dc0 con 0x556f73c1a4d0 2024-04-25T14:13:39.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.640+0000 7f17b9a0f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556f73febac0 con 0x556f73c226b0 2024-04-25T14:13:39.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.640+0000 7f17b9a0f740 10 client.0 _ll_register_callbacks cb 0x556f73eec600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:13:39.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.640+0000 7f17b10fb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f73c43390 0x556f73ffae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.640+0000 7f17b10fb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f73c43390 0x556f73ffae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40930/0 (socket says 172.21.15.96:40930) 2024-04-25T14:13:39.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.640+0000 7f17b10fb640 1 -- 172.21.15.96:0/3056125215 learned_addr learned my addr 172.21.15.96:0/3056125215 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:39.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.640+0000 7f17b08fa640 1 --2- 172.21.15.96:0/3056125215 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f73c1a4d0 0x556f73ff39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.640+0000 7f17abfff640 1 --2- 172.21.15.96:0/3056125215 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f73c226b0 0x556f73ff3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.641+0000 7f17b10fb640 1 -- 172.21.15.96:0/3056125215 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f73c226b0 msgr2=0x556f73ff3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:39.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.641+0000 7f17b10fb640 1 --2- 172.21.15.96:0/3056125215 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f73c226b0 0x556f73ff3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.641+0000 7f17b10fb640 1 -- 172.21.15.96:0/3056125215 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f73c1a4d0 msgr2=0x556f73ff39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:39.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.641+0000 7f17b10fb640 1 --2- 172.21.15.96:0/3056125215 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f73c1a4d0 0x556f73ff39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.641+0000 7f17b10fb640 1 -- 172.21.15.96:0/3056125215 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f73c5b3f0 con 0x556f73c43390 2024-04-25T14:13:39.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.641+0000 7f17b08fa640 1 --2- 172.21.15.96:0/3056125215 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f73c1a4d0 0x556f73ff39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:13:39.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.641+0000 7f17b10fb640 1 --2- 172.21.15.96:0/3056125215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f73c43390 0x556f73ffae70 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f17a40a2ee0 tx=0x7f17a40042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69d969f2da11ae10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:39.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.641+0000 7f178bfff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:13:39.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.641+0000 7f178bfff640 1 -- 172.21.15.96:0/3056125215 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a4020040 con 0x556f73c43390 2024-04-25T14:13:39.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.641+0000 7f178bfff640 1 -- 172.21.15.96:0/3056125215 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f17a40a60c0 con 0x556f73c43390 2024-04-25T14:13:39.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.641+0000 7f17b9a0f740 1 -- 172.21.15.96:0/3056125215 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17a000be20 con 0x556f73c43390 2024-04-25T14:13:39.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.641+0000 7f178bfff640 1 -- 172.21.15.96:0/3056125215 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f17a40a6e60 con 0x556f73c43390 2024-04-25T14:13:39.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.641+0000 7f17b9a0f740 10 client.15540 fetch_fsmap learned FSMap version 26 2024-04-25T14:13:39.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.641+0000 7f17b9a0f740 1 -- 172.21.15.96:0/3056125215 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556f73d257d0 con 0x556f73c43390 2024-04-25T14:13:39.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.641+0000 7f178bfff640 1 -- 172.21.15.96:0/3056125215 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17a000be20 con 0x556f73c43390 2024-04-25T14:13:39.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.641+0000 7f178bfff640 1 -- 172.21.15.96:0/3056125215 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f17a40a5b30 con 0x556f73c43390 2024-04-25T14:13:39.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.642+0000 7f17b9a0f740 10 client.15540 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:13:39.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.642+0000 7f17b9a0f740 10 client.15540 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T14:13:39.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.642+0000 7f17b9a0f740 20 client.15540 populate_metadata read hostname 'smithi096' 2024-04-25T14:13:39.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.642+0000 7f17b9a0f740 1 --2- 172.21.15.96:0/3056125215 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x556f73c01f60 0x556f740017c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.642+0000 7f17b9a0f740 4 client.15540 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:13:39.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.642+0000 7f17b9a0f740 1 -- 172.21.15.96:0/3056125215 --> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f73f57990 con 0x556f73c01f60 2024-04-25T14:13:39.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.642+0000 7f17b08fa640 1 --2- 172.21.15.96:0/3056125215 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x556f73c01f60 0x556f740017c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.642+0000 7f178bfff640 10 client.15540 ms_handle_connect on v2:172.21.15.97:6834/3872301726 2024-04-25T14:13:39.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.642+0000 7f17b08fa640 1 --2- 172.21.15.96:0/3056125215 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x556f73c01f60 0x556f740017c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=70f7b03966c8193e server_cookie=fdc16ebe559954fe in_seq=0 out_seq=0 2024-04-25T14:13:39.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.643+0000 7f178bfff640 1 -- 172.21.15.96:0/3056125215 <== mds.3 v2:172.21.15.97:6834/3872301726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556f73f57990 con 0x556f73c01f60 2024-04-25T14:13:39.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.643+0000 7f178bfff640 10 client.15540 handle_command_reply: tid=0 2024-04-25T14:13:39.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.643+0000 7f17b9a0f740 1 client.15540 shutdown 2024-04-25T14:13:39.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.643+0000 7f17b9a0f740 1 -- 172.21.15.96:0/3056125215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f73c43390 msgr2=0x556f73ffae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.643+0000 7f17b9a0f740 1 --2- 172.21.15.96:0/3056125215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f73c43390 0x556f73ffae70 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f17a40a2ee0 tx=0x7f17a40042e0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.643+0000 7f17b9a0f740 1 -- 172.21.15.96:0/3056125215 shutdown_connections 2024-04-25T14:13:39.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.643+0000 7f17b9a0f740 1 --2- 172.21.15.96:0/3056125215 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x556f73c01f60 0x556f740017c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.643+0000 7f17b9a0f740 1 --2- 172.21.15.96:0/3056125215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f73c43390 0x556f73ffae70 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.643+0000 7f17b9a0f740 1 --2- 172.21.15.96:0/3056125215 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f73c226b0 0x556f73ff3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.643+0000 7f178bfff640 0 client.15540 ms_handle_reset on v2:172.21.15.97:6834/3872301726 2024-04-25T14:13:39.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.643+0000 7f17b9a0f740 1 --2- 172.21.15.96:0/3056125215 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f73c1a4d0 0x556f73ff39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.643+0000 7f17b9a0f740 1 -- 172.21.15.96:0/3056125215 >> 172.21.15.96:0/3056125215 conn(0x556f73c374e0 msgr2=0x556f73fe3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:39.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.644+0000 7f17b9a0f740 1 -- 172.21.15.96:0/3056125215 shutdown_connections 2024-04-25T14:13:39.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.644+0000 7f17b9a0f740 1 -- 172.21.15.96:0/3056125215 wait complete. 2024-04-25T14:13:39.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.644+0000 7f17b9a0f740 20 client.15540 trim_cache size 0 max 16384 2024-04-25T14:13:39.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.657+0000 7f17b9a0f740 1 Processor -- start 2024-04-25T14:13:39.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.657+0000 7f17b9a0f740 1 -- start start 2024-04-25T14:13:39.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.657+0000 7f17b9a0f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f73c1a4d0 0x556f73fdc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.657+0000 7f17b9a0f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f73c226b0 0x556f73fe09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.657+0000 7f17b9a0f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f73c43390 0x556f73fe0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.657+0000 7f17b9a0f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f17ac1a9070 con 0x556f73c43390 2024-04-25T14:13:39.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.657+0000 7f17b9a0f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f17ac1aabb0 con 0x556f73c1a4d0 2024-04-25T14:13:39.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.657+0000 7f17b9a0f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f17ac1a9740 con 0x556f73c226b0 2024-04-25T14:13:39.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.657+0000 7f17b10fb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f73c43390 0x556f73fe0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.657+0000 7f17b10fb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f73c43390 0x556f73fe0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40944/0 (socket says 172.21.15.96:40944) 2024-04-25T14:13:39.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.657+0000 7f17b10fb640 1 -- 172.21.15.96:0/3658448024 learned_addr learned my addr 172.21.15.96:0/3658448024 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:39.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.658+0000 7f17b08fa640 1 --2- 172.21.15.96:0/3658448024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f73c1a4d0 0x556f73fdc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.658+0000 7f17abfff640 1 --2- 172.21.15.96:0/3658448024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f73c226b0 0x556f73fe09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.658+0000 7f17b10fb640 1 -- 172.21.15.96:0/3658448024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f73c226b0 msgr2=0x556f73fe09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:39.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.658+0000 7f17b10fb640 1 --2- 172.21.15.96:0/3658448024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f73c226b0 0x556f73fe09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.658+0000 7f17b10fb640 1 -- 172.21.15.96:0/3658448024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f73c1a4d0 msgr2=0x556f73fdc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:39.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.658+0000 7f17b10fb640 1 --2- 172.21.15.96:0/3658448024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f73c1a4d0 0x556f73fdc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.658+0000 7f17b10fb640 1 -- 172.21.15.96:0/3658448024 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f73fff4f0 con 0x556f73c43390 2024-04-25T14:13:39.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.658+0000 7f17abfff640 1 --2- 172.21.15.96:0/3658448024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f73c226b0 0x556f73fe09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:39.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.658+0000 7f17b10fb640 1 --2- 172.21.15.96:0/3658448024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f73c43390 0x556f73fe0f20 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f17a4005c40 tx=0x7f17a40099f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8304111273eb7722 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:39.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.658+0000 7f17967fc640 1 -- 172.21.15.96:0/3658448024 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a4020070 con 0x556f73c43390 2024-04-25T14:13:39.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.658+0000 7f17967fc640 1 -- 172.21.15.96:0/3658448024 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17a40aa020 con 0x556f73c43390 2024-04-25T14:13:39.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.658+0000 7f17967fc640 1 -- 172.21.15.96:0/3658448024 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a40a27d0 con 0x556f73c43390 2024-04-25T14:13:39.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.659+0000 7f17b9a0f740 1 -- 172.21.15.96:0/3658448024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f73c43390 msgr2=0x556f73fe0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.659+0000 7f17b9a0f740 1 --2- 172.21.15.96:0/3658448024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f73c43390 0x556f73fe0f20 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f17a4005c40 tx=0x7f17a40099f0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.659+0000 7f17b9a0f740 1 -- 172.21.15.96:0/3658448024 shutdown_connections 2024-04-25T14:13:39.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.659+0000 7f17b9a0f740 1 --2- 172.21.15.96:0/3658448024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f73c43390 0x556f73fe0f20 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.659+0000 7f17b9a0f740 1 --2- 172.21.15.96:0/3658448024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f73c226b0 0x556f73fe09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.659+0000 7f17b9a0f740 1 --2- 172.21.15.96:0/3658448024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f73c1a4d0 0x556f73fdc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.659+0000 7f17b9a0f740 1 -- 172.21.15.96:0/3658448024 >> 172.21.15.96:0/3658448024 conn(0x556f73c374e0 msgr2=0x556f73fee720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:39.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.659+0000 7f17b9a0f740 1 -- 172.21.15.96:0/3658448024 shutdown_connections 2024-04-25T14:13:39.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.659+0000 7f17b9a0f740 1 -- 172.21.15.96:0/3658448024 wait complete. 2024-04-25T14:13:39.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.659+0000 7f17b9a0f740 1 -- start start 2024-04-25T14:13:39.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.659+0000 7f17b9a0f740 1 Processor -- start 2024-04-25T14:13:39.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.660+0000 7f17b9a0f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f73c1a4d0 0x556f73ff74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.660+0000 7f17b9a0f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f73c226b0 0x556f73ff7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.660+0000 7f17b9a0f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f73c43390 0x556f74067b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.660+0000 7f17b9a0f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f17ac1aa4e0 con 0x556f73c43390 2024-04-25T14:13:39.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.660+0000 7f17b9a0f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f17ac1a9e10 con 0x556f73c226b0 2024-04-25T14:13:39.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.660+0000 7f17b9a0f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f17ac1a7050 con 0x556f73c1a4d0 2024-04-25T14:13:39.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.660+0000 7f17b9a0f740 10 client.0 _ll_register_callbacks cb 0x7f17a4009940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:13:39.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.660+0000 7f17b10fb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f73c43390 0x556f74067b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.660+0000 7f17b10fb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f73c43390 0x556f74067b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40950/0 (socket says 172.21.15.96:40950) 2024-04-25T14:13:39.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.660+0000 7f17b10fb640 1 -- 172.21.15.96:0/2337731972 learned_addr learned my addr 172.21.15.96:0/2337731972 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:39.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.660+0000 7f17abfff640 1 --2- 172.21.15.96:0/2337731972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f73c226b0 0x556f73ff7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.660+0000 7f17b08fa640 1 --2- 172.21.15.96:0/2337731972 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f73c1a4d0 0x556f73ff74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.660+0000 7f17b10fb640 1 -- 172.21.15.96:0/2337731972 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f73c1a4d0 msgr2=0x556f73ff74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:39.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.660+0000 7f17b10fb640 1 --2- 172.21.15.96:0/2337731972 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f73c1a4d0 0x556f73ff74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.660+0000 7f17b10fb640 1 -- 172.21.15.96:0/2337731972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f73c226b0 msgr2=0x556f73ff7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:39.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.660+0000 7f17b10fb640 1 --2- 172.21.15.96:0/2337731972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f73c226b0 0x556f73ff7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.660+0000 7f17b10fb640 1 -- 172.21.15.96:0/2337731972 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f74068060 con 0x556f73c43390 2024-04-25T14:13:39.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.660+0000 7f17b10fb640 1 --2- 172.21.15.96:0/2337731972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f73c43390 0x556f74067b20 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f17a400dfd0 tx=0x7f17a40a62f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fccf5fc931f9a234 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:39.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.660+0000 7f178bfff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:13:39.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.660+0000 7f178bfff640 1 -- 172.21.15.96:0/2337731972 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a40afe60 con 0x556f73c43390 2024-04-25T14:13:39.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.660+0000 7f178bfff640 1 -- 172.21.15.96:0/2337731972 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f17a408e070 con 0x556f73c43390 2024-04-25T14:13:39.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.660+0000 7f17b9a0f740 1 -- 172.21.15.96:0/2337731972 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17a40aa020 con 0x556f73c43390 2024-04-25T14:13:39.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.661+0000 7f178bfff640 1 -- 172.21.15.96:0/2337731972 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f17a4020050 con 0x556f73c43390 2024-04-25T14:13:39.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.661+0000 7f17b9a0f740 10 client.15546 fetch_fsmap learned FSMap version 26 2024-04-25T14:13:39.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.661+0000 7f17b9a0f740 1 -- 172.21.15.96:0/2337731972 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556f74069270 con 0x556f73c43390 2024-04-25T14:13:39.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.661+0000 7f178bfff640 1 -- 172.21.15.96:0/2337731972 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17a40aa020 con 0x556f73c43390 2024-04-25T14:13:39.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.661+0000 7f178bfff640 1 -- 172.21.15.96:0/2337731972 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f17a40a2040 con 0x556f73c43390 2024-04-25T14:13:39.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.661+0000 7f17b9a0f740 10 client.15546 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:13:39.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.661+0000 7f17b9a0f740 10 client.15546 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T14:13:39.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.661+0000 7f17b9a0f740 20 client.15546 populate_metadata read hostname 'smithi096' 2024-04-25T14:13:39.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.661+0000 7f17b9a0f740 1 --2- 172.21.15.96:0/2337731972 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x556f73bd73a0 0x556f7406b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.661+0000 7f17b9a0f740 4 client.15546 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:13:39.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.661+0000 7f17b9a0f740 1 -- 172.21.15.96:0/2337731972 --> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556f73f51800 con 0x556f73bd73a0 2024-04-25T14:13:39.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.661+0000 7f17b08fa640 1 --2- 172.21.15.96:0/2337731972 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x556f73bd73a0 0x556f7406b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.662+0000 7f178bfff640 10 client.15546 ms_handle_connect on v2:172.21.15.97:6834/3872301726 2024-04-25T14:13:39.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.662+0000 7f17b08fa640 1 --2- 172.21.15.96:0/2337731972 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x556f73bd73a0 0x556f7406b8f0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=87df2e06fb1339f0 server_cookie=fd3bc8674ad8ba4f in_seq=0 out_seq=0 2024-04-25T14:13:39.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.662+0000 7f178bfff640 1 -- 172.21.15.96:0/2337731972 <== mds.3 v2:172.21.15.97:6834/3872301726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556f73f51800 con 0x556f73bd73a0 2024-04-25T14:13:39.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.662+0000 7f178bfff640 10 client.15546 handle_command_reply: tid=0 2024-04-25T14:13:39.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.662+0000 7f17b9a0f740 1 client.15546 shutdown 2024-04-25T14:13:39.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.662+0000 7f17b9a0f740 1 -- 172.21.15.96:0/2337731972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f73c43390 msgr2=0x556f74067b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.662+0000 7f17b9a0f740 1 --2- 172.21.15.96:0/2337731972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f73c43390 0x556f74067b20 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f17a400dfd0 tx=0x7f17a40a62f0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.662+0000 7f17b9a0f740 1 -- 172.21.15.96:0/2337731972 shutdown_connections 2024-04-25T14:13:39.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.662+0000 7f17b9a0f740 1 --2- 172.21.15.96:0/2337731972 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x556f73bd73a0 0x556f7406b8f0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.662+0000 7f17b9a0f740 1 --2- 172.21.15.96:0/2337731972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f73c43390 0x556f74067b20 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.662+0000 7f17b9a0f740 1 --2- 172.21.15.96:0/2337731972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f73c226b0 0x556f73ff7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.662+0000 7f178bfff640 0 client.15546 ms_handle_reset on v2:172.21.15.97:6834/3872301726 2024-04-25T14:13:39.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.662+0000 7f17b9a0f740 1 --2- 172.21.15.96:0/2337731972 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f73c1a4d0 0x556f73ff74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.662+0000 7f17b9a0f740 1 -- 172.21.15.96:0/2337731972 >> 172.21.15.96:0/2337731972 conn(0x556f73c374e0 msgr2=0x556f73ff20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:39.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.662+0000 7f17b9a0f740 1 -- 172.21.15.96:0/2337731972 shutdown_connections 2024-04-25T14:13:39.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.662+0000 7f17b9a0f740 1 -- 172.21.15.96:0/2337731972 wait complete. 2024-04-25T14:13:39.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.662+0000 7f17b9a0f740 20 client.15546 trim_cache size 0 max 16384 2024-04-25T14:13:39.668 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-04-25T14:13:39.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.663+0000 7f17b2b5e640 1 -- 172.21.15.96:0/4151656367 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f179c0afaf0 msgr2=0x7f179c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.663+0000 7f17b2b5e640 1 --2- 172.21.15.96:0/4151656367 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f179c0afaf0 0x7f179c0b2000 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1798009230 tx=0x7f1798000bc0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.663+0000 7f17b2b5e640 1 -- 172.21.15.96:0/4151656367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17ac109a30 msgr2=0x7f17ac1a16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.663+0000 7f17b2b5e640 1 --2- 172.21.15.96:0/4151656367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17ac109a30 0x7f17ac1a16e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f17a4009fd0 tx=0x7f17a400b260 comp rx=0 tx=0).stop 2024-04-25T14:13:39.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.663+0000 7f17b2b5e640 1 -- 172.21.15.96:0/4151656367 shutdown_connections 2024-04-25T14:13:39.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.663+0000 7f17b2b5e640 1 --2- 172.21.15.96:0/4151656367 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f179c0afaf0 0x7f179c0b2000 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.663+0000 7f17b2b5e640 1 --2- 172.21.15.96:0/4151656367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17ac109a30 0x7f17ac1a16e0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.663+0000 7f17b2b5e640 1 --2- 172.21.15.96:0/4151656367 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17ac104d30 0x7f17ac19a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.663+0000 7f17b2b5e640 1 --2- 172.21.15.96:0/4151656367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17ac103af0 0x7f17ac19a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.663+0000 7f17b2b5e640 1 -- 172.21.15.96:0/4151656367 >> 172.21.15.96:0/4151656367 conn(0x7f17ac0fece0 msgr2=0x7f17ac105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:39.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.663+0000 7f17b2b5e640 1 -- 172.21.15.96:0/4151656367 shutdown_connections 2024-04-25T14:13:39.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.663+0000 7f17b2b5e640 1 -- 172.21.15.96:0/4151656367 wait complete. 2024-04-25T14:13:39.673 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T14:13:39.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.784+0000 7f4118397640 1 -- 172.21.15.96:0/851872069 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4110104270 msgr2=0x7f4110106690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.784+0000 7f4118397640 1 --2- 172.21.15.96:0/851872069 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4110104270 0x7f4110106690 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f410c009900 tx=0x7f410c02a3f0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.784+0000 7f4118397640 1 -- 172.21.15.96:0/851872069 shutdown_connections 2024-04-25T14:13:39.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.784+0000 7f4118397640 1 --2- 172.21.15.96:0/851872069 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4110109770 0x7f411010bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.784+0000 7f4118397640 1 --2- 172.21.15.96:0/851872069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4110106c60 0x7f41101090e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.784+0000 7f4118397640 1 --2- 172.21.15.96:0/851872069 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4110104270 0x7f4110106690 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.784+0000 7f4118397640 1 -- 172.21.15.96:0/851872069 >> 172.21.15.96:0/851872069 conn(0x7f41100fdbd0 msgr2=0x7f4110100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:39.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.784+0000 7f4118397640 1 -- 172.21.15.96:0/851872069 shutdown_connections 2024-04-25T14:13:39.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.784+0000 7f4118397640 1 -- 172.21.15.96:0/851872069 wait complete. 2024-04-25T14:13:39.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.784+0000 7f4118397640 1 Processor -- start 2024-04-25T14:13:39.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.784+0000 7f4118397640 1 -- start start 2024-04-25T14:13:39.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.784+0000 7f4118397640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4110104270 0x7f411019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.784+0000 7f4118397640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4110106c60 0x7f411019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.784+0000 7f4118397640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4110109770 0x7f41101a1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.785+0000 7f4118397640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f411010f5c0 con 0x7f4110109770 2024-04-25T14:13:39.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.785+0000 7f4118397640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f411010f440 con 0x7f4110106c60 2024-04-25T14:13:39.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.785+0000 7f4118397640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f411010f740 con 0x7f4110104270 2024-04-25T14:13:39.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.785+0000 7f4116934640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4110109770 0x7f41101a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.785+0000 7f4116934640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4110109770 0x7f41101a1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40956/0 (socket says 172.21.15.96:40956) 2024-04-25T14:13:39.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.785+0000 7f4116934640 1 -- 172.21.15.96:0/1309194854 learned_addr learned my addr 172.21.15.96:0/1309194854 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:39.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.785+0000 7f4115932640 1 --2- 172.21.15.96:0/1309194854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4110106c60 0x7f411019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.785+0000 7f4116133640 1 --2- 172.21.15.96:0/1309194854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4110104270 0x7f411019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.785+0000 7f4116934640 1 -- 172.21.15.96:0/1309194854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4110104270 msgr2=0x7f411019a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.785+0000 7f4116934640 1 --2- 172.21.15.96:0/1309194854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4110104270 0x7f411019a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.785+0000 7f4116934640 1 -- 172.21.15.96:0/1309194854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4110106c60 msgr2=0x7f411019a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.785+0000 7f4116934640 1 --2- 172.21.15.96:0/1309194854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4110106c60 0x7f411019a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.785+0000 7f4116934640 1 -- 172.21.15.96:0/1309194854 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41101a1f40 con 0x7f4110109770 2024-04-25T14:13:39.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.785+0000 7f4115932640 1 --2- 172.21.15.96:0/1309194854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4110106c60 0x7f411019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:39.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.785+0000 7f4116934640 1 --2- 172.21.15.96:0/1309194854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4110109770 0x7f41101a1980 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f410800cce0 tx=0x7f410800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:39.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.785+0000 7f41077fe640 1 -- 172.21.15.96:0/1309194854 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410800f450 con 0x7f4110109770 2024-04-25T14:13:39.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.785+0000 7f41077fe640 1 -- 172.21.15.96:0/1309194854 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f410800faa0 con 0x7f4110109770 2024-04-25T14:13:39.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.785+0000 7f41077fe640 1 -- 172.21.15.96:0/1309194854 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4108018b70 con 0x7f4110109770 2024-04-25T14:13:39.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.785+0000 7f4118397640 1 -- 172.21.15.96:0/1309194854 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41101a2230 con 0x7f4110109770 2024-04-25T14:13:39.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.785+0000 7f4118397640 1 -- 172.21.15.96:0/1309194854 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41101a2730 con 0x7f4110109770 2024-04-25T14:13:39.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.788+0000 7f411f245740 1 Processor -- start 2024-04-25T14:13:39.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.789+0000 7f41077fe640 1 -- 172.21.15.96:0/1309194854 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f4108018d10 con 0x7f4110109770 2024-04-25T14:13:39.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.789+0000 7f411f245740 1 -- start start 2024-04-25T14:13:39.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.789+0000 7f411f245740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c9ebb87390 0x55c9ebf29810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.789+0000 7f411f245740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ebb5e4d0 0x55c9ebf29d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.789+0000 7f411f245740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9ebb666b0 0x55c9ebf2aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.789+0000 7f411f245740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c9ebe8dd80 con 0x55c9ebb5e4d0 2024-04-25T14:13:39.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.789+0000 7f411f245740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c9ebe47a20 con 0x55c9ebb87390 2024-04-25T14:13:39.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.789+0000 7f411f245740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c9ebe89d40 con 0x55c9ebb666b0 2024-04-25T14:13:39.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.789+0000 7f4116133640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ebb5e4d0 0x55c9ebf29d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.789+0000 7f4116133640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ebb5e4d0 0x55c9ebf29d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40970/0 (socket says 172.21.15.96:40970) 2024-04-25T14:13:39.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.789+0000 7f4116133640 1 -- 172.21.15.96:0/650418553 learned_addr learned my addr 172.21.15.96:0/650418553 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:39.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.789+0000 7f4115932640 1 --2- 172.21.15.96:0/650418553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c9ebb87390 0x55c9ebf29810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.789+0000 7f41077fe640 1 --2- 172.21.15.96:0/1309194854 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f40fc0afaf0 0x7f40fc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.790+0000 7f4115932640 1 --2- 172.21.15.96:0/650418553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9ebb666b0 0x55c9ebf2aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.790+0000 7f4116133640 1 -- 172.21.15.96:0/650418553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9ebb666b0 msgr2=0x55c9ebf2aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:39.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.790+0000 7f4116133640 1 --2- 172.21.15.96:0/650418553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9ebb666b0 0x55c9ebf2aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.790+0000 7f4116133640 1 -- 172.21.15.96:0/650418553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c9ebb87390 msgr2=0x55c9ebf29810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:39.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.790+0000 7f4116133640 1 --2- 172.21.15.96:0/650418553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c9ebb87390 0x55c9ebf29810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.790+0000 7f4116133640 1 -- 172.21.15.96:0/650418553 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9ebe52c30 con 0x55c9ebb5e4d0 2024-04-25T14:13:39.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.790+0000 7f4115932640 1 --2- 172.21.15.96:0/650418553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9ebb666b0 0x55c9ebf2aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:39.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.790+0000 7f4116934640 1 --2- 172.21.15.96:0/1309194854 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f40fc0afaf0 0x7f40fc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.790+0000 7f41077fe640 1 -- 172.21.15.96:0/1309194854 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f41080df140 con 0x7f4110109770 2024-04-25T14:13:39.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.790+0000 7f4116133640 1 --2- 172.21.15.96:0/650418553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ebb5e4d0 0x55c9ebf29d90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f410c0096c0 tx=0x7f410c031c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe6c41a6f120760 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:39.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.790+0000 7f4104bfe640 1 -- 172.21.15.96:0/650418553 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410c035030 con 0x55c9ebb5e4d0 2024-04-25T14:13:39.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.790+0000 7f4104bfe640 1 -- 172.21.15.96:0/650418553 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f410c031e50 con 0x55c9ebb5e4d0 2024-04-25T14:13:39.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.790+0000 7f4104bfe640 1 -- 172.21.15.96:0/650418553 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410c003c30 con 0x55c9ebb5e4d0 2024-04-25T14:13:39.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.791+0000 7f411f245740 1 -- 172.21.15.96:0/650418553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ebb5e4d0 msgr2=0x55c9ebf29d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.791+0000 7f411f245740 1 --2- 172.21.15.96:0/650418553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ebb5e4d0 0x55c9ebf29d90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f410c0096c0 tx=0x7f410c031c60 comp rx=0 tx=0).stop 2024-04-25T14:13:39.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.791+0000 7f4116934640 1 --2- 172.21.15.96:0/1309194854 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f40fc0afaf0 0x7f40fc0b2000 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f41080a7300 tx=0x7f41080ca040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:39.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.791+0000 7f411f245740 1 -- 172.21.15.96:0/650418553 shutdown_connections 2024-04-25T14:13:39.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.791+0000 7f411f245740 1 --2- 172.21.15.96:0/650418553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9ebb666b0 0x55c9ebf2aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.791+0000 7f411f245740 1 --2- 172.21.15.96:0/650418553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ebb5e4d0 0x55c9ebf29d90 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.791+0000 7f411f245740 1 --2- 172.21.15.96:0/650418553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c9ebb87390 0x55c9ebf29810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.791+0000 7f411f245740 1 -- 172.21.15.96:0/650418553 >> 172.21.15.96:0/650418553 conn(0x55c9ebb7b4e0 msgr2=0x55c9ebf25020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:39.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.791+0000 7f411f245740 1 -- 172.21.15.96:0/650418553 shutdown_connections 2024-04-25T14:13:39.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.791+0000 7f411f245740 1 -- 172.21.15.96:0/650418553 wait complete. 2024-04-25T14:13:39.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.791+0000 7f411f245740 1 -- start start 2024-04-25T14:13:39.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.792+0000 7f411f245740 1 Processor -- start 2024-04-25T14:13:39.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.792+0000 7f411f245740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c9ebb5e4d0 0x55c9ebf33590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.792+0000 7f411f245740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ebb666b0 0x55c9ebf33ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.792+0000 7f411f245740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9ebb87390 0x55c9ebf3aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.792+0000 7f411f245740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c9ebf2b4d0 con 0x55c9ebb666b0 2024-04-25T14:13:39.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.792+0000 7f411f245740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c9ebe14dc0 con 0x55c9ebb5e4d0 2024-04-25T14:13:39.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.792+0000 7f411f245740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c9ebf2b650 con 0x55c9ebb87390 2024-04-25T14:13:39.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.792+0000 7f411f245740 10 client.0 _ll_register_callbacks cb 0x55c9ebe30600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:13:39.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.792+0000 7f4116133640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ebb666b0 0x55c9ebf33ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.792+0000 7f4116133640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ebb666b0 0x55c9ebf33ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40972/0 (socket says 172.21.15.96:40972) 2024-04-25T14:13:39.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.792+0000 7f4116133640 1 -- 172.21.15.96:0/1399991856 learned_addr learned my addr 172.21.15.96:0/1399991856 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:39.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.792+0000 7f4115932640 1 --2- 172.21.15.96:0/1399991856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c9ebb5e4d0 0x55c9ebf33590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.792+0000 7f4115932640 1 --2- 172.21.15.96:0/1399991856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9ebb87390 0x55c9ebf3aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.792+0000 7f4116133640 1 -- 172.21.15.96:0/1399991856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9ebb87390 msgr2=0x55c9ebf3aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:39.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.792+0000 7f4116133640 1 --2- 172.21.15.96:0/1399991856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9ebb87390 0x55c9ebf3aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.792+0000 7f4116133640 1 -- 172.21.15.96:0/1399991856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c9ebb5e4d0 msgr2=0x55c9ebf33590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:39.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.792+0000 7f4116133640 1 --2- 172.21.15.96:0/1399991856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c9ebb5e4d0 0x55c9ebf33590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.792+0000 7f4116133640 1 -- 172.21.15.96:0/1399991856 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9ebb9f3f0 con 0x55c9ebb666b0 2024-04-25T14:13:39.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.792+0000 7f4116133640 1 --2- 172.21.15.96:0/1399991856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ebb666b0 0x55c9ebf33ad0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f410c031440 tx=0x7f410c0317d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c93ccc0a481bad2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:39.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.792+0000 7f40f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:13:39.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.792+0000 7f40f9ffb640 1 -- 172.21.15.96:0/1399991856 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410c035070 con 0x55c9ebb666b0 2024-04-25T14:13:39.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.792+0000 7f40f9ffb640 1 -- 172.21.15.96:0/1399991856 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f410c003be0 con 0x55c9ebb666b0 2024-04-25T14:13:39.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.792+0000 7f411f245740 1 -- 172.21.15.96:0/1399991856 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f410c031e50 con 0x55c9ebb666b0 2024-04-25T14:13:39.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.793+0000 7f40f9ffb640 1 -- 172.21.15.96:0/1399991856 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f410c054070 con 0x55c9ebb666b0 2024-04-25T14:13:39.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.793+0000 7f411f245740 10 client.15558 fetch_fsmap learned FSMap version 26 2024-04-25T14:13:39.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.793+0000 7f411f245740 1 -- 172.21.15.96:0/1399991856 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c9ebc697d0 con 0x55c9ebb666b0 2024-04-25T14:13:39.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.793+0000 7f40f9ffb640 1 -- 172.21.15.96:0/1399991856 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f410c031e50 con 0x55c9ebb666b0 2024-04-25T14:13:39.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.793+0000 7f40f9ffb640 1 -- 172.21.15.96:0/1399991856 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f410c03c7a0 con 0x55c9ebb666b0 2024-04-25T14:13:39.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.793+0000 7f411f245740 10 client.15558 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:13:39.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.793+0000 7f411f245740 10 client.15558 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:13:39.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.793+0000 7f411f245740 20 client.15558 populate_metadata read hostname 'smithi096' 2024-04-25T14:13:39.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.793+0000 7f411f245740 1 --2- 172.21.15.96:0/1399991856 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x55c9ebb45f60 0x55c9ebf40df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.793+0000 7f411f245740 4 client.15558 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:13:39.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.793+0000 7f411f245740 1 -- 172.21.15.96:0/1399991856 --> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9ebe9b990 con 0x55c9ebb45f60 2024-04-25T14:13:39.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.793+0000 7f4115932640 1 --2- 172.21.15.96:0/1399991856 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x55c9ebb45f60 0x55c9ebf40df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.794+0000 7f4115932640 1 --2- 172.21.15.96:0/1399991856 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x55c9ebb45f60 0x55c9ebf40df0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1f3401a9a1e2dacd server_cookie=ec759b428875ad68 in_seq=0 out_seq=0 2024-04-25T14:13:39.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.794+0000 7f40f9ffb640 10 client.15558 ms_handle_connect on v2:172.21.15.151:6834/1313823638 2024-04-25T14:13:39.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.795+0000 7f40f9ffb640 1 -- 172.21.15.96:0/1399991856 <== mds.2 v2:172.21.15.151:6834/1313823638 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c9ebe9b990 con 0x55c9ebb45f60 2024-04-25T14:13:39.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.795+0000 7f40f9ffb640 10 client.15558 handle_command_reply: tid=0 2024-04-25T14:13:39.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.795+0000 7f411f245740 1 client.15558 shutdown 2024-04-25T14:13:39.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.795+0000 7f411f245740 1 -- 172.21.15.96:0/1399991856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ebb666b0 msgr2=0x55c9ebf33ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.795+0000 7f411f245740 1 --2- 172.21.15.96:0/1399991856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ebb666b0 0x55c9ebf33ad0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f410c031440 tx=0x7f410c0317d0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.795+0000 7f411f245740 1 -- 172.21.15.96:0/1399991856 shutdown_connections 2024-04-25T14:13:39.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.795+0000 7f411f245740 1 --2- 172.21.15.96:0/1399991856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9ebb87390 0x55c9ebf3aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.795+0000 7f411f245740 1 --2- 172.21.15.96:0/1399991856 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x55c9ebb45f60 0x55c9ebf40df0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.795+0000 7f411f245740 1 --2- 172.21.15.96:0/1399991856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ebb666b0 0x55c9ebf33ad0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.795+0000 7f411f245740 1 --2- 172.21.15.96:0/1399991856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c9ebb5e4d0 0x55c9ebf33590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.795+0000 7f40f9ffb640 0 client.15558 ms_handle_reset on v2:172.21.15.151:6834/1313823638 2024-04-25T14:13:39.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.795+0000 7f411f245740 1 -- 172.21.15.96:0/1399991856 >> 172.21.15.96:0/1399991856 conn(0x55c9ebb7b4e0 msgr2=0x55c9ebf27f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:39.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.795+0000 7f411f245740 1 -- 172.21.15.96:0/1399991856 shutdown_connections 2024-04-25T14:13:39.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.795+0000 7f411f245740 1 -- 172.21.15.96:0/1399991856 wait complete. 2024-04-25T14:13:39.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.795+0000 7f411f245740 20 client.15558 trim_cache size 0 max 16384 2024-04-25T14:13:39.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.808+0000 7f411f245740 1 Processor -- start 2024-04-25T14:13:39.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.808+0000 7f411f245740 1 -- start start 2024-04-25T14:13:39.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.808+0000 7f411f245740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c9ebb5e4d0 0x55c9ebf28530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.808+0000 7f411f245740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9ebb666b0 0x55c9ebf46560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.808+0000 7f411f245740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ebb87390 0x55c9ebf46aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.808+0000 7f411f245740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f41101ab140 con 0x55c9ebb87390 2024-04-25T14:13:39.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.808+0000 7f411f245740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f41101acc80 con 0x55c9ebb5e4d0 2024-04-25T14:13:39.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.808+0000 7f411f245740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f41101ab810 con 0x55c9ebb666b0 2024-04-25T14:13:39.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.809+0000 7f4115932640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ebb87390 0x55c9ebf46aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.809+0000 7f4115932640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ebb87390 0x55c9ebf46aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40984/0 (socket says 172.21.15.96:40984) 2024-04-25T14:13:39.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.809+0000 7f4115932640 1 -- 172.21.15.96:0/1385719557 learned_addr learned my addr 172.21.15.96:0/1385719557 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:39.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.809+0000 7f4116133640 1 --2- 172.21.15.96:0/1385719557 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9ebb666b0 0x55c9ebf46560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.809+0000 7f4115932640 1 --2- 172.21.15.96:0/1385719557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c9ebb5e4d0 0x55c9ebf28530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.809+0000 7f4115932640 1 -- 172.21.15.96:0/1385719557 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9ebb666b0 msgr2=0x55c9ebf46560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:39.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.809+0000 7f4115932640 1 --2- 172.21.15.96:0/1385719557 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9ebb666b0 0x55c9ebf46560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.809+0000 7f4115932640 1 -- 172.21.15.96:0/1385719557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c9ebb5e4d0 msgr2=0x55c9ebf28530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:39.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.809+0000 7f4115932640 1 --2- 172.21.15.96:0/1385719557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c9ebb5e4d0 0x55c9ebf28530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.809+0000 7f4115932640 1 -- 172.21.15.96:0/1385719557 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9ebf28a70 con 0x55c9ebb87390 2024-04-25T14:13:39.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.809+0000 7f4115932640 1 --2- 172.21.15.96:0/1385719557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ebb87390 0x55c9ebf46aa0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x55c9ebf41500 tx=0x7f41000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e28f7f7cebbb4a04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:39.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.809+0000 7f40fbfff640 1 -- 172.21.15.96:0/1385719557 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4100004030 con 0x55c9ebb87390 2024-04-25T14:13:39.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.809+0000 7f40fbfff640 1 -- 172.21.15.96:0/1385719557 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4100011910 con 0x55c9ebb87390 2024-04-25T14:13:39.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.809+0000 7f40fbfff640 1 -- 172.21.15.96:0/1385719557 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4100005550 con 0x55c9ebb87390 2024-04-25T14:13:39.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.810+0000 7f411f245740 1 -- 172.21.15.96:0/1385719557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ebb87390 msgr2=0x55c9ebf46aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.810+0000 7f411f245740 1 --2- 172.21.15.96:0/1385719557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ebb87390 0x55c9ebf46aa0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x55c9ebf41500 tx=0x7f41000046c0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.810+0000 7f411f245740 1 -- 172.21.15.96:0/1385719557 shutdown_connections 2024-04-25T14:13:39.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.810+0000 7f411f245740 1 --2- 172.21.15.96:0/1385719557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ebb87390 0x55c9ebf46aa0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.810+0000 7f411f245740 1 --2- 172.21.15.96:0/1385719557 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9ebb666b0 0x55c9ebf46560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.810+0000 7f411f245740 1 --2- 172.21.15.96:0/1385719557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c9ebb5e4d0 0x55c9ebf28530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.810+0000 7f411f245740 1 -- 172.21.15.96:0/1385719557 >> 172.21.15.96:0/1385719557 conn(0x55c9ebb7b4e0 msgr2=0x55c9ebf27e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:39.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.810+0000 7f411f245740 1 -- 172.21.15.96:0/1385719557 shutdown_connections 2024-04-25T14:13:39.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.810+0000 7f411f245740 1 -- 172.21.15.96:0/1385719557 wait complete. 2024-04-25T14:13:39.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.810+0000 7f411f245740 1 -- start start 2024-04-25T14:13:39.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.810+0000 7f411f245740 1 Processor -- start 2024-04-25T14:13:39.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.810+0000 7f411f245740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c9ebb5e4d0 0x55c9ebfa3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.810+0000 7f411f245740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9ebb666b0 0x55c9ebfa40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.810+0000 7f411f245740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ebb87390 0x55c9ebfaaf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.810+0000 7f411f245740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f41101ac5b0 con 0x55c9ebb87390 2024-04-25T14:13:39.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.810+0000 7f411f245740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f41101abee0 con 0x55c9ebb5e4d0 2024-04-25T14:13:39.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.810+0000 7f411f245740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f41101a9120 con 0x55c9ebb666b0 2024-04-25T14:13:39.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.810+0000 7f411f245740 10 client.0 _ll_register_callbacks cb 0x7f410000b500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:13:39.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.811+0000 7f4115932640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ebb87390 0x55c9ebfaaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.811+0000 7f4115932640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ebb87390 0x55c9ebfaaf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40994/0 (socket says 172.21.15.96:40994) 2024-04-25T14:13:39.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.811+0000 7f4115932640 1 -- 172.21.15.96:0/3231425270 learned_addr learned my addr 172.21.15.96:0/3231425270 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:39.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.811+0000 7f4116133640 1 --2- 172.21.15.96:0/3231425270 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9ebb666b0 0x55c9ebfa40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.811+0000 7f4115932640 1 --2- 172.21.15.96:0/3231425270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c9ebb5e4d0 0x55c9ebfa3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.811+0000 7f4115932640 1 -- 172.21.15.96:0/3231425270 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9ebb666b0 msgr2=0x55c9ebfa40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:39.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.811+0000 7f4115932640 1 --2- 172.21.15.96:0/3231425270 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9ebb666b0 0x55c9ebfa40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.811+0000 7f4115932640 1 -- 172.21.15.96:0/3231425270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c9ebb5e4d0 msgr2=0x55c9ebfa3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:39.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.811+0000 7f4115932640 1 --2- 172.21.15.96:0/3231425270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c9ebb5e4d0 0x55c9ebfa3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.811+0000 7f4115932640 1 -- 172.21.15.96:0/3231425270 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9ebfab550 con 0x55c9ebb87390 2024-04-25T14:13:39.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.811+0000 7f4115932640 1 --2- 172.21.15.96:0/3231425270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ebb87390 0x55c9ebfaaf90 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f4100005300 tx=0x7f4100005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1957d76c46a12683 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:39.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.811+0000 7f40f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:13:39.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.811+0000 7f40f9ffb640 1 -- 172.21.15.96:0/3231425270 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4100032070 con 0x55c9ebb87390 2024-04-25T14:13:39.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.811+0000 7f40f9ffb640 1 -- 172.21.15.96:0/3231425270 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f4100011cc0 con 0x55c9ebb87390 2024-04-25T14:13:39.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.811+0000 7f411f245740 1 -- 172.21.15.96:0/3231425270 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4100011910 con 0x55c9ebb87390 2024-04-25T14:13:39.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.811+0000 7f40f9ffb640 1 -- 172.21.15.96:0/3231425270 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4100004730 con 0x55c9ebb87390 2024-04-25T14:13:39.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.811+0000 7f411f245740 10 client.15564 fetch_fsmap learned FSMap version 26 2024-04-25T14:13:39.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.811+0000 7f411f245740 1 -- 172.21.15.96:0/3231425270 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c9ebfac7d0 con 0x55c9ebb87390 2024-04-25T14:13:39.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.812+0000 7f40f9ffb640 1 -- 172.21.15.96:0/3231425270 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4100011910 con 0x55c9ebb87390 2024-04-25T14:13:39.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.812+0000 7f40f9ffb640 1 -- 172.21.15.96:0/3231425270 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4100035030 con 0x55c9ebb87390 2024-04-25T14:13:39.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.812+0000 7f411f245740 10 client.15564 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:13:39.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.812+0000 7f411f245740 10 client.15564 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:13:39.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.812+0000 7f411f245740 20 client.15564 populate_metadata read hostname 'smithi096' 2024-04-25T14:13:39.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.812+0000 7f411f245740 1 --2- 172.21.15.96:0/3231425270 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x55c9ebb1b3a0 0x55c9ebfaf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.812+0000 7f411f245740 4 client.15564 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:13:39.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.812+0000 7f411f245740 1 -- 172.21.15.96:0/3231425270 --> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c9ebe95800 con 0x55c9ebb1b3a0 2024-04-25T14:13:39.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.812+0000 7f4116133640 1 --2- 172.21.15.96:0/3231425270 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x55c9ebb1b3a0 0x55c9ebfaf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.813+0000 7f40f9ffb640 10 client.15564 ms_handle_connect on v2:172.21.15.151:6834/1313823638 2024-04-25T14:13:39.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.813+0000 7f4116133640 1 --2- 172.21.15.96:0/3231425270 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x55c9ebb1b3a0 0x55c9ebfaf160 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f94bc12b53c2eaed server_cookie=e72883eb0d3b222d in_seq=0 out_seq=0 2024-04-25T14:13:39.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.813+0000 7f40f9ffb640 1 -- 172.21.15.96:0/3231425270 <== mds.2 v2:172.21.15.151:6834/1313823638 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c9ebe95800 con 0x55c9ebb1b3a0 2024-04-25T14:13:39.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.813+0000 7f40f9ffb640 10 client.15564 handle_command_reply: tid=0 2024-04-25T14:13:39.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.813+0000 7f411f245740 1 client.15564 shutdown 2024-04-25T14:13:39.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.813+0000 7f411f245740 1 -- 172.21.15.96:0/3231425270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ebb87390 msgr2=0x55c9ebfaaf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.813+0000 7f411f245740 1 --2- 172.21.15.96:0/3231425270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ebb87390 0x55c9ebfaaf90 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f4100005300 tx=0x7f4100005020 comp rx=0 tx=0).stop 2024-04-25T14:13:39.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.813+0000 7f411f245740 1 -- 172.21.15.96:0/3231425270 shutdown_connections 2024-04-25T14:13:39.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.813+0000 7f411f245740 1 --2- 172.21.15.96:0/3231425270 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x55c9ebb1b3a0 0x55c9ebfaf160 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.813+0000 7f411f245740 1 --2- 172.21.15.96:0/3231425270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9ebb87390 0x55c9ebfaaf90 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.813+0000 7f411f245740 1 --2- 172.21.15.96:0/3231425270 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9ebb666b0 0x55c9ebfa40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.813+0000 7f411f245740 1 --2- 172.21.15.96:0/3231425270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c9ebb5e4d0 0x55c9ebfa3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.813+0000 7f40f9ffb640 0 client.15564 ms_handle_reset on v2:172.21.15.151:6834/1313823638 2024-04-25T14:13:39.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.813+0000 7f411f245740 1 -- 172.21.15.96:0/3231425270 >> 172.21.15.96:0/3231425270 conn(0x55c9ebb7b4e0 msgr2=0x55c9ebf31cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:39.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.813+0000 7f411f245740 1 -- 172.21.15.96:0/3231425270 shutdown_connections 2024-04-25T14:13:39.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.813+0000 7f411f245740 1 -- 172.21.15.96:0/3231425270 wait complete. 2024-04-25T14:13:39.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.813+0000 7f411f245740 20 client.15564 trim_cache size 0 max 16384 2024-04-25T14:13:39.818 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-04-25T14:13:39.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.814+0000 7f4118397640 1 -- 172.21.15.96:0/1309194854 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f40fc0afaf0 msgr2=0x7f40fc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.814+0000 7f4118397640 1 --2- 172.21.15.96:0/1309194854 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f40fc0afaf0 0x7f40fc0b2000 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f41080a7300 tx=0x7f41080ca040 comp rx=0 tx=0).stop 2024-04-25T14:13:39.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.814+0000 7f4118397640 1 -- 172.21.15.96:0/1309194854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4110109770 msgr2=0x7f41101a1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.814+0000 7f4118397640 1 --2- 172.21.15.96:0/1309194854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4110109770 0x7f41101a1980 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f410800cce0 tx=0x7f410800a620 comp rx=0 tx=0).stop 2024-04-25T14:13:39.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.814+0000 7f4118397640 1 -- 172.21.15.96:0/1309194854 shutdown_connections 2024-04-25T14:13:39.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.814+0000 7f4118397640 1 --2- 172.21.15.96:0/1309194854 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f40fc0afaf0 0x7f40fc0b2000 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.814+0000 7f4118397640 1 --2- 172.21.15.96:0/1309194854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4110109770 0x7f41101a1980 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.814+0000 7f4118397640 1 --2- 172.21.15.96:0/1309194854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4110106c60 0x7f411019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.814+0000 7f4118397640 1 --2- 172.21.15.96:0/1309194854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4110104270 0x7f411019a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.814+0000 7f4118397640 1 -- 172.21.15.96:0/1309194854 >> 172.21.15.96:0/1309194854 conn(0x7f41100fdbd0 msgr2=0x7f4110105b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:39.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.814+0000 7f4118397640 1 -- 172.21.15.96:0/1309194854 shutdown_connections 2024-04-25T14:13:39.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.814+0000 7f4118397640 1 -- 172.21.15.96:0/1309194854 wait complete. 2024-04-25T14:13:39.824 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T14:13:39.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.935+0000 7f7e74430640 1 -- 172.21.15.96:0/1990320216 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e6c104d30 msgr2=0x7f7e6c105140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.935+0000 7f7e74430640 1 --2- 172.21.15.96:0/1990320216 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e6c104d30 0x7f7e6c105140 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f7e60009900 tx=0x7f7e6002a3f0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.935+0000 7f7e74430640 1 -- 172.21.15.96:0/1990320216 shutdown_connections 2024-04-25T14:13:39.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.935+0000 7f7e74430640 1 --2- 172.21.15.96:0/1990320216 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e6c109a30 0x7f7e6c10be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.935+0000 7f7e74430640 1 --2- 172.21.15.96:0/1990320216 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e6c104d30 0x7f7e6c105140 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.935+0000 7f7e74430640 1 --2- 172.21.15.96:0/1990320216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e6c103af0 0x7f7e6c1047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.935+0000 7f7e74430640 1 -- 172.21.15.96:0/1990320216 >> 172.21.15.96:0/1990320216 conn(0x7f7e6c0fece0 msgr2=0x7f7e6c101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:39.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.935+0000 7f7e74430640 1 -- 172.21.15.96:0/1990320216 shutdown_connections 2024-04-25T14:13:39.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.935+0000 7f7e74430640 1 -- 172.21.15.96:0/1990320216 wait complete. 2024-04-25T14:13:39.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.935+0000 7f7e74430640 1 Processor -- start 2024-04-25T14:13:39.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.935+0000 7f7e74430640 1 -- start start 2024-04-25T14:13:39.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.936+0000 7f7e74430640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e6c103af0 0x7f7e6c19a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.936+0000 7f7e74430640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e6c104d30 0x7f7e6c19a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.936+0000 7f7e74430640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e6c109a30 0x7f7e6c1a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.936+0000 7f7e74430640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c10f620 con 0x7f7e6c103af0 2024-04-25T14:13:39.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.936+0000 7f7e74430640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c10f4a0 con 0x7f7e6c104d30 2024-04-25T14:13:39.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.936+0000 7f7e74430640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c10f7a0 con 0x7f7e6c109a30 2024-04-25T14:13:39.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.936+0000 7f7e721cc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e6c103af0 0x7f7e6c19a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.936+0000 7f7e721cc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e6c103af0 0x7f7e6c19a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41012/0 (socket says 172.21.15.96:41012) 2024-04-25T14:13:39.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.936+0000 7f7e721cc640 1 -- 172.21.15.96:0/3373176018 learned_addr learned my addr 172.21.15.96:0/3373176018 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:39.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.936+0000 7f7e719cb640 1 --2- 172.21.15.96:0/3373176018 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e6c104d30 0x7f7e6c19a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.936+0000 7f7e729cd640 1 --2- 172.21.15.96:0/3373176018 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e6c109a30 0x7f7e6c1a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.936+0000 7f7e721cc640 1 -- 172.21.15.96:0/3373176018 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e6c109a30 msgr2=0x7f7e6c1a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.936+0000 7f7e721cc640 1 --2- 172.21.15.96:0/3373176018 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e6c109a30 0x7f7e6c1a16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.936+0000 7f7e721cc640 1 -- 172.21.15.96:0/3373176018 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e6c104d30 msgr2=0x7f7e6c19a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.936+0000 7f7e721cc640 1 --2- 172.21.15.96:0/3373176018 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e6c104d30 0x7f7e6c19a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.936+0000 7f7e721cc640 1 -- 172.21.15.96:0/3373176018 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e6c1a1c70 con 0x7f7e6c103af0 2024-04-25T14:13:39.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.936+0000 7f7e729cd640 1 --2- 172.21.15.96:0/3373176018 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e6c109a30 0x7f7e6c1a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:39.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.936+0000 7f7e721cc640 1 --2- 172.21.15.96:0/3373176018 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e6c103af0 0x7f7e6c19a1e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f7e6800c970 tx=0x7f7e6800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:39.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.936+0000 7f7e5f7fe640 1 -- 172.21.15.96:0/3373176018 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e68007c70 con 0x7f7e6c103af0 2024-04-25T14:13:39.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.936+0000 7f7e5f7fe640 1 -- 172.21.15.96:0/3373176018 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e68007e10 con 0x7f7e6c103af0 2024-04-25T14:13:39.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.936+0000 7f7e74430640 1 -- 172.21.15.96:0/3373176018 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e6c1a1f60 con 0x7f7e6c103af0 2024-04-25T14:13:39.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.936+0000 7f7e5f7fe640 1 -- 172.21.15.96:0/3373176018 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e680187b0 con 0x7f7e6c103af0 2024-04-25T14:13:39.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.936+0000 7f7e74430640 1 -- 172.21.15.96:0/3373176018 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e6c1a23c0 con 0x7f7e6c103af0 2024-04-25T14:13:39.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:39 smithi151 ceph-mon[32990]: scrub queued for path: / 2024-04-25T14:13:39.940 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:39 smithi151 ceph-mon[32990]: scrub summary: idle+waiting paths [/] 2024-04-25T14:13:39.940 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:39 smithi151 ceph-mon[32990]: scrub summary: active paths [/] 2024-04-25T14:13:39.940 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:39 smithi151 ceph-mon[32990]: scrub summary: idle+waiting paths [/] 2024-04-25T14:13:39.940 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:39 smithi151 ceph-mon[32990]: scrub summary: idle 2024-04-25T14:13:39.940 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:39 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/3232318215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:13:39.940 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:39 smithi151 ceph-mon[32990]: pgmap v347: 97 pgs: 97 active+clean; 648 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 938 B/s wr, 9 op/s 2024-04-25T14:13:39.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.939+0000 7f7e7b3b5740 1 Processor -- start 2024-04-25T14:13:39.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.939+0000 7f7e5f7fe640 1 -- 172.21.15.96:0/3373176018 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f7e6800fab0 con 0x7f7e6c103af0 2024-04-25T14:13:39.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.939+0000 7f7e7b3b5740 1 -- start start 2024-04-25T14:13:39.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.939+0000 7f7e7b3b5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f3858c7390 0x55f385c69810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.940+0000 7f7e7b3b5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f38589e4d0 0x55f385c69d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.940+0000 7f7e7b3b5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3858a66b0 0x55f385c6aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.940+0000 7f7e7b3b5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f385bcdd80 con 0x55f3858a66b0 2024-04-25T14:13:39.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.940+0000 7f7e7b3b5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f385b87a20 con 0x55f3858c7390 2024-04-25T14:13:39.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.940+0000 7f7e7b3b5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f385bc9d40 con 0x55f38589e4d0 2024-04-25T14:13:39.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.940+0000 7f7e719cb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3858a66b0 0x55f385c6aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.940+0000 7f7e719cb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3858a66b0 0x55f385c6aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41028/0 (socket says 172.21.15.96:41028) 2024-04-25T14:13:39.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.940+0000 7f7e719cb640 1 -- 172.21.15.96:0/4107159080 learned_addr learned my addr 172.21.15.96:0/4107159080 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:39.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.940+0000 7f7e721cc640 1 --2- 172.21.15.96:0/4107159080 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f38589e4d0 0x55f385c69d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.940+0000 7f7e729cd640 1 --2- 172.21.15.96:0/4107159080 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f3858c7390 0x55f385c69810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.940+0000 7f7e5f7fe640 1 --2- 172.21.15.96:0/3373176018 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f7e580afaf0 0x7f7e580b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.940+0000 7f7e719cb640 1 -- 172.21.15.96:0/4107159080 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f38589e4d0 msgr2=0x55f385c69d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:39.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.940+0000 7f7e719cb640 1 --2- 172.21.15.96:0/4107159080 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f38589e4d0 0x55f385c69d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.940+0000 7f7e719cb640 1 -- 172.21.15.96:0/4107159080 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f3858c7390 msgr2=0x55f385c69810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:39.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.940+0000 7f7e719cb640 1 --2- 172.21.15.96:0/4107159080 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f3858c7390 0x55f385c69810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.940+0000 7f7e719cb640 1 -- 172.21.15.96:0/4107159080 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f385b92c30 con 0x55f3858a66b0 2024-04-25T14:13:39.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.940+0000 7f7e729cd640 1 --2- 172.21.15.96:0/3373176018 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f7e580afaf0 0x7f7e580b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.940+0000 7f7e5f7fe640 1 -- 172.21.15.96:0/3373176018 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f7e680dec60 con 0x7f7e6c103af0 2024-04-25T14:13:39.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.940+0000 7f7e719cb640 1 --2- 172.21.15.96:0/4107159080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3858a66b0 0x55f385c6aa20 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f7e60031500 tx=0x7f7e60031de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bca0f050e199d41 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:39.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.940+0000 7f7e5cbfe640 1 -- 172.21.15.96:0/4107159080 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e60035030 con 0x55f3858a66b0 2024-04-25T14:13:39.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.941+0000 7f7e5cbfe640 1 -- 172.21.15.96:0/4107159080 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e60040070 con 0x55f3858a66b0 2024-04-25T14:13:39.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.941+0000 7f7e5cbfe640 1 -- 172.21.15.96:0/4107159080 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e60003b80 con 0x55f3858a66b0 2024-04-25T14:13:39.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.941+0000 7f7e729cd640 1 --2- 172.21.15.96:0/3373176018 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f7e580afaf0 0x7f7e580b2000 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55f385c6a860 tx=0x7f7e64004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:39.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.941+0000 7f7e7b3b5740 1 -- 172.21.15.96:0/4107159080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3858a66b0 msgr2=0x55f385c6aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.941+0000 7f7e7b3b5740 1 --2- 172.21.15.96:0/4107159080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3858a66b0 0x55f385c6aa20 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f7e60031500 tx=0x7f7e60031de0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.941+0000 7f7e7b3b5740 1 -- 172.21.15.96:0/4107159080 shutdown_connections 2024-04-25T14:13:39.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.941+0000 7f7e7b3b5740 1 --2- 172.21.15.96:0/4107159080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3858a66b0 0x55f385c6aa20 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.941+0000 7f7e7b3b5740 1 --2- 172.21.15.96:0/4107159080 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f38589e4d0 0x55f385c69d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.941+0000 7f7e7b3b5740 1 --2- 172.21.15.96:0/4107159080 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f3858c7390 0x55f385c69810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.941+0000 7f7e7b3b5740 1 -- 172.21.15.96:0/4107159080 >> 172.21.15.96:0/4107159080 conn(0x55f3858bb4e0 msgr2=0x55f385c65020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:39.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.942+0000 7f7e7b3b5740 1 -- 172.21.15.96:0/4107159080 shutdown_connections 2024-04-25T14:13:39.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.942+0000 7f7e7b3b5740 1 -- 172.21.15.96:0/4107159080 wait complete. 2024-04-25T14:13:39.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.942+0000 7f7e7b3b5740 1 -- start start 2024-04-25T14:13:39.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.942+0000 7f7e7b3b5740 1 Processor -- start 2024-04-25T14:13:39.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.942+0000 7f7e7b3b5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f38589e4d0 0x55f385c73520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.942+0000 7f7e7b3b5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3858a66b0 0x55f385c73a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.942+0000 7f7e7b3b5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3858c7390 0x55f385c7a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.942+0000 7f7e7b3b5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f385c6b4d0 con 0x55f3858a66b0 2024-04-25T14:13:39.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.942+0000 7f7e7b3b5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f385b54dc0 con 0x55f38589e4d0 2024-04-25T14:13:39.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.942+0000 7f7e7b3b5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f385c6b650 con 0x55f3858c7390 2024-04-25T14:13:39.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.942+0000 7f7e7b3b5740 10 client.0 _ll_register_callbacks cb 0x55f385b70600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:13:39.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.942+0000 7f7e719cb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3858a66b0 0x55f385c73a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.942+0000 7f7e719cb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3858a66b0 0x55f385c73a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41032/0 (socket says 172.21.15.96:41032) 2024-04-25T14:13:39.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.942+0000 7f7e719cb640 1 -- 172.21.15.96:0/3517573294 learned_addr learned my addr 172.21.15.96:0/3517573294 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:39.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.942+0000 7f7e721cc640 1 --2- 172.21.15.96:0/3517573294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f38589e4d0 0x55f385c73520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.942+0000 7f7e729cd640 1 --2- 172.21.15.96:0/3517573294 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3858c7390 0x55f385c7a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.943+0000 7f7e719cb640 1 -- 172.21.15.96:0/3517573294 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3858c7390 msgr2=0x55f385c7a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:39.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.943+0000 7f7e719cb640 1 --2- 172.21.15.96:0/3517573294 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3858c7390 0x55f385c7a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.943+0000 7f7e719cb640 1 -- 172.21.15.96:0/3517573294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f38589e4d0 msgr2=0x55f385c73520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:39.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.943+0000 7f7e719cb640 1 --2- 172.21.15.96:0/3517573294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f38589e4d0 0x55f385c73520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.943+0000 7f7e719cb640 1 -- 172.21.15.96:0/3517573294 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3858df3f0 con 0x55f3858a66b0 2024-04-25T14:13:39.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.943+0000 7f7e719cb640 1 --2- 172.21.15.96:0/3517573294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3858a66b0 0x55f385c73a60 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7e60031800 tx=0x7f7e60031830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6f19da5e589504 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:39.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.943+0000 7f7e55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:13:39.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.943+0000 7f7e55ffb640 1 -- 172.21.15.96:0/3517573294 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e60035040 con 0x55f3858a66b0 2024-04-25T14:13:39.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.943+0000 7f7e55ffb640 1 -- 172.21.15.96:0/3517573294 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f7e60004250 con 0x55f3858a66b0 2024-04-25T14:13:39.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.943+0000 7f7e7b3b5740 1 -- 172.21.15.96:0/3517573294 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e60040070 con 0x55f3858a66b0 2024-04-25T14:13:39.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.943+0000 7f7e55ffb640 1 -- 172.21.15.96:0/3517573294 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7e60004450 con 0x55f3858a66b0 2024-04-25T14:13:39.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.943+0000 7f7e7b3b5740 10 client.15576 fetch_fsmap learned FSMap version 26 2024-04-25T14:13:39.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.943+0000 7f7e7b3b5740 1 -- 172.21.15.96:0/3517573294 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f3859a97d0 con 0x55f3858a66b0 2024-04-25T14:13:39.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.943+0000 7f7e55ffb640 1 -- 172.21.15.96:0/3517573294 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e60040070 con 0x55f3858a66b0 2024-04-25T14:13:39.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.944+0000 7f7e55ffb640 1 -- 172.21.15.96:0/3517573294 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7e60045930 con 0x55f3858a66b0 2024-04-25T14:13:39.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.944+0000 7f7e7b3b5740 10 client.15576 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:13:39.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.944+0000 7f7e7b3b5740 10 client.15576 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:13:39.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.944+0000 7f7e7b3b5740 20 client.15576 populate_metadata read hostname 'smithi096' 2024-04-25T14:13:39.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.944+0000 7f7e7b3b5740 1 --2- 172.21.15.96:0/3517573294 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x55f385885f60 0x55f385c80e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.944+0000 7f7e7b3b5740 4 client.15576 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:13:39.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.944+0000 7f7e7b3b5740 1 -- 172.21.15.96:0/3517573294 --> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f385bdb990 con 0x55f385885f60 2024-04-25T14:13:39.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.944+0000 7f7e721cc640 1 --2- 172.21.15.96:0/3517573294 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x55f385885f60 0x55f385c80e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.944+0000 7f7e55ffb640 10 client.15576 ms_handle_connect on v2:172.21.15.151:6838/245699981 2024-04-25T14:13:39.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.944+0000 7f7e721cc640 1 --2- 172.21.15.96:0/3517573294 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x55f385885f60 0x55f385c80e10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=785bf589593c19f3 server_cookie=a57126ca39914b48 in_seq=0 out_seq=0 2024-04-25T14:13:39.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.945+0000 7f7e55ffb640 1 -- 172.21.15.96:0/3517573294 <== mds.1 v2:172.21.15.151:6838/245699981 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f385bdb990 con 0x55f385885f60 2024-04-25T14:13:39.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.945+0000 7f7e55ffb640 10 client.15576 handle_command_reply: tid=0 2024-04-25T14:13:39.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.945+0000 7f7e7b3b5740 1 client.15576 shutdown 2024-04-25T14:13:39.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.945+0000 7f7e7b3b5740 1 -- 172.21.15.96:0/3517573294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3858a66b0 msgr2=0x55f385c73a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.945+0000 7f7e7b3b5740 1 --2- 172.21.15.96:0/3517573294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3858a66b0 0x55f385c73a60 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7e60031800 tx=0x7f7e60031830 comp rx=0 tx=0).stop 2024-04-25T14:13:39.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.945+0000 7f7e7b3b5740 1 -- 172.21.15.96:0/3517573294 shutdown_connections 2024-04-25T14:13:39.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.945+0000 7f7e7b3b5740 1 --2- 172.21.15.96:0/3517573294 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x55f385885f60 0x55f385c80e10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.945+0000 7f7e7b3b5740 1 --2- 172.21.15.96:0/3517573294 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3858c7390 0x55f385c7a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.945+0000 7f7e55ffb640 0 client.15576 ms_handle_reset on v2:172.21.15.151:6838/245699981 2024-04-25T14:13:39.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.945+0000 7f7e7b3b5740 1 --2- 172.21.15.96:0/3517573294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3858a66b0 0x55f385c73a60 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.945+0000 7f7e7b3b5740 1 --2- 172.21.15.96:0/3517573294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f38589e4d0 0x55f385c73520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.945+0000 7f7e7b3b5740 1 -- 172.21.15.96:0/3517573294 >> 172.21.15.96:0/3517573294 conn(0x55f3858bb4e0 msgr2=0x55f385c67f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:39.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.945+0000 7f7e7b3b5740 1 -- 172.21.15.96:0/3517573294 shutdown_connections 2024-04-25T14:13:39.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.945+0000 7f7e7b3b5740 1 -- 172.21.15.96:0/3517573294 wait complete. 2024-04-25T14:13:39.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.946+0000 7f7e7b3b5740 20 client.15576 trim_cache size 0 max 16384 2024-04-25T14:13:39.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.959+0000 7f7e7b3b5740 1 Processor -- start 2024-04-25T14:13:39.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.959+0000 7f7e7b3b5740 1 -- start start 2024-04-25T14:13:39.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.959+0000 7f7e7b3b5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f385885f60 0x55f385c72110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.959+0000 7f7e7b3b5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f38589e4d0 0x55f385c86470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.959+0000 7f7e7b3b5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3858a66b0 0x55f385c869b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.959+0000 7f7e7b3b5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c1ab1e0 con 0x55f3858a66b0 2024-04-25T14:13:39.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.959+0000 7f7e7b3b5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c1acd20 con 0x55f38589e4d0 2024-04-25T14:13:39.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.959+0000 7f7e7b3b5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c1ab8b0 con 0x55f385885f60 2024-04-25T14:13:39.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.959+0000 7f7e729cd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3858a66b0 0x55f385c869b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.959+0000 7f7e729cd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3858a66b0 0x55f385c869b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41036/0 (socket says 172.21.15.96:41036) 2024-04-25T14:13:39.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.959+0000 7f7e729cd640 1 -- 172.21.15.96:0/629735535 learned_addr learned my addr 172.21.15.96:0/629735535 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:39.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.960+0000 7f7e719cb640 1 --2- 172.21.15.96:0/629735535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f38589e4d0 0x55f385c86470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.960+0000 7f7e729cd640 1 -- 172.21.15.96:0/629735535 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f385885f60 msgr2=0x55f385c72110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:39.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.960+0000 7f7e729cd640 1 --2- 172.21.15.96:0/629735535 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f385885f60 0x55f385c72110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.960+0000 7f7e729cd640 1 -- 172.21.15.96:0/629735535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f38589e4d0 msgr2=0x55f385c86470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:39.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.960+0000 7f7e729cd640 1 --2- 172.21.15.96:0/629735535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f38589e4d0 0x55f385c86470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.960+0000 7f7e729cd640 1 -- 172.21.15.96:0/629735535 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f385c71680 con 0x55f3858a66b0 2024-04-25T14:13:39.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.960+0000 7f7e729cd640 1 --2- 172.21.15.96:0/629735535 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3858a66b0 0x55f385c869b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7e6401d7d0 tx=0x7f7e64009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6107e0cb1451cfc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:39.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.960+0000 7f7e57fff640 1 -- 172.21.15.96:0/629735535 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e64012b70 con 0x55f3858a66b0 2024-04-25T14:13:39.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.960+0000 7f7e57fff640 1 -- 172.21.15.96:0/629735535 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e64012d10 con 0x55f3858a66b0 2024-04-25T14:13:39.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.960+0000 7f7e57fff640 1 -- 172.21.15.96:0/629735535 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6401a7b0 con 0x55f3858a66b0 2024-04-25T14:13:39.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.961+0000 7f7e7b3b5740 1 -- 172.21.15.96:0/629735535 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3858a66b0 msgr2=0x55f385c869b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.961+0000 7f7e7b3b5740 1 --2- 172.21.15.96:0/629735535 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3858a66b0 0x55f385c869b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7e6401d7d0 tx=0x7f7e64009f90 comp rx=0 tx=0).stop 2024-04-25T14:13:39.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.961+0000 7f7e7b3b5740 1 -- 172.21.15.96:0/629735535 shutdown_connections 2024-04-25T14:13:39.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.961+0000 7f7e7b3b5740 1 --2- 172.21.15.96:0/629735535 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3858a66b0 0x55f385c869b0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.961+0000 7f7e7b3b5740 1 --2- 172.21.15.96:0/629735535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f38589e4d0 0x55f385c86470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.961+0000 7f7e7b3b5740 1 --2- 172.21.15.96:0/629735535 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f385885f60 0x55f385c72110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.961+0000 7f7e7b3b5740 1 -- 172.21.15.96:0/629735535 >> 172.21.15.96:0/629735535 conn(0x55f3858bb4e0 msgr2=0x55f385c6e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:39.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.961+0000 7f7e7b3b5740 1 -- 172.21.15.96:0/629735535 shutdown_connections 2024-04-25T14:13:39.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.961+0000 7f7e7b3b5740 1 -- 172.21.15.96:0/629735535 wait complete. 2024-04-25T14:13:39.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.961+0000 7f7e7b3b5740 1 -- start start 2024-04-25T14:13:39.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.961+0000 7f7e7b3b5740 1 Processor -- start 2024-04-25T14:13:39.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.961+0000 7f7e7b3b5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f385885f60 0x55f385ce4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.961+0000 7f7e7b3b5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f38589e4d0 0x55f385ce4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.961+0000 7f7e7b3b5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3858a66b0 0x55f385ceb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.961+0000 7f7e7b3b5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c1ac650 con 0x55f3858a66b0 2024-04-25T14:13:39.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.961+0000 7f7e7b3b5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c1abf80 con 0x55f385885f60 2024-04-25T14:13:39.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.961+0000 7f7e7b3b5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c1a91c0 con 0x55f38589e4d0 2024-04-25T14:13:39.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.961+0000 7f7e7b3b5740 10 client.0 _ll_register_callbacks cb 0x7f7e60031d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:13:39.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.961+0000 7f7e729cd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3858a66b0 0x55f385ceb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.961+0000 7f7e729cd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3858a66b0 0x55f385ceb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41044/0 (socket says 172.21.15.96:41044) 2024-04-25T14:13:39.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.961+0000 7f7e729cd640 1 -- 172.21.15.96:0/665990475 learned_addr learned my addr 172.21.15.96:0/665990475 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:39.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.962+0000 7f7e721cc640 1 --2- 172.21.15.96:0/665990475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f385885f60 0x55f385ce4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.962+0000 7f7e719cb640 1 --2- 172.21.15.96:0/665990475 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f38589e4d0 0x55f385ce4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.962+0000 7f7e729cd640 1 -- 172.21.15.96:0/665990475 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f38589e4d0 msgr2=0x55f385ce4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:39.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.962+0000 7f7e729cd640 1 --2- 172.21.15.96:0/665990475 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f38589e4d0 0x55f385ce4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.962+0000 7f7e729cd640 1 -- 172.21.15.96:0/665990475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f385885f60 msgr2=0x55f385ce4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:39.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.962+0000 7f7e729cd640 1 --2- 172.21.15.96:0/665990475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f385885f60 0x55f385ce4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.962+0000 7f7e729cd640 1 -- 172.21.15.96:0/665990475 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f385cebd90 con 0x55f3858a66b0 2024-04-25T14:13:39.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.962+0000 7f7e721cc640 1 --2- 172.21.15.96:0/665990475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f385885f60 0x55f385ce4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:39.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.962+0000 7f7e729cd640 1 --2- 172.21.15.96:0/665990475 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3858a66b0 0x55f385ceb7d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f7e64004ce0 tx=0x7f7e64004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c8a748fd9ac2306 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:39.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.962+0000 7f7e55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:13:39.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.962+0000 7f7e55ffb640 1 -- 172.21.15.96:0/665990475 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e64027890 con 0x55f3858a66b0 2024-04-25T14:13:39.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.962+0000 7f7e55ffb640 1 -- 172.21.15.96:0/665990475 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f7e64027a30 con 0x55f3858a66b0 2024-04-25T14:13:39.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.962+0000 7f7e7b3b5740 1 -- 172.21.15.96:0/665990475 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e64012d10 con 0x55f3858a66b0 2024-04-25T14:13:39.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.962+0000 7f7e55ffb640 1 -- 172.21.15.96:0/665990475 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7e64005040 con 0x55f3858a66b0 2024-04-25T14:13:39.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.962+0000 7f7e7b3b5740 10 client.15582 fetch_fsmap learned FSMap version 26 2024-04-25T14:13:39.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.962+0000 7f7e7b3b5740 1 -- 172.21.15.96:0/665990475 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f385ced040 con 0x55f3858a66b0 2024-04-25T14:13:39.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.962+0000 7f7e55ffb640 1 -- 172.21.15.96:0/665990475 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e64012d10 con 0x55f3858a66b0 2024-04-25T14:13:39.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.962+0000 7f7e55ffb640 1 -- 172.21.15.96:0/665990475 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7e6402e020 con 0x55f3858a66b0 2024-04-25T14:13:39.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.963+0000 7f7e7b3b5740 10 client.15582 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:13:39.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.963+0000 7f7e7b3b5740 10 client.15582 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:13:39.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.963+0000 7f7e7b3b5740 20 client.15582 populate_metadata read hostname 'smithi096' 2024-04-25T14:13:39.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.963+0000 7f7e7b3b5740 1 --2- 172.21.15.96:0/665990475 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x55f3858c7390 0x55f385cef9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.963+0000 7f7e7b3b5740 4 client.15582 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:13:39.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.963+0000 7f7e7b3b5740 1 -- 172.21.15.96:0/665990475 --> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f385bd5800 con 0x55f3858c7390 2024-04-25T14:13:39.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.963+0000 7f7e721cc640 1 --2- 172.21.15.96:0/665990475 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x55f3858c7390 0x55f385cef9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.963+0000 7f7e55ffb640 10 client.15582 ms_handle_connect on v2:172.21.15.151:6838/245699981 2024-04-25T14:13:39.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.963+0000 7f7e721cc640 1 --2- 172.21.15.96:0/665990475 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x55f3858c7390 0x55f385cef9b0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8b255f47d891bd16 server_cookie=3d4090e5bed0569d in_seq=0 out_seq=0 2024-04-25T14:13:39.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.963+0000 7f7e55ffb640 1 -- 172.21.15.96:0/665990475 <== mds.1 v2:172.21.15.151:6838/245699981 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f385bd5800 con 0x55f3858c7390 2024-04-25T14:13:39.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.963+0000 7f7e55ffb640 10 client.15582 handle_command_reply: tid=0 2024-04-25T14:13:39.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.963+0000 7f7e7b3b5740 1 client.15582 shutdown 2024-04-25T14:13:39.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.964+0000 7f7e7b3b5740 1 -- 172.21.15.96:0/665990475 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3858a66b0 msgr2=0x55f385ceb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.964+0000 7f7e7b3b5740 1 --2- 172.21.15.96:0/665990475 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3858a66b0 0x55f385ceb7d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f7e64004ce0 tx=0x7f7e64004d10 comp rx=0 tx=0).stop 2024-04-25T14:13:39.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.964+0000 7f7e7b3b5740 1 -- 172.21.15.96:0/665990475 shutdown_connections 2024-04-25T14:13:39.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.964+0000 7f7e7b3b5740 1 --2- 172.21.15.96:0/665990475 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x55f3858c7390 0x55f385cef9b0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.964+0000 7f7e7b3b5740 1 --2- 172.21.15.96:0/665990475 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3858a66b0 0x55f385ceb7d0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.964+0000 7f7e7b3b5740 1 --2- 172.21.15.96:0/665990475 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f38589e4d0 0x55f385ce4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.964+0000 7f7e55ffb640 0 client.15582 ms_handle_reset on v2:172.21.15.151:6838/245699981 2024-04-25T14:13:39.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.964+0000 7f7e7b3b5740 1 --2- 172.21.15.96:0/665990475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f385885f60 0x55f385ce4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.964+0000 7f7e7b3b5740 1 -- 172.21.15.96:0/665990475 >> 172.21.15.96:0/665990475 conn(0x55f3858bb4e0 msgr2=0x55f385c67e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:39.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.964+0000 7f7e7b3b5740 1 -- 172.21.15.96:0/665990475 shutdown_connections 2024-04-25T14:13:39.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.964+0000 7f7e7b3b5740 1 -- 172.21.15.96:0/665990475 wait complete. 2024-04-25T14:13:39.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.964+0000 7f7e7b3b5740 20 client.15582 trim_cache size 0 max 16384 2024-04-25T14:13:39.970 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-04-25T14:13:39.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.964+0000 7f7e74430640 1 -- 172.21.15.96:0/3373176018 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f7e580afaf0 msgr2=0x7f7e580b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.964+0000 7f7e74430640 1 --2- 172.21.15.96:0/3373176018 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f7e580afaf0 0x7f7e580b2000 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55f385c6a860 tx=0x7f7e64004a40 comp rx=0 tx=0).stop 2024-04-25T14:13:39.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.964+0000 7f7e74430640 1 -- 172.21.15.96:0/3373176018 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e6c103af0 msgr2=0x7f7e6c19a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.964+0000 7f7e74430640 1 --2- 172.21.15.96:0/3373176018 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e6c103af0 0x7f7e6c19a1e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f7e6800c970 tx=0x7f7e6800ce30 comp rx=0 tx=0).stop 2024-04-25T14:13:39.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.964+0000 7f7e74430640 1 -- 172.21.15.96:0/3373176018 shutdown_connections 2024-04-25T14:13:39.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.964+0000 7f7e74430640 1 --2- 172.21.15.96:0/3373176018 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e6c109a30 0x7f7e6c1a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.964+0000 7f7e74430640 1 --2- 172.21.15.96:0/3373176018 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e6c104d30 0x7f7e6c19a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.964+0000 7f7e74430640 1 --2- 172.21.15.96:0/3373176018 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f7e580afaf0 0x7f7e580b2000 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.964+0000 7f7e74430640 1 --2- 172.21.15.96:0/3373176018 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e6c103af0 0x7f7e6c19a1e0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.964+0000 7f7e74430640 1 -- 172.21.15.96:0/3373176018 >> 172.21.15.96:0/3373176018 conn(0x7f7e6c0fece0 msgr2=0x7f7e6c105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:39.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.964+0000 7f7e74430640 1 -- 172.21.15.96:0/3373176018 shutdown_connections 2024-04-25T14:13:39.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:39.964+0000 7f7e74430640 1 -- 172.21.15.96:0/3373176018 wait complete. 2024-04-25T14:13:39.973 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:39 smithi096 ceph-mon[29207]: scrub queued for path: / 2024-04-25T14:13:39.974 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:39 smithi096 ceph-mon[29207]: scrub summary: idle+waiting paths [/] 2024-04-25T14:13:39.974 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:39 smithi096 ceph-mon[29207]: scrub summary: active paths [/] 2024-04-25T14:13:39.974 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:39 smithi096 ceph-mon[29207]: scrub summary: idle+waiting paths [/] 2024-04-25T14:13:39.974 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:39 smithi096 ceph-mon[29207]: scrub summary: idle 2024-04-25T14:13:39.974 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:39 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/3232318215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:13:39.974 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:39 smithi096 ceph-mon[29207]: pgmap v347: 97 pgs: 97 active+clean; 648 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 938 B/s wr, 9 op/s 2024-04-25T14:13:39.975 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T14:13:40.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.086+0000 7f8edf420640 1 -- 172.21.15.96:0/270415065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ed8103af0 msgr2=0x7f8ed81047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:40.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.086+0000 7f8edf420640 1 --2- 172.21.15.96:0/270415065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ed8103af0 0x7f8ed81047f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0009ca0 tx=0x7f8ed002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:13:40.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.086+0000 7f8edf420640 1 -- 172.21.15.96:0/270415065 shutdown_connections 2024-04-25T14:13:40.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.086+0000 7f8edf420640 1 --2- 172.21.15.96:0/270415065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ed8109a30 0x7f8ed810be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:40.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.086+0000 7f8edf420640 1 --2- 172.21.15.96:0/270415065 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ed8104d30 0x7f8ed8105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:40.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.086+0000 7f8edf420640 1 --2- 172.21.15.96:0/270415065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ed8103af0 0x7f8ed81047f0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:40.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.086+0000 7f8edf420640 1 -- 172.21.15.96:0/270415065 >> 172.21.15.96:0/270415065 conn(0x7f8ed80fece0 msgr2=0x7f8ed8101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:40.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.086+0000 7f8edf420640 1 -- 172.21.15.96:0/270415065 shutdown_connections 2024-04-25T14:13:40.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.086+0000 7f8edf420640 1 -- 172.21.15.96:0/270415065 wait complete. 2024-04-25T14:13:40.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.086+0000 7f8edf420640 1 Processor -- start 2024-04-25T14:13:40.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.086+0000 7f8edf420640 1 -- start start 2024-04-25T14:13:40.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.087+0000 7f8edf420640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ed8103af0 0x7f8ed819a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:40.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.087+0000 7f8edf420640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ed8104d30 0x7f8ed819a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:40.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.087+0000 7f8edf420640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ed8109a30 0x7f8ed81a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:40.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.087+0000 7f8edf420640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed810f620 con 0x7f8ed8109a30 2024-04-25T14:13:40.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.087+0000 7f8edf420640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed810f4a0 con 0x7f8ed8104d30 2024-04-25T14:13:40.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.087+0000 7f8edf420640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed810f7a0 con 0x7f8ed8103af0 2024-04-25T14:13:40.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.087+0000 7f8edd9bd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ed8109a30 0x7f8ed81a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:40.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.087+0000 7f8edd9bd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ed8109a30 0x7f8ed81a1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41062/0 (socket says 172.21.15.96:41062) 2024-04-25T14:13:40.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.087+0000 7f8edd9bd640 1 -- 172.21.15.96:0/4161992420 learned_addr learned my addr 172.21.15.96:0/4161992420 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:40.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.087+0000 7f8edc9bb640 1 --2- 172.21.15.96:0/4161992420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ed8104d30 0x7f8ed819a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:40.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.087+0000 7f8edd1bc640 1 --2- 172.21.15.96:0/4161992420 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ed8103af0 0x7f8ed819a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:40.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.087+0000 7f8edd9bd640 1 -- 172.21.15.96:0/4161992420 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ed8103af0 msgr2=0x7f8ed819a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:40.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.087+0000 7f8edd9bd640 1 --2- 172.21.15.96:0/4161992420 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ed8103af0 0x7f8ed819a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:40.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.087+0000 7f8edd9bd640 1 -- 172.21.15.96:0/4161992420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ed8104d30 msgr2=0x7f8ed819a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:40.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.087+0000 7f8edd9bd640 1 --2- 172.21.15.96:0/4161992420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ed8104d30 0x7f8ed819a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:40.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.087+0000 7f8edd9bd640 1 -- 172.21.15.96:0/4161992420 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ed81a1d90 con 0x7f8ed8109a30 2024-04-25T14:13:40.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.087+0000 7f8edd9bd640 1 --2- 172.21.15.96:0/4161992420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ed8109a30 0x7f8ed81a1720 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8ed400d750 tx=0x7f8ed400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:40.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.087+0000 7f8ece7fc640 1 -- 172.21.15.96:0/4161992420 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4017070 con 0x7f8ed8109a30 2024-04-25T14:13:40.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.087+0000 7f8ece7fc640 1 -- 172.21.15.96:0/4161992420 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ed40054d0 con 0x7f8ed8109a30 2024-04-25T14:13:40.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.087+0000 7f8edf420640 1 -- 172.21.15.96:0/4161992420 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ed81a2080 con 0x7f8ed8109a30 2024-04-25T14:13:40.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.087+0000 7f8ece7fc640 1 -- 172.21.15.96:0/4161992420 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4004c60 con 0x7f8ed8109a30 2024-04-25T14:13:40.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.087+0000 7f8edf420640 1 -- 172.21.15.96:0/4161992420 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ed81a2620 con 0x7f8ed8109a30 2024-04-25T14:13:40.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.090+0000 7f8ee63af740 1 Processor -- start 2024-04-25T14:13:40.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.090+0000 7f8ece7fc640 1 -- 172.21.15.96:0/4161992420 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f8ed4009bd0 con 0x7f8ed8109a30 2024-04-25T14:13:40.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.090+0000 7f8ee63af740 1 -- start start 2024-04-25T14:13:40.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.090+0000 7f8ee63af740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559405c24390 0x559405fc6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:40.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.090+0000 7f8ee63af740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559405bfb4d0 0x559405fc6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:40.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.090+0000 7f8ee63af740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559405c036b0 0x559405fc7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:40.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.090+0000 7f8ee63af740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559405f2ad80 con 0x559405c24390 2024-04-25T14:13:40.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.090+0000 7f8ee63af740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559405ee4a20 con 0x559405bfb4d0 2024-04-25T14:13:40.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.090+0000 7f8ee63af740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559405f26d40 con 0x559405c036b0 2024-04-25T14:13:40.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.091+0000 7f8edc9bb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559405c24390 0x559405fc6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:40.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.091+0000 7f8edc9bb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559405c24390 0x559405fc6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41074/0 (socket says 172.21.15.96:41074) 2024-04-25T14:13:40.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.091+0000 7f8edc9bb640 1 -- 172.21.15.96:0/991208539 learned_addr learned my addr 172.21.15.96:0/991208539 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:40.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.091+0000 7f8edd1bc640 1 --2- 172.21.15.96:0/991208539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559405bfb4d0 0x559405fc6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:40.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.091+0000 7f8edc9bb640 1 --2- 172.21.15.96:0/991208539 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559405c036b0 0x559405fc7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:40.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.091+0000 7f8ece7fc640 1 --2- 172.21.15.96:0/4161992420 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f8ec40afb70 0x7f8ec40b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:40.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.091+0000 7f8edc9bb640 1 -- 172.21.15.96:0/991208539 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559405c036b0 msgr2=0x559405fc7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:40.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.091+0000 7f8edc9bb640 1 --2- 172.21.15.96:0/991208539 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559405c036b0 0x559405fc7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:40.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.091+0000 7f8edc9bb640 1 -- 172.21.15.96:0/991208539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559405bfb4d0 msgr2=0x559405fc6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:40.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.091+0000 7f8edc9bb640 1 --2- 172.21.15.96:0/991208539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559405bfb4d0 0x559405fc6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:40.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.091+0000 7f8edc9bb640 1 -- 172.21.15.96:0/991208539 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559405eefc30 con 0x559405c24390 2024-04-25T14:13:40.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.091+0000 7f8edd1bc640 1 --2- 172.21.15.96:0/991208539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559405bfb4d0 0x559405fc6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:40.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.091+0000 7f8edd9bd640 1 --2- 172.21.15.96:0/4161992420 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f8ec40afb70 0x7f8ec40b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:40.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.091+0000 7f8edc9bb640 1 --2- 172.21.15.96:0/991208539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559405c24390 0x559405fc6810 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8013530 tx=0x7f8ec800b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f26065f16422eb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:40.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.091+0000 7f8ece7fc640 1 -- 172.21.15.96:0/4161992420 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f8ed40de5f0 con 0x7f8ed8109a30 2024-04-25T14:13:40.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.091+0000 7f8ec37fe640 1 -- 172.21.15.96:0/991208539 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8019030 con 0x559405c24390 2024-04-25T14:13:40.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.091+0000 7f8ec37fe640 1 -- 172.21.15.96:0/991208539 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ec8011970 con 0x559405c24390 2024-04-25T14:13:40.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.091+0000 7f8ec37fe640 1 -- 172.21.15.96:0/991208539 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec801d760 con 0x559405c24390 2024-04-25T14:13:40.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.092+0000 7f8ee63af740 1 -- 172.21.15.96:0/991208539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559405c24390 msgr2=0x559405fc6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:40.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.092+0000 7f8edd9bd640 1 --2- 172.21.15.96:0/4161992420 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f8ec40afb70 0x7f8ec40b2080 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8ed4004770 tx=0x7f8ed40c9040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:40.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.092+0000 7f8ee63af740 1 --2- 172.21.15.96:0/991208539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559405c24390 0x559405fc6810 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8013530 tx=0x7f8ec800b690 comp rx=0 tx=0).stop 2024-04-25T14:13:40.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.092+0000 7f8ee63af740 1 -- 172.21.15.96:0/991208539 shutdown_connections 2024-04-25T14:13:40.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.092+0000 7f8ee63af740 1 --2- 172.21.15.96:0/991208539 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559405c036b0 0x559405fc7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:40.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.092+0000 7f8ee63af740 1 --2- 172.21.15.96:0/991208539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559405bfb4d0 0x559405fc6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:40.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.092+0000 7f8ee63af740 1 --2- 172.21.15.96:0/991208539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559405c24390 0x559405fc6810 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:40.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.092+0000 7f8ee63af740 1 -- 172.21.15.96:0/991208539 >> 172.21.15.96:0/991208539 conn(0x559405c184e0 msgr2=0x559405fc2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:40.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.092+0000 7f8ee63af740 1 -- 172.21.15.96:0/991208539 shutdown_connections 2024-04-25T14:13:40.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.092+0000 7f8ee63af740 1 -- 172.21.15.96:0/991208539 wait complete. 2024-04-25T14:13:40.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.093+0000 7f8ee63af740 1 -- start start 2024-04-25T14:13:40.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.093+0000 7f8ee63af740 1 Processor -- start 2024-04-25T14:13:40.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.093+0000 7f8ee63af740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559405bfb4d0 0x559405fd0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:40.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.093+0000 7f8ee63af740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559405c036b0 0x559405fd0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:40.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.093+0000 7f8ee63af740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559405c24390 0x559405fd79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:40.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.093+0000 7f8ee63af740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559405fc8530 con 0x559405bfb4d0 2024-04-25T14:13:40.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.093+0000 7f8ee63af740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559405eb1dc0 con 0x559405c24390 2024-04-25T14:13:40.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.093+0000 7f8ee63af740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559405fc86b0 con 0x559405c036b0 2024-04-25T14:13:40.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.093+0000 7f8ee63af740 10 client.0 _ll_register_callbacks cb 0x559405ecd600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:13:40.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.093+0000 7f8edc9bb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559405bfb4d0 0x559405fd0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:40.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.093+0000 7f8edc9bb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559405bfb4d0 0x559405fd0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41076/0 (socket says 172.21.15.96:41076) 2024-04-25T14:13:40.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.093+0000 7f8edc9bb640 1 -- 172.21.15.96:0/4165970598 learned_addr learned my addr 172.21.15.96:0/4165970598 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:40.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.093+0000 7f8edc9bb640 1 --2- 172.21.15.96:0/4165970598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559405c24390 0x559405fd79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:40.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.094+0000 7f8edd1bc640 1 --2- 172.21.15.96:0/4165970598 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559405c036b0 0x559405fd0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:40.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.094+0000 7f8edc9bb640 1 -- 172.21.15.96:0/4165970598 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559405c036b0 msgr2=0x559405fd0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:40.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.094+0000 7f8edc9bb640 1 --2- 172.21.15.96:0/4165970598 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559405c036b0 0x559405fd0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:40.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.094+0000 7f8edc9bb640 1 -- 172.21.15.96:0/4165970598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559405c24390 msgr2=0x559405fd79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:40.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.094+0000 7f8edc9bb640 1 --2- 172.21.15.96:0/4165970598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559405c24390 0x559405fd79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:40.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.094+0000 7f8edc9bb640 1 -- 172.21.15.96:0/4165970598 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559405c3c3f0 con 0x559405bfb4d0 2024-04-25T14:13:40.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.094+0000 7f8edc9bb640 1 --2- 172.21.15.96:0/4165970598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559405bfb4d0 0x559405fd0510 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8ec800bf20 tx=0x7f8ec8005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72dedb5c3ead7c50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:40.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.094+0000 7f8ec0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:13:40.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.094+0000 7f8ec0ff9640 1 -- 172.21.15.96:0/4165970598 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8019030 con 0x559405bfb4d0 2024-04-25T14:13:40.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.094+0000 7f8ec0ff9640 1 -- 172.21.15.96:0/4165970598 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f8ec8027040 con 0x559405bfb4d0 2024-04-25T14:13:40.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.094+0000 7f8ee63af740 1 -- 172.21.15.96:0/4165970598 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ec8011970 con 0x559405bfb4d0 2024-04-25T14:13:40.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.094+0000 7f8ec0ff9640 1 -- 172.21.15.96:0/4165970598 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8ec8005020 con 0x559405bfb4d0 2024-04-25T14:13:40.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.094+0000 7f8ee63af740 10 client.15594 fetch_fsmap learned FSMap version 26 2024-04-25T14:13:40.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.094+0000 7f8ee63af740 1 -- 172.21.15.96:0/4165970598 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559405d067d0 con 0x559405bfb4d0 2024-04-25T14:13:40.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.094+0000 7f8ec0ff9640 1 -- 172.21.15.96:0/4165970598 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ec8011970 con 0x559405bfb4d0 2024-04-25T14:13:40.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.094+0000 7f8ec0ff9640 1 -- 172.21.15.96:0/4165970598 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8ec802a020 con 0x559405bfb4d0 2024-04-25T14:13:40.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.095+0000 7f8ee63af740 10 client.15594 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:13:40.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.095+0000 7f8ee63af740 10 client.15594 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:13:40.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.095+0000 7f8ee63af740 20 client.15594 populate_metadata read hostname 'smithi096' 2024-04-25T14:13:40.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.095+0000 7f8ee63af740 1 --2- 172.21.15.96:0/4165970598 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x559405be2f60 0x559405fddf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:40.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.095+0000 7f8ee63af740 4 client.15594 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:13:40.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.095+0000 7f8ee63af740 1 -- 172.21.15.96:0/4165970598 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559405f38990 con 0x559405be2f60 2024-04-25T14:13:40.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.095+0000 7f8edd1bc640 1 --2- 172.21.15.96:0/4165970598 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x559405be2f60 0x559405fddf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:40.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.095+0000 7f8edd1bc640 1 --2- 172.21.15.96:0/4165970598 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x559405be2f60 0x559405fddf70 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=756d2275ca3018ce server_cookie=29d66fdbd33a5629 in_seq=0 out_seq=0 2024-04-25T14:13:40.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.095+0000 7f8ec0ff9640 10 client.15594 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:13:40.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.096+0000 7f8ec0ff9640 1 -- 172.21.15.96:0/4165970598 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559405f38990 con 0x559405be2f60 2024-04-25T14:13:40.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.096+0000 7f8ec0ff9640 10 client.15594 handle_command_reply: tid=0 2024-04-25T14:13:40.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.096+0000 7f8ee63af740 1 client.15594 shutdown 2024-04-25T14:13:40.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.096+0000 7f8ee63af740 1 -- 172.21.15.96:0/4165970598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559405bfb4d0 msgr2=0x559405fd0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:40.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.096+0000 7f8ee63af740 1 --2- 172.21.15.96:0/4165970598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559405bfb4d0 0x559405fd0510 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8ec800bf20 tx=0x7f8ec8005f30 comp rx=0 tx=0).stop 2024-04-25T14:13:40.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.096+0000 7f8ee63af740 1 -- 172.21.15.96:0/4165970598 shutdown_connections 2024-04-25T14:13:40.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.096+0000 7f8ee63af740 1 --2- 172.21.15.96:0/4165970598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559405c24390 0x559405fd79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:40.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.096+0000 7f8ee63af740 1 --2- 172.21.15.96:0/4165970598 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559405c036b0 0x559405fd0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:40.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.096+0000 7f8ee63af740 1 --2- 172.21.15.96:0/4165970598 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x559405be2f60 0x559405fddf70 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:40.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.096+0000 7f8ee63af740 1 --2- 172.21.15.96:0/4165970598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559405bfb4d0 0x559405fd0510 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:40.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.096+0000 7f8ec0ff9640 0 client.15594 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:13:40.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.096+0000 7f8ee63af740 1 -- 172.21.15.96:0/4165970598 >> 172.21.15.96:0/4165970598 conn(0x559405c184e0 msgr2=0x559405fc4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:40.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.097+0000 7f8ee63af740 1 -- 172.21.15.96:0/4165970598 shutdown_connections 2024-04-25T14:13:40.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.097+0000 7f8ee63af740 1 -- 172.21.15.96:0/4165970598 wait complete. 2024-04-25T14:13:40.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.097+0000 7f8ee63af740 20 client.15594 trim_cache size 0 max 16384 2024-04-25T14:13:40.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.110+0000 7f8ee63af740 1 Processor -- start 2024-04-25T14:13:40.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.110+0000 7f8ee63af740 1 -- start start 2024-04-25T14:13:40.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.110+0000 7f8ee63af740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559405bfb4d0 0x559405fc2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:40.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.110+0000 7f8ee63af740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559405c036b0 0x559405fd7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:40.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.110+0000 7f8ee63af740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559405c24390 0x559405fd00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:40.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.110+0000 7f8ee63af740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed81ab3f0 con 0x559405c036b0 2024-04-25T14:13:40.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.110+0000 7f8ee63af740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed81acf30 con 0x559405c24390 2024-04-25T14:13:40.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.110+0000 7f8ee63af740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed81abac0 con 0x559405bfb4d0 2024-04-25T14:13:40.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.110+0000 7f8edd1bc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559405c036b0 0x559405fd7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:40.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.110+0000 7f8edd1bc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559405c036b0 0x559405fd7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41090/0 (socket says 172.21.15.96:41090) 2024-04-25T14:13:40.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.110+0000 7f8edd1bc640 1 -- 172.21.15.96:0/4009652827 learned_addr learned my addr 172.21.15.96:0/4009652827 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:40.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.110+0000 7f8edd1bc640 1 -- 172.21.15.96:0/4009652827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559405bfb4d0 msgr2=0x559405fc2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:40.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.110+0000 7f8edd1bc640 1 --2- 172.21.15.96:0/4009652827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559405bfb4d0 0x559405fc2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:40.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.110+0000 7f8edc9bb640 1 --2- 172.21.15.96:0/4009652827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559405c24390 0x559405fd00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:40.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.110+0000 7f8edd1bc640 1 -- 172.21.15.96:0/4009652827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559405c24390 msgr2=0x559405fd00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:40.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.110+0000 7f8edd1bc640 1 --2- 172.21.15.96:0/4009652827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559405c24390 0x559405fd00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:40.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.110+0000 7f8edd1bc640 1 -- 172.21.15.96:0/4009652827 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559405fcbca0 con 0x559405c036b0 2024-04-25T14:13:40.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.111+0000 7f8edd1bc640 1 --2- 172.21.15.96:0/4009652827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559405c036b0 0x559405fd7940 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f8ed00096c0 tx=0x7f8ed0031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b900619d2c129afc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:40.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.111+0000 7f8ec2ffd640 1 -- 172.21.15.96:0/4009652827 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed0031710 con 0x559405c036b0 2024-04-25T14:13:40.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.111+0000 7f8ec2ffd640 1 -- 172.21.15.96:0/4009652827 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ed0036040 con 0x559405c036b0 2024-04-25T14:13:40.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.111+0000 7f8ec2ffd640 1 -- 172.21.15.96:0/4009652827 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed002f660 con 0x559405c036b0 2024-04-25T14:13:40.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.111+0000 7f8ee63af740 1 -- 172.21.15.96:0/4009652827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559405c036b0 msgr2=0x559405fd7940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:40.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.112+0000 7f8ee63af740 1 --2- 172.21.15.96:0/4009652827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559405c036b0 0x559405fd7940 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f8ed00096c0 tx=0x7f8ed0031b10 comp rx=0 tx=0).stop 2024-04-25T14:13:40.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.112+0000 7f8ee63af740 1 -- 172.21.15.96:0/4009652827 shutdown_connections 2024-04-25T14:13:40.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.112+0000 7f8ee63af740 1 --2- 172.21.15.96:0/4009652827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559405c24390 0x559405fd00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:40.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.112+0000 7f8ee63af740 1 --2- 172.21.15.96:0/4009652827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559405c036b0 0x559405fd7940 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:40.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.112+0000 7f8ee63af740 1 --2- 172.21.15.96:0/4009652827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559405bfb4d0 0x559405fc2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:40.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.112+0000 7f8ee63af740 1 -- 172.21.15.96:0/4009652827 >> 172.21.15.96:0/4009652827 conn(0x559405c184e0 msgr2=0x559405fc1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:40.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.112+0000 7f8ee63af740 1 -- 172.21.15.96:0/4009652827 shutdown_connections 2024-04-25T14:13:40.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.112+0000 7f8ee63af740 1 -- 172.21.15.96:0/4009652827 wait complete. 2024-04-25T14:13:40.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.112+0000 7f8ee63af740 1 -- start start 2024-04-25T14:13:40.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.112+0000 7f8ee63af740 1 Processor -- start 2024-04-25T14:13:40.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.112+0000 7f8ee63af740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559405bfb4d0 0x559405fd6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:40.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.112+0000 7f8ee63af740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559405c036b0 0x559405fd7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:40.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.112+0000 7f8ee63af740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559405c24390 0x5594060440c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:40.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.112+0000 7f8ee63af740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed81ac860 con 0x559405c24390 2024-04-25T14:13:40.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.112+0000 7f8ee63af740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed81ac190 con 0x559405c036b0 2024-04-25T14:13:40.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.112+0000 7f8ee63af740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed81a93d0 con 0x559405bfb4d0 2024-04-25T14:13:40.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.112+0000 7f8ee63af740 10 client.0 _ll_register_callbacks cb 0x7f8ec8013400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:13:40.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.113+0000 7f8edc9bb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559405c24390 0x5594060440c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:40.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.113+0000 7f8edc9bb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559405c24390 0x5594060440c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41104/0 (socket says 172.21.15.96:41104) 2024-04-25T14:13:40.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.113+0000 7f8edc9bb640 1 -- 172.21.15.96:0/1935915703 learned_addr learned my addr 172.21.15.96:0/1935915703 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:40.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.113+0000 7f8edc9bb640 1 --2- 172.21.15.96:0/1935915703 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559405bfb4d0 0x559405fd6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:40.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.113+0000 7f8edd1bc640 1 --2- 172.21.15.96:0/1935915703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559405c036b0 0x559405fd7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:40.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.113+0000 7f8edc9bb640 1 -- 172.21.15.96:0/1935915703 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559405bfb4d0 msgr2=0x559405fd6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:40.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.113+0000 7f8edc9bb640 1 --2- 172.21.15.96:0/1935915703 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559405bfb4d0 0x559405fd6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:40.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.113+0000 7f8edc9bb640 1 -- 172.21.15.96:0/1935915703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559405c036b0 msgr2=0x559405fd7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:40.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.113+0000 7f8edc9bb640 1 --2- 172.21.15.96:0/1935915703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559405c036b0 0x559405fd7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:40.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.113+0000 7f8edc9bb640 1 -- 172.21.15.96:0/1935915703 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559406044650 con 0x559405c24390 2024-04-25T14:13:40.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.113+0000 7f8edc9bb640 1 --2- 172.21.15.96:0/1935915703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559405c24390 0x5594060440c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8004f10 tx=0x7f8ec8001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dced3be942462da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:40.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.113+0000 7f8ec0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:13:40.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.113+0000 7f8ec0ff9640 1 -- 172.21.15.96:0/1935915703 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8019020 con 0x559405c24390 2024-04-25T14:13:40.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.113+0000 7f8ec0ff9640 1 -- 172.21.15.96:0/1935915703 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f8ec80116e0 con 0x559405c24390 2024-04-25T14:13:40.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.113+0000 7f8ee63af740 1 -- 172.21.15.96:0/1935915703 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ed0036040 con 0x559405c24390 2024-04-25T14:13:40.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.113+0000 7f8ec0ff9640 1 -- 172.21.15.96:0/1935915703 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8ec800b6a0 con 0x559405c24390 2024-04-25T14:13:40.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.113+0000 7f8ee63af740 10 client.15600 fetch_fsmap learned FSMap version 26 2024-04-25T14:13:40.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.113+0000 7f8ee63af740 1 -- 172.21.15.96:0/1935915703 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5594060458d0 con 0x559405c24390 2024-04-25T14:13:40.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.113+0000 7f8ec0ff9640 1 -- 172.21.15.96:0/1935915703 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ed0036040 con 0x559405c24390 2024-04-25T14:13:40.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.114+0000 7f8ec0ff9640 1 -- 172.21.15.96:0/1935915703 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8ec802a030 con 0x559405c24390 2024-04-25T14:13:40.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.114+0000 7f8ee63af740 10 client.15600 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:13:40.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.114+0000 7f8ee63af740 10 client.15600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:13:40.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.114+0000 7f8ee63af740 20 client.15600 populate_metadata read hostname 'smithi096' 2024-04-25T14:13:40.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.114+0000 7f8ee63af740 1 --2- 172.21.15.96:0/1935915703 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x559405bb83a0 0x559406048290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:40.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.114+0000 7f8ee63af740 4 client.15600 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:13:40.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.114+0000 7f8ee63af740 1 -- 172.21.15.96:0/1935915703 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559405f32800 con 0x559405bb83a0 2024-04-25T14:13:40.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.114+0000 7f8edd1bc640 1 --2- 172.21.15.96:0/1935915703 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x559405bb83a0 0x559406048290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:40.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.115+0000 7f8ec0ff9640 10 client.15600 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:13:40.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.115+0000 7f8edd1bc640 1 --2- 172.21.15.96:0/1935915703 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x559405bb83a0 0x559406048290 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7bacaba31bfd9a8 server_cookie=102954e05261bca0 in_seq=0 out_seq=0 2024-04-25T14:13:40.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.115+0000 7f8ec0ff9640 1 -- 172.21.15.96:0/1935915703 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559405f32800 con 0x559405bb83a0 2024-04-25T14:13:40.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.115+0000 7f8ec0ff9640 10 client.15600 handle_command_reply: tid=0 2024-04-25T14:13:40.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.115+0000 7f8ee63af740 1 client.15600 shutdown 2024-04-25T14:13:40.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.115+0000 7f8ee63af740 1 -- 172.21.15.96:0/1935915703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559405c24390 msgr2=0x5594060440c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:40.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.115+0000 7f8ee63af740 1 --2- 172.21.15.96:0/1935915703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559405c24390 0x5594060440c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8004f10 tx=0x7f8ec8001e50 comp rx=0 tx=0).stop 2024-04-25T14:13:40.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.115+0000 7f8ee63af740 1 -- 172.21.15.96:0/1935915703 shutdown_connections 2024-04-25T14:13:40.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.115+0000 7f8ee63af740 1 --2- 172.21.15.96:0/1935915703 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x559405bb83a0 0x559406048290 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:40.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.115+0000 7f8ee63af740 1 --2- 172.21.15.96:0/1935915703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559405c24390 0x5594060440c0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:40.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.115+0000 7f8ee63af740 1 --2- 172.21.15.96:0/1935915703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559405c036b0 0x559405fd7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:40.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.115+0000 7f8ec0ff9640 0 client.15600 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:13:40.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.115+0000 7f8ee63af740 1 --2- 172.21.15.96:0/1935915703 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559405bfb4d0 0x559405fd6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:40.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.115+0000 7f8ee63af740 1 -- 172.21.15.96:0/1935915703 >> 172.21.15.96:0/1935915703 conn(0x559405c184e0 msgr2=0x559405fc4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:40.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.115+0000 7f8ee63af740 1 -- 172.21.15.96:0/1935915703 shutdown_connections 2024-04-25T14:13:40.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.115+0000 7f8ee63af740 1 -- 172.21.15.96:0/1935915703 wait complete. 2024-04-25T14:13:40.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.115+0000 7f8ee63af740 20 client.15600 trim_cache size 0 max 16384 2024-04-25T14:13:40.121 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-04-25T14:13:40.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.116+0000 7f8edf420640 1 -- 172.21.15.96:0/4161992420 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f8ec40afb70 msgr2=0x7f8ec40b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:40.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.116+0000 7f8edf420640 1 --2- 172.21.15.96:0/4161992420 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f8ec40afb70 0x7f8ec40b2080 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8ed4004770 tx=0x7f8ed40c9040 comp rx=0 tx=0).stop 2024-04-25T14:13:40.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.116+0000 7f8edf420640 1 -- 172.21.15.96:0/4161992420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ed8109a30 msgr2=0x7f8ed81a1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:40.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.116+0000 7f8edf420640 1 --2- 172.21.15.96:0/4161992420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ed8109a30 0x7f8ed81a1720 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8ed400d750 tx=0x7f8ed400dc10 comp rx=0 tx=0).stop 2024-04-25T14:13:40.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.116+0000 7f8edf420640 1 -- 172.21.15.96:0/4161992420 shutdown_connections 2024-04-25T14:13:40.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.116+0000 7f8edf420640 1 --2- 172.21.15.96:0/4161992420 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f8ec40afb70 0x7f8ec40b2080 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:40.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.116+0000 7f8edf420640 1 --2- 172.21.15.96:0/4161992420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ed8109a30 0x7f8ed81a1720 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:40.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.116+0000 7f8edf420640 1 --2- 172.21.15.96:0/4161992420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ed8104d30 0x7f8ed819a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:40.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.116+0000 7f8edf420640 1 --2- 172.21.15.96:0/4161992420 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ed8103af0 0x7f8ed819a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:40.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.116+0000 7f8edf420640 1 -- 172.21.15.96:0/4161992420 >> 172.21.15.96:0/4161992420 conn(0x7f8ed80fece0 msgr2=0x7f8ed8105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:40.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.116+0000 7f8edf420640 1 -- 172.21.15.96:0/4161992420 shutdown_connections 2024-04-25T14:13:40.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:40.116+0000 7f8edf420640 1 -- 172.21.15.96:0/4161992420 wait complete. 2024-04-25T14:13:40.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:39 smithi097 ceph-mon[32720]: scrub queued for path: / 2024-04-25T14:13:40.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:39 smithi097 ceph-mon[32720]: scrub summary: idle+waiting paths [/] 2024-04-25T14:13:40.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:39 smithi097 ceph-mon[32720]: scrub summary: active paths [/] 2024-04-25T14:13:40.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:39 smithi097 ceph-mon[32720]: scrub summary: idle+waiting paths [/] 2024-04-25T14:13:40.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:39 smithi097 ceph-mon[32720]: scrub summary: idle 2024-04-25T14:13:40.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:39 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/3232318215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:13:40.173 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:39 smithi097 ceph-mon[32720]: pgmap v347: 97 pgs: 97 active+clean; 648 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 938 B/s wr, 9 op/s 2024-04-25T14:13:41.126 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T14:13:41.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:13:41.126 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T14:13:41.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.253+0000 7f625fb8f640 1 -- 172.21.15.96:0/2807679271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6258108940 msgr2=0x7f625810ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:41.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.253+0000 7f625fb8f640 1 --2- 172.21.15.96:0/2807679271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6258108940 0x7f625810ad60 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f6250009ca0 tx=0x7f625002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.253+0000 7f625fb8f640 1 -- 172.21.15.96:0/2807679271 shutdown_connections 2024-04-25T14:13:41.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.253+0000 7f625fb8f640 1 --2- 172.21.15.96:0/2807679271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6258108940 0x7f625810ad60 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.253+0000 7f625fb8f640 1 --2- 172.21.15.96:0/2807679271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6258103c40 0x7f6258104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.253+0000 7f625fb8f640 1 --2- 172.21.15.96:0/2807679271 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6258102a00 0x7f6258103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.253+0000 7f625fb8f640 1 -- 172.21.15.96:0/2807679271 >> 172.21.15.96:0/2807679271 conn(0x7f62580fdbf0 msgr2=0x7f62581000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:41.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.253+0000 7f625fb8f640 1 -- 172.21.15.96:0/2807679271 shutdown_connections 2024-04-25T14:13:41.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.253+0000 7f625fb8f640 1 -- 172.21.15.96:0/2807679271 wait complete. 2024-04-25T14:13:41.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.254+0000 7f625fb8f640 1 Processor -- start 2024-04-25T14:13:41.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.254+0000 7f625fb8f640 1 -- start start 2024-04-25T14:13:41.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.254+0000 7f625fb8f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6258102a00 0x7f6258199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:41.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.254+0000 7f625fb8f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6258103c40 0x7f62581995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:41.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.254+0000 7f625fb8f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6258108940 0x7f62581a0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:41.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.254+0000 7f625fb8f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f625810e510 con 0x7f6258102a00 2024-04-25T14:13:41.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.254+0000 7f625fb8f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f625810e390 con 0x7f6258108940 2024-04-25T14:13:41.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.254+0000 7f625fb8f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f625810e690 con 0x7f6258103c40 2024-04-25T14:13:41.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.254+0000 7f625d92b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6258102a00 0x7f6258199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:41.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.254+0000 7f625d92b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6258102a00 0x7f6258199080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41122/0 (socket says 172.21.15.96:41122) 2024-04-25T14:13:41.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.254+0000 7f625d92b640 1 -- 172.21.15.96:0/1445386134 learned_addr learned my addr 172.21.15.96:0/1445386134 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:41.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.254+0000 7f625e12c640 1 --2- 172.21.15.96:0/1445386134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6258108940 0x7f62581a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:41.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.255+0000 7f625d12a640 1 --2- 172.21.15.96:0/1445386134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6258103c40 0x7f62581995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:41.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.255+0000 7f625d92b640 1 -- 172.21.15.96:0/1445386134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6258103c40 msgr2=0x7f62581995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:41.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.255+0000 7f625d92b640 1 --2- 172.21.15.96:0/1445386134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6258103c40 0x7f62581995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.255+0000 7f625d92b640 1 -- 172.21.15.96:0/1445386134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6258108940 msgr2=0x7f62581a0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:41.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.255+0000 7f625d92b640 1 --2- 172.21.15.96:0/1445386134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6258108940 0x7f62581a0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.255+0000 7f625d92b640 1 -- 172.21.15.96:0/1445386134 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62581a0b10 con 0x7f6258102a00 2024-04-25T14:13:41.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.255+0000 7f625d92b640 1 --2- 172.21.15.96:0/1445386134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6258102a00 0x7f6258199080 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f625400c970 tx=0x7f625400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:41.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.255+0000 7f624effd640 1 -- 172.21.15.96:0/1445386134 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6254004220 con 0x7f6258102a00 2024-04-25T14:13:41.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.255+0000 7f624effd640 1 -- 172.21.15.96:0/1445386134 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62540043c0 con 0x7f6258102a00 2024-04-25T14:13:41.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.255+0000 7f624effd640 1 -- 172.21.15.96:0/1445386134 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625400f450 con 0x7f6258102a00 2024-04-25T14:13:41.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.255+0000 7f625fb8f640 1 -- 172.21.15.96:0/1445386134 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62581a0e00 con 0x7f6258102a00 2024-04-25T14:13:41.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.255+0000 7f625fb8f640 1 -- 172.21.15.96:0/1445386134 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62581a12c0 con 0x7f6258102a00 2024-04-25T14:13:41.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.258+0000 7f6266a43740 1 Processor -- start 2024-04-25T14:13:41.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.258+0000 7f6266a43740 1 -- start start 2024-04-25T14:13:41.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.258+0000 7f624effd640 1 -- 172.21.15.96:0/1445386134 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f62540079b0 con 0x7f6258102a00 2024-04-25T14:13:41.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.258+0000 7f6266a43740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55718e5d0410 0x55718e9736b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:41.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.258+0000 7f6266a43740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55718e5a7550 0x55718e973c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:41.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.258+0000 7f6266a43740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55718e5af730 0x55718e97ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:41.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.258+0000 7f6266a43740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55718e8d6b00 con 0x55718e5a7550 2024-04-25T14:13:41.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.258+0000 7f6266a43740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55718e872920 con 0x55718e5d0410 2024-04-25T14:13:41.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.258+0000 7f6266a43740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55718e8d2ac0 con 0x55718e5af730 2024-04-25T14:13:41.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.258+0000 7f625d92b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55718e5a7550 0x55718e973c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:41.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.258+0000 7f625d92b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55718e5a7550 0x55718e973c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41132/0 (socket says 172.21.15.96:41132) 2024-04-25T14:13:41.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.258+0000 7f625d92b640 1 -- 172.21.15.96:0/4174166995 learned_addr learned my addr 172.21.15.96:0/4174166995 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:41.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.258+0000 7f625e12c640 1 --2- 172.21.15.96:0/4174166995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55718e5d0410 0x55718e9736b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:41.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.258+0000 7f625d12a640 1 --2- 172.21.15.96:0/4174166995 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55718e5af730 0x55718e97ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:41.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.259+0000 7f625d92b640 1 -- 172.21.15.96:0/4174166995 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55718e5af730 msgr2=0x55718e97ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:41.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.259+0000 7f625d92b640 1 --2- 172.21.15.96:0/4174166995 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55718e5af730 0x55718e97ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.259+0000 7f625d92b640 1 -- 172.21.15.96:0/4174166995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55718e5d0410 msgr2=0x55718e9736b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:41.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.259+0000 7f625d92b640 1 --2- 172.21.15.96:0/4174166995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55718e5d0410 0x55718e9736b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.259+0000 7f625d92b640 1 -- 172.21.15.96:0/4174166995 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55718e6b07e0 con 0x55718e5a7550 2024-04-25T14:13:41.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.259+0000 7f625d12a640 1 --2- 172.21.15.96:0/4174166995 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55718e5af730 0x55718e97ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:41.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.259+0000 7f624effd640 1 --2- 172.21.15.96:0/1445386134 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f62440afb70 0x7f62440b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:41.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.259+0000 7f625d92b640 1 --2- 172.21.15.96:0/4174166995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55718e5a7550 0x55718e973c30 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f62540e2f00 tx=0x7f62540e5460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5551db9a8d0ae846 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:41.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.259+0000 7f625e12c640 1 --2- 172.21.15.96:0/1445386134 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f62440afb70 0x7f62440b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:41.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.259+0000 7f6243fff640 1 -- 172.21.15.96:0/4174166995 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62540a3160 con 0x55718e5a7550 2024-04-25T14:13:41.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.259+0000 7f624effd640 1 -- 172.21.15.96:0/1445386134 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f62540dede0 con 0x7f6258102a00 2024-04-25T14:13:41.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.259+0000 7f6243fff640 1 -- 172.21.15.96:0/4174166995 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62540a6db0 con 0x55718e5a7550 2024-04-25T14:13:41.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.259+0000 7f6243fff640 1 -- 172.21.15.96:0/4174166995 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62540a6a10 con 0x55718e5a7550 2024-04-25T14:13:41.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.260+0000 7f6266a43740 1 -- 172.21.15.96:0/4174166995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55718e5a7550 msgr2=0x55718e973c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:41.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.260+0000 7f625e12c640 1 --2- 172.21.15.96:0/1445386134 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f62440afb70 0x7f62440b2080 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x55718e974880 tx=0x7f6250031af0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:41.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.260+0000 7f6266a43740 1 --2- 172.21.15.96:0/4174166995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55718e5a7550 0x55718e973c30 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f62540e2f00 tx=0x7f62540e5460 comp rx=0 tx=0).stop 2024-04-25T14:13:41.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.260+0000 7f6266a43740 1 -- 172.21.15.96:0/4174166995 shutdown_connections 2024-04-25T14:13:41.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.260+0000 7f6266a43740 1 --2- 172.21.15.96:0/4174166995 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55718e5af730 0x55718e97ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.260+0000 7f6266a43740 1 --2- 172.21.15.96:0/4174166995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55718e5a7550 0x55718e973c30 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.260+0000 7f6266a43740 1 --2- 172.21.15.96:0/4174166995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55718e5d0410 0x55718e9736b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.260+0000 7f6266a43740 1 -- 172.21.15.96:0/4174166995 >> 172.21.15.96:0/4174166995 conn(0x55718e5c4560 msgr2=0x55718e96f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:41.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.260+0000 7f6266a43740 1 -- 172.21.15.96:0/4174166995 shutdown_connections 2024-04-25T14:13:41.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.260+0000 7f6266a43740 1 -- 172.21.15.96:0/4174166995 wait complete. 2024-04-25T14:13:41.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.261+0000 7f6266a43740 1 -- start start 2024-04-25T14:13:41.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.261+0000 7f6266a43740 1 Processor -- start 2024-04-25T14:13:41.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.261+0000 7f6266a43740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55718e5a7550 0x55718e97d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:41.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.261+0000 7f6266a43740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55718e5af730 0x55718e97daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:41.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.261+0000 7f6266a43740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55718e5d0410 0x55718e984a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:41.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.261+0000 7f6266a43740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55718e97b580 con 0x55718e5a7550 2024-04-25T14:13:41.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.261+0000 7f6266a43740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55718e8705d0 con 0x55718e5d0410 2024-04-25T14:13:41.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.261+0000 7f6266a43740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55718e97b700 con 0x55718e5af730 2024-04-25T14:13:41.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.261+0000 7f6266a43740 10 client.0 _ll_register_callbacks cb 0x55718e85e080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:13:41.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.261+0000 7f625d92b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55718e5a7550 0x55718e97d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:41.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.261+0000 7f625d92b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55718e5a7550 0x55718e97d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41144/0 (socket says 172.21.15.96:41144) 2024-04-25T14:13:41.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.261+0000 7f625d92b640 1 -- 172.21.15.96:0/1806442011 learned_addr learned my addr 172.21.15.96:0/1806442011 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:41.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.261+0000 7f625e12c640 1 --2- 172.21.15.96:0/1806442011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55718e5d0410 0x55718e984a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:41.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.261+0000 7f625d92b640 1 -- 172.21.15.96:0/1806442011 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55718e5af730 msgr2=0x55718e97daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:41.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.261+0000 7f625d92b640 1 --2- 172.21.15.96:0/1806442011 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55718e5af730 0x55718e97daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.261+0000 7f625d92b640 1 -- 172.21.15.96:0/1806442011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55718e5d0410 msgr2=0x55718e984a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:41.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.261+0000 7f625d92b640 1 --2- 172.21.15.96:0/1806442011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55718e5d0410 0x55718e984a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.261+0000 7f625d92b640 1 -- 172.21.15.96:0/1806442011 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55718e710f70 con 0x55718e5a7550 2024-04-25T14:13:41.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.262+0000 7f625d92b640 1 --2- 172.21.15.96:0/1806442011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55718e5a7550 0x55718e97d560 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f62540039b0 tx=0x7f62540e2690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b35050f3eea707a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:41.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.262+0000 7f62417fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:13:41.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.262+0000 7f62417fa640 1 -- 172.21.15.96:0/1806442011 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625401d040 con 0x55718e5a7550 2024-04-25T14:13:41.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.262+0000 7f62417fa640 1 -- 172.21.15.96:0/1806442011 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f6254007ca0 con 0x55718e5a7550 2024-04-25T14:13:41.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.262+0000 7f6266a43740 1 -- 172.21.15.96:0/1806442011 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62540a6db0 con 0x55718e5a7550 2024-04-25T14:13:41.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.262+0000 7f62417fa640 1 -- 172.21.15.96:0/1806442011 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f62540a6bf0 con 0x55718e5a7550 2024-04-25T14:13:41.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.262+0000 7f6266a43740 10 client.15612 fetch_fsmap learned FSMap version 26 2024-04-25T14:13:41.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.262+0000 7f6266a43740 1 -- 172.21.15.96:0/1806442011 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55718e5a8170 con 0x55718e5a7550 2024-04-25T14:13:41.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.262+0000 7f62417fa640 1 -- 172.21.15.96:0/1806442011 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62540a6db0 con 0x55718e5a7550 2024-04-25T14:13:41.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.262+0000 7f62417fa640 1 -- 172.21.15.96:0/1806442011 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f62540e50d0 con 0x55718e5a7550 2024-04-25T14:13:41.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.262+0000 7f6266a43740 10 client.15612 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:13:41.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.262+0000 7f6266a43740 10 client.15612 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:13:41.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.262+0000 7f6266a43740 20 client.15612 populate_metadata read hostname 'smithi096' 2024-04-25T14:13:41.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.262+0000 7f6266a43740 1 --2- 172.21.15.96:0/1806442011 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55718e58f1f0 0x55718e98aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:41.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.262+0000 7f6266a43740 4 client.15612 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:13:41.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.262+0000 7f6266a43740 1 -- 172.21.15.96:0/1806442011 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55718e8e4710 con 0x55718e58f1f0 2024-04-25T14:13:41.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.263+0000 7f625d12a640 1 --2- 172.21.15.96:0/1806442011 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55718e58f1f0 0x55718e98aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:41.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.263+0000 7f625d12a640 1 --2- 172.21.15.96:0/1806442011 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55718e58f1f0 0x55718e98aa30 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12d31be7784ddc61 server_cookie=911d4f47720ce190 in_seq=0 out_seq=0 2024-04-25T14:13:41.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.263+0000 7f62417fa640 10 client.15612 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:13:41.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.264+0000 7f62417fa640 1 -- 172.21.15.96:0/1806442011 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55718e8e4710 con 0x55718e58f1f0 2024-04-25T14:13:41.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.264+0000 7f62417fa640 10 client.15612 handle_command_reply: tid=0 2024-04-25T14:13:41.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.264+0000 7f6266a43740 1 client.15612 shutdown 2024-04-25T14:13:41.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.264+0000 7f6266a43740 1 -- 172.21.15.96:0/1806442011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55718e5a7550 msgr2=0x55718e97d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:41.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.264+0000 7f6266a43740 1 --2- 172.21.15.96:0/1806442011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55718e5a7550 0x55718e97d560 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f62540039b0 tx=0x7f62540e2690 comp rx=0 tx=0).stop 2024-04-25T14:13:41.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.264+0000 7f6266a43740 1 -- 172.21.15.96:0/1806442011 shutdown_connections 2024-04-25T14:13:41.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.264+0000 7f6266a43740 1 --2- 172.21.15.96:0/1806442011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55718e5d0410 0x55718e984a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.264+0000 7f6266a43740 1 --2- 172.21.15.96:0/1806442011 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55718e5af730 0x55718e97daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.264+0000 7f6266a43740 1 --2- 172.21.15.96:0/1806442011 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55718e58f1f0 0x55718e98aa30 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.264+0000 7f6266a43740 1 --2- 172.21.15.96:0/1806442011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55718e5a7550 0x55718e97d560 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.264+0000 7f62417fa640 0 client.15612 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:13:41.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.264+0000 7f6266a43740 1 -- 172.21.15.96:0/1806442011 >> 172.21.15.96:0/1806442011 conn(0x55718e5c4560 msgr2=0x55718e96bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:41.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.264+0000 7f6266a43740 1 -- 172.21.15.96:0/1806442011 shutdown_connections 2024-04-25T14:13:41.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.264+0000 7f6266a43740 1 -- 172.21.15.96:0/1806442011 wait complete. 2024-04-25T14:13:41.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.264+0000 7f6266a43740 20 client.15612 trim_cache size 0 max 16384 2024-04-25T14:13:41.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.283+0000 7f6266a43740 1 Processor -- start 2024-04-25T14:13:41.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.283+0000 7f6266a43740 1 -- start start 2024-04-25T14:13:41.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.283+0000 7f6266a43740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55718e5a7550 0x55718e96efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:41.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.283+0000 7f6266a43740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55718e5af730 0x55718e97d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:41.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.283+0000 7f6266a43740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55718e5d0410 0x55718e97d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:41.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.283+0000 7f6266a43740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f62581a90d0 con 0x55718e5d0410 2024-04-25T14:13:41.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.283+0000 7f6266a43740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f62581aac10 con 0x55718e5af730 2024-04-25T14:13:41.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.283+0000 7f6266a43740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f62581a97a0 con 0x55718e5a7550 2024-04-25T14:13:41.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.283+0000 7f625e12c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55718e5d0410 0x55718e97d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:41.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.283+0000 7f625e12c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55718e5d0410 0x55718e97d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41160/0 (socket says 172.21.15.96:41160) 2024-04-25T14:13:41.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.283+0000 7f625e12c640 1 -- 172.21.15.96:0/889528371 learned_addr learned my addr 172.21.15.96:0/889528371 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:41.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.283+0000 7f625d92b640 1 --2- 172.21.15.96:0/889528371 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55718e5a7550 0x55718e96efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:41.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.283+0000 7f625e12c640 1 -- 172.21.15.96:0/889528371 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55718e5a7550 msgr2=0x55718e96efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:41.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.283+0000 7f625e12c640 1 --2- 172.21.15.96:0/889528371 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55718e5a7550 0x55718e96efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.283+0000 7f625e12c640 1 -- 172.21.15.96:0/889528371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55718e5af730 msgr2=0x55718e97d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:41.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.283+0000 7f625d12a640 1 --2- 172.21.15.96:0/889528371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55718e5af730 0x55718e97d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:41.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.283+0000 7f625e12c640 1 --2- 172.21.15.96:0/889528371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55718e5af730 0x55718e97d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.283+0000 7f625e12c640 1 -- 172.21.15.96:0/889528371 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55718e9783b0 con 0x55718e5d0410 2024-04-25T14:13:41.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.283+0000 7f625d92b640 1 --2- 172.21.15.96:0/889528371 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55718e5a7550 0x55718e96efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:41.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.283+0000 7f625e12c640 1 --2- 172.21.15.96:0/889528371 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55718e5d0410 0x55718e97d670 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f6250035000 tx=0x7f6250041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e48a190ababf5918 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:41.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.284+0000 7f62437fe640 1 -- 172.21.15.96:0/889528371 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625002f660 con 0x55718e5d0410 2024-04-25T14:13:41.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.284+0000 7f62437fe640 1 -- 172.21.15.96:0/889528371 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6250036040 con 0x55718e5d0410 2024-04-25T14:13:41.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.284+0000 7f62437fe640 1 -- 172.21.15.96:0/889528371 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6250030a60 con 0x55718e5d0410 2024-04-25T14:13:41.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.284+0000 7f6266a43740 1 -- 172.21.15.96:0/889528371 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55718e5d0410 msgr2=0x55718e97d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:41.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.284+0000 7f6266a43740 1 --2- 172.21.15.96:0/889528371 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55718e5d0410 0x55718e97d670 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f6250035000 tx=0x7f6250041000 comp rx=0 tx=0).stop 2024-04-25T14:13:41.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.284+0000 7f6266a43740 1 -- 172.21.15.96:0/889528371 shutdown_connections 2024-04-25T14:13:41.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.284+0000 7f6266a43740 1 --2- 172.21.15.96:0/889528371 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55718e5d0410 0x55718e97d670 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.284+0000 7f6266a43740 1 --2- 172.21.15.96:0/889528371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55718e5af730 0x55718e97d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.284+0000 7f6266a43740 1 --2- 172.21.15.96:0/889528371 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55718e5a7550 0x55718e96efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.284+0000 7f6266a43740 1 -- 172.21.15.96:0/889528371 >> 172.21.15.96:0/889528371 conn(0x55718e5c4560 msgr2=0x55718e96e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:41.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.284+0000 7f6266a43740 1 -- 172.21.15.96:0/889528371 shutdown_connections 2024-04-25T14:13:41.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.284+0000 7f6266a43740 1 -- 172.21.15.96:0/889528371 wait complete. 2024-04-25T14:13:41.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.285+0000 7f6266a43740 1 -- start start 2024-04-25T14:13:41.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.285+0000 7f6266a43740 1 Processor -- start 2024-04-25T14:13:41.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.285+0000 7f6266a43740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55718e5a7550 0x55718e981cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:41.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.285+0000 7f6266a43740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55718e5af730 0x55718e982200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:41.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.285+0000 7f6266a43740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55718e5d0410 0x55718e9830c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:41.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.285+0000 7f6266a43740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f62581aa540 con 0x55718e5a7550 2024-04-25T14:13:41.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.285+0000 7f6266a43740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f62581a9e70 con 0x55718e5d0410 2024-04-25T14:13:41.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.285+0000 7f6266a43740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f62581a70b0 con 0x55718e5af730 2024-04-25T14:13:41.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.285+0000 7f6266a43740 10 client.0 _ll_register_callbacks cb 0x7f62540e2e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:13:41.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.285+0000 7f625d92b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55718e5a7550 0x55718e981cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:41.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.285+0000 7f625d92b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55718e5a7550 0x55718e981cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41164/0 (socket says 172.21.15.96:41164) 2024-04-25T14:13:41.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.285+0000 7f625d92b640 1 -- 172.21.15.96:0/3421051928 learned_addr learned my addr 172.21.15.96:0/3421051928 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:41.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.285+0000 7f625e12c640 1 --2- 172.21.15.96:0/3421051928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55718e5d0410 0x55718e9830c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:41.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.285+0000 7f625d12a640 1 --2- 172.21.15.96:0/3421051928 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55718e5af730 0x55718e982200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:41.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.286+0000 7f625d92b640 1 -- 172.21.15.96:0/3421051928 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55718e5af730 msgr2=0x55718e982200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:41.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.286+0000 7f625d92b640 1 --2- 172.21.15.96:0/3421051928 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55718e5af730 0x55718e982200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.286+0000 7f625d92b640 1 -- 172.21.15.96:0/3421051928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55718e5d0410 msgr2=0x55718e9830c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:41.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.286+0000 7f625d92b640 1 --2- 172.21.15.96:0/3421051928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55718e5d0410 0x55718e9830c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.286+0000 7f625d92b640 1 -- 172.21.15.96:0/3421051928 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55718e983680 con 0x55718e5a7550 2024-04-25T14:13:41.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.286+0000 7f625d92b640 1 --2- 172.21.15.96:0/3421051928 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55718e5a7550 0x55718e981cc0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f62540a4510 tx=0x7f6254007ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=341be02653c33761 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:41.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.286+0000 7f62417fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:13:41.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.286+0000 7f62417fa640 1 -- 172.21.15.96:0/3421051928 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625401d030 con 0x55718e5a7550 2024-04-25T14:13:41.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.286+0000 7f62417fa640 1 -- 172.21.15.96:0/3421051928 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f62540afe80 con 0x55718e5a7550 2024-04-25T14:13:41.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.286+0000 7f6266a43740 1 -- 172.21.15.96:0/3421051928 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6250036040 con 0x55718e5a7550 2024-04-25T14:13:41.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.286+0000 7f62417fa640 1 -- 172.21.15.96:0/3421051928 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f62540a7070 con 0x55718e5a7550 2024-04-25T14:13:41.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.286+0000 7f6266a43740 10 client.15618 fetch_fsmap learned FSMap version 26 2024-04-25T14:13:41.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.286+0000 7f6266a43740 1 -- 172.21.15.96:0/3421051928 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55718e9f3060 con 0x55718e5a7550 2024-04-25T14:13:41.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.286+0000 7f62417fa640 1 -- 172.21.15.96:0/3421051928 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6250036040 con 0x55718e5a7550 2024-04-25T14:13:41.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.286+0000 7f62417fa640 1 -- 172.21.15.96:0/3421051928 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f62540a7500 con 0x55718e5a7550 2024-04-25T14:13:41.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.286+0000 7f6266a43740 10 client.15618 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:13:41.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.286+0000 7f6266a43740 10 client.15618 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:13:41.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.286+0000 7f6266a43740 20 client.15618 populate_metadata read hostname 'smithi096' 2024-04-25T14:13:41.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.286+0000 7f6266a43740 1 --2- 172.21.15.96:0/3421051928 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55718e564630 0x55718e9f57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:41.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.286+0000 7f6266a43740 4 client.15618 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T14:13:41.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.286+0000 7f6266a43740 1 -- 172.21.15.96:0/3421051928 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55718e8de580 con 0x55718e564630 2024-04-25T14:13:41.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.287+0000 7f625d12a640 1 --2- 172.21.15.96:0/3421051928 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55718e564630 0x55718e9f57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:41.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.287+0000 7f625d12a640 1 --2- 172.21.15.96:0/3421051928 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55718e564630 0x55718e9f57e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c02767d135874b01 server_cookie=2f61d1e7c0b0d751 in_seq=0 out_seq=0 2024-04-25T14:13:41.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.287+0000 7f62417fa640 10 client.15618 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:13:41.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.288+0000 7f62417fa640 1 -- 172.21.15.96:0/3421051928 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55718e8de580 con 0x55718e564630 2024-04-25T14:13:41.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.288+0000 7f62417fa640 10 client.15618 handle_command_reply: tid=0 2024-04-25T14:13:41.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.288+0000 7f6266a43740 1 client.15618 shutdown 2024-04-25T14:13:41.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.288+0000 7f6266a43740 1 -- 172.21.15.96:0/3421051928 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55718e5a7550 msgr2=0x55718e981cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:41.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.288+0000 7f6266a43740 1 --2- 172.21.15.96:0/3421051928 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55718e5a7550 0x55718e981cc0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f62540a4510 tx=0x7f6254007ed0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.289+0000 7f6266a43740 1 -- 172.21.15.96:0/3421051928 shutdown_connections 2024-04-25T14:13:41.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.289+0000 7f6266a43740 1 --2- 172.21.15.96:0/3421051928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55718e5d0410 0x55718e9830c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.289+0000 7f6266a43740 1 --2- 172.21.15.96:0/3421051928 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55718e5af730 0x55718e982200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.289+0000 7f6266a43740 1 --2- 172.21.15.96:0/3421051928 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55718e564630 0x55718e9f57e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.289+0000 7f6266a43740 1 --2- 172.21.15.96:0/3421051928 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55718e5a7550 0x55718e981cc0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.289+0000 7f62417fa640 0 client.15618 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:13:41.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.289+0000 7f6266a43740 1 -- 172.21.15.96:0/3421051928 >> 172.21.15.96:0/3421051928 conn(0x55718e5c4560 msgr2=0x55718e97d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:41.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.289+0000 7f6266a43740 1 -- 172.21.15.96:0/3421051928 shutdown_connections 2024-04-25T14:13:41.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.289+0000 7f6266a43740 1 -- 172.21.15.96:0/3421051928 wait complete. 2024-04-25T14:13:41.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.289+0000 7f6266a43740 20 client.15618 trim_cache size 0 max 16384 2024-04-25T14:13:41.294 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T14:13:41.294 INFO:teuthology.orchestra.run.smithi096.stdout: "return_code": 0, 2024-04-25T14:13:41.294 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_tag": "c6fd2c96-201c-45ea-8c66-6409bd03e99c", 2024-04-25T14:13:41.294 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": "asynchronous" 2024-04-25T14:13:41.294 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T14:13:41.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.289+0000 7f625fb8f640 1 -- 172.21.15.96:0/1445386134 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f62440afb70 msgr2=0x7f62440b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:41.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.289+0000 7f625fb8f640 1 --2- 172.21.15.96:0/1445386134 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f62440afb70 0x7f62440b2080 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x55718e974880 tx=0x7f6250031af0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.289+0000 7f625fb8f640 1 -- 172.21.15.96:0/1445386134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6258102a00 msgr2=0x7f6258199080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:41.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.289+0000 7f625fb8f640 1 --2- 172.21.15.96:0/1445386134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6258102a00 0x7f6258199080 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f625400c970 tx=0x7f625400ce30 comp rx=0 tx=0).stop 2024-04-25T14:13:41.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.289+0000 7f625fb8f640 1 -- 172.21.15.96:0/1445386134 shutdown_connections 2024-04-25T14:13:41.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.289+0000 7f625fb8f640 1 --2- 172.21.15.96:0/1445386134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6258108940 0x7f62581a0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.289+0000 7f625fb8f640 1 --2- 172.21.15.96:0/1445386134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6258103c40 0x7f62581995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.289+0000 7f625fb8f640 1 --2- 172.21.15.96:0/1445386134 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f62440afb70 0x7f62440b2080 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.289+0000 7f625fb8f640 1 --2- 172.21.15.96:0/1445386134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6258102a00 0x7f6258199080 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.289+0000 7f625fb8f640 1 -- 172.21.15.96:0/1445386134 >> 172.21.15.96:0/1445386134 conn(0x7f62580fdbf0 msgr2=0x7f6258104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:41.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.289+0000 7f625fb8f640 1 -- 172.21.15.96:0/1445386134 shutdown_connections 2024-04-25T14:13:41.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.289+0000 7f625fb8f640 1 -- 172.21.15.96:0/1445386134 wait complete. 2024-04-25T14:13:41.299 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:13:41.299 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:13:41.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.410+0000 7f90a2d82640 1 -- 172.21.15.96:0/2744757326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f909c108920 msgr2=0x7f909c10ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:41.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.410+0000 7f90a2d82640 1 --2- 172.21.15.96:0/2744757326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f909c108920 0x7f909c10ad40 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f9094009ca0 tx=0x7f909402a8c0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.410+0000 7f90a2d82640 1 -- 172.21.15.96:0/2744757326 shutdown_connections 2024-04-25T14:13:41.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.410+0000 7f90a2d82640 1 --2- 172.21.15.96:0/2744757326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f909c108920 0x7f909c10ad40 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.410+0000 7f90a2d82640 1 --2- 172.21.15.96:0/2744757326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f909c103c20 0x7f909c104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.410+0000 7f90a2d82640 1 --2- 172.21.15.96:0/2744757326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f909c1029e0 0x7f909c1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.410+0000 7f90a2d82640 1 -- 172.21.15.96:0/2744757326 >> 172.21.15.96:0/2744757326 conn(0x7f909c0fdbd0 msgr2=0x7f909c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:41.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.410+0000 7f90a2d82640 1 -- 172.21.15.96:0/2744757326 shutdown_connections 2024-04-25T14:13:41.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.410+0000 7f90a2d82640 1 -- 172.21.15.96:0/2744757326 wait complete. 2024-04-25T14:13:41.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.410+0000 7f90a2d82640 1 Processor -- start 2024-04-25T14:13:41.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.410+0000 7f90a2d82640 1 -- start start 2024-04-25T14:13:41.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.411+0000 7f90a2d82640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f909c1029e0 0x7f909c19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:41.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.411+0000 7f90a2d82640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f909c103c20 0x7f909c19aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:41.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.411+0000 7f90a2d82640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f909c108920 0x7f909c1a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:41.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.411+0000 7f90a2d82640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f909c10e4f0 con 0x7f909c108920 2024-04-25T14:13:41.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.411+0000 7f90a2d82640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f909c10e370 con 0x7f909c103c20 2024-04-25T14:13:41.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.411+0000 7f90a2d82640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f909c10e670 con 0x7f909c1029e0 2024-04-25T14:13:41.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.411+0000 7f90a131f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f909c108920 0x7f909c1a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:41.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.411+0000 7f90a131f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f909c108920 0x7f909c1a19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41178/0 (socket says 172.21.15.96:41178) 2024-04-25T14:13:41.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.411+0000 7f90a131f640 1 -- 172.21.15.96:0/2990568314 learned_addr learned my addr 172.21.15.96:0/2990568314 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:41.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.411+0000 7f909bfff640 1 --2- 172.21.15.96:0/2990568314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f909c103c20 0x7f909c19aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:41.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.411+0000 7f90a0b1e640 1 --2- 172.21.15.96:0/2990568314 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f909c1029e0 0x7f909c19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:41.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.411+0000 7f90a131f640 1 -- 172.21.15.96:0/2990568314 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f909c1029e0 msgr2=0x7f909c19a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:41.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.411+0000 7f90a131f640 1 --2- 172.21.15.96:0/2990568314 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f909c1029e0 0x7f909c19a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.411+0000 7f90a131f640 1 -- 172.21.15.96:0/2990568314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f909c103c20 msgr2=0x7f909c19aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:41.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.411+0000 7f90a131f640 1 --2- 172.21.15.96:0/2990568314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f909c103c20 0x7f909c19aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.411+0000 7f90a131f640 1 -- 172.21.15.96:0/2990568314 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f909c1a2020 con 0x7f909c108920 2024-04-25T14:13:41.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.411+0000 7f909bfff640 1 --2- 172.21.15.96:0/2990568314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f909c103c20 0x7f909c19aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:41.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.411+0000 7f90a131f640 1 --2- 172.21.15.96:0/2990568314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f909c108920 0x7f909c1a19b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f90940097b0 tx=0x7f9094003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:41.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.411+0000 7f9099ffb640 1 -- 172.21.15.96:0/2990568314 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9094004240 con 0x7f909c108920 2024-04-25T14:13:41.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.411+0000 7f9099ffb640 1 -- 172.21.15.96:0/2990568314 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f909402f380 con 0x7f909c108920 2024-04-25T14:13:41.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.412+0000 7f90a2d82640 1 -- 172.21.15.96:0/2990568314 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f909c1a22b0 con 0x7f909c108920 2024-04-25T14:13:41.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.412+0000 7f9099ffb640 1 -- 172.21.15.96:0/2990568314 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909402fc30 con 0x7f909c108920 2024-04-25T14:13:41.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.412+0000 7f90a2d82640 1 -- 172.21.15.96:0/2990568314 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f909c1a2800 con 0x7f909c108920 2024-04-25T14:13:41.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.414+0000 7f90a9c36740 1 Processor -- start 2024-04-25T14:13:41.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.414+0000 7f9099ffb640 1 -- 172.21.15.96:0/2990568314 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f90940043e0 con 0x7f909c108920 2024-04-25T14:13:41.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.414+0000 7f90a9c36740 1 -- start start 2024-04-25T14:13:41.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.414+0000 7f90a9c36740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583bd09c390 0x5583bd43da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:41.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.414+0000 7f90a9c36740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583bd0734d0 0x5583bd43dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:41.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.414+0000 7f90a9c36740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583bd07b6b0 0x5583bd43ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:41.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.414+0000 7f90a9c36740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5583bd3a2d80 con 0x5583bd07b6b0 2024-04-25T14:13:41.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.414+0000 7f90a9c36740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5583bd35c8b0 con 0x5583bd09c390 2024-04-25T14:13:41.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.414+0000 7f90a9c36740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5583bd39ed40 con 0x5583bd0734d0 2024-04-25T14:13:41.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.415+0000 7f909bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583bd09c390 0x5583bd43da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:41.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.415+0000 7f909bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583bd09c390 0x5583bd43da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:57978/0 (socket says 172.21.15.96:57978) 2024-04-25T14:13:41.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.415+0000 7f909bfff640 1 -- 172.21.15.96:0/3522025769 learned_addr learned my addr 172.21.15.96:0/3522025769 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:41.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.415+0000 7f909bfff640 1 --2- 172.21.15.96:0/3522025769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583bd07b6b0 0x5583bd43ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:41.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.415+0000 7f90a0b1e640 1 --2- 172.21.15.96:0/3522025769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583bd0734d0 0x5583bd43dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:41.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.415+0000 7f9099ffb640 1 --2- 172.21.15.96:0/2990568314 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f908c0afaf0 0x7f908c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:41.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.415+0000 7f909bfff640 1 -- 172.21.15.96:0/3522025769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583bd0734d0 msgr2=0x5583bd43dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:41.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.415+0000 7f909bfff640 1 --2- 172.21.15.96:0/3522025769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583bd0734d0 0x5583bd43dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.415+0000 7f909bfff640 1 -- 172.21.15.96:0/3522025769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583bd07b6b0 msgr2=0x5583bd43ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:41.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.415+0000 7f909bfff640 1 --2- 172.21.15.96:0/3522025769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583bd07b6b0 0x5583bd43ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.415+0000 7f909bfff640 1 -- 172.21.15.96:0/3522025769 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583bd32da40 con 0x5583bd09c390 2024-04-25T14:13:41.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.415+0000 7f90a131f640 1 --2- 172.21.15.96:0/2990568314 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f908c0afaf0 0x7f908c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:41.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.415+0000 7f9099ffb640 1 -- 172.21.15.96:0/2990568314 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f9094100460 con 0x7f909c108920 2024-04-25T14:13:41.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.415+0000 7f909bfff640 1 --2- 172.21.15.96:0/3522025769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583bd09c390 0x5583bd43da60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9088013970 tx=0x7f908800b4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ede20cb06fd9c2aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:41.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.415+0000 7f9086ffd640 1 -- 172.21.15.96:0/3522025769 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9088004020 con 0x5583bd09c390 2024-04-25T14:13:41.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.415+0000 7f90a131f640 1 --2- 172.21.15.96:0/2990568314 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f908c0afaf0 0x7f908c0b2000 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f90940c8620 tx=0x7f90940b1040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:41.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.415+0000 7f9086ffd640 1 -- 172.21.15.96:0/3522025769 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f908800ee10 con 0x5583bd09c390 2024-04-25T14:13:41.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.415+0000 7f9086ffd640 1 -- 172.21.15.96:0/3522025769 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9088003850 con 0x5583bd09c390 2024-04-25T14:13:41.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.416+0000 7f90a9c36740 1 -- 172.21.15.96:0/3522025769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583bd09c390 msgr2=0x5583bd43da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:41.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.416+0000 7f90a9c36740 1 --2- 172.21.15.96:0/3522025769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583bd09c390 0x5583bd43da60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9088013970 tx=0x7f908800b4a0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.416+0000 7f90a9c36740 1 -- 172.21.15.96:0/3522025769 shutdown_connections 2024-04-25T14:13:41.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.416+0000 7f90a9c36740 1 --2- 172.21.15.96:0/3522025769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583bd07b6b0 0x5583bd43ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.416+0000 7f90a9c36740 1 --2- 172.21.15.96:0/3522025769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583bd0734d0 0x5583bd43dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.416+0000 7f90a9c36740 1 --2- 172.21.15.96:0/3522025769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583bd09c390 0x5583bd43da60 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.416+0000 7f90a9c36740 1 -- 172.21.15.96:0/3522025769 >> 172.21.15.96:0/3522025769 conn(0x5583bd0904e0 msgr2=0x5583bd439270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:41.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.416+0000 7f90a9c36740 1 -- 172.21.15.96:0/3522025769 shutdown_connections 2024-04-25T14:13:41.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.416+0000 7f90a9c36740 1 -- 172.21.15.96:0/3522025769 wait complete. 2024-04-25T14:13:41.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.417+0000 7f90a9c36740 1 -- start start 2024-04-25T14:13:41.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.417+0000 7f90a9c36740 1 Processor -- start 2024-04-25T14:13:41.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.417+0000 7f90a9c36740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583bd0734d0 0x5583bd447990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:41.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.417+0000 7f90a9c36740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583bd07b6b0 0x5583bd447ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:41.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.417+0000 7f90a9c36740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583bd09c390 0x5583bd44edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:41.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.417+0000 7f90a9c36740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5583bd43f780 con 0x5583bd07b6b0 2024-04-25T14:13:41.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.417+0000 7f90a9c36740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5583bd329d10 con 0x5583bd09c390 2024-04-25T14:13:41.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.417+0000 7f90a9c36740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5583bd43f900 con 0x5583bd0734d0 2024-04-25T14:13:41.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.417+0000 7f90a9c36740 10 client.0 _ll_register_callbacks cb 0x5583bd356210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:13:41.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.417+0000 7f90a0b1e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583bd07b6b0 0x5583bd447ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:41.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.417+0000 7f90a0b1e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583bd07b6b0 0x5583bd447ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41192/0 (socket says 172.21.15.96:41192) 2024-04-25T14:13:41.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.417+0000 7f90a0b1e640 1 -- 172.21.15.96:0/1974851933 learned_addr learned my addr 172.21.15.96:0/1974851933 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:41.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.417+0000 7f909bfff640 1 --2- 172.21.15.96:0/1974851933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583bd0734d0 0x5583bd447990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:41.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.417+0000 7f909bfff640 1 --2- 172.21.15.96:0/1974851933 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583bd09c390 0x5583bd44edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:41.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.418+0000 7f90a0b1e640 1 -- 172.21.15.96:0/1974851933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583bd0734d0 msgr2=0x5583bd447990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:41.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.418+0000 7f90a0b1e640 1 --2- 172.21.15.96:0/1974851933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583bd0734d0 0x5583bd447990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.418+0000 7f90a0b1e640 1 -- 172.21.15.96:0/1974851933 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583bd09c390 msgr2=0x5583bd44edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:41.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.418+0000 7f90a0b1e640 1 --2- 172.21.15.96:0/1974851933 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583bd09c390 0x5583bd44edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.418+0000 7f90a0b1e640 1 -- 172.21.15.96:0/1974851933 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583bd17e7d0 con 0x5583bd07b6b0 2024-04-25T14:13:41.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.418+0000 7f90a0b1e640 1 --2- 172.21.15.96:0/1974851933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583bd07b6b0 0x5583bd447ed0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f909000be40 tx=0x7f9090012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6b5f8dbcef4fa24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:41.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.418+0000 7f907bfff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:13:41.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.418+0000 7f907bfff640 1 -- 172.21.15.96:0/1974851933 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90900108a0 con 0x5583bd07b6b0 2024-04-25T14:13:41.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.418+0000 7f907bfff640 1 -- 172.21.15.96:0/1974851933 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f9090010a40 con 0x5583bd07b6b0 2024-04-25T14:13:41.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.418+0000 7f90a9c36740 1 -- 172.21.15.96:0/1974851933 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f908800ee10 con 0x5583bd07b6b0 2024-04-25T14:13:41.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.418+0000 7f907bfff640 1 -- 172.21.15.96:0/1974851933 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9090010c40 con 0x5583bd07b6b0 2024-04-25T14:13:41.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.418+0000 7f90a9c36740 10 client.15630 fetch_fsmap learned FSMap version 26 2024-04-25T14:13:41.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.418+0000 7f90a9c36740 1 -- 172.21.15.96:0/1974851933 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5583bd0b43f0 con 0x5583bd07b6b0 2024-04-25T14:13:41.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.418+0000 7f907bfff640 1 -- 172.21.15.96:0/1974851933 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f908800ee10 con 0x5583bd07b6b0 2024-04-25T14:13:41.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.419+0000 7f907bfff640 1 -- 172.21.15.96:0/1974851933 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9090019070 con 0x5583bd07b6b0 2024-04-25T14:13:41.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.419+0000 7f90a9c36740 10 client.15630 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:13:41.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.419+0000 7f90a9c36740 10 client.15630 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:13:41.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.419+0000 7f90a9c36740 20 client.15630 populate_metadata read hostname 'smithi096' 2024-04-25T14:13:41.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.419+0000 7f90a9c36740 1 --2- 172.21.15.96:0/1974851933 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5583bd05af60 0x5583bd455340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:41.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.419+0000 7f90a9c36740 4 client.15630 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:13:41.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.419+0000 7f90a9c36740 1 -- 172.21.15.96:0/1974851933 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583bd3b0990 con 0x5583bd05af60 2024-04-25T14:13:41.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.419+0000 7f909bfff640 1 --2- 172.21.15.96:0/1974851933 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5583bd05af60 0x5583bd455340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:41.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.419+0000 7f907bfff640 10 client.15630 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:13:41.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.419+0000 7f909bfff640 1 --2- 172.21.15.96:0/1974851933 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5583bd05af60 0x5583bd455340 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98d8912280e103cc server_cookie=3f3ff5880325eca in_seq=0 out_seq=0 2024-04-25T14:13:41.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.420+0000 7f907bfff640 1 -- 172.21.15.96:0/1974851933 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5583bd3b0990 con 0x5583bd05af60 2024-04-25T14:13:41.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.420+0000 7f907bfff640 10 client.15630 handle_command_reply: tid=0 2024-04-25T14:13:41.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.420+0000 7f90a9c36740 1 client.15630 shutdown 2024-04-25T14:13:41.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.420+0000 7f90a9c36740 1 -- 172.21.15.96:0/1974851933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583bd07b6b0 msgr2=0x5583bd447ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:41.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.420+0000 7f90a9c36740 1 --2- 172.21.15.96:0/1974851933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583bd07b6b0 0x5583bd447ed0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f909000be40 tx=0x7f9090012820 comp rx=0 tx=0).stop 2024-04-25T14:13:41.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.420+0000 7f90a9c36740 1 -- 172.21.15.96:0/1974851933 shutdown_connections 2024-04-25T14:13:41.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.420+0000 7f90a9c36740 1 --2- 172.21.15.96:0/1974851933 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583bd09c390 0x5583bd44edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.420+0000 7f90a9c36740 1 --2- 172.21.15.96:0/1974851933 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5583bd05af60 0x5583bd455340 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.420+0000 7f90a9c36740 1 --2- 172.21.15.96:0/1974851933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583bd07b6b0 0x5583bd447ed0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.420+0000 7f907bfff640 0 client.15630 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:13:41.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.420+0000 7f90a9c36740 1 --2- 172.21.15.96:0/1974851933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583bd0734d0 0x5583bd447990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.420+0000 7f90a9c36740 1 -- 172.21.15.96:0/1974851933 >> 172.21.15.96:0/1974851933 conn(0x5583bd0904e0 msgr2=0x5583bd43c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:41.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.420+0000 7f90a9c36740 1 -- 172.21.15.96:0/1974851933 shutdown_connections 2024-04-25T14:13:41.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.420+0000 7f90a9c36740 1 -- 172.21.15.96:0/1974851933 wait complete. 2024-04-25T14:13:41.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.420+0000 7f90a9c36740 20 client.15630 trim_cache size 0 max 16384 2024-04-25T14:13:41.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.438+0000 7f90a9c36740 1 Processor -- start 2024-04-25T14:13:41.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.438+0000 7f90a9c36740 1 -- start start 2024-04-25T14:13:41.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.438+0000 7f90a9c36740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583bd0734d0 0x5583bd445f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:41.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.439+0000 7f90a9c36740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583bd07b6b0 0x5583bd4464b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:41.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.439+0000 7f90a9c36740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583bd09c390 0x5583bd438c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:41.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.439+0000 7f90a9c36740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f909c1ab360 con 0x5583bd07b6b0 2024-04-25T14:13:41.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.439+0000 7f90a9c36740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f909c1acea0 con 0x5583bd09c390 2024-04-25T14:13:41.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.439+0000 7f90a9c36740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f909c1aba30 con 0x5583bd0734d0 2024-04-25T14:13:41.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.439+0000 7f90a0b1e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583bd07b6b0 0x5583bd4464b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:41.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.439+0000 7f90a0b1e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583bd07b6b0 0x5583bd4464b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41198/0 (socket says 172.21.15.96:41198) 2024-04-25T14:13:41.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.439+0000 7f90a0b1e640 1 -- 172.21.15.96:0/1836384589 learned_addr learned my addr 172.21.15.96:0/1836384589 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:41.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.439+0000 7f909bfff640 1 --2- 172.21.15.96:0/1836384589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583bd0734d0 0x5583bd445f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:41.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.439+0000 7f90a0b1e640 1 -- 172.21.15.96:0/1836384589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583bd0734d0 msgr2=0x5583bd445f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:41.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.439+0000 7f90a0b1e640 1 --2- 172.21.15.96:0/1836384589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583bd0734d0 0x5583bd445f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.439+0000 7f90a0b1e640 1 -- 172.21.15.96:0/1836384589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583bd09c390 msgr2=0x5583bd438c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:41.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.439+0000 7f90a0b1e640 1 --2- 172.21.15.96:0/1836384589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583bd09c390 0x5583bd438c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.439+0000 7f90a0b1e640 1 -- 172.21.15.96:0/1836384589 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583bd0740f0 con 0x5583bd07b6b0 2024-04-25T14:13:41.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.439+0000 7f90a0b1e640 1 --2- 172.21.15.96:0/1836384589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583bd07b6b0 0x5583bd4464b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f909000b850 tx=0x7f909000bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fd67d84763c7eb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:41.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.439+0000 7f90867fc640 1 -- 172.21.15.96:0/1836384589 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9090025830 con 0x5583bd07b6b0 2024-04-25T14:13:41.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.439+0000 7f90867fc640 1 -- 172.21.15.96:0/1836384589 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9090025a00 con 0x5583bd07b6b0 2024-04-25T14:13:41.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.439+0000 7f90867fc640 1 -- 172.21.15.96:0/1836384589 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9090004440 con 0x5583bd07b6b0 2024-04-25T14:13:41.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.440+0000 7f90a9c36740 1 -- 172.21.15.96:0/1836384589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583bd07b6b0 msgr2=0x5583bd4464b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:41.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.440+0000 7f90a9c36740 1 --2- 172.21.15.96:0/1836384589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583bd07b6b0 0x5583bd4464b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f909000b850 tx=0x7f909000bab0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.440+0000 7f90a9c36740 1 -- 172.21.15.96:0/1836384589 shutdown_connections 2024-04-25T14:13:41.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.440+0000 7f90a9c36740 1 --2- 172.21.15.96:0/1836384589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583bd09c390 0x5583bd438c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.440+0000 7f90a9c36740 1 --2- 172.21.15.96:0/1836384589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583bd07b6b0 0x5583bd4464b0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.440+0000 7f90a9c36740 1 --2- 172.21.15.96:0/1836384589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583bd0734d0 0x5583bd445f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.440+0000 7f90a9c36740 1 -- 172.21.15.96:0/1836384589 >> 172.21.15.96:0/1836384589 conn(0x5583bd0904e0 msgr2=0x5583bd4426e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:41.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.440+0000 7f90a9c36740 1 -- 172.21.15.96:0/1836384589 shutdown_connections 2024-04-25T14:13:41.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.440+0000 7f90a9c36740 1 -- 172.21.15.96:0/1836384589 wait complete. 2024-04-25T14:13:41.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.440+0000 7f90a9c36740 1 -- start start 2024-04-25T14:13:41.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.440+0000 7f90a9c36740 1 Processor -- start 2024-04-25T14:13:41.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.441+0000 7f90a9c36740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583bd0734d0 0x5583bd44b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:41.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.441+0000 7f90a9c36740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583bd07b6b0 0x5583bd44b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:41.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.441+0000 7f90a9c36740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583bd09c390 0x5583bd4bb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:41.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.441+0000 7f90a9c36740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f909c1ac7d0 con 0x5583bd0734d0 2024-04-25T14:13:41.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.441+0000 7f90a9c36740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f909c1ac100 con 0x5583bd07b6b0 2024-04-25T14:13:41.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.441+0000 7f90a9c36740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f909c1a9340 con 0x5583bd09c390 2024-04-25T14:13:41.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.441+0000 7f90a9c36740 10 client.0 _ll_register_callbacks cb 0x7f909000b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:13:41.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.441+0000 7f909bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583bd0734d0 0x5583bd44b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:41.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.441+0000 7f909bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583bd0734d0 0x5583bd44b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41202/0 (socket says 172.21.15.96:41202) 2024-04-25T14:13:41.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.441+0000 7f909bfff640 1 -- 172.21.15.96:0/2826739682 learned_addr learned my addr 172.21.15.96:0/2826739682 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:41.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.441+0000 7f90a0b1e640 1 --2- 172.21.15.96:0/2826739682 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583bd07b6b0 0x5583bd44b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:41.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.441+0000 7f909bfff640 1 -- 172.21.15.96:0/2826739682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583bd09c390 msgr2=0x5583bd4bb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:41.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.441+0000 7f909bfff640 1 --2- 172.21.15.96:0/2826739682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583bd09c390 0x5583bd4bb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.441+0000 7f909bfff640 1 -- 172.21.15.96:0/2826739682 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583bd07b6b0 msgr2=0x5583bd44b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:41.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.441+0000 7f909bfff640 1 --2- 172.21.15.96:0/2826739682 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583bd07b6b0 0x5583bd44b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.441+0000 7f909bfff640 1 -- 172.21.15.96:0/2826739682 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583bd3634b0 con 0x5583bd0734d0 2024-04-25T14:13:41.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.441+0000 7f90a0b1e640 1 --2- 172.21.15.96:0/2826739682 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583bd07b6b0 0x5583bd44b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:41.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.441+0000 7f909bfff640 1 --2- 172.21.15.96:0/2826739682 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583bd0734d0 0x5583bd44b2e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9088013970 tx=0x7f908800bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb91e4a5e0497c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:41.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.441+0000 7f907bfff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:13:41.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.442+0000 7f907bfff640 1 -- 172.21.15.96:0/2826739682 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90880131f0 con 0x5583bd0734d0 2024-04-25T14:13:41.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.442+0000 7f907bfff640 1 -- 172.21.15.96:0/2826739682 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f908800e960 con 0x5583bd0734d0 2024-04-25T14:13:41.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.442+0000 7f90a9c36740 1 -- 172.21.15.96:0/2826739682 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9090025a00 con 0x5583bd0734d0 2024-04-25T14:13:41.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.442+0000 7f907bfff640 1 -- 172.21.15.96:0/2826739682 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f908800eb60 con 0x5583bd0734d0 2024-04-25T14:13:41.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.442+0000 7f90a9c36740 10 client.15636 fetch_fsmap learned FSMap version 26 2024-04-25T14:13:41.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.442+0000 7f90a9c36740 1 -- 172.21.15.96:0/2826739682 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5583bd4bcb70 con 0x5583bd0734d0 2024-04-25T14:13:41.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.442+0000 7f907bfff640 1 -- 172.21.15.96:0/2826739682 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9090025a00 con 0x5583bd0734d0 2024-04-25T14:13:41.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.442+0000 7f907bfff640 1 -- 172.21.15.96:0/2826739682 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9088004660 con 0x5583bd0734d0 2024-04-25T14:13:41.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.442+0000 7f90a9c36740 10 client.15636 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:13:41.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.442+0000 7f90a9c36740 10 client.15636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:13:41.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.442+0000 7f90a9c36740 20 client.15636 populate_metadata read hostname 'smithi096' 2024-04-25T14:13:41.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.442+0000 7f90a9c36740 1 --2- 172.21.15.96:0/2826739682 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5583bd0303a0 0x5583bd4bf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:41.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.442+0000 7f90a9c36740 4 client.15636 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:13:41.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.442+0000 7f90a9c36740 1 -- 172.21.15.96:0/2826739682 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5583bd3aa800 con 0x5583bd0303a0 2024-04-25T14:13:41.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.442+0000 7f90a0b1e640 1 --2- 172.21.15.96:0/2826739682 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5583bd0303a0 0x5583bd4bf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:41.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.443+0000 7f907bfff640 10 client.15636 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:13:41.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.443+0000 7f90a0b1e640 1 --2- 172.21.15.96:0/2826739682 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5583bd0303a0 0x5583bd4bf450 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2748ff73b610ee1 server_cookie=34feea1fc37ec6e9 in_seq=0 out_seq=0 2024-04-25T14:13:41.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.443+0000 7f907bfff640 1 -- 172.21.15.96:0/2826739682 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5583bd3aa800 con 0x5583bd0303a0 2024-04-25T14:13:41.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.443+0000 7f907bfff640 10 client.15636 handle_command_reply: tid=0 2024-04-25T14:13:41.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.443+0000 7f90a9c36740 1 client.15636 shutdown 2024-04-25T14:13:41.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.443+0000 7f90a9c36740 1 -- 172.21.15.96:0/2826739682 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583bd0734d0 msgr2=0x5583bd44b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:41.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.443+0000 7f90a9c36740 1 --2- 172.21.15.96:0/2826739682 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583bd0734d0 0x5583bd44b2e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9088013970 tx=0x7f908800bde0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.443+0000 7f90a9c36740 1 -- 172.21.15.96:0/2826739682 shutdown_connections 2024-04-25T14:13:41.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.443+0000 7f90a9c36740 1 --2- 172.21.15.96:0/2826739682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583bd09c390 0x5583bd4bb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.443+0000 7f90a9c36740 1 --2- 172.21.15.96:0/2826739682 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583bd07b6b0 0x5583bd44b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.443+0000 7f90a9c36740 1 --2- 172.21.15.96:0/2826739682 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5583bd0303a0 0x5583bd4bf450 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.443+0000 7f90a9c36740 1 --2- 172.21.15.96:0/2826739682 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583bd0734d0 0x5583bd44b2e0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.443+0000 7f907bfff640 0 client.15636 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:13:41.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.443+0000 7f90a9c36740 1 -- 172.21.15.96:0/2826739682 >> 172.21.15.96:0/2826739682 conn(0x5583bd0904e0 msgr2=0x5583bd445f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:41.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.444+0000 7f90a9c36740 1 -- 172.21.15.96:0/2826739682 shutdown_connections 2024-04-25T14:13:41.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.444+0000 7f90a9c36740 1 -- 172.21.15.96:0/2826739682 wait complete. 2024-04-25T14:13:41.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.444+0000 7f90a9c36740 20 client.15636 trim_cache size 0 max 16384 2024-04-25T14:13:41.451 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T14:13:41.451 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T14:13:41.451 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": { 2024-04-25T14:13:41.451 INFO:teuthology.orchestra.run.smithi096.stdout: "c6fd2c96-201c-45ea-8c66-6409bd03e99c": { 2024-04-25T14:13:41.451 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-04-25T14:13:41.451 INFO:teuthology.orchestra.run.smithi096.stdout: "tag": "c6fd2c96-201c-45ea-8c66-6409bd03e99c", 2024-04-25T14:13:41.451 INFO:teuthology.orchestra.run.smithi096.stdout: "options": "recursive,force" 2024-04-25T14:13:41.451 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:13:41.452 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:13:41.452 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T14:13:41.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.444+0000 7f90a2d82640 1 -- 172.21.15.96:0/2990568314 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f908c0afaf0 msgr2=0x7f908c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:41.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.444+0000 7f90a2d82640 1 --2- 172.21.15.96:0/2990568314 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f908c0afaf0 0x7f908c0b2000 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f90940c8620 tx=0x7f90940b1040 comp rx=0 tx=0).stop 2024-04-25T14:13:41.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.444+0000 7f90a2d82640 1 -- 172.21.15.96:0/2990568314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f909c108920 msgr2=0x7f909c1a19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:41.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.444+0000 7f90a2d82640 1 --2- 172.21.15.96:0/2990568314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f909c108920 0x7f909c1a19b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f90940097b0 tx=0x7f9094003c20 comp rx=0 tx=0).stop 2024-04-25T14:13:41.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.444+0000 7f90a2d82640 1 -- 172.21.15.96:0/2990568314 shutdown_connections 2024-04-25T14:13:41.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.444+0000 7f90a2d82640 1 --2- 172.21.15.96:0/2990568314 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f908c0afaf0 0x7f908c0b2000 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.444+0000 7f90a2d82640 1 --2- 172.21.15.96:0/2990568314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f909c108920 0x7f909c1a19b0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.444+0000 7f90a2d82640 1 --2- 172.21.15.96:0/2990568314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f909c103c20 0x7f909c19aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.444+0000 7f90a2d82640 1 --2- 172.21.15.96:0/2990568314 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f909c1029e0 0x7f909c19a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:41.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.444+0000 7f90a2d82640 1 -- 172.21.15.96:0/2990568314 >> 172.21.15.96:0/2990568314 conn(0x7f909c0fdbd0 msgr2=0x7f909c109d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:41.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.444+0000 7f90a2d82640 1 -- 172.21.15.96:0/2990568314 shutdown_connections 2024-04-25T14:13:41.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:13:41.444+0000 7f90a2d82640 1 -- 172.21.15.96:0/2990568314 wait complete. 2024-04-25T14:13:41.454 INFO:tasks.cephfs.filesystem:scrub status for tag:c6fd2c96-201c-45ea-8c66-6409bd03e99c - {'path': '/', 'tag': 'c6fd2c96-201c-45ea-8c66-6409bd03e99c', 'options': 'recursive,force'} 2024-04-25T14:13:42.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:42 smithi096 ceph-mon[29207]: scrub queued for path: / 2024-04-25T14:13:42.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:42 smithi096 ceph-mon[29207]: scrub summary: idle+waiting paths [/] 2024-04-25T14:13:42.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:42 smithi096 ceph-mon[29207]: scrub summary: active paths [/] 2024-04-25T14:13:42.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:42 smithi096 ceph-mon[29207]: scrub summary: idle+waiting paths [/] 2024-04-25T14:13:42.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:42 smithi096 ceph-mon[29207]: pgmap v348: 97 pgs: 97 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-04-25T14:13:42.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:42 smithi097 ceph-mon[32720]: scrub queued for path: / 2024-04-25T14:13:42.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:42 smithi097 ceph-mon[32720]: scrub summary: idle+waiting paths [/] 2024-04-25T14:13:42.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:42 smithi097 ceph-mon[32720]: scrub summary: active paths [/] 2024-04-25T14:13:42.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:42 smithi097 ceph-mon[32720]: scrub summary: idle+waiting paths [/] 2024-04-25T14:13:42.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:42 smithi097 ceph-mon[32720]: pgmap v348: 97 pgs: 97 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-04-25T14:13:42.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:42 smithi151 ceph-mon[32990]: scrub queued for path: / 2024-04-25T14:13:42.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:42 smithi151 ceph-mon[32990]: scrub summary: idle+waiting paths [/] 2024-04-25T14:13:42.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:42 smithi151 ceph-mon[32990]: scrub summary: active paths [/] 2024-04-25T14:13:42.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:42 smithi151 ceph-mon[32990]: scrub summary: idle+waiting paths [/] 2024-04-25T14:13:42.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:42 smithi151 ceph-mon[32990]: pgmap v348: 97 pgs: 97 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-04-25T14:13:44.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:44 smithi096 ceph-mon[29207]: pgmap v349: 97 pgs: 97 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-04-25T14:13:44.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:44 smithi096 ceph-mon[29207]: scrub summary: idle 2024-04-25T14:13:44.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:44 smithi097 ceph-mon[32720]: pgmap v349: 97 pgs: 97 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-04-25T14:13:44.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:44 smithi097 ceph-mon[32720]: scrub summary: idle 2024-04-25T14:13:44.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:44 smithi151 ceph-mon[32990]: pgmap v349: 97 pgs: 97 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-04-25T14:13:44.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:44 smithi151 ceph-mon[32990]: scrub summary: idle 2024-04-25T14:13:46.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:46 smithi096 ceph-mon[29207]: pgmap v350: 97 pgs: 97 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-04-25T14:13:46.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:46 smithi097 ceph-mon[32720]: pgmap v350: 97 pgs: 97 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-04-25T14:13:46.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:46 smithi151 ceph-mon[32990]: pgmap v350: 97 pgs: 97 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-04-25T14:13:48.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:48 smithi096 ceph-mon[29207]: pgmap v351: 97 pgs: 97 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-04-25T14:13:48.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:48 smithi097 ceph-mon[32720]: pgmap v351: 97 pgs: 97 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-04-25T14:13:48.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:48 smithi151 ceph-mon[32990]: pgmap v351: 97 pgs: 97 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-04-25T14:13:49.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:49 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:13:49.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:49 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:13:49.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:49 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:13:50.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:50 smithi096 ceph-mon[29207]: pgmap v352: 97 pgs: 97 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 9 op/s 2024-04-25T14:13:50.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:50 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:13:50.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:50 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:13:50.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:50 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:13:50.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:50 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:13:50.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:50 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:13:50.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:50 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:13:50.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:50 smithi097 ceph-mon[32720]: pgmap v352: 97 pgs: 97 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 9 op/s 2024-04-25T14:13:50.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:50 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:13:50.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:50 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:13:50.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:50 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:13:50.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:50 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:13:50.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:50 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:13:50.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:50 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:13:50.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:50 smithi151 ceph-mon[32990]: pgmap v352: 97 pgs: 97 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 9 op/s 2024-04-25T14:13:50.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:50 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:13:50.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:50 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:13:50.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:50 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:13:50.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:50 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:13:50.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:50 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:13:50.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:50 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:13:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.747+0000 7fa480bf8640 1 -- 172.21.15.96:0/3736110174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa47c102a20 msgr2=0x7fa47c103720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.747+0000 7fa480bf8640 1 --2- 172.21.15.96:0/3736110174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa47c102a20 0x7fa47c103720 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa464009900 tx=0x7fa46402a410 comp rx=0 tx=0).stop 2024-04-25T14:13:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.747+0000 7fa480bf8640 1 -- 172.21.15.96:0/3736110174 shutdown_connections 2024-04-25T14:13:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.747+0000 7fa480bf8640 1 --2- 172.21.15.96:0/3736110174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa47c108960 0x7fa47c10ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.747+0000 7fa480bf8640 1 --2- 172.21.15.96:0/3736110174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa47c103c60 0x7fa47c104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.747+0000 7fa480bf8640 1 --2- 172.21.15.96:0/3736110174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa47c102a20 0x7fa47c103720 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.747+0000 7fa480bf8640 1 -- 172.21.15.96:0/3736110174 >> 172.21.15.96:0/3736110174 conn(0x7fa47c0fdc10 msgr2=0x7fa47c1000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.747+0000 7fa480bf8640 1 -- 172.21.15.96:0/3736110174 shutdown_connections 2024-04-25T14:13:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.747+0000 7fa480bf8640 1 -- 172.21.15.96:0/3736110174 wait complete. 2024-04-25T14:13:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.748+0000 7fa480bf8640 1 Processor -- start 2024-04-25T14:13:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.748+0000 7fa480bf8640 1 -- start start 2024-04-25T14:13:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.748+0000 7fa480bf8640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa47c102a20 0x7fa47c19a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.748+0000 7fa480bf8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa47c103c60 0x7fa47c19a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.748+0000 7fa480bf8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa47c108960 0x7fa47c1a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.748+0000 7fa480bf8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa47c10e520 con 0x7fa47c108960 2024-04-25T14:13:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.748+0000 7fa480bf8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa47c10e3a0 con 0x7fa47c103c60 2024-04-25T14:13:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.748+0000 7fa480bf8640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa47c10e6a0 con 0x7fa47c102a20 2024-04-25T14:13:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.748+0000 7fa47ad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa47c108960 0x7fa47c1a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.748+0000 7fa47ad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa47c108960 0x7fa47c1a1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35660/0 (socket says 172.21.15.96:35660) 2024-04-25T14:13:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.748+0000 7fa47ad9d640 1 -- 172.21.15.96:0/3204420130 learned_addr learned my addr 172.21.15.96:0/3204420130 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.748+0000 7fa479d9b640 1 --2- 172.21.15.96:0/3204420130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa47c103c60 0x7fa47c19a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.748+0000 7fa47a59c640 1 --2- 172.21.15.96:0/3204420130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa47c102a20 0x7fa47c19a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.749+0000 7fa47ad9d640 1 -- 172.21.15.96:0/3204420130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa47c102a20 msgr2=0x7fa47c19a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.749+0000 7fa47ad9d640 1 --2- 172.21.15.96:0/3204420130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa47c102a20 0x7fa47c19a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.749+0000 7fa47ad9d640 1 -- 172.21.15.96:0/3204420130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa47c103c60 msgr2=0x7fa47c19a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.749+0000 7fa47ad9d640 1 --2- 172.21.15.96:0/3204420130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa47c103c60 0x7fa47c19a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.749+0000 7fa47ad9d640 1 -- 172.21.15.96:0/3204420130 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa47c1a1dc0 con 0x7fa47c108960 2024-04-25T14:13:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.749+0000 7fa47ad9d640 1 --2- 172.21.15.96:0/3204420130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa47c108960 0x7fa47c1a1750 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa47400eaa0 tx=0x7fa47400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.749+0000 7fa4737fe640 1 -- 172.21.15.96:0/3204420130 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47400cdb0 con 0x7fa47c108960 2024-04-25T14:13:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.749+0000 7fa4737fe640 1 -- 172.21.15.96:0/3204420130 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa474005020 con 0x7fa47c108960 2024-04-25T14:13:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.749+0000 7fa480bf8640 1 -- 172.21.15.96:0/3204420130 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa47c1a20b0 con 0x7fa47c108960 2024-04-25T14:13:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.749+0000 7fa4737fe640 1 -- 172.21.15.96:0/3204420130 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4740116d0 con 0x7fa47c108960 2024-04-25T14:13:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.749+0000 7fa480bf8640 1 -- 172.21.15.96:0/3204420130 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa47c1a25b0 con 0x7fa47c108960 2024-04-25T14:13:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.751+0000 7fa4737fe640 1 -- 172.21.15.96:0/3204420130 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fa4740051d0 con 0x7fa47c108960 2024-04-25T14:13:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.752+0000 7fa487b89740 1 Processor -- start 2024-04-25T14:13:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.752+0000 7fa487b89740 1 -- start start 2024-04-25T14:13:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.752+0000 7fa487b89740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f33d3851b0 0x55f33d725970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.752+0000 7fa487b89740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f33d35c2f0 0x55f33d725ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.752+0000 7fa487b89740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f33d3644d0 0x55f33d72cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.752+0000 7fa487b89740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f33d68b460 con 0x55f33d3644d0 2024-04-25T14:13:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.752+0000 7fa487b89740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f33d645590 con 0x55f33d35c2f0 2024-04-25T14:13:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.752+0000 7fa487b89740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f33d687420 con 0x55f33d3851b0 2024-04-25T14:13:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.752+0000 7fa479d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f33d3644d0 0x55f33d72cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.752+0000 7fa4737fe640 1 --2- 172.21.15.96:0/3204420130 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa4680afae0 0x7fa4680b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.752+0000 7fa479d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f33d3644d0 0x55f33d72cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35666/0 (socket says 172.21.15.96:35666) 2024-04-25T14:13:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.752+0000 7fa479d9b640 1 -- 172.21.15.96:0/1758534680 learned_addr learned my addr 172.21.15.96:0/1758534680 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.752+0000 7fa47a59c640 1 --2- 172.21.15.96:0/1758534680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f33d35c2f0 0x55f33d725ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.752+0000 7fa47ad9d640 1 --2- 172.21.15.96:0/3204420130 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa4680afae0 0x7fa4680b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.753+0000 7fa479d9b640 1 --2- 172.21.15.96:0/1758534680 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f33d3851b0 0x55f33d725970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.753+0000 7fa479d9b640 1 -- 172.21.15.96:0/1758534680 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f33d3851b0 msgr2=0x55f33d725970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.753+0000 7fa479d9b640 1 --2- 172.21.15.96:0/1758534680 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f33d3851b0 0x55f33d725970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.753+0000 7fa479d9b640 1 -- 172.21.15.96:0/1758534680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f33d35c2f0 msgr2=0x55f33d725ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.753+0000 7fa479d9b640 1 --2- 172.21.15.96:0/1758534680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f33d35c2f0 0x55f33d725ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.753+0000 7fa4737fe640 1 -- 172.21.15.96:0/3204420130 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fa4740dedb0 con 0x7fa47c108960 2024-04-25T14:13:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.753+0000 7fa479d9b640 1 -- 172.21.15.96:0/1758534680 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f33d6520d0 con 0x55f33d3644d0 2024-04-25T14:13:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.753+0000 7fa47a59c640 1 --2- 172.21.15.96:0/1758534680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f33d35c2f0 0x55f33d725ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:13:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.753+0000 7fa479d9b640 1 --2- 172.21.15.96:0/1758534680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f33d3644d0 0x55f33d72cdf0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa46c005880 tx=0x7fa46c004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=119e464495b5b106 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.753+0000 7fa47ad9d640 1 --2- 172.21.15.96:0/3204420130 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa4680afae0 0x7fa4680b1ff0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa4740dcc00 tx=0x7fa474074040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.753+0000 7fa470ff9640 1 -- 172.21.15.96:0/1758534680 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c005020 con 0x55f33d3644d0 2024-04-25T14:13:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.753+0000 7fa470ff9640 1 -- 172.21.15.96:0/1758534680 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa46c0047a0 con 0x55f33d3644d0 2024-04-25T14:13:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.753+0000 7fa470ff9640 1 -- 172.21.15.96:0/1758534680 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c0106c0 con 0x55f33d3644d0 2024-04-25T14:13:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.754+0000 7fa487b89740 1 -- 172.21.15.96:0/1758534680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f33d3644d0 msgr2=0x55f33d72cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.754+0000 7fa487b89740 1 --2- 172.21.15.96:0/1758534680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f33d3644d0 0x55f33d72cdf0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa46c005880 tx=0x7fa46c004610 comp rx=0 tx=0).stop 2024-04-25T14:13:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.754+0000 7fa487b89740 1 -- 172.21.15.96:0/1758534680 shutdown_connections 2024-04-25T14:13:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.754+0000 7fa487b89740 1 --2- 172.21.15.96:0/1758534680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f33d3644d0 0x55f33d72cdf0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.754+0000 7fa487b89740 1 --2- 172.21.15.96:0/1758534680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f33d35c2f0 0x55f33d725ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.754+0000 7fa487b89740 1 --2- 172.21.15.96:0/1758534680 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f33d3851b0 0x55f33d725970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.754+0000 7fa487b89740 1 -- 172.21.15.96:0/1758534680 >> 172.21.15.96:0/1758534680 conn(0x55f33d379300 msgr2=0x55f33d721300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.754+0000 7fa487b89740 1 -- 172.21.15.96:0/1758534680 shutdown_connections 2024-04-25T14:13:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.754+0000 7fa487b89740 1 -- 172.21.15.96:0/1758534680 wait complete. 2024-04-25T14:13:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.754+0000 7fa487b89740 1 -- start start 2024-04-25T14:13:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.755+0000 7fa487b89740 1 Processor -- start 2024-04-25T14:13:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.755+0000 7fa487b89740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f33d35c2f0 0x55f33d72f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.755+0000 7fa487b89740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f33d3644d0 0x55f33d72fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.755+0000 7fa487b89740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f33d3851b0 0x55f33d736ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.755+0000 7fa487b89740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f33d72d840 con 0x55f33d3644d0 2024-04-25T14:13:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.755+0000 7fa487b89740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f33d612b60 con 0x55f33d3851b0 2024-04-25T14:13:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.755+0000 7fa487b89740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f33d72d9c0 con 0x55f33d35c2f0 2024-04-25T14:13:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.755+0000 7fa47a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f33d3644d0 0x55f33d72fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.755+0000 7fa487b89740 10 client.0 _ll_register_callbacks cb 0x55f33d6372b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:13:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.755+0000 7fa47a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f33d3644d0 0x55f33d72fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35668/0 (socket says 172.21.15.96:35668) 2024-04-25T14:13:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.755+0000 7fa47a59c640 1 -- 172.21.15.96:0/1968139639 learned_addr learned my addr 172.21.15.96:0/1968139639 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.755+0000 7fa479d9b640 1 --2- 172.21.15.96:0/1968139639 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f33d35c2f0 0x55f33d72f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.755+0000 7fa479d9b640 1 --2- 172.21.15.96:0/1968139639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f33d3851b0 0x55f33d736ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.755+0000 7fa47a59c640 1 -- 172.21.15.96:0/1968139639 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f33d35c2f0 msgr2=0x55f33d72f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.755+0000 7fa47a59c640 1 --2- 172.21.15.96:0/1968139639 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f33d35c2f0 0x55f33d72f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.755+0000 7fa47a59c640 1 -- 172.21.15.96:0/1968139639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f33d3851b0 msgr2=0x55f33d736ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.755+0000 7fa47a59c640 1 --2- 172.21.15.96:0/1968139639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f33d3851b0 0x55f33d736ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.755+0000 7fa47a59c640 1 -- 172.21.15.96:0/1968139639 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f33d39d210 con 0x55f33d3644d0 2024-04-25T14:13:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.755+0000 7fa479d9b640 1 --2- 172.21.15.96:0/1968139639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f33d3851b0 0x55f33d736ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.755+0000 7fa47a59c640 1 --2- 172.21.15.96:0/1968139639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f33d3644d0 0x55f33d72fb50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa464009900 tx=0x7fa464031e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f78b36fdcd0fa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.755+0000 7fa461ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:13:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.756+0000 7fa461ffb640 1 -- 172.21.15.96:0/1968139639 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa464035040 con 0x55f33d3644d0 2024-04-25T14:13:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.756+0000 7fa487b89740 1 -- 172.21.15.96:0/1968139639 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa46c0047a0 con 0x55f33d3644d0 2024-04-25T14:13:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.756+0000 7fa461ffb640 1 -- 172.21.15.96:0/1968139639 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fa464004040 con 0x55f33d3644d0 2024-04-25T14:13:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.756+0000 7fa461ffb640 1 -- 172.21.15.96:0/1968139639 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa46404c070 con 0x55f33d3644d0 2024-04-25T14:13:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.756+0000 7fa487b89740 10 client.15648 fetch_fsmap learned FSMap version 26 2024-04-25T14:13:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.756+0000 7fa487b89740 1 -- 172.21.15.96:0/1968139639 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f33d469620 con 0x55f33d3644d0 2024-04-25T14:13:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.756+0000 7fa461ffb640 1 -- 172.21.15.96:0/1968139639 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa46c0047a0 con 0x55f33d3644d0 2024-04-25T14:13:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.756+0000 7fa461ffb640 1 -- 172.21.15.96:0/1968139639 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa464044020 con 0x55f33d3644d0 2024-04-25T14:13:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.756+0000 7fa487b89740 10 client.15648 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:13:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.756+0000 7fa487b89740 10 client.15648 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:13:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.756+0000 7fa487b89740 20 client.15648 populate_metadata read hostname 'smithi096' 2024-04-25T14:13:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.756+0000 7fa487b89740 1 --2- 172.21.15.96:0/1968139639 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55f33d343d80 0x55f33d73d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.756+0000 7fa487b89740 4 client.15648 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:13:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.756+0000 7fa487b89740 1 -- 172.21.15.96:0/1968139639 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f33d699070 con 0x55f33d343d80 2024-04-25T14:13:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.756+0000 7fa479d9b640 1 --2- 172.21.15.96:0/1968139639 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55f33d343d80 0x55f33d73d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.757+0000 7fa479d9b640 1 --2- 172.21.15.96:0/1968139639 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55f33d343d80 0x55f33d73d2e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0f738c949889b86 server_cookie=135d7db2f0594442 in_seq=0 out_seq=0 2024-04-25T14:13:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.757+0000 7fa461ffb640 10 client.15648 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:13:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.758+0000 7fa461ffb640 1 -- 172.21.15.96:0/1968139639 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f33d699070 con 0x55f33d343d80 2024-04-25T14:13:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.758+0000 7fa461ffb640 10 client.15648 handle_command_reply: tid=0 2024-04-25T14:13:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.758+0000 7fa487b89740 1 client.15648 shutdown 2024-04-25T14:13:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.758+0000 7fa487b89740 1 -- 172.21.15.96:0/1968139639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f33d3644d0 msgr2=0x55f33d72fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.758+0000 7fa487b89740 1 --2- 172.21.15.96:0/1968139639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f33d3644d0 0x55f33d72fb50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa464009900 tx=0x7fa464031e60 comp rx=0 tx=0).stop 2024-04-25T14:13:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.758+0000 7fa487b89740 1 -- 172.21.15.96:0/1968139639 shutdown_connections 2024-04-25T14:13:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.758+0000 7fa487b89740 1 --2- 172.21.15.96:0/1968139639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f33d3851b0 0x55f33d736ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.758+0000 7fa487b89740 1 --2- 172.21.15.96:0/1968139639 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55f33d343d80 0x55f33d73d2e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.758+0000 7fa487b89740 1 --2- 172.21.15.96:0/1968139639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f33d3644d0 0x55f33d72fb50 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.758+0000 7fa487b89740 1 --2- 172.21.15.96:0/1968139639 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f33d35c2f0 0x55f33d72f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.758+0000 7fa461ffb640 0 client.15648 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:13:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.758+0000 7fa487b89740 1 -- 172.21.15.96:0/1968139639 >> 172.21.15.96:0/1968139639 conn(0x55f33d379300 msgr2=0x55f33d724210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.758+0000 7fa487b89740 1 -- 172.21.15.96:0/1968139639 shutdown_connections 2024-04-25T14:13:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.758+0000 7fa487b89740 1 -- 172.21.15.96:0/1968139639 wait complete. 2024-04-25T14:13:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.758+0000 7fa487b89740 20 client.15648 trim_cache size 0 max 16384 2024-04-25T14:13:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.773+0000 7fa487b89740 1 Processor -- start 2024-04-25T14:13:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.773+0000 7fa487b89740 1 -- start start 2024-04-25T14:13:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.773+0000 7fa487b89740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f33d35c2f0 0x55f33d7213d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.773+0000 7fa487b89740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f33d3644d0 0x55f33d7369d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.773+0000 7fa487b89740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f33d3851b0 0x55f33d724100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.773+0000 7fa487b89740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa47c1ab320 con 0x55f33d3644d0 2024-04-25T14:13:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.773+0000 7fa487b89740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa47c1ace60 con 0x55f33d35c2f0 2024-04-25T14:13:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.773+0000 7fa487b89740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa47c1ab9f0 con 0x55f33d3851b0 2024-04-25T14:13:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.773+0000 7fa47a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f33d3644d0 0x55f33d7369d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.773+0000 7fa47a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f33d3644d0 0x55f33d7369d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35676/0 (socket says 172.21.15.96:35676) 2024-04-25T14:13:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.773+0000 7fa47a59c640 1 -- 172.21.15.96:0/1015624206 learned_addr learned my addr 172.21.15.96:0/1015624206 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.773+0000 7fa479d9b640 1 --2- 172.21.15.96:0/1015624206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f33d35c2f0 0x55f33d7213d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.773+0000 7fa47a59c640 1 -- 172.21.15.96:0/1015624206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f33d3851b0 msgr2=0x55f33d724100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.773+0000 7fa47a59c640 1 --2- 172.21.15.96:0/1015624206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f33d3851b0 0x55f33d724100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.773+0000 7fa47a59c640 1 -- 172.21.15.96:0/1015624206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f33d35c2f0 msgr2=0x55f33d7213d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.773+0000 7fa47a59c640 1 --2- 172.21.15.96:0/1015624206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f33d35c2f0 0x55f33d7213d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.773+0000 7fa47a59c640 1 -- 172.21.15.96:0/1015624206 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f33d736f10 con 0x55f33d3644d0 2024-04-25T14:13:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.773+0000 7fa479d9b640 1 --2- 172.21.15.96:0/1015624206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f33d35c2f0 0x55f33d7213d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.774+0000 7fa47a59c640 1 --2- 172.21.15.96:0/1015624206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f33d3644d0 0x55f33d7369d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa464009850 tx=0x7fa46402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c0216a38a4ec64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.774+0000 7fa463fff640 1 -- 172.21.15.96:0/1015624206 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa464056070 con 0x55f33d3644d0 2024-04-25T14:13:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.774+0000 7fa463fff640 1 -- 172.21.15.96:0/1015624206 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa464005850 con 0x55f33d3644d0 2024-04-25T14:13:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.774+0000 7fa463fff640 1 -- 172.21.15.96:0/1015624206 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa464035070 con 0x55f33d3644d0 2024-04-25T14:13:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.774+0000 7fa487b89740 1 -- 172.21.15.96:0/1015624206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f33d3644d0 msgr2=0x55f33d7369d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.774+0000 7fa487b89740 1 --2- 172.21.15.96:0/1015624206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f33d3644d0 0x55f33d7369d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa464009850 tx=0x7fa46402a410 comp rx=0 tx=0).stop 2024-04-25T14:13:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.774+0000 7fa487b89740 1 -- 172.21.15.96:0/1015624206 shutdown_connections 2024-04-25T14:13:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.774+0000 7fa487b89740 1 --2- 172.21.15.96:0/1015624206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f33d3851b0 0x55f33d724100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.774+0000 7fa487b89740 1 --2- 172.21.15.96:0/1015624206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f33d3644d0 0x55f33d7369d0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:51.782 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.774+0000 7fa487b89740 1 --2- 172.21.15.96:0/1015624206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f33d35c2f0 0x55f33d7213d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:51.782 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.775+0000 7fa487b89740 1 -- 172.21.15.96:0/1015624206 >> 172.21.15.96:0/1015624206 conn(0x55f33d379300 msgr2=0x55f33d720cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:51.782 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.775+0000 7fa487b89740 1 -- 172.21.15.96:0/1015624206 shutdown_connections 2024-04-25T14:13:51.782 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.775+0000 7fa487b89740 1 -- 172.21.15.96:0/1015624206 wait complete. 2024-04-25T14:13:51.782 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.775+0000 7fa487b89740 1 -- start start 2024-04-25T14:13:51.782 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.775+0000 7fa487b89740 1 Processor -- start 2024-04-25T14:13:51.782 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.775+0000 7fa487b89740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f33d35c2f0 0x55f33d734150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:51.782 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.775+0000 7fa487b89740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f33d3644d0 0x55f33d734690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:51.782 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.775+0000 7fa487b89740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f33d3851b0 0x55f33d735500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:51.782 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.775+0000 7fa487b89740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa47c1ac790 con 0x55f33d35c2f0 2024-04-25T14:13:51.782 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.775+0000 7fa487b89740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa47c1ac0c0 con 0x55f33d3644d0 2024-04-25T14:13:51.782 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.775+0000 7fa487b89740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa47c1a9300 con 0x55f33d3851b0 2024-04-25T14:13:51.782 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.775+0000 7fa487b89740 10 client.0 _ll_register_callbacks cb 0x7fa464005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:13:51.782 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.775+0000 7fa479d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f33d35c2f0 0x55f33d734150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.775+0000 7fa479d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f33d35c2f0 0x55f33d734150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35692/0 (socket says 172.21.15.96:35692) 2024-04-25T14:13:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.775+0000 7fa479d9b640 1 -- 172.21.15.96:0/1868467264 learned_addr learned my addr 172.21.15.96:0/1868467264 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:13:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.775+0000 7fa47a59c640 1 --2- 172.21.15.96:0/1868467264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f33d3644d0 0x55f33d734690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.775+0000 7fa479d9b640 1 --2- 172.21.15.96:0/1868467264 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f33d3851b0 0x55f33d735500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.776+0000 7fa479d9b640 1 -- 172.21.15.96:0/1868467264 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f33d3851b0 msgr2=0x55f33d735500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.776+0000 7fa479d9b640 1 --2- 172.21.15.96:0/1868467264 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f33d3851b0 0x55f33d735500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.776+0000 7fa479d9b640 1 -- 172.21.15.96:0/1868467264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f33d3644d0 msgr2=0x55f33d734690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.776+0000 7fa479d9b640 1 --2- 172.21.15.96:0/1868467264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f33d3644d0 0x55f33d734690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.776+0000 7fa479d9b640 1 -- 172.21.15.96:0/1868467264 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f33d735ac0 con 0x55f33d35c2f0 2024-04-25T14:13:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.776+0000 7fa479d9b640 1 --2- 172.21.15.96:0/1868467264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f33d35c2f0 0x55f33d734150 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa46c00bfd0 tx=0x7fa46c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed8efde3d744d5b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.776+0000 7fa461ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:13:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.776+0000 7fa461ffb640 1 -- 172.21.15.96:0/1868467264 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c004780 con 0x55f33d35c2f0 2024-04-25T14:13:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.776+0000 7fa461ffb640 1 -- 172.21.15.96:0/1868467264 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fa46c004040 con 0x55f33d35c2f0 2024-04-25T14:13:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.776+0000 7fa487b89740 1 -- 172.21.15.96:0/1868467264 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa464005850 con 0x55f33d35c2f0 2024-04-25T14:13:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.776+0000 7fa461ffb640 1 -- 172.21.15.96:0/1868467264 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa46c0049d0 con 0x55f33d35c2f0 2024-04-25T14:13:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.776+0000 7fa487b89740 10 client.15654 fetch_fsmap learned FSMap version 26 2024-04-25T14:13:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.776+0000 7fa487b89740 1 -- 172.21.15.96:0/1868467264 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f33d736cf0 con 0x55f33d35c2f0 2024-04-25T14:13:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.776+0000 7fa461ffb640 1 -- 172.21.15.96:0/1868467264 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa464005850 con 0x55f33d35c2f0 2024-04-25T14:13:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.776+0000 7fa461ffb640 1 -- 172.21.15.96:0/1868467264 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa46c02e050 con 0x55f33d35c2f0 2024-04-25T14:13:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.776+0000 7fa487b89740 10 client.15654 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:13:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.776+0000 7fa487b89740 10 client.15654 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:13:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.776+0000 7fa487b89740 20 client.15654 populate_metadata read hostname 'smithi096' 2024-04-25T14:13:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.776+0000 7fa487b89740 1 --2- 172.21.15.96:0/1868467264 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55f33d3191c0 0x55f33d7a7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.776+0000 7fa487b89740 4 client.15654 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:13:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.776+0000 7fa487b89740 1 -- 172.21.15.96:0/1868467264 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f33d692ee0 con 0x55f33d3191c0 2024-04-25T14:13:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.777+0000 7fa47a59c640 1 --2- 172.21.15.96:0/1868467264 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55f33d3191c0 0x55f33d7a7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.777+0000 7fa461ffb640 10 client.15654 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:13:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.777+0000 7fa47a59c640 1 --2- 172.21.15.96:0/1868467264 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55f33d3191c0 0x55f33d7a7cf0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18f6ddd58ae1c22 server_cookie=b201d5f66507c089 in_seq=0 out_seq=0 2024-04-25T14:13:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.789+0000 7fa461ffb640 1 -- 172.21.15.96:0/1868467264 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f33d692ee0 con 0x55f33d3191c0 2024-04-25T14:13:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.789+0000 7fa461ffb640 10 client.15654 handle_command_reply: tid=0 2024-04-25T14:13:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.789+0000 7fa487b89740 1 client.15654 shutdown 2024-04-25T14:13:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.789+0000 7fa487b89740 1 -- 172.21.15.96:0/1868467264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f33d35c2f0 msgr2=0x55f33d734150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.789+0000 7fa487b89740 1 --2- 172.21.15.96:0/1868467264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f33d35c2f0 0x55f33d734150 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa46c00bfd0 tx=0x7fa46c005fd0 comp rx=0 tx=0).stop 2024-04-25T14:13:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.789+0000 7fa487b89740 1 -- 172.21.15.96:0/1868467264 shutdown_connections 2024-04-25T14:13:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.789+0000 7fa487b89740 1 --2- 172.21.15.96:0/1868467264 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f33d3851b0 0x55f33d735500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.789+0000 7fa487b89740 1 --2- 172.21.15.96:0/1868467264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f33d3644d0 0x55f33d734690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.789+0000 7fa487b89740 1 --2- 172.21.15.96:0/1868467264 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55f33d3191c0 0x55f33d7a7cf0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.789+0000 7fa487b89740 1 --2- 172.21.15.96:0/1868467264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f33d35c2f0 0x55f33d734150 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.789+0000 7fa487b89740 1 -- 172.21.15.96:0/1868467264 >> 172.21.15.96:0/1868467264 conn(0x55f33d379300 msgr2=0x55f33d72dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.789+0000 7fa461ffb640 0 client.15654 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:13:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.789+0000 7fa487b89740 1 -- 172.21.15.96:0/1868467264 shutdown_connections 2024-04-25T14:13:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.789+0000 7fa487b89740 1 -- 172.21.15.96:0/1868467264 wait complete. 2024-04-25T14:13:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.789+0000 7fa487b89740 20 client.15654 trim_cache size 0 max 16384 2024-04-25T14:13:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T14:13:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T14:13:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T14:13:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T14:13:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.790+0000 7fa480bf8640 1 -- 172.21.15.96:0/3204420130 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa4680afae0 msgr2=0x7fa4680b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.790+0000 7fa480bf8640 1 --2- 172.21.15.96:0/3204420130 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa4680afae0 0x7fa4680b1ff0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa4740dcc00 tx=0x7fa474074040 comp rx=0 tx=0).stop 2024-04-25T14:13:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.790+0000 7fa480bf8640 1 -- 172.21.15.96:0/3204420130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa47c108960 msgr2=0x7fa47c1a1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.790+0000 7fa480bf8640 1 --2- 172.21.15.96:0/3204420130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa47c108960 0x7fa47c1a1750 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa47400eaa0 tx=0x7fa47400ef60 comp rx=0 tx=0).stop 2024-04-25T14:13:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.790+0000 7fa480bf8640 1 -- 172.21.15.96:0/3204420130 shutdown_connections 2024-04-25T14:13:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.790+0000 7fa480bf8640 1 --2- 172.21.15.96:0/3204420130 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fa4680afae0 0x7fa4680b1ff0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.790+0000 7fa480bf8640 1 --2- 172.21.15.96:0/3204420130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa47c108960 0x7fa47c1a1750 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.790+0000 7fa480bf8640 1 --2- 172.21.15.96:0/3204420130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa47c103c60 0x7fa47c19a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.790+0000 7fa480bf8640 1 --2- 172.21.15.96:0/3204420130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa47c102a20 0x7fa47c19a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.790+0000 7fa480bf8640 1 -- 172.21.15.96:0/3204420130 >> 172.21.15.96:0/3204420130 conn(0x7fa47c0fdc10 msgr2=0x7fa47c109ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.790+0000 7fa480bf8640 1 -- 172.21.15.96:0/3204420130 shutdown_connections 2024-04-25T14:13:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:13:51.790+0000 7fa480bf8640 1 -- 172.21.15.96:0/3204420130 wait complete. 2024-04-25T14:13:51.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:51 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:13:51.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:51 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:13:51.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:51 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:13:51.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:51 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:13:51.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:51 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:13:51.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:51 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:13:51.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:51 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:13:51.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:51 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:13:51.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:51 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:13:52.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:52 smithi096 ceph-mon[29207]: pgmap v353: 97 pgs: 97 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1023 B/s wr, 12 op/s 2024-04-25T14:13:52.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:52 smithi097 ceph-mon[32720]: pgmap v353: 97 pgs: 97 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1023 B/s wr, 12 op/s 2024-04-25T14:13:52.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:52 smithi151 ceph-mon[32990]: pgmap v353: 97 pgs: 97 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1023 B/s wr, 12 op/s 2024-04-25T14:13:54.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:54 smithi096 ceph-mon[29207]: pgmap v354: 97 pgs: 97 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-04-25T14:13:54.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:54 smithi097 ceph-mon[32720]: pgmap v354: 97 pgs: 97 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-04-25T14:13:54.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:54 smithi151 ceph-mon[32990]: pgmap v354: 97 pgs: 97 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-04-25T14:13:56.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:56 smithi096 ceph-mon[29207]: pgmap v355: 97 pgs: 97 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-25T14:13:56.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:56 smithi097 ceph-mon[32720]: pgmap v355: 97 pgs: 97 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-25T14:13:56.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:56 smithi151 ceph-mon[32990]: pgmap v355: 97 pgs: 97 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-25T14:13:58.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:13:58 smithi096 ceph-mon[29207]: pgmap v356: 97 pgs: 97 active+clean; 656 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.5 KiB/s wr, 16 op/s 2024-04-25T14:13:58.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:13:58 smithi097 ceph-mon[32720]: pgmap v356: 97 pgs: 97 active+clean; 656 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.5 KiB/s wr, 16 op/s 2024-04-25T14:13:58.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:13:58 smithi151 ceph-mon[32990]: pgmap v356: 97 pgs: 97 active+clean; 656 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.5 KiB/s wr, 16 op/s 2024-04-25T14:14:00.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:00 smithi096 ceph-mon[29207]: pgmap v357: 97 pgs: 97 active+clean; 656 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-04-25T14:14:00.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:00 smithi097 ceph-mon[32720]: pgmap v357: 97 pgs: 97 active+clean; 656 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-04-25T14:14:00.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:00 smithi151 ceph-mon[32990]: pgmap v357: 97 pgs: 97 active+clean; 656 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-04-25T14:14:02.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:02 smithi096 ceph-mon[29207]: pgmap v358: 97 pgs: 97 active+clean; 656 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.5 KiB/s wr, 16 op/s 2024-04-25T14:14:02.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:02 smithi097 ceph-mon[32720]: pgmap v358: 97 pgs: 97 active+clean; 656 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.5 KiB/s wr, 16 op/s 2024-04-25T14:14:02.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:02 smithi151 ceph-mon[32990]: pgmap v358: 97 pgs: 97 active+clean; 656 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.5 KiB/s wr, 16 op/s 2024-04-25T14:14:04.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:04 smithi096 ceph-mon[29207]: pgmap v359: 97 pgs: 97 active+clean; 656 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-04-25T14:14:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.897+0000 7fd1fc540640 1 -- 172.21.15.96:0/771945214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1f4108f30 msgr2=0x7fd1f410b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.897+0000 7fd1fc540640 1 --2- 172.21.15.96:0/771945214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1f4108f30 0x7fd1f410b3d0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0009900 tx=0x7fd1f002a410 comp rx=0 tx=0).stop 2024-04-25T14:14:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.897+0000 7fd1fc540640 1 -- 172.21.15.96:0/771945214 shutdown_connections 2024-04-25T14:14:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.897+0000 7fd1fc540640 1 --2- 172.21.15.96:0/771945214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1f410ba60 0x7fd1f410df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.897+0000 7fd1fc540640 1 --2- 172.21.15.96:0/771945214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1f4108f30 0x7fd1f410b3d0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.897+0000 7fd1fc540640 1 --2- 172.21.15.96:0/771945214 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd1f4104080 0x7fd1f4104490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.897+0000 7fd1fc540640 1 -- 172.21.15.96:0/771945214 >> 172.21.15.96:0/771945214 conn(0x7fd1f40fd400 msgr2=0x7fd1f40ff8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.897+0000 7fd1fc540640 1 -- 172.21.15.96:0/771945214 shutdown_connections 2024-04-25T14:14:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.897+0000 7fd1fc540640 1 -- 172.21.15.96:0/771945214 wait complete. 2024-04-25T14:14:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.898+0000 7fd1fc540640 1 Processor -- start 2024-04-25T14:14:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.898+0000 7fd1fc540640 1 -- start start 2024-04-25T14:14:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.898+0000 7fd1fc540640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd1f4104080 0x7fd1f419c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.898+0000 7fd1fc540640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1f4108f30 0x7fd1f419c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.898+0000 7fd1fc540640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1f410ba60 0x7fd1f41a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.898+0000 7fd1fc540640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f4111890 con 0x7fd1f410ba60 2024-04-25T14:14:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.898+0000 7fd1fc540640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f4111710 con 0x7fd1f4108f30 2024-04-25T14:14:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.898+0000 7fd1fc540640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f4111a10 con 0x7fd1f4104080 2024-04-25T14:14:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.898+0000 7fd1faadd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1f410ba60 0x7fd1f41a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:04.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.898+0000 7fd1faadd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1f410ba60 0x7fd1f41a3930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36480/0 (socket says 172.21.15.96:36480) 2024-04-25T14:14:04.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.898+0000 7fd1faadd640 1 -- 172.21.15.96:0/1747511787 learned_addr learned my addr 172.21.15.96:0/1747511787 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:04.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.898+0000 7fd1fa2dc640 1 --2- 172.21.15.96:0/1747511787 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd1f4104080 0x7fd1f419c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:04.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.898+0000 7fd1f9adb640 1 --2- 172.21.15.96:0/1747511787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1f4108f30 0x7fd1f419c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:04.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.899+0000 7fd1faadd640 1 -- 172.21.15.96:0/1747511787 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd1f4104080 msgr2=0x7fd1f419c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:04.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.899+0000 7fd1faadd640 1 --2- 172.21.15.96:0/1747511787 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd1f4104080 0x7fd1f419c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:04.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.899+0000 7fd1faadd640 1 -- 172.21.15.96:0/1747511787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1f4108f30 msgr2=0x7fd1f419c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:04.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.899+0000 7fd1faadd640 1 --2- 172.21.15.96:0/1747511787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1f4108f30 0x7fd1f419c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:04.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.899+0000 7fd1faadd640 1 -- 172.21.15.96:0/1747511787 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1f41a3fa0 con 0x7fd1f410ba60 2024-04-25T14:14:04.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.899+0000 7fd1faadd640 1 --2- 172.21.15.96:0/1747511787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1f410ba60 0x7fd1f41a3930 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec00cad0 tx=0x7fd1ec00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:04.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.899+0000 7fd1eb7fe640 1 -- 172.21.15.96:0/1747511787 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec00ae20 con 0x7fd1f410ba60 2024-04-25T14:14:04.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.899+0000 7fd1eb7fe640 1 -- 172.21.15.96:0/1747511787 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1ec005020 con 0x7fd1f410ba60 2024-04-25T14:14:04.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.899+0000 7fd1fc540640 1 -- 172.21.15.96:0/1747511787 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1f41a4290 con 0x7fd1f410ba60 2024-04-25T14:14:04.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.899+0000 7fd1eb7fe640 1 -- 172.21.15.96:0/1747511787 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec00f6b0 con 0x7fd1f410ba60 2024-04-25T14:14:04.902 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.899+0000 7fd1fc540640 1 -- 172.21.15.96:0/1747511787 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1f41a47e0 con 0x7fd1f410ba60 2024-04-25T14:14:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.902+0000 7fd2033f1740 1 Processor -- start 2024-04-25T14:14:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.902+0000 7fd1eb7fe640 1 -- 172.21.15.96:0/1747511787 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fd1ec0051d0 con 0x7fd1f410ba60 2024-04-25T14:14:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.902+0000 7fd2033f1740 1 -- start start 2024-04-25T14:14:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.902+0000 7fd2033f1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56433fabd1b0 0x56433fe5d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.902+0000 7fd2033f1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56433fa942f0 0x56433fe5def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.902+0000 7fd2033f1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433fa9c4d0 0x56433fe64df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.902+0000 7fd2033f1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56433fdc3460 con 0x56433fa9c4d0 2024-04-25T14:14:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.902+0000 7fd2033f1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56433fd7d590 con 0x56433fa942f0 2024-04-25T14:14:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.902+0000 7fd2033f1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56433fdbf420 con 0x56433fabd1b0 2024-04-25T14:14:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.902+0000 7fd1fa2dc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56433fa942f0 0x56433fe5def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.902+0000 7fd1f9adb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433fa9c4d0 0x56433fe64df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.902+0000 7fd1fa2dc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56433fa942f0 0x56433fe5def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:47468/0 (socket says 172.21.15.96:47468) 2024-04-25T14:14:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.902+0000 7fd1fa2dc640 1 -- 172.21.15.96:0/1527873475 learned_addr learned my addr 172.21.15.96:0/1527873475 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.903+0000 7fd1f9adb640 1 --2- 172.21.15.96:0/1527873475 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56433fabd1b0 0x56433fe5d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.903+0000 7fd1eb7fe640 1 --2- 172.21.15.96:0/1747511787 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd1e00afb60 0x7fd1e00b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.903+0000 7fd1fa2dc640 1 -- 172.21.15.96:0/1527873475 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56433fabd1b0 msgr2=0x56433fe5d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.903+0000 7fd1fa2dc640 1 --2- 172.21.15.96:0/1527873475 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56433fabd1b0 0x56433fe5d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.903+0000 7fd1fa2dc640 1 -- 172.21.15.96:0/1527873475 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433fa9c4d0 msgr2=0x56433fe64df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.903+0000 7fd1faadd640 1 --2- 172.21.15.96:0/1747511787 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd1e00afb60 0x7fd1e00b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.903+0000 7fd1fa2dc640 1 --2- 172.21.15.96:0/1527873475 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433fa9c4d0 0x56433fe64df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.903+0000 7fd1fa2dc640 1 -- 172.21.15.96:0/1527873475 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56433fd8a0d0 con 0x56433fa942f0 2024-04-25T14:14:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.903+0000 7fd1f9adb640 1 --2- 172.21.15.96:0/1527873475 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433fa9c4d0 0x56433fe64df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:14:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.903+0000 7fd1eb7fe640 1 -- 172.21.15.96:0/1747511787 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fd1ec0dec40 con 0x7fd1f410ba60 2024-04-25T14:14:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.903+0000 7fd1fa2dc640 1 --2- 172.21.15.96:0/1527873475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56433fa942f0 0x56433fe5def0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd1e400b6c0 tx=0x7fd1e400bb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=384df5eb2b6155a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.903+0000 7fd1faadd640 1 --2- 172.21.15.96:0/1747511787 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd1e00afb60 0x7fd1e00b2070 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec0dca90 tx=0x7fd1ec075c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.903+0000 7fd1e8bfe640 1 -- 172.21.15.96:0/1527873475 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e4003bd0 con 0x56433fa942f0 2024-04-25T14:14:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.903+0000 7fd1e8bfe640 1 -- 172.21.15.96:0/1527873475 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1e4015070 con 0x56433fa942f0 2024-04-25T14:14:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.903+0000 7fd1e8bfe640 1 -- 172.21.15.96:0/1527873475 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e4004cd0 con 0x56433fa942f0 2024-04-25T14:14:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.904+0000 7fd2033f1740 1 -- 172.21.15.96:0/1527873475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56433fa942f0 msgr2=0x56433fe5def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.904+0000 7fd2033f1740 1 --2- 172.21.15.96:0/1527873475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56433fa942f0 0x56433fe5def0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd1e400b6c0 tx=0x7fd1e400bb80 comp rx=0 tx=0).stop 2024-04-25T14:14:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.904+0000 7fd2033f1740 1 -- 172.21.15.96:0/1527873475 shutdown_connections 2024-04-25T14:14:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.904+0000 7fd2033f1740 1 --2- 172.21.15.96:0/1527873475 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433fa9c4d0 0x56433fe64df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.904+0000 7fd2033f1740 1 --2- 172.21.15.96:0/1527873475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56433fa942f0 0x56433fe5def0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.904+0000 7fd2033f1740 1 --2- 172.21.15.96:0/1527873475 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56433fabd1b0 0x56433fe5d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.904+0000 7fd2033f1740 1 -- 172.21.15.96:0/1527873475 >> 172.21.15.96:0/1527873475 conn(0x56433fab1300 msgr2=0x56433fe59300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.904+0000 7fd2033f1740 1 -- 172.21.15.96:0/1527873475 shutdown_connections 2024-04-25T14:14:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.904+0000 7fd2033f1740 1 -- 172.21.15.96:0/1527873475 wait complete. 2024-04-25T14:14:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.905+0000 7fd2033f1740 1 -- start start 2024-04-25T14:14:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.905+0000 7fd2033f1740 1 Processor -- start 2024-04-25T14:14:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.905+0000 7fd2033f1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433fa942f0 0x56433fe67620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.905+0000 7fd2033f1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56433fa9c4d0 0x56433fe67b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.905+0000 7fd2033f1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56433fabd1b0 0x56433fe6eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.905+0000 7fd2033f1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56433fe65840 con 0x56433fa942f0 2024-04-25T14:14:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.905+0000 7fd2033f1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56433fd4ab60 con 0x56433fa9c4d0 2024-04-25T14:14:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.905+0000 7fd2033f1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56433fe659c0 con 0x56433fabd1b0 2024-04-25T14:14:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.905+0000 7fd1f9adb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433fa942f0 0x56433fe67620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.905+0000 7fd2033f1740 10 client.0 _ll_register_callbacks cb 0x56433fd6f2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:14:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.905+0000 7fd1f9adb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433fa942f0 0x56433fe67620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36492/0 (socket says 172.21.15.96:36492) 2024-04-25T14:14:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.905+0000 7fd1f9adb640 1 -- 172.21.15.96:0/1706319268 learned_addr learned my addr 172.21.15.96:0/1706319268 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.905+0000 7fd1fa2dc640 1 --2- 172.21.15.96:0/1706319268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56433fa9c4d0 0x56433fe67b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.906+0000 7fd1f9adb640 1 -- 172.21.15.96:0/1706319268 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56433fabd1b0 msgr2=0x56433fe6eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.906+0000 7fd1f9adb640 1 --2- 172.21.15.96:0/1706319268 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56433fabd1b0 0x56433fe6eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.906+0000 7fd1f9adb640 1 -- 172.21.15.96:0/1706319268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56433fa9c4d0 msgr2=0x56433fe67b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.906+0000 7fd1f9adb640 1 --2- 172.21.15.96:0/1706319268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56433fa9c4d0 0x56433fe67b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.906+0000 7fd1f9adb640 1 -- 172.21.15.96:0/1706319268 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56433fad5210 con 0x56433fa942f0 2024-04-25T14:14:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.906+0000 7fd1f9adb640 1 --2- 172.21.15.96:0/1706319268 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433fa942f0 0x56433fe67620 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0005710 tx=0x7fd1f00314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2f09f559b1a3813 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.906+0000 7fd1ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:14:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.906+0000 7fd1ddffb640 1 -- 172.21.15.96:0/1706319268 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f0035040 con 0x56433fa942f0 2024-04-25T14:14:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.906+0000 7fd1ddffb640 1 -- 172.21.15.96:0/1706319268 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fd1f0005220 con 0x56433fa942f0 2024-04-25T14:14:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.906+0000 7fd2033f1740 1 -- 172.21.15.96:0/1706319268 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1e4015070 con 0x56433fa942f0 2024-04-25T14:14:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.906+0000 7fd1ddffb640 1 -- 172.21.15.96:0/1706319268 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd1f0005420 con 0x56433fa942f0 2024-04-25T14:14:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.906+0000 7fd2033f1740 10 client.15666 fetch_fsmap learned FSMap version 26 2024-04-25T14:14:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.906+0000 7fd2033f1740 1 -- 172.21.15.96:0/1706319268 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56433fba1620 con 0x56433fa942f0 2024-04-25T14:14:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.906+0000 7fd1ddffb640 1 -- 172.21.15.96:0/1706319268 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1e4015070 con 0x56433fa942f0 2024-04-25T14:14:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.906+0000 7fd1ddffb640 1 -- 172.21.15.96:0/1706319268 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd1f003a040 con 0x56433fa942f0 2024-04-25T14:14:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.906+0000 7fd2033f1740 10 client.15666 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:14:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.906+0000 7fd2033f1740 10 client.15666 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:14:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.906+0000 7fd2033f1740 20 client.15666 populate_metadata read hostname 'smithi096' 2024-04-25T14:14:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.906+0000 7fd2033f1740 1 --2- 172.21.15.96:0/1706319268 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x56433fa7bd80 0x56433fe752a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.907+0000 7fd2033f1740 4 client.15666 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:14:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.907+0000 7fd2033f1740 1 -- 172.21.15.96:0/1706319268 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56433fdd1070 con 0x56433fa7bd80 2024-04-25T14:14:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.907+0000 7fd1fa2dc640 1 --2- 172.21.15.96:0/1706319268 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x56433fa7bd80 0x56433fe752a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.907+0000 7fd1ddffb640 10 client.15666 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.907+0000 7fd1fa2dc640 1 --2- 172.21.15.96:0/1706319268 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x56433fa7bd80 0x56433fe752a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e3297717e20321e server_cookie=8be4fecf1e97efe0 in_seq=0 out_seq=0 2024-04-25T14:14:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.908+0000 7fd1ddffb640 1 -- 172.21.15.96:0/1706319268 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56433fdd1070 con 0x56433fa7bd80 2024-04-25T14:14:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.908+0000 7fd1ddffb640 10 client.15666 handle_command_reply: tid=0 2024-04-25T14:14:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.908+0000 7fd2033f1740 1 client.15666 shutdown 2024-04-25T14:14:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.908+0000 7fd2033f1740 1 -- 172.21.15.96:0/1706319268 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433fa942f0 msgr2=0x56433fe67620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.908+0000 7fd2033f1740 1 --2- 172.21.15.96:0/1706319268 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433fa942f0 0x56433fe67620 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0005710 tx=0x7fd1f00314a0 comp rx=0 tx=0).stop 2024-04-25T14:14:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.908+0000 7fd2033f1740 1 -- 172.21.15.96:0/1706319268 shutdown_connections 2024-04-25T14:14:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.908+0000 7fd2033f1740 1 --2- 172.21.15.96:0/1706319268 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56433fabd1b0 0x56433fe6eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.908+0000 7fd2033f1740 1 --2- 172.21.15.96:0/1706319268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56433fa9c4d0 0x56433fe67b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.908+0000 7fd2033f1740 1 --2- 172.21.15.96:0/1706319268 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x56433fa7bd80 0x56433fe752a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.908+0000 7fd2033f1740 1 --2- 172.21.15.96:0/1706319268 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433fa942f0 0x56433fe67620 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.908+0000 7fd2033f1740 1 -- 172.21.15.96:0/1706319268 >> 172.21.15.96:0/1706319268 conn(0x56433fab1300 msgr2=0x56433fe5c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.908+0000 7fd1ddffb640 0 client.15666 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.909+0000 7fd2033f1740 1 -- 172.21.15.96:0/1706319268 shutdown_connections 2024-04-25T14:14:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.909+0000 7fd2033f1740 1 -- 172.21.15.96:0/1706319268 wait complete. 2024-04-25T14:14:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.909+0000 7fd2033f1740 20 client.15666 trim_cache size 0 max 16384 2024-04-25T14:14:04.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:04 smithi097 ceph-mon[32720]: pgmap v359: 97 pgs: 97 active+clean; 656 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-04-25T14:14:04.928 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.924+0000 7fd2033f1740 1 Processor -- start 2024-04-25T14:14:04.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.924+0000 7fd2033f1740 1 -- start start 2024-04-25T14:14:04.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.924+0000 7fd2033f1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433fa942f0 0x56433fe62110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:04.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.924+0000 7fd2033f1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56433fa9c4d0 0x56433fe6ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:04.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.924+0000 7fd2033f1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56433fabd1b0 0x56433fe59320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:04.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.924+0000 7fd2033f1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f41ad500 con 0x56433fa942f0 2024-04-25T14:14:04.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.924+0000 7fd2033f1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f41af040 con 0x56433fabd1b0 2024-04-25T14:14:04.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.924+0000 7fd2033f1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f41adbd0 con 0x56433fa9c4d0 2024-04-25T14:14:04.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.924+0000 7fd1f9adb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433fa942f0 0x56433fe62110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:04.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.924+0000 7fd1f9adb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433fa942f0 0x56433fe62110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36506/0 (socket says 172.21.15.96:36506) 2024-04-25T14:14:04.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.924+0000 7fd1f9adb640 1 -- 172.21.15.96:0/1521878784 learned_addr learned my addr 172.21.15.96:0/1521878784 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:04.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.924+0000 7fd1fa2dc640 1 --2- 172.21.15.96:0/1521878784 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56433fa9c4d0 0x56433fe6ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:04.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.924+0000 7fd1f9adb640 1 --2- 172.21.15.96:0/1521878784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56433fabd1b0 0x56433fe59320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.924+0000 7fd1f9adb640 1 -- 172.21.15.96:0/1521878784 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56433fa9c4d0 msgr2=0x56433fe6ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.924+0000 7fd1f9adb640 1 --2- 172.21.15.96:0/1521878784 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56433fa9c4d0 0x56433fe6ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.924+0000 7fd1f9adb640 1 -- 172.21.15.96:0/1521878784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56433fabd1b0 msgr2=0x56433fe59320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.924+0000 7fd1f9adb640 1 --2- 172.21.15.96:0/1521878784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56433fabd1b0 0x56433fe59320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.924+0000 7fd1f9adb640 1 -- 172.21.15.96:0/1521878784 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56433fe5d480 con 0x56433fa942f0 2024-04-25T14:14:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.924+0000 7fd1fa2dc640 1 --2- 172.21.15.96:0/1521878784 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56433fa9c4d0 0x56433fe6ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:14:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.925+0000 7fd1f9adb640 1 --2- 172.21.15.96:0/1521878784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433fa942f0 0x56433fe62110 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0005ab0 tx=0x7fd1f0032f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83fe7e5ee23d35c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.925+0000 7fd1dffff640 1 -- 172.21.15.96:0/1521878784 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f0035040 con 0x56433fa942f0 2024-04-25T14:14:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.925+0000 7fd1dffff640 1 -- 172.21.15.96:0/1521878784 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1f0042070 con 0x56433fa942f0 2024-04-25T14:14:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.925+0000 7fd1dffff640 1 -- 172.21.15.96:0/1521878784 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f00325a0 con 0x56433fa942f0 2024-04-25T14:14:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.925+0000 7fd2033f1740 1 -- 172.21.15.96:0/1521878784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433fa942f0 msgr2=0x56433fe62110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.925+0000 7fd2033f1740 1 --2- 172.21.15.96:0/1521878784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433fa942f0 0x56433fe62110 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0005ab0 tx=0x7fd1f0032f10 comp rx=0 tx=0).stop 2024-04-25T14:14:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.926+0000 7fd2033f1740 1 -- 172.21.15.96:0/1521878784 shutdown_connections 2024-04-25T14:14:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.926+0000 7fd2033f1740 1 --2- 172.21.15.96:0/1521878784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56433fabd1b0 0x56433fe59320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.926+0000 7fd2033f1740 1 --2- 172.21.15.96:0/1521878784 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56433fa9c4d0 0x56433fe6ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.926+0000 7fd2033f1740 1 --2- 172.21.15.96:0/1521878784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433fa942f0 0x56433fe62110 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.926+0000 7fd2033f1740 1 -- 172.21.15.96:0/1521878784 >> 172.21.15.96:0/1521878784 conn(0x56433fab1300 msgr2=0x56433fe58c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.926+0000 7fd2033f1740 1 -- 172.21.15.96:0/1521878784 shutdown_connections 2024-04-25T14:14:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.926+0000 7fd2033f1740 1 -- 172.21.15.96:0/1521878784 wait complete. 2024-04-25T14:14:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.926+0000 7fd2033f1740 1 -- start start 2024-04-25T14:14:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.926+0000 7fd2033f1740 1 Processor -- start 2024-04-25T14:14:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.926+0000 7fd2033f1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433fa942f0 0x56433fe6c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.926+0000 7fd2033f1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56433fa9c4d0 0x56433fe6c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.926+0000 7fd2033f1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56433fabd1b0 0x56433fe6d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.926+0000 7fd2033f1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f41ae970 con 0x56433fa942f0 2024-04-25T14:14:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.926+0000 7fd2033f1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f41ae2a0 con 0x56433fa9c4d0 2024-04-25T14:14:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.926+0000 7fd2033f1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f41ab4e0 con 0x56433fabd1b0 2024-04-25T14:14:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.926+0000 7fd2033f1740 10 client.0 _ll_register_callbacks cb 0x7fd1f00319e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:14:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.926+0000 7fd1f9adb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433fa942f0 0x56433fe6c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.926+0000 7fd1f9adb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433fa942f0 0x56433fe6c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36514/0 (socket says 172.21.15.96:36514) 2024-04-25T14:14:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.926+0000 7fd1f9adb640 1 -- 172.21.15.96:0/2894015258 learned_addr learned my addr 172.21.15.96:0/2894015258 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.926+0000 7fd1fa2dc640 1 --2- 172.21.15.96:0/2894015258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56433fa9c4d0 0x56433fe6c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.926+0000 7fd1f9adb640 1 --2- 172.21.15.96:0/2894015258 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56433fabd1b0 0x56433fe6d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.927+0000 7fd1f9adb640 1 -- 172.21.15.96:0/2894015258 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56433fabd1b0 msgr2=0x56433fe6d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.927+0000 7fd1f9adb640 1 --2- 172.21.15.96:0/2894015258 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56433fabd1b0 0x56433fe6d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.927+0000 7fd1f9adb640 1 -- 172.21.15.96:0/2894015258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56433fa9c4d0 msgr2=0x56433fe6c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.927+0000 7fd1f9adb640 1 --2- 172.21.15.96:0/2894015258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56433fa9c4d0 0x56433fe6c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.927+0000 7fd1f9adb640 1 -- 172.21.15.96:0/2894015258 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56433fe6dad0 con 0x56433fa942f0 2024-04-25T14:14:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.927+0000 7fd1f9adb640 1 --2- 172.21.15.96:0/2894015258 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433fa942f0 0x56433fe6c0f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0033000 tx=0x7fd1f0031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1a4edb81f5598c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.927+0000 7fd1ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:14:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.927+0000 7fd1ddffb640 1 -- 172.21.15.96:0/2894015258 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f0035040 con 0x56433fa942f0 2024-04-25T14:14:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.927+0000 7fd1ddffb640 1 -- 172.21.15.96:0/2894015258 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fd1f0003ce0 con 0x56433fa942f0 2024-04-25T14:14:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.927+0000 7fd2033f1740 1 -- 172.21.15.96:0/2894015258 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1f0042070 con 0x56433fa942f0 2024-04-25T14:14:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.927+0000 7fd1ddffb640 1 -- 172.21.15.96:0/2894015258 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd1f0032320 con 0x56433fa942f0 2024-04-25T14:14:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.927+0000 7fd2033f1740 10 client.15672 fetch_fsmap learned FSMap version 26 2024-04-25T14:14:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.927+0000 7fd2033f1740 1 -- 172.21.15.96:0/2894015258 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56433fe6ed60 con 0x56433fa942f0 2024-04-25T14:14:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.927+0000 7fd1ddffb640 1 -- 172.21.15.96:0/2894015258 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1f0042070 con 0x56433fa942f0 2024-04-25T14:14:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.927+0000 7fd1ddffb640 1 -- 172.21.15.96:0/2894015258 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd1f0053730 con 0x56433fa942f0 2024-04-25T14:14:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.927+0000 7fd2033f1740 10 client.15672 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:14:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.927+0000 7fd2033f1740 10 client.15672 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:14:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.927+0000 7fd2033f1740 20 client.15672 populate_metadata read hostname 'smithi096' 2024-04-25T14:14:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.927+0000 7fd2033f1740 1 --2- 172.21.15.96:0/2894015258 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x56433fa511c0 0x56433fedfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.927+0000 7fd2033f1740 4 client.15672 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:14:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.927+0000 7fd2033f1740 1 -- 172.21.15.96:0/2894015258 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56433fdcaee0 con 0x56433fa511c0 2024-04-25T14:14:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.928+0000 7fd1fa2dc640 1 --2- 172.21.15.96:0/2894015258 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x56433fa511c0 0x56433fedfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.928+0000 7fd1ddffb640 10 client.15672 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.928+0000 7fd1fa2dc640 1 --2- 172.21.15.96:0/2894015258 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x56433fa511c0 0x56433fedfdc0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2a97fdae16f8fc server_cookie=727f6bd6cd05eed7 in_seq=0 out_seq=0 2024-04-25T14:14:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.932+0000 7fd1ddffb640 1 -- 172.21.15.96:0/2894015258 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56433fdcaee0 con 0x56433fa511c0 2024-04-25T14:14:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.932+0000 7fd1ddffb640 10 client.15672 handle_command_reply: tid=0 2024-04-25T14:14:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.932+0000 7fd2033f1740 1 client.15672 shutdown 2024-04-25T14:14:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.932+0000 7fd2033f1740 1 -- 172.21.15.96:0/2894015258 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433fa942f0 msgr2=0x56433fe6c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.932+0000 7fd2033f1740 1 --2- 172.21.15.96:0/2894015258 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433fa942f0 0x56433fe6c0f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0033000 tx=0x7fd1f0031400 comp rx=0 tx=0).stop 2024-04-25T14:14:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.932+0000 7fd2033f1740 1 -- 172.21.15.96:0/2894015258 shutdown_connections 2024-04-25T14:14:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.932+0000 7fd2033f1740 1 --2- 172.21.15.96:0/2894015258 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56433fabd1b0 0x56433fe6d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.932+0000 7fd2033f1740 1 --2- 172.21.15.96:0/2894015258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56433fa9c4d0 0x56433fe6c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.932+0000 7fd2033f1740 1 --2- 172.21.15.96:0/2894015258 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x56433fa511c0 0x56433fedfdc0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.932+0000 7fd2033f1740 1 --2- 172.21.15.96:0/2894015258 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56433fa942f0 0x56433fe6c0f0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.932+0000 7fd1ddffb640 0 client.15672 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.932+0000 7fd2033f1740 1 -- 172.21.15.96:0/2894015258 >> 172.21.15.96:0/2894015258 conn(0x56433fab1300 msgr2=0x56433fe79a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.932+0000 7fd2033f1740 1 -- 172.21.15.96:0/2894015258 shutdown_connections 2024-04-25T14:14:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.932+0000 7fd2033f1740 1 -- 172.21.15.96:0/2894015258 wait complete. 2024-04-25T14:14:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.932+0000 7fd2033f1740 20 client.15672 trim_cache size 0 max 16384 2024-04-25T14:14:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T14:14:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T14:14:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T14:14:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T14:14:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.933+0000 7fd1fc540640 1 -- 172.21.15.96:0/1747511787 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd1e00afb60 msgr2=0x7fd1e00b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.933+0000 7fd1fc540640 1 --2- 172.21.15.96:0/1747511787 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd1e00afb60 0x7fd1e00b2070 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec0dca90 tx=0x7fd1ec075c50 comp rx=0 tx=0).stop 2024-04-25T14:14:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.933+0000 7fd1fc540640 1 -- 172.21.15.96:0/1747511787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1f410ba60 msgr2=0x7fd1f41a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.933+0000 7fd1fc540640 1 --2- 172.21.15.96:0/1747511787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1f410ba60 0x7fd1f41a3930 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec00cad0 tx=0x7fd1ec00cf90 comp rx=0 tx=0).stop 2024-04-25T14:14:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.933+0000 7fd1fc540640 1 -- 172.21.15.96:0/1747511787 shutdown_connections 2024-04-25T14:14:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.933+0000 7fd1fc540640 1 --2- 172.21.15.96:0/1747511787 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd1e00afb60 0x7fd1e00b2070 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.933+0000 7fd1fc540640 1 --2- 172.21.15.96:0/1747511787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1f410ba60 0x7fd1f41a3930 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.933+0000 7fd1fc540640 1 --2- 172.21.15.96:0/1747511787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1f4108f30 0x7fd1f419c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.933+0000 7fd1fc540640 1 --2- 172.21.15.96:0/1747511787 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd1f4104080 0x7fd1f419c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.933+0000 7fd1fc540640 1 -- 172.21.15.96:0/1747511787 >> 172.21.15.96:0/1747511787 conn(0x7fd1f40fd400 msgr2=0x7fd1f410ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.933+0000 7fd1fc540640 1 -- 172.21.15.96:0/1747511787 shutdown_connections 2024-04-25T14:14:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:04.933+0000 7fd1fc540640 1 -- 172.21.15.96:0/1747511787 wait complete. 2024-04-25T14:14:04.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:04 smithi151 ceph-mon[32990]: pgmap v359: 97 pgs: 97 active+clean; 656 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-04-25T14:14:06.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:06 smithi096 ceph-mon[29207]: pgmap v360: 97 pgs: 97 active+clean; 656 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-25T14:14:06.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:06 smithi097 ceph-mon[32720]: pgmap v360: 97 pgs: 97 active+clean; 656 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-25T14:14:06.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:06 smithi151 ceph-mon[32990]: pgmap v360: 97 pgs: 97 active+clean; 656 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-25T14:14:08.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:08 smithi096 ceph-mon[29207]: pgmap v361: 97 pgs: 97 active+clean; 657 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.2 KiB/s wr, 15 op/s 2024-04-25T14:14:08.883 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:08 smithi151 ceph-mon[32990]: pgmap v361: 97 pgs: 97 active+clean; 657 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.2 KiB/s wr, 15 op/s 2024-04-25T14:14:08.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:08 smithi097 ceph-mon[32720]: pgmap v361: 97 pgs: 97 active+clean; 657 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.2 KiB/s wr, 15 op/s 2024-04-25T14:14:10.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:10 smithi096 ceph-mon[29207]: pgmap v362: 97 pgs: 97 active+clean; 657 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-25T14:14:10.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:10 smithi097 ceph-mon[32720]: pgmap v362: 97 pgs: 97 active+clean; 657 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-25T14:14:10.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:10 smithi151 ceph-mon[32990]: pgmap v362: 97 pgs: 97 active+clean; 657 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-25T14:14:11.456 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:14:11.456 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:14:11.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.584+0000 7fd43f36c640 1 -- 172.21.15.96:0/1805416290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4381029f0 msgr2=0x7fd4381036f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:11.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.584+0000 7fd43f36c640 1 --2- 172.21.15.96:0/1805416290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4381029f0 0x7fd4381036f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd434009900 tx=0x7fd43402a410 comp rx=0 tx=0).stop 2024-04-25T14:14:11.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.584+0000 7fd43f36c640 1 -- 172.21.15.96:0/1805416290 shutdown_connections 2024-04-25T14:14:11.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.584+0000 7fd43f36c640 1 --2- 172.21.15.96:0/1805416290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd438108930 0x7fd43810ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:11.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.584+0000 7fd43f36c640 1 --2- 172.21.15.96:0/1805416290 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd438103c30 0x7fd438104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:11.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.584+0000 7fd43f36c640 1 --2- 172.21.15.96:0/1805416290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4381029f0 0x7fd4381036f0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:11.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.584+0000 7fd43f36c640 1 -- 172.21.15.96:0/1805416290 >> 172.21.15.96:0/1805416290 conn(0x7fd4380fdbc0 msgr2=0x7fd438100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:11.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.585+0000 7fd43f36c640 1 -- 172.21.15.96:0/1805416290 shutdown_connections 2024-04-25T14:14:11.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.585+0000 7fd43f36c640 1 -- 172.21.15.96:0/1805416290 wait complete. 2024-04-25T14:14:11.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.585+0000 7fd43f36c640 1 Processor -- start 2024-04-25T14:14:11.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.585+0000 7fd43f36c640 1 -- start start 2024-04-25T14:14:11.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.585+0000 7fd43f36c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4381029f0 0x7fd43819a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:11.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.585+0000 7fd43f36c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd438103c30 0x7fd43819a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:11.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.585+0000 7fd43f36c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd438108930 0x7fd4381a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:11.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.585+0000 7fd43f36c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd43810f620 con 0x7fd438108930 2024-04-25T14:14:11.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.585+0000 7fd43f36c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd43810f4a0 con 0x7fd438103c30 2024-04-25T14:14:11.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.585+0000 7fd43f36c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd43810f7a0 con 0x7fd4381029f0 2024-04-25T14:14:11.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.586+0000 7fd43d909640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd438108930 0x7fd4381a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:11.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.586+0000 7fd43d909640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd438108930 0x7fd4381a1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58520/0 (socket says 172.21.15.96:58520) 2024-04-25T14:14:11.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.586+0000 7fd43d909640 1 -- 172.21.15.96:0/2603553965 learned_addr learned my addr 172.21.15.96:0/2603553965 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:11.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.586+0000 7fd43d108640 1 --2- 172.21.15.96:0/2603553965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4381029f0 0x7fd43819a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:11.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.586+0000 7fd43c907640 1 --2- 172.21.15.96:0/2603553965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd438103c30 0x7fd43819a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:11.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.586+0000 7fd43d909640 1 -- 172.21.15.96:0/2603553965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4381029f0 msgr2=0x7fd43819a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:11.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.586+0000 7fd43d909640 1 --2- 172.21.15.96:0/2603553965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4381029f0 0x7fd43819a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:11.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.586+0000 7fd43d909640 1 -- 172.21.15.96:0/2603553965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd438103c30 msgr2=0x7fd43819a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:11.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.586+0000 7fd43d909640 1 --2- 172.21.15.96:0/2603553965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd438103c30 0x7fd43819a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:11.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.586+0000 7fd43d909640 1 -- 172.21.15.96:0/2603553965 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4381a1db0 con 0x7fd438108930 2024-04-25T14:14:11.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.586+0000 7fd43c907640 1 --2- 172.21.15.96:0/2603553965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd438103c30 0x7fd43819a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:14:11.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.586+0000 7fd43d108640 1 --2- 172.21.15.96:0/2603553965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4381029f0 0x7fd43819a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:14:11.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.586+0000 7fd43d909640 1 --2- 172.21.15.96:0/2603553965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd438108930 0x7fd4381a1740 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd43000eb10 tx=0x7fd43000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:11.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.586+0000 7fd42e7fc640 1 -- 172.21.15.96:0/2603553965 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd430004b20 con 0x7fd438108930 2024-04-25T14:14:11.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.586+0000 7fd42e7fc640 1 -- 172.21.15.96:0/2603553965 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd430004cc0 con 0x7fd438108930 2024-04-25T14:14:11.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.586+0000 7fd42e7fc640 1 -- 172.21.15.96:0/2603553965 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd430010670 con 0x7fd438108930 2024-04-25T14:14:11.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.586+0000 7fd43f36c640 1 -- 172.21.15.96:0/2603553965 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4381a20a0 con 0x7fd438108930 2024-04-25T14:14:11.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.586+0000 7fd43f36c640 1 -- 172.21.15.96:0/2603553965 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4381a25a0 con 0x7fd438108930 2024-04-25T14:14:11.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.589+0000 7fd44621b740 1 Processor -- start 2024-04-25T14:14:11.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.589+0000 7fd42e7fc640 1 -- 172.21.15.96:0/2603553965 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fd430005020 con 0x7fd438108930 2024-04-25T14:14:11.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.589+0000 7fd44621b740 1 -- start start 2024-04-25T14:14:11.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.589+0000 7fd44621b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b042b14390 0x55b042eb5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:11.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.589+0000 7fd44621b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b042aeb4d0 0x55b042eb5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:11.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.589+0000 7fd44621b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b042af36b0 0x55b042eb6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:11.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.589+0000 7fd44621b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b042e1ad80 con 0x55b042b14390 2024-04-25T14:14:11.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.589+0000 7fd44621b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b042dd48b0 con 0x55b042aeb4d0 2024-04-25T14:14:11.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.589+0000 7fd44621b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b042e16d40 con 0x55b042af36b0 2024-04-25T14:14:11.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.589+0000 7fd43c907640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b042b14390 0x55b042eb5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:11.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.589+0000 7fd43c907640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b042b14390 0x55b042eb5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58522/0 (socket says 172.21.15.96:58522) 2024-04-25T14:14:11.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.589+0000 7fd43c907640 1 -- 172.21.15.96:0/3471423317 learned_addr learned my addr 172.21.15.96:0/3471423317 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:11.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.589+0000 7fd43d108640 1 --2- 172.21.15.96:0/3471423317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b042aeb4d0 0x55b042eb5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:11.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.589+0000 7fd43c907640 1 --2- 172.21.15.96:0/3471423317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b042af36b0 0x55b042eb6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:11.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.590+0000 7fd43c907640 1 -- 172.21.15.96:0/3471423317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b042af36b0 msgr2=0x55b042eb6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:11.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.590+0000 7fd43c907640 1 --2- 172.21.15.96:0/3471423317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b042af36b0 0x55b042eb6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:11.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.590+0000 7fd43c907640 1 -- 172.21.15.96:0/3471423317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b042aeb4d0 msgr2=0x55b042eb5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:11.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.590+0000 7fd43c907640 1 --2- 172.21.15.96:0/3471423317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b042aeb4d0 0x55b042eb5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:11.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.590+0000 7fd42e7fc640 1 --2- 172.21.15.96:0/2603553965 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd4240afaf0 0x7fd4240b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:11.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.590+0000 7fd43c907640 1 -- 172.21.15.96:0/3471423317 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b042da5a40 con 0x55b042b14390 2024-04-25T14:14:11.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.590+0000 7fd42e7fc640 1 -- 172.21.15.96:0/2603553965 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fd4300df340 con 0x7fd438108930 2024-04-25T14:14:11.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.590+0000 7fd43c907640 1 --2- 172.21.15.96:0/2603553965 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd4240afaf0 0x7fd4240b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:11.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.590+0000 7fd43c907640 1 --2- 172.21.15.96:0/3471423317 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b042b14390 0x55b042eb5a60 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd428015590 tx=0x7fd428009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=664d2b2f0f6c8f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:11.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.590+0000 7fd4237fe640 1 -- 172.21.15.96:0/3471423317 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4280106d0 con 0x55b042b14390 2024-04-25T14:14:11.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.590+0000 7fd4237fe640 1 -- 172.21.15.96:0/3471423317 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd428010e40 con 0x55b042b14390 2024-04-25T14:14:11.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.590+0000 7fd4237fe640 1 -- 172.21.15.96:0/3471423317 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd428022a60 con 0x55b042b14390 2024-04-25T14:14:11.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.591+0000 7fd44621b740 1 -- 172.21.15.96:0/3471423317 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b042b14390 msgr2=0x55b042eb5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:11.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.591+0000 7fd43c907640 1 --2- 172.21.15.96:0/2603553965 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd4240afaf0 0x7fd4240b2000 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd428020000 tx=0x7fd42801d000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:11.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.591+0000 7fd44621b740 1 --2- 172.21.15.96:0/3471423317 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b042b14390 0x55b042eb5a60 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd428015590 tx=0x7fd428009630 comp rx=0 tx=0).stop 2024-04-25T14:14:11.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.591+0000 7fd44621b740 1 -- 172.21.15.96:0/3471423317 shutdown_connections 2024-04-25T14:14:11.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.591+0000 7fd44621b740 1 --2- 172.21.15.96:0/3471423317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b042af36b0 0x55b042eb6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:11.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.591+0000 7fd44621b740 1 --2- 172.21.15.96:0/3471423317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b042aeb4d0 0x55b042eb5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:11.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.591+0000 7fd44621b740 1 --2- 172.21.15.96:0/3471423317 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b042b14390 0x55b042eb5a60 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:11.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.591+0000 7fd44621b740 1 -- 172.21.15.96:0/3471423317 >> 172.21.15.96:0/3471423317 conn(0x55b042b084e0 msgr2=0x55b042eb1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:11.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.591+0000 7fd44621b740 1 -- 172.21.15.96:0/3471423317 shutdown_connections 2024-04-25T14:14:11.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.591+0000 7fd44621b740 1 -- 172.21.15.96:0/3471423317 wait complete. 2024-04-25T14:14:11.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.592+0000 7fd44621b740 1 -- start start 2024-04-25T14:14:11.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.592+0000 7fd44621b740 1 Processor -- start 2024-04-25T14:14:11.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.592+0000 7fd44621b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b042aeb4d0 0x55b042ebf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:11.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.592+0000 7fd44621b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b042af36b0 0x55b042ebfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:11.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.592+0000 7fd44621b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b042b14390 0x55b042ec6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:11.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.592+0000 7fd44621b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b042eb7780 con 0x55b042b14390 2024-04-25T14:14:11.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.592+0000 7fd44621b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b042da1d10 con 0x55b042af36b0 2024-04-25T14:14:11.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.592+0000 7fd44621b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b042eb7900 con 0x55b042aeb4d0 2024-04-25T14:14:11.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.592+0000 7fd44621b740 10 client.0 _ll_register_callbacks cb 0x55b042dce210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:14:11.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.592+0000 7fd43d909640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b042b14390 0x55b042ec6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:11.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.592+0000 7fd43d909640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b042b14390 0x55b042ec6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58528/0 (socket says 172.21.15.96:58528) 2024-04-25T14:14:11.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.592+0000 7fd43d909640 1 -- 172.21.15.96:0/2971052294 learned_addr learned my addr 172.21.15.96:0/2971052294 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:11.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.592+0000 7fd43c907640 1 --2- 172.21.15.96:0/2971052294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b042af36b0 0x55b042ebfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:11.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.592+0000 7fd43d108640 1 --2- 172.21.15.96:0/2971052294 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b042aeb4d0 0x55b042ebf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:11.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.593+0000 7fd43d909640 1 -- 172.21.15.96:0/2971052294 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b042aeb4d0 msgr2=0x55b042ebf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:11.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.593+0000 7fd43d909640 1 --2- 172.21.15.96:0/2971052294 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b042aeb4d0 0x55b042ebf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:11.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.593+0000 7fd43d909640 1 -- 172.21.15.96:0/2971052294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b042af36b0 msgr2=0x55b042ebfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:11.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.593+0000 7fd43d909640 1 --2- 172.21.15.96:0/2971052294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b042af36b0 0x55b042ebfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:11.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.593+0000 7fd43d909640 1 -- 172.21.15.96:0/2971052294 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b042bf67d0 con 0x55b042b14390 2024-04-25T14:14:11.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.593+0000 7fd43d909640 1 --2- 172.21.15.96:0/2971052294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b042b14390 0x55b042ec6d70 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd4300a3880 tx=0x7fd4300a7210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22251f75ddd4dec9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:11.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.593+0000 7fd420ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:14:11.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.593+0000 7fd420ff9640 1 -- 172.21.15.96:0/2971052294 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4300c1060 con 0x55b042b14390 2024-04-25T14:14:11.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.593+0000 7fd420ff9640 1 -- 172.21.15.96:0/2971052294 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fd430005310 con 0x55b042b14390 2024-04-25T14:14:11.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.593+0000 7fd44621b740 1 -- 172.21.15.96:0/2971052294 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd428010e40 con 0x55b042b14390 2024-04-25T14:14:11.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.593+0000 7fd420ff9640 1 -- 172.21.15.96:0/2971052294 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd4300a7800 con 0x55b042b14390 2024-04-25T14:14:11.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.593+0000 7fd44621b740 10 client.15684 fetch_fsmap learned FSMap version 26 2024-04-25T14:14:11.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.593+0000 7fd44621b740 1 -- 172.21.15.96:0/2971052294 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b042b2c3f0 con 0x55b042b14390 2024-04-25T14:14:11.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.593+0000 7fd420ff9640 1 -- 172.21.15.96:0/2971052294 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd428010e40 con 0x55b042b14390 2024-04-25T14:14:11.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.594+0000 7fd420ff9640 1 -- 172.21.15.96:0/2971052294 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd4300900b0 con 0x55b042b14390 2024-04-25T14:14:11.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.594+0000 7fd44621b740 10 client.15684 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:14:11.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.594+0000 7fd44621b740 10 client.15684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:14:11.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.594+0000 7fd44621b740 20 client.15684 populate_metadata read hostname 'smithi096' 2024-04-25T14:14:11.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.594+0000 7fd44621b740 1 --2- 172.21.15.96:0/2971052294 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55b042ad2f60 0x55b042ecd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:11.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.594+0000 7fd44621b740 4 client.15684 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:14:11.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.594+0000 7fd44621b740 1 -- 172.21.15.96:0/2971052294 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b042e28990 con 0x55b042ad2f60 2024-04-25T14:14:11.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.594+0000 7fd43d108640 1 --2- 172.21.15.96:0/2971052294 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55b042ad2f60 0x55b042ecd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:11.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.594+0000 7fd420ff9640 10 client.15684 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:11.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.594+0000 7fd43d108640 1 --2- 172.21.15.96:0/2971052294 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55b042ad2f60 0x55b042ecd3b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=715bf66d53b100ad server_cookie=82f8164ecf22f013 in_seq=0 out_seq=0 2024-04-25T14:14:11.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.595+0000 7fd420ff9640 1 -- 172.21.15.96:0/2971052294 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b042e28990 con 0x55b042ad2f60 2024-04-25T14:14:11.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.595+0000 7fd420ff9640 10 client.15684 handle_command_reply: tid=0 2024-04-25T14:14:11.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.595+0000 7fd44621b740 1 client.15684 shutdown 2024-04-25T14:14:11.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.596+0000 7fd44621b740 1 -- 172.21.15.96:0/2971052294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b042b14390 msgr2=0x55b042ec6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:11.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.596+0000 7fd44621b740 1 --2- 172.21.15.96:0/2971052294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b042b14390 0x55b042ec6d70 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd4300a3880 tx=0x7fd4300a7210 comp rx=0 tx=0).stop 2024-04-25T14:14:11.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.596+0000 7fd44621b740 1 -- 172.21.15.96:0/2971052294 shutdown_connections 2024-04-25T14:14:11.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.596+0000 7fd44621b740 1 --2- 172.21.15.96:0/2971052294 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55b042ad2f60 0x55b042ecd3b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:11.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.596+0000 7fd44621b740 1 --2- 172.21.15.96:0/2971052294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b042b14390 0x55b042ec6d70 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:11.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.596+0000 7fd44621b740 1 --2- 172.21.15.96:0/2971052294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b042af36b0 0x55b042ebfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:11.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.596+0000 7fd44621b740 1 --2- 172.21.15.96:0/2971052294 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b042aeb4d0 0x55b042ebf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:11.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.596+0000 7fd420ff9640 0 client.15684 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:11.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.596+0000 7fd44621b740 1 -- 172.21.15.96:0/2971052294 >> 172.21.15.96:0/2971052294 conn(0x55b042b084e0 msgr2=0x55b042eb4180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:11.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.596+0000 7fd44621b740 1 -- 172.21.15.96:0/2971052294 shutdown_connections 2024-04-25T14:14:11.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.596+0000 7fd44621b740 1 -- 172.21.15.96:0/2971052294 wait complete. 2024-04-25T14:14:11.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.596+0000 7fd44621b740 20 client.15684 trim_cache size 0 max 16384 2024-04-25T14:14:11.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.614+0000 7fd44621b740 1 Processor -- start 2024-04-25T14:14:11.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.614+0000 7fd44621b740 1 -- start start 2024-04-25T14:14:11.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.614+0000 7fd44621b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b042aeb4d0 0x55b042eb1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:11.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.614+0000 7fd44621b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b042af36b0 0x55b042ebdf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:11.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.614+0000 7fd44621b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b042b14390 0x55b042ebe440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:11.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.614+0000 7fd44621b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd4381ab390 con 0x55b042b14390 2024-04-25T14:14:11.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.614+0000 7fd44621b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd4381aced0 con 0x55b042aeb4d0 2024-04-25T14:14:11.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.614+0000 7fd44621b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd4381aba60 con 0x55b042af36b0 2024-04-25T14:14:11.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.614+0000 7fd43d909640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b042b14390 0x55b042ebe440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:11.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.614+0000 7fd43d909640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b042b14390 0x55b042ebe440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58532/0 (socket says 172.21.15.96:58532) 2024-04-25T14:14:11.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.614+0000 7fd43d909640 1 -- 172.21.15.96:0/1238937725 learned_addr learned my addr 172.21.15.96:0/1238937725 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:11.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.614+0000 7fd43d108640 1 --2- 172.21.15.96:0/1238937725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b042aeb4d0 0x55b042eb1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:11.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.614+0000 7fd43c907640 1 --2- 172.21.15.96:0/1238937725 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b042af36b0 0x55b042ebdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:11.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.615+0000 7fd43d909640 1 -- 172.21.15.96:0/1238937725 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b042af36b0 msgr2=0x55b042ebdf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:11.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.615+0000 7fd43d909640 1 --2- 172.21.15.96:0/1238937725 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b042af36b0 0x55b042ebdf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:11.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.615+0000 7fd43d909640 1 -- 172.21.15.96:0/1238937725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b042aeb4d0 msgr2=0x55b042eb1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:11.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.615+0000 7fd43d909640 1 --2- 172.21.15.96:0/1238937725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b042aeb4d0 0x55b042eb1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:11.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.615+0000 7fd43d909640 1 -- 172.21.15.96:0/1238937725 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b042aec0f0 con 0x55b042b14390 2024-04-25T14:14:11.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.615+0000 7fd43d909640 1 --2- 172.21.15.96:0/1238937725 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b042b14390 0x55b042ebe440 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fd4300dce70 tx=0x7fd430005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c40c708989323dd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:11.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.615+0000 7fd422ffd640 1 -- 172.21.15.96:0/1238937725 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4300a36c0 con 0x55b042b14390 2024-04-25T14:14:11.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.615+0000 7fd422ffd640 1 -- 172.21.15.96:0/1238937725 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4300abbb0 con 0x55b042b14390 2024-04-25T14:14:11.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.615+0000 7fd422ffd640 1 -- 172.21.15.96:0/1238937725 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd430090230 con 0x55b042b14390 2024-04-25T14:14:11.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.616+0000 7fd44621b740 1 -- 172.21.15.96:0/1238937725 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b042b14390 msgr2=0x55b042ebe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:11.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.616+0000 7fd44621b740 1 --2- 172.21.15.96:0/1238937725 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b042b14390 0x55b042ebe440 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fd4300dce70 tx=0x7fd430005730 comp rx=0 tx=0).stop 2024-04-25T14:14:11.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.616+0000 7fd44621b740 1 -- 172.21.15.96:0/1238937725 shutdown_connections 2024-04-25T14:14:11.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.616+0000 7fd44621b740 1 --2- 172.21.15.96:0/1238937725 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b042b14390 0x55b042ebe440 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:11.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.616+0000 7fd44621b740 1 --2- 172.21.15.96:0/1238937725 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b042af36b0 0x55b042ebdf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:11.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.616+0000 7fd44621b740 1 --2- 172.21.15.96:0/1238937725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b042aeb4d0 0x55b042eb1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:11.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.616+0000 7fd44621b740 1 -- 172.21.15.96:0/1238937725 >> 172.21.15.96:0/1238937725 conn(0x55b042b084e0 msgr2=0x55b042eb0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:11.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.616+0000 7fd44621b740 1 -- 172.21.15.96:0/1238937725 shutdown_connections 2024-04-25T14:14:11.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.616+0000 7fd44621b740 1 -- 172.21.15.96:0/1238937725 wait complete. 2024-04-25T14:14:11.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.616+0000 7fd44621b740 1 -- start start 2024-04-25T14:14:11.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.616+0000 7fd44621b740 1 Processor -- start 2024-04-25T14:14:11.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.616+0000 7fd44621b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b042aeb4d0 0x55b042ec3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:11.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.617+0000 7fd44621b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b042af36b0 0x55b042ec4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:11.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.617+0000 7fd44621b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b042b14390 0x55b042ec5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:11.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.617+0000 7fd44621b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd4381ac800 con 0x55b042aeb4d0 2024-04-25T14:14:11.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.617+0000 7fd44621b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd4381ac130 con 0x55b042b14390 2024-04-25T14:14:11.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.617+0000 7fd44621b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd4381a9370 con 0x55b042af36b0 2024-04-25T14:14:11.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.617+0000 7fd44621b740 10 client.0 _ll_register_callbacks cb 0x7fd430004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:14:11.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.617+0000 7fd43d108640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b042aeb4d0 0x55b042ec3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:11.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.617+0000 7fd43d108640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b042aeb4d0 0x55b042ec3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58534/0 (socket says 172.21.15.96:58534) 2024-04-25T14:14:11.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.617+0000 7fd43d108640 1 -- 172.21.15.96:0/643669488 learned_addr learned my addr 172.21.15.96:0/643669488 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:11.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.617+0000 7fd43d108640 1 -- 172.21.15.96:0/643669488 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b042af36b0 msgr2=0x55b042ec4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:11.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.617+0000 7fd43d909640 1 --2- 172.21.15.96:0/643669488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b042b14390 0x55b042ec5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:11.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.617+0000 7fd43d108640 1 --2- 172.21.15.96:0/643669488 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b042af36b0 0x55b042ec4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:11.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.617+0000 7fd43d108640 1 -- 172.21.15.96:0/643669488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b042b14390 msgr2=0x55b042ec5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:11.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.617+0000 7fd43d108640 1 --2- 172.21.15.96:0/643669488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b042b14390 0x55b042ec5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:11.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.617+0000 7fd43d108640 1 -- 172.21.15.96:0/643669488 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b042ddb4b0 con 0x55b042aeb4d0 2024-04-25T14:14:11.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.617+0000 7fd43d108640 1 --2- 172.21.15.96:0/643669488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b042aeb4d0 0x55b042ec3ff0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd434009900 tx=0x7fd4340041a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b20609c40cc0e274 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:11.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.617+0000 7fd420ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:14:11.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.617+0000 7fd420ff9640 1 -- 172.21.15.96:0/643669488 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd434035040 con 0x55b042aeb4d0 2024-04-25T14:14:11.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.617+0000 7fd44621b740 1 -- 172.21.15.96:0/643669488 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4300abbb0 con 0x55b042aeb4d0 2024-04-25T14:14:11.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.617+0000 7fd420ff9640 1 -- 172.21.15.96:0/643669488 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fd434005240 con 0x55b042aeb4d0 2024-04-25T14:14:11.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.617+0000 7fd420ff9640 1 -- 172.21.15.96:0/643669488 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd434052050 con 0x55b042aeb4d0 2024-04-25T14:14:11.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.617+0000 7fd44621b740 10 client.15690 fetch_fsmap learned FSMap version 26 2024-04-25T14:14:11.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.617+0000 7fd44621b740 1 -- 172.21.15.96:0/643669488 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b042f359d0 con 0x55b042aeb4d0 2024-04-25T14:14:11.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.618+0000 7fd420ff9640 1 -- 172.21.15.96:0/643669488 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4300abbb0 con 0x55b042aeb4d0 2024-04-25T14:14:11.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.618+0000 7fd420ff9640 1 -- 172.21.15.96:0/643669488 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd434044320 con 0x55b042aeb4d0 2024-04-25T14:14:11.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.618+0000 7fd44621b740 10 client.15690 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:14:11.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.618+0000 7fd44621b740 10 client.15690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:14:11.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.618+0000 7fd44621b740 20 client.15690 populate_metadata read hostname 'smithi096' 2024-04-25T14:14:11.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.618+0000 7fd44621b740 1 --2- 172.21.15.96:0/643669488 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55b042aa83a0 0x55b042f383e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:11.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.618+0000 7fd44621b740 4 client.15690 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:14:11.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.618+0000 7fd44621b740 1 -- 172.21.15.96:0/643669488 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b042e22800 con 0x55b042aa83a0 2024-04-25T14:14:11.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.618+0000 7fd43c907640 1 --2- 172.21.15.96:0/643669488 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55b042aa83a0 0x55b042f383e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:11.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.619+0000 7fd420ff9640 10 client.15690 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:11.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.619+0000 7fd43c907640 1 --2- 172.21.15.96:0/643669488 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55b042aa83a0 0x55b042f383e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af5a0c04c430c981 server_cookie=8ade03c7401e4682 in_seq=0 out_seq=0 2024-04-25T14:14:11.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.619+0000 7fd420ff9640 1 -- 172.21.15.96:0/643669488 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b042e22800 con 0x55b042aa83a0 2024-04-25T14:14:11.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.619+0000 7fd420ff9640 10 client.15690 handle_command_reply: tid=0 2024-04-25T14:14:11.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.619+0000 7fd44621b740 1 client.15690 shutdown 2024-04-25T14:14:11.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.619+0000 7fd44621b740 1 -- 172.21.15.96:0/643669488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b042aeb4d0 msgr2=0x55b042ec3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:11.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.619+0000 7fd44621b740 1 --2- 172.21.15.96:0/643669488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b042aeb4d0 0x55b042ec3ff0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd434009900 tx=0x7fd4340041a0 comp rx=0 tx=0).stop 2024-04-25T14:14:11.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.619+0000 7fd44621b740 1 -- 172.21.15.96:0/643669488 shutdown_connections 2024-04-25T14:14:11.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.619+0000 7fd44621b740 1 --2- 172.21.15.96:0/643669488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b042b14390 0x55b042ec5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:11.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.619+0000 7fd44621b740 1 --2- 172.21.15.96:0/643669488 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b042af36b0 0x55b042ec4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:11.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.619+0000 7fd44621b740 1 --2- 172.21.15.96:0/643669488 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55b042aa83a0 0x55b042f383e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:11.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.619+0000 7fd44621b740 1 --2- 172.21.15.96:0/643669488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b042aeb4d0 0x55b042ec3ff0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:11.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.619+0000 7fd420ff9640 0 client.15690 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:11.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.619+0000 7fd44621b740 1 -- 172.21.15.96:0/643669488 >> 172.21.15.96:0/643669488 conn(0x55b042b084e0 msgr2=0x55b042ebdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:11.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.619+0000 7fd44621b740 1 -- 172.21.15.96:0/643669488 shutdown_connections 2024-04-25T14:14:11.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.619+0000 7fd44621b740 1 -- 172.21.15.96:0/643669488 wait complete. 2024-04-25T14:14:11.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.619+0000 7fd44621b740 20 client.15690 trim_cache size 0 max 16384 2024-04-25T14:14:11.626 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T14:14:11.627 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "no active scrubs running", 2024-04-25T14:14:11.627 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": {} 2024-04-25T14:14:11.627 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T14:14:11.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.620+0000 7fd43f36c640 1 -- 172.21.15.96:0/2603553965 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd4240afaf0 msgr2=0x7fd4240b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:11.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.620+0000 7fd43f36c640 1 --2- 172.21.15.96:0/2603553965 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd4240afaf0 0x7fd4240b2000 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd428020000 tx=0x7fd42801d000 comp rx=0 tx=0).stop 2024-04-25T14:14:11.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.620+0000 7fd43f36c640 1 -- 172.21.15.96:0/2603553965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd438108930 msgr2=0x7fd4381a1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:11.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.620+0000 7fd43f36c640 1 --2- 172.21.15.96:0/2603553965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd438108930 0x7fd4381a1740 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd43000eb10 tx=0x7fd43000c2d0 comp rx=0 tx=0).stop 2024-04-25T14:14:11.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.620+0000 7fd43f36c640 1 -- 172.21.15.96:0/2603553965 shutdown_connections 2024-04-25T14:14:11.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.620+0000 7fd43f36c640 1 --2- 172.21.15.96:0/2603553965 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd4240afaf0 0x7fd4240b2000 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:11.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.620+0000 7fd43f36c640 1 --2- 172.21.15.96:0/2603553965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd438108930 0x7fd4381a1740 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:11.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.620+0000 7fd43f36c640 1 --2- 172.21.15.96:0/2603553965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd438103c30 0x7fd43819a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:11.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.620+0000 7fd43f36c640 1 --2- 172.21.15.96:0/2603553965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4381029f0 0x7fd43819a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:11.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.620+0000 7fd43f36c640 1 -- 172.21.15.96:0/2603553965 >> 172.21.15.96:0/2603553965 conn(0x7fd4380fdbc0 msgr2=0x7fd438104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:11.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.620+0000 7fd43f36c640 1 -- 172.21.15.96:0/2603553965 shutdown_connections 2024-04-25T14:14:11.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.620+0000 7fd43f36c640 1 -- 172.21.15.96:0/2603553965 wait complete. 2024-04-25T14:14:11.635 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T14:14:11.636 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:14:11.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.746+0000 7fd37a834640 1 -- 172.21.15.96:0/3501837861 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd374103ef0 msgr2=0x7fd3741027d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:11.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.746+0000 7fd37a834640 1 --2- 172.21.15.96:0/3501837861 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd374103ef0 0x7fd3741027d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fd364009900 tx=0x7fd36402a410 comp rx=0 tx=0).stop 2024-04-25T14:14:11.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.746+0000 7fd37a834640 1 -- 172.21.15.96:0/3501837861 shutdown_connections 2024-04-25T14:14:11.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.746+0000 7fd37a834640 1 --2- 172.21.15.96:0/3501837861 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd374108a20 0x7fd37410ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:11.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.746+0000 7fd37a834640 1 --2- 172.21.15.96:0/3501837861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3741043d0 0x7fd374102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:11.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.746+0000 7fd37a834640 1 --2- 172.21.15.96:0/3501837861 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd374103ef0 0x7fd3741027d0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:11.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.746+0000 7fd37a834640 1 -- 172.21.15.96:0/3501837861 >> 172.21.15.96:0/3501837861 conn(0x7fd3740fd370 msgr2=0x7fd3740ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:11.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.746+0000 7fd37a834640 1 -- 172.21.15.96:0/3501837861 shutdown_connections 2024-04-25T14:14:11.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.746+0000 7fd37a834640 1 -- 172.21.15.96:0/3501837861 wait complete. 2024-04-25T14:14:11.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.746+0000 7fd37a834640 1 Processor -- start 2024-04-25T14:14:11.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.746+0000 7fd37a834640 1 -- start start 2024-04-25T14:14:11.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.747+0000 7fd37a834640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd374103ef0 0x7fd374102130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:11.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.747+0000 7fd37a834640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd374108a20 0x7fd37419ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:11.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.747+0000 7fd37a834640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd37419f100 0x7fd3741a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:11.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.747+0000 7fd37a834640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd37410e500 con 0x7fd374108a20 2024-04-25T14:14:11.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.747+0000 7fd37a834640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd37410e380 con 0x7fd37419f100 2024-04-25T14:14:11.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.747+0000 7fd37a834640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd37410e680 con 0x7fd374103ef0 2024-04-25T14:14:11.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.747+0000 7fd3737fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd374108a20 0x7fd37419ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:11.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.747+0000 7fd3737fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd374108a20 0x7fd37419ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58538/0 (socket says 172.21.15.96:58538) 2024-04-25T14:14:11.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.747+0000 7fd3737fe640 1 -- 172.21.15.96:0/313502187 learned_addr learned my addr 172.21.15.96:0/313502187 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:11.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.747+0000 7fd373fff640 1 --2- 172.21.15.96:0/313502187 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd374103ef0 0x7fd374102130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:11.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.747+0000 7fd378dd1640 1 --2- 172.21.15.96:0/313502187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd37419f100 0x7fd3741a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:11.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.747+0000 7fd3737fe640 1 -- 172.21.15.96:0/313502187 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd374103ef0 msgr2=0x7fd374102130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:11.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.747+0000 7fd3737fe640 1 --2- 172.21.15.96:0/313502187 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd374103ef0 0x7fd374102130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:11.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.747+0000 7fd3737fe640 1 -- 172.21.15.96:0/313502187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd37419f100 msgr2=0x7fd3741a1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:11.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.747+0000 7fd3737fe640 1 --2- 172.21.15.96:0/313502187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd37419f100 0x7fd3741a1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:11.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.747+0000 7fd3737fe640 1 -- 172.21.15.96:0/313502187 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3741a1d40 con 0x7fd374108a20 2024-04-25T14:14:11.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.747+0000 7fd373fff640 1 --2- 172.21.15.96:0/313502187 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd374103ef0 0x7fd374102130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:14:11.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.747+0000 7fd3737fe640 1 --2- 172.21.15.96:0/313502187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd374108a20 0x7fd37419ebc0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fd36800e9f0 tx=0x7fd36800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:11.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.747+0000 7fd3717fa640 1 -- 172.21.15.96:0/313502187 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd36800cd80 con 0x7fd374108a20 2024-04-25T14:14:11.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.747+0000 7fd3717fa640 1 -- 172.21.15.96:0/313502187 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3680108e0 con 0x7fd374108a20 2024-04-25T14:14:11.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.747+0000 7fd37a834640 1 -- 172.21.15.96:0/313502187 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3741a1fd0 con 0x7fd374108a20 2024-04-25T14:14:11.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.747+0000 7fd3717fa640 1 -- 172.21.15.96:0/313502187 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd368018980 con 0x7fd374108a20 2024-04-25T14:14:11.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.748+0000 7fd37a834640 1 -- 172.21.15.96:0/313502187 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3741a2510 con 0x7fd374108a20 2024-04-25T14:14:11.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.749+0000 7fd37a834640 1 -- 172.21.15.96:0/313502187 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd374107120 con 0x7fd374108a20 2024-04-25T14:14:11.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.750+0000 7fd3717fa640 1 -- 172.21.15.96:0/313502187 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fd368010430 con 0x7fd374108a20 2024-04-25T14:14:11.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.750+0000 7fd3717fa640 1 --2- 172.21.15.96:0/313502187 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd35c0afb70 0x7fd35c0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:11.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.751+0000 7fd373fff640 1 --2- 172.21.15.96:0/313502187 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd35c0afb70 0x7fd35c0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:11.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.751+0000 7fd3717fa640 1 -- 172.21.15.96:0/313502187 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fd3680df020 con 0x7fd374108a20 2024-04-25T14:14:11.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.751+0000 7fd373fff640 1 --2- 172.21.15.96:0/313502187 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd35c0afb70 0x7fd35c0b2080 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd364000c00 tx=0x7fd3640314c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:11.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.755+0000 7fd3717fa640 1 -- 172.21.15.96:0/313502187 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd3680a7be0 con 0x7fd374108a20 2024-04-25T14:14:11.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.972+0000 7fd37a834640 1 -- 172.21.15.96:0/313502187 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd37402db50 con 0x7fd374108a20 2024-04-25T14:14:11.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.973+0000 7fd3717fa640 1 -- 172.21.15.96:0/313502187 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11927 (secure 0 0 0) 0x7fd3680a74e0 con 0x7fd374108a20 2024-04-25T14:14:11.973 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:14:11.973 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/606227917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":606227917},{"type":"v1","addr":"172.21.15.96:6835","nonce":606227917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/3265300710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3265300710},{"type":"v1","addr":"172.21.15.96:6837","nonce":3265300710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T14:11:28.886961+0000","modified":"2024-04-25T14:13:32.120297+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24517,"mds_2":24478,"mds_3":24455,"mds_4":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/4080308509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":4080308509},{"type":"v1","addr":"172.21.15.96:6839","nonce":4080308509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/1606748971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":1606748971},{"type":"v1","addr":"172.21.15.96:6841","nonce":1606748971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6833/1995324831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1995324831},{"type":"v1","addr":"172.21.15.97:6833","nonce":1995324831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":4,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.151:6833/1382721608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1382721608},{"type":"v1","addr":"172.21.15.151:6833","nonce":1382721608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6835/3872301726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3872301726},{"type":"v1","addr":"172.21.15.97:6835","nonce":3872301726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.151:6835/1313823638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":1313823638},{"type":"v1","addr":"172.21.15.151:6835","nonce":1313823638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/2691348366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2691348366},{"type":"v1","addr":"172.21.15.151:6837","nonce":2691348366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3306780081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3306780081},{"type":"v1","addr":"172.21.15.97:6837","nonce":3306780081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.151:6839/245699981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":245699981},{"type":"v1","addr":"172.21.15.151:6839","nonce":245699981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6839/2850442269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":2850442269},{"type":"v1","addr":"172.21.15.97:6839","nonce":2850442269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24427,24455,24478,24517,24536]},"id":1}]} 2024-04-25T14:14:11.974 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 26 2024-04-25T14:14:11.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.975+0000 7fd37a834640 1 -- 172.21.15.96:0/313502187 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd35c0afb70 msgr2=0x7fd35c0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:11.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.975+0000 7fd37a834640 1 --2- 172.21.15.96:0/313502187 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd35c0afb70 0x7fd35c0b2080 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd364000c00 tx=0x7fd3640314c0 comp rx=0 tx=0).stop 2024-04-25T14:14:11.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.975+0000 7fd37a834640 1 -- 172.21.15.96:0/313502187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd374108a20 msgr2=0x7fd37419ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:11.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.975+0000 7fd37a834640 1 --2- 172.21.15.96:0/313502187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd374108a20 0x7fd37419ebc0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fd36800e9f0 tx=0x7fd36800eeb0 comp rx=0 tx=0).stop 2024-04-25T14:14:11.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.975+0000 7fd37a834640 1 -- 172.21.15.96:0/313502187 shutdown_connections 2024-04-25T14:14:11.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.975+0000 7fd37a834640 1 --2- 172.21.15.96:0/313502187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd37419f100 0x7fd3741a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:11.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.976+0000 7fd37a834640 1 --2- 172.21.15.96:0/313502187 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd35c0afb70 0x7fd35c0b2080 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:11.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.976+0000 7fd37a834640 1 --2- 172.21.15.96:0/313502187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd374108a20 0x7fd37419ebc0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:11.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.976+0000 7fd37a834640 1 --2- 172.21.15.96:0/313502187 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd374103ef0 0x7fd374102130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:11.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.976+0000 7fd37a834640 1 -- 172.21.15.96:0/313502187 >> 172.21.15.96:0/313502187 conn(0x7fd3740fd370 msgr2=0x7fd3740ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:11.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.976+0000 7fd37a834640 1 -- 172.21.15.96:0/313502187 shutdown_connections 2024-04-25T14:14:11.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:11.976+0000 7fd37a834640 1 -- 172.21.15.96:0/313502187 wait complete. 2024-04-25T14:14:11.987 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T14:14:12.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.096+0000 7fbf6e061640 1 -- 172.21.15.96:0/3901195356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf68108930 msgr2=0x7fbf6810ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.096+0000 7fbf6e061640 1 --2- 172.21.15.96:0/3901195356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf68108930 0x7fbf6810ad50 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fbf60009ca0 tx=0x7fbf6002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.097+0000 7fbf6e061640 1 -- 172.21.15.96:0/3901195356 shutdown_connections 2024-04-25T14:14:12.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.097+0000 7fbf6e061640 1 --2- 172.21.15.96:0/3901195356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf68108930 0x7fbf6810ad50 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.097+0000 7fbf6e061640 1 --2- 172.21.15.96:0/3901195356 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf68103c30 0x7fbf68104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.097+0000 7fbf6e061640 1 --2- 172.21.15.96:0/3901195356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf681029f0 0x7fbf681036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.097+0000 7fbf6e061640 1 -- 172.21.15.96:0/3901195356 >> 172.21.15.96:0/3901195356 conn(0x7fbf680fdbc0 msgr2=0x7fbf68100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:12.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.097+0000 7fbf6e061640 1 -- 172.21.15.96:0/3901195356 shutdown_connections 2024-04-25T14:14:12.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.097+0000 7fbf6e061640 1 -- 172.21.15.96:0/3901195356 wait complete. 2024-04-25T14:14:12.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.097+0000 7fbf6e061640 1 Processor -- start 2024-04-25T14:14:12.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.097+0000 7fbf6e061640 1 -- start start 2024-04-25T14:14:12.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.098+0000 7fbf6e061640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf681029f0 0x7fbf68199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.098+0000 7fbf6e061640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf68103c30 0x7fbf68199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.098+0000 7fbf6e061640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf68108930 0x7fbf681a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.098+0000 7fbf6e061640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6810e520 con 0x7fbf68108930 2024-04-25T14:14:12.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.098+0000 7fbf6e061640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6810e3a0 con 0x7fbf681029f0 2024-04-25T14:14:12.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.098+0000 7fbf6e061640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6810e6a0 con 0x7fbf68103c30 2024-04-25T14:14:12.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.098+0000 7fbf67fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf68108930 0x7fbf681a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.098+0000 7fbf67fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf68108930 0x7fbf681a0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58560/0 (socket says 172.21.15.96:58560) 2024-04-25T14:14:12.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.098+0000 7fbf67fff640 1 -- 172.21.15.96:0/3541912186 learned_addr learned my addr 172.21.15.96:0/3541912186 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:12.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.098+0000 7fbf677fe640 1 --2- 172.21.15.96:0/3541912186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf681029f0 0x7fbf68199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.098+0000 7fbf66ffd640 1 --2- 172.21.15.96:0/3541912186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf68103c30 0x7fbf68199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.098+0000 7fbf67fff640 1 -- 172.21.15.96:0/3541912186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf68103c30 msgr2=0x7fbf68199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.098+0000 7fbf67fff640 1 --2- 172.21.15.96:0/3541912186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf68103c30 0x7fbf68199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.098+0000 7fbf67fff640 1 -- 172.21.15.96:0/3541912186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf681029f0 msgr2=0x7fbf68199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.098+0000 7fbf67fff640 1 --2- 172.21.15.96:0/3541912186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf681029f0 0x7fbf68199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.098+0000 7fbf67fff640 1 -- 172.21.15.96:0/3541912186 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf681a0c00 con 0x7fbf68108930 2024-04-25T14:14:12.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.098+0000 7fbf67fff640 1 --2- 172.21.15.96:0/3541912186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf68108930 0x7fbf681a0590 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fbf600097b0 tx=0x7fbf60031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:12.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.098+0000 7fbf64ff9640 1 -- 172.21.15.96:0/3541912186 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf60003bf0 con 0x7fbf68108930 2024-04-25T14:14:12.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.098+0000 7fbf64ff9640 1 -- 172.21.15.96:0/3541912186 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf60003d90 con 0x7fbf68108930 2024-04-25T14:14:12.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.098+0000 7fbf64ff9640 1 -- 172.21.15.96:0/3541912186 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6002fc30 con 0x7fbf68108930 2024-04-25T14:14:12.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.098+0000 7fbf6e061640 1 -- 172.21.15.96:0/3541912186 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf681a0e90 con 0x7fbf68108930 2024-04-25T14:14:12.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.099+0000 7fbf6e061640 1 -- 172.21.15.96:0/3541912186 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf681a12f0 con 0x7fbf68108930 2024-04-25T14:14:12.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.101+0000 7fbf74fef740 1 Processor -- start 2024-04-25T14:14:12.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.101+0000 7fbf64ff9640 1 -- 172.21.15.96:0/3541912186 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fbf600305c0 con 0x7fbf68108930 2024-04-25T14:14:12.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.101+0000 7fbf74fef740 1 -- start start 2024-04-25T14:14:12.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.101+0000 7fbf74fef740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630c061d390 0x5630c09c1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.101+0000 7fbf74fef740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630c05f44d0 0x5630c09c1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.101+0000 7fbf74fef740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630c05fc6b0 0x5630c09c2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.101+0000 7fbf74fef740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5630c0923d80 con 0x5630c05f44d0 2024-04-25T14:14:12.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.101+0000 7fbf74fef740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5630c08dda20 con 0x5630c05fc6b0 2024-04-25T14:14:12.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.101+0000 7fbf74fef740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5630c091fd40 con 0x5630c061d390 2024-04-25T14:14:12.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.102+0000 7fbf677fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630c05f44d0 0x5630c09c1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.102+0000 7fbf677fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630c05f44d0 0x5630c09c1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58576/0 (socket says 172.21.15.96:58576) 2024-04-25T14:14:12.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.102+0000 7fbf677fe640 1 -- 172.21.15.96:0/3828582106 learned_addr learned my addr 172.21.15.96:0/3828582106 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:12.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.102+0000 7fbf66ffd640 1 --2- 172.21.15.96:0/3828582106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630c05fc6b0 0x5630c09c2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.102+0000 7fbf66ffd640 1 --2- 172.21.15.96:0/3828582106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630c061d390 0x5630c09c1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.102+0000 7fbf64ff9640 1 --2- 172.21.15.96:0/3541912186 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fbf500afaf0 0x7fbf500b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.102+0000 7fbf677fe640 1 -- 172.21.15.96:0/3828582106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630c061d390 msgr2=0x5630c09c1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:12.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.102+0000 7fbf677fe640 1 --2- 172.21.15.96:0/3828582106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630c061d390 0x5630c09c1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.102+0000 7fbf677fe640 1 -- 172.21.15.96:0/3828582106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630c05fc6b0 msgr2=0x5630c09c2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:12.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.102+0000 7fbf677fe640 1 --2- 172.21.15.96:0/3828582106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630c05fc6b0 0x5630c09c2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.102+0000 7fbf677fe640 1 -- 172.21.15.96:0/3828582106 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630c08e8c30 con 0x5630c05f44d0 2024-04-25T14:14:12.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.102+0000 7fbf66ffd640 1 --2- 172.21.15.96:0/3828582106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630c05fc6b0 0x5630c09c2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:14:12.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.102+0000 7fbf67fff640 1 --2- 172.21.15.96:0/3541912186 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fbf500afaf0 0x7fbf500b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.102+0000 7fbf64ff9640 1 -- 172.21.15.96:0/3541912186 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fbf601002f0 con 0x7fbf68108930 2024-04-25T14:14:12.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.102+0000 7fbf677fe640 1 --2- 172.21.15.96:0/3828582106 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630c05f44d0 0x5630c09c1fa0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fbf58009c60 tx=0x7fbf58011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7f6395e0c8a68a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:12.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.102+0000 7fbf4dffb640 1 -- 172.21.15.96:0/3828582106 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf58004220 con 0x5630c05f44d0 2024-04-25T14:14:12.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.102+0000 7fbf4dffb640 1 -- 172.21.15.96:0/3828582106 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf580043c0 con 0x5630c05f44d0 2024-04-25T14:14:12.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.102+0000 7fbf4dffb640 1 -- 172.21.15.96:0/3828582106 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf58004ce0 con 0x5630c05f44d0 2024-04-25T14:14:12.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.103+0000 7fbf74fef740 1 -- 172.21.15.96:0/3828582106 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630c05f44d0 msgr2=0x5630c09c1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.103+0000 7fbf67fff640 1 --2- 172.21.15.96:0/3541912186 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fbf500afaf0 0x7fbf500b2000 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbf600c8740 tx=0x7fbf600b1040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:12.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.103+0000 7fbf74fef740 1 --2- 172.21.15.96:0/3828582106 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630c05f44d0 0x5630c09c1fa0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fbf58009c60 tx=0x7fbf58011810 comp rx=0 tx=0).stop 2024-04-25T14:14:12.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.103+0000 7fbf74fef740 1 -- 172.21.15.96:0/3828582106 shutdown_connections 2024-04-25T14:14:12.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.103+0000 7fbf74fef740 1 --2- 172.21.15.96:0/3828582106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630c05fc6b0 0x5630c09c2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.103+0000 7fbf74fef740 1 --2- 172.21.15.96:0/3828582106 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630c05f44d0 0x5630c09c1fa0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.103+0000 7fbf74fef740 1 --2- 172.21.15.96:0/3828582106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630c061d390 0x5630c09c1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.103+0000 7fbf74fef740 1 -- 172.21.15.96:0/3828582106 >> 172.21.15.96:0/3828582106 conn(0x5630c06114e0 msgr2=0x5630c09bb020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:12.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.103+0000 7fbf74fef740 1 -- 172.21.15.96:0/3828582106 shutdown_connections 2024-04-25T14:14:12.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.103+0000 7fbf74fef740 1 -- 172.21.15.96:0/3828582106 wait complete. 2024-04-25T14:14:12.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.104+0000 7fbf74fef740 1 -- start start 2024-04-25T14:14:12.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.104+0000 7fbf74fef740 1 Processor -- start 2024-04-25T14:14:12.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.104+0000 7fbf74fef740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630c05f44d0 0x5630c09cb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.104+0000 7fbf74fef740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630c05fc6b0 0x5630c09cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.104+0000 7fbf74fef740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630c061d390 0x5630c09d2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.104+0000 7fbf74fef740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5630c09c3740 con 0x5630c05f44d0 2024-04-25T14:14:12.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.104+0000 7fbf74fef740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5630c08aadc0 con 0x5630c05fc6b0 2024-04-25T14:14:12.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.104+0000 7fbf74fef740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5630c09c38c0 con 0x5630c061d390 2024-04-25T14:14:12.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.104+0000 7fbf74fef740 10 client.0 _ll_register_callbacks cb 0x5630c08c6600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:14:12.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.104+0000 7fbf66ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630c05f44d0 0x5630c09cb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.104+0000 7fbf66ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630c05f44d0 0x5630c09cb710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58592/0 (socket says 172.21.15.96:58592) 2024-04-25T14:14:12.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.104+0000 7fbf66ffd640 1 -- 172.21.15.96:0/1427182635 learned_addr learned my addr 172.21.15.96:0/1427182635 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:12.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.104+0000 7fbf677fe640 1 --2- 172.21.15.96:0/1427182635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630c05fc6b0 0x5630c09cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.104+0000 7fbf66ffd640 1 --2- 172.21.15.96:0/1427182635 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630c061d390 0x5630c09d2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.105+0000 7fbf66ffd640 1 -- 172.21.15.96:0/1427182635 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630c061d390 msgr2=0x5630c09d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:12.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.105+0000 7fbf66ffd640 1 --2- 172.21.15.96:0/1427182635 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630c061d390 0x5630c09d2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.105+0000 7fbf66ffd640 1 -- 172.21.15.96:0/1427182635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630c05fc6b0 msgr2=0x5630c09cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:12.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.105+0000 7fbf66ffd640 1 --2- 172.21.15.96:0/1427182635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630c05fc6b0 0x5630c09cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.105+0000 7fbf66ffd640 1 -- 172.21.15.96:0/1427182635 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630c06353f0 con 0x5630c05f44d0 2024-04-25T14:14:12.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.105+0000 7fbf66ffd640 1 --2- 172.21.15.96:0/1427182635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630c05f44d0 0x5630c09cb710 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c00cf90 tx=0x7fbf5c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e40ec3d8fa8063cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:12.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.105+0000 7fbf46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:14:12.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.105+0000 7fbf46ffd640 1 -- 172.21.15.96:0/1427182635 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c01c040 con 0x5630c05f44d0 2024-04-25T14:14:12.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.105+0000 7fbf74fef740 1 -- 172.21.15.96:0/1427182635 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf580043c0 con 0x5630c05f44d0 2024-04-25T14:14:12.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.105+0000 7fbf46ffd640 1 -- 172.21.15.96:0/1427182635 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fbf5c005220 con 0x5630c05f44d0 2024-04-25T14:14:12.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.105+0000 7fbf46ffd640 1 -- 172.21.15.96:0/1427182635 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbf5c028050 con 0x5630c05f44d0 2024-04-25T14:14:12.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.105+0000 7fbf74fef740 10 client.15708 fetch_fsmap learned FSMap version 26 2024-04-25T14:14:12.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.105+0000 7fbf74fef740 1 -- 172.21.15.96:0/1427182635 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5630c06ff7d0 con 0x5630c05f44d0 2024-04-25T14:14:12.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.105+0000 7fbf46ffd640 1 -- 172.21.15.96:0/1427182635 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf580043c0 con 0x5630c05f44d0 2024-04-25T14:14:12.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.105+0000 7fbf46ffd640 1 -- 172.21.15.96:0/1427182635 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbf5c0203d0 con 0x5630c05f44d0 2024-04-25T14:14:12.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.105+0000 7fbf74fef740 10 client.15708 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:14:12.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.105+0000 7fbf74fef740 10 client.15708 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-25T14:14:12.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.105+0000 7fbf74fef740 20 client.15708 populate_metadata read hostname 'smithi096' 2024-04-25T14:14:12.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.105+0000 7fbf74fef740 1 --2- 172.21.15.96:0/1427182635 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x5630c05dbf60 0x5630c09d9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.105+0000 7fbf74fef740 4 client.15708 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:14:12.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.105+0000 7fbf74fef740 1 -- 172.21.15.96:0/1427182635 --> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630c0931990 con 0x5630c05dbf60 2024-04-25T14:14:12.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.106+0000 7fbf677fe640 1 --2- 172.21.15.96:0/1427182635 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x5630c05dbf60 0x5630c09d9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.106+0000 7fbf46ffd640 10 client.15708 ms_handle_connect on v2:172.21.15.151:6832/1382721608 2024-04-25T14:14:12.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.106+0000 7fbf677fe640 1 --2- 172.21.15.96:0/1427182635 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x5630c05dbf60 0x5630c09d9400 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=da4d23b79396a9f4 server_cookie=44827a9a927798a in_seq=0 out_seq=0 2024-04-25T14:14:12.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.107+0000 7fbf46ffd640 1 -- 172.21.15.96:0/1427182635 <== mds.4 v2:172.21.15.151:6832/1382721608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5630c0931990 con 0x5630c05dbf60 2024-04-25T14:14:12.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.107+0000 7fbf46ffd640 10 client.15708 handle_command_reply: tid=0 2024-04-25T14:14:12.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.107+0000 7fbf74fef740 1 client.15708 shutdown 2024-04-25T14:14:12.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.107+0000 7fbf74fef740 1 -- 172.21.15.96:0/1427182635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630c05f44d0 msgr2=0x5630c09cb710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.107+0000 7fbf74fef740 1 --2- 172.21.15.96:0/1427182635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630c05f44d0 0x5630c09cb710 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c00cf90 tx=0x7fbf5c00cc90 comp rx=0 tx=0).stop 2024-04-25T14:14:12.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.107+0000 7fbf74fef740 1 -- 172.21.15.96:0/1427182635 shutdown_connections 2024-04-25T14:14:12.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.107+0000 7fbf74fef740 1 --2- 172.21.15.96:0/1427182635 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x5630c05dbf60 0x5630c09d9400 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.107+0000 7fbf74fef740 1 --2- 172.21.15.96:0/1427182635 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630c061d390 0x5630c09d2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.107+0000 7fbf74fef740 1 --2- 172.21.15.96:0/1427182635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630c05fc6b0 0x5630c09cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.107+0000 7fbf46ffd640 0 client.15708 ms_handle_reset on v2:172.21.15.151:6832/1382721608 2024-04-25T14:14:12.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.107+0000 7fbf74fef740 1 --2- 172.21.15.96:0/1427182635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630c05f44d0 0x5630c09cb710 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.107+0000 7fbf74fef740 1 -- 172.21.15.96:0/1427182635 >> 172.21.15.96:0/1427182635 conn(0x5630c06114e0 msgr2=0x5630c09bdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:12.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.107+0000 7fbf74fef740 1 -- 172.21.15.96:0/1427182635 shutdown_connections 2024-04-25T14:14:12.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.107+0000 7fbf74fef740 1 -- 172.21.15.96:0/1427182635 wait complete. 2024-04-25T14:14:12.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.108+0000 7fbf74fef740 20 client.15708 trim_cache size 0 max 16384 2024-04-25T14:14:12.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.121+0000 7fbf74fef740 1 Processor -- start 2024-04-25T14:14:12.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.121+0000 7fbf74fef740 1 -- start start 2024-04-25T14:14:12.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.121+0000 7fbf74fef740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630c05f44d0 0x5630c09cb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.121+0000 7fbf74fef740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630c05fc6b0 0x5630c09d2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.121+0000 7fbf74fef740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630c061d390 0x5630c09cb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.121+0000 7fbf74fef740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbf681a90a0 con 0x5630c061d390 2024-04-25T14:14:12.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.121+0000 7fbf74fef740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbf681aabe0 con 0x5630c05f44d0 2024-04-25T14:14:12.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.121+0000 7fbf74fef740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbf681a9770 con 0x5630c05fc6b0 2024-04-25T14:14:12.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.121+0000 7fbf66ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630c061d390 0x5630c09cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.121+0000 7fbf66ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630c061d390 0x5630c09cb820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58596/0 (socket says 172.21.15.96:58596) 2024-04-25T14:14:12.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.121+0000 7fbf66ffd640 1 -- 172.21.15.96:0/2612842440 learned_addr learned my addr 172.21.15.96:0/2612842440 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:12.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.121+0000 7fbf66ffd640 1 --2- 172.21.15.96:0/2612842440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630c05f44d0 0x5630c09cb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.121+0000 7fbf66ffd640 1 -- 172.21.15.96:0/2612842440 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630c05fc6b0 msgr2=0x5630c09d2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:12.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.121+0000 7fbf66ffd640 1 --2- 172.21.15.96:0/2612842440 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630c05fc6b0 0x5630c09d2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.121+0000 7fbf66ffd640 1 -- 172.21.15.96:0/2612842440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630c05f44d0 msgr2=0x5630c09cb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:12.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.121+0000 7fbf66ffd640 1 --2- 172.21.15.96:0/2612842440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630c05f44d0 0x5630c09cb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.121+0000 7fbf66ffd640 1 -- 172.21.15.96:0/2612842440 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630c09c6bd0 con 0x5630c061d390 2024-04-25T14:14:12.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.121+0000 7fbf66ffd640 1 --2- 172.21.15.96:0/2612842440 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630c061d390 0x5630c09cb820 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c011cc0 tx=0x7fbf5c014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd7af024ee6fcd5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:12.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.122+0000 7fbf4d7fa640 1 -- 172.21.15.96:0/2612842440 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c032030 con 0x5630c061d390 2024-04-25T14:14:12.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.122+0000 7fbf4d7fa640 1 -- 172.21.15.96:0/2612842440 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf5c016030 con 0x5630c061d390 2024-04-25T14:14:12.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.122+0000 7fbf4d7fa640 1 -- 172.21.15.96:0/2612842440 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c025030 con 0x5630c061d390 2024-04-25T14:14:12.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.122+0000 7fbf74fef740 1 -- 172.21.15.96:0/2612842440 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630c061d390 msgr2=0x5630c09cb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.122+0000 7fbf74fef740 1 --2- 172.21.15.96:0/2612842440 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630c061d390 0x5630c09cb820 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c011cc0 tx=0x7fbf5c014040 comp rx=0 tx=0).stop 2024-04-25T14:14:12.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.122+0000 7fbf74fef740 1 -- 172.21.15.96:0/2612842440 shutdown_connections 2024-04-25T14:14:12.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.122+0000 7fbf74fef740 1 --2- 172.21.15.96:0/2612842440 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630c061d390 0x5630c09cb820 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.122+0000 7fbf74fef740 1 --2- 172.21.15.96:0/2612842440 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630c05fc6b0 0x5630c09d2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.122+0000 7fbf74fef740 1 --2- 172.21.15.96:0/2612842440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630c05f44d0 0x5630c09cb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.122+0000 7fbf74fef740 1 -- 172.21.15.96:0/2612842440 >> 172.21.15.96:0/2612842440 conn(0x5630c06114e0 msgr2=0x5630c09c64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:12.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.122+0000 7fbf74fef740 1 -- 172.21.15.96:0/2612842440 shutdown_connections 2024-04-25T14:14:12.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.123+0000 7fbf74fef740 1 -- 172.21.15.96:0/2612842440 wait complete. 2024-04-25T14:14:12.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.123+0000 7fbf74fef740 1 -- start start 2024-04-25T14:14:12.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.123+0000 7fbf74fef740 1 Processor -- start 2024-04-25T14:14:12.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.123+0000 7fbf74fef740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630c05f44d0 0x5630c09cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.123+0000 7fbf74fef740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630c05fc6b0 0x5630c09d0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.123+0000 7fbf74fef740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630c061d390 0x5630c0a3f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.123+0000 7fbf74fef740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbf681aa510 con 0x5630c05f44d0 2024-04-25T14:14:12.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.123+0000 7fbf74fef740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbf681a9e40 con 0x5630c061d390 2024-04-25T14:14:12.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.123+0000 7fbf74fef740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbf681a7080 con 0x5630c05fc6b0 2024-04-25T14:14:12.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.123+0000 7fbf74fef740 10 client.0 _ll_register_callbacks cb 0x7fbf5c00cbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:14:12.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.123+0000 7fbf66ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630c05f44d0 0x5630c09cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.123+0000 7fbf66ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630c05f44d0 0x5630c09cfb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58600/0 (socket says 172.21.15.96:58600) 2024-04-25T14:14:12.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.123+0000 7fbf66ffd640 1 -- 172.21.15.96:0/2549975441 learned_addr learned my addr 172.21.15.96:0/2549975441 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:12.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.123+0000 7fbf677fe640 1 --2- 172.21.15.96:0/2549975441 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630c05fc6b0 0x5630c09d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.124+0000 7fbf66ffd640 1 --2- 172.21.15.96:0/2549975441 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630c061d390 0x5630c0a3f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.124+0000 7fbf66ffd640 1 -- 172.21.15.96:0/2549975441 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630c05fc6b0 msgr2=0x5630c09d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:12.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.124+0000 7fbf66ffd640 1 --2- 172.21.15.96:0/2549975441 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630c05fc6b0 0x5630c09d0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.124+0000 7fbf66ffd640 1 -- 172.21.15.96:0/2549975441 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630c061d390 msgr2=0x5630c0a3f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:12.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.124+0000 7fbf66ffd640 1 --2- 172.21.15.96:0/2549975441 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630c061d390 0x5630c0a3f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.124+0000 7fbf66ffd640 1 -- 172.21.15.96:0/2549975441 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630c0a3fca0 con 0x5630c05f44d0 2024-04-25T14:14:12.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.124+0000 7fbf677fe640 1 --2- 172.21.15.96:0/2549975441 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630c05fc6b0 0x5630c09d0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:14:12.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.124+0000 7fbf66ffd640 1 --2- 172.21.15.96:0/2549975441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630c05f44d0 0x5630c09cfb00 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c00d010 tx=0x7fbf5c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e20a25a508360ab1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:12.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.124+0000 7fbf46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:14:12.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.124+0000 7fbf46ffd640 1 -- 172.21.15.96:0/2549975441 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c025070 con 0x5630c05f44d0 2024-04-25T14:14:12.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.124+0000 7fbf46ffd640 1 -- 172.21.15.96:0/2549975441 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fbf5c020b90 con 0x5630c05f44d0 2024-04-25T14:14:12.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.124+0000 7fbf74fef740 1 -- 172.21.15.96:0/2549975441 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf5c016030 con 0x5630c05f44d0 2024-04-25T14:14:12.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.124+0000 7fbf46ffd640 1 -- 172.21.15.96:0/2549975441 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbf5c032050 con 0x5630c05f44d0 2024-04-25T14:14:12.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.124+0000 7fbf74fef740 10 client.15714 fetch_fsmap learned FSMap version 26 2024-04-25T14:14:12.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.124+0000 7fbf74fef740 1 -- 172.21.15.96:0/2549975441 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5630c0a40e60 con 0x5630c05f44d0 2024-04-25T14:14:12.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.124+0000 7fbf46ffd640 1 -- 172.21.15.96:0/2549975441 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf5c016030 con 0x5630c05f44d0 2024-04-25T14:14:12.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.124+0000 7fbf46ffd640 1 -- 172.21.15.96:0/2549975441 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbf5c021460 con 0x5630c05f44d0 2024-04-25T14:14:12.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.124+0000 7fbf74fef740 10 client.15714 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:14:12.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.124+0000 7fbf74fef740 10 client.15714 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-25T14:14:12.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.124+0000 7fbf74fef740 20 client.15714 populate_metadata read hostname 'smithi096' 2024-04-25T14:14:12.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.125+0000 7fbf74fef740 1 --2- 172.21.15.96:0/2549975441 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x5630c05b13a0 0x5630c0a43790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.125+0000 7fbf74fef740 4 client.15714 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:14:12.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.125+0000 7fbf74fef740 1 -- 172.21.15.96:0/2549975441 --> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5630c092b800 con 0x5630c05b13a0 2024-04-25T14:14:12.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.125+0000 7fbf677fe640 1 --2- 172.21.15.96:0/2549975441 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x5630c05b13a0 0x5630c0a43790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.125+0000 7fbf46ffd640 10 client.15714 ms_handle_connect on v2:172.21.15.151:6832/1382721608 2024-04-25T14:14:12.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.125+0000 7fbf677fe640 1 --2- 172.21.15.96:0/2549975441 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x5630c05b13a0 0x5630c0a43790 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=188ca99713a73f server_cookie=f99219c2e76e0d80 in_seq=0 out_seq=0 2024-04-25T14:14:12.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.125+0000 7fbf46ffd640 1 -- 172.21.15.96:0/2549975441 <== mds.4 v2:172.21.15.151:6832/1382721608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5630c092b800 con 0x5630c05b13a0 2024-04-25T14:14:12.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.125+0000 7fbf46ffd640 10 client.15714 handle_command_reply: tid=0 2024-04-25T14:14:12.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.125+0000 7fbf74fef740 1 client.15714 shutdown 2024-04-25T14:14:12.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.126+0000 7fbf74fef740 1 -- 172.21.15.96:0/2549975441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630c05f44d0 msgr2=0x5630c09cfb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.126+0000 7fbf74fef740 1 --2- 172.21.15.96:0/2549975441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630c05f44d0 0x5630c09cfb00 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c00d010 tx=0x7fbf5c00cf60 comp rx=0 tx=0).stop 2024-04-25T14:14:12.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.126+0000 7fbf74fef740 1 -- 172.21.15.96:0/2549975441 shutdown_connections 2024-04-25T14:14:12.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.126+0000 7fbf74fef740 1 --2- 172.21.15.96:0/2549975441 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x5630c05b13a0 0x5630c0a43790 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.126+0000 7fbf74fef740 1 --2- 172.21.15.96:0/2549975441 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630c061d390 0x5630c0a3f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.126+0000 7fbf46ffd640 0 client.15714 ms_handle_reset on v2:172.21.15.151:6832/1382721608 2024-04-25T14:14:12.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.126+0000 7fbf74fef740 1 --2- 172.21.15.96:0/2549975441 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630c05fc6b0 0x5630c09d0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.126+0000 7fbf74fef740 1 --2- 172.21.15.96:0/2549975441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630c05f44d0 0x5630c09cfb00 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.126+0000 7fbf74fef740 1 -- 172.21.15.96:0/2549975441 >> 172.21.15.96:0/2549975441 conn(0x5630c06114e0 msgr2=0x5630c09c64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:12.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.126+0000 7fbf74fef740 1 -- 172.21.15.96:0/2549975441 shutdown_connections 2024-04-25T14:14:12.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.126+0000 7fbf74fef740 1 -- 172.21.15.96:0/2549975441 wait complete. 2024-04-25T14:14:12.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.126+0000 7fbf74fef740 20 client.15714 trim_cache size 0 max 16384 2024-04-25T14:14:12.142 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-04-25T14:14:12.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.126+0000 7fbf6e061640 1 -- 172.21.15.96:0/3541912186 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fbf500afaf0 msgr2=0x7fbf500b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.126+0000 7fbf6e061640 1 --2- 172.21.15.96:0/3541912186 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fbf500afaf0 0x7fbf500b2000 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbf600c8740 tx=0x7fbf600b1040 comp rx=0 tx=0).stop 2024-04-25T14:14:12.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.126+0000 7fbf6e061640 1 -- 172.21.15.96:0/3541912186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf68108930 msgr2=0x7fbf681a0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.126+0000 7fbf6e061640 1 --2- 172.21.15.96:0/3541912186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf68108930 0x7fbf681a0590 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fbf600097b0 tx=0x7fbf60031840 comp rx=0 tx=0).stop 2024-04-25T14:14:12.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.126+0000 7fbf6e061640 1 -- 172.21.15.96:0/3541912186 shutdown_connections 2024-04-25T14:14:12.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.126+0000 7fbf6e061640 1 --2- 172.21.15.96:0/3541912186 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fbf500afaf0 0x7fbf500b2000 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.126+0000 7fbf6e061640 1 --2- 172.21.15.96:0/3541912186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf68108930 0x7fbf681a0590 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.126+0000 7fbf6e061640 1 --2- 172.21.15.96:0/3541912186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf68103c30 0x7fbf68199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.126+0000 7fbf6e061640 1 --2- 172.21.15.96:0/3541912186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf681029f0 0x7fbf68199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.126+0000 7fbf6e061640 1 -- 172.21.15.96:0/3541912186 >> 172.21.15.96:0/3541912186 conn(0x7fbf680fdbc0 msgr2=0x7fbf68104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:12.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.126+0000 7fbf6e061640 1 -- 172.21.15.96:0/3541912186 shutdown_connections 2024-04-25T14:14:12.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.126+0000 7fbf6e061640 1 -- 172.21.15.96:0/3541912186 wait complete. 2024-04-25T14:14:12.145 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T14:14:12.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.255+0000 7f45dead1640 1 -- 172.21.15.96:0/1816260063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45d81029e0 msgr2=0x7f45d81036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.255+0000 7f45dead1640 1 --2- 172.21.15.96:0/1816260063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45d81029e0 0x7f45d81036e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f45cc009900 tx=0x7f45cc02a410 comp rx=0 tx=0).stop 2024-04-25T14:14:12.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.255+0000 7f45dead1640 1 -- 172.21.15.96:0/1816260063 shutdown_connections 2024-04-25T14:14:12.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.255+0000 7f45dead1640 1 --2- 172.21.15.96:0/1816260063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45d8108920 0x7f45d810ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.255+0000 7f45dead1640 1 --2- 172.21.15.96:0/1816260063 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45d8103c20 0x7f45d8104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.255+0000 7f45dead1640 1 --2- 172.21.15.96:0/1816260063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45d81029e0 0x7f45d81036e0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.255+0000 7f45dead1640 1 -- 172.21.15.96:0/1816260063 >> 172.21.15.96:0/1816260063 conn(0x7f45d80fdbd0 msgr2=0x7f45d8100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:12.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.255+0000 7f45dead1640 1 -- 172.21.15.96:0/1816260063 shutdown_connections 2024-04-25T14:14:12.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.255+0000 7f45dead1640 1 -- 172.21.15.96:0/1816260063 wait complete. 2024-04-25T14:14:12.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.255+0000 7f45dead1640 1 Processor -- start 2024-04-25T14:14:12.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.255+0000 7f45dead1640 1 -- start start 2024-04-25T14:14:12.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.256+0000 7f45dead1640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45d81029e0 0x7f45d8199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.256+0000 7f45dead1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45d8103c20 0x7f45d8199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.256+0000 7f45dead1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45d8108920 0x7f45d81a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.256+0000 7f45dead1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f45d810e4e0 con 0x7f45d8103c20 2024-04-25T14:14:12.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.256+0000 7f45dead1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f45d810e360 con 0x7f45d8108920 2024-04-25T14:14:12.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.256+0000 7f45dead1640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f45d810e660 con 0x7f45d81029e0 2024-04-25T14:14:12.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.256+0000 7f45d7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45d8103c20 0x7f45d8199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.256+0000 7f45d7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45d8103c20 0x7f45d8199950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58620/0 (socket says 172.21.15.96:58620) 2024-04-25T14:14:12.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.256+0000 7f45d7fff640 1 -- 172.21.15.96:0/3129295498 learned_addr learned my addr 172.21.15.96:0/3129295498 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:12.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.256+0000 7f45dc86d640 1 --2- 172.21.15.96:0/3129295498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45d81029e0 0x7f45d8199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.256+0000 7f45dd06e640 1 --2- 172.21.15.96:0/3129295498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45d8108920 0x7f45d81a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.256+0000 7f45d7fff640 1 -- 172.21.15.96:0/3129295498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45d81029e0 msgr2=0x7f45d8199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.256+0000 7f45d7fff640 1 --2- 172.21.15.96:0/3129295498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45d81029e0 0x7f45d8199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.256+0000 7f45d7fff640 1 -- 172.21.15.96:0/3129295498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45d8108920 msgr2=0x7f45d81a08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.256+0000 7f45d7fff640 1 --2- 172.21.15.96:0/3129295498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45d8108920 0x7f45d81a08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.256+0000 7f45d7fff640 1 -- 172.21.15.96:0/3129295498 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d81a0f50 con 0x7f45d8103c20 2024-04-25T14:14:12.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.256+0000 7f45dc86d640 1 --2- 172.21.15.96:0/3129295498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45d81029e0 0x7f45d8199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:14:12.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.256+0000 7f45d7fff640 1 --2- 172.21.15.96:0/3129295498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45d8103c20 0x7f45d8199950 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f45c400e9f0 tx=0x7f45c400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:12.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.256+0000 7f45d5ffb640 1 -- 172.21.15.96:0/3129295498 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c400cd80 con 0x7f45d8103c20 2024-04-25T14:14:12.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.256+0000 7f45d5ffb640 1 -- 172.21.15.96:0/3129295498 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45c40108e0 con 0x7f45d8103c20 2024-04-25T14:14:12.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.256+0000 7f45dead1640 1 -- 172.21.15.96:0/3129295498 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45d81a1240 con 0x7f45d8103c20 2024-04-25T14:14:12.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.256+0000 7f45d5ffb640 1 -- 172.21.15.96:0/3129295498 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c4018980 con 0x7f45d8103c20 2024-04-25T14:14:12.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.256+0000 7f45dead1640 1 -- 172.21.15.96:0/3129295498 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45d81a17e0 con 0x7f45d8103c20 2024-04-25T14:14:12.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.259+0000 7f45e597d740 1 Processor -- start 2024-04-25T14:14:12.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.259+0000 7f45e597d740 1 -- start start 2024-04-25T14:14:12.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.259+0000 7f45d5ffb640 1 -- 172.21.15.96:0/3129295498 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f45c4010430 con 0x7f45d8103c20 2024-04-25T14:14:12.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.259+0000 7f45e597d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3743a4390 0x55f374748a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.259+0000 7f45e597d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f37437b4d0 0x55f374748fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.259+0000 7f45e597d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3743836b0 0x55f374749c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.259+0000 7f45e597d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f3746aad80 con 0x55f3743836b0 2024-04-25T14:14:12.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.259+0000 7f45e597d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f374664a20 con 0x55f37437b4d0 2024-04-25T14:14:12.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.259+0000 7f45e597d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f3746a6d40 con 0x55f3743a4390 2024-04-25T14:14:12.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.259+0000 7f45d7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3743836b0 0x55f374749c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.260+0000 7f45d7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3743836b0 0x55f374749c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58632/0 (socket says 172.21.15.96:58632) 2024-04-25T14:14:12.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.260+0000 7f45dd06e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3743a4390 0x55f374748a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.260+0000 7f45d7fff640 1 -- 172.21.15.96:0/2529789859 learned_addr learned my addr 172.21.15.96:0/2529789859 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:12.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.260+0000 7f45dc86d640 1 --2- 172.21.15.96:0/2529789859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f37437b4d0 0x55f374748fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.260+0000 7f45d5ffb640 1 --2- 172.21.15.96:0/3129295498 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f45c80afb70 0x7f45c80b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.260+0000 7f45d7fff640 1 -- 172.21.15.96:0/2529789859 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3743a4390 msgr2=0x55f374748a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:12.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.260+0000 7f45d7fff640 1 --2- 172.21.15.96:0/2529789859 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3743a4390 0x55f374748a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.260+0000 7f45d7fff640 1 -- 172.21.15.96:0/2529789859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f37437b4d0 msgr2=0x55f374748fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:12.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.260+0000 7f45d7fff640 1 --2- 172.21.15.96:0/2529789859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f37437b4d0 0x55f374748fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.260+0000 7f45d7fff640 1 -- 172.21.15.96:0/2529789859 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f37466fc30 con 0x55f3743836b0 2024-04-25T14:14:12.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.260+0000 7f45dc86d640 1 --2- 172.21.15.96:0/2529789859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f37437b4d0 0x55f374748fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:14:12.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.260+0000 7f45dd06e640 1 --2- 172.21.15.96:0/3129295498 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f45c80afb70 0x7f45c80b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.260+0000 7f45d5ffb640 1 -- 172.21.15.96:0/3129295498 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f45c40deff0 con 0x7f45d8103c20 2024-04-25T14:14:12.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.260+0000 7f45d7fff640 1 --2- 172.21.15.96:0/2529789859 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3743836b0 0x55f374749c30 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f45c40c6a70 tx=0x7f45c40a3bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47644350f16f383e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:12.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.260+0000 7f45c2ffd640 1 -- 172.21.15.96:0/2529789859 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c4026030 con 0x55f3743836b0 2024-04-25T14:14:12.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.260+0000 7f45c2ffd640 1 -- 172.21.15.96:0/2529789859 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45c4010720 con 0x55f3743836b0 2024-04-25T14:14:12.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.260+0000 7f45c2ffd640 1 -- 172.21.15.96:0/2529789859 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c40a6c20 con 0x55f3743836b0 2024-04-25T14:14:12.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.260+0000 7f45dd06e640 1 --2- 172.21.15.96:0/3129295498 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f45c80afb70 0x7f45c80b2080 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x55f374749a70 tx=0x7f45d0004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:12.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.261+0000 7f45e597d740 1 -- 172.21.15.96:0/2529789859 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3743836b0 msgr2=0x55f374749c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.261+0000 7f45e597d740 1 --2- 172.21.15.96:0/2529789859 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3743836b0 0x55f374749c30 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f45c40c6a70 tx=0x7f45c40a3bc0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.261+0000 7f45e597d740 1 -- 172.21.15.96:0/2529789859 shutdown_connections 2024-04-25T14:14:12.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.261+0000 7f45e597d740 1 --2- 172.21.15.96:0/2529789859 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3743836b0 0x55f374749c30 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.261+0000 7f45e597d740 1 --2- 172.21.15.96:0/2529789859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f37437b4d0 0x55f374748fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.261+0000 7f45e597d740 1 --2- 172.21.15.96:0/2529789859 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3743a4390 0x55f374748a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.261+0000 7f45e597d740 1 -- 172.21.15.96:0/2529789859 >> 172.21.15.96:0/2529789859 conn(0x55f3743984e0 msgr2=0x55f374742020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:12.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.261+0000 7f45e597d740 1 -- 172.21.15.96:0/2529789859 shutdown_connections 2024-04-25T14:14:12.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.261+0000 7f45e597d740 1 -- 172.21.15.96:0/2529789859 wait complete. 2024-04-25T14:14:12.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.262+0000 7f45e597d740 1 -- start start 2024-04-25T14:14:12.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.262+0000 7f45e597d740 1 Processor -- start 2024-04-25T14:14:12.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.262+0000 7f45e597d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f37437b4d0 0x55f3747526a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.262+0000 7f45e597d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f3743836b0 0x55f374752be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.262+0000 7f45e597d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3743a4390 0x55f374759b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.262+0000 7f45e597d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f37474a680 con 0x55f3743a4390 2024-04-25T14:14:12.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.262+0000 7f45e597d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f374631dc0 con 0x55f3743836b0 2024-04-25T14:14:12.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.262+0000 7f45e597d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f37474a800 con 0x55f37437b4d0 2024-04-25T14:14:12.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.262+0000 7f45e597d740 10 client.0 _ll_register_callbacks cb 0x55f37464d600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:14:12.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.262+0000 7f45dd06e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3743a4390 0x55f374759b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.262+0000 7f45dd06e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3743a4390 0x55f374759b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58642/0 (socket says 172.21.15.96:58642) 2024-04-25T14:14:12.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.262+0000 7f45dd06e640 1 -- 172.21.15.96:0/967613403 learned_addr learned my addr 172.21.15.96:0/967613403 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:12.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.262+0000 7f45d7fff640 1 --2- 172.21.15.96:0/967613403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f3743836b0 0x55f374752be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.262+0000 7f45dc86d640 1 --2- 172.21.15.96:0/967613403 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f37437b4d0 0x55f3747526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.263+0000 7f45dd06e640 1 -- 172.21.15.96:0/967613403 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f37437b4d0 msgr2=0x55f3747526a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:12.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.263+0000 7f45dd06e640 1 --2- 172.21.15.96:0/967613403 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f37437b4d0 0x55f3747526a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.263+0000 7f45dd06e640 1 -- 172.21.15.96:0/967613403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f3743836b0 msgr2=0x55f374752be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:12.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.263+0000 7f45dd06e640 1 --2- 172.21.15.96:0/967613403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f3743836b0 0x55f374752be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.263+0000 7f45dd06e640 1 -- 172.21.15.96:0/967613403 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3743bc3f0 con 0x55f3743a4390 2024-04-25T14:14:12.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.263+0000 7f45d7fff640 1 --2- 172.21.15.96:0/967613403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f3743836b0 0x55f374752be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:14:12.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.263+0000 7f45dc86d640 1 --2- 172.21.15.96:0/967613403 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f37437b4d0 0x55f3747526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:14:12.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.263+0000 7f45dd06e640 1 --2- 172.21.15.96:0/967613403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3743a4390 0x55f374759b40 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f45d001e8e0 tx=0x7f45d0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ac474ff84aa5687 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:12.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.263+0000 7f45b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:14:12.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.263+0000 7f45b7fff640 1 -- 172.21.15.96:0/967613403 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d002b060 con 0x55f3743a4390 2024-04-25T14:14:12.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.263+0000 7f45b7fff640 1 -- 172.21.15.96:0/967613403 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f45d001cd60 con 0x55f3743a4390 2024-04-25T14:14:12.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.263+0000 7f45e597d740 1 -- 172.21.15.96:0/967613403 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f3746b8990 con 0x55f3743a4390 2024-04-25T14:14:12.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.263+0000 7f45b7fff640 1 -- 172.21.15.96:0/967613403 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f45d001ee60 con 0x55f3743a4390 2024-04-25T14:14:12.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.263+0000 7f45e597d740 10 client.15726 fetch_fsmap learned FSMap version 26 2024-04-25T14:14:12.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.263+0000 7f45e597d740 1 -- 172.21.15.96:0/967613403 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f3744867d0 con 0x55f3743a4390 2024-04-25T14:14:12.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.263+0000 7f45b7fff640 1 -- 172.21.15.96:0/967613403 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f3746b8990 con 0x55f3743a4390 2024-04-25T14:14:12.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.263+0000 7f45b7fff640 1 -- 172.21.15.96:0/967613403 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f45d0031630 con 0x55f3743a4390 2024-04-25T14:14:12.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.263+0000 7f45e597d740 10 client.15726 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:14:12.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.263+0000 7f45e597d740 10 client.15726 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T14:14:12.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.263+0000 7f45e597d740 20 client.15726 populate_metadata read hostname 'smithi096' 2024-04-25T14:14:12.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.263+0000 7f45e597d740 1 --2- 172.21.15.96:0/967613403 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x55f374362f60 0x55f374760420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.263+0000 7f45e597d740 4 client.15726 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:14:12.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.263+0000 7f45e597d740 1 -- 172.21.15.96:0/967613403 --> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3746b2800 con 0x55f374362f60 2024-04-25T14:14:12.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.264+0000 7f45dc86d640 1 --2- 172.21.15.96:0/967613403 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x55f374362f60 0x55f374760420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.264+0000 7f45b7fff640 10 client.15726 ms_handle_connect on v2:172.21.15.97:6834/3872301726 2024-04-25T14:14:12.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.264+0000 7f45dc86d640 1 --2- 172.21.15.96:0/967613403 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x55f374362f60 0x55f374760420 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f4640f68d8d2fc2c server_cookie=f799581f0a6f2612 in_seq=0 out_seq=0 2024-04-25T14:14:12.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.265+0000 7f45b7fff640 1 -- 172.21.15.96:0/967613403 <== mds.3 v2:172.21.15.97:6834/3872301726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f3746b2800 con 0x55f374362f60 2024-04-25T14:14:12.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.265+0000 7f45b7fff640 10 client.15726 handle_command_reply: tid=0 2024-04-25T14:14:12.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.265+0000 7f45e597d740 1 client.15726 shutdown 2024-04-25T14:14:12.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.265+0000 7f45e597d740 1 -- 172.21.15.96:0/967613403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3743a4390 msgr2=0x55f374759b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.265+0000 7f45e597d740 1 --2- 172.21.15.96:0/967613403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3743a4390 0x55f374759b40 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f45d001e8e0 tx=0x7f45d0004ab0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.265+0000 7f45e597d740 1 -- 172.21.15.96:0/967613403 shutdown_connections 2024-04-25T14:14:12.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.265+0000 7f45e597d740 1 --2- 172.21.15.96:0/967613403 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x55f374362f60 0x55f374760420 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.265+0000 7f45e597d740 1 --2- 172.21.15.96:0/967613403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3743a4390 0x55f374759b40 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.265+0000 7f45b7fff640 0 client.15726 ms_handle_reset on v2:172.21.15.97:6834/3872301726 2024-04-25T14:14:12.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.265+0000 7f45e597d740 1 --2- 172.21.15.96:0/967613403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f3743836b0 0x55f374752be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.265+0000 7f45e597d740 1 --2- 172.21.15.96:0/967613403 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f37437b4d0 0x55f3747526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.265+0000 7f45e597d740 1 -- 172.21.15.96:0/967613403 >> 172.21.15.96:0/967613403 conn(0x55f3743984e0 msgr2=0x55f374744f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:12.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.266+0000 7f45e597d740 1 -- 172.21.15.96:0/967613403 shutdown_connections 2024-04-25T14:14:12.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.266+0000 7f45e597d740 1 -- 172.21.15.96:0/967613403 wait complete. 2024-04-25T14:14:12.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.266+0000 7f45e597d740 20 client.15726 trim_cache size 0 max 16384 2024-04-25T14:14:12.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.279+0000 7f45e597d740 1 Processor -- start 2024-04-25T14:14:12.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.279+0000 7f45e597d740 1 -- start start 2024-04-25T14:14:12.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.279+0000 7f45e597d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f374362f60 0x55f374741a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.279+0000 7f45e597d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f37437b4d0 0x55f374759a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.279+0000 7f45e597d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3743836b0 0x55f374741f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.279+0000 7f45e597d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f45d81a91a0 con 0x55f3743836b0 2024-04-25T14:14:12.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.279+0000 7f45e597d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f45d81aace0 con 0x55f37437b4d0 2024-04-25T14:14:12.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.279+0000 7f45e597d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f45d81a9870 con 0x55f374362f60 2024-04-25T14:14:12.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.279+0000 7f45dd06e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3743836b0 0x55f374741f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.279+0000 7f45dd06e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3743836b0 0x55f374741f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58650/0 (socket says 172.21.15.96:58650) 2024-04-25T14:14:12.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.279+0000 7f45dd06e640 1 -- 172.21.15.96:0/2008888899 learned_addr learned my addr 172.21.15.96:0/2008888899 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:12.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.279+0000 7f45dd06e640 1 -- 172.21.15.96:0/2008888899 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f374362f60 msgr2=0x55f374741a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:12.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.279+0000 7f45dd06e640 1 --2- 172.21.15.96:0/2008888899 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f374362f60 0x55f374741a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.279+0000 7f45dd06e640 1 -- 172.21.15.96:0/2008888899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f37437b4d0 msgr2=0x55f374759a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:12.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.279+0000 7f45dd06e640 1 --2- 172.21.15.96:0/2008888899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f37437b4d0 0x55f374759a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.279+0000 7f45dd06e640 1 -- 172.21.15.96:0/2008888899 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f374750bf0 con 0x55f3743836b0 2024-04-25T14:14:12.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.279+0000 7f45dd06e640 1 --2- 172.21.15.96:0/2008888899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3743836b0 0x55f374741f90 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f45d0009d10 tx=0x7f45d0012d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e12ad5a181b71e49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:12.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.280+0000 7f45c27fc640 1 -- 172.21.15.96:0/2008888899 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d003e070 con 0x55f3743836b0 2024-04-25T14:14:12.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.280+0000 7f45c27fc640 1 -- 172.21.15.96:0/2008888899 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45d001cd60 con 0x55f3743836b0 2024-04-25T14:14:12.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.280+0000 7f45c27fc640 1 -- 172.21.15.96:0/2008888899 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d0039c20 con 0x55f3743836b0 2024-04-25T14:14:12.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.280+0000 7f45e597d740 1 -- 172.21.15.96:0/2008888899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3743836b0 msgr2=0x55f374741f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.280+0000 7f45e597d740 1 --2- 172.21.15.96:0/2008888899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3743836b0 0x55f374741f90 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f45d0009d10 tx=0x7f45d0012d70 comp rx=0 tx=0).stop 2024-04-25T14:14:12.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.280+0000 7f45e597d740 1 -- 172.21.15.96:0/2008888899 shutdown_connections 2024-04-25T14:14:12.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.280+0000 7f45e597d740 1 --2- 172.21.15.96:0/2008888899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3743836b0 0x55f374741f90 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.280+0000 7f45e597d740 1 --2- 172.21.15.96:0/2008888899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f37437b4d0 0x55f374759a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.280+0000 7f45e597d740 1 --2- 172.21.15.96:0/2008888899 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f374362f60 0x55f374741a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.280+0000 7f45e597d740 1 -- 172.21.15.96:0/2008888899 >> 172.21.15.96:0/2008888899 conn(0x55f3743984e0 msgr2=0x55f37474d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:12.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.280+0000 7f45e597d740 1 -- 172.21.15.96:0/2008888899 shutdown_connections 2024-04-25T14:14:12.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.280+0000 7f45e597d740 1 -- 172.21.15.96:0/2008888899 wait complete. 2024-04-25T14:14:12.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.281+0000 7f45e597d740 1 -- start start 2024-04-25T14:14:12.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.281+0000 7f45e597d740 1 Processor -- start 2024-04-25T14:14:12.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.281+0000 7f45e597d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f374362f60 0x55f3747573a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.281+0000 7f45e597d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f37437b4d0 0x55f3747578e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.281+0000 7f45e597d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3743836b0 0x55f374758810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.281+0000 7f45e597d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f45d81aa610 con 0x55f3743836b0 2024-04-25T14:14:12.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.281+0000 7f45e597d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f45d81a9f40 con 0x55f37437b4d0 2024-04-25T14:14:12.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.281+0000 7f45e597d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f45d81a7180 con 0x55f374362f60 2024-04-25T14:14:12.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.281+0000 7f45e597d740 10 client.0 _ll_register_callbacks cb 0x7f45d0009bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:14:12.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.281+0000 7f45dd06e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3743836b0 0x55f374758810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.281+0000 7f45dd06e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3743836b0 0x55f374758810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58654/0 (socket says 172.21.15.96:58654) 2024-04-25T14:14:12.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.281+0000 7f45dd06e640 1 -- 172.21.15.96:0/3159511133 learned_addr learned my addr 172.21.15.96:0/3159511133 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:12.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.281+0000 7f45d7fff640 1 --2- 172.21.15.96:0/3159511133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f37437b4d0 0x55f3747578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.281+0000 7f45dc86d640 1 --2- 172.21.15.96:0/3159511133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f374362f60 0x55f3747573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.281+0000 7f45dd06e640 1 -- 172.21.15.96:0/3159511133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f374362f60 msgr2=0x55f3747573a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:12.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.281+0000 7f45dd06e640 1 --2- 172.21.15.96:0/3159511133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f374362f60 0x55f3747573a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.281+0000 7f45dd06e640 1 -- 172.21.15.96:0/3159511133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f37437b4d0 msgr2=0x55f3747578e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:12.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.282+0000 7f45dd06e640 1 --2- 172.21.15.96:0/3159511133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f37437b4d0 0x55f3747578e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.282+0000 7f45dd06e640 1 -- 172.21.15.96:0/3159511133 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f374759fa0 con 0x55f3743836b0 2024-04-25T14:14:12.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.282+0000 7f45dc86d640 1 --2- 172.21.15.96:0/3159511133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f374362f60 0x55f3747573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:14:12.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.282+0000 7f45dd06e640 1 --2- 172.21.15.96:0/3159511133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3743836b0 0x55f374758810 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f45d0005d50 tx=0x7f45d0031730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da9e8e0cd489277f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:12.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.282+0000 7f45b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:14:12.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.282+0000 7f45b7fff640 1 -- 172.21.15.96:0/3159511133 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d003e070 con 0x55f3743836b0 2024-04-25T14:14:12.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.282+0000 7f45b7fff640 1 -- 172.21.15.96:0/3159511133 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f45d0039bb0 con 0x55f3743836b0 2024-04-25T14:14:12.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.282+0000 7f45e597d740 1 -- 172.21.15.96:0/3159511133 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45d001cd60 con 0x55f3743836b0 2024-04-25T14:14:12.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.282+0000 7f45b7fff640 1 -- 172.21.15.96:0/3159511133 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f45d0039db0 con 0x55f3743836b0 2024-04-25T14:14:12.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.282+0000 7f45e597d740 10 client.15732 fetch_fsmap learned FSMap version 26 2024-04-25T14:14:12.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.282+0000 7f45e597d740 1 -- 172.21.15.96:0/3159511133 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f3747c8790 con 0x55f3743836b0 2024-04-25T14:14:12.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.282+0000 7f45b7fff640 1 -- 172.21.15.96:0/3159511133 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45d001cd60 con 0x55f3743836b0 2024-04-25T14:14:12.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.282+0000 7f45b7fff640 1 -- 172.21.15.96:0/3159511133 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f45d004c440 con 0x55f3743836b0 2024-04-25T14:14:12.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.282+0000 7f45e597d740 10 client.15732 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:14:12.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.282+0000 7f45e597d740 10 client.15732 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T14:14:12.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.282+0000 7f45e597d740 20 client.15732 populate_metadata read hostname 'smithi096' 2024-04-25T14:14:12.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.282+0000 7f45e597d740 1 --2- 172.21.15.96:0/3159511133 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x55f3743a4390 0x55f3747caef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.282+0000 7f45e597d740 4 client.15732 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:14:12.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.282+0000 7f45e597d740 1 -- 172.21.15.96:0/3159511133 --> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f37462d610 con 0x55f3743a4390 2024-04-25T14:14:12.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.283+0000 7f45dc86d640 1 --2- 172.21.15.96:0/3159511133 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x55f3743a4390 0x55f3747caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.283+0000 7f45b7fff640 10 client.15732 ms_handle_connect on v2:172.21.15.97:6834/3872301726 2024-04-25T14:14:12.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.283+0000 7f45dc86d640 1 --2- 172.21.15.96:0/3159511133 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x55f3743a4390 0x55f3747caef0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d3d68f4b075c3676 server_cookie=d5620a9c59c8921b in_seq=0 out_seq=0 2024-04-25T14:14:12.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.283+0000 7f45b7fff640 1 -- 172.21.15.96:0/3159511133 <== mds.3 v2:172.21.15.97:6834/3872301726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f37462d610 con 0x55f3743a4390 2024-04-25T14:14:12.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.283+0000 7f45b7fff640 10 client.15732 handle_command_reply: tid=0 2024-04-25T14:14:12.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.283+0000 7f45e597d740 1 client.15732 shutdown 2024-04-25T14:14:12.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.283+0000 7f45e597d740 1 -- 172.21.15.96:0/3159511133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3743836b0 msgr2=0x55f374758810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.283+0000 7f45e597d740 1 --2- 172.21.15.96:0/3159511133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3743836b0 0x55f374758810 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f45d0005d50 tx=0x7f45d0031730 comp rx=0 tx=0).stop 2024-04-25T14:14:12.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.283+0000 7f45e597d740 1 -- 172.21.15.96:0/3159511133 shutdown_connections 2024-04-25T14:14:12.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.283+0000 7f45e597d740 1 --2- 172.21.15.96:0/3159511133 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x55f3743a4390 0x55f3747caef0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.283+0000 7f45e597d740 1 --2- 172.21.15.96:0/3159511133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f3743836b0 0x55f374758810 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.283+0000 7f45e597d740 1 --2- 172.21.15.96:0/3159511133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f37437b4d0 0x55f3747578e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.283+0000 7f45b7fff640 0 client.15732 ms_handle_reset on v2:172.21.15.97:6834/3872301726 2024-04-25T14:14:12.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.283+0000 7f45e597d740 1 --2- 172.21.15.96:0/3159511133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f374362f60 0x55f3747573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.283+0000 7f45e597d740 1 -- 172.21.15.96:0/3159511133 >> 172.21.15.96:0/3159511133 conn(0x55f3743984e0 msgr2=0x55f374744e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:12.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.284+0000 7f45e597d740 1 -- 172.21.15.96:0/3159511133 shutdown_connections 2024-04-25T14:14:12.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.284+0000 7f45e597d740 1 -- 172.21.15.96:0/3159511133 wait complete. 2024-04-25T14:14:12.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.284+0000 7f45e597d740 20 client.15732 trim_cache size 0 max 16384 2024-04-25T14:14:12.290 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-04-25T14:14:12.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.284+0000 7f45dead1640 1 -- 172.21.15.96:0/3129295498 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f45c80afb70 msgr2=0x7f45c80b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.284+0000 7f45dead1640 1 --2- 172.21.15.96:0/3129295498 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f45c80afb70 0x7f45c80b2080 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x55f374749a70 tx=0x7f45d0004a40 comp rx=0 tx=0).stop 2024-04-25T14:14:12.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.284+0000 7f45dead1640 1 -- 172.21.15.96:0/3129295498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45d8103c20 msgr2=0x7f45d8199950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.284+0000 7f45dead1640 1 --2- 172.21.15.96:0/3129295498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45d8103c20 0x7f45d8199950 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f45c400e9f0 tx=0x7f45c400eeb0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.284+0000 7f45dead1640 1 -- 172.21.15.96:0/3129295498 shutdown_connections 2024-04-25T14:14:12.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.284+0000 7f45dead1640 1 --2- 172.21.15.96:0/3129295498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45d8108920 0x7f45d81a08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.284+0000 7f45dead1640 1 --2- 172.21.15.96:0/3129295498 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f45c80afb70 0x7f45c80b2080 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.284+0000 7f45dead1640 1 --2- 172.21.15.96:0/3129295498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45d8103c20 0x7f45d8199950 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.284+0000 7f45dead1640 1 --2- 172.21.15.96:0/3129295498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45d81029e0 0x7f45d8199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.284+0000 7f45dead1640 1 -- 172.21.15.96:0/3129295498 >> 172.21.15.96:0/3129295498 conn(0x7f45d80fdbd0 msgr2=0x7f45d8104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:12.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.284+0000 7f45dead1640 1 -- 172.21.15.96:0/3129295498 shutdown_connections 2024-04-25T14:14:12.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.284+0000 7f45dead1640 1 -- 172.21.15.96:0/3129295498 wait complete. 2024-04-25T14:14:12.295 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T14:14:12.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.404+0000 7f238993f640 1 -- 172.21.15.96:0/2408833257 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2384108920 msgr2=0x7f238410ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.404+0000 7f238993f640 1 --2- 172.21.15.96:0/2408833257 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2384108920 0x7f238410ad40 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f237c009ca0 tx=0x7f237c02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.404+0000 7f238993f640 1 -- 172.21.15.96:0/2408833257 shutdown_connections 2024-04-25T14:14:12.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.404+0000 7f238993f640 1 --2- 172.21.15.96:0/2408833257 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2384108920 0x7f238410ad40 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.404+0000 7f238993f640 1 --2- 172.21.15.96:0/2408833257 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2384103c20 0x7f2384104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.404+0000 7f238993f640 1 --2- 172.21.15.96:0/2408833257 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23841029e0 0x7f23841036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.404+0000 7f238993f640 1 -- 172.21.15.96:0/2408833257 >> 172.21.15.96:0/2408833257 conn(0x7f23840fdbd0 msgr2=0x7f2384100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:12.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.404+0000 7f238993f640 1 -- 172.21.15.96:0/2408833257 shutdown_connections 2024-04-25T14:14:12.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.404+0000 7f238993f640 1 -- 172.21.15.96:0/2408833257 wait complete. 2024-04-25T14:14:12.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.404+0000 7f238993f640 1 Processor -- start 2024-04-25T14:14:12.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.405+0000 7f238993f640 1 -- start start 2024-04-25T14:14:12.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.405+0000 7f238993f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23841029e0 0x7f238419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.405+0000 7f238993f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2384103c20 0x7f238419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.405+0000 7f238993f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2384108920 0x7f23841a1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.405+0000 7f238993f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f238410e510 con 0x7f23841029e0 2024-04-25T14:14:12.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.405+0000 7f238993f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f238410e390 con 0x7f2384103c20 2024-04-25T14:14:12.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.405+0000 7f238993f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f238410e690 con 0x7f2384108920 2024-04-25T14:14:12.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.405+0000 7f2382ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23841029e0 0x7f238419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.405+0000 7f2382ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23841029e0 0x7f238419a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58670/0 (socket says 172.21.15.96:58670) 2024-04-25T14:14:12.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.405+0000 7f2382ffd640 1 -- 172.21.15.96:0/4156489921 learned_addr learned my addr 172.21.15.96:0/4156489921 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:12.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.405+0000 7f23827fc640 1 --2- 172.21.15.96:0/4156489921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2384103c20 0x7f238419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.405+0000 7f2382ffd640 1 -- 172.21.15.96:0/4156489921 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2384108920 msgr2=0x7f23841a1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.405+0000 7f2382ffd640 1 --2- 172.21.15.96:0/4156489921 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2384108920 0x7f23841a1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.405+0000 7f2382ffd640 1 -- 172.21.15.96:0/4156489921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2384103c20 msgr2=0x7f238419aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.405+0000 7f2382ffd640 1 --2- 172.21.15.96:0/4156489921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2384103c20 0x7f238419aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.405+0000 7f2382ffd640 1 -- 172.21.15.96:0/4156489921 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23841a20f0 con 0x7f23841029e0 2024-04-25T14:14:12.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.405+0000 7f2382ffd640 1 --2- 172.21.15.96:0/4156489921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23841029e0 0x7f238419a5b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f2378005d00 tx=0x7f237800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:12.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.405+0000 7f238893d640 1 -- 172.21.15.96:0/4156489921 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2378017070 con 0x7f23841029e0 2024-04-25T14:14:12.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.406+0000 7f238993f640 1 -- 172.21.15.96:0/4156489921 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23841a23e0 con 0x7f23841029e0 2024-04-25T14:14:12.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.406+0000 7f238893d640 1 -- 172.21.15.96:0/4156489921 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2378009c30 con 0x7f23841029e0 2024-04-25T14:14:12.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.406+0000 7f238893d640 1 -- 172.21.15.96:0/4156489921 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237800f620 con 0x7f23841029e0 2024-04-25T14:14:12.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.406+0000 7f238993f640 1 -- 172.21.15.96:0/4156489921 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23841a2930 con 0x7f23841029e0 2024-04-25T14:14:12.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.408+0000 7f23907f3740 1 Processor -- start 2024-04-25T14:14:12.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.408+0000 7f238893d640 1 -- 172.21.15.96:0/4156489921 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f237801a070 con 0x7f23841029e0 2024-04-25T14:14:12.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.408+0000 7f23907f3740 1 -- start start 2024-04-25T14:14:12.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.408+0000 7f23907f3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd88321390 0x55fd886c3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.408+0000 7f23907f3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd882f84d0 0x55fd886c3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.408+0000 7f23907f3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fd883006b0 0x55fd886c4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.408+0000 7f23907f3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55fd88627d80 con 0x55fd88321390 2024-04-25T14:14:12.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.408+0000 7f23907f3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55fd885e1a20 con 0x55fd882f84d0 2024-04-25T14:14:12.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.408+0000 7f23907f3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55fd88623d40 con 0x55fd883006b0 2024-04-25T14:14:12.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.408+0000 7f23837fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd88321390 0x55fd886c3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.408+0000 7f23837fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd88321390 0x55fd886c3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58676/0 (socket says 172.21.15.96:58676) 2024-04-25T14:14:12.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.408+0000 7f23837fe640 1 -- 172.21.15.96:0/265694740 learned_addr learned my addr 172.21.15.96:0/265694740 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:12.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.408+0000 7f2382ffd640 1 --2- 172.21.15.96:0/265694740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd882f84d0 0x55fd886c3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.408+0000 7f23827fc640 1 --2- 172.21.15.96:0/265694740 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fd883006b0 0x55fd886c4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.409+0000 7f2382ffd640 1 -- 172.21.15.96:0/265694740 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fd883006b0 msgr2=0x55fd886c4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:12.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.409+0000 7f2382ffd640 1 --2- 172.21.15.96:0/265694740 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fd883006b0 0x55fd886c4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.409+0000 7f2382ffd640 1 -- 172.21.15.96:0/265694740 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd88321390 msgr2=0x55fd886c3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:12.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.409+0000 7f2382ffd640 1 --2- 172.21.15.96:0/265694740 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd88321390 0x55fd886c3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.409+0000 7f2382ffd640 1 -- 172.21.15.96:0/265694740 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd885ecc30 con 0x55fd882f84d0 2024-04-25T14:14:12.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.409+0000 7f23837fe640 1 --2- 172.21.15.96:0/265694740 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd88321390 0x55fd886c3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:14:12.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.409+0000 7f238893d640 1 --2- 172.21.15.96:0/4156489921 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f236c0afaf0 0x7f236c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.409+0000 7f2382ffd640 1 --2- 172.21.15.96:0/265694740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd882f84d0 0x55fd886c3d90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f23780dc150 tx=0x7f23780e6690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da119c7f8c2feb87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:12.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.409+0000 7f23837fe640 1 --2- 172.21.15.96:0/4156489921 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f236c0afaf0 0x7f236c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.409+0000 7f238893d640 1 -- 172.21.15.96:0/4156489921 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f23780de6a0 con 0x7f23841029e0 2024-04-25T14:14:12.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.409+0000 7f2369ffb640 1 -- 172.21.15.96:0/265694740 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23780a2a20 con 0x55fd882f84d0 2024-04-25T14:14:12.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.409+0000 7f2369ffb640 1 -- 172.21.15.96:0/265694740 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23780a6670 con 0x55fd882f84d0 2024-04-25T14:14:12.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.409+0000 7f2369ffb640 1 -- 172.21.15.96:0/265694740 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23780a62d0 con 0x55fd882f84d0 2024-04-25T14:14:12.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.410+0000 7f23907f3740 1 -- 172.21.15.96:0/265694740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd882f84d0 msgr2=0x55fd886c3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.410+0000 7f23837fe640 1 --2- 172.21.15.96:0/4156489921 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f236c0afaf0 0x7f236c0b2000 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x55fd886c4860 tx=0x7f237c031b10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:12.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.410+0000 7f23907f3740 1 --2- 172.21.15.96:0/265694740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd882f84d0 0x55fd886c3d90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f23780dc150 tx=0x7f23780e6690 comp rx=0 tx=0).stop 2024-04-25T14:14:12.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.410+0000 7f23907f3740 1 -- 172.21.15.96:0/265694740 shutdown_connections 2024-04-25T14:14:12.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.410+0000 7f23907f3740 1 --2- 172.21.15.96:0/265694740 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fd883006b0 0x55fd886c4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.410+0000 7f23907f3740 1 --2- 172.21.15.96:0/265694740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd882f84d0 0x55fd886c3d90 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.410+0000 7f23907f3740 1 --2- 172.21.15.96:0/265694740 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd88321390 0x55fd886c3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.410+0000 7f23907f3740 1 -- 172.21.15.96:0/265694740 >> 172.21.15.96:0/265694740 conn(0x55fd883154e0 msgr2=0x55fd886bf020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:12.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.410+0000 7f23907f3740 1 -- 172.21.15.96:0/265694740 shutdown_connections 2024-04-25T14:14:12.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.410+0000 7f23907f3740 1 -- 172.21.15.96:0/265694740 wait complete. 2024-04-25T14:14:12.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.410+0000 7f23907f3740 1 -- start start 2024-04-25T14:14:12.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.411+0000 7f23907f3740 1 Processor -- start 2024-04-25T14:14:12.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.411+0000 7f23907f3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd882f84d0 0x55fd886cd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.411+0000 7f23907f3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd883006b0 0x55fd886cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.411+0000 7f23907f3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fd88321390 0x55fd886d4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.411+0000 7f23907f3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55fd886c5530 con 0x55fd882f84d0 2024-04-25T14:14:12.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.411+0000 7f23907f3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55fd885aedc0 con 0x55fd883006b0 2024-04-25T14:14:12.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.411+0000 7f23907f3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55fd886c56b0 con 0x55fd88321390 2024-04-25T14:14:12.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.411+0000 7f23907f3740 10 client.0 _ll_register_callbacks cb 0x55fd885ca600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:14:12.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.411+0000 7f2382ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd882f84d0 0x55fd886cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.411+0000 7f2382ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd882f84d0 0x55fd886cd630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58678/0 (socket says 172.21.15.96:58678) 2024-04-25T14:14:12.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.411+0000 7f2382ffd640 1 -- 172.21.15.96:0/1815050984 learned_addr learned my addr 172.21.15.96:0/1815050984 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:12.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.411+0000 7f23827fc640 1 --2- 172.21.15.96:0/1815050984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd883006b0 0x55fd886cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.411+0000 7f23837fe640 1 --2- 172.21.15.96:0/1815050984 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fd88321390 0x55fd886d4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.411+0000 7f2382ffd640 1 -- 172.21.15.96:0/1815050984 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fd88321390 msgr2=0x55fd886d4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:12.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.411+0000 7f2382ffd640 1 --2- 172.21.15.96:0/1815050984 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fd88321390 0x55fd886d4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.411+0000 7f2382ffd640 1 -- 172.21.15.96:0/1815050984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd883006b0 msgr2=0x55fd886cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:12.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.411+0000 7f2382ffd640 1 --2- 172.21.15.96:0/1815050984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd883006b0 0x55fd886cdb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.411+0000 7f2382ffd640 1 -- 172.21.15.96:0/1815050984 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd883393f0 con 0x55fd882f84d0 2024-04-25T14:14:12.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.411+0000 7f23837fe640 1 --2- 172.21.15.96:0/1815050984 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fd88321390 0x55fd886d4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:14:12.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.411+0000 7f23827fc640 1 --2- 172.21.15.96:0/1815050984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd883006b0 0x55fd886cdb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:14:12.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.412+0000 7f2382ffd640 1 --2- 172.21.15.96:0/1815050984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd882f84d0 0x55fd886cd630 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f23780dc4f0 tx=0x7f2378009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5e5a4481776232d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:12.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.412+0000 7f2362ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:14:12.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.412+0000 7f2362ffd640 1 -- 172.21.15.96:0/1815050984 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2378021070 con 0x55fd882f84d0 2024-04-25T14:14:12.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.412+0000 7f2362ffd640 1 -- 172.21.15.96:0/1815050984 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f23780a2df0 con 0x55fd882f84d0 2024-04-25T14:14:12.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.412+0000 7f23907f3740 1 -- 172.21.15.96:0/1815050984 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23780a6670 con 0x55fd882f84d0 2024-04-25T14:14:12.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.412+0000 7f2362ffd640 1 -- 172.21.15.96:0/1815050984 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f23780a64b0 con 0x55fd882f84d0 2024-04-25T14:14:12.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.412+0000 7f23907f3740 10 client.15744 fetch_fsmap learned FSMap version 26 2024-04-25T14:14:12.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.412+0000 7f23907f3740 1 -- 172.21.15.96:0/1815050984 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fd884037d0 con 0x55fd882f84d0 2024-04-25T14:14:12.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.412+0000 7f2362ffd640 1 -- 172.21.15.96:0/1815050984 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23780a6670 con 0x55fd882f84d0 2024-04-25T14:14:12.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.412+0000 7f2362ffd640 1 -- 172.21.15.96:0/1815050984 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f23780a2030 con 0x55fd882f84d0 2024-04-25T14:14:12.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.412+0000 7f23907f3740 10 client.15744 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:14:12.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.412+0000 7f23907f3740 10 client.15744 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:14:12.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.412+0000 7f23907f3740 20 client.15744 populate_metadata read hostname 'smithi096' 2024-04-25T14:14:12.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.412+0000 7f23907f3740 1 --2- 172.21.15.96:0/1815050984 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x55fd882dff60 0x55fd886daf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.412+0000 7f23907f3740 4 client.15744 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:14:12.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.412+0000 7f23907f3740 1 -- 172.21.15.96:0/1815050984 --> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd88635990 con 0x55fd882dff60 2024-04-25T14:14:12.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.413+0000 7f23827fc640 1 --2- 172.21.15.96:0/1815050984 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x55fd882dff60 0x55fd886daf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.413+0000 7f2362ffd640 10 client.15744 ms_handle_connect on v2:172.21.15.151:6834/1313823638 2024-04-25T14:14:12.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.413+0000 7f23827fc640 1 --2- 172.21.15.96:0/1815050984 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x55fd882dff60 0x55fd886daf20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1a5269a9c7e4e5cb server_cookie=3713baa1843552fa in_seq=0 out_seq=0 2024-04-25T14:14:12.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.414+0000 7f2362ffd640 1 -- 172.21.15.96:0/1815050984 <== mds.2 v2:172.21.15.151:6834/1313823638 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fd88635990 con 0x55fd882dff60 2024-04-25T14:14:12.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.414+0000 7f2362ffd640 10 client.15744 handle_command_reply: tid=0 2024-04-25T14:14:12.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.414+0000 7f23907f3740 1 client.15744 shutdown 2024-04-25T14:14:12.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.414+0000 7f23907f3740 1 -- 172.21.15.96:0/1815050984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd882f84d0 msgr2=0x55fd886cd630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.414+0000 7f23907f3740 1 --2- 172.21.15.96:0/1815050984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd882f84d0 0x55fd886cd630 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f23780dc4f0 tx=0x7f2378009740 comp rx=0 tx=0).stop 2024-04-25T14:14:12.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.414+0000 7f23907f3740 1 -- 172.21.15.96:0/1815050984 shutdown_connections 2024-04-25T14:14:12.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.414+0000 7f23907f3740 1 --2- 172.21.15.96:0/1815050984 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fd88321390 0x55fd886d4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.414+0000 7f23907f3740 1 --2- 172.21.15.96:0/1815050984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd883006b0 0x55fd886cdb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.414+0000 7f23907f3740 1 --2- 172.21.15.96:0/1815050984 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x55fd882dff60 0x55fd886daf20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.414+0000 7f23907f3740 1 --2- 172.21.15.96:0/1815050984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd882f84d0 0x55fd886cd630 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.414+0000 7f2362ffd640 0 client.15744 ms_handle_reset on v2:172.21.15.151:6834/1313823638 2024-04-25T14:14:12.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.414+0000 7f23907f3740 1 -- 172.21.15.96:0/1815050984 >> 172.21.15.96:0/1815050984 conn(0x55fd883154e0 msgr2=0x55fd886be040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:12.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.414+0000 7f23907f3740 1 -- 172.21.15.96:0/1815050984 shutdown_connections 2024-04-25T14:14:12.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.414+0000 7f23907f3740 1 -- 172.21.15.96:0/1815050984 wait complete. 2024-04-25T14:14:12.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.414+0000 7f23907f3740 20 client.15744 trim_cache size 0 max 16384 2024-04-25T14:14:12.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.427+0000 7f23907f3740 1 Processor -- start 2024-04-25T14:14:12.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.427+0000 7f23907f3740 1 -- start start 2024-04-25T14:14:12.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.428+0000 7f23907f3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd882f84d0 0x55fd886c8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.428+0000 7f23907f3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd883006b0 0x55fd886cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.428+0000 7f23907f3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fd88321390 0x55fd886df5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.428+0000 7f23907f3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f23841ab420 con 0x55fd882f84d0 2024-04-25T14:14:12.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.428+0000 7f23907f3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f23841acf60 con 0x55fd883006b0 2024-04-25T14:14:12.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.428+0000 7f23907f3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f23841abaf0 con 0x55fd88321390 2024-04-25T14:14:12.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.428+0000 7f2382ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd882f84d0 0x55fd886c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.428+0000 7f2382ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd882f84d0 0x55fd886c8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58688/0 (socket says 172.21.15.96:58688) 2024-04-25T14:14:12.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.428+0000 7f2382ffd640 1 -- 172.21.15.96:0/4267603188 learned_addr learned my addr 172.21.15.96:0/4267603188 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:12.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.428+0000 7f23837fe640 1 --2- 172.21.15.96:0/4267603188 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fd88321390 0x55fd886df5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.428+0000 7f23827fc640 1 --2- 172.21.15.96:0/4267603188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd883006b0 0x55fd886cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.428+0000 7f2382ffd640 1 -- 172.21.15.96:0/4267603188 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fd88321390 msgr2=0x55fd886df5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:12.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.428+0000 7f2382ffd640 1 --2- 172.21.15.96:0/4267603188 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fd88321390 0x55fd886df5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.428+0000 7f2382ffd640 1 -- 172.21.15.96:0/4267603188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd883006b0 msgr2=0x55fd886cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:12.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.428+0000 7f2382ffd640 1 --2- 172.21.15.96:0/4267603188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd883006b0 0x55fd886cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.428+0000 7f2382ffd640 1 -- 172.21.15.96:0/4267603188 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd886b92b0 con 0x55fd882f84d0 2024-04-25T14:14:12.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.428+0000 7f23837fe640 1 --2- 172.21.15.96:0/4267603188 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fd88321390 0x55fd886df5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:14:12.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.428+0000 7f23827fc640 1 --2- 172.21.15.96:0/4267603188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd883006b0 0x55fd886cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:14:12.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.428+0000 7f2382ffd640 1 --2- 172.21.15.96:0/4267603188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd882f84d0 0x55fd886c8980 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f23780a6b80 tx=0x7f23780097b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72fac2890e0a14c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:12.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.428+0000 7f23697fa640 1 -- 172.21.15.96:0/4267603188 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2378021020 con 0x55fd882f84d0 2024-04-25T14:14:12.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.428+0000 7f23697fa640 1 -- 172.21.15.96:0/4267603188 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23780e6e90 con 0x55fd882f84d0 2024-04-25T14:14:12.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.428+0000 7f23697fa640 1 -- 172.21.15.96:0/4267603188 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23780a6930 con 0x55fd882f84d0 2024-04-25T14:14:12.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.429+0000 7f23907f3740 1 -- 172.21.15.96:0/4267603188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd882f84d0 msgr2=0x55fd886c8980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.429+0000 7f23907f3740 1 --2- 172.21.15.96:0/4267603188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd882f84d0 0x55fd886c8980 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f23780a6b80 tx=0x7f23780097b0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.429+0000 7f23907f3740 1 -- 172.21.15.96:0/4267603188 shutdown_connections 2024-04-25T14:14:12.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.429+0000 7f23907f3740 1 --2- 172.21.15.96:0/4267603188 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fd88321390 0x55fd886df5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.429+0000 7f23907f3740 1 --2- 172.21.15.96:0/4267603188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd883006b0 0x55fd886cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.429+0000 7f23907f3740 1 --2- 172.21.15.96:0/4267603188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd882f84d0 0x55fd886c8980 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.429+0000 7f23907f3740 1 -- 172.21.15.96:0/4267603188 >> 172.21.15.96:0/4267603188 conn(0x55fd883154e0 msgr2=0x55fd886cd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:12.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.429+0000 7f23907f3740 1 -- 172.21.15.96:0/4267603188 shutdown_connections 2024-04-25T14:14:12.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.429+0000 7f23907f3740 1 -- 172.21.15.96:0/4267603188 wait complete. 2024-04-25T14:14:12.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.429+0000 7f23907f3740 1 -- start start 2024-04-25T14:14:12.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.430+0000 7f23907f3740 1 Processor -- start 2024-04-25T14:14:12.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.430+0000 7f23907f3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd882f84d0 0x55fd886d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.430+0000 7f23907f3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd883006b0 0x55fd886d1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.430+0000 7f23907f3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fd88321390 0x55fd886d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.430+0000 7f23907f3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f23841ac890 con 0x55fd883006b0 2024-04-25T14:14:12.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.430+0000 7f23907f3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f23841ac1c0 con 0x55fd882f84d0 2024-04-25T14:14:12.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.430+0000 7f23907f3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f23841a9400 con 0x55fd88321390 2024-04-25T14:14:12.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.430+0000 7f23907f3740 10 client.0 _ll_register_callbacks cb 0x7f237c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:14:12.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.430+0000 7f23827fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd883006b0 0x55fd886d1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.430+0000 7f23827fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd883006b0 0x55fd886d1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58704/0 (socket says 172.21.15.96:58704) 2024-04-25T14:14:12.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.430+0000 7f23827fc640 1 -- 172.21.15.96:0/1826687548 learned_addr learned my addr 172.21.15.96:0/1826687548 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:12.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.430+0000 7f23837fe640 1 --2- 172.21.15.96:0/1826687548 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fd88321390 0x55fd886d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.430+0000 7f2382ffd640 1 --2- 172.21.15.96:0/1826687548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd882f84d0 0x55fd886d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.430+0000 7f23827fc640 1 -- 172.21.15.96:0/1826687548 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fd88321390 msgr2=0x55fd886d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:12.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.430+0000 7f23827fc640 1 --2- 172.21.15.96:0/1826687548 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fd88321390 0x55fd886d29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.430+0000 7f23827fc640 1 -- 172.21.15.96:0/1826687548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd882f84d0 msgr2=0x55fd886d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:12.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.430+0000 7f23827fc640 1 --2- 172.21.15.96:0/1826687548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd882f84d0 0x55fd886d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.430+0000 7f23827fc640 1 -- 172.21.15.96:0/1826687548 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd886d2f80 con 0x55fd883006b0 2024-04-25T14:14:12.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.430+0000 7f23827fc640 1 --2- 172.21.15.96:0/1826687548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd883006b0 0x55fd886d1a90 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x55fd886bfdb0 tx=0x7f2374010a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc4caf722c32a89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:12.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.430+0000 7f2362ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:14:12.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.431+0000 7f2362ffd640 1 -- 172.21.15.96:0/1826687548 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2374005220 con 0x55fd883006b0 2024-04-25T14:14:12.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.431+0000 7f2362ffd640 1 -- 172.21.15.96:0/1826687548 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f23740053c0 con 0x55fd883006b0 2024-04-25T14:14:12.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.431+0000 7f23907f3740 1 -- 172.21.15.96:0/1826687548 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23780e6e90 con 0x55fd883006b0 2024-04-25T14:14:12.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.431+0000 7f2362ffd640 1 -- 172.21.15.96:0/1826687548 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f237402f050 con 0x55fd883006b0 2024-04-25T14:14:12.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.431+0000 7f23907f3740 10 client.15750 fetch_fsmap learned FSMap version 26 2024-04-25T14:14:12.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.431+0000 7f23907f3740 1 -- 172.21.15.96:0/1826687548 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fd886d41e0 con 0x55fd883006b0 2024-04-25T14:14:12.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.431+0000 7f2362ffd640 1 -- 172.21.15.96:0/1826687548 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23780e6e90 con 0x55fd883006b0 2024-04-25T14:14:12.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.431+0000 7f2362ffd640 1 -- 172.21.15.96:0/1826687548 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f237401e9e0 con 0x55fd883006b0 2024-04-25T14:14:12.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.431+0000 7f23907f3740 10 client.15750 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:14:12.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.431+0000 7f23907f3740 10 client.15750 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:14:12.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.431+0000 7f23907f3740 20 client.15750 populate_metadata read hostname 'smithi096' 2024-04-25T14:14:12.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.431+0000 7f23907f3740 1 --2- 172.21.15.96:0/1826687548 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x55fd882b53a0 0x55fd887456b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.431+0000 7f23907f3740 4 client.15750 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:14:12.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.431+0000 7f23907f3740 1 -- 172.21.15.96:0/1826687548 --> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fd8862f800 con 0x55fd882b53a0 2024-04-25T14:14:12.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.431+0000 7f2382ffd640 1 --2- 172.21.15.96:0/1826687548 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x55fd882b53a0 0x55fd887456b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.432+0000 7f2362ffd640 10 client.15750 ms_handle_connect on v2:172.21.15.151:6834/1313823638 2024-04-25T14:14:12.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.432+0000 7f2382ffd640 1 --2- 172.21.15.96:0/1826687548 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x55fd882b53a0 0x55fd887456b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=329fc5ece47834bc server_cookie=34e5b64bd6b04f87 in_seq=0 out_seq=0 2024-04-25T14:14:12.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.432+0000 7f2362ffd640 1 -- 172.21.15.96:0/1826687548 <== mds.2 v2:172.21.15.151:6834/1313823638 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fd8862f800 con 0x55fd882b53a0 2024-04-25T14:14:12.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.432+0000 7f2362ffd640 10 client.15750 handle_command_reply: tid=0 2024-04-25T14:14:12.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.432+0000 7f23907f3740 1 client.15750 shutdown 2024-04-25T14:14:12.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.432+0000 7f23907f3740 1 -- 172.21.15.96:0/1826687548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd883006b0 msgr2=0x55fd886d1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.432+0000 7f23907f3740 1 --2- 172.21.15.96:0/1826687548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd883006b0 0x55fd886d1a90 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x55fd886bfdb0 tx=0x7f2374010a30 comp rx=0 tx=0).stop 2024-04-25T14:14:12.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.432+0000 7f23907f3740 1 -- 172.21.15.96:0/1826687548 shutdown_connections 2024-04-25T14:14:12.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.432+0000 7f23907f3740 1 --2- 172.21.15.96:0/1826687548 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fd88321390 0x55fd886d29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.432+0000 7f23907f3740 1 --2- 172.21.15.96:0/1826687548 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x55fd882b53a0 0x55fd887456b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.432+0000 7f23907f3740 1 --2- 172.21.15.96:0/1826687548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd883006b0 0x55fd886d1a90 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.432+0000 7f23907f3740 1 --2- 172.21.15.96:0/1826687548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd882f84d0 0x55fd886d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.432+0000 7f2362ffd640 0 client.15750 ms_handle_reset on v2:172.21.15.151:6834/1313823638 2024-04-25T14:14:12.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.432+0000 7f23907f3740 1 -- 172.21.15.96:0/1826687548 >> 172.21.15.96:0/1826687548 conn(0x55fd883154e0 msgr2=0x55fd886cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:12.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.432+0000 7f23907f3740 1 -- 172.21.15.96:0/1826687548 shutdown_connections 2024-04-25T14:14:12.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.432+0000 7f23907f3740 1 -- 172.21.15.96:0/1826687548 wait complete. 2024-04-25T14:14:12.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.432+0000 7f23907f3740 20 client.15750 trim_cache size 0 max 16384 2024-04-25T14:14:12.440 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-04-25T14:14:12.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.433+0000 7f238993f640 1 -- 172.21.15.96:0/4156489921 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f236c0afaf0 msgr2=0x7f236c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.433+0000 7f238993f640 1 --2- 172.21.15.96:0/4156489921 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f236c0afaf0 0x7f236c0b2000 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x55fd886c4860 tx=0x7f237c031b10 comp rx=0 tx=0).stop 2024-04-25T14:14:12.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.433+0000 7f238993f640 1 -- 172.21.15.96:0/4156489921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23841029e0 msgr2=0x7f238419a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.433+0000 7f238993f640 1 --2- 172.21.15.96:0/4156489921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23841029e0 0x7f238419a5b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f2378005d00 tx=0x7f237800dde0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.433+0000 7f238993f640 1 -- 172.21.15.96:0/4156489921 shutdown_connections 2024-04-25T14:14:12.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.433+0000 7f238993f640 1 --2- 172.21.15.96:0/4156489921 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2384108920 0x7f23841a1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.433+0000 7f238993f640 1 --2- 172.21.15.96:0/4156489921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2384103c20 0x7f238419aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.433+0000 7f238993f640 1 --2- 172.21.15.96:0/4156489921 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f236c0afaf0 0x7f236c0b2000 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.433+0000 7f238993f640 1 --2- 172.21.15.96:0/4156489921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23841029e0 0x7f238419a5b0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.433+0000 7f238993f640 1 -- 172.21.15.96:0/4156489921 >> 172.21.15.96:0/4156489921 conn(0x7f23840fdbd0 msgr2=0x7f2384109e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:12.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.433+0000 7f238993f640 1 -- 172.21.15.96:0/4156489921 shutdown_connections 2024-04-25T14:14:12.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.433+0000 7f238993f640 1 -- 172.21.15.96:0/4156489921 wait complete. 2024-04-25T14:14:12.444 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T14:14:12.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.554+0000 7f19a4d5f640 1 -- 172.21.15.96:0/2428488061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19a0108920 msgr2=0x7f19a010ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.554+0000 7f19a4d5f640 1 --2- 172.21.15.96:0/2428488061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19a0108920 0x7f19a010ad40 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1998009cc0 tx=0x7f199802cad0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.554+0000 7f19a4d5f640 1 -- 172.21.15.96:0/2428488061 shutdown_connections 2024-04-25T14:14:12.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.554+0000 7f19a4d5f640 1 --2- 172.21.15.96:0/2428488061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19a0108920 0x7f19a010ad40 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.554+0000 7f19a4d5f640 1 --2- 172.21.15.96:0/2428488061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19a0103c20 0x7f19a0104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.554+0000 7f19a4d5f640 1 --2- 172.21.15.96:0/2428488061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f19a01029e0 0x7f19a01036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.554+0000 7f19a4d5f640 1 -- 172.21.15.96:0/2428488061 >> 172.21.15.96:0/2428488061 conn(0x7f19a00fdbd0 msgr2=0x7f19a0100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:12.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.554+0000 7f19a4d5f640 1 -- 172.21.15.96:0/2428488061 shutdown_connections 2024-04-25T14:14:12.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.554+0000 7f19a4d5f640 1 -- 172.21.15.96:0/2428488061 wait complete. 2024-04-25T14:14:12.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.555+0000 7f19a4d5f640 1 Processor -- start 2024-04-25T14:14:12.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.555+0000 7f19a4d5f640 1 -- start start 2024-04-25T14:14:12.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.555+0000 7f19a4d5f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f19a01029e0 0x7f19a01982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.555+0000 7f19a4d5f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19a0103c20 0x7f19a0198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.555+0000 7f19a4d5f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19a0108920 0x7f19a019f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.555+0000 7f19a4d5f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f19a010d400 con 0x7f19a0103c20 2024-04-25T14:14:12.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.555+0000 7f19a4d5f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f19a010d280 con 0x7f19a0108920 2024-04-25T14:14:12.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.555+0000 7f19a4d5f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f19a010d580 con 0x7f19a01029e0 2024-04-25T14:14:12.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.555+0000 7f199dd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19a0103c20 0x7f19a0198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.555+0000 7f199dd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19a0103c20 0x7f19a0198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58710/0 (socket says 172.21.15.96:58710) 2024-04-25T14:14:12.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.555+0000 7f199dd9b640 1 -- 172.21.15.96:0/3193234908 learned_addr learned my addr 172.21.15.96:0/3193234908 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:12.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.555+0000 7f199ed9d640 1 --2- 172.21.15.96:0/3193234908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19a0108920 0x7f19a019f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.555+0000 7f199e59c640 1 --2- 172.21.15.96:0/3193234908 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f19a01029e0 0x7f19a01982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.555+0000 7f199dd9b640 1 -- 172.21.15.96:0/3193234908 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f19a01029e0 msgr2=0x7f19a01982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.555+0000 7f199dd9b640 1 --2- 172.21.15.96:0/3193234908 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f19a01029e0 0x7f19a01982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.555+0000 7f199dd9b640 1 -- 172.21.15.96:0/3193234908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19a0108920 msgr2=0x7f19a019f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.555+0000 7f199dd9b640 1 --2- 172.21.15.96:0/3193234908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19a0108920 0x7f19a019f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.555+0000 7f199dd9b640 1 -- 172.21.15.96:0/3193234908 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19a019fe30 con 0x7f19a0103c20 2024-04-25T14:14:12.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.555+0000 7f199ed9d640 1 --2- 172.21.15.96:0/3193234908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19a0108920 0x7f19a019f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:14:12.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.555+0000 7f199e59c640 1 --2- 172.21.15.96:0/3193234908 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f19a01029e0 0x7f19a01982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:14:12.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.556+0000 7f199dd9b640 1 --2- 172.21.15.96:0/3193234908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19a0103c20 0x7f19a0198830 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f199000d950 tx=0x7f199000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:12.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.556+0000 7f198f7fe640 1 -- 172.21.15.96:0/3193234908 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1990017070 con 0x7f19a0103c20 2024-04-25T14:14:12.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.556+0000 7f198f7fe640 1 -- 172.21.15.96:0/3193234908 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1990009c30 con 0x7f19a0103c20 2024-04-25T14:14:12.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.556+0000 7f19a4d5f640 1 -- 172.21.15.96:0/3193234908 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19a01a0120 con 0x7f19a0103c20 2024-04-25T14:14:12.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.556+0000 7f198f7fe640 1 -- 172.21.15.96:0/3193234908 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199000f620 con 0x7f19a0103c20 2024-04-25T14:14:12.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.556+0000 7f19a4d5f640 1 -- 172.21.15.96:0/3193234908 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19a01a0670 con 0x7f19a0103c20 2024-04-25T14:14:12.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.558+0000 7f198f7fe640 1 -- 172.21.15.96:0/3193234908 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f1990010460 con 0x7f19a0103c20 2024-04-25T14:14:12.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.559+0000 7f19abc0b740 1 Processor -- start 2024-04-25T14:14:12.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.559+0000 7f19abc0b740 1 -- start start 2024-04-25T14:14:12.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.559+0000 7f19abc0b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56020a980390 0x56020ad22810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.559+0000 7f19abc0b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56020a9574d0 0x56020ad22d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.559+0000 7f19abc0b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56020a95f6b0 0x56020ad23a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.559+0000 7f19abc0b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56020ac86d80 con 0x56020a95f6b0 2024-04-25T14:14:12.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.559+0000 7f19abc0b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56020ac40a20 con 0x56020a980390 2024-04-25T14:14:12.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.559+0000 7f19abc0b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56020ac82d40 con 0x56020a9574d0 2024-04-25T14:14:12.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.559+0000 7f198f7fe640 1 --2- 172.21.15.96:0/3193234908 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f19880afaf0 0x7f19880b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.559+0000 7f199dd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56020a95f6b0 0x56020ad23a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.559+0000 7f199dd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56020a95f6b0 0x56020ad23a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58722/0 (socket says 172.21.15.96:58722) 2024-04-25T14:14:12.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.559+0000 7f199dd9b640 1 -- 172.21.15.96:0/1924094494 learned_addr learned my addr 172.21.15.96:0/1924094494 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:12.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.559+0000 7f199e59c640 1 --2- 172.21.15.96:0/1924094494 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56020a9574d0 0x56020ad22d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.559+0000 7f199ed9d640 1 --2- 172.21.15.96:0/3193234908 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f19880afaf0 0x7f19880b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.559+0000 7f199ed9d640 1 --2- 172.21.15.96:0/1924094494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56020a980390 0x56020ad22810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.560+0000 7f198f7fe640 1 -- 172.21.15.96:0/3193234908 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f19900de960 con 0x7f19a0103c20 2024-04-25T14:14:12.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.560+0000 7f199dd9b640 1 -- 172.21.15.96:0/1924094494 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56020a9574d0 msgr2=0x56020ad22d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:12.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.560+0000 7f199dd9b640 1 --2- 172.21.15.96:0/1924094494 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56020a9574d0 0x56020ad22d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.560+0000 7f199dd9b640 1 -- 172.21.15.96:0/1924094494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56020a980390 msgr2=0x56020ad22810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:12.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.560+0000 7f199dd9b640 1 --2- 172.21.15.96:0/1924094494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56020a980390 0x56020ad22810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.560+0000 7f199dd9b640 1 -- 172.21.15.96:0/1924094494 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56020ac4bc30 con 0x56020a95f6b0 2024-04-25T14:14:12.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.560+0000 7f199dd9b640 1 --2- 172.21.15.96:0/1924094494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56020a95f6b0 0x56020ad23a20 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1990000c00 tx=0x7f19900a6cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53e51cfe7e11b82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:12.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.560+0000 7f199ed9d640 1 --2- 172.21.15.96:0/3193234908 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f19880afaf0 0x7f19880b2000 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1998006010 tx=0x7f1998003db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:12.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.560+0000 7f198cff9640 1 -- 172.21.15.96:0/1924094494 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1990020070 con 0x56020a95f6b0 2024-04-25T14:14:12.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.560+0000 7f198cff9640 1 -- 172.21.15.96:0/1924094494 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19900a6930 con 0x56020a95f6b0 2024-04-25T14:14:12.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.560+0000 7f198cff9640 1 -- 172.21.15.96:0/1924094494 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19900a30b0 con 0x56020a95f6b0 2024-04-25T14:14:12.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.561+0000 7f19abc0b740 1 -- 172.21.15.96:0/1924094494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56020a95f6b0 msgr2=0x56020ad23a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.561+0000 7f19abc0b740 1 --2- 172.21.15.96:0/1924094494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56020a95f6b0 0x56020ad23a20 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1990000c00 tx=0x7f19900a6cb0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.561+0000 7f19abc0b740 1 -- 172.21.15.96:0/1924094494 shutdown_connections 2024-04-25T14:14:12.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.561+0000 7f19abc0b740 1 --2- 172.21.15.96:0/1924094494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56020a95f6b0 0x56020ad23a20 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.561+0000 7f19abc0b740 1 --2- 172.21.15.96:0/1924094494 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56020a9574d0 0x56020ad22d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.561+0000 7f19abc0b740 1 --2- 172.21.15.96:0/1924094494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56020a980390 0x56020ad22810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.561+0000 7f19abc0b740 1 -- 172.21.15.96:0/1924094494 >> 172.21.15.96:0/1924094494 conn(0x56020a9744e0 msgr2=0x56020ad1e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:12.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.561+0000 7f19abc0b740 1 -- 172.21.15.96:0/1924094494 shutdown_connections 2024-04-25T14:14:12.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.561+0000 7f19abc0b740 1 -- 172.21.15.96:0/1924094494 wait complete. 2024-04-25T14:14:12.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.561+0000 7f19abc0b740 1 -- start start 2024-04-25T14:14:12.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.561+0000 7f19abc0b740 1 Processor -- start 2024-04-25T14:14:12.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.562+0000 7f19abc0b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56020a9574d0 0x56020ad2c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.562+0000 7f19abc0b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56020a95f6b0 0x56020ad2cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.562+0000 7f19abc0b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56020a980390 0x56020ad33a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.562+0000 7f19abc0b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56020ad244d0 con 0x56020a980390 2024-04-25T14:14:12.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.562+0000 7f19abc0b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56020ac0ddc0 con 0x56020a95f6b0 2024-04-25T14:14:12.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.562+0000 7f19abc0b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56020ad24650 con 0x56020a9574d0 2024-04-25T14:14:12.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.562+0000 7f19abc0b740 10 client.0 _ll_register_callbacks cb 0x56020ac29600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:14:12.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.562+0000 7f199ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56020a980390 0x56020ad33a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.562+0000 7f199ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56020a980390 0x56020ad33a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58728/0 (socket says 172.21.15.96:58728) 2024-04-25T14:14:12.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.562+0000 7f199ed9d640 1 -- 172.21.15.96:0/1205487343 learned_addr learned my addr 172.21.15.96:0/1205487343 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:12.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.562+0000 7f199ed9d640 1 -- 172.21.15.96:0/1205487343 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56020a9574d0 msgr2=0x56020ad2c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:12.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.562+0000 7f199ed9d640 1 --2- 172.21.15.96:0/1205487343 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56020a9574d0 0x56020ad2c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.562+0000 7f199dd9b640 1 --2- 172.21.15.96:0/1205487343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56020a95f6b0 0x56020ad2cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.562+0000 7f199ed9d640 1 -- 172.21.15.96:0/1205487343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56020a95f6b0 msgr2=0x56020ad2cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:12.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.562+0000 7f199ed9d640 1 --2- 172.21.15.96:0/1205487343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56020a95f6b0 0x56020ad2cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.562+0000 7f199ed9d640 1 -- 172.21.15.96:0/1205487343 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56020a9983f0 con 0x56020a980390 2024-04-25T14:14:12.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.562+0000 7f199dd9b640 1 --2- 172.21.15.96:0/1205487343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56020a95f6b0 0x56020ad2cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:14:12.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.562+0000 7f199ed9d640 1 --2- 172.21.15.96:0/1205487343 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56020a980390 0x56020ad33a90 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1998009cc0 tx=0x7f1998048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23b14ee40ca021d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:12.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.562+0000 7f1985ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:14:12.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.562+0000 7f1985ffb640 1 -- 172.21.15.96:0/1205487343 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1998058060 con 0x56020a980390 2024-04-25T14:14:12.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.562+0000 7f1985ffb640 1 -- 172.21.15.96:0/1205487343 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f19980046b0 con 0x56020a980390 2024-04-25T14:14:12.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.562+0000 7f19abc0b740 1 -- 172.21.15.96:0/1205487343 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19900a6930 con 0x56020a980390 2024-04-25T14:14:12.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.563+0000 7f1985ffb640 1 -- 172.21.15.96:0/1205487343 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f199802e630 con 0x56020a980390 2024-04-25T14:14:12.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.563+0000 7f19abc0b740 10 client.15762 fetch_fsmap learned FSMap version 26 2024-04-25T14:14:12.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.563+0000 7f19abc0b740 1 -- 172.21.15.96:0/1205487343 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56020aa627d0 con 0x56020a980390 2024-04-25T14:14:12.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.563+0000 7f1985ffb640 1 -- 172.21.15.96:0/1205487343 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19900a6930 con 0x56020a980390 2024-04-25T14:14:12.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.563+0000 7f1985ffb640 1 -- 172.21.15.96:0/1205487343 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f199802e040 con 0x56020a980390 2024-04-25T14:14:12.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.563+0000 7f19abc0b740 10 client.15762 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:14:12.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.563+0000 7f19abc0b740 10 client.15762 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:14:12.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.563+0000 7f19abc0b740 20 client.15762 populate_metadata read hostname 'smithi096' 2024-04-25T14:14:12.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.563+0000 7f19abc0b740 1 --2- 172.21.15.96:0/1205487343 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x56020a93ef60 0x56020ad3a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.563+0000 7f19abc0b740 4 client.15762 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:14:12.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.563+0000 7f19abc0b740 1 -- 172.21.15.96:0/1205487343 --> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56020ac94990 con 0x56020a93ef60 2024-04-25T14:14:12.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.563+0000 7f199e59c640 1 --2- 172.21.15.96:0/1205487343 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x56020a93ef60 0x56020ad3a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.564+0000 7f1985ffb640 10 client.15762 ms_handle_connect on v2:172.21.15.151:6838/245699981 2024-04-25T14:14:12.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.564+0000 7f199e59c640 1 --2- 172.21.15.96:0/1205487343 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x56020a93ef60 0x56020ad3a430 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8632ce53ca46b3d5 server_cookie=b07f597d2f9d975d in_seq=0 out_seq=0 2024-04-25T14:14:12.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.565+0000 7f1985ffb640 1 -- 172.21.15.96:0/1205487343 <== mds.1 v2:172.21.15.151:6838/245699981 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56020ac94990 con 0x56020a93ef60 2024-04-25T14:14:12.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.565+0000 7f1985ffb640 10 client.15762 handle_command_reply: tid=0 2024-04-25T14:14:12.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.565+0000 7f19abc0b740 1 client.15762 shutdown 2024-04-25T14:14:12.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.565+0000 7f19abc0b740 1 -- 172.21.15.96:0/1205487343 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56020a980390 msgr2=0x56020ad33a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.565+0000 7f19abc0b740 1 --2- 172.21.15.96:0/1205487343 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56020a980390 0x56020ad33a90 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1998009cc0 tx=0x7f1998048000 comp rx=0 tx=0).stop 2024-04-25T14:14:12.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.565+0000 7f19abc0b740 1 -- 172.21.15.96:0/1205487343 shutdown_connections 2024-04-25T14:14:12.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.565+0000 7f19abc0b740 1 --2- 172.21.15.96:0/1205487343 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x56020a93ef60 0x56020ad3a430 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.565+0000 7f19abc0b740 1 --2- 172.21.15.96:0/1205487343 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56020a980390 0x56020ad33a90 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.565+0000 7f1985ffb640 0 client.15762 ms_handle_reset on v2:172.21.15.151:6838/245699981 2024-04-25T14:14:12.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.565+0000 7f19abc0b740 1 --2- 172.21.15.96:0/1205487343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56020a95f6b0 0x56020ad2cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.565+0000 7f19abc0b740 1 --2- 172.21.15.96:0/1205487343 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56020a9574d0 0x56020ad2c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.565+0000 7f19abc0b740 1 -- 172.21.15.96:0/1205487343 >> 172.21.15.96:0/1205487343 conn(0x56020a9744e0 msgr2=0x56020ad20f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:12.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.565+0000 7f19abc0b740 1 -- 172.21.15.96:0/1205487343 shutdown_connections 2024-04-25T14:14:12.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.565+0000 7f19abc0b740 1 -- 172.21.15.96:0/1205487343 wait complete. 2024-04-25T14:14:12.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.565+0000 7f19abc0b740 20 client.15762 trim_cache size 0 max 16384 2024-04-25T14:14:12.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.578+0000 7f19abc0b740 1 Processor -- start 2024-04-25T14:14:12.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.578+0000 7f19abc0b740 1 -- start start 2024-04-25T14:14:12.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.578+0000 7f19abc0b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56020a9574d0 0x56020ad27350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.578+0000 7f19abc0b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56020a95f6b0 0x56020ad27890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.578+0000 7f19abc0b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56020a980390 0x56020ad339b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.578+0000 7f19abc0b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f19a01a9210 con 0x56020a9574d0 2024-04-25T14:14:12.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.578+0000 7f19abc0b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f19a01aad50 con 0x56020a980390 2024-04-25T14:14:12.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.578+0000 7f19abc0b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f19a01a98e0 con 0x56020a95f6b0 2024-04-25T14:14:12.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.579+0000 7f199e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56020a9574d0 0x56020ad27350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.579+0000 7f199e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56020a9574d0 0x56020ad27350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58732/0 (socket says 172.21.15.96:58732) 2024-04-25T14:14:12.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.579+0000 7f199e59c640 1 -- 172.21.15.96:0/2977873370 learned_addr learned my addr 172.21.15.96:0/2977873370 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:12.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.579+0000 7f199ed9d640 1 --2- 172.21.15.96:0/2977873370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56020a980390 0x56020ad339b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.579+0000 7f199e59c640 1 -- 172.21.15.96:0/2977873370 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56020a95f6b0 msgr2=0x56020ad27890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:12.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.579+0000 7f199e59c640 1 --2- 172.21.15.96:0/2977873370 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56020a95f6b0 0x56020ad27890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.579+0000 7f199e59c640 1 -- 172.21.15.96:0/2977873370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56020a980390 msgr2=0x56020ad339b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:12.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.579+0000 7f199e59c640 1 --2- 172.21.15.96:0/2977873370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56020a980390 0x56020ad339b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.579+0000 7f199e59c640 1 -- 172.21.15.96:0/2977873370 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56020ad33ef0 con 0x56020a9574d0 2024-04-25T14:14:12.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.579+0000 7f199e59c640 1 --2- 172.21.15.96:0/2977873370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56020a9574d0 0x56020ad27350 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f199400bb60 tx=0x7f199400b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d396da375cc638 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:12.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.579+0000 7f1987fff640 1 -- 172.21.15.96:0/2977873370 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199401d070 con 0x56020a9574d0 2024-04-25T14:14:12.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.579+0000 7f1987fff640 1 -- 172.21.15.96:0/2977873370 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1994019850 con 0x56020a9574d0 2024-04-25T14:14:12.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.579+0000 7f1987fff640 1 -- 172.21.15.96:0/2977873370 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199400eba0 con 0x56020a9574d0 2024-04-25T14:14:12.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.580+0000 7f19abc0b740 1 -- 172.21.15.96:0/2977873370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56020a9574d0 msgr2=0x56020ad27350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.580+0000 7f19abc0b740 1 --2- 172.21.15.96:0/2977873370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56020a9574d0 0x56020ad27350 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f199400bb60 tx=0x7f199400b700 comp rx=0 tx=0).stop 2024-04-25T14:14:12.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.580+0000 7f19abc0b740 1 -- 172.21.15.96:0/2977873370 shutdown_connections 2024-04-25T14:14:12.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.580+0000 7f19abc0b740 1 --2- 172.21.15.96:0/2977873370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56020a980390 0x56020ad339b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.580+0000 7f19abc0b740 1 --2- 172.21.15.96:0/2977873370 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56020a95f6b0 0x56020ad27890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.580+0000 7f19abc0b740 1 --2- 172.21.15.96:0/2977873370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56020a9574d0 0x56020ad27350 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.580+0000 7f19abc0b740 1 -- 172.21.15.96:0/2977873370 >> 172.21.15.96:0/2977873370 conn(0x56020a9744e0 msgr2=0x56020ad2acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:12.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.580+0000 7f19abc0b740 1 -- 172.21.15.96:0/2977873370 shutdown_connections 2024-04-25T14:14:12.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.580+0000 7f19abc0b740 1 -- 172.21.15.96:0/2977873370 wait complete. 2024-04-25T14:14:12.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.580+0000 7f19abc0b740 1 -- start start 2024-04-25T14:14:12.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.580+0000 7f19abc0b740 1 Processor -- start 2024-04-25T14:14:12.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.580+0000 7f19abc0b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56020a9574d0 0x56020ad32a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.581+0000 7f19abc0b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56020a95f6b0 0x56020ad32fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.581+0000 7f19abc0b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56020a980390 0x56020ada0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.581+0000 7f19abc0b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f19a01aa680 con 0x56020a9574d0 2024-04-25T14:14:12.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.581+0000 7f19abc0b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f19a01a9fb0 con 0x56020a95f6b0 2024-04-25T14:14:12.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.581+0000 7f19abc0b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f19a01a71f0 con 0x56020a980390 2024-04-25T14:14:12.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.581+0000 7f19abc0b740 10 client.0 _ll_register_callbacks cb 0x7f199802d150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:14:12.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.581+0000 7f199e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56020a9574d0 0x56020ad32a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.581+0000 7f199e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56020a9574d0 0x56020ad32a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58734/0 (socket says 172.21.15.96:58734) 2024-04-25T14:14:12.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.581+0000 7f199e59c640 1 -- 172.21.15.96:0/4092292386 learned_addr learned my addr 172.21.15.96:0/4092292386 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:12.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.581+0000 7f199dd9b640 1 --2- 172.21.15.96:0/4092292386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56020a95f6b0 0x56020ad32fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.581+0000 7f199e59c640 1 -- 172.21.15.96:0/4092292386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56020a980390 msgr2=0x56020ada0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:12.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.581+0000 7f199e59c640 1 --2- 172.21.15.96:0/4092292386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56020a980390 0x56020ada0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.581+0000 7f199e59c640 1 -- 172.21.15.96:0/4092292386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56020a95f6b0 msgr2=0x56020ad32fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:12.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.581+0000 7f199e59c640 1 --2- 172.21.15.96:0/4092292386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56020a95f6b0 0x56020ad32fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.581+0000 7f199e59c640 1 -- 172.21.15.96:0/4092292386 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56020ada0d50 con 0x56020a9574d0 2024-04-25T14:14:12.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.581+0000 7f199e59c640 1 --2- 172.21.15.96:0/4092292386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56020a9574d0 0x56020ad32a60 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f199400b9e0 tx=0x7f199400fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd1359ba4e85d2ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:12.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.581+0000 7f1985ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:14:12.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.581+0000 7f1985ffb640 1 -- 172.21.15.96:0/4092292386 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199401d070 con 0x56020a9574d0 2024-04-25T14:14:12.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.581+0000 7f1985ffb640 1 -- 172.21.15.96:0/4092292386 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f199400e8e0 con 0x56020a9574d0 2024-04-25T14:14:12.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.581+0000 7f19abc0b740 1 -- 172.21.15.96:0/4092292386 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1994019850 con 0x56020a9574d0 2024-04-25T14:14:12.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.581+0000 7f1985ffb640 1 -- 172.21.15.96:0/4092292386 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1994032070 con 0x56020a9574d0 2024-04-25T14:14:12.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.581+0000 7f19abc0b740 10 client.15768 fetch_fsmap learned FSMap version 26 2024-04-25T14:14:12.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.581+0000 7f19abc0b740 1 -- 172.21.15.96:0/4092292386 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56020ada2050 con 0x56020a9574d0 2024-04-25T14:14:12.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.582+0000 7f1985ffb640 1 -- 172.21.15.96:0/4092292386 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1994019850 con 0x56020a9574d0 2024-04-25T14:14:12.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.582+0000 7f1985ffb640 1 -- 172.21.15.96:0/4092292386 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f19940047c0 con 0x56020a9574d0 2024-04-25T14:14:12.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.582+0000 7f19abc0b740 10 client.15768 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:14:12.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.582+0000 7f19abc0b740 10 client.15768 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:14:12.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.582+0000 7f19abc0b740 20 client.15768 populate_metadata read hostname 'smithi096' 2024-04-25T14:14:12.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.582+0000 7f19abc0b740 1 --2- 172.21.15.96:0/4092292386 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x56020a9143a0 0x56020ada4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.582+0000 7f19abc0b740 4 client.15768 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:14:12.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.582+0000 7f19abc0b740 1 -- 172.21.15.96:0/4092292386 --> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56020ac8e800 con 0x56020a9143a0 2024-04-25T14:14:12.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.582+0000 7f199dd9b640 1 --2- 172.21.15.96:0/4092292386 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x56020a9143a0 0x56020ada4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.582+0000 7f1985ffb640 10 client.15768 ms_handle_connect on v2:172.21.15.151:6838/245699981 2024-04-25T14:14:12.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.582+0000 7f199dd9b640 1 --2- 172.21.15.96:0/4092292386 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x56020a9143a0 0x56020ada4a10 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d0715cd2c348344b server_cookie=2d5fdfd7ad42aca2 in_seq=0 out_seq=0 2024-04-25T14:14:12.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.583+0000 7f1985ffb640 1 -- 172.21.15.96:0/4092292386 <== mds.1 v2:172.21.15.151:6838/245699981 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56020ac8e800 con 0x56020a9143a0 2024-04-25T14:14:12.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.583+0000 7f1985ffb640 10 client.15768 handle_command_reply: tid=0 2024-04-25T14:14:12.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.583+0000 7f19abc0b740 1 client.15768 shutdown 2024-04-25T14:14:12.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.583+0000 7f19abc0b740 1 -- 172.21.15.96:0/4092292386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56020a9574d0 msgr2=0x56020ad32a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.583+0000 7f19abc0b740 1 --2- 172.21.15.96:0/4092292386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56020a9574d0 0x56020ad32a60 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f199400b9e0 tx=0x7f199400fec0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.583+0000 7f19abc0b740 1 -- 172.21.15.96:0/4092292386 shutdown_connections 2024-04-25T14:14:12.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.583+0000 7f19abc0b740 1 --2- 172.21.15.96:0/4092292386 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x56020a9143a0 0x56020ada4a10 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.583+0000 7f19abc0b740 1 --2- 172.21.15.96:0/4092292386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56020a980390 0x56020ada0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.583+0000 7f19abc0b740 1 --2- 172.21.15.96:0/4092292386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56020a95f6b0 0x56020ad32fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.583+0000 7f1985ffb640 0 client.15768 ms_handle_reset on v2:172.21.15.151:6838/245699981 2024-04-25T14:14:12.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.583+0000 7f19abc0b740 1 --2- 172.21.15.96:0/4092292386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56020a9574d0 0x56020ad32a60 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.583+0000 7f19abc0b740 1 -- 172.21.15.96:0/4092292386 >> 172.21.15.96:0/4092292386 conn(0x56020a9744e0 msgr2=0x56020ad2c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:12.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.583+0000 7f19abc0b740 1 -- 172.21.15.96:0/4092292386 shutdown_connections 2024-04-25T14:14:12.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.583+0000 7f19abc0b740 1 -- 172.21.15.96:0/4092292386 wait complete. 2024-04-25T14:14:12.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.583+0000 7f19abc0b740 20 client.15768 trim_cache size 0 max 16384 2024-04-25T14:14:12.589 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-04-25T14:14:12.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.583+0000 7f19a4d5f640 1 -- 172.21.15.96:0/3193234908 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f19880afaf0 msgr2=0x7f19880b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.583+0000 7f19a4d5f640 1 --2- 172.21.15.96:0/3193234908 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f19880afaf0 0x7f19880b2000 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1998006010 tx=0x7f1998003db0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.583+0000 7f19a4d5f640 1 -- 172.21.15.96:0/3193234908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19a0103c20 msgr2=0x7f19a0198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.583+0000 7f19a4d5f640 1 --2- 172.21.15.96:0/3193234908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19a0103c20 0x7f19a0198830 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f199000d950 tx=0x7f199000de10 comp rx=0 tx=0).stop 2024-04-25T14:14:12.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.584+0000 7f19a4d5f640 1 -- 172.21.15.96:0/3193234908 shutdown_connections 2024-04-25T14:14:12.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.584+0000 7f19a4d5f640 1 --2- 172.21.15.96:0/3193234908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19a0108920 0x7f19a019f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.584+0000 7f19a4d5f640 1 --2- 172.21.15.96:0/3193234908 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f19880afaf0 0x7f19880b2000 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.584+0000 7f19a4d5f640 1 --2- 172.21.15.96:0/3193234908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19a0103c20 0x7f19a0198830 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.584+0000 7f19a4d5f640 1 --2- 172.21.15.96:0/3193234908 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f19a01029e0 0x7f19a01982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.584+0000 7f19a4d5f640 1 -- 172.21.15.96:0/3193234908 >> 172.21.15.96:0/3193234908 conn(0x7f19a00fdbd0 msgr2=0x7f19a0104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:12.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.584+0000 7f19a4d5f640 1 -- 172.21.15.96:0/3193234908 shutdown_connections 2024-04-25T14:14:12.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.584+0000 7f19a4d5f640 1 -- 172.21.15.96:0/3193234908 wait complete. 2024-04-25T14:14:12.593 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:12 smithi096 ceph-mon[29207]: pgmap v363: 97 pgs: 97 active+clean; 657 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-25T14:14:12.593 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:12 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/313502187' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:14:12.594 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T14:14:12.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.704+0000 7f5e6a6ee640 1 -- 172.21.15.96:0/1848396136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e64103940 msgr2=0x7f5e64103df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.704+0000 7f5e6a6ee640 1 --2- 172.21.15.96:0/1848396136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e64103940 0x7f5e64103df0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f5e58009900 tx=0x7f5e5802a410 comp rx=0 tx=0).stop 2024-04-25T14:14:12.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.704+0000 7f5e6a6ee640 1 -- 172.21.15.96:0/1848396136 shutdown_connections 2024-04-25T14:14:12.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.704+0000 7f5e6a6ee640 1 --2- 172.21.15.96:0/1848396136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e64104480 0x7f5e6410abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.704+0000 7f5e6a6ee640 1 --2- 172.21.15.96:0/1848396136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e64103940 0x7f5e64103df0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.704+0000 7f5e6a6ee640 1 --2- 172.21.15.96:0/1848396136 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5e64102f60 0x7f5e64103370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.704+0000 7f5e6a6ee640 1 -- 172.21.15.96:0/1848396136 >> 172.21.15.96:0/1848396136 conn(0x7f5e640fd380 msgr2=0x7f5e640ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:12.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.704+0000 7f5e6a6ee640 1 -- 172.21.15.96:0/1848396136 shutdown_connections 2024-04-25T14:14:12.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.704+0000 7f5e6a6ee640 1 -- 172.21.15.96:0/1848396136 wait complete. 2024-04-25T14:14:12.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.705+0000 7f5e6a6ee640 1 Processor -- start 2024-04-25T14:14:12.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.705+0000 7f5e6a6ee640 1 -- start start 2024-04-25T14:14:12.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.705+0000 7f5e6a6ee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e64102f60 0x7f5e6419a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.705+0000 7f5e6a6ee640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5e64103940 0x7f5e6419a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.705+0000 7f5e6a6ee640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e64104480 0x7f5e641a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.705+0000 7f5e6a6ee640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6410f640 con 0x7f5e64102f60 2024-04-25T14:14:12.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.705+0000 7f5e6a6ee640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6410f4c0 con 0x7f5e64104480 2024-04-25T14:14:12.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.705+0000 7f5e6a6ee640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6410f7c0 con 0x7f5e64103940 2024-04-25T14:14:12.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.705+0000 7f5e63fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e64102f60 0x7f5e6419a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.705+0000 7f5e63fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e64102f60 0x7f5e6419a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58756/0 (socket says 172.21.15.96:58756) 2024-04-25T14:14:12.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.705+0000 7f5e63fff640 1 -- 172.21.15.96:0/3381590783 learned_addr learned my addr 172.21.15.96:0/3381590783 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:12.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.705+0000 7f5e68c8b640 1 --2- 172.21.15.96:0/3381590783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e64104480 0x7f5e641a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.706+0000 7f5e637fe640 1 --2- 172.21.15.96:0/3381590783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5e64103940 0x7f5e6419a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.706+0000 7f5e63fff640 1 -- 172.21.15.96:0/3381590783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5e64103940 msgr2=0x7f5e6419a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.706+0000 7f5e63fff640 1 --2- 172.21.15.96:0/3381590783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5e64103940 0x7f5e6419a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.706+0000 7f5e63fff640 1 -- 172.21.15.96:0/3381590783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e64104480 msgr2=0x7f5e641a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.706+0000 7f5e63fff640 1 --2- 172.21.15.96:0/3381590783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e64104480 0x7f5e641a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.706+0000 7f5e63fff640 1 -- 172.21.15.96:0/3381590783 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e641a1db0 con 0x7f5e64102f60 2024-04-25T14:14:12.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.706+0000 7f5e637fe640 1 --2- 172.21.15.96:0/3381590783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5e64103940 0x7f5e6419a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:14:12.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.706+0000 7f5e63fff640 1 --2- 172.21.15.96:0/3381590783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e64102f60 0x7f5e6419a270 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5e5400c970 tx=0x7f5e5400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:12.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.706+0000 7f5e617fa640 1 -- 172.21.15.96:0/3381590783 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e54007bb0 con 0x7f5e64102f60 2024-04-25T14:14:12.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.706+0000 7f5e617fa640 1 -- 172.21.15.96:0/3381590783 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e54007d50 con 0x7f5e64102f60 2024-04-25T14:14:12.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.706+0000 7f5e6a6ee640 1 -- 172.21.15.96:0/3381590783 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e641a20a0 con 0x7f5e64102f60 2024-04-25T14:14:12.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.706+0000 7f5e617fa640 1 -- 172.21.15.96:0/3381590783 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5400f450 con 0x7f5e64102f60 2024-04-25T14:14:12.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.706+0000 7f5e6a6ee640 1 -- 172.21.15.96:0/3381590783 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e641a25f0 con 0x7f5e64102f60 2024-04-25T14:14:12.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.708+0000 7f5e7159e740 1 Processor -- start 2024-04-25T14:14:12.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.708+0000 7f5e7159e740 1 -- start start 2024-04-25T14:14:12.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.708+0000 7f5e617fa640 1 -- 172.21.15.96:0/3381590783 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f5e5400f5f0 con 0x7f5e64102f60 2024-04-25T14:14:12.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.709+0000 7f5e7159e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dcfd03d390 0x55dcfd3df810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.709+0000 7f5e7159e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dcfd0144d0 0x55dcfd3dfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.709+0000 7f5e7159e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcfd01c6b0 0x55dcfd3e0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.709+0000 7f5e7159e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55dcfd343d80 con 0x55dcfd01c6b0 2024-04-25T14:14:12.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.709+0000 7f5e7159e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55dcfd2fda20 con 0x55dcfd03d390 2024-04-25T14:14:12.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.709+0000 7f5e7159e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55dcfd33fd40 con 0x55dcfd0144d0 2024-04-25T14:14:12.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.709+0000 7f5e637fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcfd01c6b0 0x55dcfd3e0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.709+0000 7f5e637fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcfd01c6b0 0x55dcfd3e0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58768/0 (socket says 172.21.15.96:58768) 2024-04-25T14:14:12.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.709+0000 7f5e637fe640 1 -- 172.21.15.96:0/1175975776 learned_addr learned my addr 172.21.15.96:0/1175975776 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:12.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.709+0000 7f5e68c8b640 1 --2- 172.21.15.96:0/1175975776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dcfd03d390 0x55dcfd3df810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.709+0000 7f5e63fff640 1 --2- 172.21.15.96:0/1175975776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dcfd0144d0 0x55dcfd3dfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.709+0000 7f5e637fe640 1 -- 172.21.15.96:0/1175975776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dcfd0144d0 msgr2=0x55dcfd3dfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:12.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.709+0000 7f5e637fe640 1 --2- 172.21.15.96:0/1175975776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dcfd0144d0 0x55dcfd3dfd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.709+0000 7f5e637fe640 1 -- 172.21.15.96:0/1175975776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dcfd03d390 msgr2=0x55dcfd3df810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:12.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.709+0000 7f5e637fe640 1 --2- 172.21.15.96:0/1175975776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dcfd03d390 0x55dcfd3df810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.709+0000 7f5e637fe640 1 -- 172.21.15.96:0/1175975776 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcfd308c30 con 0x55dcfd01c6b0 2024-04-25T14:14:12.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.709+0000 7f5e68c8b640 1 --2- 172.21.15.96:0/1175975776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dcfd03d390 0x55dcfd3df810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:14:12.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.709+0000 7f5e637fe640 1 --2- 172.21.15.96:0/1175975776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcfd01c6b0 0x55dcfd3e0a20 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f5e58006010 tx=0x7f5e58032c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f0052f9008ebe9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:12.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.709+0000 7f5e617fa640 1 --2- 172.21.15.96:0/3381590783 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f5e4c0afaf0 0x7f5e4c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.709+0000 7f5e4a7fc640 1 -- 172.21.15.96:0/1175975776 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5803f030 con 0x55dcfd01c6b0 2024-04-25T14:14:12.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.709+0000 7f5e68c8b640 1 --2- 172.21.15.96:0/3381590783 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f5e4c0afaf0 0x7f5e4c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.709+0000 7f5e617fa640 1 -- 172.21.15.96:0/3381590783 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f5e540de930 con 0x7f5e64102f60 2024-04-25T14:14:12.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.709+0000 7f5e4a7fc640 1 -- 172.21.15.96:0/1175975776 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e58032e40 con 0x55dcfd01c6b0 2024-04-25T14:14:12.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.709+0000 7f5e4a7fc640 1 -- 172.21.15.96:0/1175975776 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e58035070 con 0x55dcfd01c6b0 2024-04-25T14:14:12.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.710+0000 7f5e7159e740 1 -- 172.21.15.96:0/1175975776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcfd01c6b0 msgr2=0x55dcfd3e0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.710+0000 7f5e68c8b640 1 --2- 172.21.15.96:0/3381590783 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f5e4c0afaf0 0x7f5e4c0b2000 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x55dcfd3e0860 tx=0x7f5e5c009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:12.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.710+0000 7f5e7159e740 1 --2- 172.21.15.96:0/1175975776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcfd01c6b0 0x55dcfd3e0a20 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f5e58006010 tx=0x7f5e58032c50 comp rx=0 tx=0).stop 2024-04-25T14:14:12.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.710+0000 7f5e7159e740 1 -- 172.21.15.96:0/1175975776 shutdown_connections 2024-04-25T14:14:12.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.710+0000 7f5e7159e740 1 --2- 172.21.15.96:0/1175975776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcfd01c6b0 0x55dcfd3e0a20 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.710+0000 7f5e7159e740 1 --2- 172.21.15.96:0/1175975776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dcfd0144d0 0x55dcfd3dfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.710+0000 7f5e7159e740 1 --2- 172.21.15.96:0/1175975776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dcfd03d390 0x55dcfd3df810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.710+0000 7f5e7159e740 1 -- 172.21.15.96:0/1175975776 >> 172.21.15.96:0/1175975776 conn(0x55dcfd0314e0 msgr2=0x55dcfd3db020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:12.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.710+0000 7f5e7159e740 1 -- 172.21.15.96:0/1175975776 shutdown_connections 2024-04-25T14:14:12.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.710+0000 7f5e7159e740 1 -- 172.21.15.96:0/1175975776 wait complete. 2024-04-25T14:14:12.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.711+0000 7f5e7159e740 1 -- start start 2024-04-25T14:14:12.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.711+0000 7f5e7159e740 1 Processor -- start 2024-04-25T14:14:12.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.711+0000 7f5e7159e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dcfd0144d0 0x55dcfd3e9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.711+0000 7f5e7159e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcfd01c6b0 0x55dcfd3e9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.711+0000 7f5e7159e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dcfd03d390 0x55dcfd3f09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.711+0000 7f5e7159e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55dcfd3e14d0 con 0x55dcfd01c6b0 2024-04-25T14:14:12.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.711+0000 7f5e7159e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55dcfd2cadc0 con 0x55dcfd03d390 2024-04-25T14:14:12.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.711+0000 7f5e7159e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55dcfd3e1650 con 0x55dcfd0144d0 2024-04-25T14:14:12.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.711+0000 7f5e7159e740 10 client.0 _ll_register_callbacks cb 0x55dcfd2e6600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:14:12.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.711+0000 7f5e637fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcfd01c6b0 0x55dcfd3e9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.711+0000 7f5e637fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcfd01c6b0 0x55dcfd3e9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58770/0 (socket says 172.21.15.96:58770) 2024-04-25T14:14:12.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.711+0000 7f5e637fe640 1 -- 172.21.15.96:0/2877644447 learned_addr learned my addr 172.21.15.96:0/2877644447 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:12.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.711+0000 7f5e68c8b640 1 --2- 172.21.15.96:0/2877644447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dcfd03d390 0x55dcfd3f09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.712+0000 7f5e63fff640 1 --2- 172.21.15.96:0/2877644447 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dcfd0144d0 0x55dcfd3e9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.712+0000 7f5e637fe640 1 -- 172.21.15.96:0/2877644447 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dcfd0144d0 msgr2=0x55dcfd3e9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:12.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.712+0000 7f5e637fe640 1 --2- 172.21.15.96:0/2877644447 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dcfd0144d0 0x55dcfd3e9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.712+0000 7f5e637fe640 1 -- 172.21.15.96:0/2877644447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dcfd03d390 msgr2=0x55dcfd3f09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:12.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.712+0000 7f5e637fe640 1 --2- 172.21.15.96:0/2877644447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dcfd03d390 0x55dcfd3f09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.712+0000 7f5e637fe640 1 -- 172.21.15.96:0/2877644447 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcfd0553f0 con 0x55dcfd01c6b0 2024-04-25T14:14:12.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.712+0000 7f5e637fe640 1 --2- 172.21.15.96:0/2877644447 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcfd01c6b0 0x55dcfd3e9a80 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5e580055d0 tx=0x7f5e58005600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7618df475d73e28d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:12.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.712+0000 7f5e437fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:14:12.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.712+0000 7f5e437fe640 1 -- 172.21.15.96:0/2877644447 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e58035050 con 0x55dcfd01c6b0 2024-04-25T14:14:12.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.712+0000 7f5e437fe640 1 -- 172.21.15.96:0/2877644447 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f5e580318a0 con 0x55dcfd01c6b0 2024-04-25T14:14:12.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.712+0000 7f5e7159e740 1 -- 172.21.15.96:0/2877644447 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55dcfd351990 con 0x55dcfd01c6b0 2024-04-25T14:14:12.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.712+0000 7f5e437fe640 1 -- 172.21.15.96:0/2877644447 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5e58052070 con 0x55dcfd01c6b0 2024-04-25T14:14:12.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.712+0000 7f5e7159e740 10 client.15780 fetch_fsmap learned FSMap version 26 2024-04-25T14:14:12.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.712+0000 7f5e7159e740 1 -- 172.21.15.96:0/2877644447 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55dcfd11f7d0 con 0x55dcfd01c6b0 2024-04-25T14:14:12.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.712+0000 7f5e437fe640 1 -- 172.21.15.96:0/2877644447 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55dcfd351990 con 0x55dcfd01c6b0 2024-04-25T14:14:12.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.713+0000 7f5e437fe640 1 -- 172.21.15.96:0/2877644447 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5e580035f0 con 0x55dcfd01c6b0 2024-04-25T14:14:12.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.713+0000 7f5e7159e740 10 client.15780 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:14:12.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.713+0000 7f5e7159e740 10 client.15780 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:14:12.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.713+0000 7f5e7159e740 20 client.15780 populate_metadata read hostname 'smithi096' 2024-04-25T14:14:12.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.713+0000 7f5e7159e740 1 --2- 172.21.15.96:0/2877644447 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55dcfcffbf60 0x55dcfd3f6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.713+0000 7f5e7159e740 4 client.15780 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:14:12.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.713+0000 7f5e7159e740 1 -- 172.21.15.96:0/2877644447 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcfd34b800 con 0x55dcfcffbf60 2024-04-25T14:14:12.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.713+0000 7f5e63fff640 1 --2- 172.21.15.96:0/2877644447 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55dcfcffbf60 0x55dcfd3f6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.713+0000 7f5e437fe640 10 client.15780 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:12.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.713+0000 7f5e63fff640 1 --2- 172.21.15.96:0/2877644447 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55dcfcffbf60 0x55dcfd3f6e30 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b94eb692d830cac3 server_cookie=ef6c8c67413d127f in_seq=0 out_seq=0 2024-04-25T14:14:12.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.714+0000 7f5e437fe640 1 -- 172.21.15.96:0/2877644447 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dcfd34b800 con 0x55dcfcffbf60 2024-04-25T14:14:12.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.714+0000 7f5e437fe640 10 client.15780 handle_command_reply: tid=0 2024-04-25T14:14:12.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.714+0000 7f5e7159e740 1 client.15780 shutdown 2024-04-25T14:14:12.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.714+0000 7f5e7159e740 1 -- 172.21.15.96:0/2877644447 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcfd01c6b0 msgr2=0x55dcfd3e9a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.714+0000 7f5e7159e740 1 --2- 172.21.15.96:0/2877644447 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcfd01c6b0 0x55dcfd3e9a80 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5e580055d0 tx=0x7f5e58005600 comp rx=0 tx=0).stop 2024-04-25T14:14:12.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.714+0000 7f5e7159e740 1 -- 172.21.15.96:0/2877644447 shutdown_connections 2024-04-25T14:14:12.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.714+0000 7f5e7159e740 1 --2- 172.21.15.96:0/2877644447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dcfd03d390 0x55dcfd3f09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.714+0000 7f5e7159e740 1 --2- 172.21.15.96:0/2877644447 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55dcfcffbf60 0x55dcfd3f6e30 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.714+0000 7f5e7159e740 1 --2- 172.21.15.96:0/2877644447 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcfd01c6b0 0x55dcfd3e9a80 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.714+0000 7f5e7159e740 1 --2- 172.21.15.96:0/2877644447 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dcfd0144d0 0x55dcfd3e9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.714+0000 7f5e437fe640 0 client.15780 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:12.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.714+0000 7f5e7159e740 1 -- 172.21.15.96:0/2877644447 >> 172.21.15.96:0/2877644447 conn(0x55dcfd0314e0 msgr2=0x55dcfd3ddf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:12.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.714+0000 7f5e7159e740 1 -- 172.21.15.96:0/2877644447 shutdown_connections 2024-04-25T14:14:12.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.714+0000 7f5e7159e740 1 -- 172.21.15.96:0/2877644447 wait complete. 2024-04-25T14:14:12.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.715+0000 7f5e7159e740 20 client.15780 trim_cache size 0 max 16384 2024-04-25T14:14:12.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.728+0000 7f5e7159e740 1 Processor -- start 2024-04-25T14:14:12.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.728+0000 7f5e7159e740 1 -- start start 2024-04-25T14:14:12.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.728+0000 7f5e7159e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcfd0144d0 0x55dcfd3e42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.728+0000 7f5e7159e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dcfd01c6b0 0x55dcfd3e4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.728+0000 7f5e7159e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dcfd03d390 0x55dcfd3fc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.728+0000 7f5e7159e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5e641ab3a0 con 0x55dcfd0144d0 2024-04-25T14:14:12.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.728+0000 7f5e7159e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5e641acee0 con 0x55dcfd03d390 2024-04-25T14:14:12.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.728+0000 7f5e7159e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5e641aba70 con 0x55dcfd01c6b0 2024-04-25T14:14:12.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.728+0000 7f5e63fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcfd0144d0 0x55dcfd3e42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.728+0000 7f5e63fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcfd0144d0 0x55dcfd3e42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58776/0 (socket says 172.21.15.96:58776) 2024-04-25T14:14:12.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.728+0000 7f5e63fff640 1 -- 172.21.15.96:0/4208442407 learned_addr learned my addr 172.21.15.96:0/4208442407 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:12.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.728+0000 7f5e63fff640 1 -- 172.21.15.96:0/4208442407 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dcfd01c6b0 msgr2=0x55dcfd3e4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:12.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.728+0000 7f5e68c8b640 1 --2- 172.21.15.96:0/4208442407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dcfd03d390 0x55dcfd3fc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.728+0000 7f5e63fff640 1 --2- 172.21.15.96:0/4208442407 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dcfd01c6b0 0x55dcfd3e4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.728+0000 7f5e63fff640 1 -- 172.21.15.96:0/4208442407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dcfd03d390 msgr2=0x55dcfd3fc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:12.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.728+0000 7f5e63fff640 1 --2- 172.21.15.96:0/4208442407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dcfd03d390 0x55dcfd3fc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.728+0000 7f5e63fff640 1 -- 172.21.15.96:0/4208442407 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcfd3e7660 con 0x55dcfd0144d0 2024-04-25T14:14:12.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.729+0000 7f5e63fff640 1 --2- 172.21.15.96:0/4208442407 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcfd0144d0 0x55dcfd3e42d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5e540dc410 tx=0x7f5e540dc2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=411860534a579a5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:12.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.729+0000 7f5e49ffb640 1 -- 172.21.15.96:0/4208442407 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5401d040 con 0x55dcfd0144d0 2024-04-25T14:14:12.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.729+0000 7f5e49ffb640 1 -- 172.21.15.96:0/4208442407 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e540aa070 con 0x55dcfd0144d0 2024-04-25T14:14:12.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.729+0000 7f5e49ffb640 1 -- 172.21.15.96:0/4208442407 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e54004750 con 0x55dcfd0144d0 2024-04-25T14:14:12.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.729+0000 7f5e7159e740 1 -- 172.21.15.96:0/4208442407 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcfd0144d0 msgr2=0x55dcfd3e42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.729+0000 7f5e7159e740 1 --2- 172.21.15.96:0/4208442407 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcfd0144d0 0x55dcfd3e42d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5e540dc410 tx=0x7f5e540dc2e0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.729+0000 7f5e7159e740 1 -- 172.21.15.96:0/4208442407 shutdown_connections 2024-04-25T14:14:12.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.729+0000 7f5e7159e740 1 --2- 172.21.15.96:0/4208442407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dcfd03d390 0x55dcfd3fc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.729+0000 7f5e7159e740 1 --2- 172.21.15.96:0/4208442407 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dcfd01c6b0 0x55dcfd3e4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.729+0000 7f5e7159e740 1 --2- 172.21.15.96:0/4208442407 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcfd0144d0 0x55dcfd3e42d0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.729+0000 7f5e7159e740 1 -- 172.21.15.96:0/4208442407 >> 172.21.15.96:0/4208442407 conn(0x55dcfd0314e0 msgr2=0x55dcfd3df790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:12.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.730+0000 7f5e7159e740 1 -- 172.21.15.96:0/4208442407 shutdown_connections 2024-04-25T14:14:12.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.730+0000 7f5e7159e740 1 -- 172.21.15.96:0/4208442407 wait complete. 2024-04-25T14:14:12.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.730+0000 7f5e7159e740 1 -- start start 2024-04-25T14:14:12.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.730+0000 7f5e7159e740 1 Processor -- start 2024-04-25T14:14:12.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.730+0000 7f5e7159e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dcfd0144d0 0x55dcfd459080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.730+0000 7f5e7159e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcfd01c6b0 0x55dcfd45b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.730+0000 7f5e7159e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dcfd03d390 0x55dcfd45db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.730+0000 7f5e7159e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5e641ac810 con 0x55dcfd01c6b0 2024-04-25T14:14:12.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.730+0000 7f5e7159e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5e641ac140 con 0x55dcfd0144d0 2024-04-25T14:14:12.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.730+0000 7f5e7159e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5e641a9380 con 0x55dcfd03d390 2024-04-25T14:14:12.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.730+0000 7f5e7159e740 10 client.0 _ll_register_callbacks cb 0x7f5e58032b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:14:12.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.730+0000 7f5e637fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcfd01c6b0 0x55dcfd45b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.730+0000 7f5e637fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcfd01c6b0 0x55dcfd45b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58790/0 (socket says 172.21.15.96:58790) 2024-04-25T14:14:12.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.730+0000 7f5e637fe640 1 -- 172.21.15.96:0/3417159536 learned_addr learned my addr 172.21.15.96:0/3417159536 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:12.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.731+0000 7f5e63fff640 1 --2- 172.21.15.96:0/3417159536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dcfd0144d0 0x55dcfd459080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.731+0000 7f5e68c8b640 1 --2- 172.21.15.96:0/3417159536 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dcfd03d390 0x55dcfd45db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.731+0000 7f5e637fe640 1 -- 172.21.15.96:0/3417159536 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dcfd03d390 msgr2=0x55dcfd45db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:12.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.731+0000 7f5e637fe640 1 --2- 172.21.15.96:0/3417159536 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dcfd03d390 0x55dcfd45db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.731+0000 7f5e637fe640 1 -- 172.21.15.96:0/3417159536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dcfd0144d0 msgr2=0x55dcfd459080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:12.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.731+0000 7f5e637fe640 1 --2- 172.21.15.96:0/3417159536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dcfd0144d0 0x55dcfd459080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.731+0000 7f5e637fe640 1 -- 172.21.15.96:0/3417159536 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcfd45e110 con 0x55dcfd01c6b0 2024-04-25T14:14:12.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.731+0000 7f5e637fe640 1 --2- 172.21.15.96:0/3417159536 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcfd01c6b0 0x55dcfd45b5d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f5e580039e0 tx=0x7f5e580038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f2b39a8a10294aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:12.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.731+0000 7f5e437fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:14:12.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.731+0000 7f5e437fe640 1 -- 172.21.15.96:0/3417159536 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e58035070 con 0x55dcfd01c6b0 2024-04-25T14:14:12.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.731+0000 7f5e437fe640 1 -- 172.21.15.96:0/3417159536 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f5e58004060 con 0x55dcfd01c6b0 2024-04-25T14:14:12.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.731+0000 7f5e7159e740 1 -- 172.21.15.96:0/3417159536 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e540aa070 con 0x55dcfd01c6b0 2024-04-25T14:14:12.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.731+0000 7f5e437fe640 1 -- 172.21.15.96:0/3417159536 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5e5803f040 con 0x55dcfd01c6b0 2024-04-25T14:14:12.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.731+0000 7f5e7159e740 10 client.15786 fetch_fsmap learned FSMap version 26 2024-04-25T14:14:12.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.731+0000 7f5e7159e740 1 -- 172.21.15.96:0/3417159536 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55dcfd45f390 con 0x55dcfd01c6b0 2024-04-25T14:14:12.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.731+0000 7f5e437fe640 1 -- 172.21.15.96:0/3417159536 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e540aa070 con 0x55dcfd01c6b0 2024-04-25T14:14:12.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.732+0000 7f5e437fe640 1 -- 172.21.15.96:0/3417159536 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5e58031c10 con 0x55dcfd01c6b0 2024-04-25T14:14:12.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.732+0000 7f5e7159e740 10 client.15786 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:14:12.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.732+0000 7f5e7159e740 10 client.15786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:14:12.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.732+0000 7f5e7159e740 20 client.15786 populate_metadata read hostname 'smithi096' 2024-04-25T14:14:12.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.732+0000 7f5e7159e740 1 --2- 172.21.15.96:0/3417159536 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55dcfcfd13a0 0x55dcfd461c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.732+0000 7f5e7159e740 4 client.15786 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:14:12.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.732+0000 7f5e7159e740 1 -- 172.21.15.96:0/3417159536 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dcfd2c6610 con 0x55dcfcfd13a0 2024-04-25T14:14:12.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.732+0000 7f5e63fff640 1 --2- 172.21.15.96:0/3417159536 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55dcfcfd13a0 0x55dcfd461c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.732+0000 7f5e437fe640 10 client.15786 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:12.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.732+0000 7f5e63fff640 1 --2- 172.21.15.96:0/3417159536 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55dcfcfd13a0 0x55dcfd461c90 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b95275dcc428f42b server_cookie=27d467cab4834c30 in_seq=0 out_seq=0 2024-04-25T14:14:12.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.732+0000 7f5e437fe640 1 -- 172.21.15.96:0/3417159536 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dcfd2c6610 con 0x55dcfcfd13a0 2024-04-25T14:14:12.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.732+0000 7f5e437fe640 10 client.15786 handle_command_reply: tid=0 2024-04-25T14:14:12.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.732+0000 7f5e7159e740 1 client.15786 shutdown 2024-04-25T14:14:12.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.733+0000 7f5e7159e740 1 -- 172.21.15.96:0/3417159536 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcfd01c6b0 msgr2=0x55dcfd45b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.733+0000 7f5e7159e740 1 --2- 172.21.15.96:0/3417159536 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcfd01c6b0 0x55dcfd45b5d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f5e580039e0 tx=0x7f5e580038a0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.733+0000 7f5e7159e740 1 -- 172.21.15.96:0/3417159536 shutdown_connections 2024-04-25T14:14:12.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.733+0000 7f5e7159e740 1 --2- 172.21.15.96:0/3417159536 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dcfd03d390 0x55dcfd45db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.733+0000 7f5e7159e740 1 --2- 172.21.15.96:0/3417159536 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55dcfcfd13a0 0x55dcfd461c90 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.733+0000 7f5e7159e740 1 --2- 172.21.15.96:0/3417159536 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcfd01c6b0 0x55dcfd45b5d0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.733+0000 7f5e7159e740 1 --2- 172.21.15.96:0/3417159536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dcfd0144d0 0x55dcfd459080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.733+0000 7f5e437fe640 0 client.15786 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:12.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.733+0000 7f5e7159e740 1 -- 172.21.15.96:0/3417159536 >> 172.21.15.96:0/3417159536 conn(0x55dcfd0314e0 msgr2=0x55dcfd3e42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:12.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.733+0000 7f5e7159e740 1 -- 172.21.15.96:0/3417159536 shutdown_connections 2024-04-25T14:14:12.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.733+0000 7f5e7159e740 1 -- 172.21.15.96:0/3417159536 wait complete. 2024-04-25T14:14:12.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.733+0000 7f5e7159e740 20 client.15786 trim_cache size 0 max 16384 2024-04-25T14:14:12.740 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-04-25T14:14:12.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.733+0000 7f5e6a6ee640 1 -- 172.21.15.96:0/3381590783 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f5e4c0afaf0 msgr2=0x7f5e4c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.733+0000 7f5e6a6ee640 1 --2- 172.21.15.96:0/3381590783 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f5e4c0afaf0 0x7f5e4c0b2000 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x55dcfd3e0860 tx=0x7f5e5c009600 comp rx=0 tx=0).stop 2024-04-25T14:14:12.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.733+0000 7f5e6a6ee640 1 -- 172.21.15.96:0/3381590783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e64102f60 msgr2=0x7f5e6419a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.733+0000 7f5e6a6ee640 1 --2- 172.21.15.96:0/3381590783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e64102f60 0x7f5e6419a270 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5e5400c970 tx=0x7f5e5400ce30 comp rx=0 tx=0).stop 2024-04-25T14:14:12.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.733+0000 7f5e6a6ee640 1 -- 172.21.15.96:0/3381590783 shutdown_connections 2024-04-25T14:14:12.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.733+0000 7f5e6a6ee640 1 --2- 172.21.15.96:0/3381590783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e64104480 0x7f5e641a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.733+0000 7f5e6a6ee640 1 --2- 172.21.15.96:0/3381590783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5e64103940 0x7f5e6419a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.733+0000 7f5e6a6ee640 1 --2- 172.21.15.96:0/3381590783 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f5e4c0afaf0 0x7f5e4c0b2000 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.733+0000 7f5e6a6ee640 1 --2- 172.21.15.96:0/3381590783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e64102f60 0x7f5e6419a270 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.733+0000 7f5e6a6ee640 1 -- 172.21.15.96:0/3381590783 >> 172.21.15.96:0/3381590783 conn(0x7f5e640fd380 msgr2=0x7f5e64104ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:12.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.733+0000 7f5e6a6ee640 1 -- 172.21.15.96:0/3381590783 shutdown_connections 2024-04-25T14:14:12.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:12.733+0000 7f5e6a6ee640 1 -- 172.21.15.96:0/3381590783 wait complete. 2024-04-25T14:14:12.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:12 smithi097 ceph-mon[32720]: pgmap v363: 97 pgs: 97 active+clean; 657 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-25T14:14:12.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:12 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/313502187' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:14:12.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:12 smithi151 ceph-mon[32990]: pgmap v363: 97 pgs: 97 active+clean; 657 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-25T14:14:12.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:12 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/313502187' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:14:13.744 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T14:14:13.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:14:13.745 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T14:14:13.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.871+0000 7fdd282ca640 1 -- 172.21.15.96:0/1724112192 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd20103af0 msgr2=0x7fdd201047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:13.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.871+0000 7fdd282ca640 1 --2- 172.21.15.96:0/1724112192 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd20103af0 0x7fdd201047f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c009ca0 tx=0x7fdd1c02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.871+0000 7fdd282ca640 1 -- 172.21.15.96:0/1724112192 shutdown_connections 2024-04-25T14:14:13.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.871+0000 7fdd282ca640 1 --2- 172.21.15.96:0/1724112192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd20109a30 0x7fdd2010be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.871+0000 7fdd282ca640 1 --2- 172.21.15.96:0/1724112192 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd20104d30 0x7fdd20105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.871+0000 7fdd282ca640 1 --2- 172.21.15.96:0/1724112192 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd20103af0 0x7fdd201047f0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.871+0000 7fdd282ca640 1 -- 172.21.15.96:0/1724112192 >> 172.21.15.96:0/1724112192 conn(0x7fdd200fece0 msgr2=0x7fdd20101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:13.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.871+0000 7fdd282ca640 1 -- 172.21.15.96:0/1724112192 shutdown_connections 2024-04-25T14:14:13.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.871+0000 7fdd282ca640 1 -- 172.21.15.96:0/1724112192 wait complete. 2024-04-25T14:14:13.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.871+0000 7fdd282ca640 1 Processor -- start 2024-04-25T14:14:13.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.871+0000 7fdd282ca640 1 -- start start 2024-04-25T14:14:13.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.871+0000 7fdd282ca640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd20103af0 0x7fdd2019a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:13.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.872+0000 7fdd282ca640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd20104d30 0x7fdd2019a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:13.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.872+0000 7fdd282ca640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd20109a30 0x7fdd201a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:13.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.872+0000 7fdd282ca640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2010f620 con 0x7fdd20103af0 2024-04-25T14:14:13.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.872+0000 7fdd282ca640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2010f4a0 con 0x7fdd20104d30 2024-04-25T14:14:13.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.872+0000 7fdd282ca640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2010f7a0 con 0x7fdd20109a30 2024-04-25T14:14:13.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.872+0000 7fdd26066640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd20103af0 0x7fdd2019a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:13.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.872+0000 7fdd26066640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd20103af0 0x7fdd2019a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58822/0 (socket says 172.21.15.96:58822) 2024-04-25T14:14:13.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.872+0000 7fdd26066640 1 -- 172.21.15.96:0/1698893262 learned_addr learned my addr 172.21.15.96:0/1698893262 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:13.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.872+0000 7fdd25865640 1 --2- 172.21.15.96:0/1698893262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd20104d30 0x7fdd2019a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:13.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.872+0000 7fdd26867640 1 --2- 172.21.15.96:0/1698893262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd20109a30 0x7fdd201a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:13.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.872+0000 7fdd26066640 1 -- 172.21.15.96:0/1698893262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd20109a30 msgr2=0x7fdd201a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:13.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.872+0000 7fdd26066640 1 --2- 172.21.15.96:0/1698893262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd20109a30 0x7fdd201a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.872+0000 7fdd26066640 1 -- 172.21.15.96:0/1698893262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd20104d30 msgr2=0x7fdd2019a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:13.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.872+0000 7fdd26066640 1 --2- 172.21.15.96:0/1698893262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd20104d30 0x7fdd2019a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.872+0000 7fdd26066640 1 -- 172.21.15.96:0/1698893262 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd201a1d70 con 0x7fdd20103af0 2024-04-25T14:14:13.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.872+0000 7fdd26066640 1 --2- 172.21.15.96:0/1698893262 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd20103af0 0x7fdd2019a230 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c000f80 tx=0x7fdd1c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:13.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.872+0000 7fdd177fe640 1 -- 172.21.15.96:0/1698893262 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c02f380 con 0x7fdd20103af0 2024-04-25T14:14:13.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.872+0000 7fdd177fe640 1 -- 172.21.15.96:0/1698893262 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd1c0317c0 con 0x7fdd20103af0 2024-04-25T14:14:13.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.872+0000 7fdd282ca640 1 -- 172.21.15.96:0/1698893262 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd201a2000 con 0x7fdd20103af0 2024-04-25T14:14:13.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.872+0000 7fdd177fe640 1 -- 172.21.15.96:0/1698893262 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c02fc50 con 0x7fdd20103af0 2024-04-25T14:14:13.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.873+0000 7fdd282ca640 1 -- 172.21.15.96:0/1698893262 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd201a2550 con 0x7fdd20103af0 2024-04-25T14:14:13.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.875+0000 7fdd177fe640 1 -- 172.21.15.96:0/1698893262 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fdd1c004040 con 0x7fdd20103af0 2024-04-25T14:14:13.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.875+0000 7fdd2f179740 1 Processor -- start 2024-04-25T14:14:13.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.875+0000 7fdd2f179740 1 -- start start 2024-04-25T14:14:13.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.876+0000 7fdd2f179740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56263a765410 0x56263ab0a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:13.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.876+0000 7fdd2f179740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56263a73c550 0x56263ab0ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:13.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.876+0000 7fdd2f179740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263a744730 0x56263ab11d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:13.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.876+0000 7fdd2f179740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56263aa6bb00 con 0x56263a744730 2024-04-25T14:14:13.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.876+0000 7fdd2f179740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56263aa07920 con 0x56263a73c550 2024-04-25T14:14:13.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.876+0000 7fdd2f179740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56263aa67ac0 con 0x56263a765410 2024-04-25T14:14:13.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.876+0000 7fdd177fe640 1 --2- 172.21.15.96:0/1698893262 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fdd0c0afaf0 0x7fdd0c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:13.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.876+0000 7fdd25865640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263a744730 0x56263ab11d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:13.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.876+0000 7fdd25865640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263a744730 0x56263ab11d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58826/0 (socket says 172.21.15.96:58826) 2024-04-25T14:14:13.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.876+0000 7fdd25865640 1 -- 172.21.15.96:0/574791570 learned_addr learned my addr 172.21.15.96:0/574791570 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:13.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.876+0000 7fdd26867640 1 --2- 172.21.15.96:0/1698893262 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fdd0c0afaf0 0x7fdd0c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:13.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.876+0000 7fdd26867640 1 --2- 172.21.15.96:0/574791570 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56263a765410 0x56263ab0a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:13.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.876+0000 7fdd26066640 1 --2- 172.21.15.96:0/574791570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56263a73c550 0x56263ab0ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:13.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.876+0000 7fdd177fe640 1 -- 172.21.15.96:0/1698893262 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fdd1c100380 con 0x7fdd20103af0 2024-04-25T14:14:13.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.876+0000 7fdd25865640 1 -- 172.21.15.96:0/574791570 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56263a765410 msgr2=0x56263ab0a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:13.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.876+0000 7fdd25865640 1 --2- 172.21.15.96:0/574791570 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56263a765410 0x56263ab0a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.876+0000 7fdd25865640 1 -- 172.21.15.96:0/574791570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56263a73c550 msgr2=0x56263ab0ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:13.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.876+0000 7fdd25865640 1 --2- 172.21.15.96:0/574791570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56263a73c550 0x56263ab0ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.876+0000 7fdd25865640 1 -- 172.21.15.96:0/574791570 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56263a8457e0 con 0x56263a744730 2024-04-25T14:14:13.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.876+0000 7fdd25865640 1 --2- 172.21.15.96:0/574791570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263a744730 0x56263ab11d30 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fdd10009c20 tx=0x7fdd100117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c8edda9dcba7016 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:13.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.876+0000 7fdd26867640 1 --2- 172.21.15.96:0/1698893262 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fdd0c0afaf0 0x7fdd0c0b2000 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fdd1800cfd0 tx=0x7fdd1800c2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:13.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.877+0000 7fdd14bfe640 1 -- 172.21.15.96:0/574791570 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd10012450 con 0x56263a744730 2024-04-25T14:14:13.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.877+0000 7fdd14bfe640 1 -- 172.21.15.96:0/574791570 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd10012aa0 con 0x56263a744730 2024-04-25T14:14:13.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.877+0000 7fdd14bfe640 1 -- 172.21.15.96:0/574791570 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1000faa0 con 0x56263a744730 2024-04-25T14:14:13.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.877+0000 7fdd2f179740 1 -- 172.21.15.96:0/574791570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263a744730 msgr2=0x56263ab11d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:13.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.877+0000 7fdd2f179740 1 --2- 172.21.15.96:0/574791570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263a744730 0x56263ab11d30 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fdd10009c20 tx=0x7fdd100117b0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.877+0000 7fdd2f179740 1 -- 172.21.15.96:0/574791570 shutdown_connections 2024-04-25T14:14:13.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.877+0000 7fdd2f179740 1 --2- 172.21.15.96:0/574791570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263a744730 0x56263ab11d30 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.877+0000 7fdd2f179740 1 --2- 172.21.15.96:0/574791570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56263a73c550 0x56263ab0ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.877+0000 7fdd2f179740 1 --2- 172.21.15.96:0/574791570 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56263a765410 0x56263ab0a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.877+0000 7fdd2f179740 1 -- 172.21.15.96:0/574791570 >> 172.21.15.96:0/574791570 conn(0x56263a759560 msgr2=0x56263ab01e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:13.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.878+0000 7fdd2f179740 1 -- 172.21.15.96:0/574791570 shutdown_connections 2024-04-25T14:14:13.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.878+0000 7fdd2f179740 1 -- 172.21.15.96:0/574791570 wait complete. 2024-04-25T14:14:13.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.878+0000 7fdd2f179740 1 -- start start 2024-04-25T14:14:13.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.878+0000 7fdd2f179740 1 Processor -- start 2024-04-25T14:14:13.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.878+0000 7fdd2f179740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263a73c550 0x56263ab146e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:13.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.878+0000 7fdd2f179740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56263a744730 0x56263ab14c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:13.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.878+0000 7fdd2f179740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56263a765410 0x56263ab1bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:13.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.878+0000 7fdd2f179740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56263ab12780 con 0x56263a73c550 2024-04-25T14:14:13.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.878+0000 7fdd2f179740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56263aa055d0 con 0x56263a744730 2024-04-25T14:14:13.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.878+0000 7fdd2f179740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56263ab12900 con 0x56263a765410 2024-04-25T14:14:13.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.878+0000 7fdd2f179740 10 client.0 _ll_register_callbacks cb 0x56263a9f3080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:14:13.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.878+0000 7fdd26066640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263a73c550 0x56263ab146e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:13.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.878+0000 7fdd26066640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263a73c550 0x56263ab146e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58828/0 (socket says 172.21.15.96:58828) 2024-04-25T14:14:13.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.878+0000 7fdd26066640 1 -- 172.21.15.96:0/1656072850 learned_addr learned my addr 172.21.15.96:0/1656072850 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:13.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.878+0000 7fdd25865640 1 --2- 172.21.15.96:0/1656072850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56263a744730 0x56263ab14c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:13.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.879+0000 7fdd26867640 1 --2- 172.21.15.96:0/1656072850 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56263a765410 0x56263ab1bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:13.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.879+0000 7fdd26066640 1 -- 172.21.15.96:0/1656072850 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56263a765410 msgr2=0x56263ab1bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:13.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.879+0000 7fdd26066640 1 --2- 172.21.15.96:0/1656072850 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56263a765410 0x56263ab1bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.879+0000 7fdd26066640 1 -- 172.21.15.96:0/1656072850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56263a744730 msgr2=0x56263ab14c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:13.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.879+0000 7fdd26066640 1 --2- 172.21.15.96:0/1656072850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56263a744730 0x56263ab14c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.879+0000 7fdd26066640 1 -- 172.21.15.96:0/1656072850 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56263a8a5f70 con 0x56263a73c550 2024-04-25T14:14:13.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.879+0000 7fdd25865640 1 --2- 172.21.15.96:0/1656072850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56263a744730 0x56263ab14c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:14:13.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.879+0000 7fdd26066640 1 --2- 172.21.15.96:0/1656072850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263a73c550 0x56263ab146e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c0c87d0 tx=0x7fdd1c02f780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad69795d19a5303 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:13.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.879+0000 7fdd09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:14:13.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.879+0000 7fdd09ffb640 1 -- 172.21.15.96:0/1656072850 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c0c8f40 con 0x56263a73c550 2024-04-25T14:14:13.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.879+0000 7fdd09ffb640 1 -- 172.21.15.96:0/1656072850 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fdd1c004780 con 0x56263a73c550 2024-04-25T14:14:13.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.879+0000 7fdd2f179740 1 -- 172.21.15.96:0/1656072850 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd10012aa0 con 0x56263a73c550 2024-04-25T14:14:13.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.879+0000 7fdd09ffb640 1 -- 172.21.15.96:0/1656072850 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdd1c08f1c0 con 0x56263a73c550 2024-04-25T14:14:13.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.879+0000 7fdd2f179740 10 client.15798 fetch_fsmap learned FSMap version 26 2024-04-25T14:14:13.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.879+0000 7fdd2f179740 1 -- 172.21.15.96:0/1656072850 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56263a73d170 con 0x56263a73c550 2024-04-25T14:14:13.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.879+0000 7fdd09ffb640 1 -- 172.21.15.96:0/1656072850 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd10012aa0 con 0x56263a73c550 2024-04-25T14:14:13.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.879+0000 7fdd09ffb640 1 -- 172.21.15.96:0/1656072850 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fdd1c0afce0 con 0x56263a73c550 2024-04-25T14:14:13.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.880+0000 7fdd2f179740 10 client.15798 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:14:13.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.880+0000 7fdd2f179740 10 client.15798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:14:13.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.880+0000 7fdd2f179740 20 client.15798 populate_metadata read hostname 'smithi096' 2024-04-25T14:14:13.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.880+0000 7fdd2f179740 1 --2- 172.21.15.96:0/1656072850 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x56263a7241f0 0x56263ab21f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:13.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.880+0000 7fdd2f179740 4 client.15798 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:14:13.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.880+0000 7fdd2f179740 1 -- 172.21.15.96:0/1656072850 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56263aa79710 con 0x56263a7241f0 2024-04-25T14:14:13.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.880+0000 7fdd25865640 1 --2- 172.21.15.96:0/1656072850 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x56263a7241f0 0x56263ab21f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:13.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.880+0000 7fdd09ffb640 10 client.15798 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:13.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.880+0000 7fdd25865640 1 --2- 172.21.15.96:0/1656072850 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x56263a7241f0 0x56263ab21f00 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27daef788dd48e83 server_cookie=9959aac8b652a1f2 in_seq=0 out_seq=0 2024-04-25T14:14:13.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.881+0000 7fdd09ffb640 1 -- 172.21.15.96:0/1656072850 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56263aa79710 con 0x56263a7241f0 2024-04-25T14:14:13.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.881+0000 7fdd09ffb640 10 client.15798 handle_command_reply: tid=0 2024-04-25T14:14:13.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.881+0000 7fdd2f179740 1 client.15798 shutdown 2024-04-25T14:14:13.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.881+0000 7fdd2f179740 1 -- 172.21.15.96:0/1656072850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263a73c550 msgr2=0x56263ab146e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:13.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.881+0000 7fdd2f179740 1 --2- 172.21.15.96:0/1656072850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263a73c550 0x56263ab146e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c0c87d0 tx=0x7fdd1c02f780 comp rx=0 tx=0).stop 2024-04-25T14:14:13.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.881+0000 7fdd2f179740 1 -- 172.21.15.96:0/1656072850 shutdown_connections 2024-04-25T14:14:13.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.881+0000 7fdd2f179740 1 --2- 172.21.15.96:0/1656072850 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56263a765410 0x56263ab1bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.881+0000 7fdd2f179740 1 --2- 172.21.15.96:0/1656072850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56263a744730 0x56263ab14c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.881+0000 7fdd2f179740 1 --2- 172.21.15.96:0/1656072850 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x56263a7241f0 0x56263ab21f00 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.881+0000 7fdd2f179740 1 --2- 172.21.15.96:0/1656072850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263a73c550 0x56263ab146e0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.881+0000 7fdd09ffb640 0 client.15798 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:13.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.881+0000 7fdd2f179740 1 -- 172.21.15.96:0/1656072850 >> 172.21.15.96:0/1656072850 conn(0x56263a759560 msgr2=0x56263ab00ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:13.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.881+0000 7fdd2f179740 1 -- 172.21.15.96:0/1656072850 shutdown_connections 2024-04-25T14:14:13.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.881+0000 7fdd2f179740 1 -- 172.21.15.96:0/1656072850 wait complete. 2024-04-25T14:14:13.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.881+0000 7fdd2f179740 20 client.15798 trim_cache size 0 max 16384 2024-04-25T14:14:13.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.899+0000 7fdd2f179740 1 Processor -- start 2024-04-25T14:14:13.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.899+0000 7fdd2f179740 1 -- start start 2024-04-25T14:14:13.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.899+0000 7fdd2f179740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56263a73c550 0x56263ab05340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:13.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.899+0000 7fdd2f179740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56263a744730 0x56263ab05880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:13.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.899+0000 7fdd2f179740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263a765410 0x56263ab1bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:13.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.899+0000 7fdd2f179740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdd201a9150 con 0x56263a765410 2024-04-25T14:14:13.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.899+0000 7fdd2f179740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdd201aac90 con 0x56263a744730 2024-04-25T14:14:13.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.899+0000 7fdd2f179740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdd201a9820 con 0x56263a73c550 2024-04-25T14:14:13.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.899+0000 7fdd26867640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263a765410 0x56263ab1bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:13.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.900+0000 7fdd26867640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263a765410 0x56263ab1bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58832/0 (socket says 172.21.15.96:58832) 2024-04-25T14:14:13.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.900+0000 7fdd26867640 1 -- 172.21.15.96:0/550376753 learned_addr learned my addr 172.21.15.96:0/550376753 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:13.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.900+0000 7fdd25865640 1 --2- 172.21.15.96:0/550376753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56263a744730 0x56263ab05880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:13.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.900+0000 7fdd26066640 1 --2- 172.21.15.96:0/550376753 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56263a73c550 0x56263ab05340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:13.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.900+0000 7fdd26867640 1 -- 172.21.15.96:0/550376753 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56263a73c550 msgr2=0x56263ab05340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:13.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.900+0000 7fdd26867640 1 --2- 172.21.15.96:0/550376753 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56263a73c550 0x56263ab05340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.900+0000 7fdd26867640 1 -- 172.21.15.96:0/550376753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56263a744730 msgr2=0x56263ab05880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:13.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.900+0000 7fdd26867640 1 --2- 172.21.15.96:0/550376753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56263a744730 0x56263ab05880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.900+0000 7fdd26867640 1 -- 172.21.15.96:0/550376753 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56263ab029c0 con 0x56263a765410 2024-04-25T14:14:13.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.900+0000 7fdd26867640 1 --2- 172.21.15.96:0/550376753 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263a765410 0x56263ab1bb10 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fdd1800cfa0 tx=0x7fdd1800b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e5ba4428b8de04a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:13.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.900+0000 7fdd0bfff640 1 -- 172.21.15.96:0/550376753 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd18026030 con 0x56263a765410 2024-04-25T14:14:13.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.900+0000 7fdd0bfff640 1 -- 172.21.15.96:0/550376753 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd18012900 con 0x56263a765410 2024-04-25T14:14:13.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.900+0000 7fdd0bfff640 1 -- 172.21.15.96:0/550376753 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd180228a0 con 0x56263a765410 2024-04-25T14:14:13.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.901+0000 7fdd2f179740 1 -- 172.21.15.96:0/550376753 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263a765410 msgr2=0x56263ab1bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:13.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.901+0000 7fdd2f179740 1 --2- 172.21.15.96:0/550376753 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263a765410 0x56263ab1bb10 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fdd1800cfa0 tx=0x7fdd1800b000 comp rx=0 tx=0).stop 2024-04-25T14:14:13.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.901+0000 7fdd2f179740 1 -- 172.21.15.96:0/550376753 shutdown_connections 2024-04-25T14:14:13.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.901+0000 7fdd2f179740 1 --2- 172.21.15.96:0/550376753 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263a765410 0x56263ab1bb10 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.901+0000 7fdd2f179740 1 --2- 172.21.15.96:0/550376753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56263a744730 0x56263ab05880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.901+0000 7fdd2f179740 1 --2- 172.21.15.96:0/550376753 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56263a73c550 0x56263ab05340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.901+0000 7fdd2f179740 1 -- 172.21.15.96:0/550376753 >> 172.21.15.96:0/550376753 conn(0x56263a759560 msgr2=0x56263ab04c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:13.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.901+0000 7fdd2f179740 1 -- 172.21.15.96:0/550376753 shutdown_connections 2024-04-25T14:14:13.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.901+0000 7fdd2f179740 1 -- 172.21.15.96:0/550376753 wait complete. 2024-04-25T14:14:13.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.901+0000 7fdd2f179740 1 -- start start 2024-04-25T14:14:13.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.901+0000 7fdd2f179740 1 Processor -- start 2024-04-25T14:14:13.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.902+0000 7fdd2f179740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263a73c550 0x56263ab84280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:13.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.902+0000 7fdd2f179740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56263a744730 0x56263ab867d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:13.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.902+0000 7fdd2f179740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56263a765410 0x56263ab88d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:13.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.902+0000 7fdd2f179740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdd201aa5c0 con 0x56263a73c550 2024-04-25T14:14:13.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.902+0000 7fdd2f179740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdd201a9ef0 con 0x56263a744730 2024-04-25T14:14:13.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.902+0000 7fdd2f179740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdd201a7130 con 0x56263a765410 2024-04-25T14:14:13.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.902+0000 7fdd2f179740 10 client.0 _ll_register_callbacks cb 0x7fdd1c0c47c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:14:13.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.902+0000 7fdd26066640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263a73c550 0x56263ab84280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:13.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.902+0000 7fdd26066640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263a73c550 0x56263ab84280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58846/0 (socket says 172.21.15.96:58846) 2024-04-25T14:14:13.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.902+0000 7fdd26066640 1 -- 172.21.15.96:0/2674882632 learned_addr learned my addr 172.21.15.96:0/2674882632 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:13.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.902+0000 7fdd25865640 1 --2- 172.21.15.96:0/2674882632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56263a744730 0x56263ab867d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:13.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.902+0000 7fdd26867640 1 --2- 172.21.15.96:0/2674882632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56263a765410 0x56263ab88d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:13.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.902+0000 7fdd26066640 1 -- 172.21.15.96:0/2674882632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56263a765410 msgr2=0x56263ab88d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:13.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.902+0000 7fdd26066640 1 --2- 172.21.15.96:0/2674882632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56263a765410 0x56263ab88d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.902+0000 7fdd26066640 1 -- 172.21.15.96:0/2674882632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56263a744730 msgr2=0x56263ab867d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:13.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.902+0000 7fdd26066640 1 --2- 172.21.15.96:0/2674882632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56263a744730 0x56263ab867d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.902+0000 7fdd26066640 1 -- 172.21.15.96:0/2674882632 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56263ab892b0 con 0x56263a73c550 2024-04-25T14:14:13.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.902+0000 7fdd25865640 1 --2- 172.21.15.96:0/2674882632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56263a744730 0x56263ab867d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:14:13.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.902+0000 7fdd26066640 1 --2- 172.21.15.96:0/2674882632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263a73c550 0x56263ab84280 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c0c4870 tx=0x7fdd1c004990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13291bc113161e18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:13.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.902+0000 7fdd09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:14:13.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.902+0000 7fdd09ffb640 1 -- 172.21.15.96:0/2674882632 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c009540 con 0x56263a73c550 2024-04-25T14:14:13.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.902+0000 7fdd09ffb640 1 -- 172.21.15.96:0/2674882632 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fdd1c0b1110 con 0x56263a73c550 2024-04-25T14:14:13.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.902+0000 7fdd2f179740 1 -- 172.21.15.96:0/2674882632 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd18012900 con 0x56263a73c550 2024-04-25T14:14:13.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.902+0000 7fdd09ffb640 1 -- 172.21.15.96:0/2674882632 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdd1c0c8f40 con 0x56263a73c550 2024-04-25T14:14:13.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.902+0000 7fdd2f179740 10 client.15804 fetch_fsmap learned FSMap version 26 2024-04-25T14:14:13.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.902+0000 7fdd2f179740 1 -- 172.21.15.96:0/2674882632 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56263ab8a5e0 con 0x56263a73c550 2024-04-25T14:14:13.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.903+0000 7fdd09ffb640 1 -- 172.21.15.96:0/2674882632 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd18012900 con 0x56263a73c550 2024-04-25T14:14:13.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.903+0000 7fdd09ffb640 1 -- 172.21.15.96:0/2674882632 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fdd1c0afce0 con 0x56263a73c550 2024-04-25T14:14:13.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.903+0000 7fdd2f179740 10 client.15804 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:14:13.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.903+0000 7fdd2f179740 10 client.15804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:14:13.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.903+0000 7fdd2f179740 20 client.15804 populate_metadata read hostname 'smithi096' 2024-04-25T14:14:13.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.903+0000 7fdd2f179740 1 --2- 172.21.15.96:0/2674882632 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x56263a6f9630 0x56263ab8ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:13.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.903+0000 7fdd2f179740 4 client.15804 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T14:14:13.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.903+0000 7fdd2f179740 1 -- 172.21.15.96:0/2674882632 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56263aa73580 con 0x56263a6f9630 2024-04-25T14:14:13.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.903+0000 7fdd25865640 1 --2- 172.21.15.96:0/2674882632 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x56263a6f9630 0x56263ab8ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:13.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.903+0000 7fdd09ffb640 10 client.15804 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:13.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.903+0000 7fdd25865640 1 --2- 172.21.15.96:0/2674882632 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x56263a6f9630 0x56263ab8ce00 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c28b94aaa96f90d4 server_cookie=77d1fab615ff5dc2 in_seq=0 out_seq=0 2024-04-25T14:14:13.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.905+0000 7fdd09ffb640 1 -- 172.21.15.96:0/2674882632 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56263aa73580 con 0x56263a6f9630 2024-04-25T14:14:13.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.905+0000 7fdd09ffb640 10 client.15804 handle_command_reply: tid=0 2024-04-25T14:14:13.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.905+0000 7fdd2f179740 1 client.15804 shutdown 2024-04-25T14:14:13.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.905+0000 7fdd2f179740 1 -- 172.21.15.96:0/2674882632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263a73c550 msgr2=0x56263ab84280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:13.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.905+0000 7fdd2f179740 1 --2- 172.21.15.96:0/2674882632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263a73c550 0x56263ab84280 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c0c4870 tx=0x7fdd1c004990 comp rx=0 tx=0).stop 2024-04-25T14:14:13.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.905+0000 7fdd2f179740 1 -- 172.21.15.96:0/2674882632 shutdown_connections 2024-04-25T14:14:13.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.905+0000 7fdd2f179740 1 --2- 172.21.15.96:0/2674882632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56263a765410 0x56263ab88d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.905+0000 7fdd2f179740 1 --2- 172.21.15.96:0/2674882632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56263a744730 0x56263ab867d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.905+0000 7fdd2f179740 1 --2- 172.21.15.96:0/2674882632 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x56263a6f9630 0x56263ab8ce00 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.905+0000 7fdd2f179740 1 --2- 172.21.15.96:0/2674882632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56263a73c550 0x56263ab84280 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.905+0000 7fdd09ffb640 0 client.15804 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:13.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.905+0000 7fdd2f179740 1 -- 172.21.15.96:0/2674882632 >> 172.21.15.96:0/2674882632 conn(0x56263a759560 msgr2=0x56263ab05660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:13.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.905+0000 7fdd2f179740 1 -- 172.21.15.96:0/2674882632 shutdown_connections 2024-04-25T14:14:13.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.905+0000 7fdd2f179740 1 -- 172.21.15.96:0/2674882632 wait complete. 2024-04-25T14:14:13.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.905+0000 7fdd2f179740 20 client.15804 trim_cache size 0 max 16384 2024-04-25T14:14:13.912 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T14:14:13.912 INFO:teuthology.orchestra.run.smithi096.stdout: "return_code": 0, 2024-04-25T14:14:13.912 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_tag": "28b7b7cd-cdd8-42dc-96f1-37c0dc0150ad", 2024-04-25T14:14:13.912 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": "asynchronous" 2024-04-25T14:14:13.912 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T14:14:13.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.905+0000 7fdd282ca640 1 -- 172.21.15.96:0/1698893262 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fdd0c0afaf0 msgr2=0x7fdd0c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:13.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.905+0000 7fdd282ca640 1 --2- 172.21.15.96:0/1698893262 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fdd0c0afaf0 0x7fdd0c0b2000 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fdd1800cfd0 tx=0x7fdd1800c2d0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.905+0000 7fdd282ca640 1 -- 172.21.15.96:0/1698893262 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd20103af0 msgr2=0x7fdd2019a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:13.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.905+0000 7fdd282ca640 1 --2- 172.21.15.96:0/1698893262 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd20103af0 0x7fdd2019a230 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c000f80 tx=0x7fdd1c003ba0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.905+0000 7fdd282ca640 1 -- 172.21.15.96:0/1698893262 shutdown_connections 2024-04-25T14:14:13.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.905+0000 7fdd282ca640 1 --2- 172.21.15.96:0/1698893262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd20109a30 0x7fdd201a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.905+0000 7fdd282ca640 1 --2- 172.21.15.96:0/1698893262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd20104d30 0x7fdd2019a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.905+0000 7fdd282ca640 1 --2- 172.21.15.96:0/1698893262 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fdd0c0afaf0 0x7fdd0c0b2000 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.905+0000 7fdd282ca640 1 --2- 172.21.15.96:0/1698893262 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd20103af0 0x7fdd2019a230 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.905+0000 7fdd282ca640 1 -- 172.21.15.96:0/1698893262 >> 172.21.15.96:0/1698893262 conn(0x7fdd200fece0 msgr2=0x7fdd20105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:13.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.906+0000 7fdd282ca640 1 -- 172.21.15.96:0/1698893262 shutdown_connections 2024-04-25T14:14:13.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:13.906+0000 7fdd282ca640 1 -- 172.21.15.96:0/1698893262 wait complete. 2024-04-25T14:14:13.916 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:14:13.916 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:14:14.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.026+0000 7eff2cba8640 1 -- 172.21.15.96:0/1064577498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff281033d0 msgr2=0x7eff281037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:14.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.026+0000 7eff2cba8640 1 --2- 172.21.15.96:0/1064577498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff281033d0 0x7eff281037e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7eff1c009900 tx=0x7eff1c02a410 comp rx=0 tx=0).stop 2024-04-25T14:14:14.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.026+0000 7eff2cba8640 1 -- 172.21.15.96:0/1064577498 shutdown_connections 2024-04-25T14:14:14.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.026+0000 7eff2cba8640 1 --2- 172.21.15.96:0/1064577498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff281080d0 0x7eff2810a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:14.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.026+0000 7eff2cba8640 1 --2- 172.21.15.96:0/1064577498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff281033d0 0x7eff281037e0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:14.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.026+0000 7eff2cba8640 1 --2- 172.21.15.96:0/1064577498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff28102190 0x7eff28102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:14.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.026+0000 7eff2cba8640 1 -- 172.21.15.96:0/1064577498 >> 172.21.15.96:0/1064577498 conn(0x7eff280fd380 msgr2=0x7eff280ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:14.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.026+0000 7eff2cba8640 1 -- 172.21.15.96:0/1064577498 shutdown_connections 2024-04-25T14:14:14.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.026+0000 7eff2cba8640 1 -- 172.21.15.96:0/1064577498 wait complete. 2024-04-25T14:14:14.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.027+0000 7eff2cba8640 1 Processor -- start 2024-04-25T14:14:14.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.027+0000 7eff2cba8640 1 -- start start 2024-04-25T14:14:14.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.027+0000 7eff2cba8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff28102190 0x7eff28198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:14.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.027+0000 7eff2cba8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff281033d0 0x7eff28198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:14.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.027+0000 7eff2cba8640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff281080d0 0x7eff2819f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:14.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.027+0000 7eff2cba8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7eff2810d420 con 0x7eff28102190 2024-04-25T14:14:14.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.027+0000 7eff2cba8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7eff2810d2a0 con 0x7eff281033d0 2024-04-25T14:14:14.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.027+0000 7eff2cba8640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7eff2810d5a0 con 0x7eff281080d0 2024-04-25T14:14:14.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.027+0000 7eff2659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff28102190 0x7eff28198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:14.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.027+0000 7eff2659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff28102190 0x7eff28198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58874/0 (socket says 172.21.15.96:58874) 2024-04-25T14:14:14.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.027+0000 7eff2659c640 1 -- 172.21.15.96:0/3912326781 learned_addr learned my addr 172.21.15.96:0/3912326781 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:14.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.027+0000 7eff25d9b640 1 --2- 172.21.15.96:0/3912326781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff281033d0 0x7eff28198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:14.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.027+0000 7eff26d9d640 1 --2- 172.21.15.96:0/3912326781 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff281080d0 0x7eff2819f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:14.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.027+0000 7eff2659c640 1 -- 172.21.15.96:0/3912326781 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff281080d0 msgr2=0x7eff2819f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:14.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.027+0000 7eff2659c640 1 --2- 172.21.15.96:0/3912326781 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff281080d0 0x7eff2819f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:14.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.027+0000 7eff2659c640 1 -- 172.21.15.96:0/3912326781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff281033d0 msgr2=0x7eff28198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:14.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.027+0000 7eff2659c640 1 --2- 172.21.15.96:0/3912326781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff281033d0 0x7eff28198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:14.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.028+0000 7eff2659c640 1 -- 172.21.15.96:0/3912326781 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff2819fb60 con 0x7eff28102190 2024-04-25T14:14:14.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.028+0000 7eff26d9d640 1 --2- 172.21.15.96:0/3912326781 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff281080d0 0x7eff2819f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:14:14.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.028+0000 7eff2659c640 1 --2- 172.21.15.96:0/3912326781 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff28102190 0x7eff28198020 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7eff1800e9f0 tx=0x7eff1800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:14.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.028+0000 7eff177fe640 1 -- 172.21.15.96:0/3912326781 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1800ccc0 con 0x7eff28102190 2024-04-25T14:14:14.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.028+0000 7eff177fe640 1 -- 172.21.15.96:0/3912326781 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff180035f0 con 0x7eff28102190 2024-04-25T14:14:14.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.028+0000 7eff177fe640 1 -- 172.21.15.96:0/3912326781 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1800ce60 con 0x7eff28102190 2024-04-25T14:14:14.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.028+0000 7eff2cba8640 1 -- 172.21.15.96:0/3912326781 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff2819fe50 con 0x7eff28102190 2024-04-25T14:14:14.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.028+0000 7eff2cba8640 1 -- 172.21.15.96:0/3912326781 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff281a0310 con 0x7eff28102190 2024-04-25T14:14:14.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.030+0000 7eff33a56740 1 Processor -- start 2024-04-25T14:14:14.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.030+0000 7eff177fe640 1 -- 172.21.15.96:0/3912326781 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7eff18003aa0 con 0x7eff28102190 2024-04-25T14:14:14.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.031+0000 7eff33a56740 1 -- start start 2024-04-25T14:14:14.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.031+0000 7eff33a56740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626108ff390 0x562610ca0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:14.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.031+0000 7eff33a56740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626108d64d0 0x562610ca0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:14.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.031+0000 7eff33a56740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626108de6b0 0x562610ca1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:14.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.031+0000 7eff33a56740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562610c05d80 con 0x5626108de6b0 2024-04-25T14:14:14.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.031+0000 7eff33a56740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562610bbf8b0 con 0x5626108d64d0 2024-04-25T14:14:14.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.031+0000 7eff33a56740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562610c01d40 con 0x5626108ff390 2024-04-25T14:14:14.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.031+0000 7eff25d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626108de6b0 0x562610ca1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:14.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.031+0000 7eff25d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626108de6b0 0x562610ca1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58884/0 (socket says 172.21.15.96:58884) 2024-04-25T14:14:14.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.031+0000 7eff25d9b640 1 -- 172.21.15.96:0/1352622311 learned_addr learned my addr 172.21.15.96:0/1352622311 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:14.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.031+0000 7eff2659c640 1 --2- 172.21.15.96:0/1352622311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626108d64d0 0x562610ca0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:14.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.031+0000 7eff26d9d640 1 --2- 172.21.15.96:0/1352622311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626108ff390 0x562610ca0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:14.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.031+0000 7eff177fe640 1 --2- 172.21.15.96:0/3912326781 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7eff100afb70 0x7eff100b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:14.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.031+0000 7eff25d9b640 1 -- 172.21.15.96:0/1352622311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626108ff390 msgr2=0x562610ca0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:14.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.031+0000 7eff25d9b640 1 --2- 172.21.15.96:0/1352622311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626108ff390 0x562610ca0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:14.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.031+0000 7eff25d9b640 1 -- 172.21.15.96:0/1352622311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626108d64d0 msgr2=0x562610ca0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:14.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.031+0000 7eff25d9b640 1 --2- 172.21.15.96:0/1352622311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626108d64d0 0x562610ca0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:14.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.031+0000 7eff25d9b640 1 -- 172.21.15.96:0/1352622311 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562610b90a40 con 0x5626108de6b0 2024-04-25T14:14:14.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.031+0000 7eff2659c640 1 --2- 172.21.15.96:0/1352622311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626108d64d0 0x562610ca0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:14:14.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.031+0000 7eff26d9d640 1 --2- 172.21.15.96:0/3912326781 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7eff100afb70 0x7eff100b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:14.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.031+0000 7eff177fe640 1 -- 172.21.15.96:0/3912326781 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7eff180defd0 con 0x7eff28102190 2024-04-25T14:14:14.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.032+0000 7eff25d9b640 1 --2- 172.21.15.96:0/1352622311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626108de6b0 0x562610ca1c70 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7eff1c031500 tx=0x7eff1c009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ac399b3d432edbb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:14.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.032+0000 7eff14ff9640 1 -- 172.21.15.96:0/1352622311 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1c035030 con 0x5626108de6b0 2024-04-25T14:14:14.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.032+0000 7eff26d9d640 1 --2- 172.21.15.96:0/3912326781 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7eff100afb70 0x7eff100b2080 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x562610ca1ab0 tx=0x7eff20004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:14.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.032+0000 7eff14ff9640 1 -- 172.21.15.96:0/1352622311 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff1c040070 con 0x5626108de6b0 2024-04-25T14:14:14.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.032+0000 7eff14ff9640 1 -- 172.21.15.96:0/1352622311 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1c003970 con 0x5626108de6b0 2024-04-25T14:14:14.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.033+0000 7eff33a56740 1 -- 172.21.15.96:0/1352622311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626108de6b0 msgr2=0x562610ca1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:14.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.033+0000 7eff33a56740 1 --2- 172.21.15.96:0/1352622311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626108de6b0 0x562610ca1c70 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7eff1c031500 tx=0x7eff1c009620 comp rx=0 tx=0).stop 2024-04-25T14:14:14.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.033+0000 7eff33a56740 1 -- 172.21.15.96:0/1352622311 shutdown_connections 2024-04-25T14:14:14.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.033+0000 7eff33a56740 1 --2- 172.21.15.96:0/1352622311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626108de6b0 0x562610ca1c70 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:14.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.033+0000 7eff33a56740 1 --2- 172.21.15.96:0/1352622311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626108d64d0 0x562610ca0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:14.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.033+0000 7eff33a56740 1 --2- 172.21.15.96:0/1352622311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626108ff390 0x562610ca0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:14.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.033+0000 7eff33a56740 1 -- 172.21.15.96:0/1352622311 >> 172.21.15.96:0/1352622311 conn(0x5626108f34e0 msgr2=0x562610c9c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:14.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.033+0000 7eff33a56740 1 -- 172.21.15.96:0/1352622311 shutdown_connections 2024-04-25T14:14:14.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.033+0000 7eff33a56740 1 -- 172.21.15.96:0/1352622311 wait complete. 2024-04-25T14:14:14.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.033+0000 7eff33a56740 1 -- start start 2024-04-25T14:14:14.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.033+0000 7eff33a56740 1 Processor -- start 2024-04-25T14:14:14.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.033+0000 7eff33a56740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626108d64d0 0x562610caa930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:14.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.033+0000 7eff33a56740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626108de6b0 0x562610caae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:14.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.033+0000 7eff33a56740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626108ff390 0x562610cb1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:14.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.033+0000 7eff33a56740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562610ca2720 con 0x5626108de6b0 2024-04-25T14:14:14.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.033+0000 7eff33a56740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562610b8cd10 con 0x5626108ff390 2024-04-25T14:14:14.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.033+0000 7eff33a56740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562610ca28a0 con 0x5626108d64d0 2024-04-25T14:14:14.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.033+0000 7eff33a56740 10 client.0 _ll_register_callbacks cb 0x562610bb9210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:14:14.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.034+0000 7eff25d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626108de6b0 0x562610caae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:14.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.034+0000 7eff25d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626108de6b0 0x562610caae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58898/0 (socket says 172.21.15.96:58898) 2024-04-25T14:14:14.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.034+0000 7eff25d9b640 1 -- 172.21.15.96:0/2766981675 learned_addr learned my addr 172.21.15.96:0/2766981675 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:14.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.034+0000 7eff2659c640 1 --2- 172.21.15.96:0/2766981675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626108d64d0 0x562610caa930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:14.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.034+0000 7eff26d9d640 1 --2- 172.21.15.96:0/2766981675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626108ff390 0x562610cb1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:14.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.034+0000 7eff25d9b640 1 -- 172.21.15.96:0/2766981675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626108d64d0 msgr2=0x562610caa930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:14.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.034+0000 7eff25d9b640 1 --2- 172.21.15.96:0/2766981675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626108d64d0 0x562610caa930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:14.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.034+0000 7eff25d9b640 1 -- 172.21.15.96:0/2766981675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626108ff390 msgr2=0x562610cb1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:14.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.034+0000 7eff25d9b640 1 --2- 172.21.15.96:0/2766981675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626108ff390 0x562610cb1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:14.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.034+0000 7eff25d9b640 1 -- 172.21.15.96:0/2766981675 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626109e17d0 con 0x5626108de6b0 2024-04-25T14:14:14.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.034+0000 7eff2659c640 1 --2- 172.21.15.96:0/2766981675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626108d64d0 0x562610caa930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:14:14.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.034+0000 7eff26d9d640 1 --2- 172.21.15.96:0/2766981675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626108ff390 0x562610cb1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:14:14.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.034+0000 7eff25d9b640 1 --2- 172.21.15.96:0/2766981675 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626108de6b0 0x562610caae70 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7eff1c031440 tx=0x7eff1c0317d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fd8b8197adca2f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:14.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.034+0000 7eff0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:14:14.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.034+0000 7eff0dffb640 1 -- 172.21.15.96:0/2766981675 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1c053060 con 0x5626108de6b0 2024-04-25T14:14:14.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.034+0000 7eff0dffb640 1 -- 172.21.15.96:0/2766981675 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7eff1c0036f0 con 0x5626108de6b0 2024-04-25T14:14:14.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.034+0000 7eff33a56740 1 -- 172.21.15.96:0/2766981675 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff1c040070 con 0x5626108de6b0 2024-04-25T14:14:14.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.034+0000 7eff0dffb640 1 -- 172.21.15.96:0/2766981675 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7eff1c035070 con 0x5626108de6b0 2024-04-25T14:14:14.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.034+0000 7eff33a56740 10 client.15816 fetch_fsmap learned FSMap version 26 2024-04-25T14:14:14.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.034+0000 7eff33a56740 1 -- 172.21.15.96:0/2766981675 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5626109173f0 con 0x5626108de6b0 2024-04-25T14:14:14.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.034+0000 7eff0dffb640 1 -- 172.21.15.96:0/2766981675 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff1c040070 con 0x5626108de6b0 2024-04-25T14:14:14.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.035+0000 7eff0dffb640 1 -- 172.21.15.96:0/2766981675 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7eff1c04d030 con 0x5626108de6b0 2024-04-25T14:14:14.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.035+0000 7eff33a56740 10 client.15816 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:14:14.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.035+0000 7eff33a56740 10 client.15816 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:14:14.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.035+0000 7eff33a56740 20 client.15816 populate_metadata read hostname 'smithi096' 2024-04-25T14:14:14.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.035+0000 7eff33a56740 1 --2- 172.21.15.96:0/2766981675 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5626108bdf60 0x562610cb7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:14.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.035+0000 7eff33a56740 4 client.15816 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:14:14.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.035+0000 7eff33a56740 1 -- 172.21.15.96:0/2766981675 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562610c13990 con 0x5626108bdf60 2024-04-25T14:14:14.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.035+0000 7eff2659c640 1 --2- 172.21.15.96:0/2766981675 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5626108bdf60 0x562610cb7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:14.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.035+0000 7eff0dffb640 10 client.15816 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:14.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.035+0000 7eff2659c640 1 --2- 172.21.15.96:0/2766981675 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5626108bdf60 0x562610cb7e90 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e28d8b35eda3d server_cookie=69de686de9bc2ea3 in_seq=0 out_seq=0 2024-04-25T14:14:14.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.036+0000 7eff0dffb640 1 -- 172.21.15.96:0/2766981675 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562610c13990 con 0x5626108bdf60 2024-04-25T14:14:14.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.036+0000 7eff0dffb640 10 client.15816 handle_command_reply: tid=0 2024-04-25T14:14:14.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.036+0000 7eff33a56740 1 client.15816 shutdown 2024-04-25T14:14:14.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.036+0000 7eff33a56740 1 -- 172.21.15.96:0/2766981675 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626108de6b0 msgr2=0x562610caae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:14.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.036+0000 7eff33a56740 1 --2- 172.21.15.96:0/2766981675 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626108de6b0 0x562610caae70 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7eff1c031440 tx=0x7eff1c0317d0 comp rx=0 tx=0).stop 2024-04-25T14:14:14.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.036+0000 7eff33a56740 1 -- 172.21.15.96:0/2766981675 shutdown_connections 2024-04-25T14:14:14.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.036+0000 7eff33a56740 1 --2- 172.21.15.96:0/2766981675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626108ff390 0x562610cb1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:14.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.036+0000 7eff33a56740 1 --2- 172.21.15.96:0/2766981675 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5626108bdf60 0x562610cb7e90 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:14.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.036+0000 7eff33a56740 1 --2- 172.21.15.96:0/2766981675 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626108de6b0 0x562610caae70 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:14.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.036+0000 7eff33a56740 1 --2- 172.21.15.96:0/2766981675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626108d64d0 0x562610caa930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:14.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.036+0000 7eff0dffb640 0 client.15816 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:14.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.036+0000 7eff33a56740 1 -- 172.21.15.96:0/2766981675 >> 172.21.15.96:0/2766981675 conn(0x5626108f34e0 msgr2=0x562610c9b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:14.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.036+0000 7eff33a56740 1 -- 172.21.15.96:0/2766981675 shutdown_connections 2024-04-25T14:14:14.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.036+0000 7eff33a56740 1 -- 172.21.15.96:0/2766981675 wait complete. 2024-04-25T14:14:14.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.036+0000 7eff33a56740 20 client.15816 trim_cache size 0 max 16384 2024-04-25T14:14:14.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.054+0000 7eff33a56740 1 Processor -- start 2024-04-25T14:14:14.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.054+0000 7eff33a56740 1 -- start start 2024-04-25T14:14:14.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.054+0000 7eff33a56740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626108d64d0 0x562610ca9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:14.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.054+0000 7eff33a56740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626108de6b0 0x562610c9f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:14.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.054+0000 7eff33a56740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626108ff390 0x562610c9fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:14.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.054+0000 7eff33a56740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7eff281ab370 con 0x5626108d64d0 2024-04-25T14:14:14.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.054+0000 7eff33a56740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7eff281aceb0 con 0x5626108ff390 2024-04-25T14:14:14.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.054+0000 7eff33a56740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7eff281aba40 con 0x5626108de6b0 2024-04-25T14:14:14.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.054+0000 7eff2659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626108d64d0 0x562610ca9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:14.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.054+0000 7eff2659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626108d64d0 0x562610ca9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58912/0 (socket says 172.21.15.96:58912) 2024-04-25T14:14:14.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.054+0000 7eff2659c640 1 -- 172.21.15.96:0/1673559248 learned_addr learned my addr 172.21.15.96:0/1673559248 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:14.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.054+0000 7eff26d9d640 1 --2- 172.21.15.96:0/1673559248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626108ff390 0x562610c9fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:14.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.054+0000 7eff25d9b640 1 --2- 172.21.15.96:0/1673559248 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626108de6b0 0x562610c9f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:14.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.054+0000 7eff2659c640 1 -- 172.21.15.96:0/1673559248 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626108de6b0 msgr2=0x562610c9f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:14.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.054+0000 7eff2659c640 1 --2- 172.21.15.96:0/1673559248 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626108de6b0 0x562610c9f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:14.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.054+0000 7eff2659c640 1 -- 172.21.15.96:0/1673559248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626108ff390 msgr2=0x562610c9fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:14.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.054+0000 7eff2659c640 1 --2- 172.21.15.96:0/1673559248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626108ff390 0x562610c9fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:14.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.055+0000 7eff2659c640 1 -- 172.21.15.96:0/1673559248 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626108d70f0 con 0x5626108d64d0 2024-04-25T14:14:14.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.055+0000 7eff2659c640 1 --2- 172.21.15.96:0/1673559248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626108d64d0 0x562610ca9930 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7eff180a73c0 tx=0x7eff180dc920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=828a533484923c1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:14.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.055+0000 7eff0ffff640 1 -- 172.21.15.96:0/1673559248 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff180a6d80 con 0x5626108d64d0 2024-04-25T14:14:14.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.055+0000 7eff0ffff640 1 -- 172.21.15.96:0/1673559248 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff180a6f40 con 0x5626108d64d0 2024-04-25T14:14:14.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.055+0000 7eff0ffff640 1 -- 172.21.15.96:0/1673559248 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff180a6ba0 con 0x5626108d64d0 2024-04-25T14:14:14.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.055+0000 7eff33a56740 1 -- 172.21.15.96:0/1673559248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626108d64d0 msgr2=0x562610ca9930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:14.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.055+0000 7eff33a56740 1 --2- 172.21.15.96:0/1673559248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626108d64d0 0x562610ca9930 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7eff180a73c0 tx=0x7eff180dc920 comp rx=0 tx=0).stop 2024-04-25T14:14:14.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.055+0000 7eff33a56740 1 -- 172.21.15.96:0/1673559248 shutdown_connections 2024-04-25T14:14:14.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.055+0000 7eff33a56740 1 --2- 172.21.15.96:0/1673559248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626108ff390 0x562610c9fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:14.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.055+0000 7eff33a56740 1 --2- 172.21.15.96:0/1673559248 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626108de6b0 0x562610c9f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:14.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.056+0000 7eff33a56740 1 --2- 172.21.15.96:0/1673559248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626108d64d0 0x562610ca9930 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:14.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.056+0000 7eff33a56740 1 -- 172.21.15.96:0/1673559248 >> 172.21.15.96:0/1673559248 conn(0x5626108f34e0 msgr2=0x562610c9f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:14.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.056+0000 7eff33a56740 1 -- 172.21.15.96:0/1673559248 shutdown_connections 2024-04-25T14:14:14.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.056+0000 7eff33a56740 1 -- 172.21.15.96:0/1673559248 wait complete. 2024-04-25T14:14:14.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.056+0000 7eff33a56740 1 -- start start 2024-04-25T14:14:14.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.056+0000 7eff33a56740 1 Processor -- start 2024-04-25T14:14:14.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.056+0000 7eff33a56740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626108d64d0 0x562610caf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:14.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.056+0000 7eff33a56740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626108de6b0 0x562610cafac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:14.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.056+0000 7eff33a56740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626108ff390 0x562610d1e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:14.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.056+0000 7eff33a56740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7eff281ac7e0 con 0x5626108d64d0 2024-04-25T14:14:14.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.056+0000 7eff33a56740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7eff281ac110 con 0x5626108ff390 2024-04-25T14:14:14.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.056+0000 7eff33a56740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7eff281a9350 con 0x5626108de6b0 2024-04-25T14:14:14.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.056+0000 7eff33a56740 10 client.0 _ll_register_callbacks cb 0x7eff180e28e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:14:14.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.056+0000 7eff2659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626108d64d0 0x562610caf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:14.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.056+0000 7eff2659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626108d64d0 0x562610caf580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58918/0 (socket says 172.21.15.96:58918) 2024-04-25T14:14:14.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.056+0000 7eff2659c640 1 -- 172.21.15.96:0/3190857862 learned_addr learned my addr 172.21.15.96:0/3190857862 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:14.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.056+0000 7eff26d9d640 1 --2- 172.21.15.96:0/3190857862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626108ff390 0x562610d1e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:14.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.056+0000 7eff25d9b640 1 --2- 172.21.15.96:0/3190857862 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626108de6b0 0x562610cafac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:14.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.057+0000 7eff2659c640 1 -- 172.21.15.96:0/3190857862 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626108de6b0 msgr2=0x562610cafac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:14.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.057+0000 7eff2659c640 1 --2- 172.21.15.96:0/3190857862 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626108de6b0 0x562610cafac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:14.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.057+0000 7eff2659c640 1 -- 172.21.15.96:0/3190857862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626108ff390 msgr2=0x562610d1e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:14.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.057+0000 7eff2659c640 1 --2- 172.21.15.96:0/3190857862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626108ff390 0x562610d1e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:14.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.057+0000 7eff2659c640 1 -- 172.21.15.96:0/3190857862 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562610bc64b0 con 0x5626108d64d0 2024-04-25T14:14:14.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.057+0000 7eff2659c640 1 --2- 172.21.15.96:0/3190857862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626108d64d0 0x562610caf580 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7eff180a34b0 tx=0x7eff180e2a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7be20f207d17c836 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:14.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.057+0000 7eff0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:14:14.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.057+0000 7eff0dffb640 1 -- 172.21.15.96:0/3190857862 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff180a6d80 con 0x5626108d64d0 2024-04-25T14:14:14.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.057+0000 7eff0dffb640 1 -- 172.21.15.96:0/3190857862 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7eff180a36c0 con 0x5626108d64d0 2024-04-25T14:14:14.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.057+0000 7eff33a56740 1 -- 172.21.15.96:0/3190857862 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff180a6f40 con 0x5626108d64d0 2024-04-25T14:14:14.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.057+0000 7eff0dffb640 1 -- 172.21.15.96:0/3190857862 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7eff180a7430 con 0x5626108d64d0 2024-04-25T14:14:14.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.057+0000 7eff33a56740 10 client.15822 fetch_fsmap learned FSMap version 26 2024-04-25T14:14:14.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.057+0000 7eff33a56740 1 -- 172.21.15.96:0/3190857862 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562610d1f5c0 con 0x5626108d64d0 2024-04-25T14:14:14.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.057+0000 7eff0dffb640 1 -- 172.21.15.96:0/3190857862 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff180a6f40 con 0x5626108d64d0 2024-04-25T14:14:14.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.057+0000 7eff0dffb640 1 -- 172.21.15.96:0/3190857862 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7eff180aebf0 con 0x5626108d64d0 2024-04-25T14:14:14.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.057+0000 7eff33a56740 10 client.15822 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:14:14.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.057+0000 7eff33a56740 10 client.15822 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:14:14.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.057+0000 7eff33a56740 20 client.15822 populate_metadata read hostname 'smithi096' 2024-04-25T14:14:14.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.057+0000 7eff33a56740 1 --2- 172.21.15.96:0/3190857862 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5626108933a0 0x562610d21f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:14.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.057+0000 7eff33a56740 4 client.15822 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:14:14.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.057+0000 7eff33a56740 1 -- 172.21.15.96:0/3190857862 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562610c0d800 con 0x5626108933a0 2024-04-25T14:14:14.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.058+0000 7eff25d9b640 1 --2- 172.21.15.96:0/3190857862 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5626108933a0 0x562610d21f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:14.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.058+0000 7eff25d9b640 1 --2- 172.21.15.96:0/3190857862 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5626108933a0 0x562610d21f20 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73d380be3606719f server_cookie=d41c569a50858239 in_seq=0 out_seq=0 2024-04-25T14:14:14.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.058+0000 7eff0dffb640 10 client.15822 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:14.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.058+0000 7eff0dffb640 1 -- 172.21.15.96:0/3190857862 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x562610c0d800 con 0x5626108933a0 2024-04-25T14:14:14.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.058+0000 7eff0dffb640 10 client.15822 handle_command_reply: tid=0 2024-04-25T14:14:14.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.058+0000 7eff33a56740 1 client.15822 shutdown 2024-04-25T14:14:14.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.058+0000 7eff33a56740 1 -- 172.21.15.96:0/3190857862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626108d64d0 msgr2=0x562610caf580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:14.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.058+0000 7eff33a56740 1 --2- 172.21.15.96:0/3190857862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626108d64d0 0x562610caf580 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7eff180a34b0 tx=0x7eff180e2a10 comp rx=0 tx=0).stop 2024-04-25T14:14:14.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.058+0000 7eff33a56740 1 -- 172.21.15.96:0/3190857862 shutdown_connections 2024-04-25T14:14:14.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.058+0000 7eff33a56740 1 --2- 172.21.15.96:0/3190857862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626108ff390 0x562610d1e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:14.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.058+0000 7eff33a56740 1 --2- 172.21.15.96:0/3190857862 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626108de6b0 0x562610cafac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:14.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.058+0000 7eff33a56740 1 --2- 172.21.15.96:0/3190857862 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5626108933a0 0x562610d21f20 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:14.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.058+0000 7eff33a56740 1 --2- 172.21.15.96:0/3190857862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626108d64d0 0x562610caf580 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:14.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.058+0000 7eff0dffb640 0 client.15822 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:14.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.058+0000 7eff33a56740 1 -- 172.21.15.96:0/3190857862 >> 172.21.15.96:0/3190857862 conn(0x5626108f34e0 msgr2=0x562610c9faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:14.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.058+0000 7eff33a56740 1 -- 172.21.15.96:0/3190857862 shutdown_connections 2024-04-25T14:14:14.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.059+0000 7eff33a56740 1 -- 172.21.15.96:0/3190857862 wait complete. 2024-04-25T14:14:14.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.059+0000 7eff33a56740 20 client.15822 trim_cache size 0 max 16384 2024-04-25T14:14:14.080 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T14:14:14.080 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T14:14:14.080 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": { 2024-04-25T14:14:14.080 INFO:teuthology.orchestra.run.smithi096.stdout: "28b7b7cd-cdd8-42dc-96f1-37c0dc0150ad": { 2024-04-25T14:14:14.081 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-04-25T14:14:14.081 INFO:teuthology.orchestra.run.smithi096.stdout: "tag": "28b7b7cd-cdd8-42dc-96f1-37c0dc0150ad", 2024-04-25T14:14:14.081 INFO:teuthology.orchestra.run.smithi096.stdout: "options": "recursive,force" 2024-04-25T14:14:14.081 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:14:14.081 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:14:14.081 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T14:14:14.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.059+0000 7eff2cba8640 1 -- 172.21.15.96:0/3912326781 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7eff100afb70 msgr2=0x7eff100b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:14.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.059+0000 7eff2cba8640 1 --2- 172.21.15.96:0/3912326781 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7eff100afb70 0x7eff100b2080 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x562610ca1ab0 tx=0x7eff20004a40 comp rx=0 tx=0).stop 2024-04-25T14:14:14.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.059+0000 7eff2cba8640 1 -- 172.21.15.96:0/3912326781 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff28102190 msgr2=0x7eff28198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:14.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.059+0000 7eff2cba8640 1 --2- 172.21.15.96:0/3912326781 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff28102190 0x7eff28198020 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7eff1800e9f0 tx=0x7eff1800eeb0 comp rx=0 tx=0).stop 2024-04-25T14:14:14.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.059+0000 7eff2cba8640 1 -- 172.21.15.96:0/3912326781 shutdown_connections 2024-04-25T14:14:14.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.059+0000 7eff2cba8640 1 --2- 172.21.15.96:0/3912326781 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff281080d0 0x7eff2819f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:14.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.059+0000 7eff2cba8640 1 --2- 172.21.15.96:0/3912326781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff281033d0 0x7eff28198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:14.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.059+0000 7eff2cba8640 1 --2- 172.21.15.96:0/3912326781 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7eff100afb70 0x7eff100b2080 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:14.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.059+0000 7eff2cba8640 1 --2- 172.21.15.96:0/3912326781 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff28102190 0x7eff28198020 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:14.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.059+0000 7eff2cba8640 1 -- 172.21.15.96:0/3912326781 >> 172.21.15.96:0/3912326781 conn(0x7eff280fd380 msgr2=0x7eff281040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:14.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.059+0000 7eff2cba8640 1 -- 172.21.15.96:0/3912326781 shutdown_connections 2024-04-25T14:14:14.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:14.059+0000 7eff2cba8640 1 -- 172.21.15.96:0/3912326781 wait complete. 2024-04-25T14:14:14.084 INFO:tasks.cephfs.filesystem:scrub status for tag:28b7b7cd-cdd8-42dc-96f1-37c0dc0150ad - {'path': '/', 'tag': '28b7b7cd-cdd8-42dc-96f1-37c0dc0150ad', 'options': 'recursive,force'} 2024-04-25T14:14:14.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:14 smithi096 ceph-mon[29207]: pgmap v364: 97 pgs: 97 active+clean; 657 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T14:14:14.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:14 smithi097 ceph-mon[32720]: pgmap v364: 97 pgs: 97 active+clean; 657 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T14:14:14.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:14 smithi151 ceph-mon[32990]: pgmap v364: 97 pgs: 97 active+clean; 657 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T14:14:15.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:15 smithi096 ceph-mon[29207]: scrub queued for path: / 2024-04-25T14:14:15.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:15 smithi096 ceph-mon[29207]: scrub summary: idle+waiting paths [/] 2024-04-25T14:14:15.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:15 smithi096 ceph-mon[29207]: scrub summary: active paths [/] 2024-04-25T14:14:15.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:15 smithi096 ceph-mon[29207]: scrub summary: idle+waiting paths [/] 2024-04-25T14:14:15.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:15 smithi097 ceph-mon[32720]: scrub queued for path: / 2024-04-25T14:14:15.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:15 smithi097 ceph-mon[32720]: scrub summary: idle+waiting paths [/] 2024-04-25T14:14:15.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:15 smithi097 ceph-mon[32720]: scrub summary: active paths [/] 2024-04-25T14:14:15.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:15 smithi097 ceph-mon[32720]: scrub summary: idle+waiting paths [/] 2024-04-25T14:14:15.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:15 smithi151 ceph-mon[32990]: scrub queued for path: / 2024-04-25T14:14:15.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:15 smithi151 ceph-mon[32990]: scrub summary: idle+waiting paths [/] 2024-04-25T14:14:15.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:15 smithi151 ceph-mon[32990]: scrub summary: active paths [/] 2024-04-25T14:14:15.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:15 smithi151 ceph-mon[32990]: scrub summary: idle+waiting paths [/] 2024-04-25T14:14:16.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:16 smithi096 ceph-mon[29207]: pgmap v365: 97 pgs: 97 active+clean; 657 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T14:14:16.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:16 smithi097 ceph-mon[32720]: pgmap v365: 97 pgs: 97 active+clean; 657 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T14:14:16.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:16 smithi151 ceph-mon[32990]: pgmap v365: 97 pgs: 97 active+clean; 657 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T14:14:18.041 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.040+0000 7fd9b2914640 1 -- 172.21.15.96:0/1130578414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9ac103c50 msgr2=0x7fd9ac104060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:18.041 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.040+0000 7fd9b2914640 1 --2- 172.21.15.96:0/1130578414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9ac103c50 0x7fd9ac104060 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0009900 tx=0x7fd9a002a3f0 comp rx=0 tx=0).stop 2024-04-25T14:14:18.041 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.040+0000 7fd9b2914640 1 -- 172.21.15.96:0/1130578414 shutdown_connections 2024-04-25T14:14:18.041 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.040+0000 7fd9b2914640 1 --2- 172.21.15.96:0/1130578414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9ac108950 0x7fd9ac10ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:18.041 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.040+0000 7fd9b2914640 1 --2- 172.21.15.96:0/1130578414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9ac103c50 0x7fd9ac104060 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:18.041 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.040+0000 7fd9b2914640 1 --2- 172.21.15.96:0/1130578414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9ac102a10 0x7fd9ac103710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:18.041 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.040+0000 7fd9b2914640 1 -- 172.21.15.96:0/1130578414 >> 172.21.15.96:0/1130578414 conn(0x7fd9ac0fdc00 msgr2=0x7fd9ac1000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:18.041 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.040+0000 7fd9b2914640 1 -- 172.21.15.96:0/1130578414 shutdown_connections 2024-04-25T14:14:18.041 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.040+0000 7fd9b2914640 1 -- 172.21.15.96:0/1130578414 wait complete. 2024-04-25T14:14:18.041 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.041+0000 7fd9b2914640 1 Processor -- start 2024-04-25T14:14:18.042 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.041+0000 7fd9b2914640 1 -- start start 2024-04-25T14:14:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.041+0000 7fd9b2914640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9ac102a10 0x7fd9ac075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.041+0000 7fd9b2914640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9ac103c50 0x7fd9ac19f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.041+0000 7fd9b2914640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9ac108950 0x7fd9ac1a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.041+0000 7fd9b2914640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ac10e530 con 0x7fd9ac102a10 2024-04-25T14:14:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.041+0000 7fd9b2914640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ac10e3b0 con 0x7fd9ac103c50 2024-04-25T14:14:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.041+0000 7fd9b2914640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ac10e6b0 con 0x7fd9ac108950 2024-04-25T14:14:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.042+0000 7fd9abfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9ac102a10 0x7fd9ac075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.042+0000 7fd9abfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9ac102a10 0x7fd9ac075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58942/0 (socket says 172.21.15.96:58942) 2024-04-25T14:14:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.042+0000 7fd9abfff640 1 -- 172.21.15.96:0/2670584324 learned_addr learned my addr 172.21.15.96:0/2670584324 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.042+0000 7fd9b0eb1640 1 --2- 172.21.15.96:0/2670584324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9ac108950 0x7fd9ac1a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.042+0000 7fd9ab7fe640 1 --2- 172.21.15.96:0/2670584324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9ac103c50 0x7fd9ac19f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.042+0000 7fd9abfff640 1 -- 172.21.15.96:0/2670584324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9ac108950 msgr2=0x7fd9ac1a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.042+0000 7fd9abfff640 1 --2- 172.21.15.96:0/2670584324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9ac108950 0x7fd9ac1a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.042+0000 7fd9abfff640 1 -- 172.21.15.96:0/2670584324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9ac103c50 msgr2=0x7fd9ac19f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.042+0000 7fd9abfff640 1 --2- 172.21.15.96:0/2670584324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9ac103c50 0x7fd9ac19f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.042+0000 7fd9abfff640 1 -- 172.21.15.96:0/2670584324 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9ac1a1cf0 con 0x7fd9ac102a10 2024-04-25T14:14:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.042+0000 7fd9b0eb1640 1 --2- 172.21.15.96:0/2670584324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9ac108950 0x7fd9ac1a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:14:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.042+0000 7fd9ab7fe640 1 --2- 172.21.15.96:0/2670584324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9ac103c50 0x7fd9ac19f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:14:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.042+0000 7fd9abfff640 1 --2- 172.21.15.96:0/2670584324 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9ac102a10 0x7fd9ac075840 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd99c00c970 tx=0x7fd99c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.042+0000 7fd9a97fa640 1 -- 172.21.15.96:0/2670584324 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c004220 con 0x7fd9ac102a10 2024-04-25T14:14:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.042+0000 7fd9a97fa640 1 -- 172.21.15.96:0/2670584324 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd99c0043c0 con 0x7fd9ac102a10 2024-04-25T14:14:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.042+0000 7fd9b2914640 1 -- 172.21.15.96:0/2670584324 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9ac1a1fe0 con 0x7fd9ac102a10 2024-04-25T14:14:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.042+0000 7fd9a97fa640 1 -- 172.21.15.96:0/2670584324 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c004a00 con 0x7fd9ac102a10 2024-04-25T14:14:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.042+0000 7fd9b2914640 1 -- 172.21.15.96:0/2670584324 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9ac1a23f0 con 0x7fd9ac102a10 2024-04-25T14:14:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.045+0000 7fd9a97fa640 1 -- 172.21.15.96:0/2670584324 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fd99c0079b0 con 0x7fd9ac102a10 2024-04-25T14:14:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.045+0000 7fd9b97c8740 1 Processor -- start 2024-04-25T14:14:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.045+0000 7fd9b97c8740 1 -- start start 2024-04-25T14:14:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.045+0000 7fd9b97c8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630486e41b0 0x563048a84970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.045+0000 7fd9b97c8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630486bb2f0 0x563048a84ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.045+0000 7fd9b97c8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630486c34d0 0x563048a8bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.045+0000 7fd9b97c8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5630489ea460 con 0x5630486bb2f0 2024-04-25T14:14:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.045+0000 7fd9b97c8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5630489a4590 con 0x5630486c34d0 2024-04-25T14:14:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.045+0000 7fd9b97c8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5630489e6420 con 0x5630486e41b0 2024-04-25T14:14:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.045+0000 7fd9abfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630486bb2f0 0x563048a84ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.045+0000 7fd9abfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630486bb2f0 0x563048a84ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58956/0 (socket says 172.21.15.96:58956) 2024-04-25T14:14:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.045+0000 7fd9abfff640 1 -- 172.21.15.96:0/4214341367 learned_addr learned my addr 172.21.15.96:0/4214341367 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.045+0000 7fd9ab7fe640 1 --2- 172.21.15.96:0/4214341367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630486c34d0 0x563048a8bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.046+0000 7fd9b0eb1640 1 --2- 172.21.15.96:0/4214341367 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630486e41b0 0x563048a84970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.046+0000 7fd9a97fa640 1 --2- 172.21.15.96:0/2670584324 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd9940afae0 0x7fd9940b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.046+0000 7fd9b0eb1640 1 --2- 172.21.15.96:0/2670584324 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd9940afae0 0x7fd9940b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.046+0000 7fd9abfff640 1 -- 172.21.15.96:0/4214341367 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630486e41b0 msgr2=0x563048a84970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.046+0000 7fd9abfff640 1 --2- 172.21.15.96:0/4214341367 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630486e41b0 0x563048a84970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.046+0000 7fd9abfff640 1 -- 172.21.15.96:0/4214341367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630486c34d0 msgr2=0x563048a8bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.046+0000 7fd9abfff640 1 --2- 172.21.15.96:0/4214341367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630486c34d0 0x563048a8bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.046+0000 7fd9a97fa640 1 -- 172.21.15.96:0/2670584324 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fd99c0dead0 con 0x7fd9ac102a10 2024-04-25T14:14:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.046+0000 7fd9abfff640 1 -- 172.21.15.96:0/4214341367 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630489b10d0 con 0x5630486bb2f0 2024-04-25T14:14:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.046+0000 7fd9ab7fe640 1 --2- 172.21.15.96:0/4214341367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630486c34d0 0x563048a8bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:14:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.046+0000 7fd9abfff640 1 --2- 172.21.15.96:0/4214341367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630486bb2f0 0x563048a84ef0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd99c0a3010 tx=0x7fd99c0a2f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51b1df1fa298f800 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.046+0000 7fd9b0eb1640 1 --2- 172.21.15.96:0/2670584324 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd9940afae0 0x7fd9940b1ff0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x563048a85b40 tx=0x7fd9a4004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.046+0000 7fd9927fc640 1 -- 172.21.15.96:0/4214341367 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c01d070 con 0x5630486bb2f0 2024-04-25T14:14:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.046+0000 7fd9927fc640 1 -- 172.21.15.96:0/4214341367 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd99c0a6aa0 con 0x5630486bb2f0 2024-04-25T14:14:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.046+0000 7fd9927fc640 1 -- 172.21.15.96:0/4214341367 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c0dc6c0 con 0x5630486bb2f0 2024-04-25T14:14:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.047+0000 7fd9b97c8740 1 -- 172.21.15.96:0/4214341367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630486bb2f0 msgr2=0x563048a84ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.047+0000 7fd9b97c8740 1 --2- 172.21.15.96:0/4214341367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630486bb2f0 0x563048a84ef0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd99c0a3010 tx=0x7fd99c0a2f80 comp rx=0 tx=0).stop 2024-04-25T14:14:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.047+0000 7fd9b97c8740 1 -- 172.21.15.96:0/4214341367 shutdown_connections 2024-04-25T14:14:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.047+0000 7fd9b97c8740 1 --2- 172.21.15.96:0/4214341367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630486c34d0 0x563048a8bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.047+0000 7fd9b97c8740 1 --2- 172.21.15.96:0/4214341367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630486bb2f0 0x563048a84ef0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.047+0000 7fd9b97c8740 1 --2- 172.21.15.96:0/4214341367 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630486e41b0 0x563048a84970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.047+0000 7fd9b97c8740 1 -- 172.21.15.96:0/4214341367 >> 172.21.15.96:0/4214341367 conn(0x5630486d8300 msgr2=0x563048a80300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.047+0000 7fd9b97c8740 1 -- 172.21.15.96:0/4214341367 shutdown_connections 2024-04-25T14:14:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.047+0000 7fd9b97c8740 1 -- 172.21.15.96:0/4214341367 wait complete. 2024-04-25T14:14:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.048+0000 7fd9b97c8740 1 -- start start 2024-04-25T14:14:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.048+0000 7fd9b97c8740 1 Processor -- start 2024-04-25T14:14:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.048+0000 7fd9b97c8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630486bb2f0 0x563048a8e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.048+0000 7fd9b97c8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630486c34d0 0x563048a8ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.048+0000 7fd9b97c8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630486e41b0 0x563048a95a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.048+0000 7fd9b97c8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563048a8c7e0 con 0x5630486bb2f0 2024-04-25T14:14:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.048+0000 7fd9b97c8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563048971b60 con 0x5630486e41b0 2024-04-25T14:14:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.048+0000 7fd9b97c8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563048a8c960 con 0x5630486c34d0 2024-04-25T14:14:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.048+0000 7fd9b97c8740 10 client.0 _ll_register_callbacks cb 0x5630489962b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:14:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.048+0000 7fd9abfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630486bb2f0 0x563048a8e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.048+0000 7fd9abfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630486bb2f0 0x563048a8e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58960/0 (socket says 172.21.15.96:58960) 2024-04-25T14:14:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.048+0000 7fd9abfff640 1 -- 172.21.15.96:0/252999220 learned_addr learned my addr 172.21.15.96:0/252999220 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.048+0000 7fd9ab7fe640 1 --2- 172.21.15.96:0/252999220 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630486c34d0 0x563048a8ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.049+0000 7fd9b0eb1640 1 --2- 172.21.15.96:0/252999220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630486e41b0 0x563048a95a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.049+0000 7fd9abfff640 1 -- 172.21.15.96:0/252999220 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630486c34d0 msgr2=0x563048a8ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.049+0000 7fd9abfff640 1 --2- 172.21.15.96:0/252999220 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630486c34d0 0x563048a8ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.049+0000 7fd9abfff640 1 -- 172.21.15.96:0/252999220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630486e41b0 msgr2=0x563048a95a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.049+0000 7fd9abfff640 1 --2- 172.21.15.96:0/252999220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630486e41b0 0x563048a95a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.049+0000 7fd9abfff640 1 -- 172.21.15.96:0/252999220 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630486fc210 con 0x5630486bb2f0 2024-04-25T14:14:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.049+0000 7fd9abfff640 1 --2- 172.21.15.96:0/252999220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630486bb2f0 0x563048a8e590 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd99c0a6f20 tx=0x7fd99c0dee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1a521edfa1befeb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.049+0000 7fd98b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:14:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.049+0000 7fd98b7fe640 1 -- 172.21.15.96:0/252999220 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c0e5060 con 0x5630486bb2f0 2024-04-25T14:14:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.049+0000 7fd98b7fe640 1 -- 172.21.15.96:0/252999220 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fd99c0af960 con 0x5630486bb2f0 2024-04-25T14:14:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.049+0000 7fd9b97c8740 1 -- 172.21.15.96:0/252999220 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd99c0a6aa0 con 0x5630486bb2f0 2024-04-25T14:14:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.049+0000 7fd98b7fe640 1 -- 172.21.15.96:0/252999220 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd99c01d070 con 0x5630486bb2f0 2024-04-25T14:14:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.049+0000 7fd9b97c8740 10 client.15834 fetch_fsmap learned FSMap version 26 2024-04-25T14:14:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.049+0000 7fd9b97c8740 1 -- 172.21.15.96:0/252999220 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5630487c8620 con 0x5630486bb2f0 2024-04-25T14:14:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.050+0000 7fd98b7fe640 1 -- 172.21.15.96:0/252999220 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd99c0a6aa0 con 0x5630486bb2f0 2024-04-25T14:14:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.050+0000 7fd98b7fe640 1 -- 172.21.15.96:0/252999220 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd99c0a2900 con 0x5630486bb2f0 2024-04-25T14:14:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.050+0000 7fd9b97c8740 10 client.15834 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:14:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.050+0000 7fd9b97c8740 10 client.15834 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:14:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.050+0000 7fd9b97c8740 20 client.15834 populate_metadata read hostname 'smithi096' 2024-04-25T14:14:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.050+0000 7fd9b97c8740 1 --2- 172.21.15.96:0/252999220 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5630486a2d80 0x563048a9be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.050+0000 7fd9b97c8740 4 client.15834 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:14:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.050+0000 7fd9b97c8740 1 -- 172.21.15.96:0/252999220 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630489f8070 con 0x5630486a2d80 2024-04-25T14:14:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.050+0000 7fd9ab7fe640 1 --2- 172.21.15.96:0/252999220 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5630486a2d80 0x563048a9be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.051+0000 7fd9ab7fe640 1 --2- 172.21.15.96:0/252999220 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5630486a2d80 0x563048a9be80 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce796b4dd10552d4 server_cookie=7282b71a7d7d3add in_seq=0 out_seq=0 2024-04-25T14:14:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.051+0000 7fd98b7fe640 10 client.15834 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.051+0000 7fd98b7fe640 1 -- 172.21.15.96:0/252999220 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5630489f8070 con 0x5630486a2d80 2024-04-25T14:14:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.051+0000 7fd98b7fe640 10 client.15834 handle_command_reply: tid=0 2024-04-25T14:14:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.051+0000 7fd9b97c8740 1 client.15834 shutdown 2024-04-25T14:14:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.051+0000 7fd9b97c8740 1 -- 172.21.15.96:0/252999220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630486bb2f0 msgr2=0x563048a8e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.051+0000 7fd9b97c8740 1 --2- 172.21.15.96:0/252999220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630486bb2f0 0x563048a8e590 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd99c0a6f20 tx=0x7fd99c0dee10 comp rx=0 tx=0).stop 2024-04-25T14:14:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.051+0000 7fd9b97c8740 1 -- 172.21.15.96:0/252999220 shutdown_connections 2024-04-25T14:14:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.051+0000 7fd9b97c8740 1 --2- 172.21.15.96:0/252999220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630486e41b0 0x563048a95a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.051+0000 7fd9b97c8740 1 --2- 172.21.15.96:0/252999220 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630486c34d0 0x563048a8ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.051+0000 7fd9b97c8740 1 --2- 172.21.15.96:0/252999220 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5630486a2d80 0x563048a9be80 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.052+0000 7fd9b97c8740 1 --2- 172.21.15.96:0/252999220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630486bb2f0 0x563048a8e590 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.052+0000 7fd98b7fe640 0 client.15834 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.052+0000 7fd9b97c8740 1 -- 172.21.15.96:0/252999220 >> 172.21.15.96:0/252999220 conn(0x5630486d8300 msgr2=0x563048a83210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.052+0000 7fd9b97c8740 1 -- 172.21.15.96:0/252999220 shutdown_connections 2024-04-25T14:14:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.052+0000 7fd9b97c8740 1 -- 172.21.15.96:0/252999220 wait complete. 2024-04-25T14:14:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.052+0000 7fd9b97c8740 20 client.15834 trim_cache size 0 max 16384 2024-04-25T14:14:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.067+0000 7fd9b97c8740 1 Processor -- start 2024-04-25T14:14:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.067+0000 7fd9b97c8740 1 -- start start 2024-04-25T14:14:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.067+0000 7fd9b97c8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630486bb2f0 0x563048a83810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.067+0000 7fd9b97c8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630486c34d0 0x563048a959c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.067+0000 7fd9b97c8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630486e41b0 0x563048a7fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.067+0000 7fd9b97c8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ac1ab2f0 con 0x5630486bb2f0 2024-04-25T14:14:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.067+0000 7fd9b97c8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ac1ace30 con 0x5630486c34d0 2024-04-25T14:14:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.067+0000 7fd9b97c8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ac1ab9c0 con 0x5630486e41b0 2024-04-25T14:14:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.067+0000 7fd9abfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630486bb2f0 0x563048a83810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.067+0000 7fd9abfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630486bb2f0 0x563048a83810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58964/0 (socket says 172.21.15.96:58964) 2024-04-25T14:14:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.067+0000 7fd9abfff640 1 -- 172.21.15.96:0/736037566 learned_addr learned my addr 172.21.15.96:0/736037566 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.067+0000 7fd9abfff640 1 -- 172.21.15.96:0/736037566 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630486e41b0 msgr2=0x563048a7fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.067+0000 7fd9abfff640 1 --2- 172.21.15.96:0/736037566 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630486e41b0 0x563048a7fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.067+0000 7fd9abfff640 1 -- 172.21.15.96:0/736037566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630486c34d0 msgr2=0x563048a959c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.067+0000 7fd9abfff640 1 --2- 172.21.15.96:0/736037566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630486c34d0 0x563048a959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.067+0000 7fd9abfff640 1 -- 172.21.15.96:0/736037566 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563048a8c1b0 con 0x5630486bb2f0 2024-04-25T14:14:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.067+0000 7fd9abfff640 1 --2- 172.21.15.96:0/736037566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630486bb2f0 0x563048a83810 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd99c0a4200 tx=0x7fd99c0052e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7203f1fb79f96265 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.068+0000 7fd991ffb640 1 -- 172.21.15.96:0/736037566 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c0e5070 con 0x5630486bb2f0 2024-04-25T14:14:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.068+0000 7fd991ffb640 1 -- 172.21.15.96:0/736037566 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd99c0a7690 con 0x5630486bb2f0 2024-04-25T14:14:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.068+0000 7fd991ffb640 1 -- 172.21.15.96:0/736037566 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c0e24a0 con 0x5630486bb2f0 2024-04-25T14:14:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.068+0000 7fd9b97c8740 1 -- 172.21.15.96:0/736037566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630486bb2f0 msgr2=0x563048a83810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.068+0000 7fd9b97c8740 1 --2- 172.21.15.96:0/736037566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630486bb2f0 0x563048a83810 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd99c0a4200 tx=0x7fd99c0052e0 comp rx=0 tx=0).stop 2024-04-25T14:14:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.068+0000 7fd9b97c8740 1 -- 172.21.15.96:0/736037566 shutdown_connections 2024-04-25T14:14:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.068+0000 7fd9b97c8740 1 --2- 172.21.15.96:0/736037566 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630486e41b0 0x563048a7fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.068+0000 7fd9b97c8740 1 --2- 172.21.15.96:0/736037566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630486c34d0 0x563048a959c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.068+0000 7fd9b97c8740 1 --2- 172.21.15.96:0/736037566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630486bb2f0 0x563048a83810 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.068+0000 7fd9b97c8740 1 -- 172.21.15.96:0/736037566 >> 172.21.15.96:0/736037566 conn(0x5630486d8300 msgr2=0x563048a83100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.068+0000 7fd9b97c8740 1 -- 172.21.15.96:0/736037566 shutdown_connections 2024-04-25T14:14:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.068+0000 7fd9b97c8740 1 -- 172.21.15.96:0/736037566 wait complete. 2024-04-25T14:14:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.069+0000 7fd9b97c8740 1 -- start start 2024-04-25T14:14:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.069+0000 7fd9b97c8740 1 Processor -- start 2024-04-25T14:14:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.069+0000 7fd9b97c8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630486bb2f0 0x563048a93930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.069+0000 7fd9b97c8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630486c34d0 0x563048a93e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.069+0000 7fd9b97c8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630486e41b0 0x563048a94da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.069+0000 7fd9b97c8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ac1ac760 con 0x5630486e41b0 2024-04-25T14:14:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.069+0000 7fd9b97c8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ac1ac090 con 0x5630486c34d0 2024-04-25T14:14:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.069+0000 7fd9b97c8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ac1a92d0 con 0x5630486bb2f0 2024-04-25T14:14:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.069+0000 7fd9b97c8740 10 client.0 _ll_register_callbacks cb 0x7fd99c0def50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:14:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.069+0000 7fd9b0eb1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630486e41b0 0x563048a94da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.069+0000 7fd9b0eb1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630486e41b0 0x563048a94da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58968/0 (socket says 172.21.15.96:58968) 2024-04-25T14:14:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.069+0000 7fd9b0eb1640 1 -- 172.21.15.96:0/820787250 learned_addr learned my addr 172.21.15.96:0/820787250 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.069+0000 7fd9abfff640 1 --2- 172.21.15.96:0/820787250 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630486bb2f0 0x563048a93930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.070+0000 7fd9ab7fe640 1 --2- 172.21.15.96:0/820787250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630486c34d0 0x563048a93e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.070+0000 7fd9b0eb1640 1 -- 172.21.15.96:0/820787250 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630486bb2f0 msgr2=0x563048a93930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.070+0000 7fd9b0eb1640 1 --2- 172.21.15.96:0/820787250 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630486bb2f0 0x563048a93930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.070+0000 7fd9b0eb1640 1 -- 172.21.15.96:0/820787250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630486c34d0 msgr2=0x563048a93e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.070+0000 7fd9b0eb1640 1 --2- 172.21.15.96:0/820787250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630486c34d0 0x563048a93e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.070+0000 7fd9b0eb1640 1 -- 172.21.15.96:0/820787250 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563048a894a0 con 0x5630486e41b0 2024-04-25T14:14:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.070+0000 7fd9b0eb1640 1 --2- 172.21.15.96:0/820787250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630486e41b0 0x563048a94da0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4020670 tx=0x7fd9a4007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cba7e6b3c99ecb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.070+0000 7fd98b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:14:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.070+0000 7fd98b7fe640 1 -- 172.21.15.96:0/820787250 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4018070 con 0x5630486e41b0 2024-04-25T14:14:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.070+0000 7fd98b7fe640 1 -- 172.21.15.96:0/820787250 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fd9a4004d20 con 0x5630486e41b0 2024-04-25T14:14:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.070+0000 7fd9b97c8740 1 -- 172.21.15.96:0/820787250 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd99c0a7690 con 0x5630486e41b0 2024-04-25T14:14:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.070+0000 7fd98b7fe640 1 -- 172.21.15.96:0/820787250 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd9a400f650 con 0x5630486e41b0 2024-04-25T14:14:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.070+0000 7fd9b97c8740 10 client.15840 fetch_fsmap learned FSMap version 26 2024-04-25T14:14:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.070+0000 7fd9b97c8740 1 -- 172.21.15.96:0/820787250 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563048b04680 con 0x5630486e41b0 2024-04-25T14:14:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.071+0000 7fd98b7fe640 1 -- 172.21.15.96:0/820787250 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd99c0a7690 con 0x5630486e41b0 2024-04-25T14:14:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.071+0000 7fd98b7fe640 1 -- 172.21.15.96:0/820787250 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd9a4027070 con 0x5630486e41b0 2024-04-25T14:14:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.071+0000 7fd9b97c8740 10 client.15840 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:14:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.071+0000 7fd9b97c8740 10 client.15840 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:14:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.071+0000 7fd9b97c8740 20 client.15840 populate_metadata read hostname 'smithi096' 2024-04-25T14:14:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.071+0000 7fd9b97c8740 1 --2- 172.21.15.96:0/820787250 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5630486781c0 0x563048b07120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.071+0000 7fd9b97c8740 4 client.15840 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:14:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.071+0000 7fd9b97c8740 1 -- 172.21.15.96:0/820787250 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5630489f1ee0 con 0x5630486781c0 2024-04-25T14:14:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.071+0000 7fd9abfff640 1 --2- 172.21.15.96:0/820787250 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5630486781c0 0x563048b07120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.071+0000 7fd9abfff640 1 --2- 172.21.15.96:0/820787250 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5630486781c0 0x563048b07120 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b64793a7d1426eba server_cookie=80160e6acfc2f936 in_seq=0 out_seq=0 2024-04-25T14:14:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.071+0000 7fd98b7fe640 10 client.15840 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.078+0000 7fd98b7fe640 1 -- 172.21.15.96:0/820787250 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5630489f1ee0 con 0x5630486781c0 2024-04-25T14:14:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.079+0000 7fd98b7fe640 10 client.15840 handle_command_reply: tid=0 2024-04-25T14:14:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.079+0000 7fd9b97c8740 1 client.15840 shutdown 2024-04-25T14:14:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.079+0000 7fd9b97c8740 1 -- 172.21.15.96:0/820787250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630486e41b0 msgr2=0x563048a94da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.079+0000 7fd9b97c8740 1 --2- 172.21.15.96:0/820787250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630486e41b0 0x563048a94da0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4020670 tx=0x7fd9a4007d30 comp rx=0 tx=0).stop 2024-04-25T14:14:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.079+0000 7fd9b97c8740 1 -- 172.21.15.96:0/820787250 shutdown_connections 2024-04-25T14:14:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.079+0000 7fd9b97c8740 1 --2- 172.21.15.96:0/820787250 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5630486781c0 0x563048b07120 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.079+0000 7fd9b97c8740 1 --2- 172.21.15.96:0/820787250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630486e41b0 0x563048a94da0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.079+0000 7fd98b7fe640 0 client.15840 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.079+0000 7fd9b97c8740 1 --2- 172.21.15.96:0/820787250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630486c34d0 0x563048a93e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.079+0000 7fd9b97c8740 1 --2- 172.21.15.96:0/820787250 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630486bb2f0 0x563048a93930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.079+0000 7fd9b97c8740 1 -- 172.21.15.96:0/820787250 >> 172.21.15.96:0/820787250 conn(0x5630486d8300 msgr2=0x563048a8e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.079+0000 7fd9b97c8740 1 -- 172.21.15.96:0/820787250 shutdown_connections 2024-04-25T14:14:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.079+0000 7fd9b97c8740 1 -- 172.21.15.96:0/820787250 wait complete. 2024-04-25T14:14:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.079+0000 7fd9b97c8740 20 client.15840 trim_cache size 0 max 16384 2024-04-25T14:14:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T14:14:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T14:14:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T14:14:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T14:14:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.080+0000 7fd9b2914640 1 -- 172.21.15.96:0/2670584324 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd9940afae0 msgr2=0x7fd9940b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.080+0000 7fd9b2914640 1 --2- 172.21.15.96:0/2670584324 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd9940afae0 0x7fd9940b1ff0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x563048a85b40 tx=0x7fd9a4004ab0 comp rx=0 tx=0).stop 2024-04-25T14:14:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.080+0000 7fd9b2914640 1 -- 172.21.15.96:0/2670584324 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9ac102a10 msgr2=0x7fd9ac075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.080+0000 7fd9b2914640 1 --2- 172.21.15.96:0/2670584324 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9ac102a10 0x7fd9ac075840 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd99c00c970 tx=0x7fd99c00ce30 comp rx=0 tx=0).stop 2024-04-25T14:14:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.080+0000 7fd9b2914640 1 -- 172.21.15.96:0/2670584324 shutdown_connections 2024-04-25T14:14:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.080+0000 7fd9b2914640 1 --2- 172.21.15.96:0/2670584324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9ac108950 0x7fd9ac1a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.080+0000 7fd9b2914640 1 --2- 172.21.15.96:0/2670584324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9ac103c50 0x7fd9ac19f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.080+0000 7fd9b2914640 1 --2- 172.21.15.96:0/2670584324 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd9940afae0 0x7fd9940b1ff0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.080+0000 7fd9b2914640 1 --2- 172.21.15.96:0/2670584324 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9ac102a10 0x7fd9ac075840 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.080+0000 7fd9b2914640 1 -- 172.21.15.96:0/2670584324 >> 172.21.15.96:0/2670584324 conn(0x7fd9ac0fdc00 msgr2=0x7fd9ac104930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.080+0000 7fd9b2914640 1 -- 172.21.15.96:0/2670584324 shutdown_connections 2024-04-25T14:14:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:18.080+0000 7fd9b2914640 1 -- 172.21.15.96:0/2670584324 wait complete. 2024-04-25T14:14:18.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:18 smithi096 ceph-mon[29207]: pgmap v366: 97 pgs: 97 active+clean; 666 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-04-25T14:14:18.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:18 smithi097 ceph-mon[32720]: pgmap v366: 97 pgs: 97 active+clean; 666 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-04-25T14:14:18.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:18 smithi151 ceph-mon[32990]: pgmap v366: 97 pgs: 97 active+clean; 666 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-04-25T14:14:19.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:19 smithi096 ceph-mon[29207]: scrub summary: idle 2024-04-25T14:14:19.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:19 smithi097 ceph-mon[32720]: scrub summary: idle 2024-04-25T14:14:19.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:19 smithi151 ceph-mon[32990]: scrub summary: idle 2024-04-25T14:14:20.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:20 smithi096 ceph-mon[29207]: pgmap v367: 97 pgs: 97 active+clean; 666 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 853 B/s wr, 9 op/s 2024-04-25T14:14:20.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:20 smithi097 ceph-mon[32720]: pgmap v367: 97 pgs: 97 active+clean; 666 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 853 B/s wr, 9 op/s 2024-04-25T14:14:20.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:20 smithi151 ceph-mon[32990]: pgmap v367: 97 pgs: 97 active+clean; 666 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 853 B/s wr, 9 op/s 2024-04-25T14:14:22.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:22 smithi096 ceph-mon[29207]: pgmap v368: 97 pgs: 97 active+clean; 667 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-04-25T14:14:22.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:22 smithi097 ceph-mon[32720]: pgmap v368: 97 pgs: 97 active+clean; 667 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-04-25T14:14:22.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:22 smithi151 ceph-mon[32990]: pgmap v368: 97 pgs: 97 active+clean; 667 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-04-25T14:14:24.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:24 smithi096 ceph-mon[29207]: pgmap v369: 97 pgs: 97 active+clean; 667 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-25T14:14:24.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:24 smithi097 ceph-mon[32720]: pgmap v369: 97 pgs: 97 active+clean; 667 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-25T14:14:24.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:24 smithi151 ceph-mon[32990]: pgmap v369: 97 pgs: 97 active+clean; 667 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-25T14:14:26.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:26 smithi096 ceph-mon[29207]: pgmap v370: 97 pgs: 97 active+clean; 667 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-25T14:14:26.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:26 smithi097 ceph-mon[32720]: pgmap v370: 97 pgs: 97 active+clean; 667 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-25T14:14:26.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:26 smithi151 ceph-mon[32990]: pgmap v370: 97 pgs: 97 active+clean; 667 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-25T14:14:28.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:28 smithi096 ceph-mon[29207]: pgmap v371: 97 pgs: 97 active+clean; 667 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-04-25T14:14:28.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:28 smithi097 ceph-mon[32720]: pgmap v371: 97 pgs: 97 active+clean; 667 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-04-25T14:14:28.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:28 smithi151 ceph-mon[32990]: pgmap v371: 97 pgs: 97 active+clean; 667 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-04-25T14:14:30.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:30 smithi096 ceph-mon[29207]: pgmap v372: 97 pgs: 97 active+clean; 667 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 10 op/s 2024-04-25T14:14:30.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:30 smithi097 ceph-mon[32720]: pgmap v372: 97 pgs: 97 active+clean; 667 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 10 op/s 2024-04-25T14:14:30.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:30 smithi151 ceph-mon[32990]: pgmap v372: 97 pgs: 97 active+clean; 667 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 10 op/s 2024-04-25T14:14:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.203+0000 7eff85926640 1 -- 172.21.15.96:0/997335646 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff80102a20 msgr2=0x7eff80103720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.203+0000 7eff85926640 1 --2- 172.21.15.96:0/997335646 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff80102a20 0x7eff80103720 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7eff70009900 tx=0x7eff7002a410 comp rx=0 tx=0).stop 2024-04-25T14:14:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.203+0000 7eff85926640 1 -- 172.21.15.96:0/997335646 shutdown_connections 2024-04-25T14:14:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.203+0000 7eff85926640 1 --2- 172.21.15.96:0/997335646 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff80108960 0x7eff8010ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.203+0000 7eff85926640 1 --2- 172.21.15.96:0/997335646 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff80103c60 0x7eff80104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.203+0000 7eff85926640 1 --2- 172.21.15.96:0/997335646 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff80102a20 0x7eff80103720 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.203+0000 7eff85926640 1 -- 172.21.15.96:0/997335646 >> 172.21.15.96:0/997335646 conn(0x7eff800fdc10 msgr2=0x7eff801000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.203+0000 7eff85926640 1 -- 172.21.15.96:0/997335646 shutdown_connections 2024-04-25T14:14:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.203+0000 7eff85926640 1 -- 172.21.15.96:0/997335646 wait complete. 2024-04-25T14:14:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.204+0000 7eff85926640 1 Processor -- start 2024-04-25T14:14:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.204+0000 7eff85926640 1 -- start start 2024-04-25T14:14:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.204+0000 7eff85926640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff80102a20 0x7eff8019a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.204+0000 7eff85926640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff80103c60 0x7eff8019a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.204+0000 7eff85926640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff80108960 0x7eff801a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.204+0000 7eff85926640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7eff8010f650 con 0x7eff80108960 2024-04-25T14:14:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.204+0000 7eff85926640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7eff8010f4d0 con 0x7eff80102a20 2024-04-25T14:14:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.204+0000 7eff85926640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7eff8010f7d0 con 0x7eff80103c60 2024-04-25T14:14:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.205+0000 7eff7f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff80108960 0x7eff801a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.205+0000 7eff7f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff80108960 0x7eff801a1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37458/0 (socket says 172.21.15.96:37458) 2024-04-25T14:14:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.205+0000 7eff7f7fe640 1 -- 172.21.15.96:0/1762575249 learned_addr learned my addr 172.21.15.96:0/1762575249 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.205+0000 7eff7e7fc640 1 --2- 172.21.15.96:0/1762575249 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff80103c60 0x7eff8019a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.205+0000 7eff7effd640 1 --2- 172.21.15.96:0/1762575249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff80102a20 0x7eff8019a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.205+0000 7eff7f7fe640 1 -- 172.21.15.96:0/1762575249 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff80103c60 msgr2=0x7eff8019a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.205+0000 7eff7f7fe640 1 --2- 172.21.15.96:0/1762575249 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff80103c60 0x7eff8019a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.205+0000 7eff7f7fe640 1 -- 172.21.15.96:0/1762575249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff80102a20 msgr2=0x7eff8019a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:31.208 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.205+0000 7eff7f7fe640 1 --2- 172.21.15.96:0/1762575249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff80102a20 0x7eff8019a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:31.208 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.205+0000 7eff7f7fe640 1 -- 172.21.15.96:0/1762575249 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff801a1d80 con 0x7eff80108960 2024-04-25T14:14:31.208 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.205+0000 7eff7f7fe640 1 --2- 172.21.15.96:0/1762575249 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff80108960 0x7eff801a1710 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7eff7800eaa0 tx=0x7eff7800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:31.208 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.205+0000 7eff84924640 1 -- 172.21.15.96:0/1762575249 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7800cdb0 con 0x7eff80108960 2024-04-25T14:14:31.208 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.205+0000 7eff84924640 1 -- 172.21.15.96:0/1762575249 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff78005020 con 0x7eff80108960 2024-04-25T14:14:31.208 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.205+0000 7eff85926640 1 -- 172.21.15.96:0/1762575249 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff801a2070 con 0x7eff80108960 2024-04-25T14:14:31.208 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.205+0000 7eff84924640 1 -- 172.21.15.96:0/1762575249 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff780116d0 con 0x7eff80108960 2024-04-25T14:14:31.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.206+0000 7eff85926640 1 -- 172.21.15.96:0/1762575249 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff801a2570 con 0x7eff80108960 2024-04-25T14:14:31.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.209+0000 7eff8c7d8740 1 Processor -- start 2024-04-25T14:14:31.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.209+0000 7eff84924640 1 -- 172.21.15.96:0/1762575249 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7eff780051d0 con 0x7eff80108960 2024-04-25T14:14:31.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.209+0000 7eff8c7d8740 1 -- start start 2024-04-25T14:14:31.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.209+0000 7eff8c7d8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef542ef1b0 0x55ef5468f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:31.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.209+0000 7eff8c7d8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef542c62f0 0x55ef5468fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:31.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.209+0000 7eff8c7d8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef542ce4d0 0x55ef54696df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.209+0000 7eff8c7d8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ef545f5460 con 0x55ef542ce4d0 2024-04-25T14:14:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.209+0000 7eff8c7d8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ef545af590 con 0x55ef542c62f0 2024-04-25T14:14:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.209+0000 7eff8c7d8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ef545f1420 con 0x55ef542ef1b0 2024-04-25T14:14:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.209+0000 7eff7e7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef542ce4d0 0x55ef54696df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.209+0000 7eff7e7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef542ce4d0 0x55ef54696df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37468/0 (socket says 172.21.15.96:37468) 2024-04-25T14:14:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.209+0000 7eff7e7fc640 1 -- 172.21.15.96:0/3421392096 learned_addr learned my addr 172.21.15.96:0/3421392096 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.209+0000 7eff84924640 1 --2- 172.21.15.96:0/1762575249 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7eff680afae0 0x7eff680b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.209+0000 7eff7effd640 1 --2- 172.21.15.96:0/3421392096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef542c62f0 0x55ef5468fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.209+0000 7eff7e7fc640 1 --2- 172.21.15.96:0/3421392096 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef542ef1b0 0x55ef5468f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.210+0000 7eff7e7fc640 1 -- 172.21.15.96:0/3421392096 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef542ef1b0 msgr2=0x55ef5468f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.210+0000 7eff7e7fc640 1 --2- 172.21.15.96:0/3421392096 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef542ef1b0 0x55ef5468f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.210+0000 7eff7e7fc640 1 -- 172.21.15.96:0/3421392096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef542c62f0 msgr2=0x55ef5468fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.210+0000 7eff7e7fc640 1 --2- 172.21.15.96:0/3421392096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef542c62f0 0x55ef5468fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.210+0000 7eff7e7fc640 1 -- 172.21.15.96:0/3421392096 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef545bc0d0 con 0x55ef542ce4d0 2024-04-25T14:14:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.210+0000 7eff7effd640 1 --2- 172.21.15.96:0/3421392096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef542c62f0 0x55ef5468fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:14:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.210+0000 7eff7f7fe640 1 --2- 172.21.15.96:0/1762575249 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7eff680afae0 0x7eff680b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.210+0000 7eff84924640 1 -- 172.21.15.96:0/1762575249 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7eff780dedb0 con 0x7eff80108960 2024-04-25T14:14:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.210+0000 7eff7e7fc640 1 --2- 172.21.15.96:0/3421392096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef542ce4d0 0x55ef54696df0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7eff740058a0 tx=0x7eff7400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac99ecb3a8a1049f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.210+0000 7eff65ffb640 1 -- 172.21.15.96:0/3421392096 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7401e640 con 0x55ef542ce4d0 2024-04-25T14:14:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.210+0000 7eff65ffb640 1 -- 172.21.15.96:0/3421392096 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff740108e0 con 0x55ef542ce4d0 2024-04-25T14:14:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.210+0000 7eff65ffb640 1 -- 172.21.15.96:0/3421392096 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7401d920 con 0x55ef542ce4d0 2024-04-25T14:14:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.211+0000 7eff8c7d8740 1 -- 172.21.15.96:0/3421392096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef542ce4d0 msgr2=0x55ef54696df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.211+0000 7eff7f7fe640 1 --2- 172.21.15.96:0/1762575249 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7eff680afae0 0x7eff680b1ff0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7eff780a7020 tx=0x7eff780d59c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.211+0000 7eff8c7d8740 1 --2- 172.21.15.96:0/3421392096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef542ce4d0 0x55ef54696df0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7eff740058a0 tx=0x7eff7400b870 comp rx=0 tx=0).stop 2024-04-25T14:14:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.211+0000 7eff8c7d8740 1 -- 172.21.15.96:0/3421392096 shutdown_connections 2024-04-25T14:14:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.211+0000 7eff8c7d8740 1 --2- 172.21.15.96:0/3421392096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef542ce4d0 0x55ef54696df0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.211+0000 7eff8c7d8740 1 --2- 172.21.15.96:0/3421392096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef542c62f0 0x55ef5468fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.211+0000 7eff8c7d8740 1 --2- 172.21.15.96:0/3421392096 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef542ef1b0 0x55ef5468f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.211+0000 7eff8c7d8740 1 -- 172.21.15.96:0/3421392096 >> 172.21.15.96:0/3421392096 conn(0x55ef542e3300 msgr2=0x55ef5468b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.211+0000 7eff8c7d8740 1 -- 172.21.15.96:0/3421392096 shutdown_connections 2024-04-25T14:14:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.211+0000 7eff8c7d8740 1 -- 172.21.15.96:0/3421392096 wait complete. 2024-04-25T14:14:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.211+0000 7eff8c7d8740 1 -- start start 2024-04-25T14:14:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.212+0000 7eff8c7d8740 1 Processor -- start 2024-04-25T14:14:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.212+0000 7eff8c7d8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef542c62f0 0x55ef5469da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.212+0000 7eff8c7d8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef542ce4d0 0x55ef5469df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.212+0000 7eff8c7d8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef542ef1b0 0x55ef546a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.212+0000 7eff8c7d8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ef5469bc50 con 0x55ef542ef1b0 2024-04-25T14:14:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.212+0000 7eff8c7d8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ef5457cb60 con 0x55ef542ce4d0 2024-04-25T14:14:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.212+0000 7eff8c7d8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ef5469bdd0 con 0x55ef542c62f0 2024-04-25T14:14:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.212+0000 7eff8c7d8740 10 client.0 _ll_register_callbacks cb 0x55ef545a12b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:14:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.212+0000 7eff7e7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef542ef1b0 0x55ef546a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.212+0000 7eff7e7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef542ef1b0 0x55ef546a4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37470/0 (socket says 172.21.15.96:37470) 2024-04-25T14:14:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.212+0000 7eff7e7fc640 1 -- 172.21.15.96:0/4265667433 learned_addr learned my addr 172.21.15.96:0/4265667433 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.212+0000 7eff7e7fc640 1 --2- 172.21.15.96:0/4265667433 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef542c62f0 0x55ef5469da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.212+0000 7eff7effd640 1 --2- 172.21.15.96:0/4265667433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef542ce4d0 0x55ef5469df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.213+0000 7eff7e7fc640 1 -- 172.21.15.96:0/4265667433 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef542c62f0 msgr2=0x55ef5469da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.213+0000 7eff7e7fc640 1 --2- 172.21.15.96:0/4265667433 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef542c62f0 0x55ef5469da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.213+0000 7eff7e7fc640 1 -- 172.21.15.96:0/4265667433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef542ce4d0 msgr2=0x55ef5469df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.213+0000 7eff7e7fc640 1 --2- 172.21.15.96:0/4265667433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef542ce4d0 0x55ef5469df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.213+0000 7eff7e7fc640 1 -- 172.21.15.96:0/4265667433 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef54307210 con 0x55ef542ef1b0 2024-04-25T14:14:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.213+0000 7eff7effd640 1 --2- 172.21.15.96:0/4265667433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef542ce4d0 0x55ef5469df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:14:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.213+0000 7eff7e7fc640 1 --2- 172.21.15.96:0/4265667433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef542ef1b0 0x55ef546a4ec0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7eff74005870 tx=0x7eff7401d3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b2c7b835e10be82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.213+0000 7eff5effd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:14:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.213+0000 7eff5effd640 1 -- 172.21.15.96:0/4265667433 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff74010e30 con 0x55ef542ef1b0 2024-04-25T14:14:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.213+0000 7eff8c7d8740 1 -- 172.21.15.96:0/4265667433 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff740108e0 con 0x55ef542ef1b0 2024-04-25T14:14:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.213+0000 7eff5effd640 1 -- 172.21.15.96:0/4265667433 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7eff7401dab0 con 0x55ef542ef1b0 2024-04-25T14:14:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.213+0000 7eff5effd640 1 -- 172.21.15.96:0/4265667433 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7eff7401dcb0 con 0x55ef542ef1b0 2024-04-25T14:14:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.213+0000 7eff8c7d8740 10 client.15852 fetch_fsmap learned FSMap version 26 2024-04-25T14:14:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.213+0000 7eff8c7d8740 1 -- 172.21.15.96:0/4265667433 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ef543d3620 con 0x55ef542ef1b0 2024-04-25T14:14:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.213+0000 7eff5effd640 1 -- 172.21.15.96:0/4265667433 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff740108e0 con 0x55ef542ef1b0 2024-04-25T14:14:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.214+0000 7eff5effd640 1 -- 172.21.15.96:0/4265667433 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7eff74029030 con 0x55ef542ef1b0 2024-04-25T14:14:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.214+0000 7eff8c7d8740 10 client.15852 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:14:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.214+0000 7eff8c7d8740 10 client.15852 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:14:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.214+0000 7eff8c7d8740 20 client.15852 populate_metadata read hostname 'smithi096' 2024-04-25T14:14:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.214+0000 7eff8c7d8740 1 --2- 172.21.15.96:0/4265667433 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55ef542add80 0x55ef546abab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.214+0000 7eff8c7d8740 4 client.15852 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:14:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.214+0000 7eff8c7d8740 1 -- 172.21.15.96:0/4265667433 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef54603070 con 0x55ef542add80 2024-04-25T14:14:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.214+0000 7eff7effd640 1 --2- 172.21.15.96:0/4265667433 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55ef542add80 0x55ef546abab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.215+0000 7eff7effd640 1 --2- 172.21.15.96:0/4265667433 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55ef542add80 0x55ef546abab0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3441ac22b650f7 server_cookie=55ce7a8df66775a9 in_seq=0 out_seq=0 2024-04-25T14:14:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.215+0000 7eff5effd640 10 client.15852 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.215+0000 7eff5effd640 1 -- 172.21.15.96:0/4265667433 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ef54603070 con 0x55ef542add80 2024-04-25T14:14:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.215+0000 7eff5effd640 10 client.15852 handle_command_reply: tid=0 2024-04-25T14:14:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.215+0000 7eff8c7d8740 1 client.15852 shutdown 2024-04-25T14:14:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.215+0000 7eff8c7d8740 1 -- 172.21.15.96:0/4265667433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef542ef1b0 msgr2=0x55ef546a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.215+0000 7eff8c7d8740 1 --2- 172.21.15.96:0/4265667433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef542ef1b0 0x55ef546a4ec0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7eff74005870 tx=0x7eff7401d3e0 comp rx=0 tx=0).stop 2024-04-25T14:14:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.216+0000 7eff8c7d8740 1 -- 172.21.15.96:0/4265667433 shutdown_connections 2024-04-25T14:14:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.216+0000 7eff8c7d8740 1 --2- 172.21.15.96:0/4265667433 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55ef542add80 0x55ef546abab0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.216+0000 7eff8c7d8740 1 --2- 172.21.15.96:0/4265667433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef542ef1b0 0x55ef546a4ec0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.216+0000 7eff5effd640 0 client.15852 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.216+0000 7eff8c7d8740 1 --2- 172.21.15.96:0/4265667433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef542ce4d0 0x55ef5469df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.216+0000 7eff8c7d8740 1 --2- 172.21.15.96:0/4265667433 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef542c62f0 0x55ef5469da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.216+0000 7eff8c7d8740 1 -- 172.21.15.96:0/4265667433 >> 172.21.15.96:0/4265667433 conn(0x55ef542e3300 msgr2=0x55ef5468e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.216+0000 7eff8c7d8740 1 -- 172.21.15.96:0/4265667433 shutdown_connections 2024-04-25T14:14:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.216+0000 7eff8c7d8740 1 -- 172.21.15.96:0/4265667433 wait complete. 2024-04-25T14:14:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.216+0000 7eff8c7d8740 20 client.15852 trim_cache size 0 max 16384 2024-04-25T14:14:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.231+0000 7eff8c7d8740 1 Processor -- start 2024-04-25T14:14:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.231+0000 7eff8c7d8740 1 -- start start 2024-04-25T14:14:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.232+0000 7eff8c7d8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef542add80 0x55ef546b12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.232+0000 7eff8c7d8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef542c62f0 0x55ef546b17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.232+0000 7eff8c7d8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef542ce4d0 0x55ef546a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.232+0000 7eff8c7d8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7eff801a91d0 con 0x55ef542add80 2024-04-25T14:14:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.232+0000 7eff8c7d8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7eff801aad10 con 0x55ef542ce4d0 2024-04-25T14:14:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.232+0000 7eff8c7d8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7eff801a98a0 con 0x55ef542c62f0 2024-04-25T14:14:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.232+0000 7eff7e7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef542add80 0x55ef546b12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.232+0000 7eff7e7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef542add80 0x55ef546b12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37474/0 (socket says 172.21.15.96:37474) 2024-04-25T14:14:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.232+0000 7eff7e7fc640 1 -- 172.21.15.96:0/1765251627 learned_addr learned my addr 172.21.15.96:0/1765251627 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.232+0000 7eff7effd640 1 --2- 172.21.15.96:0/1765251627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef542c62f0 0x55ef546b17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.232+0000 7eff7e7fc640 1 -- 172.21.15.96:0/1765251627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef542c62f0 msgr2=0x55ef546b17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.232+0000 7eff7e7fc640 1 --2- 172.21.15.96:0/1765251627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef542c62f0 0x55ef546b17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.232+0000 7eff7e7fc640 1 -- 172.21.15.96:0/1765251627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef542ce4d0 msgr2=0x55ef546a4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.232+0000 7eff7e7fc640 1 --2- 172.21.15.96:0/1765251627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef542ce4d0 0x55ef546a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.232+0000 7eff7e7fc640 1 -- 172.21.15.96:0/1765251627 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef546a5320 con 0x55ef542add80 2024-04-25T14:14:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.232+0000 7eff7effd640 1 --2- 172.21.15.96:0/1765251627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef542c62f0 0x55ef546b17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:14:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.232+0000 7eff7e7fc640 1 --2- 172.21.15.96:0/1765251627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef542add80 0x55ef546b12b0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7eff7401fff0 tx=0x7eff74000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bff3ac092d8d1b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.232+0000 7eff657fa640 1 -- 172.21.15.96:0/1765251627 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7401de50 con 0x55ef542add80 2024-04-25T14:14:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.233+0000 7eff657fa640 1 -- 172.21.15.96:0/1765251627 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff74024040 con 0x55ef542add80 2024-04-25T14:14:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.233+0000 7eff657fa640 1 -- 172.21.15.96:0/1765251627 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff740120c0 con 0x55ef542add80 2024-04-25T14:14:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.233+0000 7eff8c7d8740 1 -- 172.21.15.96:0/1765251627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef542add80 msgr2=0x55ef546b12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.233+0000 7eff8c7d8740 1 --2- 172.21.15.96:0/1765251627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef542add80 0x55ef546b12b0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7eff7401fff0 tx=0x7eff74000bc0 comp rx=0 tx=0).stop 2024-04-25T14:14:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.233+0000 7eff8c7d8740 1 -- 172.21.15.96:0/1765251627 shutdown_connections 2024-04-25T14:14:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.233+0000 7eff8c7d8740 1 --2- 172.21.15.96:0/1765251627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef542ce4d0 0x55ef546a4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.233+0000 7eff8c7d8740 1 --2- 172.21.15.96:0/1765251627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef542c62f0 0x55ef546b17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.233+0000 7eff8c7d8740 1 --2- 172.21.15.96:0/1765251627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef542add80 0x55ef546b12b0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.233+0000 7eff8c7d8740 1 -- 172.21.15.96:0/1765251627 >> 172.21.15.96:0/1765251627 conn(0x55ef542e3300 msgr2=0x55ef5468f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.233+0000 7eff8c7d8740 1 -- 172.21.15.96:0/1765251627 shutdown_connections 2024-04-25T14:14:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.234+0000 7eff8c7d8740 1 -- 172.21.15.96:0/1765251627 wait complete. 2024-04-25T14:14:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.234+0000 7eff8c7d8740 1 -- start start 2024-04-25T14:14:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.234+0000 7eff8c7d8740 1 Processor -- start 2024-04-25T14:14:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.234+0000 7eff8c7d8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef542c62f0 0x55ef546a3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.234+0000 7eff8c7d8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef542ce4d0 0x55ef546a4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.234+0000 7eff8c7d8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef542831c0 0x55ef54711e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.234+0000 7eff8c7d8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7eff801aa640 con 0x55ef542ce4d0 2024-04-25T14:14:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.234+0000 7eff8c7d8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7eff801a9f70 con 0x55ef542831c0 2024-04-25T14:14:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.234+0000 7eff8c7d8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7eff801a71b0 con 0x55ef542c62f0 2024-04-25T14:14:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.235+0000 7eff8c7d8740 10 client.0 _ll_register_callbacks cb 0x7eff74012410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:14:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.235+0000 7eff7effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef542ce4d0 0x55ef546a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.235+0000 7eff7effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef542ce4d0 0x55ef546a4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37488/0 (socket says 172.21.15.96:37488) 2024-04-25T14:14:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.235+0000 7eff7effd640 1 -- 172.21.15.96:0/3476853972 learned_addr learned my addr 172.21.15.96:0/3476853972 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.235+0000 7eff7e7fc640 1 --2- 172.21.15.96:0/3476853972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef542831c0 0x55ef54711e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.235+0000 7eff7e7fc640 1 --2- 172.21.15.96:0/3476853972 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef542c62f0 0x55ef546a3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.235+0000 7eff7effd640 1 -- 172.21.15.96:0/3476853972 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef542c62f0 msgr2=0x55ef546a3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.235+0000 7eff7effd640 1 --2- 172.21.15.96:0/3476853972 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef542c62f0 0x55ef546a3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.235+0000 7eff7effd640 1 -- 172.21.15.96:0/3476853972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef542831c0 msgr2=0x55ef54711e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.235+0000 7eff7effd640 1 --2- 172.21.15.96:0/3476853972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef542831c0 0x55ef54711e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.235+0000 7eff7effd640 1 -- 172.21.15.96:0/3476853972 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef54712390 con 0x55ef542ce4d0 2024-04-25T14:14:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.235+0000 7eff7e7fc640 1 --2- 172.21.15.96:0/3476853972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef542831c0 0x55ef54711e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:14:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.235+0000 7eff7effd640 1 --2- 172.21.15.96:0/3476853972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef542ce4d0 0x55ef546a4270 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7eff70009900 tx=0x7eff70031730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8c90a6f2a768b68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.235+0000 7eff5effd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:14:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.235+0000 7eff5effd640 1 -- 172.21.15.96:0/3476853972 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff70035040 con 0x55ef542ce4d0 2024-04-25T14:14:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.235+0000 7eff5effd640 1 -- 172.21.15.96:0/3476853972 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7eff70004450 con 0x55ef542ce4d0 2024-04-25T14:14:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.235+0000 7eff8c7d8740 1 -- 172.21.15.96:0/3476853972 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff74024040 con 0x55ef542ce4d0 2024-04-25T14:14:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.235+0000 7eff5effd640 1 -- 172.21.15.96:0/3476853972 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7eff70053070 con 0x55ef542ce4d0 2024-04-25T14:14:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.236+0000 7eff8c7d8740 10 client.15858 fetch_fsmap learned FSMap version 26 2024-04-25T14:14:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.236+0000 7eff8c7d8740 1 -- 172.21.15.96:0/3476853972 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ef54713620 con 0x55ef542ce4d0 2024-04-25T14:14:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.236+0000 7eff5effd640 1 -- 172.21.15.96:0/3476853972 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff70031960 con 0x55ef542ce4d0 2024-04-25T14:14:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.236+0000 7eff5effd640 1 -- 172.21.15.96:0/3476853972 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7eff700039d0 con 0x55ef542ce4d0 2024-04-25T14:14:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.236+0000 7eff8c7d8740 10 client.15858 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:14:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.236+0000 7eff8c7d8740 10 client.15858 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:14:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.236+0000 7eff8c7d8740 20 client.15858 populate_metadata read hostname 'smithi096' 2024-04-25T14:14:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.236+0000 7eff8c7d8740 1 --2- 172.21.15.96:0/3476853972 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55ef54713f80 0x55ef547163a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.236+0000 7eff8c7d8740 4 client.15858 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:14:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.236+0000 7eff8c7d8740 1 -- 172.21.15.96:0/3476853972 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7eff74024040 con 0x55ef54713f80 2024-04-25T14:14:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.236+0000 7eff7e7fc640 1 --2- 172.21.15.96:0/3476853972 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55ef54713f80 0x55ef547163a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.237+0000 7eff5effd640 10 client.15858 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.237+0000 7eff7e7fc640 1 --2- 172.21.15.96:0/3476853972 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55ef54713f80 0x55ef547163a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d149767016daf38a server_cookie=5424a10c11010e60 in_seq=0 out_seq=0 2024-04-25T14:14:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.240+0000 7eff5effd640 1 -- 172.21.15.96:0/3476853972 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7eff74024040 con 0x55ef54713f80 2024-04-25T14:14:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.241+0000 7eff5effd640 10 client.15858 handle_command_reply: tid=0 2024-04-25T14:14:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.241+0000 7eff8c7d8740 1 client.15858 shutdown 2024-04-25T14:14:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.241+0000 7eff8c7d8740 1 -- 172.21.15.96:0/3476853972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef542ce4d0 msgr2=0x55ef546a4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.241+0000 7eff8c7d8740 1 --2- 172.21.15.96:0/3476853972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef542ce4d0 0x55ef546a4270 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7eff70009900 tx=0x7eff70031730 comp rx=0 tx=0).stop 2024-04-25T14:14:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.241+0000 7eff8c7d8740 1 -- 172.21.15.96:0/3476853972 shutdown_connections 2024-04-25T14:14:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.241+0000 7eff8c7d8740 1 --2- 172.21.15.96:0/3476853972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef542831c0 0x55ef54711e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.241+0000 7eff8c7d8740 1 --2- 172.21.15.96:0/3476853972 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55ef54713f80 0x55ef547163a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.241+0000 7eff8c7d8740 1 --2- 172.21.15.96:0/3476853972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef542ce4d0 0x55ef546a4270 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.241+0000 7eff8c7d8740 1 --2- 172.21.15.96:0/3476853972 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef542c62f0 0x55ef546a3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.241+0000 7eff5effd640 0 client.15858 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.241+0000 7eff8c7d8740 1 -- 172.21.15.96:0/3476853972 >> 172.21.15.96:0/3476853972 conn(0x55ef542e3300 msgr2=0x55ef5468e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.241+0000 7eff8c7d8740 1 -- 172.21.15.96:0/3476853972 shutdown_connections 2024-04-25T14:14:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.241+0000 7eff8c7d8740 1 -- 172.21.15.96:0/3476853972 wait complete. 2024-04-25T14:14:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.241+0000 7eff8c7d8740 20 client.15858 trim_cache size 0 max 16384 2024-04-25T14:14:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T14:14:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T14:14:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T14:14:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T14:14:31.248 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.242+0000 7eff85926640 1 -- 172.21.15.96:0/1762575249 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7eff680afae0 msgr2=0x7eff680b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:31.248 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.242+0000 7eff85926640 1 --2- 172.21.15.96:0/1762575249 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7eff680afae0 0x7eff680b1ff0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7eff780a7020 tx=0x7eff780d59c0 comp rx=0 tx=0).stop 2024-04-25T14:14:31.248 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.242+0000 7eff85926640 1 -- 172.21.15.96:0/1762575249 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff80108960 msgr2=0x7eff801a1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:31.248 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.242+0000 7eff85926640 1 --2- 172.21.15.96:0/1762575249 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff80108960 0x7eff801a1710 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7eff7800eaa0 tx=0x7eff7800ef60 comp rx=0 tx=0).stop 2024-04-25T14:14:31.248 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.242+0000 7eff85926640 1 -- 172.21.15.96:0/1762575249 shutdown_connections 2024-04-25T14:14:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.242+0000 7eff85926640 1 --2- 172.21.15.96:0/1762575249 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7eff680afae0 0x7eff680b1ff0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.242+0000 7eff85926640 1 --2- 172.21.15.96:0/1762575249 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff80108960 0x7eff801a1710 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.242+0000 7eff85926640 1 --2- 172.21.15.96:0/1762575249 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff80103c60 0x7eff8019a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.242+0000 7eff85926640 1 --2- 172.21.15.96:0/1762575249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff80102a20 0x7eff8019a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.242+0000 7eff85926640 1 -- 172.21.15.96:0/1762575249 >> 172.21.15.96:0/1762575249 conn(0x7eff800fdc10 msgr2=0x7eff80104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.242+0000 7eff85926640 1 -- 172.21.15.96:0/1762575249 shutdown_connections 2024-04-25T14:14:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:31.242+0000 7eff85926640 1 -- 172.21.15.96:0/1762575249 wait complete. 2024-04-25T14:14:32.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:32 smithi096 ceph-mon[29207]: pgmap v373: 97 pgs: 97 active+clean; 667 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 853 B/s wr, 13 op/s 2024-04-25T14:14:32.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:32 smithi097 ceph-mon[32720]: pgmap v373: 97 pgs: 97 active+clean; 667 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 853 B/s wr, 13 op/s 2024-04-25T14:14:32.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:32 smithi151 ceph-mon[32990]: pgmap v373: 97 pgs: 97 active+clean; 667 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 853 B/s wr, 13 op/s 2024-04-25T14:14:34.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:34 smithi096 ceph-mon[29207]: pgmap v374: 97 pgs: 97 active+clean; 667 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-25T14:14:34.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:34 smithi097 ceph-mon[32720]: pgmap v374: 97 pgs: 97 active+clean; 667 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-25T14:14:34.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:34 smithi151 ceph-mon[32990]: pgmap v374: 97 pgs: 97 active+clean; 667 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-25T14:14:36.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:36 smithi096 ceph-mon[29207]: pgmap v375: 97 pgs: 97 active+clean; 667 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-25T14:14:36.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:36 smithi097 ceph-mon[32720]: pgmap v375: 97 pgs: 97 active+clean; 667 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-25T14:14:36.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:36 smithi151 ceph-mon[32990]: pgmap v375: 97 pgs: 97 active+clean; 667 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-25T14:14:38.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:38 smithi096 ceph-mon[29207]: pgmap v376: 97 pgs: 97 active+clean; 668 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-25T14:14:38.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:38 smithi097 ceph-mon[32720]: pgmap v376: 97 pgs: 97 active+clean; 668 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-25T14:14:38.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:38 smithi151 ceph-mon[32990]: pgmap v376: 97 pgs: 97 active+clean; 668 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-25T14:14:40.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:40 smithi096 ceph-mon[29207]: pgmap v377: 97 pgs: 97 active+clean; 668 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-25T14:14:40.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:40 smithi097 ceph-mon[32720]: pgmap v377: 97 pgs: 97 active+clean; 668 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-25T14:14:40.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:40 smithi151 ceph-mon[32990]: pgmap v377: 97 pgs: 97 active+clean; 668 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-25T14:14:42.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:42 smithi096 ceph-mon[29207]: pgmap v378: 97 pgs: 97 active+clean; 668 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T14:14:42.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:42 smithi097 ceph-mon[32720]: pgmap v378: 97 pgs: 97 active+clean; 668 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T14:14:42.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:42 smithi151 ceph-mon[32990]: pgmap v378: 97 pgs: 97 active+clean; 668 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T14:14:44.086 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:14:44.086 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:14:44.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.210+0000 7f94b6020640 1 -- 172.21.15.96:0/378824471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94b0109890 msgr2=0x7f94b010bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:44.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.210+0000 7f94b6020640 1 --2- 172.21.15.96:0/378824471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94b0109890 0x7f94b010bcb0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f94a8009ca0 tx=0x7f94a802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.210+0000 7f94b6020640 1 -- 172.21.15.96:0/378824471 shutdown_connections 2024-04-25T14:14:44.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.210+0000 7f94b6020640 1 --2- 172.21.15.96:0/378824471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94b0109890 0x7f94b010bcb0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.210+0000 7f94b6020640 1 --2- 172.21.15.96:0/378824471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94b0106f30 0x7f94b0109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.210+0000 7f94b6020640 1 --2- 172.21.15.96:0/378824471 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94b01045d0 0x7f94b01069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.210+0000 7f94b6020640 1 -- 172.21.15.96:0/378824471 >> 172.21.15.96:0/378824471 conn(0x7f94b00fdbd0 msgr2=0x7f94b0100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:44.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.210+0000 7f94b6020640 1 -- 172.21.15.96:0/378824471 shutdown_connections 2024-04-25T14:14:44.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.210+0000 7f94b6020640 1 -- 172.21.15.96:0/378824471 wait complete. 2024-04-25T14:14:44.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.211+0000 7f94b6020640 1 Processor -- start 2024-04-25T14:14:44.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.211+0000 7f94b6020640 1 -- start start 2024-04-25T14:14:44.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.211+0000 7f94b6020640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94b01045d0 0x7f94b019a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.211+0000 7f94b6020640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94b0106f30 0x7f94b019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.211+0000 7f94b6020640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94b0109890 0x7f94b01a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.211+0000 7f94b6020640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f94b010f610 con 0x7f94b0109890 2024-04-25T14:14:44.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.211+0000 7f94b6020640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f94b010f490 con 0x7f94b01045d0 2024-04-25T14:14:44.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.211+0000 7f94b6020640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f94b010f790 con 0x7f94b0106f30 2024-04-25T14:14:44.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.211+0000 7f94affff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94b0109890 0x7f94b01a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.211+0000 7f94affff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94b0109890 0x7f94b01a1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34430/0 (socket says 172.21.15.96:34430) 2024-04-25T14:14:44.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.211+0000 7f94affff640 1 -- 172.21.15.96:0/715885053 learned_addr learned my addr 172.21.15.96:0/715885053 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:44.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.211+0000 7f94aeffd640 1 --2- 172.21.15.96:0/715885053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94b0106f30 0x7f94b019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.212+0000 7f94af7fe640 1 --2- 172.21.15.96:0/715885053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94b01045d0 0x7f94b019a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.212+0000 7f94affff640 1 -- 172.21.15.96:0/715885053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94b0106f30 msgr2=0x7f94b019aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:44.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.212+0000 7f94affff640 1 --2- 172.21.15.96:0/715885053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94b0106f30 0x7f94b019aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.212+0000 7f94affff640 1 -- 172.21.15.96:0/715885053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94b01045d0 msgr2=0x7f94b019a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:44.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.212+0000 7f94affff640 1 --2- 172.21.15.96:0/715885053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94b01045d0 0x7f94b019a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.212+0000 7f94affff640 1 -- 172.21.15.96:0/715885053 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94b01a2080 con 0x7f94b0109890 2024-04-25T14:14:44.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.212+0000 7f94aeffd640 1 --2- 172.21.15.96:0/715885053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94b0106f30 0x7f94b019aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:14:44.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.212+0000 7f94affff640 1 --2- 172.21.15.96:0/715885053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94b0109890 0x7f94b01a1a10 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f94a8009430 tx=0x7f94a8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:44.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.212+0000 7f94acff9640 1 -- 172.21.15.96:0/715885053 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a802f380 con 0x7f94b0109890 2024-04-25T14:14:44.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.212+0000 7f94acff9640 1 -- 172.21.15.96:0/715885053 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94a80317c0 con 0x7f94b0109890 2024-04-25T14:14:44.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.212+0000 7f94acff9640 1 -- 172.21.15.96:0/715885053 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a802fcd0 con 0x7f94b0109890 2024-04-25T14:14:44.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.213+0000 7f94b6020640 1 -- 172.21.15.96:0/715885053 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94b01a2310 con 0x7f94b0109890 2024-04-25T14:14:44.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.213+0000 7f94b6020640 1 -- 172.21.15.96:0/715885053 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94b01a2750 con 0x7f94b0109890 2024-04-25T14:14:44.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.215+0000 7f94bcfab740 1 Processor -- start 2024-04-25T14:14:44.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.216+0000 7f94acff9640 1 -- 172.21.15.96:0/715885053 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f94a8004040 con 0x7f94b0109890 2024-04-25T14:14:44.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.217+0000 7f94bcfab740 1 -- start start 2024-04-25T14:14:44.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.217+0000 7f94bcfab740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555fa9e3e390 0x555faa1dfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.217+0000 7f94bcfab740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555fa9e154d0 0x555faa1dffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.217+0000 7f94bcfab740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555fa9e1d6b0 0x555faa1e0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.217+0000 7f94bcfab740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555faa144d80 con 0x555fa9e3e390 2024-04-25T14:14:44.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.217+0000 7f94bcfab740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555faa0fe8b0 con 0x555fa9e1d6b0 2024-04-25T14:14:44.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.217+0000 7f94bcfab740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555faa140d40 con 0x555fa9e154d0 2024-04-25T14:14:44.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.217+0000 7f94aeffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555fa9e3e390 0x555faa1dfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.217+0000 7f94aeffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555fa9e3e390 0x555faa1dfa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34446/0 (socket says 172.21.15.96:34446) 2024-04-25T14:14:44.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.217+0000 7f94aeffd640 1 -- 172.21.15.96:0/4063556007 learned_addr learned my addr 172.21.15.96:0/4063556007 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:44.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.217+0000 7f94acff9640 1 --2- 172.21.15.96:0/715885053 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f949c0afc90 0x7f949c0b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.217+0000 7f94aeffd640 1 --2- 172.21.15.96:0/4063556007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555fa9e1d6b0 0x555faa1e0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.217+0000 7f94af7fe640 1 --2- 172.21.15.96:0/4063556007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555fa9e154d0 0x555faa1dffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.217+0000 7f94acff9640 1 -- 172.21.15.96:0/715885053 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f94a81002a0 con 0x7f94b0109890 2024-04-25T14:14:44.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.218+0000 7f94affff640 1 --2- 172.21.15.96:0/715885053 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f949c0afc90 0x7f949c0b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.218+0000 7f94aeffd640 1 -- 172.21.15.96:0/4063556007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555fa9e154d0 msgr2=0x555faa1dffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:44.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.218+0000 7f94aeffd640 1 --2- 172.21.15.96:0/4063556007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555fa9e154d0 0x555faa1dffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.218+0000 7f94aeffd640 1 -- 172.21.15.96:0/4063556007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555fa9e1d6b0 msgr2=0x555faa1e0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:44.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.218+0000 7f94aeffd640 1 --2- 172.21.15.96:0/4063556007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555fa9e1d6b0 0x555faa1e0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.218+0000 7f94aeffd640 1 -- 172.21.15.96:0/4063556007 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555faa0cfa40 con 0x555fa9e3e390 2024-04-25T14:14:44.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.218+0000 7f94aeffd640 1 --2- 172.21.15.96:0/4063556007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555fa9e3e390 0x555faa1dfa60 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f9498005d00 tx=0x7f949800bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f1b13ca3923d88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:44.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.218+0000 7f94a5ffb640 1 -- 172.21.15.96:0/4063556007 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949800ebe0 con 0x555fa9e3e390 2024-04-25T14:14:44.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.218+0000 7f94a5ffb640 1 -- 172.21.15.96:0/4063556007 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f949800ed80 con 0x555fa9e3e390 2024-04-25T14:14:44.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.218+0000 7f94affff640 1 --2- 172.21.15.96:0/715885053 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f949c0afc90 0x7f949c0b21a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f94a8031b50 tx=0x7f94a802a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:44.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.219+0000 7f94a5ffb640 1 -- 172.21.15.96:0/4063556007 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9498005590 con 0x555fa9e3e390 2024-04-25T14:14:44.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.219+0000 7f94bcfab740 1 -- 172.21.15.96:0/4063556007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555fa9e3e390 msgr2=0x555faa1dfa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:44.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.219+0000 7f94bcfab740 1 --2- 172.21.15.96:0/4063556007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555fa9e3e390 0x555faa1dfa60 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f9498005d00 tx=0x7f949800bc30 comp rx=0 tx=0).stop 2024-04-25T14:14:44.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.220+0000 7f94bcfab740 1 -- 172.21.15.96:0/4063556007 shutdown_connections 2024-04-25T14:14:44.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.220+0000 7f94bcfab740 1 --2- 172.21.15.96:0/4063556007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555fa9e1d6b0 0x555faa1e0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.220+0000 7f94bcfab740 1 --2- 172.21.15.96:0/4063556007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555fa9e154d0 0x555faa1dffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.220+0000 7f94bcfab740 1 --2- 172.21.15.96:0/4063556007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555fa9e3e390 0x555faa1dfa60 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.220+0000 7f94bcfab740 1 -- 172.21.15.96:0/4063556007 >> 172.21.15.96:0/4063556007 conn(0x555fa9e324e0 msgr2=0x555faa1db270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:44.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.220+0000 7f94bcfab740 1 -- 172.21.15.96:0/4063556007 shutdown_connections 2024-04-25T14:14:44.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.220+0000 7f94bcfab740 1 -- 172.21.15.96:0/4063556007 wait complete. 2024-04-25T14:14:44.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.220+0000 7f94bcfab740 1 -- start start 2024-04-25T14:14:44.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.220+0000 7f94bcfab740 1 Processor -- start 2024-04-25T14:14:44.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.221+0000 7f94bcfab740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555fa9e154d0 0x555faa1e98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.221+0000 7f94bcfab740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555fa9e1d6b0 0x555faa1e9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.221+0000 7f94aeffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555fa9e154d0 0x555faa1e98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.221+0000 7f94aeffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555fa9e154d0 0x555faa1e98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34462/0 (socket says 172.21.15.96:34462) 2024-04-25T14:14:44.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.221+0000 7f94aeffd640 1 -- 172.21.15.96:0/3564783625 learned_addr learned my addr 172.21.15.96:0/3564783625 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:44.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.221+0000 7f94bcfab740 1 --2- 172.21.15.96:0/3564783625 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555fa9e3e390 0x555faa1f0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.221+0000 7f94bcfab740 1 -- 172.21.15.96:0/3564783625 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555faa1e1710 con 0x555fa9e154d0 2024-04-25T14:14:44.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.221+0000 7f94bcfab740 1 -- 172.21.15.96:0/3564783625 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555faa0cbd10 con 0x555fa9e1d6b0 2024-04-25T14:14:44.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.221+0000 7f94bcfab740 1 -- 172.21.15.96:0/3564783625 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555faa1e1890 con 0x555fa9e3e390 2024-04-25T14:14:44.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.221+0000 7f94af7fe640 1 --2- 172.21.15.96:0/3564783625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555fa9e1d6b0 0x555faa1e9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.222+0000 7f94aeffd640 1 -- 172.21.15.96:0/3564783625 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555fa9e3e390 msgr2=0x555faa1f0d80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T14:14:44.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.222+0000 7f94aeffd640 1 --2- 172.21.15.96:0/3564783625 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555fa9e3e390 0x555faa1f0d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.222+0000 7f94aeffd640 1 -- 172.21.15.96:0/3564783625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555fa9e1d6b0 msgr2=0x555faa1e9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:44.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.222+0000 7f94aeffd640 1 --2- 172.21.15.96:0/3564783625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555fa9e1d6b0 0x555faa1e9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.222+0000 7f94aeffd640 1 -- 172.21.15.96:0/3564783625 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fa9f207d0 con 0x555fa9e154d0 2024-04-25T14:14:44.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.222+0000 7f94af7fe640 1 --2- 172.21.15.96:0/3564783625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555fa9e1d6b0 0x555faa1e9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:14:44.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.222+0000 7f94bcfab740 10 client.0 _ll_register_callbacks cb 0x555faa0f8210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:14:44.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.222+0000 7f94bcfab740 1 -- 172.21.15.96:0/3564783625 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f949800ed80 con 0x555fa9e154d0 2024-04-25T14:14:44.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.222+0000 7f94aeffd640 1 --2- 172.21.15.96:0/3564783625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555fa9e154d0 0x555faa1e98e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f9498012040 tx=0x7f94980122e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39c89afddb4d85fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:44.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.222+0000 7f948effd640 10 client.15870 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:14:44.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.222+0000 7f948effd640 1 -- 172.21.15.96:0/3564783625 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949802d060 con 0x555fa9e154d0 2024-04-25T14:14:44.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.222+0000 7f948effd640 1 -- 172.21.15.96:0/3564783625 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f949801e5e0 con 0x555fa9e154d0 2024-04-25T14:14:44.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.222+0000 7f948effd640 1 -- 172.21.15.96:0/3564783625 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f949800ebe0 con 0x555fa9e154d0 2024-04-25T14:14:44.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.222+0000 7f94bcfab740 10 client.15870 fetch_fsmap learned FSMap version 26 2024-04-25T14:14:44.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.222+0000 7f94bcfab740 1 -- 172.21.15.96:0/3564783625 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({fsmap=26}) -- 0x555fa9e563f0 con 0x555fa9e154d0 2024-04-25T14:14:44.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.223+0000 7f948effd640 1 -- 172.21.15.96:0/3564783625 <== mon.0 v2:172.21.15.96:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9498027070 con 0x555fa9e154d0 2024-04-25T14:14:44.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.223+0000 7f94bcfab740 10 client.15870 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:14:44.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.223+0000 7f94bcfab740 10 client.15870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:14:44.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.223+0000 7f94bcfab740 20 client.15870 populate_metadata read hostname 'smithi096' 2024-04-25T14:14:44.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.223+0000 7f94bcfab740 1 --2- 172.21.15.96:0/3564783625 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x555fa9dfcf60 0x555faa1f6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.223+0000 7f94bcfab740 4 client.15870 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:14:44.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.223+0000 7f94bcfab740 1 -- 172.21.15.96:0/3564783625 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555faa152990 con 0x555fa9dfcf60 2024-04-25T14:14:44.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.223+0000 7f94af7fe640 1 --2- 172.21.15.96:0/3564783625 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x555fa9dfcf60 0x555faa1f6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.224+0000 7f94af7fe640 1 --2- 172.21.15.96:0/3564783625 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x555fa9dfcf60 0x555faa1f6cf0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83f40fb688994126 server_cookie=4fd823a4aeb8d38c in_seq=0 out_seq=0 2024-04-25T14:14:44.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.224+0000 7f948effd640 10 client.15870 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:44.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.224+0000 7f948effd640 1 -- 172.21.15.96:0/3564783625 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555faa152990 con 0x555fa9dfcf60 2024-04-25T14:14:44.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.224+0000 7f948effd640 10 client.15870 handle_command_reply: tid=0 2024-04-25T14:14:44.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.224+0000 7f94bcfab740 1 client.15870 shutdown 2024-04-25T14:14:44.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.225+0000 7f94bcfab740 1 -- 172.21.15.96:0/3564783625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555fa9e154d0 msgr2=0x555faa1e98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:44.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.225+0000 7f94bcfab740 1 --2- 172.21.15.96:0/3564783625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555fa9e154d0 0x555faa1e98e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f9498012040 tx=0x7f94980122e0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.225+0000 7f94bcfab740 1 -- 172.21.15.96:0/3564783625 shutdown_connections 2024-04-25T14:14:44.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.225+0000 7f94bcfab740 1 --2- 172.21.15.96:0/3564783625 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555fa9e3e390 0x555faa1f0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.225+0000 7f94bcfab740 1 --2- 172.21.15.96:0/3564783625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555fa9e1d6b0 0x555faa1e9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.225+0000 7f94bcfab740 1 --2- 172.21.15.96:0/3564783625 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x555fa9dfcf60 0x555faa1f6cf0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.225+0000 7f94bcfab740 1 --2- 172.21.15.96:0/3564783625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555fa9e154d0 0x555faa1e98e0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.225+0000 7f94bcfab740 1 -- 172.21.15.96:0/3564783625 >> 172.21.15.96:0/3564783625 conn(0x555fa9e324e0 msgr2=0x555faa1de180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:44.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.225+0000 7f948effd640 0 client.15870 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:44.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.225+0000 7f94bcfab740 1 -- 172.21.15.96:0/3564783625 shutdown_connections 2024-04-25T14:14:44.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.225+0000 7f94bcfab740 1 -- 172.21.15.96:0/3564783625 wait complete. 2024-04-25T14:14:44.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.225+0000 7f94bcfab740 20 client.15870 trim_cache size 0 max 16384 2024-04-25T14:14:44.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.243+0000 7f94bcfab740 1 Processor -- start 2024-04-25T14:14:44.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.243+0000 7f94bcfab740 1 -- start start 2024-04-25T14:14:44.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.244+0000 7f94bcfab740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555fa9e154d0 0x555faa1e8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.244+0000 7f94bcfab740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555fa9e1d6b0 0x555faa1da0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.244+0000 7f94bcfab740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555fa9e3e390 0x555faa1f0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.244+0000 7f94bcfab740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f94b01a90e0 con 0x555fa9e154d0 2024-04-25T14:14:44.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.244+0000 7f94bcfab740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f94b01aac20 con 0x555fa9e1d6b0 2024-04-25T14:14:44.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.244+0000 7f94bcfab740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f94b01a97b0 con 0x555fa9e3e390 2024-04-25T14:14:44.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.244+0000 7f94aeffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555fa9e154d0 0x555faa1e8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.244+0000 7f94aeffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555fa9e154d0 0x555faa1e8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34470/0 (socket says 172.21.15.96:34470) 2024-04-25T14:14:44.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.244+0000 7f94aeffd640 1 -- 172.21.15.96:0/2650996098 learned_addr learned my addr 172.21.15.96:0/2650996098 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:44.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.244+0000 7f94af7fe640 1 --2- 172.21.15.96:0/2650996098 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555fa9e1d6b0 0x555faa1da0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.244+0000 7f94aeffd640 1 --2- 172.21.15.96:0/2650996098 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555fa9e3e390 0x555faa1f0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.244+0000 7f94aeffd640 1 -- 172.21.15.96:0/2650996098 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555fa9e3e390 msgr2=0x555faa1f0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:44.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.244+0000 7f94aeffd640 1 --2- 172.21.15.96:0/2650996098 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555fa9e3e390 0x555faa1f0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.244+0000 7f94aeffd640 1 -- 172.21.15.96:0/2650996098 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555fa9e1d6b0 msgr2=0x555faa1da0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:44.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.244+0000 7f94aeffd640 1 --2- 172.21.15.96:0/2650996098 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555fa9e1d6b0 0x555faa1da0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.244+0000 7f94aeffd640 1 -- 172.21.15.96:0/2650996098 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fa9e160f0 con 0x555fa9e154d0 2024-04-25T14:14:44.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.244+0000 7f94aeffd640 1 --2- 172.21.15.96:0/2650996098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555fa9e154d0 0x555faa1e8910 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f94980057b0 tx=0x7f9498023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b57374a68816852 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:44.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.245+0000 7f94a57fa640 1 -- 172.21.15.96:0/2650996098 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949802d070 con 0x555fa9e154d0 2024-04-25T14:14:44.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.245+0000 7f94a57fa640 1 -- 172.21.15.96:0/2650996098 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f949800ed80 con 0x555fa9e154d0 2024-04-25T14:14:44.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.245+0000 7f94a57fa640 1 -- 172.21.15.96:0/2650996098 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9498003850 con 0x555fa9e154d0 2024-04-25T14:14:44.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.245+0000 7f94bcfab740 1 -- 172.21.15.96:0/2650996098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555fa9e154d0 msgr2=0x555faa1e8910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:44.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.245+0000 7f94bcfab740 1 --2- 172.21.15.96:0/2650996098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555fa9e154d0 0x555faa1e8910 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f94980057b0 tx=0x7f9498023000 comp rx=0 tx=0).stop 2024-04-25T14:14:44.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.246+0000 7f94bcfab740 1 -- 172.21.15.96:0/2650996098 shutdown_connections 2024-04-25T14:14:44.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.246+0000 7f94bcfab740 1 --2- 172.21.15.96:0/2650996098 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555fa9e3e390 0x555faa1f0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.246+0000 7f94bcfab740 1 --2- 172.21.15.96:0/2650996098 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555fa9e1d6b0 0x555faa1da0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.246+0000 7f94bcfab740 1 --2- 172.21.15.96:0/2650996098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555fa9e154d0 0x555faa1e8910 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.246+0000 7f94bcfab740 1 -- 172.21.15.96:0/2650996098 >> 172.21.15.96:0/2650996098 conn(0x555fa9e324e0 msgr2=0x555faa1d99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:44.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.246+0000 7f94bcfab740 1 -- 172.21.15.96:0/2650996098 shutdown_connections 2024-04-25T14:14:44.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.246+0000 7f94bcfab740 1 -- 172.21.15.96:0/2650996098 wait complete. 2024-04-25T14:14:44.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.246+0000 7f94bcfab740 1 -- start start 2024-04-25T14:14:44.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.246+0000 7f94bcfab740 1 Processor -- start 2024-04-25T14:14:44.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.247+0000 7f94bcfab740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555fa9e154d0 0x555faa1edf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.247+0000 7f94bcfab740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555fa9e1d6b0 0x555faa1ee4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.247+0000 7f94bcfab740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555fa9e3e390 0x555faa25d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.247+0000 7f94bcfab740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f94b01aa550 con 0x555fa9e3e390 2024-04-25T14:14:44.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.247+0000 7f94bcfab740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f94b01a9e80 con 0x555fa9e1d6b0 2024-04-25T14:14:44.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.247+0000 7f94bcfab740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f94b01a70c0 con 0x555fa9e154d0 2024-04-25T14:14:44.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.247+0000 7f94bcfab740 10 client.0 _ll_register_callbacks cb 0x7f94980125f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:14:44.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.247+0000 7f94aeffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555fa9e3e390 0x555faa25d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.247+0000 7f94aeffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555fa9e3e390 0x555faa25d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34482/0 (socket says 172.21.15.96:34482) 2024-04-25T14:14:44.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.247+0000 7f94aeffd640 1 -- 172.21.15.96:0/3077314111 learned_addr learned my addr 172.21.15.96:0/3077314111 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:44.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.247+0000 7f94af7fe640 1 --2- 172.21.15.96:0/3077314111 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555fa9e1d6b0 0x555faa1ee4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.247+0000 7f94aeffd640 1 --2- 172.21.15.96:0/3077314111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555fa9e154d0 0x555faa1edf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.247+0000 7f94af7fe640 1 -- 172.21.15.96:0/3077314111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555fa9e154d0 msgr2=0x555faa1edf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:44.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.247+0000 7f94af7fe640 1 --2- 172.21.15.96:0/3077314111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555fa9e154d0 0x555faa1edf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.247+0000 7f94af7fe640 1 -- 172.21.15.96:0/3077314111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555fa9e3e390 msgr2=0x555faa25d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:44.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.247+0000 7f94af7fe640 1 --2- 172.21.15.96:0/3077314111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555fa9e3e390 0x555faa25d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.247+0000 7f94af7fe640 1 -- 172.21.15.96:0/3077314111 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555faa1054b0 con 0x555fa9e1d6b0 2024-04-25T14:14:44.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.247+0000 7f94aeffd640 1 --2- 172.21.15.96:0/3077314111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555fa9e3e390 0x555faa25d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:14:44.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.248+0000 7f94af7fe640 1 --2- 172.21.15.96:0/3077314111 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555fa9e1d6b0 0x555faa1ee4d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x555faa1f7420 tx=0x7f94a000bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=daf3f0471e7b7c40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:44.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.248+0000 7f948effd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T14:14:44.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.248+0000 7f948effd640 1 -- 172.21.15.96:0/3077314111 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a0005220 con 0x555fa9e1d6b0 2024-04-25T14:14:44.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.248+0000 7f94bcfab740 1 -- 172.21.15.96:0/3077314111 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f949800ed80 con 0x555fa9e1d6b0 2024-04-25T14:14:44.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.248+0000 7f948effd640 1 -- 172.21.15.96:0/3077314111 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f94a00053c0 con 0x555fa9e1d6b0 2024-04-25T14:14:44.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.248+0000 7f948effd640 1 -- 172.21.15.96:0/3077314111 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f94a00055e0 con 0x555fa9e1d6b0 2024-04-25T14:14:44.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.248+0000 7f94bcfab740 10 client.25379 fetch_fsmap learned FSMap version 26 2024-04-25T14:14:44.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.248+0000 7f94bcfab740 1 -- 172.21.15.96:0/3077314111 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555faa25e370 con 0x555fa9e1d6b0 2024-04-25T14:14:44.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.249+0000 7f948effd640 1 -- 172.21.15.96:0/3077314111 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f949800ed80 con 0x555fa9e1d6b0 2024-04-25T14:14:44.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.249+0000 7f948effd640 1 -- 172.21.15.96:0/3077314111 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f94a0011600 con 0x555fa9e1d6b0 2024-04-25T14:14:44.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.249+0000 7f94bcfab740 10 client.25379 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:14:44.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.249+0000 7f94bcfab740 10 client.25379 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:14:44.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.249+0000 7f94bcfab740 20 client.25379 populate_metadata read hostname 'smithi096' 2024-04-25T14:14:44.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.249+0000 7f94bcfab740 1 --2- 172.21.15.96:0/3077314111 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x555fa9dd23a0 0x555faa260d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.249+0000 7f94bcfab740 4 client.25379 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:14:44.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.249+0000 7f94bcfab740 1 -- 172.21.15.96:0/3077314111 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555faa14c800 con 0x555fa9dd23a0 2024-04-25T14:14:44.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.249+0000 7f94aeffd640 1 --2- 172.21.15.96:0/3077314111 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x555fa9dd23a0 0x555faa260d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.250+0000 7f948effd640 10 client.25379 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:44.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.250+0000 7f94aeffd640 1 --2- 172.21.15.96:0/3077314111 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x555fa9dd23a0 0x555faa260d60 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30df6288efff4c08 server_cookie=8e8443afe0e0fe9b in_seq=0 out_seq=0 2024-04-25T14:14:44.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.250+0000 7f948effd640 1 -- 172.21.15.96:0/3077314111 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555faa14c800 con 0x555fa9dd23a0 2024-04-25T14:14:44.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.250+0000 7f948effd640 10 client.25379 handle_command_reply: tid=0 2024-04-25T14:14:44.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.250+0000 7f94bcfab740 1 client.25379 shutdown 2024-04-25T14:14:44.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.250+0000 7f94bcfab740 1 -- 172.21.15.96:0/3077314111 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555fa9e1d6b0 msgr2=0x555faa1ee4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:44.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.250+0000 7f94bcfab740 1 --2- 172.21.15.96:0/3077314111 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555fa9e1d6b0 0x555faa1ee4d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x555faa1f7420 tx=0x7f94a000bed0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.250+0000 7f94bcfab740 1 -- 172.21.15.96:0/3077314111 shutdown_connections 2024-04-25T14:14:44.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.250+0000 7f94bcfab740 1 --2- 172.21.15.96:0/3077314111 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x555fa9dd23a0 0x555faa260d60 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.250+0000 7f94bcfab740 1 --2- 172.21.15.96:0/3077314111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555fa9e3e390 0x555faa25d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.250+0000 7f94bcfab740 1 --2- 172.21.15.96:0/3077314111 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555fa9e1d6b0 0x555faa1ee4d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.250+0000 7f94bcfab740 1 --2- 172.21.15.96:0/3077314111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555fa9e154d0 0x555faa1edf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.250+0000 7f94bcfab740 1 -- 172.21.15.96:0/3077314111 >> 172.21.15.96:0/3077314111 conn(0x555fa9e324e0 msgr2=0x555faa1e8670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:44.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.250+0000 7f948effd640 0 client.25379 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:44.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.250+0000 7f94bcfab740 1 -- 172.21.15.96:0/3077314111 shutdown_connections 2024-04-25T14:14:44.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.250+0000 7f94bcfab740 1 -- 172.21.15.96:0/3077314111 wait complete. 2024-04-25T14:14:44.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.250+0000 7f94bcfab740 20 client.25379 trim_cache size 0 max 16384 2024-04-25T14:14:44.262 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T14:14:44.263 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "no active scrubs running", 2024-04-25T14:14:44.263 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": {} 2024-04-25T14:14:44.263 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T14:14:44.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.251+0000 7f94b6020640 1 -- 172.21.15.96:0/715885053 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f949c0afc90 msgr2=0x7f949c0b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:44.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.251+0000 7f94b6020640 1 --2- 172.21.15.96:0/715885053 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f949c0afc90 0x7f949c0b21a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f94a8031b50 tx=0x7f94a802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.251+0000 7f94b6020640 1 -- 172.21.15.96:0/715885053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94b0109890 msgr2=0x7f94b01a1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:44.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.251+0000 7f94b6020640 1 --2- 172.21.15.96:0/715885053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94b0109890 0x7f94b01a1a10 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f94a8009430 tx=0x7f94a8003ba0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.251+0000 7f94b6020640 1 -- 172.21.15.96:0/715885053 shutdown_connections 2024-04-25T14:14:44.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.251+0000 7f94b6020640 1 --2- 172.21.15.96:0/715885053 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f949c0afc90 0x7f949c0b21a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.251+0000 7f94b6020640 1 --2- 172.21.15.96:0/715885053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94b0109890 0x7f94b01a1a10 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.251+0000 7f94b6020640 1 --2- 172.21.15.96:0/715885053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94b0106f30 0x7f94b019aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.251+0000 7f94b6020640 1 --2- 172.21.15.96:0/715885053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94b01045d0 0x7f94b019a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.251+0000 7f94b6020640 1 -- 172.21.15.96:0/715885053 >> 172.21.15.96:0/715885053 conn(0x7f94b00fdbd0 msgr2=0x7f94b0105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:44.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.251+0000 7f94b6020640 1 -- 172.21.15.96:0/715885053 shutdown_connections 2024-04-25T14:14:44.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.251+0000 7f94b6020640 1 -- 172.21.15.96:0/715885053 wait complete. 2024-04-25T14:14:44.268 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T14:14:44.268 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:14:44.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.382+0000 7fe737205640 1 -- 172.21.15.96:0/2894622049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7301080a0 msgr2=0x7fe73010a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:44.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.382+0000 7fe737205640 1 --2- 172.21.15.96:0/2894622049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7301080a0 0x7fe73010a4c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe728009be0 tx=0x7fe72802a6e0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.382+0000 7fe737205640 1 -- 172.21.15.96:0/2894622049 shutdown_connections 2024-04-25T14:14:44.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.382+0000 7fe737205640 1 --2- 172.21.15.96:0/2894622049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7301080a0 0x7fe73010a4c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.382+0000 7fe737205640 1 --2- 172.21.15.96:0/2894622049 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe7301033a0 0x7fe7301037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.382+0000 7fe737205640 1 --2- 172.21.15.96:0/2894622049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe730102160 0x7fe730102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.382+0000 7fe737205640 1 -- 172.21.15.96:0/2894622049 >> 172.21.15.96:0/2894622049 conn(0x7fe7300fd330 msgr2=0x7fe7300ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:44.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.382+0000 7fe737205640 1 -- 172.21.15.96:0/2894622049 shutdown_connections 2024-04-25T14:14:44.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.382+0000 7fe737205640 1 -- 172.21.15.96:0/2894622049 wait complete. 2024-04-25T14:14:44.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.383+0000 7fe737205640 1 Processor -- start 2024-04-25T14:14:44.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.383+0000 7fe737205640 1 -- start start 2024-04-25T14:14:44.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.384+0000 7fe737205640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe730102160 0x7fe7301a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.384+0000 7fe737205640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7301033a0 0x7fe7301a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.384+0000 7fe737205640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe7301080a0 0x7fe7301a7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.384+0000 7fe72ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7301033a0 0x7fe7301a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.384+0000 7fe72ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7301033a0 0x7fe7301a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34500/0 (socket says 172.21.15.96:34500) 2024-04-25T14:14:44.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.384+0000 7fe734fa1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe730102160 0x7fe7301a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.384+0000 7fe734fa1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe730102160 0x7fe7301a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:53958/0 (socket says 172.21.15.96:53958) 2024-04-25T14:14:44.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.384+0000 7fe72ffff640 1 -- 172.21.15.96:0/1154579808 learned_addr learned my addr 172.21.15.96:0/1154579808 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:44.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.384+0000 7fe737205640 1 -- 172.21.15.96:0/1154579808 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe73010d3f0 con 0x7fe7301033a0 2024-04-25T14:14:44.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.384+0000 7fe737205640 1 -- 172.21.15.96:0/1154579808 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe73010d270 con 0x7fe730102160 2024-04-25T14:14:44.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.384+0000 7fe737205640 1 -- 172.21.15.96:0/1154579808 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe73010d570 con 0x7fe7301080a0 2024-04-25T14:14:44.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.384+0000 7fe7357a2640 1 --2- 172.21.15.96:0/1154579808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe7301080a0 0x7fe7301a7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.385+0000 7fe72ffff640 1 -- 172.21.15.96:0/1154579808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe7301080a0 msgr2=0x7fe7301a7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:44.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.385+0000 7fe72ffff640 1 --2- 172.21.15.96:0/1154579808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe7301080a0 0x7fe7301a7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.385+0000 7fe72ffff640 1 -- 172.21.15.96:0/1154579808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe730102160 msgr2=0x7fe7301a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:44.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.385+0000 7fe72ffff640 1 --2- 172.21.15.96:0/1154579808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe730102160 0x7fe7301a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.385+0000 7fe72ffff640 1 -- 172.21.15.96:0/1154579808 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7301a8160 con 0x7fe7301033a0 2024-04-25T14:14:44.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.385+0000 7fe7357a2640 1 --2- 172.21.15.96:0/1154579808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe7301080a0 0x7fe7301a7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:14:44.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.385+0000 7fe72ffff640 1 --2- 172.21.15.96:0/1154579808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7301033a0 0x7fe7301a0b60 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe71c00e9f0 tx=0x7fe71c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:44.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.385+0000 7fe72dffb640 1 -- 172.21.15.96:0/1154579808 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71c00ccc0 con 0x7fe7301033a0 2024-04-25T14:14:44.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.385+0000 7fe72dffb640 1 -- 172.21.15.96:0/1154579808 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe71c0108e0 con 0x7fe7301033a0 2024-04-25T14:14:44.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.385+0000 7fe72dffb640 1 -- 172.21.15.96:0/1154579808 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71c018880 con 0x7fe7301033a0 2024-04-25T14:14:44.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.385+0000 7fe737205640 1 -- 172.21.15.96:0/1154579808 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7301a8450 con 0x7fe7301033a0 2024-04-25T14:14:44.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.386+0000 7fe737205640 1 -- 172.21.15.96:0/1154579808 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7301a8950 con 0x7fe7301033a0 2024-04-25T14:14:44.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.387+0000 7fe737205640 1 -- 172.21.15.96:0/1154579808 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7301067a0 con 0x7fe7301033a0 2024-04-25T14:14:44.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.389+0000 7fe72dffb640 1 -- 172.21.15.96:0/1154579808 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fe71c010430 con 0x7fe7301033a0 2024-04-25T14:14:44.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.390+0000 7fe72dffb640 1 --2- 172.21.15.96:0/1154579808 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fe7200afb70 0x7fe7200b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.391+0000 7fe734fa1640 1 --2- 172.21.15.96:0/1154579808 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fe7200afb70 0x7fe7200b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.391+0000 7fe72dffb640 1 -- 172.21.15.96:0/1154579808 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fe71c0df410 con 0x7fe7301033a0 2024-04-25T14:14:44.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.391+0000 7fe734fa1640 1 --2- 172.21.15.96:0/1154579808 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fe7200afb70 0x7fe7200b2080 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe730102d60 tx=0x7fe724009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:44.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.396+0000 7fe72dffb640 1 -- 172.21.15.96:0/1154579808 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe71c0a7fd0 con 0x7fe7301033a0 2024-04-25T14:14:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.396+0000 7faba9ffb640 1 -- 172.21.15.96:0/2605155038 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabac09b9d0 con 0x7faba409c6a0 2024-04-25T14:14:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.397+0000 7fabb16b6640 1 -- 172.21.15.96:0/2605155038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faba409c6a0 msgr2=0x7faba409ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.397+0000 7fabb16b6640 1 --2- 172.21.15.96:0/2605155038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faba409c6a0 0x7faba409ebb0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fabac068690 tx=0x7fabac097ac0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.397+0000 7fabb16b6640 1 -- 172.21.15.96:0/2605155038 shutdown_connections 2024-04-25T14:14:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.397+0000 7fabb16b6640 1 --2- 172.21.15.96:0/2605155038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faba409c6a0 0x7faba409ebb0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.397+0000 7fabb16b6640 1 --2- 172.21.15.96:0/2605155038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faba4099b70 0x7faba409c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.397+0000 7fabb16b6640 1 --2- 172.21.15.96:0/2605155038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faba4097180 0x7faba40995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.397+0000 7fabb16b6640 1 -- 172.21.15.96:0/2605155038 >> 172.21.15.96:0/2605155038 conn(0x7faba4090ae0 msgr2=0x7faba4092fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.397+0000 7fabb16b6640 1 -- 172.21.15.96:0/2605155038 shutdown_connections 2024-04-25T14:14:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.397+0000 7fabb16b6640 1 -- 172.21.15.96:0/2605155038 wait complete. 2024-04-25T14:14:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.398+0000 7fabb16b6640 1 Processor -- start 2024-04-25T14:14:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.398+0000 7fabb16b6640 1 -- start start 2024-04-25T14:14:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.398+0000 7fabb16b6640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faba4097180 0x7faba412d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.398+0000 7fabb16b6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faba4099b70 0x7faba412d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.398+0000 7fabb16b6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faba409c6a0 0x7faba4134560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.398+0000 7fabb16b6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faba40a24b0 con 0x7faba4099b70 2024-04-25T14:14:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.398+0000 7fabb16b6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7faba40a2330 con 0x7faba409c6a0 2024-04-25T14:14:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.398+0000 7fabb16b6640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faba40a2630 con 0x7faba4097180 2024-04-25T14:14:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.399+0000 7fabaa7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faba4099b70 0x7faba412d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.399+0000 7fabaa7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faba4099b70 0x7faba412d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34512/0 (socket says 172.21.15.96:34512) 2024-04-25T14:14:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.399+0000 7fabaa7fc640 1 -- 172.21.15.96:0/499178023 learned_addr learned my addr 172.21.15.96:0/499178023 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.399+0000 7fabaaffd640 1 --2- 172.21.15.96:0/499178023 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faba4097180 0x7faba412d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.399+0000 7fabab7fe640 1 --2- 172.21.15.96:0/499178023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faba409c6a0 0x7faba4134560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.399+0000 7fabaa7fc640 1 -- 172.21.15.96:0/499178023 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faba4097180 msgr2=0x7faba412d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.399+0000 7fabaa7fc640 1 --2- 172.21.15.96:0/499178023 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faba4097180 0x7faba412d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.399+0000 7fabaa7fc640 1 -- 172.21.15.96:0/499178023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faba409c6a0 msgr2=0x7faba4134560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.399+0000 7fabaa7fc640 1 --2- 172.21.15.96:0/499178023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faba409c6a0 0x7faba4134560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.399+0000 7fabaa7fc640 1 -- 172.21.15.96:0/499178023 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faba4134bd0 con 0x7faba4099b70 2024-04-25T14:14:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.399+0000 7fabaaffd640 1 --2- 172.21.15.96:0/499178023 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faba4097180 0x7faba412d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:14:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.400+0000 7fabaa7fc640 1 --2- 172.21.15.96:0/499178023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faba4099b70 0x7faba412d5d0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7faba000c890 tx=0x7faba000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.400+0000 7fab93fff640 1 -- 172.21.15.96:0/499178023 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba0004220 con 0x7faba4099b70 2024-04-25T14:14:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.400+0000 7fab93fff640 1 -- 172.21.15.96:0/499178023 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faba00043c0 con 0x7faba4099b70 2024-04-25T14:14:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.400+0000 7fabb16b6640 1 -- 172.21.15.96:0/499178023 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faba4134ec0 con 0x7faba4099b70 2024-04-25T14:14:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.400+0000 7fab93fff640 1 -- 172.21.15.96:0/499178023 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba0004a00 con 0x7faba4099b70 2024-04-25T14:14:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.400+0000 7fabb16b6640 1 -- 172.21.15.96:0/499178023 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faba4135380 con 0x7faba4099b70 2024-04-25T14:14:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.404+0000 7fabb856a740 1 Processor -- start 2024-04-25T14:14:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.404+0000 7fabb856a740 1 -- start start 2024-04-25T14:14:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.404+0000 7fabb856a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55afe409a1b0 0x55afe4445980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.404+0000 7fabb856a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afe40712f0 0x55afe4447f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.404+0000 7fabb856a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55afe40794d0 0x55afe444a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.404+0000 7fabb856a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55afe43a0460 con 0x55afe40794d0 2024-04-25T14:14:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.404+0000 7fabb856a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55afe435a590 con 0x55afe40712f0 2024-04-25T14:14:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.404+0000 7fabb856a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55afe439c420 con 0x55afe409a1b0 2024-04-25T14:14:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.404+0000 7fabaaffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afe40712f0 0x55afe4447f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.404+0000 7fabab7fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55afe409a1b0 0x55afe4445980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.404+0000 7fabab7fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55afe409a1b0 0x55afe4445980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.96:54760/0 (socket says 172.21.15.96:54760) 2024-04-25T14:14:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.404+0000 7fabab7fe640 1 -- 172.21.15.96:0/1511404289 learned_addr learned my addr 172.21.15.96:0/1511404289 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.405+0000 7fabab7fe640 1 -- 172.21.15.96:0/1511404289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afe40712f0 msgr2=0x55afe4447f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.405+0000 7fabab7fe640 1 --2- 172.21.15.96:0/1511404289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afe40712f0 0x55afe4447f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.405+0000 7fabab7fe640 1 -- 172.21.15.96:0/1511404289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55afe40794d0 msgr2=0x55afe444a6c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T14:14:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.405+0000 7fabab7fe640 1 --2- 172.21.15.96:0/1511404289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55afe40794d0 0x55afe444a6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.405+0000 7fabab7fe640 1 -- 172.21.15.96:0/1511404289 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afe43670d0 con 0x55afe409a1b0 2024-04-25T14:14:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.405+0000 7fab93fff640 1 -- 172.21.15.96:0/499178023 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7faba0004ba0 con 0x7faba4099b70 2024-04-25T14:14:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.405+0000 7fabaaffd640 1 --2- 172.21.15.96:0/1511404289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afe40712f0 0x55afe4447f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:14:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.405+0000 7fabab7fe640 1 --2- 172.21.15.96:0/1511404289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55afe409a1b0 0x55afe4445980 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fabac068a60 tx=0x7fabac09b450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20d413be036893e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.405+0000 7fab917fa640 1 -- 172.21.15.96:0/1511404289 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabac09f070 con 0x55afe409a1b0 2024-04-25T14:14:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.406+0000 7fab917fa640 1 -- 172.21.15.96:0/1511404289 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabac06b500 con 0x55afe409a1b0 2024-04-25T14:14:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.406+0000 7fab93fff640 1 --2- 172.21.15.96:0/499178023 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fab940afc30 0x7fab940b2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.406+0000 7fab917fa640 1 -- 172.21.15.96:0/1511404289 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabac09bc90 con 0x55afe409a1b0 2024-04-25T14:14:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.406+0000 7fabaaffd640 1 --2- 172.21.15.96:0/499178023 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fab940afc30 0x7fab940b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.406+0000 7fab93fff640 1 -- 172.21.15.96:0/499178023 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7faba00de8b0 con 0x7faba4099b70 2024-04-25T14:14:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.407+0000 7fabb856a740 1 -- 172.21.15.96:0/1511404289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55afe409a1b0 msgr2=0x55afe4445980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.407+0000 7fabb856a740 1 --2- 172.21.15.96:0/1511404289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55afe409a1b0 0x55afe4445980 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fabac068a60 tx=0x7fabac09b450 comp rx=0 tx=0).stop 2024-04-25T14:14:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.407+0000 7fabb856a740 1 -- 172.21.15.96:0/1511404289 shutdown_connections 2024-04-25T14:14:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.407+0000 7fabb856a740 1 --2- 172.21.15.96:0/1511404289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55afe40794d0 0x55afe444a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.407+0000 7fabb856a740 1 --2- 172.21.15.96:0/1511404289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afe40712f0 0x55afe4447f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.407+0000 7fabb856a740 1 --2- 172.21.15.96:0/1511404289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55afe409a1b0 0x55afe4445980 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.407+0000 7fabb856a740 1 -- 172.21.15.96:0/1511404289 >> 172.21.15.96:0/1511404289 conn(0x55afe408e300 msgr2=0x55afe4436430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.407+0000 7fabb856a740 1 -- 172.21.15.96:0/1511404289 shutdown_connections 2024-04-25T14:14:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.407+0000 7fabb856a740 1 -- 172.21.15.96:0/1511404289 wait complete. 2024-04-25T14:14:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.407+0000 7fabaaffd640 1 --2- 172.21.15.96:0/499178023 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fab940afc30 0x7fab940b2140 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x55afe44371c0 tx=0x7fab9c00a250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.408+0000 7fabb856a740 1 -- start start 2024-04-25T14:14:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.408+0000 7fabb856a740 1 Processor -- start 2024-04-25T14:14:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.408+0000 7fabb856a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55afe40712f0 0x55afe445e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.408+0000 7fabb856a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afe40794d0 0x55afe445e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.408+0000 7fabb856a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55afe409a1b0 0x55afe44654a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.408+0000 7fabb856a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55afe445c180 con 0x55afe409a1b0 2024-04-25T14:14:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.408+0000 7fabb856a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55afe4327b60 con 0x55afe40794d0 2024-04-25T14:14:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.408+0000 7fabb856a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55afe445c300 con 0x55afe40712f0 2024-04-25T14:14:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.408+0000 7fabb856a740 10 client.0 _ll_register_callbacks cb 0x55afe434c2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:14:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.408+0000 7fabab7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55afe409a1b0 0x55afe44654a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.408+0000 7fabab7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55afe409a1b0 0x55afe44654a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34516/0 (socket says 172.21.15.96:34516) 2024-04-25T14:14:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.408+0000 7fabab7fe640 1 -- 172.21.15.96:0/721042226 learned_addr learned my addr 172.21.15.96:0/721042226 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.408+0000 7fabaa7fc640 1 --2- 172.21.15.96:0/721042226 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afe40794d0 0x55afe445e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.409+0000 7fabaaffd640 1 --2- 172.21.15.96:0/721042226 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55afe40712f0 0x55afe445e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.409+0000 7fabaa7fc640 1 -- 172.21.15.96:0/721042226 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55afe40712f0 msgr2=0x55afe445e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.409+0000 7fabaa7fc640 1 --2- 172.21.15.96:0/721042226 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55afe40712f0 0x55afe445e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.409+0000 7fabaa7fc640 1 -- 172.21.15.96:0/721042226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55afe409a1b0 msgr2=0x55afe44654a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.409+0000 7fabaa7fc640 1 --2- 172.21.15.96:0/721042226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55afe409a1b0 0x55afe44654a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.409+0000 7fabaa7fc640 1 -- 172.21.15.96:0/721042226 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afe40b2210 con 0x55afe40794d0 2024-04-25T14:14:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.409+0000 7fabab7fe640 1 --2- 172.21.15.96:0/721042226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55afe409a1b0 0x55afe44654a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:14:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.409+0000 7fabaa7fc640 1 --2- 172.21.15.96:0/721042226 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afe40794d0 0x55afe445e540 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7faba00a6d50 tx=0x7faba00a2c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98879e5ffd414fc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.409+0000 7fab8eff5640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T14:14:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.409+0000 7fab8eff5640 1 -- 172.21.15.96:0/721042226 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba001d070 con 0x55afe40794d0 2024-04-25T14:14:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.409+0000 7fabb856a740 1 -- 172.21.15.96:0/721042226 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabac06b500 con 0x55afe40794d0 2024-04-25T14:14:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.409+0000 7fab8eff5640 1 -- 172.21.15.96:0/721042226 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7faba00c1350 con 0x55afe40794d0 2024-04-25T14:14:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.409+0000 7fab8eff5640 1 -- 172.21.15.96:0/721042226 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faba00a6530 con 0x55afe40794d0 2024-04-25T14:14:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.409+0000 7fabb856a740 10 client.25394 fetch_fsmap learned FSMap version 26 2024-04-25T14:14:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.409+0000 7fabb856a740 1 -- 172.21.15.96:0/721042226 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55afe417e620 con 0x55afe40794d0 2024-04-25T14:14:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.410+0000 7fab8eff5640 1 -- 172.21.15.96:0/721042226 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabac06b500 con 0x55afe40794d0 2024-04-25T14:14:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.410+0000 7fab8eff5640 1 -- 172.21.15.96:0/721042226 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7faba00c0030 con 0x55afe40794d0 2024-04-25T14:14:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.410+0000 7fabb856a740 10 client.25394 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:14:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.410+0000 7fabb856a740 10 client.25394 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:14:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.410+0000 7fabb856a740 20 client.25394 populate_metadata read hostname 'smithi096' 2024-04-25T14:14:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.410+0000 7fabb856a740 1 --2- 172.21.15.96:0/721042226 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55afe4058d80 0x55afe446bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.411+0000 7fabb856a740 4 client.25394 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:14:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.411+0000 7fabb856a740 1 -- 172.21.15.96:0/721042226 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55afe43ae070 con 0x55afe4058d80 2024-04-25T14:14:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.411+0000 7fabaaffd640 1 --2- 172.21.15.96:0/721042226 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55afe4058d80 0x55afe446bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.411+0000 7fab8eff5640 10 client.25394 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.411+0000 7fabaaffd640 1 --2- 172.21.15.96:0/721042226 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55afe4058d80 0x55afe446bce0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd61f27e1d6a47a4 server_cookie=c9a317a8be44b7d in_seq=0 out_seq=0 2024-04-25T14:14:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.412+0000 7fab8eff5640 1 -- 172.21.15.96:0/721042226 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55afe43ae070 con 0x55afe4058d80 2024-04-25T14:14:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.412+0000 7fab8eff5640 10 client.25394 handle_command_reply: tid=0 2024-04-25T14:14:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.412+0000 7fabb856a740 1 client.25394 shutdown 2024-04-25T14:14:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.412+0000 7fabb856a740 1 -- 172.21.15.96:0/721042226 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afe40794d0 msgr2=0x55afe445e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.412+0000 7fabb856a740 1 --2- 172.21.15.96:0/721042226 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afe40794d0 0x55afe445e540 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7faba00a6d50 tx=0x7faba00a2c80 comp rx=0 tx=0).stop 2024-04-25T14:14:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.412+0000 7fabb856a740 1 -- 172.21.15.96:0/721042226 shutdown_connections 2024-04-25T14:14:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.412+0000 7fabb856a740 1 --2- 172.21.15.96:0/721042226 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55afe4058d80 0x55afe446bce0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.412+0000 7fabb856a740 1 --2- 172.21.15.96:0/721042226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55afe409a1b0 0x55afe44654a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.412+0000 7fab8eff5640 0 client.25394 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.412+0000 7fabb856a740 1 --2- 172.21.15.96:0/721042226 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afe40794d0 0x55afe445e540 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.412+0000 7fabb856a740 1 --2- 172.21.15.96:0/721042226 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55afe40712f0 0x55afe445e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.412+0000 7fabb856a740 1 -- 172.21.15.96:0/721042226 >> 172.21.15.96:0/721042226 conn(0x55afe408e300 msgr2=0x55afe4435750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.412+0000 7fabb856a740 1 -- 172.21.15.96:0/721042226 shutdown_connections 2024-04-25T14:14:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.412+0000 7fabb856a740 1 -- 172.21.15.96:0/721042226 wait complete. 2024-04-25T14:14:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.412+0000 7fabb856a740 20 client.25394 trim_cache size 0 max 16384 2024-04-25T14:14:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.444+0000 7fabb856a740 1 Processor -- start 2024-04-25T14:14:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.444+0000 7fabb856a740 1 -- start start 2024-04-25T14:14:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.444+0000 7fabb856a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55afe40712f0 0x55afe44714e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.444+0000 7fabb856a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55afe40794d0 0x55afe44653c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.444+0000 7fabb856a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afe409a1b0 0x55afe4471a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.445+0000 7fabb856a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faba413be50 con 0x55afe40712f0 2024-04-25T14:14:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.445+0000 7fabaaffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55afe40712f0 0x55afe44714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.445+0000 7fabb856a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7faba413d990 con 0x55afe409a1b0 2024-04-25T14:14:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.445+0000 7fabaaffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55afe40712f0 0x55afe44714e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34524/0 (socket says 172.21.15.96:34524) 2024-04-25T14:14:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.445+0000 7fabaaffd640 1 -- 172.21.15.96:0/2011979025 learned_addr learned my addr 172.21.15.96:0/2011979025 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.445+0000 7fabb856a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faba413c520 con 0x55afe40794d0 2024-04-25T14:14:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.445+0000 7fabaa7fc640 1 --2- 172.21.15.96:0/2011979025 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55afe40794d0 0x55afe44653c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.445+0000 7fabab7fe640 1 --2- 172.21.15.96:0/2011979025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afe409a1b0 0x55afe4471a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.445+0000 7fabaaffd640 1 -- 172.21.15.96:0/2011979025 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55afe40794d0 msgr2=0x55afe44653c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.445+0000 7fabaaffd640 1 --2- 172.21.15.96:0/2011979025 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55afe40794d0 0x55afe44653c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.445+0000 7fabaaffd640 1 -- 172.21.15.96:0/2011979025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afe409a1b0 msgr2=0x55afe4471a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.445+0000 7fabaaffd640 1 --2- 172.21.15.96:0/2011979025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afe409a1b0 0x55afe4471a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.445+0000 7fabaaffd640 1 -- 172.21.15.96:0/2011979025 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afe44397d0 con 0x55afe40712f0 2024-04-25T14:14:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.445+0000 7fabaa7fc640 1 --2- 172.21.15.96:0/2011979025 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55afe40794d0 0x55afe44653c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:14:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.445+0000 7fabaaffd640 1 --2- 172.21.15.96:0/2011979025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55afe40712f0 0x55afe44714e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fab9c0058c0 tx=0x7fab9c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9eabbef3c085b91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.446+0000 7fab90ff9640 1 -- 172.21.15.96:0/2011979025 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9c02f070 con 0x55afe40712f0 2024-04-25T14:14:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.446+0000 7fab90ff9640 1 -- 172.21.15.96:0/2011979025 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab9c0299a0 con 0x55afe40712f0 2024-04-25T14:14:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.446+0000 7fab90ff9640 1 -- 172.21.15.96:0/2011979025 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9c01b410 con 0x55afe40712f0 2024-04-25T14:14:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.447+0000 7fabb856a740 1 -- 172.21.15.96:0/2011979025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55afe40712f0 msgr2=0x55afe44714e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.447+0000 7fabb856a740 1 --2- 172.21.15.96:0/2011979025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55afe40712f0 0x55afe44714e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fab9c0058c0 tx=0x7fab9c015000 comp rx=0 tx=0).stop 2024-04-25T14:14:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.447+0000 7fabb856a740 1 -- 172.21.15.96:0/2011979025 shutdown_connections 2024-04-25T14:14:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.447+0000 7fabb856a740 1 --2- 172.21.15.96:0/2011979025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afe409a1b0 0x55afe4471a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.447+0000 7fabb856a740 1 --2- 172.21.15.96:0/2011979025 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55afe40794d0 0x55afe44653c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.447+0000 7fabb856a740 1 --2- 172.21.15.96:0/2011979025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55afe40712f0 0x55afe44714e0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.447+0000 7fabb856a740 1 -- 172.21.15.96:0/2011979025 >> 172.21.15.96:0/2011979025 conn(0x55afe408e300 msgr2=0x55afe445d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.447+0000 7fabb856a740 1 -- 172.21.15.96:0/2011979025 shutdown_connections 2024-04-25T14:14:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.447+0000 7fabb856a740 1 -- 172.21.15.96:0/2011979025 wait complete. 2024-04-25T14:14:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.448+0000 7fabb856a740 1 -- start start 2024-04-25T14:14:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.448+0000 7fabb856a740 1 Processor -- start 2024-04-25T14:14:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.448+0000 7fabb856a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55afe40712f0 0x55afe4462270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.448+0000 7fabb856a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afe40794d0 0x55afe44627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.448+0000 7fabb856a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55afe409a1b0 0x55afe44d2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.448+0000 7fabb856a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faba413d2c0 con 0x55afe409a1b0 2024-04-25T14:14:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.448+0000 7fabab7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55afe409a1b0 0x55afe44d2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.449+0000 7fabaa7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afe40794d0 0x55afe44627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.449+0000 7fabaaffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55afe40712f0 0x55afe4462270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.449+0000 7fabaa7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afe40794d0 0x55afe44627b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:54022/0 (socket says 172.21.15.96:54022) 2024-04-25T14:14:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.449+0000 7fabaa7fc640 1 -- 172.21.15.96:0/1983336810 learned_addr learned my addr 172.21.15.96:0/1983336810 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.449+0000 7fabb856a740 1 -- 172.21.15.96:0/1983336810 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7faba413cbf0 con 0x55afe40794d0 2024-04-25T14:14:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.449+0000 7fabb856a740 1 -- 172.21.15.96:0/1983336810 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faba4139e30 con 0x55afe40712f0 2024-04-25T14:14:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.449+0000 7fabb856a740 10 client.0 _ll_register_callbacks cb 0x7faba0004570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:14:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.449+0000 7fabaaffd640 1 -- 172.21.15.96:0/1983336810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afe40794d0 msgr2=0x55afe44627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.449+0000 7fabaaffd640 1 --2- 172.21.15.96:0/1983336810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afe40794d0 0x55afe44627b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.449+0000 7fabaaffd640 1 -- 172.21.15.96:0/1983336810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55afe409a1b0 msgr2=0x55afe44d2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.449+0000 7fabaaffd640 1 --2- 172.21.15.96:0/1983336810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55afe409a1b0 0x55afe44d2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.449+0000 7fabaaffd640 1 -- 172.21.15.96:0/1983336810 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afe4465770 con 0x55afe40712f0 2024-04-25T14:14:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.449+0000 7fabab7fe640 1 --2- 172.21.15.96:0/1983336810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55afe409a1b0 0x55afe44d2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:14:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.450+0000 7fabaaffd640 1 --2- 172.21.15.96:0/1983336810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55afe40712f0 0x55afe4462270 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fab9c02d040 tx=0x7fab9c00f870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4208494853e65124 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.450+0000 7fab8eff5640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-25T14:14:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.450+0000 7fab8eff5640 1 -- 172.21.15.96:0/1983336810 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9c03a060 con 0x55afe40712f0 2024-04-25T14:14:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.450+0000 7fab8eff5640 1 -- 172.21.15.96:0/1983336810 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fab9c027640 con 0x55afe40712f0 2024-04-25T14:14:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.450+0000 7fabb856a740 1 -- 172.21.15.96:0/1983336810 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab9c0299a0 con 0x55afe40712f0 2024-04-25T14:14:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.450+0000 7fab8eff5640 1 -- 172.21.15.96:0/1983336810 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fab9c02f070 con 0x55afe40712f0 2024-04-25T14:14:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.450+0000 7fabb856a740 10 client.25336 fetch_fsmap learned FSMap version 26 2024-04-25T14:14:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.451+0000 7fabb856a740 1 -- 172.21.15.96:0/1983336810 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55afe44d3500 con 0x55afe40712f0 2024-04-25T14:14:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.451+0000 7fab8eff5640 1 -- 172.21.15.96:0/1983336810 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab9c0299a0 con 0x55afe40712f0 2024-04-25T14:14:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.451+0000 7fab8eff5640 1 -- 172.21.15.96:0/1983336810 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fab9c0049f0 con 0x55afe40712f0 2024-04-25T14:14:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.451+0000 7fabb856a740 10 client.25336 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:14:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.451+0000 7fabb856a740 10 client.25336 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:14:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.451+0000 7fabb856a740 20 client.25336 populate_metadata read hostname 'smithi096' 2024-04-25T14:14:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.452+0000 7fabb856a740 1 --2- 172.21.15.96:0/1983336810 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55afe402e1c0 0x55afe44d5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.452+0000 7fabb856a740 4 client.25336 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:14:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.452+0000 7fabb856a740 1 -- 172.21.15.96:0/1983336810 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55afe43a7ee0 con 0x55afe402e1c0 2024-04-25T14:14:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.452+0000 7fabaa7fc640 1 --2- 172.21.15.96:0/1983336810 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55afe402e1c0 0x55afe44d5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.453+0000 7fab8eff5640 10 client.25336 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.453+0000 7fabaa7fc640 1 --2- 172.21.15.96:0/1983336810 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55afe402e1c0 0x55afe44d5e70 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9eae3329828b74ce server_cookie=90654fd21523e85c in_seq=0 out_seq=0 2024-04-25T14:14:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.456+0000 7fab8eff5640 1 -- 172.21.15.96:0/1983336810 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55afe43a7ee0 con 0x55afe402e1c0 2024-04-25T14:14:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.456+0000 7fab8eff5640 10 client.25336 handle_command_reply: tid=0 2024-04-25T14:14:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.456+0000 7fabb856a740 1 client.25336 shutdown 2024-04-25T14:14:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.457+0000 7fabb856a740 1 -- 172.21.15.96:0/1983336810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55afe40712f0 msgr2=0x55afe4462270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.457+0000 7fabb856a740 1 --2- 172.21.15.96:0/1983336810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55afe40712f0 0x55afe4462270 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fab9c02d040 tx=0x7fab9c00f870 comp rx=0 tx=0).stop 2024-04-25T14:14:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.457+0000 7fabb856a740 1 -- 172.21.15.96:0/1983336810 shutdown_connections 2024-04-25T14:14:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.457+0000 7fabb856a740 1 --2- 172.21.15.96:0/1983336810 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55afe402e1c0 0x55afe44d5e70 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.457+0000 7fab8eff5640 0 client.25336 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.457+0000 7fabb856a740 1 --2- 172.21.15.96:0/1983336810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55afe409a1b0 0x55afe44d2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.457+0000 7fabb856a740 1 --2- 172.21.15.96:0/1983336810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afe40794d0 0x55afe44627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.457+0000 7fabb856a740 1 --2- 172.21.15.96:0/1983336810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55afe40712f0 0x55afe4462270 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.457+0000 7fabb856a740 1 -- 172.21.15.96:0/1983336810 >> 172.21.15.96:0/1983336810 conn(0x55afe408e300 msgr2=0x55afe44704d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.457+0000 7fabb856a740 1 -- 172.21.15.96:0/1983336810 shutdown_connections 2024-04-25T14:14:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.457+0000 7fabb856a740 1 -- 172.21.15.96:0/1983336810 wait complete. 2024-04-25T14:14:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.457+0000 7fabb856a740 20 client.25336 trim_cache size 0 max 16384 2024-04-25T14:14:44.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T14:14:44.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T14:14:44.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T14:14:44.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T14:14:44.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.458+0000 7fabb16b6640 1 -- 172.21.15.96:0/499178023 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fab940afc30 msgr2=0x7fab940b2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:44.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.458+0000 7fabb16b6640 1 --2- 172.21.15.96:0/499178023 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fab940afc30 0x7fab940b2140 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x55afe44371c0 tx=0x7fab9c00a250 comp rx=0 tx=0).stop 2024-04-25T14:14:44.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.458+0000 7fabb16b6640 1 -- 172.21.15.96:0/499178023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faba4099b70 msgr2=0x7faba412d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:44.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.458+0000 7fabb16b6640 1 --2- 172.21.15.96:0/499178023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faba4099b70 0x7faba412d5d0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7faba000c890 tx=0x7faba000cd50 comp rx=0 tx=0).stop 2024-04-25T14:14:44.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.458+0000 7fabb16b6640 1 -- 172.21.15.96:0/499178023 shutdown_connections 2024-04-25T14:14:44.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.458+0000 7fabb16b6640 1 --2- 172.21.15.96:0/499178023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faba409c6a0 0x7faba4134560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.458+0000 7fabb16b6640 1 --2- 172.21.15.96:0/499178023 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fab940afc30 0x7fab940b2140 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.458+0000 7fabb16b6640 1 --2- 172.21.15.96:0/499178023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faba4099b70 0x7faba412d5d0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.458+0000 7fabb16b6640 1 --2- 172.21.15.96:0/499178023 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faba4097180 0x7faba412d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.458+0000 7fabb16b6640 1 -- 172.21.15.96:0/499178023 >> 172.21.15.96:0/499178023 conn(0x7faba4090ae0 msgr2=0x7faba4098140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:44.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.458+0000 7fabb16b6640 1 -- 172.21.15.96:0/499178023 shutdown_connections 2024-04-25T14:14:44.461 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:44.459+0000 7fabb16b6640 1 -- 172.21.15.96:0/499178023 wait complete. 2024-04-25T14:14:44.590 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:44 smithi096 ceph-mon[29207]: pgmap v379: 97 pgs: 97 active+clean; 668 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T14:14:44.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.652+0000 7fe737205640 1 -- 172.21.15.96:0/1154579808 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe73002db50 con 0x7fe7301033a0 2024-04-25T14:14:44.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.653+0000 7fe72dffb640 1 -- 172.21.15.96:0/1154579808 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11927 (secure 0 0 0) 0x7fe71c0a78d0 con 0x7fe7301033a0 2024-04-25T14:14:44.653 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:14:44.653 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/606227917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":606227917},{"type":"v1","addr":"172.21.15.96:6835","nonce":606227917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/3265300710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3265300710},{"type":"v1","addr":"172.21.15.96:6837","nonce":3265300710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T14:11:28.886961+0000","modified":"2024-04-25T14:13:32.120297+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24517,"mds_2":24478,"mds_3":24455,"mds_4":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/4080308509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":4080308509},{"type":"v1","addr":"172.21.15.96:6839","nonce":4080308509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/1606748971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":1606748971},{"type":"v1","addr":"172.21.15.96:6841","nonce":1606748971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6833/1995324831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1995324831},{"type":"v1","addr":"172.21.15.97:6833","nonce":1995324831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":4,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.151:6833/1382721608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1382721608},{"type":"v1","addr":"172.21.15.151:6833","nonce":1382721608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6835/3872301726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3872301726},{"type":"v1","addr":"172.21.15.97:6835","nonce":3872301726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.151:6835/1313823638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":1313823638},{"type":"v1","addr":"172.21.15.151:6835","nonce":1313823638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/2691348366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2691348366},{"type":"v1","addr":"172.21.15.151:6837","nonce":2691348366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3306780081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3306780081},{"type":"v1","addr":"172.21.15.97:6837","nonce":3306780081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.151:6839/245699981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":245699981},{"type":"v1","addr":"172.21.15.151:6839","nonce":245699981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6839/2850442269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":2850442269},{"type":"v1","addr":"172.21.15.97:6839","nonce":2850442269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24427,24455,24478,24517,24536]},"id":1}]} 2024-04-25T14:14:44.654 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 26 2024-04-25T14:14:44.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.655+0000 7fe737205640 1 -- 172.21.15.96:0/1154579808 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fe7200afb70 msgr2=0x7fe7200b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:44.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.655+0000 7fe737205640 1 --2- 172.21.15.96:0/1154579808 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fe7200afb70 0x7fe7200b2080 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe730102d60 tx=0x7fe724009260 comp rx=0 tx=0).stop 2024-04-25T14:14:44.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.655+0000 7fe737205640 1 -- 172.21.15.96:0/1154579808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7301033a0 msgr2=0x7fe7301a0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:44.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.655+0000 7fe737205640 1 --2- 172.21.15.96:0/1154579808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7301033a0 0x7fe7301a0b60 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe71c00e9f0 tx=0x7fe71c00edf0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.655+0000 7fe737205640 1 -- 172.21.15.96:0/1154579808 shutdown_connections 2024-04-25T14:14:44.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.655+0000 7fe737205640 1 --2- 172.21.15.96:0/1154579808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe7301080a0 0x7fe7301a7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.655+0000 7fe737205640 1 --2- 172.21.15.96:0/1154579808 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fe7200afb70 0x7fe7200b2080 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.655+0000 7fe737205640 1 --2- 172.21.15.96:0/1154579808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7301033a0 0x7fe7301a0b60 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.655+0000 7fe737205640 1 --2- 172.21.15.96:0/1154579808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe730102160 0x7fe7301a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.655+0000 7fe737205640 1 -- 172.21.15.96:0/1154579808 >> 172.21.15.96:0/1154579808 conn(0x7fe7300fd330 msgr2=0x7fe7300ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:44.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.656+0000 7fe737205640 1 -- 172.21.15.96:0/1154579808 shutdown_connections 2024-04-25T14:14:44.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.656+0000 7fe737205640 1 -- 172.21.15.96:0/1154579808 wait complete. 2024-04-25T14:14:44.667 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T14:14:44.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.826+0000 7f23df5e3640 1 -- 172.21.15.96:0/896828205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23d8106c60 msgr2=0x7f23d81090e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:44.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.826+0000 7f23df5e3640 1 --2- 172.21.15.96:0/896828205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23d8106c60 0x7f23d81090e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f23d4009900 tx=0x7f23d402a410 comp rx=0 tx=0).stop 2024-04-25T14:14:44.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.826+0000 7f23df5e3640 1 -- 172.21.15.96:0/896828205 shutdown_connections 2024-04-25T14:14:44.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.826+0000 7f23df5e3640 1 --2- 172.21.15.96:0/896828205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23d8109770 0x7f23d810bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.826+0000 7f23df5e3640 1 --2- 172.21.15.96:0/896828205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23d8106c60 0x7f23d81090e0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.826+0000 7f23df5e3640 1 --2- 172.21.15.96:0/896828205 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23d8104270 0x7f23d8106690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.826+0000 7f23df5e3640 1 -- 172.21.15.96:0/896828205 >> 172.21.15.96:0/896828205 conn(0x7f23d80fdbd0 msgr2=0x7f23d8100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:44.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.826+0000 7f23df5e3640 1 -- 172.21.15.96:0/896828205 shutdown_connections 2024-04-25T14:14:44.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.826+0000 7f23df5e3640 1 -- 172.21.15.96:0/896828205 wait complete. 2024-04-25T14:14:44.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.827+0000 7f23df5e3640 1 Processor -- start 2024-04-25T14:14:44.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.827+0000 7f23df5e3640 1 -- start start 2024-04-25T14:14:44.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.827+0000 7f23df5e3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23d8104270 0x7f23d819a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.827+0000 7f23df5e3640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23d8106c60 0x7f23d819aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.827+0000 7f23df5e3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23d8109770 0x7f23d81a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.827+0000 7f23df5e3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f23d810f600 con 0x7f23d8109770 2024-04-25T14:14:44.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.827+0000 7f23df5e3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f23d810f480 con 0x7f23d8104270 2024-04-25T14:14:44.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.827+0000 7f23df5e3640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f23d810f780 con 0x7f23d8106c60 2024-04-25T14:14:44.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.828+0000 7f23ddb80640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23d8109770 0x7f23d81a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.828+0000 7f23ddb80640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23d8109770 0x7f23d81a1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34534/0 (socket says 172.21.15.96:34534) 2024-04-25T14:14:44.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.828+0000 7f23dd37f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23d8104270 0x7f23d819a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.828+0000 7f23ddb80640 1 -- 172.21.15.96:0/3180336887 learned_addr learned my addr 172.21.15.96:0/3180336887 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:44.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.828+0000 7f23dcb7e640 1 --2- 172.21.15.96:0/3180336887 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23d8106c60 0x7f23d819aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.828+0000 7f23dd37f640 1 -- 172.21.15.96:0/3180336887 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23d8106c60 msgr2=0x7f23d819aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:44.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.828+0000 7f23dd37f640 1 --2- 172.21.15.96:0/3180336887 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23d8106c60 0x7f23d819aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.828+0000 7f23dd37f640 1 -- 172.21.15.96:0/3180336887 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23d8109770 msgr2=0x7f23d81a1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:44.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.828+0000 7f23dd37f640 1 --2- 172.21.15.96:0/3180336887 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23d8109770 0x7f23d81a1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.828+0000 7f23dd37f640 1 -- 172.21.15.96:0/3180336887 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23d81a2090 con 0x7f23d8104270 2024-04-25T14:14:44.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.828+0000 7f23ddb80640 1 --2- 172.21.15.96:0/3180336887 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23d8109770 0x7f23d81a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:14:44.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.828+0000 7f23dd37f640 1 --2- 172.21.15.96:0/3180336887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23d8104270 0x7f23d819a550 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f23c8005fd0 tx=0x7f23c800dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:44.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.828+0000 7f23ce7fc640 1 -- 172.21.15.96:0/3180336887 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c8009c00 con 0x7f23d8104270 2024-04-25T14:14:44.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.828+0000 7f23df5e3640 1 -- 172.21.15.96:0/3180336887 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23d81a2380 con 0x7f23d8104270 2024-04-25T14:14:44.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.828+0000 7f23ce7fc640 1 -- 172.21.15.96:0/3180336887 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23c8009da0 con 0x7f23d8104270 2024-04-25T14:14:44.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.828+0000 7f23ce7fc640 1 -- 172.21.15.96:0/3180336887 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c8003e40 con 0x7f23d8104270 2024-04-25T14:14:44.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.829+0000 7f23df5e3640 1 -- 172.21.15.96:0/3180336887 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23d81a2920 con 0x7f23d8104270 2024-04-25T14:14:44.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.832+0000 7f23e656d740 1 Processor -- start 2024-04-25T14:14:44.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.832+0000 7f23e656d740 1 -- start start 2024-04-25T14:14:44.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.833+0000 7f23ce7fc640 1 -- 172.21.15.96:0/3180336887 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f23c8004000 con 0x7f23d8104270 2024-04-25T14:14:44.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.833+0000 7f23e656d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed66bbf390 0x55ed66f61810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.833+0000 7f23e656d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed66b964d0 0x55ed66f61d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.833+0000 7f23e656d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed66b9e6b0 0x55ed66f62a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.833+0000 7f23e656d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ed66ec5d80 con 0x55ed66bbf390 2024-04-25T14:14:44.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.833+0000 7f23e656d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ed66e7fa20 con 0x55ed66b964d0 2024-04-25T14:14:44.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.833+0000 7f23e656d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ed66ec1d40 con 0x55ed66b9e6b0 2024-04-25T14:14:44.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.833+0000 7f23ddb80640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed66bbf390 0x55ed66f61810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.833+0000 7f23ddb80640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed66bbf390 0x55ed66f61810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34548/0 (socket says 172.21.15.96:34548) 2024-04-25T14:14:44.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.833+0000 7f23ddb80640 1 -- 172.21.15.96:0/1447674959 learned_addr learned my addr 172.21.15.96:0/1447674959 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:44.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.833+0000 7f23dcb7e640 1 --2- 172.21.15.96:0/1447674959 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed66b9e6b0 0x55ed66f62a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.833+0000 7f23dd37f640 1 --2- 172.21.15.96:0/1447674959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed66b964d0 0x55ed66f61d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.834+0000 7f23ce7fc640 1 --2- 172.21.15.96:0/3180336887 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f23c40afb70 0x7f23c40b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.834+0000 7f23dcb7e640 1 -- 172.21.15.96:0/1447674959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed66b964d0 msgr2=0x55ed66f61d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:44.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.834+0000 7f23dcb7e640 1 --2- 172.21.15.96:0/1447674959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed66b964d0 0x55ed66f61d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.834+0000 7f23dcb7e640 1 -- 172.21.15.96:0/1447674959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed66bbf390 msgr2=0x55ed66f61810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:44.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.834+0000 7f23dcb7e640 1 --2- 172.21.15.96:0/1447674959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed66bbf390 0x55ed66f61810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.834+0000 7f23dcb7e640 1 -- 172.21.15.96:0/1447674959 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed66e8ac30 con 0x55ed66b9e6b0 2024-04-25T14:14:44.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.834+0000 7f23ddb80640 1 --2- 172.21.15.96:0/1447674959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed66bbf390 0x55ed66f61810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:14:44.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.834+0000 7f23ce7fc640 1 -- 172.21.15.96:0/3180336887 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f23c80de6e0 con 0x7f23d8104270 2024-04-25T14:14:44.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.834+0000 7f23ddb80640 1 --2- 172.21.15.96:0/3180336887 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f23c40afb70 0x7f23c40b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.834+0000 7f23dcb7e640 1 --2- 172.21.15.96:0/1447674959 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed66b9e6b0 0x55ed66f62a20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f23d4005ab0 tx=0x7f23d4031be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a68051a6cae2859b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:44.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.835+0000 7f23c37fe640 1 -- 172.21.15.96:0/1447674959 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d4035070 con 0x55ed66b9e6b0 2024-04-25T14:14:44.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.835+0000 7f23c37fe640 1 -- 172.21.15.96:0/1447674959 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23d4031d50 con 0x55ed66b9e6b0 2024-04-25T14:14:44.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.835+0000 7f23ddb80640 1 --2- 172.21.15.96:0/3180336887 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f23c40afb70 0x7f23c40b2080 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x55ed66f62860 tx=0x7f23d0004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:44.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.835+0000 7f23c37fe640 1 -- 172.21.15.96:0/1447674959 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d403da50 con 0x55ed66b9e6b0 2024-04-25T14:14:44.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.836+0000 7f23e656d740 1 -- 172.21.15.96:0/1447674959 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed66b9e6b0 msgr2=0x55ed66f62a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:44.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.836+0000 7f23e656d740 1 --2- 172.21.15.96:0/1447674959 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed66b9e6b0 0x55ed66f62a20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f23d4005ab0 tx=0x7f23d4031be0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.836+0000 7f23e656d740 1 -- 172.21.15.96:0/1447674959 shutdown_connections 2024-04-25T14:14:44.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.836+0000 7f23e656d740 1 --2- 172.21.15.96:0/1447674959 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed66b9e6b0 0x55ed66f62a20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.836+0000 7f23e656d740 1 --2- 172.21.15.96:0/1447674959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed66b964d0 0x55ed66f61d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.836+0000 7f23e656d740 1 --2- 172.21.15.96:0/1447674959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed66bbf390 0x55ed66f61810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.836+0000 7f23e656d740 1 -- 172.21.15.96:0/1447674959 >> 172.21.15.96:0/1447674959 conn(0x55ed66bb34e0 msgr2=0x55ed66f5d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:44.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.836+0000 7f23e656d740 1 -- 172.21.15.96:0/1447674959 shutdown_connections 2024-04-25T14:14:44.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.836+0000 7f23e656d740 1 -- 172.21.15.96:0/1447674959 wait complete. 2024-04-25T14:14:44.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.836+0000 7f23e656d740 1 -- start start 2024-04-25T14:14:44.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.836+0000 7f23e656d740 1 Processor -- start 2024-04-25T14:14:44.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.837+0000 7f23e656d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed66b964d0 0x55ed66f6b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.837+0000 7f23e656d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed66b9e6b0 0x55ed66f6bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.837+0000 7f23e656d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed66bbf390 0x55ed66f72a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.837+0000 7f23e656d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ed66f634d0 con 0x55ed66b964d0 2024-04-25T14:14:44.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.837+0000 7f23e656d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ed66e4cdc0 con 0x55ed66b9e6b0 2024-04-25T14:14:44.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.837+0000 7f23e656d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ed66f63650 con 0x55ed66bbf390 2024-04-25T14:14:44.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.837+0000 7f23dd37f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed66b964d0 0x55ed66f6b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.837+0000 7f23dd37f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed66b964d0 0x55ed66f6b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34554/0 (socket says 172.21.15.96:34554) 2024-04-25T14:14:44.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.837+0000 7f23dd37f640 1 -- 172.21.15.96:0/1110717154 learned_addr learned my addr 172.21.15.96:0/1110717154 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:44.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.837+0000 7f23e656d740 10 client.0 _ll_register_callbacks cb 0x55ed66e68600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:14:44.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.837+0000 7f23dcb7e640 1 --2- 172.21.15.96:0/1110717154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed66b9e6b0 0x55ed66f6bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.837+0000 7f23ddb80640 1 --2- 172.21.15.96:0/1110717154 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed66bbf390 0x55ed66f72a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.838+0000 7f23dd37f640 1 -- 172.21.15.96:0/1110717154 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed66bbf390 msgr2=0x55ed66f72a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:44.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.838+0000 7f23dd37f640 1 --2- 172.21.15.96:0/1110717154 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed66bbf390 0x55ed66f72a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.838+0000 7f23dd37f640 1 -- 172.21.15.96:0/1110717154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed66b9e6b0 msgr2=0x55ed66f6bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:44.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.838+0000 7f23dd37f640 1 --2- 172.21.15.96:0/1110717154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed66b9e6b0 0x55ed66f6bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.838+0000 7f23dd37f640 1 -- 172.21.15.96:0/1110717154 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed66bd73f0 con 0x55ed66b964d0 2024-04-25T14:14:44.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.838+0000 7f23ddb80640 1 --2- 172.21.15.96:0/1110717154 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed66bbf390 0x55ed66f72a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:14:44.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.838+0000 7f23dd37f640 1 --2- 172.21.15.96:0/1110717154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed66b964d0 0x55ed66f6b5c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f23c80a6950 tx=0x7f23c80a24c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b67c549c5f5ac38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:44.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.838+0000 7f23c0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:14:44.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.838+0000 7f23c0ff9640 1 -- 172.21.15.96:0/1110717154 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c8020030 con 0x55ed66b964d0 2024-04-25T14:14:44.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.838+0000 7f23c0ff9640 1 -- 172.21.15.96:0/1110717154 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f23c80deb90 con 0x55ed66b964d0 2024-04-25T14:14:44.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.838+0000 7f23e656d740 1 -- 172.21.15.96:0/1110717154 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23d4031d50 con 0x55ed66b964d0 2024-04-25T14:14:44.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.838+0000 7f23c0ff9640 1 -- 172.21.15.96:0/1110717154 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f23c80035f0 con 0x55ed66b964d0 2024-04-25T14:14:44.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.838+0000 7f23e656d740 10 client.15909 fetch_fsmap learned FSMap version 26 2024-04-25T14:14:44.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.838+0000 7f23e656d740 1 -- 172.21.15.96:0/1110717154 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ed66ca17d0 con 0x55ed66b964d0 2024-04-25T14:14:44.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.839+0000 7f23c0ff9640 1 -- 172.21.15.96:0/1110717154 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23d4031d50 con 0x55ed66b964d0 2024-04-25T14:14:44.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.839+0000 7f23c0ff9640 1 -- 172.21.15.96:0/1110717154 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f23c80a25c0 con 0x55ed66b964d0 2024-04-25T14:14:44.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.839+0000 7f23e656d740 10 client.15909 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:14:44.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.839+0000 7f23e656d740 10 client.15909 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-25T14:14:44.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.839+0000 7f23e656d740 20 client.15909 populate_metadata read hostname 'smithi096' 2024-04-25T14:14:44.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.839+0000 7f23e656d740 1 --2- 172.21.15.96:0/1110717154 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x55ed66b7df60 0x55ed66f79220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.839+0000 7f23e656d740 4 client.15909 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:14:44.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.839+0000 7f23e656d740 1 -- 172.21.15.96:0/1110717154 --> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed66ed3990 con 0x55ed66b7df60 2024-04-25T14:14:44.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.839+0000 7f23dcb7e640 1 --2- 172.21.15.96:0/1110717154 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x55ed66b7df60 0x55ed66f79220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.839+0000 7f23c0ff9640 10 client.15909 ms_handle_connect on v2:172.21.15.151:6832/1382721608 2024-04-25T14:14:44.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.840+0000 7f23dcb7e640 1 --2- 172.21.15.96:0/1110717154 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x55ed66b7df60 0x55ed66f79220 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e3efaab587a8857 server_cookie=1111c43006879909 in_seq=0 out_seq=0 2024-04-25T14:14:44.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.841+0000 7f23c0ff9640 1 -- 172.21.15.96:0/1110717154 <== mds.4 v2:172.21.15.151:6832/1382721608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ed66ed3990 con 0x55ed66b7df60 2024-04-25T14:14:44.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.841+0000 7f23c0ff9640 10 client.15909 handle_command_reply: tid=0 2024-04-25T14:14:44.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.841+0000 7f23e656d740 1 client.15909 shutdown 2024-04-25T14:14:44.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.841+0000 7f23e656d740 1 -- 172.21.15.96:0/1110717154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed66b964d0 msgr2=0x55ed66f6b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:44.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.841+0000 7f23e656d740 1 --2- 172.21.15.96:0/1110717154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed66b964d0 0x55ed66f6b5c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f23c80a6950 tx=0x7f23c80a24c0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.841+0000 7f23e656d740 1 -- 172.21.15.96:0/1110717154 shutdown_connections 2024-04-25T14:14:44.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.841+0000 7f23e656d740 1 --2- 172.21.15.96:0/1110717154 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x55ed66b7df60 0x55ed66f79220 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.841+0000 7f23c0ff9640 0 client.15909 ms_handle_reset on v2:172.21.15.151:6832/1382721608 2024-04-25T14:14:44.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.841+0000 7f23e656d740 1 --2- 172.21.15.96:0/1110717154 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed66bbf390 0x55ed66f72a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.841+0000 7f23e656d740 1 --2- 172.21.15.96:0/1110717154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed66b9e6b0 0x55ed66f6bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.841+0000 7f23e656d740 1 --2- 172.21.15.96:0/1110717154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed66b964d0 0x55ed66f6b5c0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.841+0000 7f23e656d740 1 -- 172.21.15.96:0/1110717154 >> 172.21.15.96:0/1110717154 conn(0x55ed66bb34e0 msgr2=0x55ed66f5ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:44.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.841+0000 7f23e656d740 1 -- 172.21.15.96:0/1110717154 shutdown_connections 2024-04-25T14:14:44.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.841+0000 7f23e656d740 1 -- 172.21.15.96:0/1110717154 wait complete. 2024-04-25T14:14:44.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.841+0000 7f23e656d740 20 client.15909 trim_cache size 0 max 16384 2024-04-25T14:14:44.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.855+0000 7f23e656d740 1 Processor -- start 2024-04-25T14:14:44.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.855+0000 7f23e656d740 1 -- start start 2024-04-25T14:14:44.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.855+0000 7f23e656d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed66b964d0 0x55ed66f5d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.856+0000 7f23e656d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed66b9e6b0 0x55ed66f5fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.856+0000 7f23e656d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed66bbf390 0x55ed66f60360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.856+0000 7f23e656d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f23d81a9120 con 0x55ed66bbf390 2024-04-25T14:14:44.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.856+0000 7f23e656d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f23d81aac60 con 0x55ed66b964d0 2024-04-25T14:14:44.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.856+0000 7f23e656d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f23d81a97f0 con 0x55ed66b9e6b0 2024-04-25T14:14:44.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.856+0000 7f23ddb80640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed66bbf390 0x55ed66f60360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.856+0000 7f23ddb80640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed66bbf390 0x55ed66f60360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34566/0 (socket says 172.21.15.96:34566) 2024-04-25T14:14:44.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.856+0000 7f23dd37f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed66b964d0 0x55ed66f5d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.856+0000 7f23ddb80640 1 -- 172.21.15.96:0/3611295494 learned_addr learned my addr 172.21.15.96:0/3611295494 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:44.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.856+0000 7f23dcb7e640 1 --2- 172.21.15.96:0/3611295494 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed66b9e6b0 0x55ed66f5fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.856+0000 7f23ddb80640 1 -- 172.21.15.96:0/3611295494 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed66b9e6b0 msgr2=0x55ed66f5fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:44.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.856+0000 7f23ddb80640 1 --2- 172.21.15.96:0/3611295494 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed66b9e6b0 0x55ed66f5fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.856+0000 7f23ddb80640 1 -- 172.21.15.96:0/3611295494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed66b964d0 msgr2=0x55ed66f5d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:44.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.856+0000 7f23ddb80640 1 --2- 172.21.15.96:0/3611295494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed66b964d0 0x55ed66f5d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.856+0000 7f23ddb80640 1 -- 172.21.15.96:0/3611295494 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed66f69a50 con 0x55ed66bbf390 2024-04-25T14:14:44.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.856+0000 7f23dd37f640 1 --2- 172.21.15.96:0/3611295494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed66b964d0 0x55ed66f5d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:14:44.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.856+0000 7f23ddb80640 1 --2- 172.21.15.96:0/3611295494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed66bbf390 0x55ed66f60360 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f23d0014310 tx=0x7f23d0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb74a761818960d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:44.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.857+0000 7f23c2ffd640 1 -- 172.21.15.96:0/3611295494 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d0012c90 con 0x55ed66bbf390 2024-04-25T14:14:44.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.857+0000 7f23c2ffd640 1 -- 172.21.15.96:0/3611295494 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23d0012e30 con 0x55ed66bbf390 2024-04-25T14:14:44.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.857+0000 7f23c2ffd640 1 -- 172.21.15.96:0/3611295494 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d001f7c0 con 0x55ed66bbf390 2024-04-25T14:14:44.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.858+0000 7f23e656d740 1 -- 172.21.15.96:0/3611295494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed66bbf390 msgr2=0x55ed66f60360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:44.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.858+0000 7f23e656d740 1 --2- 172.21.15.96:0/3611295494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed66bbf390 0x55ed66f60360 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f23d0014310 tx=0x7f23d0017000 comp rx=0 tx=0).stop 2024-04-25T14:14:44.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.858+0000 7f23e656d740 1 -- 172.21.15.96:0/3611295494 shutdown_connections 2024-04-25T14:14:44.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.858+0000 7f23e656d740 1 --2- 172.21.15.96:0/3611295494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed66bbf390 0x55ed66f60360 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.858+0000 7f23e656d740 1 --2- 172.21.15.96:0/3611295494 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed66b9e6b0 0x55ed66f5fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.858+0000 7f23e656d740 1 --2- 172.21.15.96:0/3611295494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed66b964d0 0x55ed66f5d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.858+0000 7f23e656d740 1 -- 172.21.15.96:0/3611295494 >> 172.21.15.96:0/3611295494 conn(0x55ed66bb34e0 msgr2=0x55ed66f5c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:44.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.858+0000 7f23e656d740 1 -- 172.21.15.96:0/3611295494 shutdown_connections 2024-04-25T14:14:44.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.858+0000 7f23e656d740 1 -- 172.21.15.96:0/3611295494 wait complete. 2024-04-25T14:14:44.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.858+0000 7f23e656d740 1 -- start start 2024-04-25T14:14:44.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.858+0000 7f23e656d740 1 Processor -- start 2024-04-25T14:14:44.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.859+0000 7f23e656d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed66b964d0 0x55ed66f6f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.859+0000 7f23e656d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed66b9e6b0 0x55ed66f6fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.859+0000 7f23e656d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed66bbf390 0x55ed66f70d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.859+0000 7f23e656d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f23d81aa590 con 0x55ed66b9e6b0 2024-04-25T14:14:44.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.859+0000 7f23e656d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f23d81a9ec0 con 0x55ed66b964d0 2024-04-25T14:14:44.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.859+0000 7f23e656d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f23d81a7100 con 0x55ed66bbf390 2024-04-25T14:14:44.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.859+0000 7f23e656d740 10 client.0 _ll_register_callbacks cb 0x7f23c8005d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:14:44.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.859+0000 7f23dd37f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed66b964d0 0x55ed66f6f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.859+0000 7f23dcb7e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed66b9e6b0 0x55ed66f6fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.860+0000 7f23ddb80640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed66bbf390 0x55ed66f70d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.860+0000 7f23dd37f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed66b964d0 0x55ed66f6f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:54082/0 (socket says 172.21.15.96:54082) 2024-04-25T14:14:44.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.860+0000 7f23dd37f640 1 -- 172.21.15.96:0/4286601630 learned_addr learned my addr 172.21.15.96:0/4286601630 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:44.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.860+0000 7f23dcb7e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed66b9e6b0 0x55ed66f6fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34576/0 (socket says 172.21.15.96:34576) 2024-04-25T14:14:44.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.860+0000 7f23ddb80640 1 -- 172.21.15.96:0/4286601630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed66b964d0 msgr2=0x55ed66f6f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:44.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.860+0000 7f23ddb80640 1 --2- 172.21.15.96:0/4286601630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed66b964d0 0x55ed66f6f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.860+0000 7f23ddb80640 1 -- 172.21.15.96:0/4286601630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed66b9e6b0 msgr2=0x55ed66f6fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:44.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.860+0000 7f23ddb80640 1 --2- 172.21.15.96:0/4286601630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed66b9e6b0 0x55ed66f6fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.860+0000 7f23ddb80640 1 -- 172.21.15.96:0/4286601630 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed66f66d40 con 0x55ed66bbf390 2024-04-25T14:14:44.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.860+0000 7f23dd37f640 1 --2- 172.21.15.96:0/4286601630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed66b964d0 0x55ed66f6f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:14:44.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.860+0000 7f23dcb7e640 1 --2- 172.21.15.96:0/4286601630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed66b9e6b0 0x55ed66f6fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:14:44.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.860+0000 7f23ddb80640 1 --2- 172.21.15.96:0/4286601630 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed66bbf390 0x55ed66f70d70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f23d00094f0 tx=0x7f23d0014400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7dca900c044590be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:44.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.860+0000 7f23c0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-25T14:14:44.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.861+0000 7f23c0ff9640 1 -- 172.21.15.96:0/4286601630 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d0020420 con 0x55ed66bbf390 2024-04-25T14:14:44.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.861+0000 7f23c0ff9640 1 -- 172.21.15.96:0/4286601630 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f23d00205c0 con 0x55ed66bbf390 2024-04-25T14:14:44.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.861+0000 7f23e656d740 1 -- 172.21.15.96:0/4286601630 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23d0012e30 con 0x55ed66bbf390 2024-04-25T14:14:44.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.861+0000 7f23c0ff9640 1 -- 172.21.15.96:0/4286601630 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f23d00207c0 con 0x55ed66bbf390 2024-04-25T14:14:44.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.861+0000 7f23e656d740 10 client.25354 fetch_fsmap learned FSMap version 26 2024-04-25T14:14:44.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.861+0000 7f23e656d740 1 -- 172.21.15.96:0/4286601630 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ed66f723e0 con 0x55ed66bbf390 2024-04-25T14:14:44.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.861+0000 7f23c0ff9640 1 -- 172.21.15.96:0/4286601630 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23d0012e30 con 0x55ed66bbf390 2024-04-25T14:14:44.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.861+0000 7f23c0ff9640 1 -- 172.21.15.96:0/4286601630 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f23d000f040 con 0x55ed66bbf390 2024-04-25T14:14:44.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.861+0000 7f23e656d740 10 client.25354 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:14:44.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.861+0000 7f23e656d740 10 client.25354 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-25T14:14:44.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.861+0000 7f23e656d740 20 client.25354 populate_metadata read hostname 'smithi096' 2024-04-25T14:14:44.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.861+0000 7f23e656d740 1 --2- 172.21.15.96:0/4286601630 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x55ed66b533a0 0x55ed66fe3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.861+0000 7f23e656d740 4 client.25354 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:14:44.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.861+0000 7f23e656d740 1 -- 172.21.15.96:0/4286601630 --> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ed66ecd800 con 0x55ed66b533a0 2024-04-25T14:14:44.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.862+0000 7f23dd37f640 1 --2- 172.21.15.96:0/4286601630 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x55ed66b533a0 0x55ed66fe3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.862+0000 7f23c0ff9640 10 client.25354 ms_handle_connect on v2:172.21.15.151:6832/1382721608 2024-04-25T14:14:44.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.862+0000 7f23dd37f640 1 --2- 172.21.15.96:0/4286601630 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x55ed66b533a0 0x55ed66fe3bd0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ab2913e1ff0f2216 server_cookie=a37134af2cb17c6d in_seq=0 out_seq=0 2024-04-25T14:14:44.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.862+0000 7f23c0ff9640 1 -- 172.21.15.96:0/4286601630 <== mds.4 v2:172.21.15.151:6832/1382721608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ed66ecd800 con 0x55ed66b533a0 2024-04-25T14:14:44.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.862+0000 7f23c0ff9640 10 client.25354 handle_command_reply: tid=0 2024-04-25T14:14:44.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.862+0000 7f23e656d740 1 client.25354 shutdown 2024-04-25T14:14:44.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.863+0000 7f23e656d740 1 -- 172.21.15.96:0/4286601630 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed66bbf390 msgr2=0x55ed66f70d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:44.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.863+0000 7f23e656d740 1 --2- 172.21.15.96:0/4286601630 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed66bbf390 0x55ed66f70d70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f23d00094f0 tx=0x7f23d0014400 comp rx=0 tx=0).stop 2024-04-25T14:14:44.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.863+0000 7f23e656d740 1 -- 172.21.15.96:0/4286601630 shutdown_connections 2024-04-25T14:14:44.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.863+0000 7f23e656d740 1 --2- 172.21.15.96:0/4286601630 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x55ed66b533a0 0x55ed66fe3bd0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.863+0000 7f23e656d740 1 --2- 172.21.15.96:0/4286601630 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed66bbf390 0x55ed66f70d70 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.863+0000 7f23c0ff9640 0 client.25354 ms_handle_reset on v2:172.21.15.151:6832/1382721608 2024-04-25T14:14:44.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.863+0000 7f23e656d740 1 --2- 172.21.15.96:0/4286601630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed66b9e6b0 0x55ed66f6fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.863+0000 7f23e656d740 1 --2- 172.21.15.96:0/4286601630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed66b964d0 0x55ed66f6f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.863+0000 7f23e656d740 1 -- 172.21.15.96:0/4286601630 >> 172.21.15.96:0/4286601630 conn(0x55ed66bb34e0 msgr2=0x55ed66f69cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:44.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.863+0000 7f23e656d740 1 -- 172.21.15.96:0/4286601630 shutdown_connections 2024-04-25T14:14:44.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.863+0000 7f23e656d740 1 -- 172.21.15.96:0/4286601630 wait complete. 2024-04-25T14:14:44.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.863+0000 7f23e656d740 20 client.25354 trim_cache size 0 max 16384 2024-04-25T14:14:44.870 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-04-25T14:14:44.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.864+0000 7f23df5e3640 1 -- 172.21.15.96:0/3180336887 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f23c40afb70 msgr2=0x7f23c40b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:44.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.864+0000 7f23df5e3640 1 --2- 172.21.15.96:0/3180336887 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f23c40afb70 0x7f23c40b2080 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x55ed66f62860 tx=0x7f23d0004a40 comp rx=0 tx=0).stop 2024-04-25T14:14:44.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.864+0000 7f23df5e3640 1 -- 172.21.15.96:0/3180336887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23d8104270 msgr2=0x7f23d819a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:44.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.864+0000 7f23df5e3640 1 --2- 172.21.15.96:0/3180336887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23d8104270 0x7f23d819a550 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f23c8005fd0 tx=0x7f23c800dde0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.864+0000 7f23df5e3640 1 -- 172.21.15.96:0/3180336887 shutdown_connections 2024-04-25T14:14:44.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.864+0000 7f23df5e3640 1 --2- 172.21.15.96:0/3180336887 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f23c40afb70 0x7f23c40b2080 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.864+0000 7f23df5e3640 1 --2- 172.21.15.96:0/3180336887 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23d8109770 0x7f23d81a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.864+0000 7f23df5e3640 1 --2- 172.21.15.96:0/3180336887 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23d8106c60 0x7f23d819aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.864+0000 7f23df5e3640 1 --2- 172.21.15.96:0/3180336887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23d8104270 0x7f23d819a550 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.864+0000 7f23df5e3640 1 -- 172.21.15.96:0/3180336887 >> 172.21.15.96:0/3180336887 conn(0x7f23d80fdbd0 msgr2=0x7f23d81047c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:44.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.864+0000 7f23df5e3640 1 -- 172.21.15.96:0/3180336887 shutdown_connections 2024-04-25T14:14:44.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.864+0000 7f23df5e3640 1 -- 172.21.15.96:0/3180336887 wait complete. 2024-04-25T14:14:44.875 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T14:14:44.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:44 smithi097 ceph-mon[32720]: pgmap v379: 97 pgs: 97 active+clean; 668 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T14:14:44.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:44 smithi151 ceph-mon[32990]: pgmap v379: 97 pgs: 97 active+clean; 668 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T14:14:44.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.991+0000 7ff399ab1640 1 -- 172.21.15.96:0/2078967157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3941029e0 msgr2=0x7ff3941036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:44.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.991+0000 7ff399ab1640 1 --2- 172.21.15.96:0/2078967157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3941029e0 0x7ff3941036e0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff384009900 tx=0x7ff38402a410 comp rx=0 tx=0).stop 2024-04-25T14:14:44.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.992+0000 7ff399ab1640 1 -- 172.21.15.96:0/2078967157 shutdown_connections 2024-04-25T14:14:44.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.992+0000 7ff399ab1640 1 --2- 172.21.15.96:0/2078967157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff394108920 0x7ff39410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.992+0000 7ff399ab1640 1 --2- 172.21.15.96:0/2078967157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff394103c20 0x7ff394104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.992+0000 7ff399ab1640 1 --2- 172.21.15.96:0/2078967157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3941029e0 0x7ff3941036e0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.992+0000 7ff399ab1640 1 -- 172.21.15.96:0/2078967157 >> 172.21.15.96:0/2078967157 conn(0x7ff3940fdbd0 msgr2=0x7ff394100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:44.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.992+0000 7ff399ab1640 1 -- 172.21.15.96:0/2078967157 shutdown_connections 2024-04-25T14:14:44.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.992+0000 7ff399ab1640 1 -- 172.21.15.96:0/2078967157 wait complete. 2024-04-25T14:14:44.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.993+0000 7ff399ab1640 1 Processor -- start 2024-04-25T14:14:44.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.993+0000 7ff399ab1640 1 -- start start 2024-04-25T14:14:44.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.993+0000 7ff399ab1640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3941029e0 0x7ff39419a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.993+0000 7ff399ab1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff394103c20 0x7ff39419aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.993+0000 7ff399ab1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff394108920 0x7ff3941a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:44.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.993+0000 7ff399ab1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff39410f610 con 0x7ff394108920 2024-04-25T14:14:44.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.993+0000 7ff399ab1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff39410f490 con 0x7ff394103c20 2024-04-25T14:14:44.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.993+0000 7ff399ab1640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff39410f790 con 0x7ff3941029e0 2024-04-25T14:14:44.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.993+0000 7ff393fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff394108920 0x7ff3941a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.993+0000 7ff393fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff394108920 0x7ff3941a1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34604/0 (socket says 172.21.15.96:34604) 2024-04-25T14:14:44.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.993+0000 7ff393fff640 1 -- 172.21.15.96:0/2070701008 learned_addr learned my addr 172.21.15.96:0/2070701008 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:44.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.993+0000 7ff3937fe640 1 --2- 172.21.15.96:0/2070701008 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3941029e0 0x7ff39419a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.993+0000 7ff392ffd640 1 --2- 172.21.15.96:0/2070701008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff394103c20 0x7ff39419aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:44.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.994+0000 7ff393fff640 1 -- 172.21.15.96:0/2070701008 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3941029e0 msgr2=0x7ff39419a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:44.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.994+0000 7ff393fff640 1 --2- 172.21.15.96:0/2070701008 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3941029e0 0x7ff39419a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.994+0000 7ff393fff640 1 -- 172.21.15.96:0/2070701008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff394103c20 msgr2=0x7ff39419aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:44.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.994+0000 7ff393fff640 1 --2- 172.21.15.96:0/2070701008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff394103c20 0x7ff39419aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.994+0000 7ff393fff640 1 -- 172.21.15.96:0/2070701008 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3941a20b0 con 0x7ff394108920 2024-04-25T14:14:44.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.994+0000 7ff393fff640 1 --2- 172.21.15.96:0/2070701008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff394108920 0x7ff3941a1a40 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff38c009f60 tx=0x7ff38c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:44.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.994+0000 7ff390ff9640 1 -- 172.21.15.96:0/2070701008 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c00cd80 con 0x7ff394108920 2024-04-25T14:14:44.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.994+0000 7ff390ff9640 1 -- 172.21.15.96:0/2070701008 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff38c013770 con 0x7ff394108920 2024-04-25T14:14:44.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.994+0000 7ff390ff9640 1 -- 172.21.15.96:0/2070701008 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c005380 con 0x7ff394108920 2024-04-25T14:14:44.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.994+0000 7ff399ab1640 1 -- 172.21.15.96:0/2070701008 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff39410b890 con 0x7ff394108920 2024-04-25T14:14:45.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.995+0000 7ff399ab1640 1 -- 172.21.15.96:0/2070701008 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff39410bd90 con 0x7ff394108920 2024-04-25T14:14:45.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.997+0000 7ff390ff9640 1 -- 172.21.15.96:0/2070701008 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7ff38c013920 con 0x7ff394108920 2024-04-25T14:14:45.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.998+0000 7ff390ff9640 1 --2- 172.21.15.96:0/2070701008 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ff37c0afaf0 0x7ff37c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.998+0000 7ff390ff9640 1 -- 172.21.15.96:0/2070701008 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7ff38c0e2e30 con 0x7ff394108920 2024-04-25T14:14:45.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.998+0000 7ff3937fe640 1 --2- 172.21.15.96:0/2070701008 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ff37c0afaf0 0x7ff37c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.999+0000 7ff3a05fd740 1 Processor -- start 2024-04-25T14:14:45.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.999+0000 7ff3937fe640 1 --2- 172.21.15.96:0/2070701008 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ff37c0afaf0 0x7ff37c0b2000 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff384005ab0 tx=0x7ff384031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:45.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.999+0000 7ff3a05fd740 1 -- start start 2024-04-25T14:14:45.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.999+0000 7ff3a05fd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560503d2a390 0x5605040cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.999+0000 7ff3a05fd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560503d014d0 0x5605040ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.999+0000 7ff3a05fd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560503d096b0 0x5605040cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.999+0000 7ff3a05fd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560504030d80 con 0x560503d2a390 2024-04-25T14:14:45.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.999+0000 7ff3a05fd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560503feaa20 con 0x560503d096b0 2024-04-25T14:14:45.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.999+0000 7ff3a05fd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56050402cd40 con 0x560503d014d0 2024-04-25T14:14:45.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.999+0000 7ff3937fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560503d2a390 0x5605040cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.999+0000 7ff3937fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560503d2a390 0x5605040cc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34614/0 (socket says 172.21.15.96:34614) 2024-04-25T14:14:45.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.999+0000 7ff3937fe640 1 -- 172.21.15.96:0/2320898230 learned_addr learned my addr 172.21.15.96:0/2320898230 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:45.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.999+0000 7ff393fff640 1 --2- 172.21.15.96:0/2320898230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560503d096b0 0x5605040cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.999+0000 7ff392ffd640 1 --2- 172.21.15.96:0/2320898230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560503d014d0 0x5605040ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.999+0000 7ff3937fe640 1 -- 172.21.15.96:0/2320898230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560503d014d0 msgr2=0x5605040ccd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:45.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.999+0000 7ff3937fe640 1 --2- 172.21.15.96:0/2320898230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560503d014d0 0x5605040ccd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.999+0000 7ff3937fe640 1 -- 172.21.15.96:0/2320898230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560503d096b0 msgr2=0x5605040cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:45.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.999+0000 7ff3937fe640 1 --2- 172.21.15.96:0/2320898230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560503d096b0 0x5605040cda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:44.999+0000 7ff3937fe640 1 -- 172.21.15.96:0/2320898230 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560503ff5c30 con 0x560503d2a390 2024-04-25T14:14:45.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.000+0000 7ff3937fe640 1 --2- 172.21.15.96:0/2320898230 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560503d2a390 0x5605040cc810 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff384009230 tx=0x7ff38403e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1035133e5830ddd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:45.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.000+0000 7ff379ffb640 1 -- 172.21.15.96:0/2320898230 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38404d070 con 0x560503d2a390 2024-04-25T14:14:45.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.000+0000 7ff379ffb640 1 -- 172.21.15.96:0/2320898230 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff384004440 con 0x560503d2a390 2024-04-25T14:14:45.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.000+0000 7ff379ffb640 1 -- 172.21.15.96:0/2320898230 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff384035050 con 0x560503d2a390 2024-04-25T14:14:45.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.001+0000 7ff3a05fd740 1 -- 172.21.15.96:0/2320898230 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560503d2a390 msgr2=0x5605040cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:45.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.001+0000 7ff3a05fd740 1 --2- 172.21.15.96:0/2320898230 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560503d2a390 0x5605040cc810 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff384009230 tx=0x7ff38403e070 comp rx=0 tx=0).stop 2024-04-25T14:14:45.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.001+0000 7ff3a05fd740 1 -- 172.21.15.96:0/2320898230 shutdown_connections 2024-04-25T14:14:45.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.001+0000 7ff3a05fd740 1 --2- 172.21.15.96:0/2320898230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560503d096b0 0x5605040cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.001+0000 7ff3a05fd740 1 --2- 172.21.15.96:0/2320898230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560503d014d0 0x5605040ccd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.001+0000 7ff3a05fd740 1 --2- 172.21.15.96:0/2320898230 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560503d2a390 0x5605040cc810 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.001+0000 7ff3a05fd740 1 -- 172.21.15.96:0/2320898230 >> 172.21.15.96:0/2320898230 conn(0x560503d1e4e0 msgr2=0x5605040c8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:45.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.001+0000 7ff3a05fd740 1 -- 172.21.15.96:0/2320898230 shutdown_connections 2024-04-25T14:14:45.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.001+0000 7ff3a05fd740 1 -- 172.21.15.96:0/2320898230 wait complete. 2024-04-25T14:14:45.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.001+0000 7ff3a05fd740 1 -- start start 2024-04-25T14:14:45.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.001+0000 7ff3a05fd740 1 Processor -- start 2024-04-25T14:14:45.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.002+0000 7ff3a05fd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560503d014d0 0x5605040d6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.002+0000 7ff3a05fd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560503d096b0 0x5605040d6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.002+0000 7ff3a05fd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560503ce8f60 0x5605040ddbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.002+0000 7ff3a05fd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5605040ce4d0 con 0x560503d096b0 2024-04-25T14:14:45.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.002+0000 7ff3a05fd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560503fb7dc0 con 0x560503ce8f60 2024-04-25T14:14:45.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.002+0000 7ff3a05fd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5605040ce650 con 0x560503d014d0 2024-04-25T14:14:45.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.002+0000 7ff3a05fd740 10 client.0 _ll_register_callbacks cb 0x560503fd3600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:14:45.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.002+0000 7ff392ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560503d096b0 0x5605040d6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.002+0000 7ff392ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560503d096b0 0x5605040d6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34626/0 (socket says 172.21.15.96:34626) 2024-04-25T14:14:45.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.002+0000 7ff392ffd640 1 -- 172.21.15.96:0/3279105532 learned_addr learned my addr 172.21.15.96:0/3279105532 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:45.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.002+0000 7ff3937fe640 1 --2- 172.21.15.96:0/3279105532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560503d014d0 0x5605040d6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.002+0000 7ff393fff640 1 --2- 172.21.15.96:0/3279105532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560503ce8f60 0x5605040ddbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.002+0000 7ff392ffd640 1 -- 172.21.15.96:0/3279105532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560503d014d0 msgr2=0x5605040d6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:45.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.002+0000 7ff392ffd640 1 --2- 172.21.15.96:0/3279105532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560503d014d0 0x5605040d6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.002+0000 7ff392ffd640 1 -- 172.21.15.96:0/3279105532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560503ce8f60 msgr2=0x5605040ddbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:45.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.002+0000 7ff392ffd640 1 --2- 172.21.15.96:0/3279105532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560503ce8f60 0x5605040ddbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.002+0000 7ff392ffd640 1 -- 172.21.15.96:0/3279105532 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560503d423f0 con 0x560503d096b0 2024-04-25T14:14:45.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.003+0000 7ff392ffd640 1 --2- 172.21.15.96:0/3279105532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560503d096b0 0x5605040d6c90 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7ff38800b940 tx=0x7ff38800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dc2170fabe89d46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:45.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.003+0000 7ff372ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:14:45.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.003+0000 7ff372ffd640 1 -- 172.21.15.96:0/3279105532 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38801b470 con 0x560503d096b0 2024-04-25T14:14:45.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.003+0000 7ff3a05fd740 1 -- 172.21.15.96:0/3279105532 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff384004440 con 0x560503d096b0 2024-04-25T14:14:45.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.003+0000 7ff372ffd640 1 -- 172.21.15.96:0/3279105532 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7ff38801b610 con 0x560503d096b0 2024-04-25T14:14:45.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.003+0000 7ff372ffd640 1 -- 172.21.15.96:0/3279105532 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff38801b810 con 0x560503d096b0 2024-04-25T14:14:45.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.003+0000 7ff3a05fd740 10 client.15927 fetch_fsmap learned FSMap version 26 2024-04-25T14:14:45.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.003+0000 7ff3a05fd740 1 -- 172.21.15.96:0/3279105532 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560503e0c7d0 con 0x560503d096b0 2024-04-25T14:14:45.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.004+0000 7ff372ffd640 1 -- 172.21.15.96:0/3279105532 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff384004440 con 0x560503d096b0 2024-04-25T14:14:45.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.004+0000 7ff372ffd640 1 -- 172.21.15.96:0/3279105532 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff388012c50 con 0x560503d096b0 2024-04-25T14:14:45.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.004+0000 7ff3a05fd740 10 client.15927 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:14:45.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.004+0000 7ff3a05fd740 10 client.15927 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T14:14:45.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.004+0000 7ff3a05fd740 20 client.15927 populate_metadata read hostname 'smithi096' 2024-04-25T14:14:45.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.004+0000 7ff3a05fd740 1 --2- 172.21.15.96:0/3279105532 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x560503cbe3a0 0x5605040e4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.004+0000 7ff3a05fd740 4 client.15927 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:14:45.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.004+0000 7ff3a05fd740 1 -- 172.21.15.96:0/3279105532 --> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56050403e990 con 0x560503cbe3a0 2024-04-25T14:14:45.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.004+0000 7ff3937fe640 1 --2- 172.21.15.96:0/3279105532 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x560503cbe3a0 0x5605040e4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.005+0000 7ff372ffd640 10 client.15927 ms_handle_connect on v2:172.21.15.97:6834/3872301726 2024-04-25T14:14:45.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.005+0000 7ff3937fe640 1 --2- 172.21.15.96:0/3279105532 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x560503cbe3a0 0x5605040e4570 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b6e1299915621a01 server_cookie=c9edbf2a078f1fc2 in_seq=0 out_seq=0 2024-04-25T14:14:45.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.006+0000 7ff372ffd640 1 -- 172.21.15.96:0/3279105532 <== mds.3 v2:172.21.15.97:6834/3872301726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56050403e990 con 0x560503cbe3a0 2024-04-25T14:14:45.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.006+0000 7ff372ffd640 10 client.15927 handle_command_reply: tid=0 2024-04-25T14:14:45.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.006+0000 7ff3a05fd740 1 client.15927 shutdown 2024-04-25T14:14:45.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.006+0000 7ff3a05fd740 1 -- 172.21.15.96:0/3279105532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560503d096b0 msgr2=0x5605040d6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:45.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.006+0000 7ff3a05fd740 1 --2- 172.21.15.96:0/3279105532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560503d096b0 0x5605040d6c90 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7ff38800b940 tx=0x7ff38800be00 comp rx=0 tx=0).stop 2024-04-25T14:14:45.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.006+0000 7ff3a05fd740 1 -- 172.21.15.96:0/3279105532 shutdown_connections 2024-04-25T14:14:45.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.006+0000 7ff3a05fd740 1 --2- 172.21.15.96:0/3279105532 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x560503cbe3a0 0x5605040e4570 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.006+0000 7ff3a05fd740 1 --2- 172.21.15.96:0/3279105532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560503ce8f60 0x5605040ddbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.006+0000 7ff372ffd640 0 client.15927 ms_handle_reset on v2:172.21.15.97:6834/3872301726 2024-04-25T14:14:45.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.006+0000 7ff3a05fd740 1 --2- 172.21.15.96:0/3279105532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560503d096b0 0x5605040d6c90 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.006+0000 7ff3a05fd740 1 --2- 172.21.15.96:0/3279105532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560503d014d0 0x5605040d6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.006+0000 7ff3a05fd740 1 -- 172.21.15.96:0/3279105532 >> 172.21.15.96:0/3279105532 conn(0x560503d1e4e0 msgr2=0x5605040c7040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:45.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.006+0000 7ff3a05fd740 1 -- 172.21.15.96:0/3279105532 shutdown_connections 2024-04-25T14:14:45.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.006+0000 7ff3a05fd740 1 -- 172.21.15.96:0/3279105532 wait complete. 2024-04-25T14:14:45.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.006+0000 7ff3a05fd740 20 client.15927 trim_cache size 0 max 16384 2024-04-25T14:14:45.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.020+0000 7ff3a05fd740 1 Processor -- start 2024-04-25T14:14:45.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.020+0000 7ff3a05fd740 1 -- start start 2024-04-25T14:14:45.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.020+0000 7ff3a05fd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560503cbe3a0 0x5605040d4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.020+0000 7ff3a05fd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560503ce8f60 0x5605040ddbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.020+0000 7ff3a05fd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560503d014d0 0x5605040d52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.020+0000 7ff3a05fd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff3941a90c0 con 0x560503d014d0 2024-04-25T14:14:45.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.020+0000 7ff3a05fd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff3941aac00 con 0x560503ce8f60 2024-04-25T14:14:45.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.020+0000 7ff3a05fd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff3941a9790 con 0x560503cbe3a0 2024-04-25T14:14:45.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.020+0000 7ff393fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560503d014d0 0x5605040d52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.020+0000 7ff393fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560503d014d0 0x5605040d52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34634/0 (socket says 172.21.15.96:34634) 2024-04-25T14:14:45.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.020+0000 7ff393fff640 1 -- 172.21.15.96:0/3784397862 learned_addr learned my addr 172.21.15.96:0/3784397862 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:45.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.020+0000 7ff392ffd640 1 --2- 172.21.15.96:0/3784397862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560503ce8f60 0x5605040ddbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.021+0000 7ff3937fe640 1 --2- 172.21.15.96:0/3784397862 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560503cbe3a0 0x5605040d4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.021+0000 7ff393fff640 1 -- 172.21.15.96:0/3784397862 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560503cbe3a0 msgr2=0x5605040d4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:45.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.021+0000 7ff393fff640 1 --2- 172.21.15.96:0/3784397862 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560503cbe3a0 0x5605040d4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.021+0000 7ff393fff640 1 -- 172.21.15.96:0/3784397862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560503ce8f60 msgr2=0x5605040ddbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:45.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.021+0000 7ff393fff640 1 --2- 172.21.15.96:0/3784397862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560503ce8f60 0x5605040ddbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.021+0000 7ff393fff640 1 -- 172.21.15.96:0/3784397862 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605040d4860 con 0x560503d014d0 2024-04-25T14:14:45.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.021+0000 7ff393fff640 1 --2- 172.21.15.96:0/3784397862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560503d014d0 0x5605040d52b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ff38c0e0960 tx=0x7ff38c0ab4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63d1506f76abfd5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:45.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.021+0000 7ff3797fa640 1 -- 172.21.15.96:0/3784397862 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c0a7630 con 0x560503d014d0 2024-04-25T14:14:45.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.021+0000 7ff3797fa640 1 -- 172.21.15.96:0/3784397862 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff38c013c40 con 0x560503d014d0 2024-04-25T14:14:45.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.021+0000 7ff3797fa640 1 -- 172.21.15.96:0/3784397862 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c0af4a0 con 0x560503d014d0 2024-04-25T14:14:45.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.022+0000 7ff3a05fd740 1 -- 172.21.15.96:0/3784397862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560503d014d0 msgr2=0x5605040d52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:45.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.022+0000 7ff3a05fd740 1 --2- 172.21.15.96:0/3784397862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560503d014d0 0x5605040d52b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ff38c0e0960 tx=0x7ff38c0ab4e0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.022+0000 7ff3a05fd740 1 -- 172.21.15.96:0/3784397862 shutdown_connections 2024-04-25T14:14:45.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.022+0000 7ff3a05fd740 1 --2- 172.21.15.96:0/3784397862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560503d014d0 0x5605040d52b0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.022+0000 7ff3a05fd740 1 --2- 172.21.15.96:0/3784397862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560503ce8f60 0x5605040ddbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.022+0000 7ff3a05fd740 1 --2- 172.21.15.96:0/3784397862 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560503cbe3a0 0x5605040d4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.022+0000 7ff3a05fd740 1 -- 172.21.15.96:0/3784397862 >> 172.21.15.96:0/3784397862 conn(0x560503d1e4e0 msgr2=0x5605040d14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:45.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.022+0000 7ff3a05fd740 1 -- 172.21.15.96:0/3784397862 shutdown_connections 2024-04-25T14:14:45.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.022+0000 7ff3a05fd740 1 -- 172.21.15.96:0/3784397862 wait complete. 2024-04-25T14:14:45.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.022+0000 7ff3a05fd740 1 -- start start 2024-04-25T14:14:45.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.023+0000 7ff3a05fd740 1 Processor -- start 2024-04-25T14:14:45.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.023+0000 7ff3a05fd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560503cbe3a0 0x5605040dacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.023+0000 7ff3a05fd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560503ce8f60 0x5605040db230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.023+0000 7ff3a05fd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560503d014d0 0x56050414a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.023+0000 7ff3a05fd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff3941aa530 con 0x560503ce8f60 2024-04-25T14:14:45.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.023+0000 7ff3a05fd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff3941a9e60 con 0x560503cbe3a0 2024-04-25T14:14:45.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.023+0000 7ff3a05fd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff3941a70a0 con 0x560503d014d0 2024-04-25T14:14:45.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.023+0000 7ff392ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560503ce8f60 0x5605040db230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.023+0000 7ff392ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560503ce8f60 0x5605040db230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34636/0 (socket says 172.21.15.96:34636) 2024-04-25T14:14:45.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.023+0000 7ff3a05fd740 10 client.0 _ll_register_callbacks cb 0x7ff388004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:14:45.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.023+0000 7ff392ffd640 1 -- 172.21.15.96:0/1080658978 learned_addr learned my addr 172.21.15.96:0/1080658978 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:45.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.023+0000 7ff3937fe640 1 --2- 172.21.15.96:0/1080658978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560503cbe3a0 0x5605040dacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.023+0000 7ff393fff640 1 --2- 172.21.15.96:0/1080658978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560503d014d0 0x56050414a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.023+0000 7ff392ffd640 1 -- 172.21.15.96:0/1080658978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560503d014d0 msgr2=0x56050414a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:45.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.023+0000 7ff392ffd640 1 --2- 172.21.15.96:0/1080658978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560503d014d0 0x56050414a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.023+0000 7ff392ffd640 1 -- 172.21.15.96:0/1080658978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560503cbe3a0 msgr2=0x5605040dacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:45.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.023+0000 7ff392ffd640 1 --2- 172.21.15.96:0/1080658978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560503cbe3a0 0x5605040dacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.023+0000 7ff392ffd640 1 -- 172.21.15.96:0/1080658978 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56050414ae60 con 0x560503ce8f60 2024-04-25T14:14:45.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.023+0000 7ff3937fe640 1 --2- 172.21.15.96:0/1080658978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560503cbe3a0 0x5605040dacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:14:45.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.024+0000 7ff392ffd640 1 --2- 172.21.15.96:0/1080658978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560503ce8f60 0x5605040db230 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7ff388004360 tx=0x7ff388004400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d14fde6e5ff5bba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:45.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.024+0000 7ff372ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:14:45.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.024+0000 7ff372ffd640 1 -- 172.21.15.96:0/1080658978 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff388026660 con 0x560503ce8f60 2024-04-25T14:14:45.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.024+0000 7ff372ffd640 1 -- 172.21.15.96:0/1080658978 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7ff388026800 con 0x560503ce8f60 2024-04-25T14:14:45.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.024+0000 7ff3a05fd740 1 -- 172.21.15.96:0/1080658978 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff38c013c40 con 0x560503ce8f60 2024-04-25T14:14:45.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.024+0000 7ff372ffd640 1 -- 172.21.15.96:0/1080658978 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff388026b00 con 0x560503ce8f60 2024-04-25T14:14:45.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.024+0000 7ff3a05fd740 10 client.15933 fetch_fsmap learned FSMap version 26 2024-04-25T14:14:45.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.024+0000 7ff3a05fd740 1 -- 172.21.15.96:0/1080658978 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56050414c180 con 0x560503ce8f60 2024-04-25T14:14:45.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.024+0000 7ff372ffd640 1 -- 172.21.15.96:0/1080658978 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff388016070 con 0x560503ce8f60 2024-04-25T14:14:45.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.024+0000 7ff372ffd640 1 -- 172.21.15.96:0/1080658978 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff3880050b0 con 0x560503ce8f60 2024-04-25T14:14:45.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.024+0000 7ff3a05fd740 10 client.15933 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:14:45.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.024+0000 7ff3a05fd740 10 client.15933 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T14:14:45.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.024+0000 7ff3a05fd740 20 client.15933 populate_metadata read hostname 'smithi096' 2024-04-25T14:14:45.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.025+0000 7ff3a05fd740 1 --2- 172.21.15.96:0/1080658978 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x56050414cb30 0x56050414ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.025+0000 7ff3a05fd740 4 client.15933 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:14:45.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.025+0000 7ff3a05fd740 1 -- 172.21.15.96:0/1080658978 --> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ff38c013c40 con 0x56050414cb30 2024-04-25T14:14:45.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.025+0000 7ff3937fe640 1 --2- 172.21.15.96:0/1080658978 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x56050414cb30 0x56050414ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.025+0000 7ff372ffd640 10 client.15933 ms_handle_connect on v2:172.21.15.97:6834/3872301726 2024-04-25T14:14:45.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.025+0000 7ff3937fe640 1 --2- 172.21.15.96:0/1080658978 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x56050414cb30 0x56050414ef50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ab1140e0c7325f0e server_cookie=4a27b4ce41010035 in_seq=0 out_seq=0 2024-04-25T14:14:45.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.026+0000 7ff372ffd640 1 -- 172.21.15.96:0/1080658978 <== mds.3 v2:172.21.15.97:6834/3872301726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ff38c013c40 con 0x56050414cb30 2024-04-25T14:14:45.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.026+0000 7ff372ffd640 10 client.15933 handle_command_reply: tid=0 2024-04-25T14:14:45.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.026+0000 7ff3a05fd740 1 client.15933 shutdown 2024-04-25T14:14:45.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.026+0000 7ff3a05fd740 1 -- 172.21.15.96:0/1080658978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560503ce8f60 msgr2=0x5605040db230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:45.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.026+0000 7ff3a05fd740 1 --2- 172.21.15.96:0/1080658978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560503ce8f60 0x5605040db230 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7ff388004360 tx=0x7ff388004400 comp rx=0 tx=0).stop 2024-04-25T14:14:45.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.026+0000 7ff3a05fd740 1 -- 172.21.15.96:0/1080658978 shutdown_connections 2024-04-25T14:14:45.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.026+0000 7ff3a05fd740 1 --2- 172.21.15.96:0/1080658978 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x56050414cb30 0x56050414ef50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.026+0000 7ff372ffd640 0 client.15933 ms_handle_reset on v2:172.21.15.97:6834/3872301726 2024-04-25T14:14:45.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.026+0000 7ff3a05fd740 1 --2- 172.21.15.96:0/1080658978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560503d014d0 0x56050414a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.026+0000 7ff3a05fd740 1 --2- 172.21.15.96:0/1080658978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560503ce8f60 0x5605040db230 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.026+0000 7ff3a05fd740 1 --2- 172.21.15.96:0/1080658978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560503cbe3a0 0x5605040dacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.026+0000 7ff3a05fd740 1 -- 172.21.15.96:0/1080658978 >> 172.21.15.96:0/1080658978 conn(0x560503d1e4e0 msgr2=0x5605040cae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:45.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.026+0000 7ff3a05fd740 1 -- 172.21.15.96:0/1080658978 shutdown_connections 2024-04-25T14:14:45.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.026+0000 7ff3a05fd740 1 -- 172.21.15.96:0/1080658978 wait complete. 2024-04-25T14:14:45.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.026+0000 7ff3a05fd740 20 client.15933 trim_cache size 0 max 16384 2024-04-25T14:14:45.032 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-04-25T14:14:45.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.027+0000 7ff399ab1640 1 -- 172.21.15.96:0/2070701008 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ff37c0afaf0 msgr2=0x7ff37c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:45.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.027+0000 7ff399ab1640 1 --2- 172.21.15.96:0/2070701008 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ff37c0afaf0 0x7ff37c0b2000 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff384005ab0 tx=0x7ff384031400 comp rx=0 tx=0).stop 2024-04-25T14:14:45.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.027+0000 7ff399ab1640 1 -- 172.21.15.96:0/2070701008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff394108920 msgr2=0x7ff3941a1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:45.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.027+0000 7ff399ab1640 1 --2- 172.21.15.96:0/2070701008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff394108920 0x7ff3941a1a40 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff38c009f60 tx=0x7ff38c009f90 comp rx=0 tx=0).stop 2024-04-25T14:14:45.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.027+0000 7ff399ab1640 1 -- 172.21.15.96:0/2070701008 shutdown_connections 2024-04-25T14:14:45.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.027+0000 7ff399ab1640 1 --2- 172.21.15.96:0/2070701008 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7ff37c0afaf0 0x7ff37c0b2000 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.027+0000 7ff399ab1640 1 --2- 172.21.15.96:0/2070701008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff394108920 0x7ff3941a1a40 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.027+0000 7ff399ab1640 1 --2- 172.21.15.96:0/2070701008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff394103c20 0x7ff39419aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.027+0000 7ff399ab1640 1 --2- 172.21.15.96:0/2070701008 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3941029e0 0x7ff39419a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.027+0000 7ff399ab1640 1 -- 172.21.15.96:0/2070701008 >> 172.21.15.96:0/2070701008 conn(0x7ff3940fdbd0 msgr2=0x7ff394104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:45.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.027+0000 7ff399ab1640 1 -- 172.21.15.96:0/2070701008 shutdown_connections 2024-04-25T14:14:45.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.027+0000 7ff399ab1640 1 -- 172.21.15.96:0/2070701008 wait complete. 2024-04-25T14:14:45.042 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T14:14:45.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.154+0000 7fbf1b1ce640 1 -- 172.21.15.96:0/124464483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf14101e10 msgr2=0x7fbf14102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:45.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.154+0000 7fbf1b1ce640 1 --2- 172.21.15.96:0/124464483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf14101e10 0x7fbf14102220 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fbf08009900 tx=0x7fbf0802a410 comp rx=0 tx=0).stop 2024-04-25T14:14:45.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.154+0000 7fbf1b1ce640 1 -- 172.21.15.96:0/124464483 shutdown_connections 2024-04-25T14:14:45.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.154+0000 7fbf1b1ce640 1 --2- 172.21.15.96:0/124464483 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf14109780 0x7fbf1410bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.154+0000 7fbf1b1ce640 1 --2- 172.21.15.96:0/124464483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf14106c10 0x7fbf141090f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.154+0000 7fbf1b1ce640 1 --2- 172.21.15.96:0/124464483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf14101e10 0x7fbf14102220 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.154+0000 7fbf1b1ce640 1 -- 172.21.15.96:0/124464483 >> 172.21.15.96:0/124464483 conn(0x7fbf140fd340 msgr2=0x7fbf140ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:45.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.154+0000 7fbf1b1ce640 1 -- 172.21.15.96:0/124464483 shutdown_connections 2024-04-25T14:14:45.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.154+0000 7fbf1b1ce640 1 -- 172.21.15.96:0/124464483 wait complete. 2024-04-25T14:14:45.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.155+0000 7fbf1b1ce640 1 Processor -- start 2024-04-25T14:14:45.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.155+0000 7fbf1b1ce640 1 -- start start 2024-04-25T14:14:45.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.155+0000 7fbf1b1ce640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf14101e10 0x7fbf1419a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.155+0000 7fbf1b1ce640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf14106c10 0x7fbf1419a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.155+0000 7fbf1b1ce640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf14109780 0x7fbf141a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.155+0000 7fbf1b1ce640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1410f630 con 0x7fbf14101e10 2024-04-25T14:14:45.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.155+0000 7fbf1b1ce640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1410f4b0 con 0x7fbf14109780 2024-04-25T14:14:45.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.155+0000 7fbf1b1ce640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1410f7b0 con 0x7fbf14106c10 2024-04-25T14:14:45.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.155+0000 7fbf18f6a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf14101e10 0x7fbf1419a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.155+0000 7fbf18f6a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf14101e10 0x7fbf1419a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34662/0 (socket says 172.21.15.96:34662) 2024-04-25T14:14:45.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.155+0000 7fbf18f6a640 1 -- 172.21.15.96:0/2101538966 learned_addr learned my addr 172.21.15.96:0/2101538966 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:45.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.155+0000 7fbf13fff640 1 --2- 172.21.15.96:0/2101538966 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf14106c10 0x7fbf1419a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.155+0000 7fbf1976b640 1 --2- 172.21.15.96:0/2101538966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf14109780 0x7fbf141a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.155+0000 7fbf18f6a640 1 -- 172.21.15.96:0/2101538966 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf14106c10 msgr2=0x7fbf1419a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:45.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.155+0000 7fbf18f6a640 1 --2- 172.21.15.96:0/2101538966 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf14106c10 0x7fbf1419a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.155+0000 7fbf18f6a640 1 -- 172.21.15.96:0/2101538966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf14109780 msgr2=0x7fbf141a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:45.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.156+0000 7fbf18f6a640 1 --2- 172.21.15.96:0/2101538966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf14109780 0x7fbf141a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.156+0000 7fbf18f6a640 1 -- 172.21.15.96:0/2101538966 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf141a1d40 con 0x7fbf14101e10 2024-04-25T14:14:45.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.156+0000 7fbf18f6a640 1 --2- 172.21.15.96:0/2101538966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf14101e10 0x7fbf1419a200 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fbf08031440 tx=0x7fbf08003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:45.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.156+0000 7fbf11ffb640 1 -- 172.21.15.96:0/2101538966 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0803f070 con 0x7fbf14101e10 2024-04-25T14:14:45.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.156+0000 7fbf11ffb640 1 -- 172.21.15.96:0/2101538966 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf08031c50 con 0x7fbf14101e10 2024-04-25T14:14:45.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.156+0000 7fbf1b1ce640 1 -- 172.21.15.96:0/2101538966 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf14102860 con 0x7fbf14101e10 2024-04-25T14:14:45.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.156+0000 7fbf11ffb640 1 -- 172.21.15.96:0/2101538966 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf08035030 con 0x7fbf14101e10 2024-04-25T14:14:45.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.156+0000 7fbf1b1ce640 1 -- 172.21.15.96:0/2101538966 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf14102d50 con 0x7fbf14101e10 2024-04-25T14:14:45.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.159+0000 7fbf2207f740 1 Processor -- start 2024-04-25T14:14:45.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.160+0000 7fbf2207f740 1 -- start start 2024-04-25T14:14:45.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.160+0000 7fbf2207f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec96bf0390 0x55ec96f92810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.160+0000 7fbf2207f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec96bc74d0 0x55ec96f92d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.160+0000 7fbf2207f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec96bcf6b0 0x55ec96f93a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.160+0000 7fbf2207f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ec96ef6d80 con 0x55ec96bcf6b0 2024-04-25T14:14:45.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.160+0000 7fbf2207f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ec96eb0a20 con 0x55ec96bf0390 2024-04-25T14:14:45.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.160+0000 7fbf2207f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ec96ef2d40 con 0x55ec96bc74d0 2024-04-25T14:14:45.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.160+0000 7fbf11ffb640 1 -- 172.21.15.96:0/2101538966 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fbf08042070 con 0x7fbf14101e10 2024-04-25T14:14:45.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.160+0000 7fbf13fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec96bcf6b0 0x55ec96f93a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.160+0000 7fbf13fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec96bcf6b0 0x55ec96f93a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34668/0 (socket says 172.21.15.96:34668) 2024-04-25T14:14:45.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.160+0000 7fbf13fff640 1 -- 172.21.15.96:0/3992423856 learned_addr learned my addr 172.21.15.96:0/3992423856 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:45.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.160+0000 7fbf1976b640 1 --2- 172.21.15.96:0/3992423856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec96bf0390 0x55ec96f92810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.161+0000 7fbf13fff640 1 -- 172.21.15.96:0/3992423856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec96bc74d0 msgr2=0x55ec96f92d90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T14:14:45.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.161+0000 7fbf13fff640 1 --2- 172.21.15.96:0/3992423856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec96bc74d0 0x55ec96f92d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.161+0000 7fbf13fff640 1 -- 172.21.15.96:0/3992423856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec96bf0390 msgr2=0x55ec96f92810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:45.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.161+0000 7fbf13fff640 1 --2- 172.21.15.96:0/3992423856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec96bf0390 0x55ec96f92810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.161+0000 7fbf13fff640 1 -- 172.21.15.96:0/3992423856 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec96ebbc30 con 0x55ec96bcf6b0 2024-04-25T14:14:45.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.161+0000 7fbf13fff640 1 --2- 172.21.15.96:0/3992423856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec96bcf6b0 0x55ec96f93a20 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fbf00009c90 tx=0x7fbf00011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8fae578bbf5b8da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:45.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.161+0000 7fbefeffd640 1 -- 172.21.15.96:0/3992423856 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf00005220 con 0x55ec96bcf6b0 2024-04-25T14:14:45.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.161+0000 7fbefeffd640 1 -- 172.21.15.96:0/3992423856 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf00004c50 con 0x55ec96bcf6b0 2024-04-25T14:14:45.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.161+0000 7fbf11ffb640 1 --2- 172.21.15.96:0/2101538966 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fbf040afaf0 0x7fbf040b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.161+0000 7fbf11ffb640 1 -- 172.21.15.96:0/2101538966 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fbf080ff430 con 0x7fbf14101e10 2024-04-25T14:14:45.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.162+0000 7fbf1976b640 1 --2- 172.21.15.96:0/2101538966 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fbf040afaf0 0x7fbf040b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.162+0000 7fbefeffd640 1 -- 172.21.15.96:0/3992423856 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf000196f0 con 0x55ec96bcf6b0 2024-04-25T14:14:45.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.162+0000 7fbf2207f740 1 -- 172.21.15.96:0/3992423856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec96bcf6b0 msgr2=0x55ec96f93a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:45.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.162+0000 7fbf1976b640 1 --2- 172.21.15.96:0/2101538966 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fbf040afaf0 0x7fbf040b2000 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55ec96f93860 tx=0x7fbf0c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:45.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.162+0000 7fbf2207f740 1 --2- 172.21.15.96:0/3992423856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec96bcf6b0 0x55ec96f93a20 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fbf00009c90 tx=0x7fbf00011810 comp rx=0 tx=0).stop 2024-04-25T14:14:45.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.162+0000 7fbf2207f740 1 -- 172.21.15.96:0/3992423856 shutdown_connections 2024-04-25T14:14:45.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.162+0000 7fbf2207f740 1 --2- 172.21.15.96:0/3992423856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec96bcf6b0 0x55ec96f93a20 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.162+0000 7fbf2207f740 1 --2- 172.21.15.96:0/3992423856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec96bc74d0 0x55ec96f92d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.162+0000 7fbf2207f740 1 --2- 172.21.15.96:0/3992423856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec96bf0390 0x55ec96f92810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.162+0000 7fbf2207f740 1 -- 172.21.15.96:0/3992423856 >> 172.21.15.96:0/3992423856 conn(0x55ec96be44e0 msgr2=0x55ec96f8e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:45.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.163+0000 7fbf2207f740 1 -- 172.21.15.96:0/3992423856 shutdown_connections 2024-04-25T14:14:45.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.163+0000 7fbf2207f740 1 -- 172.21.15.96:0/3992423856 wait complete. 2024-04-25T14:14:45.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.163+0000 7fbf2207f740 1 -- start start 2024-04-25T14:14:45.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.163+0000 7fbf2207f740 1 Processor -- start 2024-04-25T14:14:45.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.164+0000 7fbf2207f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec96bc74d0 0x55ec96f9c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.164+0000 7fbf2207f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec96bcf6b0 0x55ec96f9caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.164+0000 7fbf2207f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec96bf0390 0x55ec96fa3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.164+0000 7fbf2207f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ec96f945d0 con 0x55ec96bf0390 2024-04-25T14:14:45.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.164+0000 7fbf2207f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ec96e7ddc0 con 0x55ec96bc74d0 2024-04-25T14:14:45.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.164+0000 7fbf2207f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ec96f94750 con 0x55ec96bcf6b0 2024-04-25T14:14:45.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.164+0000 7fbf1976b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec96bf0390 0x55ec96fa3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.164+0000 7fbf1976b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec96bf0390 0x55ec96fa3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34678/0 (socket says 172.21.15.96:34678) 2024-04-25T14:14:45.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.164+0000 7fbf1976b640 1 -- 172.21.15.96:0/2211039840 learned_addr learned my addr 172.21.15.96:0/2211039840 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:45.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.164+0000 7fbf18f6a640 1 --2- 172.21.15.96:0/2211039840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec96bc74d0 0x55ec96f9c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.164+0000 7fbf2207f740 10 client.0 _ll_register_callbacks cb 0x55ec96e99600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:14:45.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.164+0000 7fbf13fff640 1 --2- 172.21.15.96:0/2211039840 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec96bcf6b0 0x55ec96f9caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.164+0000 7fbf1976b640 1 -- 172.21.15.96:0/2211039840 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec96bcf6b0 msgr2=0x55ec96f9caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:45.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.164+0000 7fbf1976b640 1 --2- 172.21.15.96:0/2211039840 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec96bcf6b0 0x55ec96f9caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.164+0000 7fbf1976b640 1 -- 172.21.15.96:0/2211039840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec96bc74d0 msgr2=0x55ec96f9c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:45.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.164+0000 7fbf1976b640 1 --2- 172.21.15.96:0/2211039840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec96bc74d0 0x55ec96f9c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.164+0000 7fbf1976b640 1 -- 172.21.15.96:0/2211039840 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec96c083f0 con 0x55ec96bf0390 2024-04-25T14:14:45.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.164+0000 7fbf18f6a640 1 --2- 172.21.15.96:0/2211039840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec96bc74d0 0x55ec96f9c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:14:45.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.165+0000 7fbf1976b640 1 --2- 172.21.15.96:0/2211039840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec96bf0390 0x55ec96fa3a50 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c005c40 tx=0x7fbf0c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df1d9e8ca6b5711c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:45.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.165+0000 7fbef3fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:14:45.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.165+0000 7fbef3fff640 1 -- 172.21.15.96:0/2211039840 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c00eb90 con 0x55ec96bf0390 2024-04-25T14:14:45.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.165+0000 7fbf2207f740 1 -- 172.21.15.96:0/2211039840 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf00004c50 con 0x55ec96bf0390 2024-04-25T14:14:45.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.165+0000 7fbef3fff640 1 -- 172.21.15.96:0/2211039840 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fbf0c00ed30 con 0x55ec96bf0390 2024-04-25T14:14:45.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.165+0000 7fbef3fff640 1 -- 172.21.15.96:0/2211039840 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbf0c036070 con 0x55ec96bf0390 2024-04-25T14:14:45.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.165+0000 7fbf2207f740 10 client.15945 fetch_fsmap learned FSMap version 26 2024-04-25T14:14:45.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.165+0000 7fbf2207f740 1 -- 172.21.15.96:0/2211039840 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ec96cd27d0 con 0x55ec96bf0390 2024-04-25T14:14:45.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.165+0000 7fbef3fff640 1 -- 172.21.15.96:0/2211039840 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf00004c50 con 0x55ec96bf0390 2024-04-25T14:14:45.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.166+0000 7fbef3fff640 1 -- 172.21.15.96:0/2211039840 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbf0c026510 con 0x55ec96bf0390 2024-04-25T14:14:45.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.166+0000 7fbf2207f740 10 client.15945 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:14:45.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.166+0000 7fbf2207f740 10 client.15945 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:14:45.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.166+0000 7fbf2207f740 20 client.15945 populate_metadata read hostname 'smithi096' 2024-04-25T14:14:45.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.166+0000 7fbf2207f740 1 --2- 172.21.15.96:0/2211039840 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x55ec96baef60 0x55ec96faa340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.166+0000 7fbf2207f740 4 client.15945 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:14:45.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.166+0000 7fbf2207f740 1 -- 172.21.15.96:0/2211039840 --> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec96f04990 con 0x55ec96baef60 2024-04-25T14:14:45.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.167+0000 7fbf18f6a640 1 --2- 172.21.15.96:0/2211039840 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x55ec96baef60 0x55ec96faa340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.167+0000 7fbf18f6a640 1 --2- 172.21.15.96:0/2211039840 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x55ec96baef60 0x55ec96faa340 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=629e42047d891034 server_cookie=463673d5a9050ad5 in_seq=0 out_seq=0 2024-04-25T14:14:45.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.167+0000 7fbef3fff640 10 client.15945 ms_handle_connect on v2:172.21.15.151:6834/1313823638 2024-04-25T14:14:45.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.168+0000 7fbef3fff640 1 -- 172.21.15.96:0/2211039840 <== mds.2 v2:172.21.15.151:6834/1313823638 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ec96f04990 con 0x55ec96baef60 2024-04-25T14:14:45.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.168+0000 7fbef3fff640 10 client.15945 handle_command_reply: tid=0 2024-04-25T14:14:45.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.168+0000 7fbf2207f740 1 client.15945 shutdown 2024-04-25T14:14:45.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.168+0000 7fbf2207f740 1 -- 172.21.15.96:0/2211039840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec96bf0390 msgr2=0x55ec96fa3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:45.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.168+0000 7fbf2207f740 1 --2- 172.21.15.96:0/2211039840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec96bf0390 0x55ec96fa3a50 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c005c40 tx=0x7fbf0c004ab0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.168+0000 7fbf2207f740 1 -- 172.21.15.96:0/2211039840 shutdown_connections 2024-04-25T14:14:45.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.168+0000 7fbf2207f740 1 --2- 172.21.15.96:0/2211039840 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x55ec96baef60 0x55ec96faa340 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.168+0000 7fbf2207f740 1 --2- 172.21.15.96:0/2211039840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec96bf0390 0x55ec96fa3a50 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.168+0000 7fbef3fff640 0 client.15945 ms_handle_reset on v2:172.21.15.151:6834/1313823638 2024-04-25T14:14:45.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.168+0000 7fbf2207f740 1 --2- 172.21.15.96:0/2211039840 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec96bcf6b0 0x55ec96f9caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.168+0000 7fbf2207f740 1 --2- 172.21.15.96:0/2211039840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec96bc74d0 0x55ec96f9c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.168+0000 7fbf2207f740 1 -- 172.21.15.96:0/2211039840 >> 172.21.15.96:0/2211039840 conn(0x55ec96be44e0 msgr2=0x55ec96f8d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:45.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.168+0000 7fbf2207f740 1 -- 172.21.15.96:0/2211039840 shutdown_connections 2024-04-25T14:14:45.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.168+0000 7fbf2207f740 1 -- 172.21.15.96:0/2211039840 wait complete. 2024-04-25T14:14:45.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.169+0000 7fbf2207f740 20 client.15945 trim_cache size 0 max 16384 2024-04-25T14:14:45.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.182+0000 7fbf2207f740 1 Processor -- start 2024-04-25T14:14:45.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.182+0000 7fbf2207f740 1 -- start start 2024-04-25T14:14:45.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.182+0000 7fbf2207f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec96bc74d0 0x55ec96faa270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.182+0000 7fbf2207f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec96bcf6b0 0x55ec96f9adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.182+0000 7fbf2207f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec96bf0390 0x55ec96f9b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.182+0000 7fbf2207f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbf141a9100 con 0x55ec96bcf6b0 2024-04-25T14:14:45.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.182+0000 7fbf2207f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbf141aac40 con 0x55ec96bf0390 2024-04-25T14:14:45.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.182+0000 7fbf2207f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbf141a97d0 con 0x55ec96bc74d0 2024-04-25T14:14:45.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.183+0000 7fbf13fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec96bcf6b0 0x55ec96f9adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.183+0000 7fbf13fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec96bcf6b0 0x55ec96f9adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34684/0 (socket says 172.21.15.96:34684) 2024-04-25T14:14:45.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.183+0000 7fbf13fff640 1 -- 172.21.15.96:0/2385382340 learned_addr learned my addr 172.21.15.96:0/2385382340 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:45.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.183+0000 7fbf1976b640 1 --2- 172.21.15.96:0/2385382340 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec96bf0390 0x55ec96f9b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.183+0000 7fbf18f6a640 1 --2- 172.21.15.96:0/2385382340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec96bc74d0 0x55ec96faa270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.183+0000 7fbf13fff640 1 -- 172.21.15.96:0/2385382340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec96bc74d0 msgr2=0x55ec96faa270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:45.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.183+0000 7fbf13fff640 1 --2- 172.21.15.96:0/2385382340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec96bc74d0 0x55ec96faa270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.183+0000 7fbf13fff640 1 -- 172.21.15.96:0/2385382340 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec96bf0390 msgr2=0x55ec96f9b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:45.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.183+0000 7fbf13fff640 1 --2- 172.21.15.96:0/2385382340 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec96bf0390 0x55ec96f9b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.183+0000 7fbf13fff640 1 -- 172.21.15.96:0/2385382340 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec96f9a5d0 con 0x55ec96bcf6b0 2024-04-25T14:14:45.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.183+0000 7fbf1976b640 1 --2- 172.21.15.96:0/2385382340 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec96bf0390 0x55ec96f9b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:14:45.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.183+0000 7fbf13fff640 1 --2- 172.21.15.96:0/2385382340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec96bcf6b0 0x55ec96f9adb0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fbf00009be0 tx=0x7fbf00009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24c307f8ed7561fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:45.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.183+0000 7fbefe7fc640 1 -- 172.21.15.96:0/2385382340 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf000052d0 con 0x55ec96bcf6b0 2024-04-25T14:14:45.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.184+0000 7fbefe7fc640 1 -- 172.21.15.96:0/2385382340 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf00005470 con 0x55ec96bcf6b0 2024-04-25T14:14:45.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.184+0000 7fbefe7fc640 1 -- 172.21.15.96:0/2385382340 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf000197b0 con 0x55ec96bcf6b0 2024-04-25T14:14:45.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.185+0000 7fbf2207f740 1 -- 172.21.15.96:0/2385382340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec96bcf6b0 msgr2=0x55ec96f9adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:45.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.185+0000 7fbf2207f740 1 --2- 172.21.15.96:0/2385382340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec96bcf6b0 0x55ec96f9adb0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fbf00009be0 tx=0x7fbf00009f00 comp rx=0 tx=0).stop 2024-04-25T14:14:45.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.185+0000 7fbf2207f740 1 -- 172.21.15.96:0/2385382340 shutdown_connections 2024-04-25T14:14:45.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.185+0000 7fbf2207f740 1 --2- 172.21.15.96:0/2385382340 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec96bf0390 0x55ec96f9b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.185+0000 7fbf2207f740 1 --2- 172.21.15.96:0/2385382340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec96bcf6b0 0x55ec96f9adb0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.185+0000 7fbf2207f740 1 --2- 172.21.15.96:0/2385382340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec96bc74d0 0x55ec96faa270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.185+0000 7fbf2207f740 1 -- 172.21.15.96:0/2385382340 >> 172.21.15.96:0/2385382340 conn(0x55ec96be44e0 msgr2=0x55ec96f97210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:45.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.185+0000 7fbf2207f740 1 -- 172.21.15.96:0/2385382340 shutdown_connections 2024-04-25T14:14:45.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.185+0000 7fbf2207f740 1 -- 172.21.15.96:0/2385382340 wait complete. 2024-04-25T14:14:45.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.185+0000 7fbf2207f740 1 -- start start 2024-04-25T14:14:45.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.185+0000 7fbf2207f740 1 Processor -- start 2024-04-25T14:14:45.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.186+0000 7fbf2207f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec96bc74d0 0x55ec96f9ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.186+0000 7fbf2207f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec96bcf6b0 0x55ec96fa04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.186+0000 7fbf2207f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec96bf0390 0x55ec970106a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.186+0000 7fbf2207f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbf141aa570 con 0x55ec96bcf6b0 2024-04-25T14:14:45.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.186+0000 7fbf13fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec96bcf6b0 0x55ec96fa04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.186+0000 7fbf2207f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbf141a9ea0 con 0x55ec96bf0390 2024-04-25T14:14:45.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.186+0000 7fbf2207f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbf141a70e0 con 0x55ec96bc74d0 2024-04-25T14:14:45.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.186+0000 7fbf13fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec96bcf6b0 0x55ec96fa04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34688/0 (socket says 172.21.15.96:34688) 2024-04-25T14:14:45.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.186+0000 7fbf13fff640 1 -- 172.21.15.96:0/3139765402 learned_addr learned my addr 172.21.15.96:0/3139765402 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:45.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.186+0000 7fbf1976b640 1 --2- 172.21.15.96:0/3139765402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec96bf0390 0x55ec970106a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.186+0000 7fbf18f6a640 1 --2- 172.21.15.96:0/3139765402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec96bc74d0 0x55ec96f9ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.187+0000 7fbf2207f740 10 client.0 _ll_register_callbacks cb 0x7fbf0c009cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:14:45.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.187+0000 7fbf1976b640 1 -- 172.21.15.96:0/3139765402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec96bc74d0 msgr2=0x55ec96f9ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:45.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.187+0000 7fbf1976b640 1 --2- 172.21.15.96:0/3139765402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec96bc74d0 0x55ec96f9ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.187+0000 7fbf1976b640 1 -- 172.21.15.96:0/3139765402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec96bcf6b0 msgr2=0x55ec96fa04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:45.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.187+0000 7fbf1976b640 1 --2- 172.21.15.96:0/3139765402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec96bcf6b0 0x55ec96fa04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.187+0000 7fbf1976b640 1 -- 172.21.15.96:0/3139765402 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec97010be0 con 0x55ec96bf0390 2024-04-25T14:14:45.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.187+0000 7fbf18f6a640 1 --2- 172.21.15.96:0/3139765402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec96bc74d0 0x55ec96f9ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:14:45.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.187+0000 7fbf13fff640 1 --2- 172.21.15.96:0/3139765402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec96bcf6b0 0x55ec96fa04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:14:45.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.187+0000 7fbf1976b640 1 --2- 172.21.15.96:0/3139765402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec96bf0390 0x55ec970106a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c026980 tx=0x7fbf0c017ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd03195838e06c27 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:45.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.187+0000 7fbef3fff640 10 client.25454 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T14:14:45.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.187+0000 7fbf2207f740 1 -- 172.21.15.96:0/3139765402 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf00005470 con 0x55ec96bf0390 2024-04-25T14:14:45.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.187+0000 7fbef3fff640 1 -- 172.21.15.96:0/3139765402 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c04b060 con 0x55ec96bf0390 2024-04-25T14:14:45.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.187+0000 7fbef3fff640 1 -- 172.21.15.96:0/3139765402 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fbf0c00eb90 con 0x55ec96bf0390 2024-04-25T14:14:45.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.188+0000 7fbef3fff640 1 -- 172.21.15.96:0/3139765402 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbf0c00edd0 con 0x55ec96bf0390 2024-04-25T14:14:45.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.188+0000 7fbf2207f740 10 client.25454 fetch_fsmap learned FSMap version 26 2024-04-25T14:14:45.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.188+0000 7fbf2207f740 1 -- 172.21.15.96:0/3139765402 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({fsmap=26}) -- 0x55ec97011bf0 con 0x55ec96bf0390 2024-04-25T14:14:45.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.188+0000 7fbef3fff640 1 -- 172.21.15.96:0/3139765402 <== mon.2 v2:172.21.15.97:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbf0c026410 con 0x55ec96bf0390 2024-04-25T14:14:45.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.188+0000 7fbf2207f740 10 client.25454 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:14:45.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.188+0000 7fbf2207f740 10 client.25454 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:14:45.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.188+0000 7fbf2207f740 20 client.25454 populate_metadata read hostname 'smithi096' 2024-04-25T14:14:45.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.188+0000 7fbf2207f740 1 --2- 172.21.15.96:0/3139765402 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x55ec96b843a0 0x55ec97014420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.188+0000 7fbf2207f740 4 client.25454 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:14:45.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.189+0000 7fbf2207f740 1 -- 172.21.15.96:0/3139765402 --> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fbf00005470 con 0x55ec96b843a0 2024-04-25T14:14:45.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.189+0000 7fbf18f6a640 1 --2- 172.21.15.96:0/3139765402 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x55ec96b843a0 0x55ec97014420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.189+0000 7fbef3fff640 10 client.25454 ms_handle_connect on v2:172.21.15.151:6834/1313823638 2024-04-25T14:14:45.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.189+0000 7fbf18f6a640 1 --2- 172.21.15.96:0/3139765402 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x55ec96b843a0 0x55ec97014420 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f59f71dd175ecef7 server_cookie=2e1713362180e22f in_seq=0 out_seq=0 2024-04-25T14:14:45.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.190+0000 7fbef3fff640 1 -- 172.21.15.96:0/3139765402 <== mds.2 v2:172.21.15.151:6834/1313823638 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fbf00005470 con 0x55ec96b843a0 2024-04-25T14:14:45.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.190+0000 7fbef3fff640 10 client.25454 handle_command_reply: tid=0 2024-04-25T14:14:45.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.190+0000 7fbf2207f740 1 client.25454 shutdown 2024-04-25T14:14:45.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.190+0000 7fbf2207f740 1 -- 172.21.15.96:0/3139765402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec96bf0390 msgr2=0x55ec970106a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:45.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.190+0000 7fbf2207f740 1 --2- 172.21.15.96:0/3139765402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec96bf0390 0x55ec970106a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c026980 tx=0x7fbf0c017ce0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.190+0000 7fbf2207f740 1 -- 172.21.15.96:0/3139765402 shutdown_connections 2024-04-25T14:14:45.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.190+0000 7fbf2207f740 1 --2- 172.21.15.96:0/3139765402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec96bf0390 0x55ec970106a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.190+0000 7fbf2207f740 1 --2- 172.21.15.96:0/3139765402 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x55ec96b843a0 0x55ec97014420 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.190+0000 7fbef3fff640 0 client.25454 ms_handle_reset on v2:172.21.15.151:6834/1313823638 2024-04-25T14:14:45.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.190+0000 7fbf2207f740 1 --2- 172.21.15.96:0/3139765402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec96bcf6b0 0x55ec96fa04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.190+0000 7fbf2207f740 1 --2- 172.21.15.96:0/3139765402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec96bc74d0 0x55ec96f9ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.190+0000 7fbf2207f740 1 -- 172.21.15.96:0/3139765402 >> 172.21.15.96:0/3139765402 conn(0x55ec96be44e0 msgr2=0x55ec96f9adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:45.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.190+0000 7fbf2207f740 1 -- 172.21.15.96:0/3139765402 shutdown_connections 2024-04-25T14:14:45.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.190+0000 7fbf2207f740 1 -- 172.21.15.96:0/3139765402 wait complete. 2024-04-25T14:14:45.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.190+0000 7fbf2207f740 20 client.25454 trim_cache size 0 max 16384 2024-04-25T14:14:45.195 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-04-25T14:14:45.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.191+0000 7fbf1b1ce640 1 -- 172.21.15.96:0/2101538966 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fbf040afaf0 msgr2=0x7fbf040b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:45.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.191+0000 7fbf1b1ce640 1 --2- 172.21.15.96:0/2101538966 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fbf040afaf0 0x7fbf040b2000 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55ec96f93860 tx=0x7fbf0c009500 comp rx=0 tx=0).stop 2024-04-25T14:14:45.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.191+0000 7fbf1b1ce640 1 -- 172.21.15.96:0/2101538966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf14101e10 msgr2=0x7fbf1419a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:45.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.191+0000 7fbf1b1ce640 1 --2- 172.21.15.96:0/2101538966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf14101e10 0x7fbf1419a200 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fbf08031440 tx=0x7fbf08003c10 comp rx=0 tx=0).stop 2024-04-25T14:14:45.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.191+0000 7fbf1b1ce640 1 -- 172.21.15.96:0/2101538966 shutdown_connections 2024-04-25T14:14:45.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.191+0000 7fbf1b1ce640 1 --2- 172.21.15.96:0/2101538966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf14109780 0x7fbf141a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.191+0000 7fbf1b1ce640 1 --2- 172.21.15.96:0/2101538966 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf14106c10 0x7fbf1419a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.191+0000 7fbf1b1ce640 1 --2- 172.21.15.96:0/2101538966 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fbf040afaf0 0x7fbf040b2000 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.191+0000 7fbf1b1ce640 1 --2- 172.21.15.96:0/2101538966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf14101e10 0x7fbf1419a200 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.191+0000 7fbf1b1ce640 1 -- 172.21.15.96:0/2101538966 >> 172.21.15.96:0/2101538966 conn(0x7fbf140fd340 msgr2=0x7fbf14108ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:45.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.191+0000 7fbf1b1ce640 1 -- 172.21.15.96:0/2101538966 shutdown_connections 2024-04-25T14:14:45.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.191+0000 7fbf1b1ce640 1 -- 172.21.15.96:0/2101538966 wait complete. 2024-04-25T14:14:45.203 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T14:14:45.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.313+0000 7faab09ee640 1 -- 172.21.15.96:0/285513988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faaac1029e0 msgr2=0x7faaac1036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:45.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.313+0000 7faab09ee640 1 --2- 172.21.15.96:0/285513988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faaac1029e0 0x7faaac1036e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7faa9c009900 tx=0x7faa9c02a410 comp rx=0 tx=0).stop 2024-04-25T14:14:45.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.313+0000 7faab09ee640 1 -- 172.21.15.96:0/285513988 shutdown_connections 2024-04-25T14:14:45.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.313+0000 7faab09ee640 1 --2- 172.21.15.96:0/285513988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faaac108920 0x7faaac10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.313+0000 7faab09ee640 1 --2- 172.21.15.96:0/285513988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faaac103c20 0x7faaac104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.313+0000 7faab09ee640 1 --2- 172.21.15.96:0/285513988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faaac1029e0 0x7faaac1036e0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.313+0000 7faab09ee640 1 -- 172.21.15.96:0/285513988 >> 172.21.15.96:0/285513988 conn(0x7faaac0fdbd0 msgr2=0x7faaac100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:45.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.314+0000 7faab09ee640 1 -- 172.21.15.96:0/285513988 shutdown_connections 2024-04-25T14:14:45.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.314+0000 7faab09ee640 1 -- 172.21.15.96:0/285513988 wait complete. 2024-04-25T14:14:45.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.314+0000 7faab09ee640 1 Processor -- start 2024-04-25T14:14:45.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.314+0000 7faab09ee640 1 -- start start 2024-04-25T14:14:45.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.315+0000 7faab09ee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faaac1029e0 0x7faaac06a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.315+0000 7faab09ee640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faaac103c20 0x7faaac19f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.315+0000 7faab09ee640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faaac108920 0x7faaac1a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.315+0000 7faab09ee640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faaac10e4c0 con 0x7faaac1029e0 2024-04-25T14:14:45.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.315+0000 7faab09ee640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7faaac10e340 con 0x7faaac108920 2024-04-25T14:14:45.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.315+0000 7faab09ee640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faaac10e640 con 0x7faaac103c20 2024-04-25T14:14:45.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.315+0000 7faaaa59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faaac1029e0 0x7faaac06a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.315+0000 7faaaa59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faaac1029e0 0x7faaac06a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34702/0 (socket says 172.21.15.96:34702) 2024-04-25T14:14:45.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.315+0000 7faaaa59c640 1 -- 172.21.15.96:0/725582388 learned_addr learned my addr 172.21.15.96:0/725582388 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:45.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.315+0000 7faaa9d9b640 1 --2- 172.21.15.96:0/725582388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faaac103c20 0x7faaac19f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.315+0000 7faaaad9d640 1 --2- 172.21.15.96:0/725582388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faaac108920 0x7faaac1a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.315+0000 7faaaa59c640 1 -- 172.21.15.96:0/725582388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faaac103c20 msgr2=0x7faaac19f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:45.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.315+0000 7faaaa59c640 1 --2- 172.21.15.96:0/725582388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faaac103c20 0x7faaac19f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.315+0000 7faaaa59c640 1 -- 172.21.15.96:0/725582388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faaac108920 msgr2=0x7faaac1a1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:45.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.315+0000 7faaaa59c640 1 --2- 172.21.15.96:0/725582388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faaac108920 0x7faaac1a1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.315+0000 7faaaa59c640 1 -- 172.21.15.96:0/725582388 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaac1990a0 con 0x7faaac1029e0 2024-04-25T14:14:45.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.315+0000 7faaaa59c640 1 --2- 172.21.15.96:0/725582388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faaac1029e0 0x7faaac06a900 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7faa9c006010 tx=0x7faa9c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:45.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.315+0000 7faa9b7fe640 1 -- 172.21.15.96:0/725582388 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa9c03f030 con 0x7faaac1029e0 2024-04-25T14:14:45.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.316+0000 7faab09ee640 1 -- 172.21.15.96:0/725582388 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaac199330 con 0x7faaac1029e0 2024-04-25T14:14:45.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.316+0000 7faa9b7fe640 1 -- 172.21.15.96:0/725582388 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa9c003aa0 con 0x7faaac1029e0 2024-04-25T14:14:45.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.316+0000 7faa9b7fe640 1 -- 172.21.15.96:0/725582388 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa9c035070 con 0x7faaac1029e0 2024-04-25T14:14:45.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.316+0000 7faab09ee640 1 -- 172.21.15.96:0/725582388 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaac199870 con 0x7faaac1029e0 2024-04-25T14:14:45.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.318+0000 7faab7977740 1 Processor -- start 2024-04-25T14:14:45.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.319+0000 7faab7977740 1 -- start start 2024-04-25T14:14:45.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.319+0000 7faab7977740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d669a74390 0x55d669e1f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.319+0000 7faab7977740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d669a4b4d0 0x55d669e1f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.319+0000 7faab7977740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d669a536b0 0x55d669e20230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.319+0000 7faab7977740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d669d7ad80 con 0x55d669a4b4d0 2024-04-25T14:14:45.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.319+0000 7faab7977740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d669d34a20 con 0x55d669a536b0 2024-04-25T14:14:45.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.319+0000 7faab7977740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d669d76d40 con 0x55d669a74390 2024-04-25T14:14:45.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.319+0000 7faaa9d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d669a536b0 0x55d669e20230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.319+0000 7faaa9d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d669a536b0 0x55d669e20230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:54224/0 (socket says 172.21.15.96:54224) 2024-04-25T14:14:45.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.319+0000 7faaa9d9b640 1 -- 172.21.15.96:0/3804222121 learned_addr learned my addr 172.21.15.96:0/3804222121 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:45.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.319+0000 7faa9b7fe640 1 -- 172.21.15.96:0/725582388 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7faa9c042030 con 0x7faaac1029e0 2024-04-25T14:14:45.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.320+0000 7faaaa59c640 1 --2- 172.21.15.96:0/3804222121 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d669a4b4d0 0x55d669e1f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.320+0000 7faaa9d9b640 1 -- 172.21.15.96:0/3804222121 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d669a74390 msgr2=0x55d669e1f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:45.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.320+0000 7faaa9d9b640 1 --2- 172.21.15.96:0/3804222121 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d669a74390 0x55d669e1f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.320+0000 7faaa9d9b640 1 -- 172.21.15.96:0/3804222121 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d669a4b4d0 msgr2=0x55d669e1f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:45.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.320+0000 7faaa9d9b640 1 --2- 172.21.15.96:0/3804222121 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d669a4b4d0 0x55d669e1f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.320+0000 7faaa9d9b640 1 -- 172.21.15.96:0/3804222121 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d669d3fc30 con 0x55d669a536b0 2024-04-25T14:14:45.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.320+0000 7faaaa59c640 1 --2- 172.21.15.96:0/3804222121 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d669a4b4d0 0x55d669e1f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:14:45.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.320+0000 7faaa9d9b640 1 --2- 172.21.15.96:0/3804222121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d669a536b0 0x55d669e20230 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7faaa000bd10 tx=0x7faaa00107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3a90cc49069977d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:45.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.320+0000 7faa98ff9640 1 -- 172.21.15.96:0/3804222121 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa0005220 con 0x55d669a536b0 2024-04-25T14:14:45.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.320+0000 7faa98ff9640 1 -- 172.21.15.96:0/3804222121 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faaa0004c50 con 0x55d669a536b0 2024-04-25T14:14:45.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.320+0000 7faa98ff9640 1 -- 172.21.15.96:0/3804222121 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa0011800 con 0x55d669a536b0 2024-04-25T14:14:45.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.320+0000 7faa9b7fe640 1 --2- 172.21.15.96:0/725582388 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7faa940afaf0 0x7faa940b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.320+0000 7faa9b7fe640 1 -- 172.21.15.96:0/725582388 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7faa9c0ff000 con 0x7faaac1029e0 2024-04-25T14:14:45.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.321+0000 7faaaad9d640 1 --2- 172.21.15.96:0/725582388 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7faa940afaf0 0x7faa940b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.321+0000 7faab7977740 1 -- 172.21.15.96:0/3804222121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d669a536b0 msgr2=0x55d669e20230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:45.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.321+0000 7faab7977740 1 --2- 172.21.15.96:0/3804222121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d669a536b0 0x55d669e20230 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7faaa000bd10 tx=0x7faaa00107a0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.321+0000 7faaaad9d640 1 --2- 172.21.15.96:0/725582388 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7faa940afaf0 0x7faa940b2000 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x55d669e20070 tx=0x7faaa4009a80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:45.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.321+0000 7faab7977740 1 -- 172.21.15.96:0/3804222121 shutdown_connections 2024-04-25T14:14:45.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.321+0000 7faab7977740 1 --2- 172.21.15.96:0/3804222121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d669a536b0 0x55d669e20230 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.321+0000 7faab7977740 1 --2- 172.21.15.96:0/3804222121 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d669a4b4d0 0x55d669e1f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.321+0000 7faab7977740 1 --2- 172.21.15.96:0/3804222121 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d669a74390 0x55d669e1f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.321+0000 7faab7977740 1 -- 172.21.15.96:0/3804222121 >> 172.21.15.96:0/3804222121 conn(0x55d669a684e0 msgr2=0x55d669e1a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:45.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.321+0000 7faab7977740 1 -- 172.21.15.96:0/3804222121 shutdown_connections 2024-04-25T14:14:45.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.321+0000 7faab7977740 1 -- 172.21.15.96:0/3804222121 wait complete. 2024-04-25T14:14:45.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.322+0000 7faab7977740 1 -- start start 2024-04-25T14:14:45.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.322+0000 7faab7977740 1 Processor -- start 2024-04-25T14:14:45.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.322+0000 7faab7977740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d669a4b4d0 0x55d669e133f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.322+0000 7faab7977740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d669a536b0 0x55d669e13930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.322+0000 7faab7977740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d669a74390 0x55d669e18830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.322+0000 7faab7977740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d669e20d40 con 0x55d669a536b0 2024-04-25T14:14:45.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.322+0000 7faab7977740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d669d01dc0 con 0x55d669a4b4d0 2024-04-25T14:14:45.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.322+0000 7faab7977740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d669e20ec0 con 0x55d669a74390 2024-04-25T14:14:45.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.322+0000 7faaa9d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d669a536b0 0x55d669e13930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.322+0000 7faaa9d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d669a536b0 0x55d669e13930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34724/0 (socket says 172.21.15.96:34724) 2024-04-25T14:14:45.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.322+0000 7faaa9d9b640 1 -- 172.21.15.96:0/2947236227 learned_addr learned my addr 172.21.15.96:0/2947236227 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:45.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.322+0000 7faab7977740 10 client.0 _ll_register_callbacks cb 0x55d669d1d600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:14:45.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.322+0000 7faaaa59c640 1 --2- 172.21.15.96:0/2947236227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d669a4b4d0 0x55d669e133f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.322+0000 7faaaad9d640 1 --2- 172.21.15.96:0/2947236227 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d669a74390 0x55d669e18830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.322+0000 7faaa9d9b640 1 -- 172.21.15.96:0/2947236227 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d669a74390 msgr2=0x55d669e18830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:45.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.322+0000 7faaa9d9b640 1 --2- 172.21.15.96:0/2947236227 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d669a74390 0x55d669e18830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.323+0000 7faaa9d9b640 1 -- 172.21.15.96:0/2947236227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d669a4b4d0 msgr2=0x55d669e133f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:45.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.323+0000 7faaa9d9b640 1 --2- 172.21.15.96:0/2947236227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d669a4b4d0 0x55d669e133f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.323+0000 7faaa9d9b640 1 -- 172.21.15.96:0/2947236227 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d669a8c3f0 con 0x55d669a536b0 2024-04-25T14:14:45.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.323+0000 7faaaa59c640 1 --2- 172.21.15.96:0/2947236227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d669a4b4d0 0x55d669e133f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:14:45.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.323+0000 7faaa9d9b640 1 --2- 172.21.15.96:0/2947236227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d669a536b0 0x55d669e13930 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7faaa000bce0 tx=0x7faaa00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5286098ff1a4cedf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:45.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.323+0000 7faa91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:14:45.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.323+0000 7faa91ffb640 1 -- 172.21.15.96:0/2947236227 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa002a060 con 0x55d669a536b0 2024-04-25T14:14:45.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.323+0000 7faa91ffb640 1 -- 172.21.15.96:0/2947236227 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7faaa0005010 con 0x55d669a536b0 2024-04-25T14:14:45.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.323+0000 7faab7977740 1 -- 172.21.15.96:0/2947236227 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faaa0004c50 con 0x55d669a536b0 2024-04-25T14:14:45.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.323+0000 7faa91ffb640 1 -- 172.21.15.96:0/2947236227 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faaa0005210 con 0x55d669a536b0 2024-04-25T14:14:45.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.323+0000 7faab7977740 10 client.15960 fetch_fsmap learned FSMap version 26 2024-04-25T14:14:45.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.323+0000 7faab7977740 1 -- 172.21.15.96:0/2947236227 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d669b567d0 con 0x55d669a536b0 2024-04-25T14:14:45.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.323+0000 7faa91ffb640 1 -- 172.21.15.96:0/2947236227 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faaa0004c50 con 0x55d669a536b0 2024-04-25T14:14:45.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.324+0000 7faa91ffb640 1 -- 172.21.15.96:0/2947236227 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7faaa001b4c0 con 0x55d669a536b0 2024-04-25T14:14:45.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.324+0000 7faab7977740 10 client.15960 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:14:45.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.324+0000 7faab7977740 10 client.15960 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:14:45.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.324+0000 7faab7977740 20 client.15960 populate_metadata read hostname 'smithi096' 2024-04-25T14:14:45.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.324+0000 7faab7977740 1 --2- 172.21.15.96:0/2947236227 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x55d669a32f60 0x55d669e3ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.324+0000 7faab7977740 4 client.15960 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:14:45.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.324+0000 7faab7977740 1 -- 172.21.15.96:0/2947236227 --> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d669d88990 con 0x55d669a32f60 2024-04-25T14:14:45.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.325+0000 7faaaa59c640 1 --2- 172.21.15.96:0/2947236227 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x55d669a32f60 0x55d669e3ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.325+0000 7faaaa59c640 1 --2- 172.21.15.96:0/2947236227 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x55d669a32f60 0x55d669e3ef10 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=27189f68e6b17ac1 server_cookie=b6b6b8b9515b1abb in_seq=0 out_seq=0 2024-04-25T14:14:45.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.325+0000 7faa91ffb640 10 client.15960 ms_handle_connect on v2:172.21.15.151:6838/245699981 2024-04-25T14:14:45.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.326+0000 7faa91ffb640 1 -- 172.21.15.96:0/2947236227 <== mds.1 v2:172.21.15.151:6838/245699981 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d669d88990 con 0x55d669a32f60 2024-04-25T14:14:45.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.326+0000 7faa91ffb640 10 client.15960 handle_command_reply: tid=0 2024-04-25T14:14:45.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.326+0000 7faab7977740 1 client.15960 shutdown 2024-04-25T14:14:45.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.326+0000 7faab7977740 1 -- 172.21.15.96:0/2947236227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d669a536b0 msgr2=0x55d669e13930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:45.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.326+0000 7faab7977740 1 --2- 172.21.15.96:0/2947236227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d669a536b0 0x55d669e13930 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7faaa000bce0 tx=0x7faaa00048f0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.326+0000 7faab7977740 1 -- 172.21.15.96:0/2947236227 shutdown_connections 2024-04-25T14:14:45.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.326+0000 7faab7977740 1 --2- 172.21.15.96:0/2947236227 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x55d669a32f60 0x55d669e3ef10 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.326+0000 7faab7977740 1 --2- 172.21.15.96:0/2947236227 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d669a74390 0x55d669e18830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.326+0000 7faa91ffb640 0 client.15960 ms_handle_reset on v2:172.21.15.151:6838/245699981 2024-04-25T14:14:45.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.326+0000 7faab7977740 1 --2- 172.21.15.96:0/2947236227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d669a536b0 0x55d669e13930 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.326+0000 7faab7977740 1 --2- 172.21.15.96:0/2947236227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d669a4b4d0 0x55d669e133f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.326+0000 7faab7977740 1 -- 172.21.15.96:0/2947236227 >> 172.21.15.96:0/2947236227 conn(0x55d669a684e0 msgr2=0x55d669e11040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:45.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.326+0000 7faab7977740 1 -- 172.21.15.96:0/2947236227 shutdown_connections 2024-04-25T14:14:45.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.326+0000 7faab7977740 1 -- 172.21.15.96:0/2947236227 wait complete. 2024-04-25T14:14:45.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.326+0000 7faab7977740 20 client.15960 trim_cache size 0 max 16384 2024-04-25T14:14:45.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.340+0000 7faab7977740 1 Processor -- start 2024-04-25T14:14:45.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.340+0000 7faab7977740 1 -- start start 2024-04-25T14:14:45.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.340+0000 7faab7977740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d669a4b4d0 0x55d669e23ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.340+0000 7faab7977740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d669a74390 0x55d669e240e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.340+0000 7faab7977740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d669a083a0 0x55d669e11a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.340+0000 7faab7977740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faaac1a9170 con 0x55d669a083a0 2024-04-25T14:14:45.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.340+0000 7faab7977740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7faaac1aacb0 con 0x55d669a74390 2024-04-25T14:14:45.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.340+0000 7faab7977740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faaac1a9840 con 0x55d669a4b4d0 2024-04-25T14:14:45.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.340+0000 7faaaad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d669a083a0 0x55d669e11a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.340+0000 7faaaad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d669a083a0 0x55d669e11a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34736/0 (socket says 172.21.15.96:34736) 2024-04-25T14:14:45.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.340+0000 7faaaad9d640 1 -- 172.21.15.96:0/1284134739 learned_addr learned my addr 172.21.15.96:0/1284134739 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:45.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.340+0000 7faaaa59c640 1 --2- 172.21.15.96:0/1284134739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d669a4b4d0 0x55d669e23ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.340+0000 7faaa9d9b640 1 --2- 172.21.15.96:0/1284134739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d669a74390 0x55d669e240e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.340+0000 7faaaad9d640 1 -- 172.21.15.96:0/1284134739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d669a4b4d0 msgr2=0x55d669e23ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:45.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.340+0000 7faaaad9d640 1 --2- 172.21.15.96:0/1284134739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d669a4b4d0 0x55d669e23ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.340+0000 7faaaad9d640 1 -- 172.21.15.96:0/1284134739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d669a74390 msgr2=0x55d669e240e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:45.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.341+0000 7faaaad9d640 1 --2- 172.21.15.96:0/1284134739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d669a74390 0x55d669e240e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.341+0000 7faaaad9d640 1 -- 172.21.15.96:0/1284134739 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d669e26f60 con 0x55d669a083a0 2024-04-25T14:14:45.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.341+0000 7faaa9d9b640 1 --2- 172.21.15.96:0/1284134739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d669a74390 0x55d669e240e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:14:45.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.341+0000 7faaaad9d640 1 --2- 172.21.15.96:0/1284134739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d669a083a0 0x55d669e11a20 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7faaa4011790 tx=0x7faaa4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca5b9c20df636499 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:45.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.341+0000 7faa93fff640 1 -- 172.21.15.96:0/1284134739 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa401e8c0 con 0x55d669a083a0 2024-04-25T14:14:45.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.341+0000 7faa93fff640 1 -- 172.21.15.96:0/1284134739 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faaa401f430 con 0x55d669a083a0 2024-04-25T14:14:45.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.341+0000 7faa93fff640 1 -- 172.21.15.96:0/1284134739 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa4027630 con 0x55d669a083a0 2024-04-25T14:14:45.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.342+0000 7faab7977740 1 -- 172.21.15.96:0/1284134739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d669a083a0 msgr2=0x55d669e11a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:45.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.342+0000 7faab7977740 1 --2- 172.21.15.96:0/1284134739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d669a083a0 0x55d669e11a20 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7faaa4011790 tx=0x7faaa4013000 comp rx=0 tx=0).stop 2024-04-25T14:14:45.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.342+0000 7faab7977740 1 -- 172.21.15.96:0/1284134739 shutdown_connections 2024-04-25T14:14:45.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.342+0000 7faab7977740 1 --2- 172.21.15.96:0/1284134739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d669a083a0 0x55d669e11a20 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.342+0000 7faab7977740 1 --2- 172.21.15.96:0/1284134739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d669a74390 0x55d669e240e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.342+0000 7faab7977740 1 --2- 172.21.15.96:0/1284134739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d669a4b4d0 0x55d669e23ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.342+0000 7faab7977740 1 -- 172.21.15.96:0/1284134739 >> 172.21.15.96:0/1284134739 conn(0x55d669a684e0 msgr2=0x55d669e1efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:45.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.342+0000 7faab7977740 1 -- 172.21.15.96:0/1284134739 shutdown_connections 2024-04-25T14:14:45.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.342+0000 7faab7977740 1 -- 172.21.15.96:0/1284134739 wait complete. 2024-04-25T14:14:45.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.342+0000 7faab7977740 1 -- start start 2024-04-25T14:14:45.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.343+0000 7faab7977740 1 Processor -- start 2024-04-25T14:14:45.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.343+0000 7faab7977740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d669a083a0 0x55d669e4dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.343+0000 7faab7977740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d669a4b4d0 0x55d669e4e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.343+0000 7faab7977740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d669a74390 0x55d669e4f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.343+0000 7faab7977740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faaac1aa5e0 con 0x55d669a4b4d0 2024-04-25T14:14:45.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.343+0000 7faab7977740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7faaac1a9f10 con 0x55d669a74390 2024-04-25T14:14:45.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.343+0000 7faab7977740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faaac1a7150 con 0x55d669a083a0 2024-04-25T14:14:45.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.343+0000 7faab7977740 10 client.0 _ll_register_callbacks cb 0x7faaa000bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:14:45.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.343+0000 7faaa9d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d669a4b4d0 0x55d669e4e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.343+0000 7faaa9d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d669a4b4d0 0x55d669e4e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34752/0 (socket says 172.21.15.96:34752) 2024-04-25T14:14:45.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.343+0000 7faaa9d9b640 1 -- 172.21.15.96:0/1391304586 learned_addr learned my addr 172.21.15.96:0/1391304586 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:45.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.343+0000 7faaaad9d640 1 --2- 172.21.15.96:0/1391304586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d669a74390 0x55d669e4f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.343+0000 7faaaa59c640 1 --2- 172.21.15.96:0/1391304586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d669a083a0 0x55d669e4dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.343+0000 7faaa9d9b640 1 -- 172.21.15.96:0/1391304586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d669a083a0 msgr2=0x55d669e4dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:45.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.343+0000 7faaa9d9b640 1 --2- 172.21.15.96:0/1391304586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d669a083a0 0x55d669e4dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.343+0000 7faaa9d9b640 1 -- 172.21.15.96:0/1391304586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d669a74390 msgr2=0x55d669e4f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:45.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.343+0000 7faaa9d9b640 1 --2- 172.21.15.96:0/1391304586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d669a74390 0x55d669e4f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.343+0000 7faaa9d9b640 1 -- 172.21.15.96:0/1391304586 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d669e4f650 con 0x55d669a4b4d0 2024-04-25T14:14:45.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.343+0000 7faaaa59c640 1 --2- 172.21.15.96:0/1391304586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d669a083a0 0x55d669e4dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:14:45.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.344+0000 7faaa9d9b640 1 --2- 172.21.15.96:0/1391304586 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d669a4b4d0 0x55d669e4e160 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7faaa000bc30 tx=0x7faaa000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaf359837128e70e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:45.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.344+0000 7faa91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:14:45.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.344+0000 7faa91ffb640 1 -- 172.21.15.96:0/1391304586 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa002a070 con 0x55d669a4b4d0 2024-04-25T14:14:45.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.344+0000 7faa91ffb640 1 -- 172.21.15.96:0/1391304586 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7faaa001b640 con 0x55d669a4b4d0 2024-04-25T14:14:45.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.344+0000 7faab7977740 1 -- 172.21.15.96:0/1391304586 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faaa401f430 con 0x55d669a4b4d0 2024-04-25T14:14:45.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.344+0000 7faa91ffb640 1 -- 172.21.15.96:0/1391304586 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faaa00049e0 con 0x55d669a4b4d0 2024-04-25T14:14:45.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.344+0000 7faab7977740 10 client.15966 fetch_fsmap learned FSMap version 26 2024-04-25T14:14:45.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.344+0000 7faab7977740 1 -- 172.21.15.96:0/1391304586 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d669e50940 con 0x55d669a4b4d0 2024-04-25T14:14:45.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.344+0000 7faa91ffb640 1 -- 172.21.15.96:0/1391304586 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faaa401f430 con 0x55d669a4b4d0 2024-04-25T14:14:45.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.344+0000 7faa91ffb640 1 -- 172.21.15.96:0/1391304586 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7faaa00035f0 con 0x55d669a4b4d0 2024-04-25T14:14:45.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.344+0000 7faab7977740 10 client.15966 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:14:45.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.344+0000 7faab7977740 10 client.15966 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:14:45.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.345+0000 7faab7977740 20 client.15966 populate_metadata read hostname 'smithi096' 2024-04-25T14:14:45.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.345+0000 7faab7977740 1 --2- 172.21.15.96:0/1391304586 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x55d669ea75a0 0x55d669ea99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.345+0000 7faab7977740 4 client.15966 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:14:45.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.345+0000 7faab7977740 1 -- 172.21.15.96:0/1391304586 --> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d669d82800 con 0x55d669ea75a0 2024-04-25T14:14:45.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.345+0000 7faaaa59c640 1 --2- 172.21.15.96:0/1391304586 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x55d669ea75a0 0x55d669ea99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.345+0000 7faa91ffb640 10 client.15966 ms_handle_connect on v2:172.21.15.151:6838/245699981 2024-04-25T14:14:45.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.345+0000 7faaaa59c640 1 --2- 172.21.15.96:0/1391304586 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x55d669ea75a0 0x55d669ea99c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=14aea68b59b7eaf server_cookie=e87a5279cb94479 in_seq=0 out_seq=0 2024-04-25T14:14:45.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.345+0000 7faa91ffb640 1 -- 172.21.15.96:0/1391304586 <== mds.1 v2:172.21.15.151:6838/245699981 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d669d82800 con 0x55d669ea75a0 2024-04-25T14:14:45.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.345+0000 7faa91ffb640 10 client.15966 handle_command_reply: tid=0 2024-04-25T14:14:45.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.346+0000 7faab7977740 1 client.15966 shutdown 2024-04-25T14:14:45.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.346+0000 7faab7977740 1 -- 172.21.15.96:0/1391304586 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d669a4b4d0 msgr2=0x55d669e4e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:45.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.346+0000 7faab7977740 1 --2- 172.21.15.96:0/1391304586 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d669a4b4d0 0x55d669e4e160 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7faaa000bc30 tx=0x7faaa000b960 comp rx=0 tx=0).stop 2024-04-25T14:14:45.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.346+0000 7faab7977740 1 -- 172.21.15.96:0/1391304586 shutdown_connections 2024-04-25T14:14:45.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.346+0000 7faab7977740 1 --2- 172.21.15.96:0/1391304586 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x55d669ea75a0 0x55d669ea99c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.346+0000 7faab7977740 1 --2- 172.21.15.96:0/1391304586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d669a74390 0x55d669e4f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.346+0000 7faab7977740 1 --2- 172.21.15.96:0/1391304586 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d669a4b4d0 0x55d669e4e160 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.346+0000 7faa91ffb640 0 client.15966 ms_handle_reset on v2:172.21.15.151:6838/245699981 2024-04-25T14:14:45.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.346+0000 7faab7977740 1 --2- 172.21.15.96:0/1391304586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d669a083a0 0x55d669e4dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.346+0000 7faab7977740 1 -- 172.21.15.96:0/1391304586 >> 172.21.15.96:0/1391304586 conn(0x55d669a684e0 msgr2=0x55d669e23ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:45.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.346+0000 7faab7977740 1 -- 172.21.15.96:0/1391304586 shutdown_connections 2024-04-25T14:14:45.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.346+0000 7faab7977740 1 -- 172.21.15.96:0/1391304586 wait complete. 2024-04-25T14:14:45.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.346+0000 7faab7977740 20 client.15966 trim_cache size 0 max 16384 2024-04-25T14:14:45.355 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-04-25T14:14:45.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.347+0000 7faab09ee640 1 -- 172.21.15.96:0/725582388 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7faa940afaf0 msgr2=0x7faa940b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:45.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.347+0000 7faab09ee640 1 --2- 172.21.15.96:0/725582388 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7faa940afaf0 0x7faa940b2000 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x55d669e20070 tx=0x7faaa4009a80 comp rx=0 tx=0).stop 2024-04-25T14:14:45.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.347+0000 7faab09ee640 1 -- 172.21.15.96:0/725582388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faaac1029e0 msgr2=0x7faaac06a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:45.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.347+0000 7faab09ee640 1 --2- 172.21.15.96:0/725582388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faaac1029e0 0x7faaac06a900 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7faa9c006010 tx=0x7faa9c004270 comp rx=0 tx=0).stop 2024-04-25T14:14:45.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.347+0000 7faab09ee640 1 -- 172.21.15.96:0/725582388 shutdown_connections 2024-04-25T14:14:45.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.347+0000 7faab09ee640 1 --2- 172.21.15.96:0/725582388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faaac108920 0x7faaac1a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.347+0000 7faab09ee640 1 --2- 172.21.15.96:0/725582388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faaac103c20 0x7faaac19f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.347+0000 7faab09ee640 1 --2- 172.21.15.96:0/725582388 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7faa940afaf0 0x7faa940b2000 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.347+0000 7faab09ee640 1 --2- 172.21.15.96:0/725582388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faaac1029e0 0x7faaac06a900 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.347+0000 7faab09ee640 1 -- 172.21.15.96:0/725582388 >> 172.21.15.96:0/725582388 conn(0x7faaac0fdbd0 msgr2=0x7faaac104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:45.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.347+0000 7faab09ee640 1 -- 172.21.15.96:0/725582388 shutdown_connections 2024-04-25T14:14:45.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.347+0000 7faab09ee640 1 -- 172.21.15.96:0/725582388 wait complete. 2024-04-25T14:14:45.361 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T14:14:45.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.486+0000 7f5ed03ee640 1 -- 172.21.15.96:0/2573817783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ec81029e0 msgr2=0x7f5ec81036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:45.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.486+0000 7f5ed03ee640 1 --2- 172.21.15.96:0/2573817783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ec81029e0 0x7f5ec81036e0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4009900 tx=0x7f5ec402a410 comp rx=0 tx=0).stop 2024-04-25T14:14:45.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.486+0000 7f5ed03ee640 1 -- 172.21.15.96:0/2573817783 shutdown_connections 2024-04-25T14:14:45.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.486+0000 7f5ed03ee640 1 --2- 172.21.15.96:0/2573817783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ec8108920 0x7f5ec810ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.486+0000 7f5ed03ee640 1 --2- 172.21.15.96:0/2573817783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ec8103c20 0x7f5ec8104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.486+0000 7f5ed03ee640 1 --2- 172.21.15.96:0/2573817783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ec81029e0 0x7f5ec81036e0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.486+0000 7f5ed03ee640 1 -- 172.21.15.96:0/2573817783 >> 172.21.15.96:0/2573817783 conn(0x7f5ec80fdbd0 msgr2=0x7f5ec8100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:45.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.486+0000 7f5ed03ee640 1 -- 172.21.15.96:0/2573817783 shutdown_connections 2024-04-25T14:14:45.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.486+0000 7f5ed03ee640 1 -- 172.21.15.96:0/2573817783 wait complete. 2024-04-25T14:14:45.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.487+0000 7f5ed03ee640 1 Processor -- start 2024-04-25T14:14:45.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.487+0000 7f5ed03ee640 1 -- start start 2024-04-25T14:14:45.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.487+0000 7f5ed03ee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ec81029e0 0x7f5ec8198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.487+0000 7f5ed03ee640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ec8103c20 0x7f5ec81988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.487+0000 7f5ed03ee640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ec8108920 0x7f5ec819f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.487+0000 7f5ed03ee640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec810d400 con 0x7f5ec81029e0 2024-04-25T14:14:45.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.487+0000 7f5ed03ee640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec810d280 con 0x7f5ec8108920 2024-04-25T14:14:45.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.487+0000 7f5ed03ee640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec810d580 con 0x7f5ec8103c20 2024-04-25T14:14:45.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.487+0000 7f5ece18a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ec81029e0 0x7f5ec8198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.487+0000 7f5ecd989640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ec8103c20 0x7f5ec81988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.487+0000 7f5ecd989640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ec8103c20 0x7f5ec81988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.96:54976/0 (socket says 172.21.15.96:54976) 2024-04-25T14:14:45.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.487+0000 7f5ecd989640 1 -- 172.21.15.96:0/1945156031 learned_addr learned my addr 172.21.15.96:0/1945156031 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:45.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.487+0000 7f5ece98b640 1 --2- 172.21.15.96:0/1945156031 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ec8108920 0x7f5ec819f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.488+0000 7f5ecd989640 1 -- 172.21.15.96:0/1945156031 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ec8108920 msgr2=0x7f5ec819f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:45.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.488+0000 7f5ecd989640 1 --2- 172.21.15.96:0/1945156031 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ec8108920 0x7f5ec819f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.488+0000 7f5ecd989640 1 -- 172.21.15.96:0/1945156031 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ec81029e0 msgr2=0x7f5ec8198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:45.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.488+0000 7f5ecd989640 1 --2- 172.21.15.96:0/1945156031 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ec81029e0 0x7f5ec8198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.488+0000 7f5ecd989640 1 -- 172.21.15.96:0/1945156031 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ec819fea0 con 0x7f5ec8103c20 2024-04-25T14:14:45.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.488+0000 7f5ece18a640 1 --2- 172.21.15.96:0/1945156031 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ec81029e0 0x7f5ec8198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:14:45.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.488+0000 7f5ecd989640 1 --2- 172.21.15.96:0/1945156031 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ec8103c20 0x7f5ec81988a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8011a60 tx=0x7f5eb8011f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:45.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.488+0000 7f5ebf7fe640 1 -- 172.21.15.96:0/1945156031 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb800fbc0 con 0x7f5ec8103c20 2024-04-25T14:14:45.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.488+0000 7f5ed03ee640 1 -- 172.21.15.96:0/1945156031 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ec81a0190 con 0x7f5ec8103c20 2024-04-25T14:14:45.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.488+0000 7f5ebf7fe640 1 -- 172.21.15.96:0/1945156031 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5eb800fd60 con 0x7f5ec8103c20 2024-04-25T14:14:45.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.488+0000 7f5ebf7fe640 1 -- 172.21.15.96:0/1945156031 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb8019400 con 0x7f5ec8103c20 2024-04-25T14:14:45.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.488+0000 7f5ed03ee640 1 -- 172.21.15.96:0/1945156031 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ec81a06e0 con 0x7f5ec8103c20 2024-04-25T14:14:45.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.492+0000 7f5ed7375740 1 Processor -- start 2024-04-25T14:14:45.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.492+0000 7f5ed7375740 1 -- start start 2024-04-25T14:14:45.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.492+0000 7f5ed7375740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8e7bf1390 0x55e8e7f93810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.492+0000 7f5ed7375740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8e7bc84d0 0x55e8e7f93d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.492+0000 7f5ed7375740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e8e7bd06b0 0x55e8e7f94a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.492+0000 7f5ed7375740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e8e7ef7d80 con 0x55e8e7bf1390 2024-04-25T14:14:45.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.492+0000 7f5ed7375740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e8e7eb1a20 con 0x55e8e7bc84d0 2024-04-25T14:14:45.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.492+0000 7f5ed7375740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e8e7ef3d40 con 0x55e8e7bd06b0 2024-04-25T14:14:45.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.493+0000 7f5ebf7fe640 1 -- 172.21.15.96:0/1945156031 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f5eb8004f80 con 0x7f5ec8103c20 2024-04-25T14:14:45.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.493+0000 7f5ece98b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8e7bf1390 0x55e8e7f93810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.493+0000 7f5ece98b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8e7bf1390 0x55e8e7f93810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34776/0 (socket says 172.21.15.96:34776) 2024-04-25T14:14:45.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.493+0000 7f5ece98b640 1 -- 172.21.15.96:0/567230148 learned_addr learned my addr 172.21.15.96:0/567230148 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:45.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.493+0000 7f5ece18a640 1 --2- 172.21.15.96:0/567230148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8e7bc84d0 0x55e8e7f93d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.493+0000 7f5ecd989640 1 --2- 172.21.15.96:0/567230148 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e8e7bd06b0 0x55e8e7f94a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.493+0000 7f5ece18a640 1 -- 172.21.15.96:0/567230148 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e8e7bd06b0 msgr2=0x55e8e7f94a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:45.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.493+0000 7f5ece18a640 1 --2- 172.21.15.96:0/567230148 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e8e7bd06b0 0x55e8e7f94a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.493+0000 7f5ece18a640 1 -- 172.21.15.96:0/567230148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8e7bf1390 msgr2=0x55e8e7f93810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:45.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.493+0000 7f5ece18a640 1 --2- 172.21.15.96:0/567230148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8e7bf1390 0x55e8e7f93810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.493+0000 7f5ece18a640 1 -- 172.21.15.96:0/567230148 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8e7ebcc30 con 0x55e8e7bc84d0 2024-04-25T14:14:45.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.493+0000 7f5ece98b640 1 --2- 172.21.15.96:0/567230148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8e7bf1390 0x55e8e7f93810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:14:45.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.494+0000 7f5ebf7fe640 1 --2- 172.21.15.96:0/1945156031 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f5eb40afaf0 0x7f5eb40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.494+0000 7f5ece98b640 1 --2- 172.21.15.96:0/1945156031 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f5eb40afaf0 0x7f5eb40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.494+0000 7f5ece18a640 1 --2- 172.21.15.96:0/567230148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8e7bc84d0 0x55e8e7f93d90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5ec402a930 tx=0x7f5ec4005e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65ce9f42293a7fc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:45.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.494+0000 7f5ebcbfe640 1 -- 172.21.15.96:0/567230148 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4035070 con 0x55e8e7bc84d0 2024-04-25T14:14:45.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.494+0000 7f5ebcbfe640 1 -- 172.21.15.96:0/567230148 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ec40318e0 con 0x55e8e7bc84d0 2024-04-25T14:14:45.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.494+0000 7f5ebf7fe640 1 -- 172.21.15.96:0/1945156031 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f5eb80e2570 con 0x7f5ec8103c20 2024-04-25T14:14:45.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.494+0000 7f5ebcbfe640 1 -- 172.21.15.96:0/567230148 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4003dc0 con 0x55e8e7bc84d0 2024-04-25T14:14:45.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.496+0000 7f5ed7375740 1 -- 172.21.15.96:0/567230148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8e7bc84d0 msgr2=0x55e8e7f93d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:45.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.496+0000 7f5ece98b640 1 --2- 172.21.15.96:0/1945156031 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f5eb40afaf0 0x7f5eb40b2000 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x55e8e7f94860 tx=0x7f5ec0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:45.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.496+0000 7f5ed7375740 1 --2- 172.21.15.96:0/567230148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8e7bc84d0 0x55e8e7f93d90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5ec402a930 tx=0x7f5ec4005e20 comp rx=0 tx=0).stop 2024-04-25T14:14:45.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.496+0000 7f5ed7375740 1 -- 172.21.15.96:0/567230148 shutdown_connections 2024-04-25T14:14:45.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.496+0000 7f5ed7375740 1 --2- 172.21.15.96:0/567230148 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e8e7bd06b0 0x55e8e7f94a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.496+0000 7f5ed7375740 1 --2- 172.21.15.96:0/567230148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8e7bc84d0 0x55e8e7f93d90 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.496+0000 7f5ed7375740 1 --2- 172.21.15.96:0/567230148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8e7bf1390 0x55e8e7f93810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.496+0000 7f5ed7375740 1 -- 172.21.15.96:0/567230148 >> 172.21.15.96:0/567230148 conn(0x55e8e7be54e0 msgr2=0x55e8e7f8f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:45.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.496+0000 7f5ed7375740 1 -- 172.21.15.96:0/567230148 shutdown_connections 2024-04-25T14:14:45.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.496+0000 7f5ed7375740 1 -- 172.21.15.96:0/567230148 wait complete. 2024-04-25T14:14:45.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.496+0000 7f5ed7375740 1 -- start start 2024-04-25T14:14:45.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.496+0000 7f5ed7375740 1 Processor -- start 2024-04-25T14:14:45.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.497+0000 7f5ed7375740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8e7bc84d0 0x55e8e7f9d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.497+0000 7f5ed7375740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e8e7bd06b0 0x55e8e7f9daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.497+0000 7f5ed7375740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8e7bf1390 0x55e8e7fa4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.497+0000 7f5ed7375740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e8e7f954d0 con 0x55e8e7bf1390 2024-04-25T14:14:45.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.497+0000 7f5ed7375740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e8e7e7edc0 con 0x55e8e7bc84d0 2024-04-25T14:14:45.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.497+0000 7f5ed7375740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e8e7f95650 con 0x55e8e7bd06b0 2024-04-25T14:14:45.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.497+0000 7f5ed7375740 10 client.0 _ll_register_callbacks cb 0x55e8e7e9a600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:14:45.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.497+0000 7f5ece98b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8e7bf1390 0x55e8e7fa4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.497+0000 7f5ece98b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8e7bf1390 0x55e8e7fa4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34792/0 (socket says 172.21.15.96:34792) 2024-04-25T14:14:45.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.497+0000 7f5ece98b640 1 -- 172.21.15.96:0/2586954270 learned_addr learned my addr 172.21.15.96:0/2586954270 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:45.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.497+0000 7f5ece18a640 1 --2- 172.21.15.96:0/2586954270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8e7bc84d0 0x55e8e7f9d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.497+0000 7f5ecd989640 1 --2- 172.21.15.96:0/2586954270 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e8e7bd06b0 0x55e8e7f9daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.497+0000 7f5ece98b640 1 -- 172.21.15.96:0/2586954270 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e8e7bd06b0 msgr2=0x55e8e7f9daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:45.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.497+0000 7f5ece98b640 1 --2- 172.21.15.96:0/2586954270 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e8e7bd06b0 0x55e8e7f9daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.497+0000 7f5ece98b640 1 -- 172.21.15.96:0/2586954270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8e7bc84d0 msgr2=0x55e8e7f9d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:45.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.497+0000 7f5ece98b640 1 --2- 172.21.15.96:0/2586954270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8e7bc84d0 0x55e8e7f9d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.497+0000 7f5ece98b640 1 -- 172.21.15.96:0/2586954270 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8e7c093f0 con 0x55e8e7bf1390 2024-04-25T14:14:45.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.498+0000 7f5ece98b640 1 --2- 172.21.15.96:0/2586954270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8e7bf1390 0x55e8e7fa4a00 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0003bc0 tx=0x7f5ec0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b63377b102b27d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:45.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.498+0000 7f5eb1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:14:45.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.498+0000 7f5eb1ffb640 1 -- 172.21.15.96:0/2586954270 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec0004310 con 0x55e8e7bf1390 2024-04-25T14:14:45.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.498+0000 7f5ed7375740 1 -- 172.21.15.96:0/2586954270 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ec40318e0 con 0x55e8e7bf1390 2024-04-25T14:14:45.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.498+0000 7f5eb1ffb640 1 -- 172.21.15.96:0/2586954270 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f5ec00044b0 con 0x55e8e7bf1390 2024-04-25T14:14:45.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.498+0000 7f5eb1ffb640 1 -- 172.21.15.96:0/2586954270 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5ec00046b0 con 0x55e8e7bf1390 2024-04-25T14:14:45.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.498+0000 7f5ed7375740 10 client.15978 fetch_fsmap learned FSMap version 26 2024-04-25T14:14:45.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.498+0000 7f5ed7375740 1 -- 172.21.15.96:0/2586954270 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e8e7cd37d0 con 0x55e8e7bf1390 2024-04-25T14:14:45.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.498+0000 7f5eb1ffb640 1 -- 172.21.15.96:0/2586954270 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ec40318e0 con 0x55e8e7bf1390 2024-04-25T14:14:45.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.500+0000 7f5eb1ffb640 1 -- 172.21.15.96:0/2586954270 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5ec0026070 con 0x55e8e7bf1390 2024-04-25T14:14:45.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.500+0000 7f5ed7375740 10 client.15978 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:14:45.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.500+0000 7f5ed7375740 10 client.15978 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:14:45.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.500+0000 7f5ed7375740 20 client.15978 populate_metadata read hostname 'smithi096' 2024-04-25T14:14:45.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.500+0000 7f5ed7375740 1 --2- 172.21.15.96:0/2586954270 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55e8e7baff60 0x55e8e7fab2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.500+0000 7f5ed7375740 4 client.15978 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:14:45.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.500+0000 7f5ed7375740 1 -- 172.21.15.96:0/2586954270 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8e7f05990 con 0x55e8e7baff60 2024-04-25T14:14:45.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.500+0000 7f5ece18a640 1 --2- 172.21.15.96:0/2586954270 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55e8e7baff60 0x55e8e7fab2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.501+0000 7f5ece18a640 1 --2- 172.21.15.96:0/2586954270 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55e8e7baff60 0x55e8e7fab2c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b60b95610ccc98f6 server_cookie=63dcdaeee1e49a4 in_seq=0 out_seq=0 2024-04-25T14:14:45.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.501+0000 7f5eb1ffb640 10 client.15978 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:45.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.501+0000 7f5eb1ffb640 1 -- 172.21.15.96:0/2586954270 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e8e7f05990 con 0x55e8e7baff60 2024-04-25T14:14:45.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.501+0000 7f5eb1ffb640 10 client.15978 handle_command_reply: tid=0 2024-04-25T14:14:45.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.501+0000 7f5ed7375740 1 client.15978 shutdown 2024-04-25T14:14:45.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.501+0000 7f5ed7375740 1 -- 172.21.15.96:0/2586954270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8e7bf1390 msgr2=0x55e8e7fa4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:45.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.502+0000 7f5ed7375740 1 --2- 172.21.15.96:0/2586954270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8e7bf1390 0x55e8e7fa4a00 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0003bc0 tx=0x7f5ec0004ab0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.502+0000 7f5ed7375740 1 -- 172.21.15.96:0/2586954270 shutdown_connections 2024-04-25T14:14:45.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.502+0000 7f5ed7375740 1 --2- 172.21.15.96:0/2586954270 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55e8e7baff60 0x55e8e7fab2c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.502+0000 7f5ed7375740 1 --2- 172.21.15.96:0/2586954270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8e7bf1390 0x55e8e7fa4a00 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.502+0000 7f5eb1ffb640 0 client.15978 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:45.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.502+0000 7f5ed7375740 1 --2- 172.21.15.96:0/2586954270 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e8e7bd06b0 0x55e8e7f9daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.502+0000 7f5ed7375740 1 --2- 172.21.15.96:0/2586954270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8e7bc84d0 0x55e8e7f9d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.502+0000 7f5ed7375740 1 -- 172.21.15.96:0/2586954270 >> 172.21.15.96:0/2586954270 conn(0x55e8e7be54e0 msgr2=0x55e8e7f91f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:45.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.502+0000 7f5ed7375740 1 -- 172.21.15.96:0/2586954270 shutdown_connections 2024-04-25T14:14:45.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.502+0000 7f5ed7375740 1 -- 172.21.15.96:0/2586954270 wait complete. 2024-04-25T14:14:45.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.502+0000 7f5ed7375740 20 client.15978 trim_cache size 0 max 16384 2024-04-25T14:14:45.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.515+0000 7f5ed7375740 1 Processor -- start 2024-04-25T14:14:45.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.515+0000 7f5ed7375740 1 -- start start 2024-04-25T14:14:45.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.516+0000 7f5ed7375740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e8e7bc84d0 0x55e8e7fb0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.516+0000 7f5ed7375740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8e7bd06b0 0x55e8e7fa4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.516+0000 7f5ed7375740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8e7bf1390 0x55e8e7fb0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.516+0000 7f5ed7375740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec81a9110 con 0x55e8e7bf1390 2024-04-25T14:14:45.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.516+0000 7f5ed7375740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec81aac50 con 0x55e8e7bd06b0 2024-04-25T14:14:45.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.516+0000 7f5ed7375740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec81a97e0 con 0x55e8e7bc84d0 2024-04-25T14:14:45.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.516+0000 7f5ece98b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8e7bf1390 0x55e8e7fb0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.516+0000 7f5ece98b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8e7bf1390 0x55e8e7fb0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34808/0 (socket says 172.21.15.96:34808) 2024-04-25T14:14:45.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.516+0000 7f5ece98b640 1 -- 172.21.15.96:0/758248880 learned_addr learned my addr 172.21.15.96:0/758248880 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:45.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.516+0000 7f5ece18a640 1 --2- 172.21.15.96:0/758248880 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e8e7bc84d0 0x55e8e7fb0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.516+0000 7f5ecd989640 1 --2- 172.21.15.96:0/758248880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8e7bd06b0 0x55e8e7fa4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.516+0000 7f5ece98b640 1 -- 172.21.15.96:0/758248880 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e8e7bc84d0 msgr2=0x55e8e7fb0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:45.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.516+0000 7f5ece98b640 1 --2- 172.21.15.96:0/758248880 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e8e7bc84d0 0x55e8e7fb0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.516+0000 7f5ece98b640 1 -- 172.21.15.96:0/758248880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8e7bd06b0 msgr2=0x55e8e7fa4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:45.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.516+0000 7f5ece98b640 1 --2- 172.21.15.96:0/758248880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8e7bd06b0 0x55e8e7fa4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.516+0000 7f5ece98b640 1 -- 172.21.15.96:0/758248880 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8e7f9b680 con 0x55e8e7bf1390 2024-04-25T14:14:45.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.516+0000 7f5ecd989640 1 --2- 172.21.15.96:0/758248880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8e7bd06b0 0x55e8e7fa4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:14:45.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.516+0000 7f5ece98b640 1 --2- 172.21.15.96:0/758248880 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8e7bf1390 0x55e8e7fb0fc0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0028490 tx=0x7f5ec000fe50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58372467b7931f26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:45.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.517+0000 7f5eb3fff640 1 -- 172.21.15.96:0/758248880 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec0004450 con 0x55e8e7bf1390 2024-04-25T14:14:45.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.517+0000 7f5eb3fff640 1 -- 172.21.15.96:0/758248880 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ec0037020 con 0x55e8e7bf1390 2024-04-25T14:14:45.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.517+0000 7f5eb3fff640 1 -- 172.21.15.96:0/758248880 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec001eaa0 con 0x55e8e7bf1390 2024-04-25T14:14:45.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.518+0000 7f5ed7375740 1 -- 172.21.15.96:0/758248880 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8e7bf1390 msgr2=0x55e8e7fb0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:45.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.518+0000 7f5ed7375740 1 --2- 172.21.15.96:0/758248880 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8e7bf1390 0x55e8e7fb0fc0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0028490 tx=0x7f5ec000fe50 comp rx=0 tx=0).stop 2024-04-25T14:14:45.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.518+0000 7f5ed7375740 1 -- 172.21.15.96:0/758248880 shutdown_connections 2024-04-25T14:14:45.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.518+0000 7f5ed7375740 1 --2- 172.21.15.96:0/758248880 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8e7bf1390 0x55e8e7fb0fc0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.518+0000 7f5ed7375740 1 --2- 172.21.15.96:0/758248880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8e7bd06b0 0x55e8e7fa4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.518+0000 7f5ed7375740 1 --2- 172.21.15.96:0/758248880 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e8e7bc84d0 0x55e8e7fb0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.518+0000 7f5ed7375740 1 -- 172.21.15.96:0/758248880 >> 172.21.15.96:0/758248880 conn(0x55e8e7be54e0 msgr2=0x55e8e7f982f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:45.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.518+0000 7f5ed7375740 1 -- 172.21.15.96:0/758248880 shutdown_connections 2024-04-25T14:14:45.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.518+0000 7f5ed7375740 1 -- 172.21.15.96:0/758248880 wait complete. 2024-04-25T14:14:45.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.518+0000 7f5ed7375740 1 -- start start 2024-04-25T14:14:45.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.518+0000 7f5ed7375740 1 Processor -- start 2024-04-25T14:14:45.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.519+0000 7f5ed7375740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8e7bc84d0 0x55e8e7fa3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.519+0000 7f5ed7375740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8e7bd06b0 0x55e8e7fa4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.519+0000 7f5ed7375740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e8e7bf1390 0x55e8e8011830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.519+0000 7f5ed7375740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec81aa580 con 0x55e8e7bc84d0 2024-04-25T14:14:45.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.519+0000 7f5ed7375740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec81a9eb0 con 0x55e8e7bd06b0 2024-04-25T14:14:45.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.519+0000 7f5ed7375740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec81a70f0 con 0x55e8e7bf1390 2024-04-25T14:14:45.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.519+0000 7f5ece18a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8e7bc84d0 0x55e8e7fa3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.519+0000 7f5ece18a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8e7bc84d0 0x55e8e7fa3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34812/0 (socket says 172.21.15.96:34812) 2024-04-25T14:14:45.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.519+0000 7f5ece18a640 1 -- 172.21.15.96:0/4019805903 learned_addr learned my addr 172.21.15.96:0/4019805903 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:45.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.519+0000 7f5ed7375740 10 client.0 _ll_register_callbacks cb 0x7f5ec0011e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:14:45.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.519+0000 7f5ece98b640 1 --2- 172.21.15.96:0/4019805903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e8e7bf1390 0x55e8e8011830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.519+0000 7f5ecd989640 1 --2- 172.21.15.96:0/4019805903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8e7bd06b0 0x55e8e7fa4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.519+0000 7f5ece18a640 1 -- 172.21.15.96:0/4019805903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e8e7bf1390 msgr2=0x55e8e8011830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:45.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.519+0000 7f5ece18a640 1 --2- 172.21.15.96:0/4019805903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e8e7bf1390 0x55e8e8011830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.519+0000 7f5ece18a640 1 -- 172.21.15.96:0/4019805903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8e7bd06b0 msgr2=0x55e8e7fa4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:45.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.519+0000 7f5ece18a640 1 --2- 172.21.15.96:0/4019805903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8e7bd06b0 0x55e8e7fa4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.519+0000 7f5ece18a640 1 -- 172.21.15.96:0/4019805903 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8e8011e20 con 0x55e8e7bc84d0 2024-04-25T14:14:45.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.520+0000 7f5ece18a640 1 --2- 172.21.15.96:0/4019805903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8e7bc84d0 0x55e8e7fa3c40 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4005ab0 tx=0x7f5ec40315a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dfe4a010b61f26b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:45.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.520+0000 7f5eb1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:14:45.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.520+0000 7f5eb1ffb640 1 -- 172.21.15.96:0/4019805903 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4035040 con 0x55e8e7bc84d0 2024-04-25T14:14:45.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.520+0000 7f5eb1ffb640 1 -- 172.21.15.96:0/4019805903 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f5ec4031c00 con 0x55e8e7bc84d0 2024-04-25T14:14:45.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.520+0000 7f5ed7375740 1 -- 172.21.15.96:0/4019805903 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ec0037020 con 0x55e8e7bc84d0 2024-04-25T14:14:45.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.521+0000 7f5eb1ffb640 1 -- 172.21.15.96:0/4019805903 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5ec404a040 con 0x55e8e7bc84d0 2024-04-25T14:14:45.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.521+0000 7f5ed7375740 10 client.15984 fetch_fsmap learned FSMap version 26 2024-04-25T14:14:45.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.521+0000 7f5ed7375740 1 -- 172.21.15.96:0/4019805903 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e8e8013110 con 0x55e8e7bc84d0 2024-04-25T14:14:45.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.521+0000 7f5eb1ffb640 1 -- 172.21.15.96:0/4019805903 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ec0037020 con 0x55e8e7bc84d0 2024-04-25T14:14:45.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.521+0000 7f5eb1ffb640 1 -- 172.21.15.96:0/4019805903 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5ec4052440 con 0x55e8e7bc84d0 2024-04-25T14:14:45.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.521+0000 7f5ed7375740 10 client.15984 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:14:45.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.521+0000 7f5ed7375740 10 client.15984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:14:45.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.521+0000 7f5ed7375740 20 client.15984 populate_metadata read hostname 'smithi096' 2024-04-25T14:14:45.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.521+0000 7f5ed7375740 1 --2- 172.21.15.96:0/4019805903 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55e8e7b853a0 0x55e8e8015680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.521+0000 7f5ed7375740 4 client.15984 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:14:45.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.521+0000 7f5ed7375740 1 -- 172.21.15.96:0/4019805903 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e8e7eff800 con 0x55e8e7b853a0 2024-04-25T14:14:45.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.522+0000 7f5ecd989640 1 --2- 172.21.15.96:0/4019805903 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55e8e7b853a0 0x55e8e8015680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.522+0000 7f5ecd989640 1 --2- 172.21.15.96:0/4019805903 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55e8e7b853a0 0x55e8e8015680 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=183a8cb1f6ec1bf5 server_cookie=2dc112cf867050b6 in_seq=0 out_seq=0 2024-04-25T14:14:45.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.522+0000 7f5eb1ffb640 10 client.15984 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:45.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.522+0000 7f5eb1ffb640 1 -- 172.21.15.96:0/4019805903 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e8e7eff800 con 0x55e8e7b853a0 2024-04-25T14:14:45.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.523+0000 7f5eb1ffb640 10 client.15984 handle_command_reply: tid=0 2024-04-25T14:14:45.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.523+0000 7f5ed7375740 1 client.15984 shutdown 2024-04-25T14:14:45.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.523+0000 7f5ed7375740 1 -- 172.21.15.96:0/4019805903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8e7bc84d0 msgr2=0x55e8e7fa3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:45.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.523+0000 7f5ed7375740 1 --2- 172.21.15.96:0/4019805903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8e7bc84d0 0x55e8e7fa3c40 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4005ab0 tx=0x7f5ec40315a0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.523+0000 7f5ed7375740 1 -- 172.21.15.96:0/4019805903 shutdown_connections 2024-04-25T14:14:45.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.523+0000 7f5ed7375740 1 --2- 172.21.15.96:0/4019805903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e8e7bf1390 0x55e8e8011830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.523+0000 7f5ed7375740 1 --2- 172.21.15.96:0/4019805903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8e7bd06b0 0x55e8e7fa4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.523+0000 7f5ed7375740 1 --2- 172.21.15.96:0/4019805903 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55e8e7b853a0 0x55e8e8015680 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.523+0000 7f5ed7375740 1 --2- 172.21.15.96:0/4019805903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8e7bc84d0 0x55e8e7fa3c40 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.523+0000 7f5eb1ffb640 0 client.15984 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:45.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.523+0000 7f5ed7375740 1 -- 172.21.15.96:0/4019805903 >> 172.21.15.96:0/4019805903 conn(0x55e8e7be54e0 msgr2=0x55e8e7f91e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:45.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.523+0000 7f5ed7375740 1 -- 172.21.15.96:0/4019805903 shutdown_connections 2024-04-25T14:14:45.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.523+0000 7f5ed7375740 1 -- 172.21.15.96:0/4019805903 wait complete. 2024-04-25T14:14:45.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.523+0000 7f5ed7375740 20 client.15984 trim_cache size 0 max 16384 2024-04-25T14:14:45.528 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-04-25T14:14:45.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.524+0000 7f5ed03ee640 1 -- 172.21.15.96:0/1945156031 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f5eb40afaf0 msgr2=0x7f5eb40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:45.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.524+0000 7f5ed03ee640 1 --2- 172.21.15.96:0/1945156031 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f5eb40afaf0 0x7f5eb40b2000 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x55e8e7f94860 tx=0x7f5ec0009500 comp rx=0 tx=0).stop 2024-04-25T14:14:45.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.524+0000 7f5ed03ee640 1 -- 172.21.15.96:0/1945156031 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ec8103c20 msgr2=0x7f5ec81988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:45.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.524+0000 7f5ed03ee640 1 --2- 172.21.15.96:0/1945156031 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ec8103c20 0x7f5ec81988a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8011a60 tx=0x7f5eb8011f20 comp rx=0 tx=0).stop 2024-04-25T14:14:45.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.524+0000 7f5ed03ee640 1 -- 172.21.15.96:0/1945156031 shutdown_connections 2024-04-25T14:14:45.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.524+0000 7f5ed03ee640 1 --2- 172.21.15.96:0/1945156031 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ec8108920 0x7f5ec819f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.524+0000 7f5ed03ee640 1 --2- 172.21.15.96:0/1945156031 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ec8103c20 0x7f5ec81988a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.524+0000 7f5ed03ee640 1 --2- 172.21.15.96:0/1945156031 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f5eb40afaf0 0x7f5eb40b2000 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.524+0000 7f5ed03ee640 1 --2- 172.21.15.96:0/1945156031 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ec81029e0 0x7f5ec8198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.524+0000 7f5ed03ee640 1 -- 172.21.15.96:0/1945156031 >> 172.21.15.96:0/1945156031 conn(0x7f5ec80fdbd0 msgr2=0x7f5ec8104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:45.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.524+0000 7f5ed03ee640 1 -- 172.21.15.96:0/1945156031 shutdown_connections 2024-04-25T14:14:45.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:45.524+0000 7f5ed03ee640 1 -- 172.21.15.96:0/1945156031 wait complete. 2024-04-25T14:14:45.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:45 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/1154579808' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:14:45.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:45 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/1154579808' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:14:45.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:45 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/1154579808' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:14:46.539 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T14:14:46.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:14:46.539 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T14:14:46.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.703+0000 7fb7daf00640 1 -- 172.21.15.96:0/2023777978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7d4102240 msgr2=0x7fb7d410bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:46.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.703+0000 7fb7daf00640 1 --2- 172.21.15.96:0/2023777978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7d4102240 0x7fb7d410bca0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc009ca0 tx=0x7fb7cc02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.703+0000 7fb7daf00640 1 -- 172.21.15.96:0/2023777978 shutdown_connections 2024-04-25T14:14:46.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.703+0000 7fb7daf00640 1 --2- 172.21.15.96:0/2023777978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7d4102240 0x7fb7d410bca0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.703+0000 7fb7daf00640 1 --2- 172.21.15.96:0/2023777978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7d4101740 0x7fb7d4101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.703+0000 7fb7daf00640 1 --2- 172.21.15.96:0/2023777978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7d4105090 0x7fb7d41054a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.703+0000 7fb7daf00640 1 -- 172.21.15.96:0/2023777978 >> 172.21.15.96:0/2023777978 conn(0x7fb7d40fd340 msgr2=0x7fb7d40ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:46.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.703+0000 7fb7daf00640 1 -- 172.21.15.96:0/2023777978 shutdown_connections 2024-04-25T14:14:46.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.703+0000 7fb7daf00640 1 -- 172.21.15.96:0/2023777978 wait complete. 2024-04-25T14:14:46.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.704+0000 7fb7daf00640 1 Processor -- start 2024-04-25T14:14:46.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.704+0000 7fb7daf00640 1 -- start start 2024-04-25T14:14:46.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.704+0000 7fb7daf00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7d4101740 0x7fb7d419a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:46.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.704+0000 7fb7daf00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7d4102240 0x7fb7d419a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:46.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.704+0000 7fb7daf00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7d4105090 0x7fb7d41a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:46.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.704+0000 7fb7daf00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d410f620 con 0x7fb7d4105090 2024-04-25T14:14:46.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.704+0000 7fb7daf00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d410f4a0 con 0x7fb7d4102240 2024-04-25T14:14:46.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.704+0000 7fb7daf00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d410f7a0 con 0x7fb7d4101740 2024-04-25T14:14:46.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.704+0000 7fb7d949d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7d4105090 0x7fb7d41a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:46.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.705+0000 7fb7d949d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7d4105090 0x7fb7d41a1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34830/0 (socket says 172.21.15.96:34830) 2024-04-25T14:14:46.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.705+0000 7fb7d949d640 1 -- 172.21.15.96:0/4029418213 learned_addr learned my addr 172.21.15.96:0/4029418213 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:46.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.705+0000 7fb7d8c9c640 1 --2- 172.21.15.96:0/4029418213 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7d4101740 0x7fb7d419a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:46.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.705+0000 7fb7d3fff640 1 --2- 172.21.15.96:0/4029418213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7d4102240 0x7fb7d419a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:46.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.705+0000 7fb7d949d640 1 -- 172.21.15.96:0/4029418213 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7d4101740 msgr2=0x7fb7d419a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:46.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.705+0000 7fb7d949d640 1 --2- 172.21.15.96:0/4029418213 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7d4101740 0x7fb7d419a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.705+0000 7fb7d949d640 1 -- 172.21.15.96:0/4029418213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7d4102240 msgr2=0x7fb7d419a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:46.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.705+0000 7fb7d949d640 1 --2- 172.21.15.96:0/4029418213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7d4102240 0x7fb7d419a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.705+0000 7fb7d949d640 1 -- 172.21.15.96:0/4029418213 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7d41a1ce0 con 0x7fb7d4105090 2024-04-25T14:14:46.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.705+0000 7fb7d949d640 1 --2- 172.21.15.96:0/4029418213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7d4105090 0x7fb7d41a1670 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc0097b0 tx=0x7fb7cc031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:46.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.705+0000 7fb7d1ffb640 1 -- 172.21.15.96:0/4029418213 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc003bf0 con 0x7fb7d4105090 2024-04-25T14:14:46.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.705+0000 7fb7d1ffb640 1 -- 172.21.15.96:0/4029418213 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7cc003d90 con 0x7fb7d4105090 2024-04-25T14:14:46.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.706+0000 7fb7daf00640 1 -- 172.21.15.96:0/4029418213 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7d41a1f70 con 0x7fb7d4105090 2024-04-25T14:14:46.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.706+0000 7fb7d1ffb640 1 -- 172.21.15.96:0/4029418213 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc02fc30 con 0x7fb7d4105090 2024-04-25T14:14:46.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.706+0000 7fb7daf00640 1 -- 172.21.15.96:0/4029418213 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7d41a23e0 con 0x7fb7d4105090 2024-04-25T14:14:46.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.708+0000 7fb7d1ffb640 1 -- 172.21.15.96:0/4029418213 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fb7cc004040 con 0x7fb7d4105090 2024-04-25T14:14:46.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.709+0000 7fb7d1ffb640 1 --2- 172.21.15.96:0/4029418213 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb7c40afb70 0x7fb7c40b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:46.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.709+0000 7fb7d1ffb640 1 -- 172.21.15.96:0/4029418213 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fb7cc100220 con 0x7fb7d4105090 2024-04-25T14:14:46.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.709+0000 7fb7e1db3740 1 Processor -- start 2024-04-25T14:14:46.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.709+0000 7fb7d8c9c640 1 --2- 172.21.15.96:0/4029418213 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb7c40afb70 0x7fb7c40b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:46.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.709+0000 7fb7e1db3740 1 -- start start 2024-04-25T14:14:46.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.710+0000 7fb7e1db3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606c55e4410 0x5606c59854a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:46.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.710+0000 7fb7e1db3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606c55bb550 0x5606c5985a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:46.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.710+0000 7fb7e1db3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5606c55c3730 0x5606c598c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:46.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.710+0000 7fb7e1db3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5606c58eab00 con 0x5606c55e4410 2024-04-25T14:14:46.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.710+0000 7fb7e1db3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5606c5886920 con 0x5606c55c3730 2024-04-25T14:14:46.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.710+0000 7fb7e1db3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5606c58e6ac0 con 0x5606c55bb550 2024-04-25T14:14:46.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.710+0000 7fb7d8c9c640 1 --2- 172.21.15.96:0/4029418213 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb7c40afb70 0x7fb7c40b2080 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4069bf0 tx=0x7fb7c8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:46.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.710+0000 7fb7d949d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5606c55c3730 0x5606c598c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:46.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.710+0000 7fb7d3fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606c55bb550 0x5606c5985a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:46.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.710+0000 7fb7d3fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606c55bb550 0x5606c5985a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.96:55042/0 (socket says 172.21.15.96:55042) 2024-04-25T14:14:46.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.710+0000 7fb7d949d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5606c55c3730 0x5606c598c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:54340/0 (socket says 172.21.15.96:54340) 2024-04-25T14:14:46.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.710+0000 7fb7d3fff640 1 -- 172.21.15.96:0/2030049596 learned_addr learned my addr 172.21.15.96:0/2030049596 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:46.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.710+0000 7fb7d8c9c640 1 --2- 172.21.15.96:0/2030049596 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606c55e4410 0x5606c59854a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:46.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.710+0000 7fb7d949d640 1 -- 172.21.15.96:0/2030049596 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606c55bb550 msgr2=0x5606c5985a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:46.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.710+0000 7fb7d949d640 1 --2- 172.21.15.96:0/2030049596 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606c55bb550 0x5606c5985a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.710+0000 7fb7d949d640 1 -- 172.21.15.96:0/2030049596 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606c55e4410 msgr2=0x5606c59854a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:46.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.710+0000 7fb7d949d640 1 --2- 172.21.15.96:0/2030049596 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606c55e4410 0x5606c59854a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.710+0000 7fb7d949d640 1 -- 172.21.15.96:0/2030049596 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606c56c47e0 con 0x5606c55c3730 2024-04-25T14:14:46.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.711+0000 7fb7d8c9c640 1 --2- 172.21.15.96:0/2030049596 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606c55e4410 0x5606c59854a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:14:46.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.711+0000 7fb7d949d640 1 --2- 172.21.15.96:0/2030049596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5606c55c3730 0x5606c598c920 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc0fdd50 tx=0x7fb7cc02a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8a8d2d5c4b4450f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:46.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.711+0000 7fb7beffd640 1 -- 172.21.15.96:0/2030049596 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc0c81f0 con 0x5606c55c3730 2024-04-25T14:14:46.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.711+0000 7fb7beffd640 1 -- 172.21.15.96:0/2030049596 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7cc036070 con 0x5606c55c3730 2024-04-25T14:14:46.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.711+0000 7fb7beffd640 1 -- 172.21.15.96:0/2030049596 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc0046a0 con 0x5606c55c3730 2024-04-25T14:14:46.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.712+0000 7fb7e1db3740 1 -- 172.21.15.96:0/2030049596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5606c55c3730 msgr2=0x5606c598c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:46.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.712+0000 7fb7e1db3740 1 --2- 172.21.15.96:0/2030049596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5606c55c3730 0x5606c598c920 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc0fdd50 tx=0x7fb7cc02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.712+0000 7fb7e1db3740 1 -- 172.21.15.96:0/2030049596 shutdown_connections 2024-04-25T14:14:46.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.712+0000 7fb7e1db3740 1 --2- 172.21.15.96:0/2030049596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5606c55c3730 0x5606c598c920 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.712+0000 7fb7e1db3740 1 --2- 172.21.15.96:0/2030049596 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606c55bb550 0x5606c5985a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.712+0000 7fb7e1db3740 1 --2- 172.21.15.96:0/2030049596 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606c55e4410 0x5606c59854a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.712+0000 7fb7e1db3740 1 -- 172.21.15.96:0/2030049596 >> 172.21.15.96:0/2030049596 conn(0x5606c55d8560 msgr2=0x5606c5980e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:46.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.712+0000 7fb7e1db3740 1 -- 172.21.15.96:0/2030049596 shutdown_connections 2024-04-25T14:14:46.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.712+0000 7fb7e1db3740 1 -- 172.21.15.96:0/2030049596 wait complete. 2024-04-25T14:14:46.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.713+0000 7fb7e1db3740 1 -- start start 2024-04-25T14:14:46.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.713+0000 7fb7e1db3740 1 Processor -- start 2024-04-25T14:14:46.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.713+0000 7fb7e1db3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606c55bb550 0x5606c599fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:46.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.713+0000 7fb7e1db3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5606c55c3730 0x5606c59a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:46.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.713+0000 7fb7e1db3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606c55e4410 0x5606c59a7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:46.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.714+0000 7fb7e1db3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5606c5886920 con 0x5606c55bb550 2024-04-25T14:14:46.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.714+0000 7fb7e1db3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5606c58845d0 con 0x5606c55c3730 2024-04-25T14:14:46.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.714+0000 7fb7e1db3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5606c599e140 con 0x5606c55e4410 2024-04-25T14:14:46.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.714+0000 7fb7d8c9c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606c55bb550 0x5606c599fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:46.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.714+0000 7fb7d8c9c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606c55bb550 0x5606c599fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34854/0 (socket says 172.21.15.96:34854) 2024-04-25T14:14:46.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.714+0000 7fb7d3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5606c55c3730 0x5606c59a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:46.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.714+0000 7fb7d8c9c640 1 -- 172.21.15.96:0/2158383711 learned_addr learned my addr 172.21.15.96:0/2158383711 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:46.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.714+0000 7fb7d949d640 1 --2- 172.21.15.96:0/2158383711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606c55e4410 0x5606c59a7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:46.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.714+0000 7fb7e1db3740 10 client.0 _ll_register_callbacks cb 0x5606c5872080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:14:46.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.714+0000 7fb7d3fff640 1 -- 172.21.15.96:0/2158383711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606c55e4410 msgr2=0x5606c59a7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:46.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.714+0000 7fb7d3fff640 1 --2- 172.21.15.96:0/2158383711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606c55e4410 0x5606c59a7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.714+0000 7fb7d3fff640 1 -- 172.21.15.96:0/2158383711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606c55bb550 msgr2=0x5606c599fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:46.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.714+0000 7fb7d3fff640 1 --2- 172.21.15.96:0/2158383711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606c55bb550 0x5606c599fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.714+0000 7fb7d3fff640 1 -- 172.21.15.96:0/2158383711 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606c5724f70 con 0x5606c55c3730 2024-04-25T14:14:46.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.714+0000 7fb7e1db3740 1 -- 172.21.15.96:0/2158383711 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7cc036070 con 0x5606c55c3730 2024-04-25T14:14:46.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.714+0000 7fb7d8c9c640 1 --2- 172.21.15.96:0/2158383711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606c55bb550 0x5606c599fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:14:46.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.714+0000 7fb7d3fff640 1 --2- 172.21.15.96:0/2158383711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5606c55c3730 0x5606c59a03b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0005d00 tx=0x7fb7c00118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a7898d0bd113759 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:46.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.715+0000 7fb7b3fff640 10 client.25502 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T14:14:46.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.715+0000 7fb7b3fff640 1 -- 172.21.15.96:0/2158383711 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c0004020 con 0x5606c55c3730 2024-04-25T14:14:46.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.715+0000 7fb7b3fff640 1 -- 172.21.15.96:0/2158383711 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fb7c00041c0 con 0x5606c55c3730 2024-04-25T14:14:46.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.715+0000 7fb7b3fff640 1 -- 172.21.15.96:0/2158383711 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb7c0004420 con 0x5606c55c3730 2024-04-25T14:14:46.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.715+0000 7fb7e1db3740 10 client.25502 fetch_fsmap learned FSMap version 26 2024-04-25T14:14:46.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.715+0000 7fb7e1db3740 1 -- 172.21.15.96:0/2158383711 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({fsmap=26}) -- 0x5606c55bc170 con 0x5606c55c3730 2024-04-25T14:14:46.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.715+0000 7fb7b3fff640 1 -- 172.21.15.96:0/2158383711 <== mon.2 v2:172.21.15.97:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb7c0005020 con 0x5606c55c3730 2024-04-25T14:14:46.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.715+0000 7fb7e1db3740 10 client.25502 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:14:46.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.715+0000 7fb7e1db3740 10 client.25502 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:14:46.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.715+0000 7fb7e1db3740 20 client.25502 populate_metadata read hostname 'smithi096' 2024-04-25T14:14:46.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.715+0000 7fb7e1db3740 1 --2- 172.21.15.96:0/2158383711 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5606c55a31f0 0x5606c59ad6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:46.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.716+0000 7fb7e1db3740 4 client.25502 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:14:46.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.716+0000 7fb7e1db3740 1 -- 172.21.15.96:0/2158383711 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606c58f8710 con 0x5606c55a31f0 2024-04-25T14:14:46.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.716+0000 7fb7d8c9c640 1 --2- 172.21.15.96:0/2158383711 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5606c55a31f0 0x5606c59ad6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:46.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.716+0000 7fb7b3fff640 10 client.25502 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:46.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.716+0000 7fb7d8c9c640 1 --2- 172.21.15.96:0/2158383711 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5606c55a31f0 0x5606c59ad6a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6893f3f1709acbd3 server_cookie=46eafea8c03f258c in_seq=0 out_seq=0 2024-04-25T14:14:46.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.717+0000 7fb7b3fff640 1 -- 172.21.15.96:0/2158383711 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5606c58f8710 con 0x5606c55a31f0 2024-04-25T14:14:46.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.717+0000 7fb7b3fff640 10 client.25502 handle_command_reply: tid=0 2024-04-25T14:14:46.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.717+0000 7fb7e1db3740 1 client.25502 shutdown 2024-04-25T14:14:46.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.717+0000 7fb7e1db3740 1 -- 172.21.15.96:0/2158383711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5606c55c3730 msgr2=0x5606c59a03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:46.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.717+0000 7fb7e1db3740 1 --2- 172.21.15.96:0/2158383711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5606c55c3730 0x5606c59a03b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0005d00 tx=0x7fb7c00118d0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.717+0000 7fb7e1db3740 1 -- 172.21.15.96:0/2158383711 shutdown_connections 2024-04-25T14:14:46.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.717+0000 7fb7e1db3740 1 --2- 172.21.15.96:0/2158383711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606c55e4410 0x5606c59a7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.717+0000 7fb7e1db3740 1 --2- 172.21.15.96:0/2158383711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5606c55c3730 0x5606c59a03b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.717+0000 7fb7e1db3740 1 --2- 172.21.15.96:0/2158383711 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5606c55a31f0 0x5606c59ad6a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.717+0000 7fb7b3fff640 0 client.25502 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:46.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.717+0000 7fb7e1db3740 1 --2- 172.21.15.96:0/2158383711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606c55bb550 0x5606c599fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.717+0000 7fb7e1db3740 1 -- 172.21.15.96:0/2158383711 >> 172.21.15.96:0/2158383711 conn(0x5606c55d8560 msgr2=0x5606c597fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:46.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.717+0000 7fb7e1db3740 1 -- 172.21.15.96:0/2158383711 shutdown_connections 2024-04-25T14:14:46.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.717+0000 7fb7e1db3740 1 -- 172.21.15.96:0/2158383711 wait complete. 2024-04-25T14:14:46.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.718+0000 7fb7e1db3740 20 client.25502 trim_cache size 0 max 16384 2024-04-25T14:14:46.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.753+0000 7fb7e1db3740 1 Processor -- start 2024-04-25T14:14:46.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.753+0000 7fb7e1db3740 1 -- start start 2024-04-25T14:14:46.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.754+0000 7fb7e1db3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606c55a31f0 0x5606c599f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:46.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.754+0000 7fb7e1db3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606c55bb550 0x5606c59a7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:46.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.754+0000 7fb7e1db3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5606c55c3730 0x5606c59a7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:46.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.754+0000 7fb7e1db3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d41a2ad0 con 0x5606c55bb550 2024-04-25T14:14:46.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.754+0000 7fb7e1db3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d41a4610 con 0x5606c55c3730 2024-04-25T14:14:46.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.754+0000 7fb7e1db3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d41a31a0 con 0x5606c55a31f0 2024-04-25T14:14:46.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.754+0000 7fb7d3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606c55bb550 0x5606c59a7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:46.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.754+0000 7fb7d3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606c55bb550 0x5606c59a7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34864/0 (socket says 172.21.15.96:34864) 2024-04-25T14:14:46.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.754+0000 7fb7d3fff640 1 -- 172.21.15.96:0/1845667373 learned_addr learned my addr 172.21.15.96:0/1845667373 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:46.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.754+0000 7fb7d949d640 1 --2- 172.21.15.96:0/1845667373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5606c55c3730 0x5606c59a7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:46.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.754+0000 7fb7d8c9c640 1 --2- 172.21.15.96:0/1845667373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606c55a31f0 0x5606c599f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:46.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.754+0000 7fb7d3fff640 1 -- 172.21.15.96:0/1845667373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606c55a31f0 msgr2=0x5606c599f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:46.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.754+0000 7fb7d3fff640 1 --2- 172.21.15.96:0/1845667373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606c55a31f0 0x5606c599f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.754+0000 7fb7d3fff640 1 -- 172.21.15.96:0/1845667373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5606c55c3730 msgr2=0x5606c59a7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:46.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.754+0000 7fb7d3fff640 1 --2- 172.21.15.96:0/1845667373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5606c55c3730 0x5606c59a7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.754+0000 7fb7d3fff640 1 -- 172.21.15.96:0/1845667373 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606c59a7cb0 con 0x5606c55bb550 2024-04-25T14:14:46.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.754+0000 7fb7d949d640 1 --2- 172.21.15.96:0/1845667373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5606c55c3730 0x5606c59a7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:14:46.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.755+0000 7fb7d3fff640 1 --2- 172.21.15.96:0/1845667373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606c55bb550 0x5606c59a7230 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0004690 tx=0x7fb7c0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83a5b95f5bc5850a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:46.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.755+0000 7fb7be7fc640 1 -- 172.21.15.96:0/1845667373 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c0010700 con 0x5606c55bb550 2024-04-25T14:14:46.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.755+0000 7fb7be7fc640 1 -- 172.21.15.96:0/1845667373 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7cc036070 con 0x5606c55bb550 2024-04-25T14:14:46.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.755+0000 7fb7be7fc640 1 -- 172.21.15.96:0/1845667373 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c0024a90 con 0x5606c55bb550 2024-04-25T14:14:46.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.756+0000 7fb7e1db3740 1 -- 172.21.15.96:0/1845667373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606c55bb550 msgr2=0x5606c59a7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:46.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.756+0000 7fb7e1db3740 1 --2- 172.21.15.96:0/1845667373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606c55bb550 0x5606c59a7230 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0004690 tx=0x7fb7c0017000 comp rx=0 tx=0).stop 2024-04-25T14:14:46.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.756+0000 7fb7e1db3740 1 -- 172.21.15.96:0/1845667373 shutdown_connections 2024-04-25T14:14:46.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.756+0000 7fb7e1db3740 1 --2- 172.21.15.96:0/1845667373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5606c55c3730 0x5606c59a7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.756+0000 7fb7e1db3740 1 --2- 172.21.15.96:0/1845667373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606c55bb550 0x5606c59a7230 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.756+0000 7fb7e1db3740 1 --2- 172.21.15.96:0/1845667373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606c55a31f0 0x5606c599f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.756+0000 7fb7e1db3740 1 -- 172.21.15.96:0/1845667373 >> 172.21.15.96:0/1845667373 conn(0x5606c55d8560 msgr2=0x5606c5984660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:46.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.756+0000 7fb7e1db3740 1 -- 172.21.15.96:0/1845667373 shutdown_connections 2024-04-25T14:14:46.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.756+0000 7fb7e1db3740 1 -- 172.21.15.96:0/1845667373 wait complete. 2024-04-25T14:14:46.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.757+0000 7fb7e1db3740 1 -- start start 2024-04-25T14:14:46.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.757+0000 7fb7e1db3740 1 Processor -- start 2024-04-25T14:14:46.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.757+0000 7fb7e1db3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606c55a31f0 0x5606c59a4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:46.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.757+0000 7fb7e1db3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606c55c3730 0x5606c59a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:46.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.757+0000 7fb7e1db3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5606c55e4410 0x5606c5a139a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:46.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.758+0000 7fb7e1db3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d41a3f40 con 0x5606c55c3730 2024-04-25T14:14:46.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.758+0000 7fb7d3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606c55c3730 0x5606c59a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:46.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.758+0000 7fb7d8c9c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606c55a31f0 0x5606c59a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:46.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.758+0000 7fb7d8c9c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606c55a31f0 0x5606c59a4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.96:55058/0 (socket says 172.21.15.96:55058) 2024-04-25T14:14:46.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.758+0000 7fb7d8c9c640 1 -- 172.21.15.96:0/3664564317 learned_addr learned my addr 172.21.15.96:0/3664564317 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:46.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.758+0000 7fb7e1db3740 1 -- 172.21.15.96:0/3664564317 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d41a3870 con 0x5606c55e4410 2024-04-25T14:14:46.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.758+0000 7fb7e1db3740 1 -- 172.21.15.96:0/3664564317 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d40732d0 con 0x5606c55a31f0 2024-04-25T14:14:46.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.758+0000 7fb7d949d640 1 --2- 172.21.15.96:0/3664564317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5606c55e4410 0x5606c5a139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:46.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.758+0000 7fb7e1db3740 10 client.0 _ll_register_callbacks cb 0x7fb7c000bc40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:14:46.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.758+0000 7fb7d3fff640 1 -- 172.21.15.96:0/3664564317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606c55a31f0 msgr2=0x5606c59a4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:46.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.758+0000 7fb7d3fff640 1 --2- 172.21.15.96:0/3664564317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606c55a31f0 0x5606c59a4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.758+0000 7fb7d3fff640 1 -- 172.21.15.96:0/3664564317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5606c55e4410 msgr2=0x5606c5a139a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:46.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.758+0000 7fb7d3fff640 1 --2- 172.21.15.96:0/3664564317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5606c55e4410 0x5606c5a139a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.758+0000 7fb7d3fff640 1 -- 172.21.15.96:0/3664564317 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606c5a13ee0 con 0x5606c55c3730 2024-04-25T14:14:46.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.758+0000 7fb7e1db3740 1 -- 172.21.15.96:0/3664564317 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7cc036070 con 0x5606c55c3730 2024-04-25T14:14:46.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.758+0000 7fb7d3fff640 1 --2- 172.21.15.96:0/3664564317 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606c55c3730 0x5606c59a4c40 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0001000 tx=0x7fb7c0026d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37650fbb3a443ad2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:46.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.758+0000 7fb7b3fff640 10 client.16002 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:14:46.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.758+0000 7fb7b3fff640 1 -- 172.21.15.96:0/3664564317 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c0024480 con 0x5606c55c3730 2024-04-25T14:14:46.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.759+0000 7fb7b3fff640 1 -- 172.21.15.96:0/3664564317 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fb7c0026110 con 0x5606c55c3730 2024-04-25T14:14:46.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.759+0000 7fb7b3fff640 1 -- 172.21.15.96:0/3664564317 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb7c0026350 con 0x5606c55c3730 2024-04-25T14:14:46.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.759+0000 7fb7e1db3740 10 client.16002 fetch_fsmap learned FSMap version 26 2024-04-25T14:14:46.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.759+0000 7fb7e1db3740 1 -- 172.21.15.96:0/3664564317 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({fsmap=26}) -- 0x5606c5a14fc0 con 0x5606c55c3730 2024-04-25T14:14:46.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.759+0000 7fb7b3fff640 1 -- 172.21.15.96:0/3664564317 <== mon.0 v2:172.21.15.96:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb7c00264f0 con 0x5606c55c3730 2024-04-25T14:14:46.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.759+0000 7fb7e1db3740 10 client.16002 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:14:46.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.759+0000 7fb7e1db3740 10 client.16002 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:14:46.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.759+0000 7fb7e1db3740 20 client.16002 populate_metadata read hostname 'smithi096' 2024-04-25T14:14:46.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.759+0000 7fb7e1db3740 1 --2- 172.21.15.96:0/3664564317 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5606c5578630 0x5606c5a177c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:46.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.759+0000 7fb7e1db3740 4 client.16002 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T14:14:46.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.759+0000 7fb7e1db3740 1 -- 172.21.15.96:0/3664564317 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5606c58f2580 con 0x5606c5578630 2024-04-25T14:14:46.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.760+0000 7fb7d8c9c640 1 --2- 172.21.15.96:0/3664564317 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5606c5578630 0x5606c5a177c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:46.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.760+0000 7fb7b3fff640 10 client.16002 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:46.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.760+0000 7fb7d8c9c640 1 --2- 172.21.15.96:0/3664564317 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5606c5578630 0x5606c5a177c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83c4d2594e8aab6e server_cookie=71c3ef3fa61263cb in_seq=0 out_seq=0 2024-04-25T14:14:46.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.761+0000 7fb7b3fff640 1 -- 172.21.15.96:0/3664564317 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5606c58f2580 con 0x5606c5578630 2024-04-25T14:14:46.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.761+0000 7fb7b3fff640 10 client.16002 handle_command_reply: tid=0 2024-04-25T14:14:46.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.761+0000 7fb7e1db3740 1 client.16002 shutdown 2024-04-25T14:14:46.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.762+0000 7fb7e1db3740 1 -- 172.21.15.96:0/3664564317 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606c55c3730 msgr2=0x5606c59a4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:46.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.762+0000 7fb7e1db3740 1 --2- 172.21.15.96:0/3664564317 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606c55c3730 0x5606c59a4c40 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0001000 tx=0x7fb7c0026d00 comp rx=0 tx=0).stop 2024-04-25T14:14:46.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.762+0000 7fb7e1db3740 1 -- 172.21.15.96:0/3664564317 shutdown_connections 2024-04-25T14:14:46.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.762+0000 7fb7e1db3740 1 --2- 172.21.15.96:0/3664564317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5606c55e4410 0x5606c5a139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.762+0000 7fb7e1db3740 1 --2- 172.21.15.96:0/3664564317 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5606c5578630 0x5606c5a177c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.762+0000 7fb7e1db3740 1 --2- 172.21.15.96:0/3664564317 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606c55c3730 0x5606c59a4c40 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.762+0000 7fb7b3fff640 0 client.16002 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:46.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.762+0000 7fb7e1db3740 1 --2- 172.21.15.96:0/3664564317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606c55a31f0 0x5606c59a4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.762+0000 7fb7e1db3740 1 -- 172.21.15.96:0/3664564317 >> 172.21.15.96:0/3664564317 conn(0x5606c55d8560 msgr2=0x5606c59843c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:46.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.762+0000 7fb7e1db3740 1 -- 172.21.15.96:0/3664564317 shutdown_connections 2024-04-25T14:14:46.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.762+0000 7fb7e1db3740 1 -- 172.21.15.96:0/3664564317 wait complete. 2024-04-25T14:14:46.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.763+0000 7fb7e1db3740 20 client.16002 trim_cache size 0 max 16384 2024-04-25T14:14:46.767 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T14:14:46.767 INFO:teuthology.orchestra.run.smithi096.stdout: "return_code": 0, 2024-04-25T14:14:46.767 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_tag": "628b0723-59f2-4324-8004-34bdabeceede", 2024-04-25T14:14:46.768 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": "asynchronous" 2024-04-25T14:14:46.768 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T14:14:46.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.763+0000 7fb7daf00640 1 -- 172.21.15.96:0/4029418213 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb7c40afb70 msgr2=0x7fb7c40b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:46.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.763+0000 7fb7daf00640 1 --2- 172.21.15.96:0/4029418213 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb7c40afb70 0x7fb7c40b2080 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4069bf0 tx=0x7fb7c8005f90 comp rx=0 tx=0).stop 2024-04-25T14:14:46.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.763+0000 7fb7daf00640 1 -- 172.21.15.96:0/4029418213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7d4105090 msgr2=0x7fb7d41a1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:46.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.763+0000 7fb7daf00640 1 --2- 172.21.15.96:0/4029418213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7d4105090 0x7fb7d41a1670 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc0097b0 tx=0x7fb7cc031840 comp rx=0 tx=0).stop 2024-04-25T14:14:46.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.764+0000 7fb7daf00640 1 -- 172.21.15.96:0/4029418213 shutdown_connections 2024-04-25T14:14:46.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.764+0000 7fb7daf00640 1 --2- 172.21.15.96:0/4029418213 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb7c40afb70 0x7fb7c40b2080 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.764+0000 7fb7daf00640 1 --2- 172.21.15.96:0/4029418213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7d4105090 0x7fb7d41a1670 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.764+0000 7fb7daf00640 1 --2- 172.21.15.96:0/4029418213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7d4102240 0x7fb7d419a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.764+0000 7fb7daf00640 1 --2- 172.21.15.96:0/4029418213 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7d4101740 0x7fb7d419a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.764+0000 7fb7daf00640 1 -- 172.21.15.96:0/4029418213 >> 172.21.15.96:0/4029418213 conn(0x7fb7d40fd340 msgr2=0x7fb7d4107c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:46.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.764+0000 7fb7daf00640 1 -- 172.21.15.96:0/4029418213 shutdown_connections 2024-04-25T14:14:46.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.764+0000 7fb7daf00640 1 -- 172.21.15.96:0/4029418213 wait complete. 2024-04-25T14:14:46.775 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:14:46.775 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:14:46.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:46 smithi096 ceph-mon[29207]: pgmap v380: 97 pgs: 97 active+clean; 668 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T14:14:46.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.888+0000 7f191917d640 1 -- 172.21.15.96:0/424415792 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1914108920 msgr2=0x7f191410ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:46.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.888+0000 7f191917d640 1 --2- 172.21.15.96:0/424415792 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1914108920 0x7f191410ad40 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f190c009ca0 tx=0x7f190c02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.888+0000 7f191917d640 1 -- 172.21.15.96:0/424415792 shutdown_connections 2024-04-25T14:14:46.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.888+0000 7f191917d640 1 --2- 172.21.15.96:0/424415792 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1914108920 0x7f191410ad40 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.888+0000 7f191917d640 1 --2- 172.21.15.96:0/424415792 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1914103c20 0x7f1914104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.888+0000 7f191917d640 1 --2- 172.21.15.96:0/424415792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19141029e0 0x7f19141036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.888+0000 7f191917d640 1 -- 172.21.15.96:0/424415792 >> 172.21.15.96:0/424415792 conn(0x7f19140fdbd0 msgr2=0x7f1914100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:46.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.888+0000 7f191917d640 1 -- 172.21.15.96:0/424415792 shutdown_connections 2024-04-25T14:14:46.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.888+0000 7f191917d640 1 -- 172.21.15.96:0/424415792 wait complete. 2024-04-25T14:14:46.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.889+0000 7f191917d640 1 Processor -- start 2024-04-25T14:14:46.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.889+0000 7f191917d640 1 -- start start 2024-04-25T14:14:46.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.889+0000 7f191917d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19141029e0 0x7f191419a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:46.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.889+0000 7f191917d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1914103c20 0x7f191419aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:46.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.889+0000 7f191917d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1914108920 0x7f19141a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:46.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.889+0000 7f191917d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f191410f610 con 0x7f1914108920 2024-04-25T14:14:46.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.889+0000 7f191917d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f191410f490 con 0x7f19141029e0 2024-04-25T14:14:46.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.889+0000 7f191917d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f191410f790 con 0x7f1914103c20 2024-04-25T14:14:46.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.890+0000 7f191359e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1914108920 0x7f19141a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:46.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.890+0000 7f191359e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1914108920 0x7f19141a19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34870/0 (socket says 172.21.15.96:34870) 2024-04-25T14:14:46.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.890+0000 7f191359e640 1 -- 172.21.15.96:0/2082163545 learned_addr learned my addr 172.21.15.96:0/2082163545 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:46.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.890+0000 7f1912d9d640 1 --2- 172.21.15.96:0/2082163545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19141029e0 0x7f191419a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:46.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.890+0000 7f191259c640 1 --2- 172.21.15.96:0/2082163545 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1914103c20 0x7f191419aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:46.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.890+0000 7f191359e640 1 -- 172.21.15.96:0/2082163545 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1914103c20 msgr2=0x7f191419aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:46.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.890+0000 7f191359e640 1 --2- 172.21.15.96:0/2082163545 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1914103c20 0x7f191419aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.890+0000 7f191359e640 1 -- 172.21.15.96:0/2082163545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19141029e0 msgr2=0x7f191419a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:46.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.890+0000 7f191359e640 1 --2- 172.21.15.96:0/2082163545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19141029e0 0x7f191419a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.890+0000 7f191359e640 1 -- 172.21.15.96:0/2082163545 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19141a2040 con 0x7f1914108920 2024-04-25T14:14:46.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.890+0000 7f191359e640 1 --2- 172.21.15.96:0/2082163545 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1914108920 0x7f19141a19d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f190c0097b0 tx=0x7f190c031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:46.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.890+0000 7f18fbfff640 1 -- 172.21.15.96:0/2082163545 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190c003bf0 con 0x7f1914108920 2024-04-25T14:14:46.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.890+0000 7f18fbfff640 1 -- 172.21.15.96:0/2082163545 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f190c003d90 con 0x7f1914108920 2024-04-25T14:14:46.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.891+0000 7f191917d640 1 -- 172.21.15.96:0/2082163545 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19141a22d0 con 0x7f1914108920 2024-04-25T14:14:46.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.891+0000 7f18fbfff640 1 -- 172.21.15.96:0/2082163545 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190c02fc30 con 0x7f1914108920 2024-04-25T14:14:46.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.891+0000 7f191917d640 1 -- 172.21.15.96:0/2082163545 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19141a2820 con 0x7f1914108920 2024-04-25T14:14:46.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.893+0000 7f18fbfff640 1 -- 172.21.15.96:0/2082163545 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f190c004040 con 0x7f1914108920 2024-04-25T14:14:46.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.894+0000 7f192002b740 1 Processor -- start 2024-04-25T14:14:46.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.894+0000 7f192002b740 1 -- start start 2024-04-25T14:14:46.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.894+0000 7f192002b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da3a5f6390 0x55da3a99be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:46.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.894+0000 7f192002b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da3a5cd4d0 0x55da3a99c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:46.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.894+0000 7f192002b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da3a5d56b0 0x55da3a99d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:46.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.894+0000 7f192002b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55da3a8fcd80 con 0x55da3a5d56b0 2024-04-25T14:14:46.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.894+0000 7f192002b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55da3a8b68b0 con 0x55da3a5f6390 2024-04-25T14:14:46.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.894+0000 7f192002b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55da3a8f8d40 con 0x55da3a5cd4d0 2024-04-25T14:14:46.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.894+0000 7f191259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da3a5d56b0 0x55da3a99d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:46.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.894+0000 7f191259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da3a5d56b0 0x55da3a99d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34872/0 (socket says 172.21.15.96:34872) 2024-04-25T14:14:46.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.894+0000 7f191259c640 1 -- 172.21.15.96:0/3168025776 learned_addr learned my addr 172.21.15.96:0/3168025776 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:46.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.894+0000 7f18fbfff640 1 --2- 172.21.15.96:0/2082163545 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f18fc0afaf0 0x7f18fc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:46.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.894+0000 7f1912d9d640 1 --2- 172.21.15.96:0/3168025776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da3a5cd4d0 0x55da3a99c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:46.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.895+0000 7f191259c640 1 -- 172.21.15.96:0/3168025776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da3a5cd4d0 msgr2=0x55da3a99c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:46.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.895+0000 7f191259c640 1 --2- 172.21.15.96:0/3168025776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da3a5cd4d0 0x55da3a99c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.895+0000 7f191259c640 1 -- 172.21.15.96:0/3168025776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da3a5f6390 msgr2=0x55da3a99be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:46.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.895+0000 7f191259c640 1 --2- 172.21.15.96:0/3168025776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da3a5f6390 0x55da3a99be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.895+0000 7f191359e640 1 --2- 172.21.15.96:0/2082163545 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f18fc0afaf0 0x7f18fc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:46.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.895+0000 7f191259c640 1 -- 172.21.15.96:0/3168025776 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da3a887a40 con 0x55da3a5d56b0 2024-04-25T14:14:46.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.895+0000 7f18fbfff640 1 -- 172.21.15.96:0/2082163545 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f190c100280 con 0x7f1914108920 2024-04-25T14:14:46.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.895+0000 7f191259c640 1 --2- 172.21.15.96:0/3168025776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da3a5d56b0 0x55da3a99d080 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f19080058c0 tx=0x7f1908011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c911e9e449fbd56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:46.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.895+0000 7f18f97fa640 1 -- 172.21.15.96:0/3168025776 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190801f070 con 0x55da3a5d56b0 2024-04-25T14:14:46.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.895+0000 7f191359e640 1 --2- 172.21.15.96:0/2082163545 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f18fc0afaf0 0x7f18fc0b2000 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f190c0c86d0 tx=0x7f190c0c3ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:46.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.895+0000 7f18f97fa640 1 -- 172.21.15.96:0/3168025776 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f190801a8f0 con 0x55da3a5d56b0 2024-04-25T14:14:46.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.895+0000 7f18f97fa640 1 -- 172.21.15.96:0/3168025776 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190800f980 con 0x55da3a5d56b0 2024-04-25T14:14:46.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.896+0000 7f192002b740 1 -- 172.21.15.96:0/3168025776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da3a5d56b0 msgr2=0x55da3a99d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:46.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.896+0000 7f192002b740 1 --2- 172.21.15.96:0/3168025776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da3a5d56b0 0x55da3a99d080 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f19080058c0 tx=0x7f1908011810 comp rx=0 tx=0).stop 2024-04-25T14:14:46.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.896+0000 7f192002b740 1 -- 172.21.15.96:0/3168025776 shutdown_connections 2024-04-25T14:14:46.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.896+0000 7f192002b740 1 --2- 172.21.15.96:0/3168025776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da3a5d56b0 0x55da3a99d080 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.896+0000 7f192002b740 1 --2- 172.21.15.96:0/3168025776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da3a5cd4d0 0x55da3a99c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.896+0000 7f192002b740 1 --2- 172.21.15.96:0/3168025776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da3a5f6390 0x55da3a99be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.896+0000 7f192002b740 1 -- 172.21.15.96:0/3168025776 >> 172.21.15.96:0/3168025776 conn(0x55da3a5ea4e0 msgr2=0x55da3a993270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:46.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.896+0000 7f192002b740 1 -- 172.21.15.96:0/3168025776 shutdown_connections 2024-04-25T14:14:46.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.896+0000 7f192002b740 1 -- 172.21.15.96:0/3168025776 wait complete. 2024-04-25T14:14:46.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.897+0000 7f192002b740 1 -- start start 2024-04-25T14:14:46.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.897+0000 7f192002b740 1 Processor -- start 2024-04-25T14:14:46.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.897+0000 7f192002b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da3a5cd4d0 0x55da3a9a5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:46.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.897+0000 7f192002b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da3a5d56b0 0x55da3a9a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:46.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.897+0000 7f192002b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da3a5f6390 0x55da3a9ad250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:46.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.897+0000 7f192002b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55da3a99db90 con 0x55da3a5f6390 2024-04-25T14:14:46.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.897+0000 7f192002b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55da3a883d10 con 0x55da3a5cd4d0 2024-04-25T14:14:46.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.897+0000 7f192002b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55da3a99dd10 con 0x55da3a5d56b0 2024-04-25T14:14:46.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.897+0000 7f192002b740 10 client.0 _ll_register_callbacks cb 0x55da3a8b0210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:14:46.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.897+0000 7f1912d9d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da3a5d56b0 0x55da3a9a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:46.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.897+0000 7f191259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da3a5cd4d0 0x55da3a9a5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:46.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.898+0000 7f191259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da3a5cd4d0 0x55da3a9a5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:54402/0 (socket says 172.21.15.96:54402) 2024-04-25T14:14:46.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.898+0000 7f191259c640 1 -- 172.21.15.96:0/1867503362 learned_addr learned my addr 172.21.15.96:0/1867503362 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:46.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.898+0000 7f1912d9d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da3a5d56b0 0x55da3a9a62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.96:55088/0 (socket says 172.21.15.96:55088) 2024-04-25T14:14:46.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.898+0000 7f191259c640 1 --2- 172.21.15.96:0/1867503362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da3a5f6390 0x55da3a9ad250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:46.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.898+0000 7f1912d9d640 1 -- 172.21.15.96:0/1867503362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da3a5cd4d0 msgr2=0x55da3a9a5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:46.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.898+0000 7f1912d9d640 1 --2- 172.21.15.96:0/1867503362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da3a5cd4d0 0x55da3a9a5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.898+0000 7f1912d9d640 1 -- 172.21.15.96:0/1867503362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da3a5f6390 msgr2=0x55da3a9ad250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:46.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.898+0000 7f1912d9d640 1 --2- 172.21.15.96:0/1867503362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da3a5f6390 0x55da3a9ad250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.898+0000 7f1912d9d640 1 -- 172.21.15.96:0/1867503362 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da3a6d87d0 con 0x55da3a5d56b0 2024-04-25T14:14:46.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.898+0000 7f191259c640 1 --2- 172.21.15.96:0/1867503362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da3a5cd4d0 0x55da3a9a5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:14:46.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.898+0000 7f1912d9d640 1 --2- 172.21.15.96:0/1867503362 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da3a5d56b0 0x55da3a9a62f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x55da3a99bb90 tx=0x7f190400be60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf0fcb4c6f9c4e7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:46.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.898+0000 7f18f6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-25T14:14:46.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.898+0000 7f18f6ff5640 1 -- 172.21.15.96:0/1867503362 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1904005220 con 0x55da3a5d56b0 2024-04-25T14:14:46.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.898+0000 7f18f6ff5640 1 -- 172.21.15.96:0/1867503362 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f19040053c0 con 0x55da3a5d56b0 2024-04-25T14:14:46.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.898+0000 7f192002b740 1 -- 172.21.15.96:0/1867503362 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f190801a8f0 con 0x55da3a5d56b0 2024-04-25T14:14:46.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.899+0000 7f18f6ff5640 1 -- 172.21.15.96:0/1867503362 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1904003e70 con 0x55da3a5d56b0 2024-04-25T14:14:46.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.899+0000 7f192002b740 10 client.25447 fetch_fsmap learned FSMap version 26 2024-04-25T14:14:46.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.899+0000 7f192002b740 1 -- 172.21.15.96:0/1867503362 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55da3a60e3f0 con 0x55da3a5d56b0 2024-04-25T14:14:46.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.899+0000 7f18f6ff5640 1 -- 172.21.15.96:0/1867503362 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f190801a8f0 con 0x55da3a5d56b0 2024-04-25T14:14:46.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.899+0000 7f18f6ff5640 1 -- 172.21.15.96:0/1867503362 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1904019030 con 0x55da3a5d56b0 2024-04-25T14:14:46.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.899+0000 7f192002b740 10 client.25447 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:14:46.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.899+0000 7f192002b740 10 client.25447 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:14:46.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.899+0000 7f192002b740 20 client.25447 populate_metadata read hostname 'smithi096' 2024-04-25T14:14:46.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.899+0000 7f192002b740 1 --2- 172.21.15.96:0/1867503362 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55da3a5b4f60 0x55da3a9b3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:46.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.899+0000 7f192002b740 4 client.25447 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:14:46.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.900+0000 7f192002b740 1 -- 172.21.15.96:0/1867503362 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da3a90a990 con 0x55da3a5b4f60 2024-04-25T14:14:46.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.900+0000 7f191259c640 1 --2- 172.21.15.96:0/1867503362 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55da3a5b4f60 0x55da3a9b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:46.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.900+0000 7f18f6ff5640 10 client.25447 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:46.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.900+0000 7f191259c640 1 --2- 172.21.15.96:0/1867503362 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55da3a5b4f60 0x55da3a9b3800 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c0bbd34dd379136 server_cookie=f1fa5a1d0e7eadba in_seq=0 out_seq=0 2024-04-25T14:14:46.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.901+0000 7f18f6ff5640 1 -- 172.21.15.96:0/1867503362 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55da3a90a990 con 0x55da3a5b4f60 2024-04-25T14:14:46.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.901+0000 7f18f6ff5640 10 client.25447 handle_command_reply: tid=0 2024-04-25T14:14:46.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.901+0000 7f192002b740 1 client.25447 shutdown 2024-04-25T14:14:46.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.901+0000 7f192002b740 1 -- 172.21.15.96:0/1867503362 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da3a5d56b0 msgr2=0x55da3a9a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:46.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.901+0000 7f192002b740 1 --2- 172.21.15.96:0/1867503362 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da3a5d56b0 0x55da3a9a62f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x55da3a99bb90 tx=0x7f190400be60 comp rx=0 tx=0).stop 2024-04-25T14:14:46.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.901+0000 7f192002b740 1 -- 172.21.15.96:0/1867503362 shutdown_connections 2024-04-25T14:14:46.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.901+0000 7f192002b740 1 --2- 172.21.15.96:0/1867503362 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55da3a5b4f60 0x55da3a9b3800 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.901+0000 7f192002b740 1 --2- 172.21.15.96:0/1867503362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da3a5f6390 0x55da3a9ad250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.901+0000 7f192002b740 1 --2- 172.21.15.96:0/1867503362 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da3a5d56b0 0x55da3a9a62f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.901+0000 7f192002b740 1 --2- 172.21.15.96:0/1867503362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da3a5cd4d0 0x55da3a9a5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.901+0000 7f192002b740 1 -- 172.21.15.96:0/1867503362 >> 172.21.15.96:0/1867503362 conn(0x55da3a5ea4e0 msgr2=0x55da3a996180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:46.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.901+0000 7f18f6ff5640 0 client.25447 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:46.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.901+0000 7f192002b740 1 -- 172.21.15.96:0/1867503362 shutdown_connections 2024-04-25T14:14:46.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.901+0000 7f192002b740 1 -- 172.21.15.96:0/1867503362 wait complete. 2024-04-25T14:14:46.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.901+0000 7f192002b740 20 client.25447 trim_cache size 0 max 16384 2024-04-25T14:14:46.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.919+0000 7f192002b740 1 Processor -- start 2024-04-25T14:14:46.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.919+0000 7f192002b740 1 -- start start 2024-04-25T14:14:46.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.919+0000 7f192002b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da3a5cd4d0 0x55da3a992ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:46.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.919+0000 7f192002b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da3a5d56b0 0x55da3a9931e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:46.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.919+0000 7f192002b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da3a5f6390 0x55da3a9ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:46.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.919+0000 7f192002b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f19141a9130 con 0x55da3a5f6390 2024-04-25T14:14:46.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.919+0000 7f192002b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f19141aac70 con 0x55da3a5cd4d0 2024-04-25T14:14:46.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.919+0000 7f192002b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f19141a9800 con 0x55da3a5d56b0 2024-04-25T14:14:46.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.920+0000 7f191259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da3a5f6390 0x55da3a9ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:46.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.920+0000 7f191259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da3a5f6390 0x55da3a9ad170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34884/0 (socket says 172.21.15.96:34884) 2024-04-25T14:14:46.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.920+0000 7f191259c640 1 -- 172.21.15.96:0/2122281961 learned_addr learned my addr 172.21.15.96:0/2122281961 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:46.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.920+0000 7f191259c640 1 --2- 172.21.15.96:0/2122281961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da3a5cd4d0 0x55da3a992ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:46.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.920+0000 7f1912d9d640 1 --2- 172.21.15.96:0/2122281961 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da3a5d56b0 0x55da3a9931e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:46.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.920+0000 7f191259c640 1 -- 172.21.15.96:0/2122281961 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da3a5d56b0 msgr2=0x55da3a9931e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:46.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.920+0000 7f191259c640 1 --2- 172.21.15.96:0/2122281961 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da3a5d56b0 0x55da3a9931e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.920+0000 7f191259c640 1 -- 172.21.15.96:0/2122281961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da3a5cd4d0 msgr2=0x55da3a992ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:46.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.920+0000 7f191259c640 1 --2- 172.21.15.96:0/2122281961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da3a5cd4d0 0x55da3a992ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.920+0000 7f191259c640 1 -- 172.21.15.96:0/2122281961 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da3a5ce0f0 con 0x55da3a5f6390 2024-04-25T14:14:46.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.920+0000 7f191259c640 1 --2- 172.21.15.96:0/2122281961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da3a5f6390 0x55da3a9ad170 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f190801d040 tx=0x7f1908011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5db66b6f6fb9d8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:46.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.920+0000 7f18f8ff9640 1 -- 172.21.15.96:0/2122281961 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190801f070 con 0x55da3a5f6390 2024-04-25T14:14:46.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.921+0000 7f18f8ff9640 1 -- 172.21.15.96:0/2122281961 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19080035f0 con 0x55da3a5f6390 2024-04-25T14:14:46.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.921+0000 7f18f8ff9640 1 -- 172.21.15.96:0/2122281961 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190802a650 con 0x55da3a5f6390 2024-04-25T14:14:46.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:46 smithi097 ceph-mon[32720]: pgmap v380: 97 pgs: 97 active+clean; 668 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T14:14:46.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.921+0000 7f192002b740 1 -- 172.21.15.96:0/2122281961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da3a5f6390 msgr2=0x55da3a9ad170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:46.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.921+0000 7f192002b740 1 --2- 172.21.15.96:0/2122281961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da3a5f6390 0x55da3a9ad170 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f190801d040 tx=0x7f1908011810 comp rx=0 tx=0).stop 2024-04-25T14:14:46.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.922+0000 7f192002b740 1 -- 172.21.15.96:0/2122281961 shutdown_connections 2024-04-25T14:14:46.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.922+0000 7f192002b740 1 --2- 172.21.15.96:0/2122281961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da3a5f6390 0x55da3a9ad170 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.922+0000 7f192002b740 1 --2- 172.21.15.96:0/2122281961 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da3a5d56b0 0x55da3a9931e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.922+0000 7f192002b740 1 --2- 172.21.15.96:0/2122281961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da3a5cd4d0 0x55da3a992ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.922+0000 7f192002b740 1 -- 172.21.15.96:0/2122281961 >> 172.21.15.96:0/2122281961 conn(0x55da3a5ea4e0 msgr2=0x55da3a9a43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:46.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.922+0000 7f192002b740 1 -- 172.21.15.96:0/2122281961 shutdown_connections 2024-04-25T14:14:46.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.922+0000 7f192002b740 1 -- 172.21.15.96:0/2122281961 wait complete. 2024-04-25T14:14:46.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.922+0000 7f192002b740 1 -- start start 2024-04-25T14:14:46.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.922+0000 7f192002b740 1 Processor -- start 2024-04-25T14:14:46.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.922+0000 7f192002b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da3a5cd4d0 0x55da3a9aa680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:46.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.922+0000 7f192002b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da3a5d56b0 0x55da3a9aabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:46.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.922+0000 7f192002b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da3a5f6390 0x55da3aa19b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:46.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.922+0000 7f192002b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f19141aa5a0 con 0x55da3a5f6390 2024-04-25T14:14:46.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.922+0000 7f192002b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f19141a9ed0 con 0x55da3a5d56b0 2024-04-25T14:14:46.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.922+0000 7f192002b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f19141a7110 con 0x55da3a5cd4d0 2024-04-25T14:14:46.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.923+0000 7f192002b740 10 client.0 _ll_register_callbacks cb 0x7f1904004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:14:46.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.923+0000 7f191259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da3a5f6390 0x55da3aa19b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:46.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.923+0000 7f1912d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da3a5d56b0 0x55da3a9aabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:46.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.923+0000 7f191259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da3a5f6390 0x55da3aa19b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34892/0 (socket says 172.21.15.96:34892) 2024-04-25T14:14:46.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.923+0000 7f191259c640 1 -- 172.21.15.96:0/3047067336 learned_addr learned my addr 172.21.15.96:0/3047067336 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:46.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.923+0000 7f191259c640 1 --2- 172.21.15.96:0/3047067336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da3a5cd4d0 0x55da3a9aa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:46.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.923+0000 7f1912d9d640 1 -- 172.21.15.96:0/3047067336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da3a5cd4d0 msgr2=0x55da3a9aa680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:46.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.923+0000 7f1912d9d640 1 --2- 172.21.15.96:0/3047067336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da3a5cd4d0 0x55da3a9aa680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.923+0000 7f1912d9d640 1 -- 172.21.15.96:0/3047067336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da3a5f6390 msgr2=0x55da3aa19b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:46.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.923+0000 7f1912d9d640 1 --2- 172.21.15.96:0/3047067336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da3a5f6390 0x55da3aa19b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.923+0000 7f1912d9d640 1 -- 172.21.15.96:0/3047067336 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da3a8bd4b0 con 0x55da3a5d56b0 2024-04-25T14:14:46.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.923+0000 7f191259c640 1 --2- 172.21.15.96:0/3047067336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da3a5f6390 0x55da3aa19b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:14:46.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.923+0000 7f1912d9d640 1 --2- 172.21.15.96:0/3047067336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da3a5d56b0 0x55da3a9aabc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f190400c010 tx=0x7f19040041d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=437aadde487027fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:46.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.923+0000 7f18f6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T14:14:46.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.923+0000 7f18f6ff5640 1 -- 172.21.15.96:0/3047067336 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1904033060 con 0x55da3a5d56b0 2024-04-25T14:14:46.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.923+0000 7f192002b740 1 -- 172.21.15.96:0/3047067336 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19080035f0 con 0x55da3a5d56b0 2024-04-25T14:14:46.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.923+0000 7f18f6ff5640 1 -- 172.21.15.96:0/3047067336 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f1904005240 con 0x55da3a5d56b0 2024-04-25T14:14:46.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.924+0000 7f18f6ff5640 1 -- 172.21.15.96:0/3047067336 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f190402a040 con 0x55da3a5d56b0 2024-04-25T14:14:46.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.924+0000 7f192002b740 10 client.25523 fetch_fsmap learned FSMap version 26 2024-04-25T14:14:46.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.924+0000 7f192002b740 1 -- 172.21.15.96:0/3047067336 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55da3aa1ad30 con 0x55da3a5d56b0 2024-04-25T14:14:46.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.924+0000 7f18f6ff5640 1 -- 172.21.15.96:0/3047067336 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19080035f0 con 0x55da3a5d56b0 2024-04-25T14:14:46.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.924+0000 7f18f6ff5640 1 -- 172.21.15.96:0/3047067336 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1904019040 con 0x55da3a5d56b0 2024-04-25T14:14:46.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.924+0000 7f192002b740 10 client.25523 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:14:46.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.924+0000 7f192002b740 10 client.25523 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:14:46.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.924+0000 7f192002b740 20 client.25523 populate_metadata read hostname 'smithi096' 2024-04-25T14:14:46.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.924+0000 7f192002b740 1 --2- 172.21.15.96:0/3047067336 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55da3a58a3a0 0x55da3aa1d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:46.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.924+0000 7f192002b740 4 client.25523 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:14:46.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.924+0000 7f192002b740 1 -- 172.21.15.96:0/3047067336 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55da3a904800 con 0x55da3a58a3a0 2024-04-25T14:14:46.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.925+0000 7f191259c640 1 --2- 172.21.15.96:0/3047067336 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55da3a58a3a0 0x55da3aa1d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:46.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.925+0000 7f18f6ff5640 10 client.25523 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:46.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.925+0000 7f191259c640 1 --2- 172.21.15.96:0/3047067336 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55da3a58a3a0 0x55da3aa1d690 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74c5f6f40e99fe8e server_cookie=a74613843640e8ed in_seq=0 out_seq=0 2024-04-25T14:14:46.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.925+0000 7f18f6ff5640 1 -- 172.21.15.96:0/3047067336 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55da3a904800 con 0x55da3a58a3a0 2024-04-25T14:14:46.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.925+0000 7f18f6ff5640 10 client.25523 handle_command_reply: tid=0 2024-04-25T14:14:46.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.925+0000 7f192002b740 1 client.25523 shutdown 2024-04-25T14:14:46.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.925+0000 7f192002b740 1 -- 172.21.15.96:0/3047067336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da3a5d56b0 msgr2=0x55da3a9aabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:46.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.925+0000 7f192002b740 1 --2- 172.21.15.96:0/3047067336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da3a5d56b0 0x55da3a9aabc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f190400c010 tx=0x7f19040041d0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.925+0000 7f192002b740 1 -- 172.21.15.96:0/3047067336 shutdown_connections 2024-04-25T14:14:46.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.925+0000 7f192002b740 1 --2- 172.21.15.96:0/3047067336 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55da3a58a3a0 0x55da3aa1d690 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.925+0000 7f192002b740 1 --2- 172.21.15.96:0/3047067336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da3a5f6390 0x55da3aa19b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.925+0000 7f192002b740 1 --2- 172.21.15.96:0/3047067336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da3a5d56b0 0x55da3a9aabc0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.925+0000 7f192002b740 1 --2- 172.21.15.96:0/3047067336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da3a5cd4d0 0x55da3a9aa680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.925+0000 7f18f6ff5640 0 client.25523 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:46.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.925+0000 7f192002b740 1 -- 172.21.15.96:0/3047067336 >> 172.21.15.96:0/3047067336 conn(0x55da3a5ea4e0 msgr2=0x55da3a992c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:46.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.925+0000 7f192002b740 1 -- 172.21.15.96:0/3047067336 shutdown_connections 2024-04-25T14:14:46.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.925+0000 7f192002b740 1 -- 172.21.15.96:0/3047067336 wait complete. 2024-04-25T14:14:46.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.926+0000 7f192002b740 20 client.25523 trim_cache size 0 max 16384 2024-04-25T14:14:46.931 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T14:14:46.931 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T14:14:46.931 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": { 2024-04-25T14:14:46.931 INFO:teuthology.orchestra.run.smithi096.stdout: "628b0723-59f2-4324-8004-34bdabeceede": { 2024-04-25T14:14:46.932 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-04-25T14:14:46.932 INFO:teuthology.orchestra.run.smithi096.stdout: "tag": "628b0723-59f2-4324-8004-34bdabeceede", 2024-04-25T14:14:46.932 INFO:teuthology.orchestra.run.smithi096.stdout: "options": "recursive,force" 2024-04-25T14:14:46.932 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:14:46.932 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:14:46.932 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T14:14:46.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.926+0000 7f191917d640 1 -- 172.21.15.96:0/2082163545 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f18fc0afaf0 msgr2=0x7f18fc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:46.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.926+0000 7f191917d640 1 --2- 172.21.15.96:0/2082163545 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f18fc0afaf0 0x7f18fc0b2000 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f190c0c86d0 tx=0x7f190c0c3ee0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.926+0000 7f191917d640 1 -- 172.21.15.96:0/2082163545 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1914108920 msgr2=0x7f19141a19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:46.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.926+0000 7f191917d640 1 --2- 172.21.15.96:0/2082163545 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1914108920 0x7f19141a19d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f190c0097b0 tx=0x7f190c031840 comp rx=0 tx=0).stop 2024-04-25T14:14:46.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.926+0000 7f191917d640 1 -- 172.21.15.96:0/2082163545 shutdown_connections 2024-04-25T14:14:46.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.926+0000 7f191917d640 1 --2- 172.21.15.96:0/2082163545 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f18fc0afaf0 0x7f18fc0b2000 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.926+0000 7f191917d640 1 --2- 172.21.15.96:0/2082163545 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1914108920 0x7f19141a19d0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.926+0000 7f191917d640 1 --2- 172.21.15.96:0/2082163545 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1914103c20 0x7f191419aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.926+0000 7f191917d640 1 --2- 172.21.15.96:0/2082163545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19141029e0 0x7f191419a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:46.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.926+0000 7f191917d640 1 -- 172.21.15.96:0/2082163545 >> 172.21.15.96:0/2082163545 conn(0x7f19140fdbd0 msgr2=0x7f1914104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:46.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.926+0000 7f191917d640 1 -- 172.21.15.96:0/2082163545 shutdown_connections 2024-04-25T14:14:46.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:14:46.927+0000 7f191917d640 1 -- 172.21.15.96:0/2082163545 wait complete. 2024-04-25T14:14:46.937 INFO:tasks.cephfs.filesystem:scrub status for tag:628b0723-59f2-4324-8004-34bdabeceede - {'path': '/', 'tag': '628b0723-59f2-4324-8004-34bdabeceede', 'options': 'recursive,force'} 2024-04-25T14:14:46.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:46 smithi151 ceph-mon[32990]: pgmap v380: 97 pgs: 97 active+clean; 668 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T14:14:47.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:47 smithi096 ceph-mon[29207]: scrub queued for path: / 2024-04-25T14:14:47.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:47 smithi096 ceph-mon[29207]: scrub summary: idle+waiting paths [/] 2024-04-25T14:14:47.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:47 smithi096 ceph-mon[29207]: scrub summary: active paths [/] 2024-04-25T14:14:47.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:47 smithi096 ceph-mon[29207]: scrub summary: idle+waiting paths [/] 2024-04-25T14:14:47.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:47 smithi097 ceph-mon[32720]: scrub queued for path: / 2024-04-25T14:14:47.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:47 smithi097 ceph-mon[32720]: scrub summary: idle+waiting paths [/] 2024-04-25T14:14:47.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:47 smithi097 ceph-mon[32720]: scrub summary: active paths [/] 2024-04-25T14:14:47.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:47 smithi097 ceph-mon[32720]: scrub summary: idle+waiting paths [/] 2024-04-25T14:14:47.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:47 smithi151 ceph-mon[32990]: scrub queued for path: / 2024-04-25T14:14:47.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:47 smithi151 ceph-mon[32990]: scrub summary: idle+waiting paths [/] 2024-04-25T14:14:47.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:47 smithi151 ceph-mon[32990]: scrub summary: active paths [/] 2024-04-25T14:14:47.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:47 smithi151 ceph-mon[32990]: scrub summary: idle+waiting paths [/] 2024-04-25T14:14:48.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:48 smithi096 ceph-mon[29207]: pgmap v381: 97 pgs: 97 active+clean; 669 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 12 op/s 2024-04-25T14:14:48.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:48 smithi097 ceph-mon[32720]: pgmap v381: 97 pgs: 97 active+clean; 669 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 12 op/s 2024-04-25T14:14:48.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:48 smithi151 ceph-mon[32990]: pgmap v381: 97 pgs: 97 active+clean; 669 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 12 op/s 2024-04-25T14:14:49.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:49 smithi097 ceph-mon[32720]: scrub summary: idle 2024-04-25T14:14:49.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:49 smithi097 ceph-mon[32720]: pgmap v382: 97 pgs: 97 active+clean; 669 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-25T14:14:49.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:49 smithi151 ceph-mon[32990]: scrub summary: idle 2024-04-25T14:14:49.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:49 smithi151 ceph-mon[32990]: pgmap v382: 97 pgs: 97 active+clean; 669 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-25T14:14:50.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:49 smithi096 ceph-mon[29207]: scrub summary: idle 2024-04-25T14:14:50.109 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:49 smithi096 ceph-mon[29207]: pgmap v382: 97 pgs: 97 active+clean; 669 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-25T14:14:51.422 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:51 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:14:51.437 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:51 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:14:51.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:51 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:14:52.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:52 smithi151 ceph-mon[32990]: pgmap v383: 97 pgs: 97 active+clean; 669 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-25T14:14:52.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:52 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:14:52.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:52 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:14:52.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:52 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:14:52.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:52 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:14:52.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:52 smithi096 ceph-mon[29207]: pgmap v383: 97 pgs: 97 active+clean; 669 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-25T14:14:52.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:52 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:14:52.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:52 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:14:52.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:52 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:14:52.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:52 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:14:52.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:52 smithi097 ceph-mon[32720]: pgmap v383: 97 pgs: 97 active+clean; 669 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-25T14:14:52.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:52 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:14:52.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:52 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:14:52.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:52 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:14:52.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:52 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:14:53.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:53 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:14:53.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:53 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:14:53.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:53 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:14:53.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:53 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:14:53.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:53 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:14:53.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:53 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:14:53.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:53 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:14:53.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:53 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:14:53.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:53 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:14:54.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:54 smithi151 ceph-mon[32990]: pgmap v384: 97 pgs: 97 active+clean; 669 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-25T14:14:54.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:54 smithi096 ceph-mon[29207]: pgmap v384: 97 pgs: 97 active+clean; 669 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-25T14:14:54.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:54 smithi097 ceph-mon[32720]: pgmap v384: 97 pgs: 97 active+clean; 669 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-25T14:14:56.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:56 smithi096 ceph-mon[29207]: pgmap v385: 97 pgs: 97 active+clean; 669 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-25T14:14:56.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:56 smithi097 ceph-mon[32720]: pgmap v385: 97 pgs: 97 active+clean; 669 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-25T14:14:56.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:56 smithi151 ceph-mon[32990]: pgmap v385: 97 pgs: 97 active+clean; 669 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-25T14:14:57.564 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.564+0000 7fdce42f9640 1 -- 172.21.15.96:0/3025513522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcdc102a50 msgr2=0x7fdcdc103750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:57.564 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.564+0000 7fdce42f9640 1 --2- 172.21.15.96:0/3025513522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcdc102a50 0x7fdcdc103750 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8009900 tx=0x7fdcd802a410 comp rx=0 tx=0).stop 2024-04-25T14:14:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.564+0000 7fdce42f9640 1 -- 172.21.15.96:0/3025513522 shutdown_connections 2024-04-25T14:14:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.564+0000 7fdce42f9640 1 --2- 172.21.15.96:0/3025513522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcdc108990 0x7fdcdc10adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.564+0000 7fdce42f9640 1 --2- 172.21.15.96:0/3025513522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcdc103c90 0x7fdcdc1040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.564+0000 7fdce42f9640 1 --2- 172.21.15.96:0/3025513522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcdc102a50 0x7fdcdc103750 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.564+0000 7fdce42f9640 1 -- 172.21.15.96:0/3025513522 >> 172.21.15.96:0/3025513522 conn(0x7fdcdc0fdc60 msgr2=0x7fdcdc1000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.564+0000 7fdce42f9640 1 -- 172.21.15.96:0/3025513522 shutdown_connections 2024-04-25T14:14:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.564+0000 7fdce42f9640 1 -- 172.21.15.96:0/3025513522 wait complete. 2024-04-25T14:14:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.565+0000 7fdce42f9640 1 Processor -- start 2024-04-25T14:14:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.565+0000 7fdce42f9640 1 -- start start 2024-04-25T14:14:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.566+0000 7fdce42f9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcdc102a50 0x7fdcdc1a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.566+0000 7fdce42f9640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcdc103c90 0x7fdcdc1a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:57.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.566+0000 7fdce42f9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcdc108990 0x7fdcdc1a7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:57.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.566+0000 7fdce2095640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcdc102a50 0x7fdcdc1a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:57.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.566+0000 7fdce42f9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc10d4b0 con 0x7fdcdc102a50 2024-04-25T14:14:57.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.566+0000 7fdce2095640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcdc102a50 0x7fdcdc1a0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35674/0 (socket says 172.21.15.96:35674) 2024-04-25T14:14:57.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.566+0000 7fdce2095640 1 -- 172.21.15.96:0/2166086563 learned_addr learned my addr 172.21.15.96:0/2166086563 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:57.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.566+0000 7fdce42f9640 1 -- 172.21.15.96:0/2166086563 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc10d330 con 0x7fdcdc108990 2024-04-25T14:14:57.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.566+0000 7fdce42f9640 1 -- 172.21.15.96:0/2166086563 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc10d630 con 0x7fdcdc103c90 2024-04-25T14:14:57.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.566+0000 7fdce1894640 1 --2- 172.21.15.96:0/2166086563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcdc103c90 0x7fdcdc1a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:57.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.566+0000 7fdce2896640 1 --2- 172.21.15.96:0/2166086563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcdc108990 0x7fdcdc1a7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:57.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.566+0000 7fdce2095640 1 -- 172.21.15.96:0/2166086563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcdc103c90 msgr2=0x7fdcdc1a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:57.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.566+0000 7fdce2095640 1 --2- 172.21.15.96:0/2166086563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcdc103c90 0x7fdcdc1a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:57.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.566+0000 7fdce2095640 1 -- 172.21.15.96:0/2166086563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcdc108990 msgr2=0x7fdcdc1a7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:57.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.566+0000 7fdce2095640 1 --2- 172.21.15.96:0/2166086563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcdc108990 0x7fdcdc1a7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:57.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.566+0000 7fdce2095640 1 -- 172.21.15.96:0/2166086563 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcdc1a83a0 con 0x7fdcdc102a50 2024-04-25T14:14:57.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.567+0000 7fdce2896640 1 --2- 172.21.15.96:0/2166086563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcdc108990 0x7fdcdc1a7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:14:57.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.567+0000 7fdce2095640 1 --2- 172.21.15.96:0/2166086563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcdc102a50 0x7fdcdc1a0860 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8031440 tx=0x7fdcd8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:57.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.567+0000 7fdcd37fe640 1 -- 172.21.15.96:0/2166086563 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd803f070 con 0x7fdcdc102a50 2024-04-25T14:14:57.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.567+0000 7fdcd37fe640 1 -- 172.21.15.96:0/2166086563 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcd8031c50 con 0x7fdcdc102a50 2024-04-25T14:14:57.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.567+0000 7fdcd37fe640 1 -- 172.21.15.96:0/2166086563 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd8035030 con 0x7fdcdc102a50 2024-04-25T14:14:57.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.567+0000 7fdce42f9640 1 -- 172.21.15.96:0/2166086563 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcdc1a8630 con 0x7fdcdc102a50 2024-04-25T14:14:57.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.567+0000 7fdce42f9640 1 -- 172.21.15.96:0/2166086563 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcdc1a8b70 con 0x7fdcdc102a50 2024-04-25T14:14:57.570 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.570+0000 7fdcd37fe640 1 -- 172.21.15.96:0/2166086563 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fdcd8042070 con 0x7fdcdc102a50 2024-04-25T14:14:57.571 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.571+0000 7fdceb1ae740 1 Processor -- start 2024-04-25T14:14:57.571 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.571+0000 7fdceb1ae740 1 -- start start 2024-04-25T14:14:57.572 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.572+0000 7fdcd37fe640 1 --2- 172.21.15.96:0/2166086563 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fdcc80afb50 0x7fdcc80b2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:57.572 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.572+0000 7fdceb1ae740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651d55ed1b0 0x5651d598d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:57.572 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.572+0000 7fdceb1ae740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651d55c42f0 0x5651d598def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:57.573 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.572+0000 7fdce2896640 1 --2- 172.21.15.96:0/2166086563 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fdcc80afb50 0x7fdcc80b2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:57.573 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.572+0000 7fdce2095640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651d55ed1b0 0x5651d598d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:57.573 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.572+0000 7fdce2095640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651d55ed1b0 0x5651d598d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.96:54088/0 (socket says 172.21.15.96:54088) 2024-04-25T14:14:57.573 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.572+0000 7fdce2095640 1 -- 172.21.15.96:0/1196354489 learned_addr learned my addr 172.21.15.96:0/1196354489 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:57.573 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.572+0000 7fdceb1ae740 1 --2- 172.21.15.96:0/1196354489 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651d55cc4d0 0x5651d5994df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:57.573 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.572+0000 7fdce1894640 1 --2- 172.21.15.96:0/1196354489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651d55c42f0 0x5651d598def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:57.573 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.572+0000 7fdceb1ae740 1 -- 172.21.15.96:0/1196354489 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5651d58f3460 con 0x5651d55c42f0 2024-04-25T14:14:57.573 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.573+0000 7fdceb1ae740 1 -- 172.21.15.96:0/1196354489 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5651d58ad590 con 0x5651d55cc4d0 2024-04-25T14:14:57.573 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.573+0000 7fdceb1ae740 1 -- 172.21.15.96:0/1196354489 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5651d58ef420 con 0x5651d55ed1b0 2024-04-25T14:14:57.574 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.573+0000 7fdcd37fe640 1 -- 172.21.15.96:0/2166086563 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fdcd80ff3c0 con 0x7fdcdc102a50 2024-04-25T14:14:57.574 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.573+0000 7fdce2095640 1 -- 172.21.15.96:0/1196354489 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651d55cc4d0 msgr2=0x5651d5994df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:57.574 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.573+0000 7fdce2095640 1 --2- 172.21.15.96:0/1196354489 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651d55cc4d0 0x5651d5994df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:57.574 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.573+0000 7fdce2095640 1 -- 172.21.15.96:0/1196354489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651d55c42f0 msgr2=0x5651d598def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:57.574 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.573+0000 7fdce2095640 1 --2- 172.21.15.96:0/1196354489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651d55c42f0 0x5651d598def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:57.574 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.573+0000 7fdce2095640 1 -- 172.21.15.96:0/1196354489 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651d58ba0d0 con 0x5651d55ed1b0 2024-04-25T14:14:57.575 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.573+0000 7fdce2896640 1 --2- 172.21.15.96:0/2166086563 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fdcc80afb50 0x7fdcc80b2060 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4005fd0 tx=0x7fdcd4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:57.575 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.573+0000 7fdce2095640 1 --2- 172.21.15.96:0/1196354489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651d55ed1b0 0x5651d598d970 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8006010 tx=0x7fdcd80c3870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c2227b8dc34ee0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:57.575 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.574+0000 7fdcd0bfe640 1 -- 172.21.15.96:0/1196354489 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd8049070 con 0x5651d55ed1b0 2024-04-25T14:14:57.575 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.574+0000 7fdcd0bfe640 1 -- 172.21.15.96:0/1196354489 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcd8031e00 con 0x5651d55ed1b0 2024-04-25T14:14:57.575 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.574+0000 7fdcd0bfe640 1 -- 172.21.15.96:0/1196354489 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd803b460 con 0x5651d55ed1b0 2024-04-25T14:14:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.575+0000 7fdceb1ae740 1 -- 172.21.15.96:0/1196354489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651d55ed1b0 msgr2=0x5651d598d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.575+0000 7fdceb1ae740 1 --2- 172.21.15.96:0/1196354489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651d55ed1b0 0x5651d598d970 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8006010 tx=0x7fdcd80c3870 comp rx=0 tx=0).stop 2024-04-25T14:14:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.575+0000 7fdceb1ae740 1 -- 172.21.15.96:0/1196354489 shutdown_connections 2024-04-25T14:14:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.575+0000 7fdceb1ae740 1 --2- 172.21.15.96:0/1196354489 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651d55cc4d0 0x5651d5994df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.575+0000 7fdceb1ae740 1 --2- 172.21.15.96:0/1196354489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651d55c42f0 0x5651d598def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.575+0000 7fdceb1ae740 1 --2- 172.21.15.96:0/1196354489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651d55ed1b0 0x5651d598d970 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.575+0000 7fdceb1ae740 1 -- 172.21.15.96:0/1196354489 >> 172.21.15.96:0/1196354489 conn(0x5651d55e1300 msgr2=0x5651d5989300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.575+0000 7fdceb1ae740 1 -- 172.21.15.96:0/1196354489 shutdown_connections 2024-04-25T14:14:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.575+0000 7fdceb1ae740 1 -- 172.21.15.96:0/1196354489 wait complete. 2024-04-25T14:14:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.576+0000 7fdceb1ae740 1 -- start start 2024-04-25T14:14:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.576+0000 7fdceb1ae740 1 Processor -- start 2024-04-25T14:14:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.576+0000 7fdceb1ae740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651d55c42f0 0x5651d5997370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.576+0000 7fdceb1ae740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651d55cc4d0 0x5651d59978b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.576+0000 7fdceb1ae740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651d55ed1b0 0x5651d599e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.576+0000 7fdceb1ae740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5651d5995600 con 0x5651d55c42f0 2024-04-25T14:14:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.576+0000 7fdceb1ae740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5651d587ab60 con 0x5651d55ed1b0 2024-04-25T14:14:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.576+0000 7fdceb1ae740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5651d5995780 con 0x5651d55cc4d0 2024-04-25T14:14:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.576+0000 7fdce2095640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651d55c42f0 0x5651d5997370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.576+0000 7fdceb1ae740 10 client.0 _ll_register_callbacks cb 0x5651d589f2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:14:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.576+0000 7fdce2095640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651d55c42f0 0x5651d5997370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35694/0 (socket says 172.21.15.96:35694) 2024-04-25T14:14:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.576+0000 7fdce2095640 1 -- 172.21.15.96:0/2834402338 learned_addr learned my addr 172.21.15.96:0/2834402338 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.576+0000 7fdce2896640 1 --2- 172.21.15.96:0/2834402338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651d55ed1b0 0x5651d599e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.576+0000 7fdce1894640 1 --2- 172.21.15.96:0/2834402338 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651d55cc4d0 0x5651d59978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.576+0000 7fdce2095640 1 -- 172.21.15.96:0/2834402338 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651d55cc4d0 msgr2=0x5651d59978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.576+0000 7fdce2095640 1 --2- 172.21.15.96:0/2834402338 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651d55cc4d0 0x5651d59978b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.576+0000 7fdce2095640 1 -- 172.21.15.96:0/2834402338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651d55ed1b0 msgr2=0x5651d599e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.576+0000 7fdce2095640 1 --2- 172.21.15.96:0/2834402338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651d55ed1b0 0x5651d599e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.576+0000 7fdce2095640 1 -- 172.21.15.96:0/2834402338 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651d5605210 con 0x5651d55c42f0 2024-04-25T14:14:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.577+0000 7fdce2095640 1 --2- 172.21.15.96:0/2834402338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651d55c42f0 0x5651d5997370 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fdcd80c7810 tx=0x7fdcd803b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c17d3ee5058ba19 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.577+0000 7fdcc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:14:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.577+0000 7fdcc5ffb640 1 -- 172.21.15.96:0/2834402338 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd8049070 con 0x5651d55c42f0 2024-04-25T14:14:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.577+0000 7fdceb1ae740 1 -- 172.21.15.96:0/2834402338 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5651d5901070 con 0x5651d55c42f0 2024-04-25T14:14:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.577+0000 7fdcc5ffb640 1 -- 172.21.15.96:0/2834402338 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fdcd80c3b10 con 0x5651d55c42f0 2024-04-25T14:14:57.580 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.577+0000 7fdcc5ffb640 1 -- 172.21.15.96:0/2834402338 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdcd80c7880 con 0x5651d55c42f0 2024-04-25T14:14:57.580 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.577+0000 7fdceb1ae740 10 client.16032 fetch_fsmap learned FSMap version 26 2024-04-25T14:14:57.580 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.577+0000 7fdceb1ae740 1 -- 172.21.15.96:0/2834402338 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5651d56d1620 con 0x5651d55c42f0 2024-04-25T14:14:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.578+0000 7fdcc5ffb640 1 -- 172.21.15.96:0/2834402338 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5651d5901070 con 0x5651d55c42f0 2024-04-25T14:14:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.578+0000 7fdcc5ffb640 1 -- 172.21.15.96:0/2834402338 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fdcd80d0250 con 0x5651d55c42f0 2024-04-25T14:14:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.578+0000 7fdceb1ae740 10 client.16032 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:14:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.578+0000 7fdceb1ae740 10 client.16032 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:14:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.578+0000 7fdceb1ae740 20 client.16032 populate_metadata read hostname 'smithi096' 2024-04-25T14:14:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.578+0000 7fdceb1ae740 1 --2- 172.21.15.96:0/2834402338 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5651d55abd80 0x5651d59a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.578+0000 7fdceb1ae740 4 client.16032 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:14:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.578+0000 7fdceb1ae740 1 -- 172.21.15.96:0/2834402338 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651d58faee0 con 0x5651d55abd80 2024-04-25T14:14:57.583 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.578+0000 7fdce1894640 1 --2- 172.21.15.96:0/2834402338 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5651d55abd80 0x5651d59a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:57.583 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.579+0000 7fdcc5ffb640 10 client.16032 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:57.583 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.579+0000 7fdce1894640 1 --2- 172.21.15.96:0/2834402338 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5651d55abd80 0x5651d59a4e80 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c070d8715f7529b server_cookie=142284fffc09c84e in_seq=0 out_seq=0 2024-04-25T14:14:57.583 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.580+0000 7fdcc5ffb640 1 -- 172.21.15.96:0/2834402338 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5651d58faee0 con 0x5651d55abd80 2024-04-25T14:14:57.583 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.580+0000 7fdcc5ffb640 10 client.16032 handle_command_reply: tid=0 2024-04-25T14:14:57.583 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.580+0000 7fdceb1ae740 1 client.16032 shutdown 2024-04-25T14:14:57.583 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.580+0000 7fdceb1ae740 1 -- 172.21.15.96:0/2834402338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651d55c42f0 msgr2=0x5651d5997370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:57.583 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.580+0000 7fdceb1ae740 1 --2- 172.21.15.96:0/2834402338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651d55c42f0 0x5651d5997370 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fdcd80c7810 tx=0x7fdcd803b6f0 comp rx=0 tx=0).stop 2024-04-25T14:14:57.583 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.581+0000 7fdceb1ae740 1 -- 172.21.15.96:0/2834402338 shutdown_connections 2024-04-25T14:14:57.583 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.581+0000 7fdceb1ae740 1 --2- 172.21.15.96:0/2834402338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651d55ed1b0 0x5651d599e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:57.583 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.581+0000 7fdceb1ae740 1 --2- 172.21.15.96:0/2834402338 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651d55cc4d0 0x5651d59978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:57.583 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.581+0000 7fdceb1ae740 1 --2- 172.21.15.96:0/2834402338 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5651d55abd80 0x5651d59a4e80 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:57.583 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.581+0000 7fdceb1ae740 1 --2- 172.21.15.96:0/2834402338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651d55c42f0 0x5651d5997370 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:57.583 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.581+0000 7fdceb1ae740 1 -- 172.21.15.96:0/2834402338 >> 172.21.15.96:0/2834402338 conn(0x5651d55e1300 msgr2=0x5651d5988250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:57.584 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.581+0000 7fdcc5ffb640 0 client.16032 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:57.584 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.581+0000 7fdceb1ae740 1 -- 172.21.15.96:0/2834402338 shutdown_connections 2024-04-25T14:14:57.584 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.581+0000 7fdceb1ae740 1 -- 172.21.15.96:0/2834402338 wait complete. 2024-04-25T14:14:57.584 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.582+0000 7fdceb1ae740 20 client.16032 trim_cache size 0 max 16384 2024-04-25T14:14:57.612 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.612+0000 7fdceb1ae740 1 Processor -- start 2024-04-25T14:14:57.612 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.612+0000 7fdceb1ae740 1 -- start start 2024-04-25T14:14:57.612 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.612+0000 7fdceb1ae740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651d55c42f0 0x5651d5988c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:57.612 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.612+0000 7fdceb1ae740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651d55cc4d0 0x5651d599e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:57.612 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.612+0000 7fdceb1ae740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651d55ed1b0 0x5651d59891b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:57.612 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.612+0000 7fdceb1ae740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc1b1880 con 0x5651d55c42f0 2024-04-25T14:14:57.612 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.612+0000 7fdceb1ae740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc1b33c0 con 0x5651d55ed1b0 2024-04-25T14:14:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.612+0000 7fdceb1ae740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc1b1f50 con 0x5651d55cc4d0 2024-04-25T14:14:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.612+0000 7fdce2095640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651d55c42f0 0x5651d5988c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.612+0000 7fdce2896640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651d55ed1b0 0x5651d59891b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.612+0000 7fdce1894640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651d55cc4d0 0x5651d599e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.612+0000 7fdce2896640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651d55ed1b0 0x5651d59891b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:33216/0 (socket says 172.21.15.96:33216) 2024-04-25T14:14:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.612+0000 7fdce2896640 1 -- 172.21.15.96:0/3222869669 learned_addr learned my addr 172.21.15.96:0/3222869669 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.613+0000 7fdce2896640 1 -- 172.21.15.96:0/3222869669 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651d55cc4d0 msgr2=0x5651d599e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.613+0000 7fdce2896640 1 --2- 172.21.15.96:0/3222869669 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651d55cc4d0 0x5651d599e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.613+0000 7fdce2896640 1 -- 172.21.15.96:0/3222869669 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651d55c42f0 msgr2=0x5651d5988c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.613+0000 7fdce2896640 1 --2- 172.21.15.96:0/3222869669 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651d55c42f0 0x5651d5988c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.613+0000 7fdce2896640 1 -- 172.21.15.96:0/3222869669 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651d5994ed0 con 0x5651d55ed1b0 2024-04-25T14:14:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.613+0000 7fdce1894640 1 --2- 172.21.15.96:0/3222869669 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651d55cc4d0 0x5651d599e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:14:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.613+0000 7fdce2896640 1 --2- 172.21.15.96:0/3222869669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651d55ed1b0 0x5651d59891b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdcd40119f0 tx=0x7fdcd4013070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb74de067d695572 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.613+0000 7fdcc7fff640 1 -- 172.21.15.96:0/3222869669 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd401eb80 con 0x5651d55ed1b0 2024-04-25T14:14:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.613+0000 7fdcc7fff640 1 -- 172.21.15.96:0/3222869669 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcd4023070 con 0x5651d55ed1b0 2024-04-25T14:14:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.613+0000 7fdcc7fff640 1 -- 172.21.15.96:0/3222869669 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd401c630 con 0x5651d55ed1b0 2024-04-25T14:14:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.615+0000 7fdceb1ae740 1 -- 172.21.15.96:0/3222869669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651d55ed1b0 msgr2=0x5651d59891b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.615+0000 7fdceb1ae740 1 --2- 172.21.15.96:0/3222869669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651d55ed1b0 0x5651d59891b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdcd40119f0 tx=0x7fdcd4013070 comp rx=0 tx=0).stop 2024-04-25T14:14:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.615+0000 7fdceb1ae740 1 -- 172.21.15.96:0/3222869669 shutdown_connections 2024-04-25T14:14:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.615+0000 7fdceb1ae740 1 --2- 172.21.15.96:0/3222869669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651d55ed1b0 0x5651d59891b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.615+0000 7fdceb1ae740 1 --2- 172.21.15.96:0/3222869669 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651d55cc4d0 0x5651d599e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.615+0000 7fdceb1ae740 1 --2- 172.21.15.96:0/3222869669 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651d55c42f0 0x5651d5988c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.615+0000 7fdceb1ae740 1 -- 172.21.15.96:0/3222869669 >> 172.21.15.96:0/3222869669 conn(0x5651d55e1300 msgr2=0x5651d5991b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.615+0000 7fdceb1ae740 1 -- 172.21.15.96:0/3222869669 shutdown_connections 2024-04-25T14:14:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.615+0000 7fdceb1ae740 1 -- 172.21.15.96:0/3222869669 wait complete. 2024-04-25T14:14:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.615+0000 7fdceb1ae740 1 -- start start 2024-04-25T14:14:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.615+0000 7fdceb1ae740 1 Processor -- start 2024-04-25T14:14:57.619 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.616+0000 7fdceb1ae740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651d55c42f0 0x5651d599b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:57.619 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.616+0000 7fdceb1ae740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651d55cc4d0 0x5651d599bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:57.619 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.616+0000 7fdce2095640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651d55c42f0 0x5651d599b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:57.619 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.616+0000 7fdce2095640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651d55c42f0 0x5651d599b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35706/0 (socket says 172.21.15.96:35706) 2024-04-25T14:14:57.619 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.616+0000 7fdce2095640 1 -- 172.21.15.96:0/795705241 learned_addr learned my addr 172.21.15.96:0/795705241 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:14:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.616+0000 7fdceb1ae740 1 --2- 172.21.15.96:0/795705241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651d55ed1b0 0x5651d5a0b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.616+0000 7fdceb1ae740 1 -- 172.21.15.96:0/795705241 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc1b2cf0 con 0x5651d55c42f0 2024-04-25T14:14:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.616+0000 7fdceb1ae740 1 -- 172.21.15.96:0/795705241 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc1b2620 con 0x5651d55ed1b0 2024-04-25T14:14:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.616+0000 7fdceb1ae740 1 -- 172.21.15.96:0/795705241 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc1af860 con 0x5651d55cc4d0 2024-04-25T14:14:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.616+0000 7fdce1894640 1 --2- 172.21.15.96:0/795705241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651d55cc4d0 0x5651d599bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.616+0000 7fdceb1ae740 10 client.0 _ll_register_callbacks cb 0x7fdccc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:14:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.616+0000 7fdce2896640 1 --2- 172.21.15.96:0/795705241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651d55ed1b0 0x5651d5a0b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.616+0000 7fdce1894640 1 -- 172.21.15.96:0/795705241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651d55ed1b0 msgr2=0x5651d5a0b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.616+0000 7fdce1894640 1 --2- 172.21.15.96:0/795705241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651d55ed1b0 0x5651d5a0b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.616+0000 7fdce1894640 1 -- 172.21.15.96:0/795705241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651d55c42f0 msgr2=0x5651d599b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.616+0000 7fdce1894640 1 --2- 172.21.15.96:0/795705241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651d55c42f0 0x5651d599b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.616+0000 7fdce1894640 1 -- 172.21.15.96:0/795705241 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651d5a0b760 con 0x5651d55cc4d0 2024-04-25T14:14:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.617+0000 7fdce1894640 1 --2- 172.21.15.96:0/795705241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651d55cc4d0 0x5651d599bca0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x5651d59a55f0 tx=0x7fdccc004200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=404e069283edf20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.617+0000 7fdcc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-25T14:14:57.621 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.617+0000 7fdcc5ffb640 1 -- 172.21.15.96:0/795705241 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdccc024030 con 0x5651d55cc4d0 2024-04-25T14:14:57.621 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.617+0000 7fdcc5ffb640 1 -- 172.21.15.96:0/795705241 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fdccc005020 con 0x5651d55cc4d0 2024-04-25T14:14:57.621 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.617+0000 7fdceb1ae740 1 -- 172.21.15.96:0/795705241 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcd4023070 con 0x5651d55cc4d0 2024-04-25T14:14:57.621 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.617+0000 7fdcc5ffb640 1 -- 172.21.15.96:0/795705241 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdccc005220 con 0x5651d55cc4d0 2024-04-25T14:14:57.621 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.617+0000 7fdceb1ae740 10 client.25471 fetch_fsmap learned FSMap version 26 2024-04-25T14:14:57.621 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.617+0000 7fdceb1ae740 1 -- 172.21.15.96:0/795705241 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5651d5a0c9f0 con 0x5651d55cc4d0 2024-04-25T14:14:57.621 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.617+0000 7fdcc5ffb640 1 -- 172.21.15.96:0/795705241 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdccc004c50 con 0x5651d55cc4d0 2024-04-25T14:14:57.621 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.618+0000 7fdcc5ffb640 1 -- 172.21.15.96:0/795705241 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fdccc0035f0 con 0x5651d55cc4d0 2024-04-25T14:14:57.621 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.618+0000 7fdceb1ae740 10 client.25471 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:14:57.621 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.618+0000 7fdceb1ae740 10 client.25471 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:14:57.621 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.618+0000 7fdceb1ae740 20 client.25471 populate_metadata read hostname 'smithi096' 2024-04-25T14:14:57.621 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.618+0000 7fdceb1ae740 1 --2- 172.21.15.96:0/795705241 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5651d55811c0 0x5651d5a0f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:57.621 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.618+0000 7fdceb1ae740 4 client.25471 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:14:57.621 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.618+0000 7fdceb1ae740 1 -- 172.21.15.96:0/795705241 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fdcd4023070 con 0x5651d55811c0 2024-04-25T14:14:57.622 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.618+0000 7fdce2095640 1 --2- 172.21.15.96:0/795705241 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5651d55811c0 0x5651d5a0f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:57.622 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.619+0000 7fdcc5ffb640 10 client.25471 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:57.622 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.619+0000 7fdce2095640 1 --2- 172.21.15.96:0/795705241 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5651d55811c0 0x5651d5a0f3e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8a634fa2d2aeea1 server_cookie=886d8e797f95b31b in_seq=0 out_seq=0 2024-04-25T14:14:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.622+0000 7fdcc5ffb640 1 -- 172.21.15.96:0/795705241 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fdcd4023070 con 0x5651d55811c0 2024-04-25T14:14:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.622+0000 7fdcc5ffb640 10 client.25471 handle_command_reply: tid=0 2024-04-25T14:14:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.622+0000 7fdceb1ae740 1 client.25471 shutdown 2024-04-25T14:14:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.622+0000 7fdceb1ae740 1 -- 172.21.15.96:0/795705241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651d55cc4d0 msgr2=0x5651d599bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.622+0000 7fdceb1ae740 1 --2- 172.21.15.96:0/795705241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651d55cc4d0 0x5651d599bca0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x5651d59a55f0 tx=0x7fdccc004200 comp rx=0 tx=0).stop 2024-04-25T14:14:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.623+0000 7fdceb1ae740 1 -- 172.21.15.96:0/795705241 shutdown_connections 2024-04-25T14:14:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.623+0000 7fdceb1ae740 1 --2- 172.21.15.96:0/795705241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651d55ed1b0 0x5651d5a0b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.623+0000 7fdceb1ae740 1 --2- 172.21.15.96:0/795705241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651d55cc4d0 0x5651d599bca0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.623+0000 7fdceb1ae740 1 --2- 172.21.15.96:0/795705241 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5651d55811c0 0x5651d5a0f3e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.623+0000 7fdceb1ae740 1 --2- 172.21.15.96:0/795705241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651d55c42f0 0x5651d599b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.623+0000 7fdceb1ae740 1 -- 172.21.15.96:0/795705241 >> 172.21.15.96:0/795705241 conn(0x5651d55e1300 msgr2=0x5651d5988c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.623+0000 7fdcc5ffb640 0 client.25471 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:14:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.623+0000 7fdceb1ae740 1 -- 172.21.15.96:0/795705241 shutdown_connections 2024-04-25T14:14:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.623+0000 7fdceb1ae740 1 -- 172.21.15.96:0/795705241 wait complete. 2024-04-25T14:14:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.623+0000 7fdceb1ae740 20 client.25471 trim_cache size 0 max 16384 2024-04-25T14:14:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T14:14:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T14:14:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T14:14:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T14:14:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.623+0000 7fdce42f9640 1 -- 172.21.15.96:0/2166086563 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fdcc80afb50 msgr2=0x7fdcc80b2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.624+0000 7fdce42f9640 1 --2- 172.21.15.96:0/2166086563 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fdcc80afb50 0x7fdcc80b2060 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4005fd0 tx=0x7fdcd4009500 comp rx=0 tx=0).stop 2024-04-25T14:14:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.624+0000 7fdce42f9640 1 -- 172.21.15.96:0/2166086563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcdc102a50 msgr2=0x7fdcdc1a0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.624+0000 7fdce42f9640 1 --2- 172.21.15.96:0/2166086563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcdc102a50 0x7fdcdc1a0860 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8031440 tx=0x7fdcd8003c10 comp rx=0 tx=0).stop 2024-04-25T14:14:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.624+0000 7fdce42f9640 1 -- 172.21.15.96:0/2166086563 shutdown_connections 2024-04-25T14:14:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.624+0000 7fdce42f9640 1 --2- 172.21.15.96:0/2166086563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcdc108990 0x7fdcdc1a7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.624+0000 7fdce42f9640 1 --2- 172.21.15.96:0/2166086563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcdc103c90 0x7fdcdc1a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.624+0000 7fdce42f9640 1 --2- 172.21.15.96:0/2166086563 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fdcc80afb50 0x7fdcc80b2060 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.624+0000 7fdce42f9640 1 --2- 172.21.15.96:0/2166086563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcdc102a50 0x7fdcdc1a0860 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.624+0000 7fdce42f9640 1 -- 172.21.15.96:0/2166086563 >> 172.21.15.96:0/2166086563 conn(0x7fdcdc0fdc60 msgr2=0x7fdcdc104970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.624+0000 7fdce42f9640 1 -- 172.21.15.96:0/2166086563 shutdown_connections 2024-04-25T14:14:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:14:57.624+0000 7fdce42f9640 1 -- 172.21.15.96:0/2166086563 wait complete. 2024-04-25T14:14:58.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:14:58 smithi096 ceph-mon[29207]: pgmap v386: 97 pgs: 97 active+clean; 669 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-25T14:14:58.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:14:58 smithi097 ceph-mon[32720]: pgmap v386: 97 pgs: 97 active+clean; 669 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-25T14:14:58.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:14:58 smithi151 ceph-mon[32990]: pgmap v386: 97 pgs: 97 active+clean; 669 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-25T14:15:00.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:00 smithi096 ceph-mon[29207]: pgmap v387: 97 pgs: 97 active+clean; 669 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-25T14:15:00.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:00 smithi097 ceph-mon[32720]: pgmap v387: 97 pgs: 97 active+clean; 669 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-25T14:15:00.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:00 smithi151 ceph-mon[32990]: pgmap v387: 97 pgs: 97 active+clean; 669 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-25T14:15:02.580 INFO:tasks.workunit.client.0.smithi096.stderr:Note: switching to '994e2f20102a20fa9a27966c14b625cc01884df8'. 2024-04-25T14:15:02.580 INFO:tasks.workunit.client.0.smithi096.stderr: 2024-04-25T14:15:02.581 INFO:tasks.workunit.client.0.smithi096.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-25T14:15:02.581 INFO:tasks.workunit.client.0.smithi096.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-25T14:15:02.581 INFO:tasks.workunit.client.0.smithi096.stderr:state without impacting any branches by switching back to a branch. 2024-04-25T14:15:02.581 INFO:tasks.workunit.client.0.smithi096.stderr: 2024-04-25T14:15:02.581 INFO:tasks.workunit.client.0.smithi096.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-25T14:15:02.581 INFO:tasks.workunit.client.0.smithi096.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-25T14:15:02.581 INFO:tasks.workunit.client.0.smithi096.stderr: 2024-04-25T14:15:02.581 INFO:tasks.workunit.client.0.smithi096.stderr: git switch -c 2024-04-25T14:15:02.581 INFO:tasks.workunit.client.0.smithi096.stderr: 2024-04-25T14:15:02.581 INFO:tasks.workunit.client.0.smithi096.stderr:Or undo this operation with: 2024-04-25T14:15:02.581 INFO:tasks.workunit.client.0.smithi096.stderr: 2024-04-25T14:15:02.581 INFO:tasks.workunit.client.0.smithi096.stderr: git switch - 2024-04-25T14:15:02.581 INFO:tasks.workunit.client.0.smithi096.stderr: 2024-04-25T14:15:02.582 INFO:tasks.workunit.client.0.smithi096.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-25T14:15:02.582 INFO:tasks.workunit.client.0.smithi096.stderr: 2024-04-25T14:15:02.582 INFO:tasks.workunit.client.0.smithi096.stderr:HEAD is now at 994e2f20102 Merge PR #56810 into wip-pdonnell-testing-20240425.015853-debug 2024-04-25T14:15:02.586 DEBUG:teuthology.orchestra.run.smithi096:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-25T14:15:02.614 INFO:tasks.workunit.client.0.smithi096.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-25T14:15:02.615 INFO:tasks.workunit.client.0.smithi096.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T14:15:02.616 INFO:tasks.workunit.client.0.smithi096.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-25T14:15:02.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:02 smithi096 ceph-mon[29207]: pgmap v388: 97 pgs: 97 active+clean; 670 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-25T14:15:02.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:02 smithi097 ceph-mon[32720]: pgmap v388: 97 pgs: 97 active+clean; 670 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-25T14:15:02.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:02 smithi151 ceph-mon[32990]: pgmap v388: 97 pgs: 97 active+clean; 670 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-25T14:15:03.246 INFO:tasks.workunit.client.0.smithi096.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-25T14:15:03.305 INFO:tasks.workunit.client.0.smithi096.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-25T14:15:03.345 INFO:tasks.workunit.client.0.smithi096.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T14:15:03.347 INFO:tasks.workunit.client.0.smithi096.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T14:15:03.347 INFO:tasks.workunit.client.0.smithi096.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-25T14:15:03.385 INFO:tasks.workunit.client.0.smithi096.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T14:15:03.388 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T14:15:03.389 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-25T14:15:03.444 INFO:tasks.workunit:Running workunits matching direct_io on client.0... 2024-04-25T14:15:03.445 INFO:tasks.workunit:Running workunit direct_io/big.sh... 2024-04-25T14:15:03.445 DEBUG:teuthology.orchestra.run.smithi096:workunit test direct_io/big.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=994e2f20102a20fa9a27966c14b625cc01884df8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/big.sh 2024-04-25T14:15:03.504 INFO:tasks.workunit.client.0.smithi096.stdout:test large (16MB) dio write 2024-04-25T14:15:03.504 INFO:tasks.workunit.client.0.smithi096.stderr:+ echo 'test large (16MB) dio write' 2024-04-25T14:15:03.504 INFO:tasks.workunit.client.0.smithi096.stderr:+ dd if=/dev/zero of=foo.big bs=16M count=1 oflag=direct 2024-04-25T14:15:03.589 INFO:tasks.workunit.client.0.smithi096.stderr:1+0 records in 2024-04-25T14:15:03.589 INFO:tasks.workunit.client.0.smithi096.stderr:1+0 records out 2024-04-25T14:15:03.590 INFO:tasks.workunit.client.0.smithi096.stderr:16777216 bytes (17 MB, 16 MiB) copied, 0.0831503 s, 202 MB/s 2024-04-25T14:15:03.590 INFO:tasks.workunit.client.0.smithi096.stderr:+ echo OK 2024-04-25T14:15:03.590 INFO:tasks.workunit.client.0.smithi096.stdout:OK 2024-04-25T14:15:03.591 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-25T14:15:03.591 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-25T14:15:03.665 INFO:tasks.workunit:Running workunit direct_io/direct_io_test... 2024-04-25T14:15:03.665 DEBUG:teuthology.orchestra.run.smithi096:workunit test direct_io/direct_io_test> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=994e2f20102a20fa9a27966c14b625cc01884df8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/direct_io_test 2024-04-25T14:15:04.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:04 smithi096 ceph-mon[29207]: pgmap v389: 97 pgs: 97 active+clean; 670 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T14:15:04.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:04 smithi097 ceph-mon[32720]: pgmap v389: 97 pgs: 97 active+clean; 670 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T14:15:04.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:04 smithi151 ceph-mon[32990]: pgmap v389: 97 pgs: 97 active+clean; 670 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T14:15:06.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:06 smithi096 ceph-mon[29207]: pgmap v390: 97 pgs: 97 active+clean; 4.7 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 342 KiB/s wr, 9 op/s 2024-04-25T14:15:06.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:06 smithi097 ceph-mon[32720]: pgmap v390: 97 pgs: 97 active+clean; 4.7 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 342 KiB/s wr, 9 op/s 2024-04-25T14:15:06.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:06 smithi151 ceph-mon[32990]: pgmap v390: 97 pgs: 97 active+clean; 4.7 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 342 KiB/s wr, 9 op/s 2024-04-25T14:15:08.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:08 smithi096 ceph-mon[29207]: pgmap v391: 97 pgs: 97 active+clean; 5.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.4 MiB/s wr, 689 op/s 2024-04-25T14:15:08.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:08 smithi097 ceph-mon[32720]: pgmap v391: 97 pgs: 97 active+clean; 5.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.4 MiB/s wr, 689 op/s 2024-04-25T14:15:08.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:08 smithi151 ceph-mon[32990]: pgmap v391: 97 pgs: 97 active+clean; 5.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.4 MiB/s wr, 689 op/s 2024-04-25T14:15:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.727+0000 7f36de5ab640 1 -- 172.21.15.96:0/2078753250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36d8102a20 msgr2=0x7f36d8103720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.727+0000 7f36de5ab640 1 --2- 172.21.15.96:0/2078753250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36d8102a20 0x7f36d8103720 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f36c8009900 tx=0x7f36c802a410 comp rx=0 tx=0).stop 2024-04-25T14:15:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.727+0000 7f36de5ab640 1 -- 172.21.15.96:0/2078753250 shutdown_connections 2024-04-25T14:15:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.727+0000 7f36de5ab640 1 --2- 172.21.15.96:0/2078753250 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f36d8108960 0x7f36d810ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.727+0000 7f36de5ab640 1 --2- 172.21.15.96:0/2078753250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36d8103c60 0x7f36d8104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:10.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.727+0000 7f36de5ab640 1 --2- 172.21.15.96:0/2078753250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36d8102a20 0x7f36d8103720 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:10.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.727+0000 7f36de5ab640 1 -- 172.21.15.96:0/2078753250 >> 172.21.15.96:0/2078753250 conn(0x7f36d80fdc10 msgr2=0x7f36d81000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:10.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.727+0000 7f36de5ab640 1 -- 172.21.15.96:0/2078753250 shutdown_connections 2024-04-25T14:15:10.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.728+0000 7f36de5ab640 1 -- 172.21.15.96:0/2078753250 wait complete. 2024-04-25T14:15:10.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.728+0000 7f36de5ab640 1 Processor -- start 2024-04-25T14:15:10.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.728+0000 7f36de5ab640 1 -- start start 2024-04-25T14:15:10.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.728+0000 7f36de5ab640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36d8102a20 0x7f36d81991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:10.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.728+0000 7f36de5ab640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f36d8103c60 0x7f36d8199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:10.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.728+0000 7f36de5ab640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36d8108960 0x7f36d81a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:10.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.728+0000 7f36de5ab640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f36d810e520 con 0x7f36d8108960 2024-04-25T14:15:10.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.728+0000 7f36de5ab640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f36d810e3a0 con 0x7f36d8102a20 2024-04-25T14:15:10.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.728+0000 7f36de5ab640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f36d810e6a0 con 0x7f36d8103c60 2024-04-25T14:15:10.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.729+0000 7f36dcb48640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36d8108960 0x7f36d81a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:10.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.729+0000 7f36dcb48640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36d8108960 0x7f36d81a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35332/0 (socket says 172.21.15.96:35332) 2024-04-25T14:15:10.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.729+0000 7f36dcb48640 1 -- 172.21.15.96:0/1495916756 learned_addr learned my addr 172.21.15.96:0/1495916756 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:10.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.729+0000 7f36cffff640 1 --2- 172.21.15.96:0/1495916756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f36d8103c60 0x7f36d8199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:10.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.729+0000 7f36d7fff640 1 --2- 172.21.15.96:0/1495916756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36d8102a20 0x7f36d81991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:10.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.729+0000 7f36dcb48640 1 -- 172.21.15.96:0/1495916756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f36d8103c60 msgr2=0x7f36d8199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:10.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.729+0000 7f36dcb48640 1 --2- 172.21.15.96:0/1495916756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f36d8103c60 0x7f36d8199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:10.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.729+0000 7f36dcb48640 1 -- 172.21.15.96:0/1495916756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36d8102a20 msgr2=0x7f36d81991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:10.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.729+0000 7f36dcb48640 1 --2- 172.21.15.96:0/1495916756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36d8102a20 0x7f36d81991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:10.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.729+0000 7f36dcb48640 1 -- 172.21.15.96:0/1495916756 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36d81a0c90 con 0x7f36d8108960 2024-04-25T14:15:10.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.729+0000 7f36d7fff640 1 --2- 172.21.15.96:0/1495916756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36d8102a20 0x7f36d81991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:15:10.733 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.729+0000 7f36dcb48640 1 --2- 172.21.15.96:0/1495916756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36d8108960 0x7f36d81a0620 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f36d000eb10 tx=0x7f36d000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:10.733 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.729+0000 7f36d5ffb640 1 -- 172.21.15.96:0/1495916756 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d0004b20 con 0x7f36d8108960 2024-04-25T14:15:10.733 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.729+0000 7f36d5ffb640 1 -- 172.21.15.96:0/1495916756 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36d0004cc0 con 0x7f36d8108960 2024-04-25T14:15:10.733 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.729+0000 7f36de5ab640 1 -- 172.21.15.96:0/1495916756 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36d81a0f80 con 0x7f36d8108960 2024-04-25T14:15:10.733 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.729+0000 7f36d5ffb640 1 -- 172.21.15.96:0/1495916756 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d0010670 con 0x7f36d8108960 2024-04-25T14:15:10.733 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.729+0000 7f36de5ab640 1 -- 172.21.15.96:0/1495916756 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36d81a1440 con 0x7f36d8108960 2024-04-25T14:15:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.732+0000 7f36d5ffb640 1 -- 172.21.15.96:0/1495916756 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f36d0005020 con 0x7f36d8108960 2024-04-25T14:15:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.732+0000 7f36e5458740 1 Processor -- start 2024-04-25T14:15:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.732+0000 7f36e5458740 1 -- start start 2024-04-25T14:15:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.732+0000 7f36e5458740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6689c91b0 0x55e668d6bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.732+0000 7f36e5458740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6689a02f0 0x55e668d6c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.732+0000 7f36e5458740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6689a84d0 0x55e668d73000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.732+0000 7f36e5458740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e668ccf460 con 0x55e6689c91b0 2024-04-25T14:15:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.732+0000 7f36e5458740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e668c89590 con 0x55e6689a84d0 2024-04-25T14:15:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.732+0000 7f36e5458740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e668ccb420 con 0x55e6689a02f0 2024-04-25T14:15:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.732+0000 7f36cffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6689c91b0 0x55e668d6bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.732+0000 7f36d7fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6689a02f0 0x55e668d6c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.732+0000 7f36cffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6689c91b0 0x55e668d6bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35346/0 (socket says 172.21.15.96:35346) 2024-04-25T14:15:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.732+0000 7f36cffff640 1 -- 172.21.15.96:0/4067038170 learned_addr learned my addr 172.21.15.96:0/4067038170 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.733+0000 7f36cffff640 1 --2- 172.21.15.96:0/4067038170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6689a84d0 0x55e668d73000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.733+0000 7f36d7fff640 1 -- 172.21.15.96:0/4067038170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6689a84d0 msgr2=0x55e668d73000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.733+0000 7f36d7fff640 1 --2- 172.21.15.96:0/4067038170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6689a84d0 0x55e668d73000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.733+0000 7f36d7fff640 1 -- 172.21.15.96:0/4067038170 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6689c91b0 msgr2=0x55e668d6bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.733+0000 7f36d7fff640 1 --2- 172.21.15.96:0/4067038170 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6689c91b0 0x55e668d6bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.733+0000 7f36d7fff640 1 -- 172.21.15.96:0/4067038170 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e668c960d0 con 0x55e6689a02f0 2024-04-25T14:15:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.733+0000 7f36cffff640 1 --2- 172.21.15.96:0/4067038170 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6689c91b0 0x55e668d6bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:15:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.733+0000 7f36d5ffb640 1 --2- 172.21.15.96:0/1495916756 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f36c40afb60 0x7f36c40b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.733+0000 7f36cffff640 1 --2- 172.21.15.96:0/1495916756 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f36c40afb60 0x7f36c40b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.733+0000 7f36d7fff640 1 --2- 172.21.15.96:0/4067038170 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6689a02f0 0x55e668d6c100 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f36c8009230 tx=0x7f36c8032b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c2cfdd12ae4f835 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.733+0000 7f36d5ffb640 1 -- 172.21.15.96:0/1495916756 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f36d00df2a0 con 0x7f36d8108960 2024-04-25T14:15:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.733+0000 7f36ce7fc640 1 -- 172.21.15.96:0/4067038170 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c803f030 con 0x55e6689a02f0 2024-04-25T14:15:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.733+0000 7f36ce7fc640 1 -- 172.21.15.96:0/4067038170 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36c8032d80 con 0x55e6689a02f0 2024-04-25T14:15:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.733+0000 7f36ce7fc640 1 -- 172.21.15.96:0/4067038170 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c8035070 con 0x55e6689a02f0 2024-04-25T14:15:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.734+0000 7f36e5458740 1 -- 172.21.15.96:0/4067038170 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6689a02f0 msgr2=0x55e668d6c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.734+0000 7f36cffff640 1 --2- 172.21.15.96:0/1495916756 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f36c40afb60 0x7f36c40b2070 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x55e668d6cd50 tx=0x7f36c00097b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.734+0000 7f36e5458740 1 --2- 172.21.15.96:0/4067038170 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6689a02f0 0x55e668d6c100 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f36c8009230 tx=0x7f36c8032b90 comp rx=0 tx=0).stop 2024-04-25T14:15:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.734+0000 7f36e5458740 1 -- 172.21.15.96:0/4067038170 shutdown_connections 2024-04-25T14:15:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.734+0000 7f36e5458740 1 --2- 172.21.15.96:0/4067038170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6689a84d0 0x55e668d73000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.734+0000 7f36e5458740 1 --2- 172.21.15.96:0/4067038170 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6689a02f0 0x55e668d6c100 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.734+0000 7f36e5458740 1 --2- 172.21.15.96:0/4067038170 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6689c91b0 0x55e668d6bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.734+0000 7f36e5458740 1 -- 172.21.15.96:0/4067038170 >> 172.21.15.96:0/4067038170 conn(0x55e6689bd300 msgr2=0x55e668d65300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.734+0000 7f36e5458740 1 -- 172.21.15.96:0/4067038170 shutdown_connections 2024-04-25T14:15:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.734+0000 7f36e5458740 1 -- 172.21.15.96:0/4067038170 wait complete. 2024-04-25T14:15:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.734+0000 7f36e5458740 1 -- start start 2024-04-25T14:15:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.735+0000 7f36e5458740 1 Processor -- start 2024-04-25T14:15:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.735+0000 7f36e5458740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6689a02f0 0x55e668d75790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.735+0000 7f36e5458740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6689a84d0 0x55e668d75cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.735+0000 7f36e5458740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6689c91b0 0x55e668d7cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.735+0000 7f36e5458740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e668d739f0 con 0x55e6689a84d0 2024-04-25T14:15:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.735+0000 7f36e5458740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e668c56b60 con 0x55e6689c91b0 2024-04-25T14:15:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.735+0000 7f36e5458740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e668d73b70 con 0x55e6689a02f0 2024-04-25T14:15:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.735+0000 7f36e5458740 10 client.0 _ll_register_callbacks cb 0x55e668c7b2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:15:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.735+0000 7f36cffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6689a84d0 0x55e668d75cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.735+0000 7f36cffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6689a84d0 0x55e668d75cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35354/0 (socket says 172.21.15.96:35354) 2024-04-25T14:15:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.735+0000 7f36cffff640 1 -- 172.21.15.96:0/3817164528 learned_addr learned my addr 172.21.15.96:0/3817164528 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.735+0000 7f36d7fff640 1 --2- 172.21.15.96:0/3817164528 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6689a02f0 0x55e668d75790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.735+0000 7f36dcb48640 1 --2- 172.21.15.96:0/3817164528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6689c91b0 0x55e668d7cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.735+0000 7f36cffff640 1 -- 172.21.15.96:0/3817164528 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6689a02f0 msgr2=0x55e668d75790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.735+0000 7f36cffff640 1 --2- 172.21.15.96:0/3817164528 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6689a02f0 0x55e668d75790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.735+0000 7f36cffff640 1 -- 172.21.15.96:0/3817164528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6689c91b0 msgr2=0x55e668d7cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.735+0000 7f36cffff640 1 --2- 172.21.15.96:0/3817164528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6689c91b0 0x55e668d7cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.735+0000 7f36cffff640 1 -- 172.21.15.96:0/3817164528 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6689e1210 con 0x55e6689a84d0 2024-04-25T14:15:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.735+0000 7f36d7fff640 1 --2- 172.21.15.96:0/3817164528 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6689a02f0 0x55e668d75790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:15:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.736+0000 7f36cffff640 1 --2- 172.21.15.96:0/3817164528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6689a84d0 0x55e668d75cd0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f36c0004620 tx=0x7f36c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76566fbf702d0d7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.736+0000 7f36b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:15:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.736+0000 7f36b77fe640 1 -- 172.21.15.96:0/3817164528 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c002e060 con 0x55e6689a84d0 2024-04-25T14:15:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.736+0000 7f36b77fe640 1 -- 172.21.15.96:0/3817164528 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f36c0012c40 con 0x55e6689a84d0 2024-04-25T14:15:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.736+0000 7f36e5458740 1 -- 172.21.15.96:0/3817164528 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36c8032d80 con 0x55e6689a84d0 2024-04-25T14:15:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.736+0000 7f36b77fe640 1 -- 172.21.15.96:0/3817164528 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f36c0012e40 con 0x55e6689a84d0 2024-04-25T14:15:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.736+0000 7f36e5458740 10 client.16050 fetch_fsmap learned FSMap version 26 2024-04-25T14:15:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.736+0000 7f36e5458740 1 -- 172.21.15.96:0/3817164528 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e668aad620 con 0x55e6689a84d0 2024-04-25T14:15:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.736+0000 7f36b77fe640 1 -- 172.21.15.96:0/3817164528 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36c8032d80 con 0x55e6689a84d0 2024-04-25T14:15:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.736+0000 7f36b77fe640 1 -- 172.21.15.96:0/3817164528 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f36c001d040 con 0x55e6689a84d0 2024-04-25T14:15:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.736+0000 7f36e5458740 10 client.16050 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:15:10.746 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.736+0000 7f36e5458740 10 client.16050 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:15:10.746 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.736+0000 7f36e5458740 20 client.16050 populate_metadata read hostname 'smithi096' 2024-04-25T14:15:10.746 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.736+0000 7f36e5458740 1 --2- 172.21.15.96:0/3817164528 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55e668987d80 0x55e668d835b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:10.746 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.736+0000 7f36e5458740 4 client.16050 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:15:10.746 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.736+0000 7f36e5458740 1 -- 172.21.15.96:0/3817164528 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e668cdd070 con 0x55e668987d80 2024-04-25T14:15:10.746 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.737+0000 7f36d7fff640 1 --2- 172.21.15.96:0/3817164528 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55e668987d80 0x55e668d835b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:10.746 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.737+0000 7f36d7fff640 1 --2- 172.21.15.96:0/3817164528 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55e668987d80 0x55e668d835b0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc702f6360b17256 server_cookie=5b19e7b1970e7380 in_seq=0 out_seq=0 2024-04-25T14:15:10.746 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.737+0000 7f36b77fe640 10 client.16050 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:15:10.746 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.738+0000 7f36b77fe640 1 -- 172.21.15.96:0/3817164528 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e668cdd070 con 0x55e668987d80 2024-04-25T14:15:10.746 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.738+0000 7f36b77fe640 10 client.16050 handle_command_reply: tid=0 2024-04-25T14:15:10.746 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.738+0000 7f36e5458740 1 client.16050 shutdown 2024-04-25T14:15:10.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.738+0000 7f36e5458740 1 -- 172.21.15.96:0/3817164528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6689a84d0 msgr2=0x55e668d75cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:10.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.738+0000 7f36e5458740 1 --2- 172.21.15.96:0/3817164528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6689a84d0 0x55e668d75cd0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f36c0004620 tx=0x7f36c0004730 comp rx=0 tx=0).stop 2024-04-25T14:15:10.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.738+0000 7f36e5458740 1 -- 172.21.15.96:0/3817164528 shutdown_connections 2024-04-25T14:15:10.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.738+0000 7f36e5458740 1 --2- 172.21.15.96:0/3817164528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6689c91b0 0x55e668d7cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:10.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.738+0000 7f36e5458740 1 --2- 172.21.15.96:0/3817164528 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55e668987d80 0x55e668d835b0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:10.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.738+0000 7f36e5458740 1 --2- 172.21.15.96:0/3817164528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6689a84d0 0x55e668d75cd0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:10.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.738+0000 7f36e5458740 1 --2- 172.21.15.96:0/3817164528 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6689a02f0 0x55e668d75790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:10.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.738+0000 7f36b77fe640 0 client.16050 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:15:10.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.738+0000 7f36e5458740 1 -- 172.21.15.96:0/3817164528 >> 172.21.15.96:0/3817164528 conn(0x55e6689bd300 msgr2=0x55e668d68210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:10.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.738+0000 7f36e5458740 1 -- 172.21.15.96:0/3817164528 shutdown_connections 2024-04-25T14:15:10.748 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.738+0000 7f36e5458740 1 -- 172.21.15.96:0/3817164528 wait complete. 2024-04-25T14:15:10.748 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.738+0000 7f36e5458740 20 client.16050 trim_cache size 0 max 16384 2024-04-25T14:15:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.753+0000 7f36e5458740 1 Processor -- start 2024-04-25T14:15:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.753+0000 7f36e5458740 1 -- start start 2024-04-25T14:15:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.753+0000 7f36e5458740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6689a02f0 0x55e668d74620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.753+0000 7f36e5458740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6689a84d0 0x55e668d7cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.753+0000 7f36e5458740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6689c91b0 0x55e668d68100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.753+0000 7f36e5458740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f36d81a91e0 con 0x55e6689a02f0 2024-04-25T14:15:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.753+0000 7f36e5458740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f36d81aad20 con 0x55e6689c91b0 2024-04-25T14:15:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.753+0000 7f36e5458740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f36d81a98b0 con 0x55e6689a84d0 2024-04-25T14:15:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.753+0000 7f36d7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6689a02f0 0x55e668d74620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.753+0000 7f36d7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6689a02f0 0x55e668d74620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35358/0 (socket says 172.21.15.96:35358) 2024-04-25T14:15:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.753+0000 7f36d7fff640 1 -- 172.21.15.96:0/3686825921 learned_addr learned my addr 172.21.15.96:0/3686825921 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.753+0000 7f36cffff640 1 --2- 172.21.15.96:0/3686825921 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6689a84d0 0x55e668d7cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.754+0000 7f36d7fff640 1 -- 172.21.15.96:0/3686825921 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6689a84d0 msgr2=0x55e668d7cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:10.761 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.754+0000 7f36d7fff640 1 --2- 172.21.15.96:0/3686825921 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6689a84d0 0x55e668d7cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:10.761 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.754+0000 7f36dcb48640 1 --2- 172.21.15.96:0/3686825921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6689c91b0 0x55e668d68100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:10.761 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.754+0000 7f36d7fff640 1 -- 172.21.15.96:0/3686825921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6689c91b0 msgr2=0x55e668d68100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:10.761 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.754+0000 7f36d7fff640 1 --2- 172.21.15.96:0/3686825921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6689c91b0 0x55e668d68100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:10.761 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.754+0000 7f36d7fff640 1 -- 172.21.15.96:0/3686825921 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e668d70690 con 0x55e6689a02f0 2024-04-25T14:15:10.761 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.754+0000 7f36dcb48640 1 --2- 172.21.15.96:0/3686825921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6689c91b0 0x55e668d68100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:15:10.761 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.754+0000 7f36d7fff640 1 --2- 172.21.15.96:0/3686825921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6689a02f0 0x55e668d74620 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x55e668d6cf60 tx=0x7f36c8032b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fc0831e90c0e150 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:10.761 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.754+0000 7f36cdffb640 1 -- 172.21.15.96:0/3686825921 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c803f040 con 0x55e6689a02f0 2024-04-25T14:15:10.761 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.754+0000 7f36cdffb640 1 -- 172.21.15.96:0/3686825921 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36c8051070 con 0x55e6689a02f0 2024-04-25T14:15:10.761 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.754+0000 7f36cdffb640 1 -- 172.21.15.96:0/3686825921 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c8035040 con 0x55e6689a02f0 2024-04-25T14:15:10.761 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.755+0000 7f36e5458740 1 -- 172.21.15.96:0/3686825921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6689a02f0 msgr2=0x55e668d74620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:10.761 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.755+0000 7f36e5458740 1 --2- 172.21.15.96:0/3686825921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6689a02f0 0x55e668d74620 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x55e668d6cf60 tx=0x7f36c8032b20 comp rx=0 tx=0).stop 2024-04-25T14:15:10.761 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.755+0000 7f36e5458740 1 -- 172.21.15.96:0/3686825921 shutdown_connections 2024-04-25T14:15:10.761 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.755+0000 7f36e5458740 1 --2- 172.21.15.96:0/3686825921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6689c91b0 0x55e668d68100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:10.762 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.755+0000 7f36e5458740 1 --2- 172.21.15.96:0/3686825921 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6689a84d0 0x55e668d7cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:10.762 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.755+0000 7f36e5458740 1 --2- 172.21.15.96:0/3686825921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6689a02f0 0x55e668d74620 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:10.762 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.755+0000 7f36e5458740 1 -- 172.21.15.96:0/3686825921 >> 172.21.15.96:0/3686825921 conn(0x55e6689bd300 msgr2=0x55e668d73f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:10.762 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.755+0000 7f36e5458740 1 -- 172.21.15.96:0/3686825921 shutdown_connections 2024-04-25T14:15:10.762 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.755+0000 7f36e5458740 1 -- 172.21.15.96:0/3686825921 wait complete. 2024-04-25T14:15:10.762 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.755+0000 7f36e5458740 1 -- start start 2024-04-25T14:15:10.762 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.756+0000 7f36e5458740 1 Processor -- start 2024-04-25T14:15:10.762 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.756+0000 7f36e5458740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6689a02f0 0x55e668d7a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:10.762 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.756+0000 7f36e5458740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6689a84d0 0x55e668d7a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:10.762 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.756+0000 7f36e5458740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6689c91b0 0x55e668d7b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:10.762 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.756+0000 7f36e5458740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f36d81aa650 con 0x55e6689a84d0 2024-04-25T14:15:10.762 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.756+0000 7f36e5458740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f36d81a9f80 con 0x55e6689a02f0 2024-04-25T14:15:10.762 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.756+0000 7f36e5458740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f36d81a71c0 con 0x55e6689c91b0 2024-04-25T14:15:10.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.756+0000 7f36e5458740 10 client.0 _ll_register_callbacks cb 0x7f36c00098a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:15:10.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.756+0000 7f36cffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6689a84d0 0x55e668d7a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:10.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.756+0000 7f36cffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6689a84d0 0x55e668d7a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35368/0 (socket says 172.21.15.96:35368) 2024-04-25T14:15:10.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.756+0000 7f36cffff640 1 -- 172.21.15.96:0/3966276146 learned_addr learned my addr 172.21.15.96:0/3966276146 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:10.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.756+0000 7f36dcb48640 1 --2- 172.21.15.96:0/3966276146 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6689c91b0 0x55e668d7b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:10.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.756+0000 7f36d7fff640 1 --2- 172.21.15.96:0/3966276146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6689a02f0 0x55e668d7a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:10.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.756+0000 7f36cffff640 1 -- 172.21.15.96:0/3966276146 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6689c91b0 msgr2=0x55e668d7b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:10.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.756+0000 7f36cffff640 1 --2- 172.21.15.96:0/3966276146 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6689c91b0 0x55e668d7b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:10.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.756+0000 7f36cffff640 1 -- 172.21.15.96:0/3966276146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6689a02f0 msgr2=0x55e668d7a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:10.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.756+0000 7f36cffff640 1 --2- 172.21.15.96:0/3966276146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6689a02f0 0x55e668d7a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:10.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.756+0000 7f36cffff640 1 -- 172.21.15.96:0/3966276146 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e668d7bb50 con 0x55e6689a84d0 2024-04-25T14:15:10.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.756+0000 7f36cffff640 1 --2- 172.21.15.96:0/3966276146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6689a84d0 0x55e668d7a660 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f36c0021c60 tx=0x7f36c0011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8acc49faa923568 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:10.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.756+0000 7f36b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:15:10.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.757+0000 7f36b77fe640 1 -- 172.21.15.96:0/3966276146 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c0012820 con 0x55e6689a84d0 2024-04-25T14:15:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.757+0000 7f36b77fe640 1 -- 172.21.15.96:0/3966276146 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f36c001f670 con 0x55e6689a84d0 2024-04-25T14:15:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.757+0000 7f36e5458740 1 -- 172.21.15.96:0/3966276146 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36c8051070 con 0x55e6689a84d0 2024-04-25T14:15:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.757+0000 7f36b77fe640 1 -- 172.21.15.96:0/3966276146 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f36c00293e0 con 0x55e6689a84d0 2024-04-25T14:15:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.757+0000 7f36e5458740 10 client.16056 fetch_fsmap learned FSMap version 26 2024-04-25T14:15:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.757+0000 7f36e5458740 1 -- 172.21.15.96:0/3966276146 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e668d7cdf0 con 0x55e6689a84d0 2024-04-25T14:15:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.757+0000 7f36b77fe640 1 -- 172.21.15.96:0/3966276146 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36c8051070 con 0x55e6689a84d0 2024-04-25T14:15:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.757+0000 7f36b77fe640 1 -- 172.21.15.96:0/3966276146 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f36c003b020 con 0x55e6689a84d0 2024-04-25T14:15:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.757+0000 7f36e5458740 10 client.16056 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:15:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.757+0000 7f36e5458740 10 client.16056 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:15:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.757+0000 7f36e5458740 20 client.16056 populate_metadata read hostname 'smithi096' 2024-04-25T14:15:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.757+0000 7f36e5458740 1 --2- 172.21.15.96:0/3966276146 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55e66895d1c0 0x55e668dee080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.757+0000 7f36e5458740 4 client.16056 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:15:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.757+0000 7f36e5458740 1 -- 172.21.15.96:0/3966276146 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e668cd6ee0 con 0x55e66895d1c0 2024-04-25T14:15:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.758+0000 7f36d7fff640 1 --2- 172.21.15.96:0/3966276146 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55e66895d1c0 0x55e668dee080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.758+0000 7f36d7fff640 1 --2- 172.21.15.96:0/3966276146 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55e66895d1c0 0x55e668dee080 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=285735f949d4b1a7 server_cookie=87247f65bdc01a01 in_seq=0 out_seq=0 2024-04-25T14:15:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.758+0000 7f36b77fe640 10 client.16056 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:15:10.770 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.769+0000 7f36b77fe640 1 -- 172.21.15.96:0/3966276146 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e668cd6ee0 con 0x55e66895d1c0 2024-04-25T14:15:10.770 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.769+0000 7f36b77fe640 10 client.16056 handle_command_reply: tid=0 2024-04-25T14:15:10.770 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.769+0000 7f36e5458740 1 client.16056 shutdown 2024-04-25T14:15:10.770 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.769+0000 7f36e5458740 1 -- 172.21.15.96:0/3966276146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6689a84d0 msgr2=0x55e668d7a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:10.770 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.769+0000 7f36e5458740 1 --2- 172.21.15.96:0/3966276146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6689a84d0 0x55e668d7a660 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f36c0021c60 tx=0x7f36c0011f10 comp rx=0 tx=0).stop 2024-04-25T14:15:10.770 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.769+0000 7f36e5458740 1 -- 172.21.15.96:0/3966276146 shutdown_connections 2024-04-25T14:15:10.770 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.769+0000 7f36e5458740 1 --2- 172.21.15.96:0/3966276146 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6689c91b0 0x55e668d7b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.769+0000 7f36e5458740 1 --2- 172.21.15.96:0/3966276146 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55e66895d1c0 0x55e668dee080 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.769+0000 7f36e5458740 1 --2- 172.21.15.96:0/3966276146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6689a84d0 0x55e668d7a660 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.769+0000 7f36e5458740 1 --2- 172.21.15.96:0/3966276146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e6689a02f0 0x55e668d7a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.769+0000 7f36b77fe640 0 client.16056 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:15:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.769+0000 7f36e5458740 1 -- 172.21.15.96:0/3966276146 >> 172.21.15.96:0/3966276146 conn(0x55e6689bd300 msgr2=0x55e668d73f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.769+0000 7f36e5458740 1 -- 172.21.15.96:0/3966276146 shutdown_connections 2024-04-25T14:15:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.770+0000 7f36e5458740 1 -- 172.21.15.96:0/3966276146 wait complete. 2024-04-25T14:15:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.770+0000 7f36e5458740 20 client.16056 trim_cache size 0 max 16384 2024-04-25T14:15:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T14:15:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T14:15:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T14:15:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T14:15:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.770+0000 7f36de5ab640 1 -- 172.21.15.96:0/1495916756 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f36c40afb60 msgr2=0x7f36c40b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.770+0000 7f36de5ab640 1 --2- 172.21.15.96:0/1495916756 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f36c40afb60 0x7f36c40b2070 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x55e668d6cd50 tx=0x7f36c00097b0 comp rx=0 tx=0).stop 2024-04-25T14:15:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.770+0000 7f36de5ab640 1 -- 172.21.15.96:0/1495916756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36d8108960 msgr2=0x7f36d81a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.770+0000 7f36de5ab640 1 --2- 172.21.15.96:0/1495916756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36d8108960 0x7f36d81a0620 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f36d000eb10 tx=0x7f36d000c2d0 comp rx=0 tx=0).stop 2024-04-25T14:15:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.770+0000 7f36de5ab640 1 -- 172.21.15.96:0/1495916756 shutdown_connections 2024-04-25T14:15:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.770+0000 7f36de5ab640 1 --2- 172.21.15.96:0/1495916756 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f36c40afb60 0x7f36c40b2070 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.770+0000 7f36de5ab640 1 --2- 172.21.15.96:0/1495916756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36d8108960 0x7f36d81a0620 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.770+0000 7f36de5ab640 1 --2- 172.21.15.96:0/1495916756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f36d8103c60 0x7f36d8199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.770+0000 7f36de5ab640 1 --2- 172.21.15.96:0/1495916756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36d8102a20 0x7f36d81991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.770+0000 7f36de5ab640 1 -- 172.21.15.96:0/1495916756 >> 172.21.15.96:0/1495916756 conn(0x7f36d80fdc10 msgr2=0x7f36d8104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.770+0000 7f36de5ab640 1 -- 172.21.15.96:0/1495916756 shutdown_connections 2024-04-25T14:15:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:10.770+0000 7f36de5ab640 1 -- 172.21.15.96:0/1495916756 wait complete. 2024-04-25T14:15:10.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:10 smithi096 ceph-mon[29207]: pgmap v392: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.4 MiB/s wr, 686 op/s 2024-04-25T14:15:10.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:10 smithi097 ceph-mon[32720]: pgmap v392: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.4 MiB/s wr, 686 op/s 2024-04-25T14:15:10.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:10 smithi151 ceph-mon[32990]: pgmap v392: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.4 MiB/s wr, 686 op/s 2024-04-25T14:15:12.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:12 smithi096 ceph-mon[29207]: pgmap v393: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1.4 MiB/s wr, 1.97k op/s 2024-04-25T14:15:12.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:12 smithi097 ceph-mon[32720]: pgmap v393: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1.4 MiB/s wr, 1.97k op/s 2024-04-25T14:15:12.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:12 smithi151 ceph-mon[32990]: pgmap v393: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1.4 MiB/s wr, 1.97k op/s 2024-04-25T14:15:12.998 INFO:tasks.workunit.client.0.smithi096.stdout:.......... 2024-04-25T14:15:12.998 INFO:tasks.workunit.client.0.smithi096.stdout:do_o_direct_reads: SUCCESS 2024-04-25T14:15:13.001 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-25T14:15:13.002 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-25T14:15:13.031 INFO:tasks.workunit:Running workunit direct_io/misc.sh... 2024-04-25T14:15:13.032 DEBUG:teuthology.orchestra.run.smithi096:workunit test direct_io/misc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=994e2f20102a20fa9a27966c14b625cc01884df8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/misc.sh 2024-04-25T14:15:13.093 INFO:tasks.workunit.client.0.smithi096.stdout:test read from hole 2024-04-25T14:15:13.093 INFO:tasks.workunit.client.0.smithi096.stderr:+ echo 'test read from hole' 2024-04-25T14:15:13.093 INFO:tasks.workunit.client.0.smithi096.stderr:+ dd if=/dev/zero of=dd3 bs=1 seek=1048576 count=0 2024-04-25T14:15:13.094 INFO:tasks.workunit.client.0.smithi096.stderr:0+0 records in 2024-04-25T14:15:13.095 INFO:tasks.workunit.client.0.smithi096.stderr:0+0 records out 2024-04-25T14:15:13.095 INFO:tasks.workunit.client.0.smithi096.stderr:0 bytes copied, 3.0948e-05 s, 0.0 kB/s 2024-04-25T14:15:13.095 INFO:tasks.workunit.client.0.smithi096.stderr:+ dd if=dd3 of=/tmp/ddout1 skip=8 bs=512 count=2 iflag=direct 2024-04-25T14:15:13.096 INFO:tasks.workunit.client.0.smithi096.stderr:2+0 records in 2024-04-25T14:15:13.097 INFO:tasks.workunit.client.0.smithi096.stderr:2+0 records out 2024-04-25T14:15:13.097 INFO:tasks.workunit.client.0.smithi096.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00138752 s, 738 kB/s 2024-04-25T14:15:13.097 INFO:tasks.workunit.client.0.smithi096.stderr:+ dd if=/dev/zero of=/tmp/dd3 bs=512 count=2 2024-04-25T14:15:13.097 INFO:tasks.workunit.client.0.smithi096.stderr:2+0 records in 2024-04-25T14:15:13.097 INFO:tasks.workunit.client.0.smithi096.stderr:2+0 records out 2024-04-25T14:15:13.097 INFO:tasks.workunit.client.0.smithi096.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 3.9898e-05 s, 25.7 MB/s 2024-04-25T14:15:13.097 INFO:tasks.workunit.client.0.smithi096.stderr:+ cmp /tmp/dd3 /tmp/ddout1 2024-04-25T14:15:13.098 INFO:tasks.workunit.client.0.smithi096.stdout:other thing 2024-04-25T14:15:13.098 INFO:tasks.workunit.client.0.smithi096.stderr:+ echo 'other thing' 2024-04-25T14:15:13.098 INFO:tasks.workunit.client.0.smithi096.stderr:+ dd if=/dev/urandom of=/tmp/dd10 bs=500 count=1 2024-04-25T14:15:13.099 INFO:tasks.workunit.client.0.smithi096.stderr:1+0 records in 2024-04-25T14:15:13.099 INFO:tasks.workunit.client.0.smithi096.stderr:1+0 records out 2024-04-25T14:15:13.099 INFO:tasks.workunit.client.0.smithi096.stderr:500 bytes copied, 3.6609e-05 s, 13.7 MB/s 2024-04-25T14:15:13.099 INFO:tasks.workunit.client.0.smithi096.stderr:+ dd if=/tmp/dd10 of=dd10 bs=512 seek=8388 count=1 2024-04-25T14:15:13.107 INFO:tasks.workunit.client.0.smithi096.stderr:0+1 records in 2024-04-25T14:15:13.107 INFO:tasks.workunit.client.0.smithi096.stderr:0+1 records out 2024-04-25T14:15:13.107 INFO:tasks.workunit.client.0.smithi096.stderr:500 bytes copied, 0.00680336 s, 73.5 kB/s 2024-04-25T14:15:13.107 INFO:tasks.workunit.client.0.smithi096.stderr:+ dd if=dd10 of=/tmp/dd10out bs=512 skip=8388 count=1 iflag=direct 2024-04-25T14:15:13.113 INFO:tasks.workunit.client.0.smithi096.stderr:0+1 records in 2024-04-25T14:15:13.113 INFO:tasks.workunit.client.0.smithi096.stderr:0+1 records out 2024-04-25T14:15:13.113 INFO:tasks.workunit.client.0.smithi096.stderr:500 bytes copied, 0.00512495 s, 97.6 kB/s 2024-04-25T14:15:13.113 INFO:tasks.workunit.client.0.smithi096.stderr:+ cmp /tmp/dd10 /tmp/dd10out 2024-04-25T14:15:13.114 INFO:tasks.workunit.client.0.smithi096.stdout:OK 2024-04-25T14:15:13.114 INFO:tasks.workunit.client.0.smithi096.stderr:+ echo OK 2024-04-25T14:15:13.115 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-25T14:15:13.115 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-25T14:15:13.193 INFO:tasks.workunit:Running workunit direct_io/test_short_dio_read... 2024-04-25T14:15:13.194 DEBUG:teuthology.orchestra.run.smithi096:workunit test direct_io/test_short_dio_read> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=994e2f20102a20fa9a27966c14b625cc01884df8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/test_short_dio_read 2024-04-25T14:15:13.264 INFO:tasks.workunit.client.0.smithi096.stdout:writing first 3 bytes of 10k file 2024-04-25T14:15:13.265 INFO:tasks.workunit.client.0.smithi096.stdout:reading O_DIRECT 2024-04-25T14:15:13.265 INFO:tasks.workunit.client.0.smithi096.stdout:got 10000 2024-04-25T14:15:13.265 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-25T14:15:13.265 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-25T14:15:13.340 INFO:tasks.workunit:Running workunit direct_io/test_sync_io... 2024-04-25T14:15:13.341 DEBUG:teuthology.orchestra.run.smithi096:workunit test direct_io/test_sync_io> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=994e2f20102a20fa9a27966c14b625cc01884df8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/test_sync_io 2024-04-25T14:15:14.624 INFO:tasks.workunit.client.0.smithi096.stdout:writing pattern 2024-04-25T14:15:14.624 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4190208 len 1024 2024-04-25T14:15:14.624 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4190208 len 1024 2024-04-25T14:15:14.624 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4190208 len 1024 2024-04-25T14:15:14.624 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4190208 len 1024 2024-04-25T14:15:14.624 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4190208 len 2048 2024-04-25T14:15:14.624 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4190208 len 2048 2024-04-25T14:15:14.625 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4190208 len 2048 2024-04-25T14:15:14.625 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4190208 len 2048 2024-04-25T14:15:14.625 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4190208 len 4096 2024-04-25T14:15:14.625 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4190208 len 4096 2024-04-25T14:15:14.625 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4190208 len 4096 2024-04-25T14:15:14.625 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4190208 len 4096 2024-04-25T14:15:14.625 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4190208 len 8192 2024-04-25T14:15:14.625 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4190208 len 8192 2024-04-25T14:15:14.625 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4190208 len 8192 2024-04-25T14:15:14.625 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4190208 len 8192 2024-04-25T14:15:14.625 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4190208 len 16384 2024-04-25T14:15:14.625 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4190208 len 16384 2024-04-25T14:15:14.625 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4190208 len 16384 2024-04-25T14:15:14.625 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4190208 len 16384 2024-04-25T14:15:14.626 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4190720 len 1024 2024-04-25T14:15:14.626 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4190720 len 1024 2024-04-25T14:15:14.626 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4190720 len 1024 2024-04-25T14:15:14.626 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4190720 len 1024 2024-04-25T14:15:14.626 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4190720 len 2048 2024-04-25T14:15:14.626 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4190720 len 2048 2024-04-25T14:15:14.626 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4190720 len 2048 2024-04-25T14:15:14.626 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4190720 len 2048 2024-04-25T14:15:14.626 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4190720 len 4096 2024-04-25T14:15:14.626 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4190720 len 4096 2024-04-25T14:15:14.626 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4190720 len 4096 2024-04-25T14:15:14.626 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4190720 len 4096 2024-04-25T14:15:14.626 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4190720 len 8192 2024-04-25T14:15:14.627 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4190720 len 8192 2024-04-25T14:15:14.627 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4190720 len 8192 2024-04-25T14:15:14.627 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4190720 len 8192 2024-04-25T14:15:14.627 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4190720 len 16384 2024-04-25T14:15:14.627 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4190720 len 16384 2024-04-25T14:15:14.627 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4190720 len 16384 2024-04-25T14:15:14.627 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4190720 len 16384 2024-04-25T14:15:14.627 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4191232 len 1024 2024-04-25T14:15:14.627 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4191232 len 1024 2024-04-25T14:15:14.627 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4191232 len 1024 2024-04-25T14:15:14.627 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4191232 len 1024 2024-04-25T14:15:14.627 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4191232 len 2048 2024-04-25T14:15:14.627 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4191232 len 2048 2024-04-25T14:15:14.627 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4191232 len 2048 2024-04-25T14:15:14.627 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4191232 len 2048 2024-04-25T14:15:14.628 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4191232 len 4096 2024-04-25T14:15:14.628 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4191232 len 4096 2024-04-25T14:15:14.628 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4191232 len 4096 2024-04-25T14:15:14.628 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4191232 len 4096 2024-04-25T14:15:14.628 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4191232 len 8192 2024-04-25T14:15:14.628 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4191232 len 8192 2024-04-25T14:15:14.628 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4191232 len 8192 2024-04-25T14:15:14.628 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4191232 len 8192 2024-04-25T14:15:14.628 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4191232 len 16384 2024-04-25T14:15:14.628 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4191232 len 16384 2024-04-25T14:15:14.628 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4191232 len 16384 2024-04-25T14:15:14.628 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4191232 len 16384 2024-04-25T14:15:14.628 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4191744 len 1024 2024-04-25T14:15:14.628 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4191744 len 1024 2024-04-25T14:15:14.628 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4191744 len 1024 2024-04-25T14:15:14.628 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4191744 len 1024 2024-04-25T14:15:14.629 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4191744 len 2048 2024-04-25T14:15:14.629 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4191744 len 2048 2024-04-25T14:15:14.629 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4191744 len 2048 2024-04-25T14:15:14.629 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4191744 len 2048 2024-04-25T14:15:14.629 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4191744 len 4096 2024-04-25T14:15:14.629 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4191744 len 4096 2024-04-25T14:15:14.629 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4191744 len 4096 2024-04-25T14:15:14.629 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4191744 len 4096 2024-04-25T14:15:14.629 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4191744 len 8192 2024-04-25T14:15:14.629 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4191744 len 8192 2024-04-25T14:15:14.629 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4191744 len 8192 2024-04-25T14:15:14.629 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4191744 len 8192 2024-04-25T14:15:14.629 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4191744 len 16384 2024-04-25T14:15:14.629 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4191744 len 16384 2024-04-25T14:15:14.630 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4191744 len 16384 2024-04-25T14:15:14.630 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4191744 len 16384 2024-04-25T14:15:14.630 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4192256 len 1024 2024-04-25T14:15:14.630 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4192256 len 1024 2024-04-25T14:15:14.630 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4192256 len 1024 2024-04-25T14:15:14.630 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4192256 len 1024 2024-04-25T14:15:14.630 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4192256 len 2048 2024-04-25T14:15:14.630 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4192256 len 2048 2024-04-25T14:15:14.630 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4192256 len 2048 2024-04-25T14:15:14.645 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4192256 len 2048 2024-04-25T14:15:14.645 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4192256 len 4096 2024-04-25T14:15:14.645 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4192256 len 4096 2024-04-25T14:15:14.645 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4192256 len 4096 2024-04-25T14:15:14.646 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4192256 len 4096 2024-04-25T14:15:14.646 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4192256 len 8192 2024-04-25T14:15:14.646 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4192256 len 8192 2024-04-25T14:15:14.646 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4192256 len 8192 2024-04-25T14:15:14.646 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4192256 len 8192 2024-04-25T14:15:14.646 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4192256 len 16384 2024-04-25T14:15:14.646 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4192256 len 16384 2024-04-25T14:15:14.646 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4192256 len 16384 2024-04-25T14:15:14.646 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4192256 len 16384 2024-04-25T14:15:14.646 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4192768 len 1024 2024-04-25T14:15:14.646 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4192768 len 1024 2024-04-25T14:15:14.646 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4192768 len 1024 2024-04-25T14:15:14.646 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4192768 len 1024 2024-04-25T14:15:14.646 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4192768 len 2048 2024-04-25T14:15:14.647 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4192768 len 2048 2024-04-25T14:15:14.647 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4192768 len 2048 2024-04-25T14:15:14.647 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4192768 len 2048 2024-04-25T14:15:14.647 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4192768 len 4096 2024-04-25T14:15:14.647 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4192768 len 4096 2024-04-25T14:15:14.647 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4192768 len 4096 2024-04-25T14:15:14.647 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4192768 len 4096 2024-04-25T14:15:14.647 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4192768 len 8192 2024-04-25T14:15:14.647 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4192768 len 8192 2024-04-25T14:15:14.647 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4192768 len 8192 2024-04-25T14:15:14.647 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4192768 len 8192 2024-04-25T14:15:14.647 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4192768 len 16384 2024-04-25T14:15:14.647 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4192768 len 16384 2024-04-25T14:15:14.647 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4192768 len 16384 2024-04-25T14:15:14.648 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4192768 len 16384 2024-04-25T14:15:14.648 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4193280 len 1024 2024-04-25T14:15:14.648 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4193280 len 1024 2024-04-25T14:15:14.648 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4193280 len 1024 2024-04-25T14:15:14.648 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4193280 len 1024 2024-04-25T14:15:14.648 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4193280 len 2048 2024-04-25T14:15:14.648 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4193280 len 2048 2024-04-25T14:15:14.648 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4193280 len 2048 2024-04-25T14:15:14.648 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4193280 len 2048 2024-04-25T14:15:14.648 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4193280 len 4096 2024-04-25T14:15:14.648 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4193280 len 4096 2024-04-25T14:15:14.648 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4193280 len 4096 2024-04-25T14:15:14.648 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4193280 len 4096 2024-04-25T14:15:14.648 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4193280 len 8192 2024-04-25T14:15:14.648 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4193280 len 8192 2024-04-25T14:15:14.649 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4193280 len 8192 2024-04-25T14:15:14.649 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4193280 len 8192 2024-04-25T14:15:14.649 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4193280 len 16384 2024-04-25T14:15:14.649 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4193280 len 16384 2024-04-25T14:15:14.649 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4193280 len 16384 2024-04-25T14:15:14.649 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4193280 len 16384 2024-04-25T14:15:14.649 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4193792 len 1024 2024-04-25T14:15:14.649 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4193792 len 1024 2024-04-25T14:15:14.649 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4193792 len 1024 2024-04-25T14:15:14.649 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4193792 len 1024 2024-04-25T14:15:14.649 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4193792 len 2048 2024-04-25T14:15:14.649 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4193792 len 2048 2024-04-25T14:15:14.649 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4193792 len 2048 2024-04-25T14:15:14.649 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4193792 len 2048 2024-04-25T14:15:14.649 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4193792 len 4096 2024-04-25T14:15:14.650 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4193792 len 4096 2024-04-25T14:15:14.650 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4193792 len 4096 2024-04-25T14:15:14.650 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4193792 len 4096 2024-04-25T14:15:14.650 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4193792 len 8192 2024-04-25T14:15:14.650 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4193792 len 8192 2024-04-25T14:15:14.650 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4193792 len 8192 2024-04-25T14:15:14.650 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4193792 len 8192 2024-04-25T14:15:14.650 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4193792 len 16384 2024-04-25T14:15:14.650 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4193792 len 16384 2024-04-25T14:15:14.650 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4193792 len 16384 2024-04-25T14:15:14.650 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4193792 len 16384 2024-04-25T14:15:14.650 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4194304 len 1024 2024-04-25T14:15:14.650 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4194304 len 1024 2024-04-25T14:15:14.650 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4194304 len 1024 2024-04-25T14:15:14.650 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4194304 len 1024 2024-04-25T14:15:14.651 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4194304 len 2048 2024-04-25T14:15:14.651 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4194304 len 2048 2024-04-25T14:15:14.651 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4194304 len 2048 2024-04-25T14:15:14.651 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4194304 len 2048 2024-04-25T14:15:14.651 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4194304 len 4096 2024-04-25T14:15:14.651 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4194304 len 4096 2024-04-25T14:15:14.651 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4194304 len 4096 2024-04-25T14:15:14.651 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4194304 len 4096 2024-04-25T14:15:14.651 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4194304 len 8192 2024-04-25T14:15:14.651 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4194304 len 8192 2024-04-25T14:15:14.651 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4194304 len 8192 2024-04-25T14:15:14.655 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4194304 len 8192 2024-04-25T14:15:14.655 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4194304 len 16384 2024-04-25T14:15:14.655 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4194304 len 16384 2024-04-25T14:15:14.655 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4194304 len 16384 2024-04-25T14:15:14.655 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4194304 len 16384 2024-04-25T14:15:14.655 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4194816 len 1024 2024-04-25T14:15:14.655 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4194816 len 1024 2024-04-25T14:15:14.655 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4194816 len 1024 2024-04-25T14:15:14.655 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4194816 len 1024 2024-04-25T14:15:14.655 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4194816 len 2048 2024-04-25T14:15:14.655 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4194816 len 2048 2024-04-25T14:15:14.655 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4194816 len 2048 2024-04-25T14:15:14.656 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4194816 len 2048 2024-04-25T14:15:14.656 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4194816 len 4096 2024-04-25T14:15:14.656 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4194816 len 4096 2024-04-25T14:15:14.656 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4194816 len 4096 2024-04-25T14:15:14.656 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4194816 len 4096 2024-04-25T14:15:14.656 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4194816 len 8192 2024-04-25T14:15:14.656 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4194816 len 8192 2024-04-25T14:15:14.656 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4194816 len 8192 2024-04-25T14:15:14.656 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4194816 len 8192 2024-04-25T14:15:14.656 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4194816 len 16384 2024-04-25T14:15:14.656 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4194816 len 16384 2024-04-25T14:15:14.656 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4194816 len 16384 2024-04-25T14:15:14.656 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4194816 len 16384 2024-04-25T14:15:14.656 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4195328 len 1024 2024-04-25T14:15:14.656 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4195328 len 1024 2024-04-25T14:15:14.657 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4195328 len 1024 2024-04-25T14:15:14.657 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4195328 len 1024 2024-04-25T14:15:14.657 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4195328 len 2048 2024-04-25T14:15:14.657 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4195328 len 2048 2024-04-25T14:15:14.657 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4195328 len 2048 2024-04-25T14:15:14.657 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4195328 len 2048 2024-04-25T14:15:14.657 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4195328 len 4096 2024-04-25T14:15:14.657 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4195328 len 4096 2024-04-25T14:15:14.657 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4195328 len 4096 2024-04-25T14:15:14.657 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4195328 len 4096 2024-04-25T14:15:14.657 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4195328 len 8192 2024-04-25T14:15:14.657 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4195328 len 8192 2024-04-25T14:15:14.657 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4195328 len 8192 2024-04-25T14:15:14.657 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4195328 len 8192 2024-04-25T14:15:14.657 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4195328 len 16384 2024-04-25T14:15:14.658 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4195328 len 16384 2024-04-25T14:15:14.658 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4195328 len 16384 2024-04-25T14:15:14.658 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4195328 len 16384 2024-04-25T14:15:14.658 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4195840 len 1024 2024-04-25T14:15:14.658 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4195840 len 1024 2024-04-25T14:15:14.658 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4195840 len 1024 2024-04-25T14:15:14.658 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4195840 len 1024 2024-04-25T14:15:14.658 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4195840 len 2048 2024-04-25T14:15:14.658 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4195840 len 2048 2024-04-25T14:15:14.658 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4195840 len 2048 2024-04-25T14:15:14.658 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4195840 len 2048 2024-04-25T14:15:14.658 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4195840 len 4096 2024-04-25T14:15:14.658 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4195840 len 4096 2024-04-25T14:15:14.658 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4195840 len 4096 2024-04-25T14:15:14.659 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4195840 len 4096 2024-04-25T14:15:14.659 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4195840 len 8192 2024-04-25T14:15:14.659 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4195840 len 8192 2024-04-25T14:15:14.659 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4195840 len 8192 2024-04-25T14:15:14.659 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4195840 len 8192 2024-04-25T14:15:14.659 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4195840 len 16384 2024-04-25T14:15:14.659 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4195840 len 16384 2024-04-25T14:15:14.659 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4195840 len 16384 2024-04-25T14:15:14.659 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4195840 len 16384 2024-04-25T14:15:14.659 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4196352 len 1024 2024-04-25T14:15:14.659 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4196352 len 1024 2024-04-25T14:15:14.659 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4196352 len 1024 2024-04-25T14:15:14.659 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4196352 len 1024 2024-04-25T14:15:14.659 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4196352 len 2048 2024-04-25T14:15:14.659 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4196352 len 2048 2024-04-25T14:15:14.660 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4196352 len 2048 2024-04-25T14:15:14.660 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4196352 len 2048 2024-04-25T14:15:14.660 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4196352 len 4096 2024-04-25T14:15:14.660 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4196352 len 4096 2024-04-25T14:15:14.660 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4196352 len 4096 2024-04-25T14:15:14.660 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4196352 len 4096 2024-04-25T14:15:14.660 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4196352 len 8192 2024-04-25T14:15:14.660 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4196352 len 8192 2024-04-25T14:15:14.660 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4196352 len 8192 2024-04-25T14:15:14.660 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4196352 len 8192 2024-04-25T14:15:14.660 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4196352 len 16384 2024-04-25T14:15:14.660 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4196352 len 16384 2024-04-25T14:15:14.660 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4196352 len 16384 2024-04-25T14:15:14.660 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4196352 len 16384 2024-04-25T14:15:14.660 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4196864 len 1024 2024-04-25T14:15:14.660 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4196864 len 1024 2024-04-25T14:15:14.666 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4196864 len 1024 2024-04-25T14:15:14.666 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4196864 len 1024 2024-04-25T14:15:14.666 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4196864 len 2048 2024-04-25T14:15:14.666 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4196864 len 2048 2024-04-25T14:15:14.666 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4196864 len 2048 2024-04-25T14:15:14.666 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4196864 len 2048 2024-04-25T14:15:14.666 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4196864 len 4096 2024-04-25T14:15:14.667 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4196864 len 4096 2024-04-25T14:15:14.667 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4196864 len 4096 2024-04-25T14:15:14.667 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4196864 len 4096 2024-04-25T14:15:14.667 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4196864 len 8192 2024-04-25T14:15:14.667 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4196864 len 8192 2024-04-25T14:15:14.667 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4196864 len 8192 2024-04-25T14:15:14.667 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4196864 len 8192 2024-04-25T14:15:14.667 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4196864 len 16384 2024-04-25T14:15:14.667 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4196864 len 16384 2024-04-25T14:15:14.667 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4196864 len 16384 2024-04-25T14:15:14.667 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4196864 len 16384 2024-04-25T14:15:14.667 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4197376 len 1024 2024-04-25T14:15:14.667 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4197376 len 1024 2024-04-25T14:15:14.667 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4197376 len 1024 2024-04-25T14:15:14.667 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4197376 len 1024 2024-04-25T14:15:14.667 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4197376 len 2048 2024-04-25T14:15:14.668 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4197376 len 2048 2024-04-25T14:15:14.668 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4197376 len 2048 2024-04-25T14:15:14.668 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4197376 len 2048 2024-04-25T14:15:14.668 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4197376 len 4096 2024-04-25T14:15:14.668 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4197376 len 4096 2024-04-25T14:15:14.668 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4197376 len 4096 2024-04-25T14:15:14.668 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4197376 len 4096 2024-04-25T14:15:14.668 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4197376 len 8192 2024-04-25T14:15:14.668 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4197376 len 8192 2024-04-25T14:15:14.668 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4197376 len 8192 2024-04-25T14:15:14.668 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4197376 len 8192 2024-04-25T14:15:14.668 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4197376 len 16384 2024-04-25T14:15:14.668 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4197376 len 16384 2024-04-25T14:15:14.668 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4197376 len 16384 2024-04-25T14:15:14.668 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4197376 len 16384 2024-04-25T14:15:14.669 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4197888 len 1024 2024-04-25T14:15:14.669 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4197888 len 1024 2024-04-25T14:15:14.669 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4197888 len 1024 2024-04-25T14:15:14.669 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4197888 len 1024 2024-04-25T14:15:14.669 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4197888 len 2048 2024-04-25T14:15:14.669 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4197888 len 2048 2024-04-25T14:15:14.669 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4197888 len 2048 2024-04-25T14:15:14.669 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4197888 len 2048 2024-04-25T14:15:14.669 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4197888 len 4096 2024-04-25T14:15:14.669 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4197888 len 4096 2024-04-25T14:15:14.669 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4197888 len 4096 2024-04-25T14:15:14.669 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4197888 len 4096 2024-04-25T14:15:14.669 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4197888 len 8192 2024-04-25T14:15:14.669 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4197888 len 8192 2024-04-25T14:15:14.669 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4197888 len 8192 2024-04-25T14:15:14.670 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4197888 len 8192 2024-04-25T14:15:14.670 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 0 offset 4197888 len 16384 2024-04-25T14:15:14.670 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4197888 len 16384 2024-04-25T14:15:14.670 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 0 offset 4197888 len 16384 2024-04-25T14:15:14.670 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 0 offset 4197888 len 16384 2024-04-25T14:15:14.670 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4190208 len 1024 2024-04-25T14:15:14.670 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4190208 len 1024 2024-04-25T14:15:14.670 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4190208 len 1024 2024-04-25T14:15:14.670 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4190208 len 1024 2024-04-25T14:15:14.670 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4190208 len 2048 2024-04-25T14:15:14.670 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4190208 len 2048 2024-04-25T14:15:14.670 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4190208 len 2048 2024-04-25T14:15:14.670 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4190208 len 2048 2024-04-25T14:15:14.670 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4190208 len 4096 2024-04-25T14:15:14.671 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4190208 len 4096 2024-04-25T14:15:14.671 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4190208 len 4096 2024-04-25T14:15:14.671 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4190208 len 4096 2024-04-25T14:15:14.671 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4190208 len 8192 2024-04-25T14:15:14.671 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4190208 len 8192 2024-04-25T14:15:14.671 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4190208 len 8192 2024-04-25T14:15:14.671 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4190208 len 8192 2024-04-25T14:15:14.671 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4190208 len 16384 2024-04-25T14:15:14.671 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4190208 len 16384 2024-04-25T14:15:14.671 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4190208 len 16384 2024-04-25T14:15:14.671 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4190208 len 16384 2024-04-25T14:15:14.671 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4190720 len 1024 2024-04-25T14:15:14.671 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4190720 len 1024 2024-04-25T14:15:14.671 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4190720 len 1024 2024-04-25T14:15:14.672 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4190720 len 1024 2024-04-25T14:15:14.672 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4190720 len 2048 2024-04-25T14:15:14.672 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4190720 len 2048 2024-04-25T14:15:14.672 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4190720 len 2048 2024-04-25T14:15:14.672 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4190720 len 2048 2024-04-25T14:15:14.672 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4190720 len 4096 2024-04-25T14:15:14.686 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4190720 len 4096 2024-04-25T14:15:14.686 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4190720 len 4096 2024-04-25T14:15:14.686 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4190720 len 4096 2024-04-25T14:15:14.686 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4190720 len 8192 2024-04-25T14:15:14.686 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4190720 len 8192 2024-04-25T14:15:14.687 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4190720 len 8192 2024-04-25T14:15:14.687 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4190720 len 8192 2024-04-25T14:15:14.687 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4190720 len 16384 2024-04-25T14:15:14.687 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4190720 len 16384 2024-04-25T14:15:14.687 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4190720 len 16384 2024-04-25T14:15:14.687 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4190720 len 16384 2024-04-25T14:15:14.687 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4191232 len 1024 2024-04-25T14:15:14.687 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4191232 len 1024 2024-04-25T14:15:14.687 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4191232 len 1024 2024-04-25T14:15:14.687 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4191232 len 1024 2024-04-25T14:15:14.687 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4191232 len 2048 2024-04-25T14:15:14.687 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4191232 len 2048 2024-04-25T14:15:14.687 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4191232 len 2048 2024-04-25T14:15:14.687 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4191232 len 2048 2024-04-25T14:15:14.687 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4191232 len 4096 2024-04-25T14:15:14.688 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4191232 len 4096 2024-04-25T14:15:14.688 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4191232 len 4096 2024-04-25T14:15:14.688 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4191232 len 4096 2024-04-25T14:15:14.688 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4191232 len 8192 2024-04-25T14:15:14.688 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4191232 len 8192 2024-04-25T14:15:14.688 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4191232 len 8192 2024-04-25T14:15:14.688 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4191232 len 8192 2024-04-25T14:15:14.688 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4191232 len 16384 2024-04-25T14:15:14.688 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4191232 len 16384 2024-04-25T14:15:14.688 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4191232 len 16384 2024-04-25T14:15:14.688 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4191232 len 16384 2024-04-25T14:15:14.688 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4191744 len 1024 2024-04-25T14:15:14.688 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4191744 len 1024 2024-04-25T14:15:14.688 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4191744 len 1024 2024-04-25T14:15:14.688 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4191744 len 1024 2024-04-25T14:15:14.689 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4191744 len 2048 2024-04-25T14:15:14.689 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4191744 len 2048 2024-04-25T14:15:14.689 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4191744 len 2048 2024-04-25T14:15:14.689 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4191744 len 2048 2024-04-25T14:15:14.689 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4191744 len 4096 2024-04-25T14:15:14.689 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4191744 len 4096 2024-04-25T14:15:14.689 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4191744 len 4096 2024-04-25T14:15:14.689 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4191744 len 4096 2024-04-25T14:15:14.689 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4191744 len 8192 2024-04-25T14:15:14.689 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4191744 len 8192 2024-04-25T14:15:14.689 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4191744 len 8192 2024-04-25T14:15:14.689 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4191744 len 8192 2024-04-25T14:15:14.689 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4191744 len 16384 2024-04-25T14:15:14.689 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4191744 len 16384 2024-04-25T14:15:14.689 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4191744 len 16384 2024-04-25T14:15:14.689 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4191744 len 16384 2024-04-25T14:15:14.690 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4192256 len 1024 2024-04-25T14:15:14.690 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4192256 len 1024 2024-04-25T14:15:14.690 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4192256 len 1024 2024-04-25T14:15:14.690 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4192256 len 1024 2024-04-25T14:15:14.690 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4192256 len 2048 2024-04-25T14:15:14.690 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4192256 len 2048 2024-04-25T14:15:14.690 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4192256 len 2048 2024-04-25T14:15:14.690 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4192256 len 2048 2024-04-25T14:15:14.690 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4192256 len 4096 2024-04-25T14:15:14.690 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4192256 len 4096 2024-04-25T14:15:14.690 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4192256 len 4096 2024-04-25T14:15:14.690 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4192256 len 4096 2024-04-25T14:15:14.690 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4192256 len 8192 2024-04-25T14:15:14.690 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4192256 len 8192 2024-04-25T14:15:14.691 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4192256 len 8192 2024-04-25T14:15:14.691 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4192256 len 8192 2024-04-25T14:15:14.691 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4192256 len 16384 2024-04-25T14:15:14.691 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4192256 len 16384 2024-04-25T14:15:14.691 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4192256 len 16384 2024-04-25T14:15:14.691 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4192256 len 16384 2024-04-25T14:15:14.691 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4192768 len 1024 2024-04-25T14:15:14.691 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4192768 len 1024 2024-04-25T14:15:14.691 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4192768 len 1024 2024-04-25T14:15:14.691 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4192768 len 1024 2024-04-25T14:15:14.691 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4192768 len 2048 2024-04-25T14:15:14.691 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4192768 len 2048 2024-04-25T14:15:14.691 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4192768 len 2048 2024-04-25T14:15:14.691 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4192768 len 2048 2024-04-25T14:15:14.691 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4192768 len 4096 2024-04-25T14:15:14.692 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4192768 len 4096 2024-04-25T14:15:14.692 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4192768 len 4096 2024-04-25T14:15:14.692 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4192768 len 4096 2024-04-25T14:15:14.692 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4192768 len 8192 2024-04-25T14:15:14.706 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4192768 len 8192 2024-04-25T14:15:14.707 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4192768 len 8192 2024-04-25T14:15:14.707 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4192768 len 8192 2024-04-25T14:15:14.707 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4192768 len 16384 2024-04-25T14:15:14.707 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4192768 len 16384 2024-04-25T14:15:14.707 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4192768 len 16384 2024-04-25T14:15:14.707 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4192768 len 16384 2024-04-25T14:15:14.707 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4193280 len 1024 2024-04-25T14:15:14.707 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4193280 len 1024 2024-04-25T14:15:14.707 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4193280 len 1024 2024-04-25T14:15:14.707 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4193280 len 1024 2024-04-25T14:15:14.707 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4193280 len 2048 2024-04-25T14:15:14.707 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4193280 len 2048 2024-04-25T14:15:14.707 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4193280 len 2048 2024-04-25T14:15:14.708 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4193280 len 2048 2024-04-25T14:15:14.708 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4193280 len 4096 2024-04-25T14:15:14.708 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4193280 len 4096 2024-04-25T14:15:14.708 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4193280 len 4096 2024-04-25T14:15:14.708 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4193280 len 4096 2024-04-25T14:15:14.708 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4193280 len 8192 2024-04-25T14:15:14.708 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4193280 len 8192 2024-04-25T14:15:14.708 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4193280 len 8192 2024-04-25T14:15:14.708 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4193280 len 8192 2024-04-25T14:15:14.708 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4193280 len 16384 2024-04-25T14:15:14.708 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4193280 len 16384 2024-04-25T14:15:14.708 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4193280 len 16384 2024-04-25T14:15:14.708 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4193280 len 16384 2024-04-25T14:15:14.709 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4193792 len 1024 2024-04-25T14:15:14.709 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4193792 len 1024 2024-04-25T14:15:14.709 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4193792 len 1024 2024-04-25T14:15:14.709 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4193792 len 1024 2024-04-25T14:15:14.709 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4193792 len 2048 2024-04-25T14:15:14.709 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4193792 len 2048 2024-04-25T14:15:14.709 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4193792 len 2048 2024-04-25T14:15:14.709 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4193792 len 2048 2024-04-25T14:15:14.709 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4193792 len 4096 2024-04-25T14:15:14.709 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4193792 len 4096 2024-04-25T14:15:14.709 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4193792 len 4096 2024-04-25T14:15:14.709 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4193792 len 4096 2024-04-25T14:15:14.709 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4193792 len 8192 2024-04-25T14:15:14.709 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4193792 len 8192 2024-04-25T14:15:14.709 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4193792 len 8192 2024-04-25T14:15:14.709 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4193792 len 8192 2024-04-25T14:15:14.710 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4193792 len 16384 2024-04-25T14:15:14.710 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4193792 len 16384 2024-04-25T14:15:14.710 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4193792 len 16384 2024-04-25T14:15:14.710 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4193792 len 16384 2024-04-25T14:15:14.710 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4194304 len 1024 2024-04-25T14:15:14.710 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4194304 len 1024 2024-04-25T14:15:14.710 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4194304 len 1024 2024-04-25T14:15:14.710 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4194304 len 1024 2024-04-25T14:15:14.710 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4194304 len 2048 2024-04-25T14:15:14.710 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4194304 len 2048 2024-04-25T14:15:14.710 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4194304 len 2048 2024-04-25T14:15:14.710 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4194304 len 2048 2024-04-25T14:15:14.710 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4194304 len 4096 2024-04-25T14:15:14.710 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4194304 len 4096 2024-04-25T14:15:14.711 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4194304 len 4096 2024-04-25T14:15:14.711 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4194304 len 4096 2024-04-25T14:15:14.711 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4194304 len 8192 2024-04-25T14:15:14.711 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4194304 len 8192 2024-04-25T14:15:14.711 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4194304 len 8192 2024-04-25T14:15:14.711 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4194304 len 8192 2024-04-25T14:15:14.711 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4194304 len 16384 2024-04-25T14:15:14.711 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4194304 len 16384 2024-04-25T14:15:14.711 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4194304 len 16384 2024-04-25T14:15:14.711 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4194304 len 16384 2024-04-25T14:15:14.711 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4194816 len 1024 2024-04-25T14:15:14.711 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4194816 len 1024 2024-04-25T14:15:14.711 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4194816 len 1024 2024-04-25T14:15:14.711 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4194816 len 1024 2024-04-25T14:15:14.711 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4194816 len 2048 2024-04-25T14:15:14.713 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4194816 len 2048 2024-04-25T14:15:14.713 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4194816 len 2048 2024-04-25T14:15:14.713 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4194816 len 2048 2024-04-25T14:15:14.713 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4194816 len 4096 2024-04-25T14:15:14.713 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4194816 len 4096 2024-04-25T14:15:14.713 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4194816 len 4096 2024-04-25T14:15:14.713 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4194816 len 4096 2024-04-25T14:15:14.713 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4194816 len 8192 2024-04-25T14:15:14.714 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4194816 len 8192 2024-04-25T14:15:14.714 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4194816 len 8192 2024-04-25T14:15:14.714 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4194816 len 8192 2024-04-25T14:15:14.714 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4194816 len 16384 2024-04-25T14:15:14.717 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4194816 len 16384 2024-04-25T14:15:14.717 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4194816 len 16384 2024-04-25T14:15:14.717 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4194816 len 16384 2024-04-25T14:15:14.717 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4195328 len 1024 2024-04-25T14:15:14.717 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4195328 len 1024 2024-04-25T14:15:14.717 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4195328 len 1024 2024-04-25T14:15:14.717 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4195328 len 1024 2024-04-25T14:15:14.717 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4195328 len 2048 2024-04-25T14:15:14.718 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4195328 len 2048 2024-04-25T14:15:14.718 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4195328 len 2048 2024-04-25T14:15:14.718 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4195328 len 2048 2024-04-25T14:15:14.718 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4195328 len 4096 2024-04-25T14:15:14.718 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4195328 len 4096 2024-04-25T14:15:14.718 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4195328 len 4096 2024-04-25T14:15:14.718 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4195328 len 4096 2024-04-25T14:15:14.718 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4195328 len 8192 2024-04-25T14:15:14.718 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4195328 len 8192 2024-04-25T14:15:14.718 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4195328 len 8192 2024-04-25T14:15:14.718 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4195328 len 8192 2024-04-25T14:15:14.718 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4195328 len 16384 2024-04-25T14:15:14.718 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4195328 len 16384 2024-04-25T14:15:14.718 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4195328 len 16384 2024-04-25T14:15:14.719 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4195328 len 16384 2024-04-25T14:15:14.719 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4195840 len 1024 2024-04-25T14:15:14.719 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4195840 len 1024 2024-04-25T14:15:14.719 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4195840 len 1024 2024-04-25T14:15:14.719 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4195840 len 1024 2024-04-25T14:15:14.719 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4195840 len 2048 2024-04-25T14:15:14.719 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4195840 len 2048 2024-04-25T14:15:14.719 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4195840 len 2048 2024-04-25T14:15:14.719 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4195840 len 2048 2024-04-25T14:15:14.719 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4195840 len 4096 2024-04-25T14:15:14.719 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4195840 len 4096 2024-04-25T14:15:14.719 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4195840 len 4096 2024-04-25T14:15:14.719 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4195840 len 4096 2024-04-25T14:15:14.719 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4195840 len 8192 2024-04-25T14:15:14.720 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4195840 len 8192 2024-04-25T14:15:14.720 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4195840 len 8192 2024-04-25T14:15:14.720 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4195840 len 8192 2024-04-25T14:15:14.720 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4195840 len 16384 2024-04-25T14:15:14.720 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4195840 len 16384 2024-04-25T14:15:14.720 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4195840 len 16384 2024-04-25T14:15:14.720 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4195840 len 16384 2024-04-25T14:15:14.720 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4196352 len 1024 2024-04-25T14:15:14.720 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4196352 len 1024 2024-04-25T14:15:14.720 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4196352 len 1024 2024-04-25T14:15:14.720 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4196352 len 1024 2024-04-25T14:15:14.720 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4196352 len 2048 2024-04-25T14:15:14.720 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4196352 len 2048 2024-04-25T14:15:14.720 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4196352 len 2048 2024-04-25T14:15:14.720 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4196352 len 2048 2024-04-25T14:15:14.721 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4196352 len 4096 2024-04-25T14:15:14.721 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4196352 len 4096 2024-04-25T14:15:14.721 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4196352 len 4096 2024-04-25T14:15:14.721 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4196352 len 4096 2024-04-25T14:15:14.721 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4196352 len 8192 2024-04-25T14:15:14.721 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4196352 len 8192 2024-04-25T14:15:14.721 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4196352 len 8192 2024-04-25T14:15:14.721 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4196352 len 8192 2024-04-25T14:15:14.721 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4196352 len 16384 2024-04-25T14:15:14.721 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4196352 len 16384 2024-04-25T14:15:14.721 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4196352 len 16384 2024-04-25T14:15:14.721 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4196352 len 16384 2024-04-25T14:15:14.721 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4196864 len 1024 2024-04-25T14:15:14.721 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4196864 len 1024 2024-04-25T14:15:14.721 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4196864 len 1024 2024-04-25T14:15:14.721 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4196864 len 1024 2024-04-25T14:15:14.722 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4196864 len 2048 2024-04-25T14:15:14.722 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4196864 len 2048 2024-04-25T14:15:14.722 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4196864 len 2048 2024-04-25T14:15:14.722 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4196864 len 2048 2024-04-25T14:15:14.722 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4196864 len 4096 2024-04-25T14:15:14.722 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4196864 len 4096 2024-04-25T14:15:14.722 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4196864 len 4096 2024-04-25T14:15:14.722 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4196864 len 4096 2024-04-25T14:15:14.722 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4196864 len 8192 2024-04-25T14:15:14.722 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4196864 len 8192 2024-04-25T14:15:14.722 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4196864 len 8192 2024-04-25T14:15:14.722 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4196864 len 8192 2024-04-25T14:15:14.722 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4196864 len 16384 2024-04-25T14:15:14.722 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4196864 len 16384 2024-04-25T14:15:14.722 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4196864 len 16384 2024-04-25T14:15:14.723 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4196864 len 16384 2024-04-25T14:15:14.723 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4197376 len 1024 2024-04-25T14:15:14.731 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4197376 len 1024 2024-04-25T14:15:14.732 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4197376 len 1024 2024-04-25T14:15:14.732 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4197376 len 1024 2024-04-25T14:15:14.732 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4197376 len 2048 2024-04-25T14:15:14.732 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4197376 len 2048 2024-04-25T14:15:14.732 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4197376 len 2048 2024-04-25T14:15:14.732 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4197376 len 2048 2024-04-25T14:15:14.732 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4197376 len 4096 2024-04-25T14:15:14.732 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4197376 len 4096 2024-04-25T14:15:14.732 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4197376 len 4096 2024-04-25T14:15:14.732 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4197376 len 4096 2024-04-25T14:15:14.732 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4197376 len 8192 2024-04-25T14:15:14.732 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4197376 len 8192 2024-04-25T14:15:14.732 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4197376 len 8192 2024-04-25T14:15:14.732 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4197376 len 8192 2024-04-25T14:15:14.733 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4197376 len 16384 2024-04-25T14:15:14.733 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4197376 len 16384 2024-04-25T14:15:14.733 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4197376 len 16384 2024-04-25T14:15:14.733 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4197376 len 16384 2024-04-25T14:15:14.733 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4197888 len 1024 2024-04-25T14:15:14.733 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4197888 len 1024 2024-04-25T14:15:14.733 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4197888 len 1024 2024-04-25T14:15:14.733 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4197888 len 1024 2024-04-25T14:15:14.733 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4197888 len 2048 2024-04-25T14:15:14.733 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4197888 len 2048 2024-04-25T14:15:14.733 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4197888 len 2048 2024-04-25T14:15:14.733 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4197888 len 2048 2024-04-25T14:15:14.733 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4197888 len 4096 2024-04-25T14:15:14.733 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4197888 len 4096 2024-04-25T14:15:14.733 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4197888 len 4096 2024-04-25T14:15:14.734 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4197888 len 4096 2024-04-25T14:15:14.734 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4197888 len 8192 2024-04-25T14:15:14.734 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4197888 len 8192 2024-04-25T14:15:14.734 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4197888 len 8192 2024-04-25T14:15:14.734 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4197888 len 8192 2024-04-25T14:15:14.734 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 512 offset 4197888 len 16384 2024-04-25T14:15:14.734 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4197888 len 16384 2024-04-25T14:15:14.734 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 512 offset 4197888 len 16384 2024-04-25T14:15:14.734 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 512 offset 4197888 len 16384 2024-04-25T14:15:14.734 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4190208 len 1024 2024-04-25T14:15:14.734 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4190208 len 1024 2024-04-25T14:15:14.734 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4190208 len 1024 2024-04-25T14:15:14.734 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4190208 len 1024 2024-04-25T14:15:14.734 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4190208 len 2048 2024-04-25T14:15:14.734 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4190208 len 2048 2024-04-25T14:15:14.735 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4190208 len 2048 2024-04-25T14:15:14.735 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4190208 len 2048 2024-04-25T14:15:14.735 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4190208 len 4096 2024-04-25T14:15:14.735 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4190208 len 4096 2024-04-25T14:15:14.735 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4190208 len 4096 2024-04-25T14:15:14.735 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4190208 len 4096 2024-04-25T14:15:14.735 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4190208 len 8192 2024-04-25T14:15:14.735 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4190208 len 8192 2024-04-25T14:15:14.735 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4190208 len 8192 2024-04-25T14:15:14.735 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4190208 len 8192 2024-04-25T14:15:14.735 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4190208 len 16384 2024-04-25T14:15:14.735 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4190208 len 16384 2024-04-25T14:15:14.735 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4190208 len 16384 2024-04-25T14:15:14.735 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4190208 len 16384 2024-04-25T14:15:14.735 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4190720 len 1024 2024-04-25T14:15:14.735 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4190720 len 1024 2024-04-25T14:15:14.736 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4190720 len 1024 2024-04-25T14:15:14.736 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4190720 len 1024 2024-04-25T14:15:14.736 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4190720 len 2048 2024-04-25T14:15:14.736 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4190720 len 2048 2024-04-25T14:15:14.736 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4190720 len 2048 2024-04-25T14:15:14.736 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4190720 len 2048 2024-04-25T14:15:14.736 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4190720 len 4096 2024-04-25T14:15:14.736 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4190720 len 4096 2024-04-25T14:15:14.736 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4190720 len 4096 2024-04-25T14:15:14.736 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4190720 len 4096 2024-04-25T14:15:14.736 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4190720 len 8192 2024-04-25T14:15:14.736 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4190720 len 8192 2024-04-25T14:15:14.736 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4190720 len 8192 2024-04-25T14:15:14.736 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4190720 len 8192 2024-04-25T14:15:14.736 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4190720 len 16384 2024-04-25T14:15:14.737 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4190720 len 16384 2024-04-25T14:15:14.737 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4190720 len 16384 2024-04-25T14:15:14.737 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4190720 len 16384 2024-04-25T14:15:14.737 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4191232 len 1024 2024-04-25T14:15:14.737 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4191232 len 1024 2024-04-25T14:15:14.737 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4191232 len 1024 2024-04-25T14:15:14.737 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4191232 len 1024 2024-04-25T14:15:14.754 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4191232 len 2048 2024-04-25T14:15:14.754 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4191232 len 2048 2024-04-25T14:15:14.754 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4191232 len 2048 2024-04-25T14:15:14.754 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4191232 len 2048 2024-04-25T14:15:14.754 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4191232 len 4096 2024-04-25T14:15:14.754 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4191232 len 4096 2024-04-25T14:15:14.754 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4191232 len 4096 2024-04-25T14:15:14.755 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4191232 len 4096 2024-04-25T14:15:14.755 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4191232 len 8192 2024-04-25T14:15:14.755 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4191232 len 8192 2024-04-25T14:15:14.755 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4191232 len 8192 2024-04-25T14:15:14.755 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4191232 len 8192 2024-04-25T14:15:14.755 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4191232 len 16384 2024-04-25T14:15:14.755 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4191232 len 16384 2024-04-25T14:15:14.755 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4191232 len 16384 2024-04-25T14:15:14.755 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4191232 len 16384 2024-04-25T14:15:14.755 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4191744 len 1024 2024-04-25T14:15:14.755 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4191744 len 1024 2024-04-25T14:15:14.755 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4191744 len 1024 2024-04-25T14:15:14.755 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4191744 len 1024 2024-04-25T14:15:14.755 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4191744 len 2048 2024-04-25T14:15:14.755 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4191744 len 2048 2024-04-25T14:15:14.755 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4191744 len 2048 2024-04-25T14:15:14.756 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4191744 len 2048 2024-04-25T14:15:14.756 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4191744 len 4096 2024-04-25T14:15:14.756 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4191744 len 4096 2024-04-25T14:15:14.756 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4191744 len 4096 2024-04-25T14:15:14.756 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4191744 len 4096 2024-04-25T14:15:14.756 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4191744 len 8192 2024-04-25T14:15:14.756 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4191744 len 8192 2024-04-25T14:15:14.756 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4191744 len 8192 2024-04-25T14:15:14.756 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4191744 len 8192 2024-04-25T14:15:14.756 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4191744 len 16384 2024-04-25T14:15:14.756 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4191744 len 16384 2024-04-25T14:15:14.756 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4191744 len 16384 2024-04-25T14:15:14.756 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4191744 len 16384 2024-04-25T14:15:14.756 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4192256 len 1024 2024-04-25T14:15:14.756 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4192256 len 1024 2024-04-25T14:15:14.757 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4192256 len 1024 2024-04-25T14:15:14.757 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4192256 len 1024 2024-04-25T14:15:14.757 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4192256 len 2048 2024-04-25T14:15:14.757 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4192256 len 2048 2024-04-25T14:15:14.757 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4192256 len 2048 2024-04-25T14:15:14.757 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4192256 len 2048 2024-04-25T14:15:14.757 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4192256 len 4096 2024-04-25T14:15:14.757 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4192256 len 4096 2024-04-25T14:15:14.757 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4192256 len 4096 2024-04-25T14:15:14.757 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4192256 len 4096 2024-04-25T14:15:14.757 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4192256 len 8192 2024-04-25T14:15:14.757 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4192256 len 8192 2024-04-25T14:15:14.757 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4192256 len 8192 2024-04-25T14:15:14.757 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4192256 len 8192 2024-04-25T14:15:14.758 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4192256 len 16384 2024-04-25T14:15:14.758 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4192256 len 16384 2024-04-25T14:15:14.758 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4192256 len 16384 2024-04-25T14:15:14.758 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4192256 len 16384 2024-04-25T14:15:14.758 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4192768 len 1024 2024-04-25T14:15:14.758 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4192768 len 1024 2024-04-25T14:15:14.758 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4192768 len 1024 2024-04-25T14:15:14.758 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4192768 len 1024 2024-04-25T14:15:14.758 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4192768 len 2048 2024-04-25T14:15:14.758 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4192768 len 2048 2024-04-25T14:15:14.758 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4192768 len 2048 2024-04-25T14:15:14.758 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4192768 len 2048 2024-04-25T14:15:14.758 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4192768 len 4096 2024-04-25T14:15:14.758 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4192768 len 4096 2024-04-25T14:15:14.759 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4192768 len 4096 2024-04-25T14:15:14.759 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4192768 len 4096 2024-04-25T14:15:14.759 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4192768 len 8192 2024-04-25T14:15:14.759 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4192768 len 8192 2024-04-25T14:15:14.759 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4192768 len 8192 2024-04-25T14:15:14.759 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4192768 len 8192 2024-04-25T14:15:14.759 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4192768 len 16384 2024-04-25T14:15:14.759 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4192768 len 16384 2024-04-25T14:15:14.759 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4192768 len 16384 2024-04-25T14:15:14.759 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4192768 len 16384 2024-04-25T14:15:14.759 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4193280 len 1024 2024-04-25T14:15:14.759 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4193280 len 1024 2024-04-25T14:15:14.759 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4193280 len 1024 2024-04-25T14:15:14.759 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4193280 len 1024 2024-04-25T14:15:14.759 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4193280 len 2048 2024-04-25T14:15:14.760 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4193280 len 2048 2024-04-25T14:15:14.760 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4193280 len 2048 2024-04-25T14:15:14.771 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4193280 len 2048 2024-04-25T14:15:14.771 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4193280 len 4096 2024-04-25T14:15:14.771 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4193280 len 4096 2024-04-25T14:15:14.771 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4193280 len 4096 2024-04-25T14:15:14.772 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4193280 len 4096 2024-04-25T14:15:14.772 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4193280 len 8192 2024-04-25T14:15:14.772 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4193280 len 8192 2024-04-25T14:15:14.772 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4193280 len 8192 2024-04-25T14:15:14.772 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4193280 len 8192 2024-04-25T14:15:14.772 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4193280 len 16384 2024-04-25T14:15:14.772 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4193280 len 16384 2024-04-25T14:15:14.772 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4193280 len 16384 2024-04-25T14:15:14.772 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4193280 len 16384 2024-04-25T14:15:14.772 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4193792 len 1024 2024-04-25T14:15:14.772 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4193792 len 1024 2024-04-25T14:15:14.772 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4193792 len 1024 2024-04-25T14:15:14.772 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4193792 len 1024 2024-04-25T14:15:14.772 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4193792 len 2048 2024-04-25T14:15:14.773 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4193792 len 2048 2024-04-25T14:15:14.773 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4193792 len 2048 2024-04-25T14:15:14.773 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4193792 len 2048 2024-04-25T14:15:14.773 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4193792 len 4096 2024-04-25T14:15:14.773 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4193792 len 4096 2024-04-25T14:15:14.773 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4193792 len 4096 2024-04-25T14:15:14.773 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4193792 len 4096 2024-04-25T14:15:14.773 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4193792 len 8192 2024-04-25T14:15:14.773 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4193792 len 8192 2024-04-25T14:15:14.773 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4193792 len 8192 2024-04-25T14:15:14.773 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4193792 len 8192 2024-04-25T14:15:14.773 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4193792 len 16384 2024-04-25T14:15:14.773 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4193792 len 16384 2024-04-25T14:15:14.773 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4193792 len 16384 2024-04-25T14:15:14.773 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4193792 len 16384 2024-04-25T14:15:14.773 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4194304 len 1024 2024-04-25T14:15:14.774 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4194304 len 1024 2024-04-25T14:15:14.774 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4194304 len 1024 2024-04-25T14:15:14.774 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4194304 len 1024 2024-04-25T14:15:14.774 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4194304 len 2048 2024-04-25T14:15:14.774 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4194304 len 2048 2024-04-25T14:15:14.774 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4194304 len 2048 2024-04-25T14:15:14.774 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4194304 len 2048 2024-04-25T14:15:14.774 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4194304 len 4096 2024-04-25T14:15:14.774 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4194304 len 4096 2024-04-25T14:15:14.774 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4194304 len 4096 2024-04-25T14:15:14.774 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4194304 len 4096 2024-04-25T14:15:14.774 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4194304 len 8192 2024-04-25T14:15:14.774 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4194304 len 8192 2024-04-25T14:15:14.774 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4194304 len 8192 2024-04-25T14:15:14.774 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4194304 len 8192 2024-04-25T14:15:14.775 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4194304 len 16384 2024-04-25T14:15:14.775 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4194304 len 16384 2024-04-25T14:15:14.775 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4194304 len 16384 2024-04-25T14:15:14.775 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4194304 len 16384 2024-04-25T14:15:14.775 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4194816 len 1024 2024-04-25T14:15:14.775 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4194816 len 1024 2024-04-25T14:15:14.775 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4194816 len 1024 2024-04-25T14:15:14.775 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4194816 len 1024 2024-04-25T14:15:14.775 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4194816 len 2048 2024-04-25T14:15:14.775 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4194816 len 2048 2024-04-25T14:15:14.775 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4194816 len 2048 2024-04-25T14:15:14.775 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4194816 len 2048 2024-04-25T14:15:14.775 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4194816 len 4096 2024-04-25T14:15:14.775 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4194816 len 4096 2024-04-25T14:15:14.775 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4194816 len 4096 2024-04-25T14:15:14.776 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4194816 len 4096 2024-04-25T14:15:14.776 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4194816 len 8192 2024-04-25T14:15:14.776 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4194816 len 8192 2024-04-25T14:15:14.776 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4194816 len 8192 2024-04-25T14:15:14.776 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4194816 len 8192 2024-04-25T14:15:14.776 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4194816 len 16384 2024-04-25T14:15:14.776 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4194816 len 16384 2024-04-25T14:15:14.776 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4194816 len 16384 2024-04-25T14:15:14.776 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4194816 len 16384 2024-04-25T14:15:14.776 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4195328 len 1024 2024-04-25T14:15:14.776 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4195328 len 1024 2024-04-25T14:15:14.776 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4195328 len 1024 2024-04-25T14:15:14.776 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4195328 len 1024 2024-04-25T14:15:14.776 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4195328 len 2048 2024-04-25T14:15:14.776 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4195328 len 2048 2024-04-25T14:15:14.776 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4195328 len 2048 2024-04-25T14:15:14.777 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4195328 len 2048 2024-04-25T14:15:14.777 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4195328 len 4096 2024-04-25T14:15:14.780 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4195328 len 4096 2024-04-25T14:15:14.781 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4195328 len 4096 2024-04-25T14:15:14.781 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4195328 len 4096 2024-04-25T14:15:14.781 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4195328 len 8192 2024-04-25T14:15:14.781 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4195328 len 8192 2024-04-25T14:15:14.781 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4195328 len 8192 2024-04-25T14:15:14.781 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4195328 len 8192 2024-04-25T14:15:14.781 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4195328 len 16384 2024-04-25T14:15:14.781 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4195328 len 16384 2024-04-25T14:15:14.781 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4195328 len 16384 2024-04-25T14:15:14.781 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4195328 len 16384 2024-04-25T14:15:14.781 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4195840 len 1024 2024-04-25T14:15:14.781 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4195840 len 1024 2024-04-25T14:15:14.781 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4195840 len 1024 2024-04-25T14:15:14.781 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4195840 len 1024 2024-04-25T14:15:14.781 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4195840 len 2048 2024-04-25T14:15:14.782 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4195840 len 2048 2024-04-25T14:15:14.782 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4195840 len 2048 2024-04-25T14:15:14.782 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4195840 len 2048 2024-04-25T14:15:14.782 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4195840 len 4096 2024-04-25T14:15:14.782 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4195840 len 4096 2024-04-25T14:15:14.782 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4195840 len 4096 2024-04-25T14:15:14.782 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4195840 len 4096 2024-04-25T14:15:14.782 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4195840 len 8192 2024-04-25T14:15:14.782 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4195840 len 8192 2024-04-25T14:15:14.782 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4195840 len 8192 2024-04-25T14:15:14.782 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4195840 len 8192 2024-04-25T14:15:14.782 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4195840 len 16384 2024-04-25T14:15:14.782 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4195840 len 16384 2024-04-25T14:15:14.782 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4195840 len 16384 2024-04-25T14:15:14.783 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4195840 len 16384 2024-04-25T14:15:14.783 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4196352 len 1024 2024-04-25T14:15:14.783 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4196352 len 1024 2024-04-25T14:15:14.783 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4196352 len 1024 2024-04-25T14:15:14.783 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4196352 len 1024 2024-04-25T14:15:14.783 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4196352 len 2048 2024-04-25T14:15:14.783 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4196352 len 2048 2024-04-25T14:15:14.783 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4196352 len 2048 2024-04-25T14:15:14.783 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4196352 len 2048 2024-04-25T14:15:14.783 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4196352 len 4096 2024-04-25T14:15:14.783 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4196352 len 4096 2024-04-25T14:15:14.783 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4196352 len 4096 2024-04-25T14:15:14.783 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4196352 len 4096 2024-04-25T14:15:14.783 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4196352 len 8192 2024-04-25T14:15:14.783 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4196352 len 8192 2024-04-25T14:15:14.784 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4196352 len 8192 2024-04-25T14:15:14.784 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4196352 len 8192 2024-04-25T14:15:14.784 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4196352 len 16384 2024-04-25T14:15:14.784 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4196352 len 16384 2024-04-25T14:15:14.784 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4196352 len 16384 2024-04-25T14:15:14.784 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4196352 len 16384 2024-04-25T14:15:14.784 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4196864 len 1024 2024-04-25T14:15:14.784 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4196864 len 1024 2024-04-25T14:15:14.784 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4196864 len 1024 2024-04-25T14:15:14.784 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4196864 len 1024 2024-04-25T14:15:14.784 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4196864 len 2048 2024-04-25T14:15:14.784 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4196864 len 2048 2024-04-25T14:15:14.784 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4196864 len 2048 2024-04-25T14:15:14.784 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4196864 len 2048 2024-04-25T14:15:14.784 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4196864 len 4096 2024-04-25T14:15:14.785 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4196864 len 4096 2024-04-25T14:15:14.785 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4196864 len 4096 2024-04-25T14:15:14.785 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4196864 len 4096 2024-04-25T14:15:14.785 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4196864 len 8192 2024-04-25T14:15:14.785 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4196864 len 8192 2024-04-25T14:15:14.785 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4196864 len 8192 2024-04-25T14:15:14.785 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4196864 len 8192 2024-04-25T14:15:14.785 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4196864 len 16384 2024-04-25T14:15:14.785 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4196864 len 16384 2024-04-25T14:15:14.785 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4196864 len 16384 2024-04-25T14:15:14.785 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4196864 len 16384 2024-04-25T14:15:14.785 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4197376 len 1024 2024-04-25T14:15:14.785 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4197376 len 1024 2024-04-25T14:15:14.785 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4197376 len 1024 2024-04-25T14:15:14.785 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4197376 len 1024 2024-04-25T14:15:14.786 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4197376 len 2048 2024-04-25T14:15:14.786 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4197376 len 2048 2024-04-25T14:15:14.786 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4197376 len 2048 2024-04-25T14:15:14.786 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4197376 len 2048 2024-04-25T14:15:14.786 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4197376 len 4096 2024-04-25T14:15:14.786 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4197376 len 4096 2024-04-25T14:15:14.786 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4197376 len 4096 2024-04-25T14:15:14.786 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4197376 len 4096 2024-04-25T14:15:14.796 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4197376 len 8192 2024-04-25T14:15:14.796 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4197376 len 8192 2024-04-25T14:15:14.796 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4197376 len 8192 2024-04-25T14:15:14.796 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4197376 len 8192 2024-04-25T14:15:14.796 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4197376 len 16384 2024-04-25T14:15:14.797 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4197376 len 16384 2024-04-25T14:15:14.797 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4197376 len 16384 2024-04-25T14:15:14.797 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4197376 len 16384 2024-04-25T14:15:14.797 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4197888 len 1024 2024-04-25T14:15:14.797 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4197888 len 1024 2024-04-25T14:15:14.797 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4197888 len 1024 2024-04-25T14:15:14.797 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4197888 len 1024 2024-04-25T14:15:14.797 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4197888 len 2048 2024-04-25T14:15:14.797 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4197888 len 2048 2024-04-25T14:15:14.797 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4197888 len 2048 2024-04-25T14:15:14.797 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4197888 len 2048 2024-04-25T14:15:14.797 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4197888 len 4096 2024-04-25T14:15:14.797 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4197888 len 4096 2024-04-25T14:15:14.798 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4197888 len 4096 2024-04-25T14:15:14.798 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4197888 len 4096 2024-04-25T14:15:14.798 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4197888 len 8192 2024-04-25T14:15:14.798 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4197888 len 8192 2024-04-25T14:15:14.798 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4197888 len 8192 2024-04-25T14:15:14.798 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4197888 len 8192 2024-04-25T14:15:14.798 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1024 offset 4197888 len 16384 2024-04-25T14:15:14.798 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4197888 len 16384 2024-04-25T14:15:14.798 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1024 offset 4197888 len 16384 2024-04-25T14:15:14.798 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1024 offset 4197888 len 16384 2024-04-25T14:15:14.798 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4190208 len 1024 2024-04-25T14:15:14.798 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4190208 len 1024 2024-04-25T14:15:14.798 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4190208 len 1024 2024-04-25T14:15:14.799 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4190208 len 1024 2024-04-25T14:15:14.799 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4190208 len 2048 2024-04-25T14:15:14.799 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4190208 len 2048 2024-04-25T14:15:14.799 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4190208 len 2048 2024-04-25T14:15:14.799 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4190208 len 2048 2024-04-25T14:15:14.799 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4190208 len 4096 2024-04-25T14:15:14.799 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4190208 len 4096 2024-04-25T14:15:14.799 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4190208 len 4096 2024-04-25T14:15:14.799 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4190208 len 4096 2024-04-25T14:15:14.799 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4190208 len 8192 2024-04-25T14:15:14.799 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4190208 len 8192 2024-04-25T14:15:14.799 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4190208 len 8192 2024-04-25T14:15:14.799 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4190208 len 8192 2024-04-25T14:15:14.799 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4190208 len 16384 2024-04-25T14:15:14.799 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4190208 len 16384 2024-04-25T14:15:14.799 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4190208 len 16384 2024-04-25T14:15:14.800 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4190208 len 16384 2024-04-25T14:15:14.800 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4190720 len 1024 2024-04-25T14:15:14.800 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4190720 len 1024 2024-04-25T14:15:14.800 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4190720 len 1024 2024-04-25T14:15:14.800 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4190720 len 1024 2024-04-25T14:15:14.800 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4190720 len 2048 2024-04-25T14:15:14.800 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4190720 len 2048 2024-04-25T14:15:14.800 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4190720 len 2048 2024-04-25T14:15:14.800 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4190720 len 2048 2024-04-25T14:15:14.800 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4190720 len 4096 2024-04-25T14:15:14.800 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4190720 len 4096 2024-04-25T14:15:14.800 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4190720 len 4096 2024-04-25T14:15:14.800 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4190720 len 4096 2024-04-25T14:15:14.800 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4190720 len 8192 2024-04-25T14:15:14.800 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4190720 len 8192 2024-04-25T14:15:14.801 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4190720 len 8192 2024-04-25T14:15:14.801 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4190720 len 8192 2024-04-25T14:15:14.801 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4190720 len 16384 2024-04-25T14:15:14.801 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4190720 len 16384 2024-04-25T14:15:14.801 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4190720 len 16384 2024-04-25T14:15:14.801 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4190720 len 16384 2024-04-25T14:15:14.801 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4191232 len 1024 2024-04-25T14:15:14.801 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4191232 len 1024 2024-04-25T14:15:14.801 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4191232 len 1024 2024-04-25T14:15:14.801 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4191232 len 1024 2024-04-25T14:15:14.801 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4191232 len 2048 2024-04-25T14:15:14.801 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4191232 len 2048 2024-04-25T14:15:14.801 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4191232 len 2048 2024-04-25T14:15:14.801 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4191232 len 2048 2024-04-25T14:15:14.801 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4191232 len 4096 2024-04-25T14:15:14.802 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4191232 len 4096 2024-04-25T14:15:14.802 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4191232 len 4096 2024-04-25T14:15:14.802 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4191232 len 4096 2024-04-25T14:15:14.802 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4191232 len 8192 2024-04-25T14:15:14.802 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4191232 len 8192 2024-04-25T14:15:14.819 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4191232 len 8192 2024-04-25T14:15:14.820 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4191232 len 8192 2024-04-25T14:15:14.820 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4191232 len 16384 2024-04-25T14:15:14.820 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4191232 len 16384 2024-04-25T14:15:14.820 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4191232 len 16384 2024-04-25T14:15:14.820 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4191232 len 16384 2024-04-25T14:15:14.820 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4191744 len 1024 2024-04-25T14:15:14.820 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4191744 len 1024 2024-04-25T14:15:14.820 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4191744 len 1024 2024-04-25T14:15:14.820 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4191744 len 1024 2024-04-25T14:15:14.820 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4191744 len 2048 2024-04-25T14:15:14.820 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4191744 len 2048 2024-04-25T14:15:14.820 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4191744 len 2048 2024-04-25T14:15:14.820 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4191744 len 2048 2024-04-25T14:15:14.821 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4191744 len 4096 2024-04-25T14:15:14.821 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4191744 len 4096 2024-04-25T14:15:14.821 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4191744 len 4096 2024-04-25T14:15:14.821 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4191744 len 4096 2024-04-25T14:15:14.821 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4191744 len 8192 2024-04-25T14:15:14.821 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4191744 len 8192 2024-04-25T14:15:14.821 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4191744 len 8192 2024-04-25T14:15:14.821 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4191744 len 8192 2024-04-25T14:15:14.821 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4191744 len 16384 2024-04-25T14:15:14.821 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4191744 len 16384 2024-04-25T14:15:14.821 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4191744 len 16384 2024-04-25T14:15:14.821 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4191744 len 16384 2024-04-25T14:15:14.821 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4192256 len 1024 2024-04-25T14:15:14.821 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4192256 len 1024 2024-04-25T14:15:14.821 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4192256 len 1024 2024-04-25T14:15:14.822 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4192256 len 1024 2024-04-25T14:15:14.822 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4192256 len 2048 2024-04-25T14:15:14.822 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4192256 len 2048 2024-04-25T14:15:14.822 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4192256 len 2048 2024-04-25T14:15:14.822 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4192256 len 2048 2024-04-25T14:15:14.822 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4192256 len 4096 2024-04-25T14:15:14.822 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4192256 len 4096 2024-04-25T14:15:14.822 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4192256 len 4096 2024-04-25T14:15:14.822 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4192256 len 4096 2024-04-25T14:15:14.822 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4192256 len 8192 2024-04-25T14:15:14.822 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4192256 len 8192 2024-04-25T14:15:14.822 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4192256 len 8192 2024-04-25T14:15:14.822 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4192256 len 8192 2024-04-25T14:15:14.823 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4192256 len 16384 2024-04-25T14:15:14.823 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4192256 len 16384 2024-04-25T14:15:14.823 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4192256 len 16384 2024-04-25T14:15:14.823 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4192256 len 16384 2024-04-25T14:15:14.823 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4192768 len 1024 2024-04-25T14:15:14.823 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4192768 len 1024 2024-04-25T14:15:14.823 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4192768 len 1024 2024-04-25T14:15:14.823 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4192768 len 1024 2024-04-25T14:15:14.823 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4192768 len 2048 2024-04-25T14:15:14.823 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4192768 len 2048 2024-04-25T14:15:14.823 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4192768 len 2048 2024-04-25T14:15:14.823 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4192768 len 2048 2024-04-25T14:15:14.823 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4192768 len 4096 2024-04-25T14:15:14.823 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4192768 len 4096 2024-04-25T14:15:14.823 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4192768 len 4096 2024-04-25T14:15:14.823 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4192768 len 4096 2024-04-25T14:15:14.824 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4192768 len 8192 2024-04-25T14:15:14.824 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4192768 len 8192 2024-04-25T14:15:14.824 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4192768 len 8192 2024-04-25T14:15:14.824 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4192768 len 8192 2024-04-25T14:15:14.824 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4192768 len 16384 2024-04-25T14:15:14.824 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4192768 len 16384 2024-04-25T14:15:14.824 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4192768 len 16384 2024-04-25T14:15:14.824 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4192768 len 16384 2024-04-25T14:15:14.824 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4193280 len 1024 2024-04-25T14:15:14.824 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4193280 len 1024 2024-04-25T14:15:14.824 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4193280 len 1024 2024-04-25T14:15:14.824 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4193280 len 1024 2024-04-25T14:15:14.824 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4193280 len 2048 2024-04-25T14:15:14.824 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4193280 len 2048 2024-04-25T14:15:14.824 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4193280 len 2048 2024-04-25T14:15:14.825 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4193280 len 2048 2024-04-25T14:15:14.825 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4193280 len 4096 2024-04-25T14:15:14.825 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4193280 len 4096 2024-04-25T14:15:14.825 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4193280 len 4096 2024-04-25T14:15:14.825 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4193280 len 4096 2024-04-25T14:15:14.825 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4193280 len 8192 2024-04-25T14:15:14.825 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4193280 len 8192 2024-04-25T14:15:14.825 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4193280 len 8192 2024-04-25T14:15:14.825 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4193280 len 8192 2024-04-25T14:15:14.833 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4193280 len 16384 2024-04-25T14:15:14.833 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4193280 len 16384 2024-04-25T14:15:14.833 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4193280 len 16384 2024-04-25T14:15:14.833 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4193280 len 16384 2024-04-25T14:15:14.833 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4193792 len 1024 2024-04-25T14:15:14.834 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4193792 len 1024 2024-04-25T14:15:14.834 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4193792 len 1024 2024-04-25T14:15:14.834 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4193792 len 1024 2024-04-25T14:15:14.834 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4193792 len 2048 2024-04-25T14:15:14.834 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4193792 len 2048 2024-04-25T14:15:14.834 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4193792 len 2048 2024-04-25T14:15:14.834 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4193792 len 2048 2024-04-25T14:15:14.834 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4193792 len 4096 2024-04-25T14:15:14.834 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4193792 len 4096 2024-04-25T14:15:14.834 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4193792 len 4096 2024-04-25T14:15:14.834 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4193792 len 4096 2024-04-25T14:15:14.834 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4193792 len 8192 2024-04-25T14:15:14.834 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4193792 len 8192 2024-04-25T14:15:14.834 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4193792 len 8192 2024-04-25T14:15:14.835 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4193792 len 8192 2024-04-25T14:15:14.835 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4193792 len 16384 2024-04-25T14:15:14.835 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4193792 len 16384 2024-04-25T14:15:14.835 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4193792 len 16384 2024-04-25T14:15:14.835 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4193792 len 16384 2024-04-25T14:15:14.835 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4194304 len 1024 2024-04-25T14:15:14.835 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4194304 len 1024 2024-04-25T14:15:14.835 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4194304 len 1024 2024-04-25T14:15:14.835 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4194304 len 1024 2024-04-25T14:15:14.835 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4194304 len 2048 2024-04-25T14:15:14.835 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4194304 len 2048 2024-04-25T14:15:14.835 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4194304 len 2048 2024-04-25T14:15:14.835 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4194304 len 2048 2024-04-25T14:15:14.835 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4194304 len 4096 2024-04-25T14:15:14.835 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4194304 len 4096 2024-04-25T14:15:14.835 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4194304 len 4096 2024-04-25T14:15:14.836 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4194304 len 4096 2024-04-25T14:15:14.836 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4194304 len 8192 2024-04-25T14:15:14.836 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4194304 len 8192 2024-04-25T14:15:14.836 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4194304 len 8192 2024-04-25T14:15:14.836 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4194304 len 8192 2024-04-25T14:15:14.836 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4194304 len 16384 2024-04-25T14:15:14.836 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4194304 len 16384 2024-04-25T14:15:14.836 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4194304 len 16384 2024-04-25T14:15:14.836 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4194304 len 16384 2024-04-25T14:15:14.836 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4194816 len 1024 2024-04-25T14:15:14.836 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4194816 len 1024 2024-04-25T14:15:14.836 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4194816 len 1024 2024-04-25T14:15:14.836 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4194816 len 1024 2024-04-25T14:15:14.836 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4194816 len 2048 2024-04-25T14:15:14.836 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4194816 len 2048 2024-04-25T14:15:14.836 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4194816 len 2048 2024-04-25T14:15:14.836 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4194816 len 2048 2024-04-25T14:15:14.837 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4194816 len 4096 2024-04-25T14:15:14.837 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4194816 len 4096 2024-04-25T14:15:14.837 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4194816 len 4096 2024-04-25T14:15:14.837 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4194816 len 4096 2024-04-25T14:15:14.837 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4194816 len 8192 2024-04-25T14:15:14.837 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4194816 len 8192 2024-04-25T14:15:14.837 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4194816 len 8192 2024-04-25T14:15:14.837 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4194816 len 8192 2024-04-25T14:15:14.837 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4194816 len 16384 2024-04-25T14:15:14.837 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4194816 len 16384 2024-04-25T14:15:14.837 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4194816 len 16384 2024-04-25T14:15:14.837 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4194816 len 16384 2024-04-25T14:15:14.837 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4195328 len 1024 2024-04-25T14:15:14.837 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4195328 len 1024 2024-04-25T14:15:14.837 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4195328 len 1024 2024-04-25T14:15:14.837 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4195328 len 1024 2024-04-25T14:15:14.837 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4195328 len 2048 2024-04-25T14:15:14.838 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4195328 len 2048 2024-04-25T14:15:14.838 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4195328 len 2048 2024-04-25T14:15:14.838 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4195328 len 2048 2024-04-25T14:15:14.838 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4195328 len 4096 2024-04-25T14:15:14.838 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4195328 len 4096 2024-04-25T14:15:14.838 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4195328 len 4096 2024-04-25T14:15:14.838 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4195328 len 4096 2024-04-25T14:15:14.838 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4195328 len 8192 2024-04-25T14:15:14.838 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4195328 len 8192 2024-04-25T14:15:14.838 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4195328 len 8192 2024-04-25T14:15:14.838 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4195328 len 8192 2024-04-25T14:15:14.838 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4195328 len 16384 2024-04-25T14:15:14.838 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4195328 len 16384 2024-04-25T14:15:14.838 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4195328 len 16384 2024-04-25T14:15:14.842 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4195328 len 16384 2024-04-25T14:15:14.842 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4195840 len 1024 2024-04-25T14:15:14.842 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4195840 len 1024 2024-04-25T14:15:14.843 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4195840 len 1024 2024-04-25T14:15:14.843 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4195840 len 1024 2024-04-25T14:15:14.843 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4195840 len 2048 2024-04-25T14:15:14.843 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4195840 len 2048 2024-04-25T14:15:14.843 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4195840 len 2048 2024-04-25T14:15:14.843 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4195840 len 2048 2024-04-25T14:15:14.843 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4195840 len 4096 2024-04-25T14:15:14.843 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4195840 len 4096 2024-04-25T14:15:14.843 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4195840 len 4096 2024-04-25T14:15:14.843 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4195840 len 4096 2024-04-25T14:15:14.843 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4195840 len 8192 2024-04-25T14:15:14.843 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4195840 len 8192 2024-04-25T14:15:14.843 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4195840 len 8192 2024-04-25T14:15:14.843 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4195840 len 8192 2024-04-25T14:15:14.843 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4195840 len 16384 2024-04-25T14:15:14.843 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4195840 len 16384 2024-04-25T14:15:14.844 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4195840 len 16384 2024-04-25T14:15:14.844 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4195840 len 16384 2024-04-25T14:15:14.844 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4196352 len 1024 2024-04-25T14:15:14.844 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4196352 len 1024 2024-04-25T14:15:14.844 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4196352 len 1024 2024-04-25T14:15:14.844 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4196352 len 1024 2024-04-25T14:15:14.844 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4196352 len 2048 2024-04-25T14:15:14.844 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4196352 len 2048 2024-04-25T14:15:14.844 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4196352 len 2048 2024-04-25T14:15:14.844 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4196352 len 2048 2024-04-25T14:15:14.844 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4196352 len 4096 2024-04-25T14:15:14.844 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4196352 len 4096 2024-04-25T14:15:14.844 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4196352 len 4096 2024-04-25T14:15:14.844 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4196352 len 4096 2024-04-25T14:15:14.844 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4196352 len 8192 2024-04-25T14:15:14.844 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4196352 len 8192 2024-04-25T14:15:14.845 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4196352 len 8192 2024-04-25T14:15:14.845 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4196352 len 8192 2024-04-25T14:15:14.845 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4196352 len 16384 2024-04-25T14:15:14.845 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4196352 len 16384 2024-04-25T14:15:14.845 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4196352 len 16384 2024-04-25T14:15:14.845 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4196352 len 16384 2024-04-25T14:15:14.845 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4196864 len 1024 2024-04-25T14:15:14.845 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4196864 len 1024 2024-04-25T14:15:14.845 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4196864 len 1024 2024-04-25T14:15:14.845 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4196864 len 1024 2024-04-25T14:15:14.845 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4196864 len 2048 2024-04-25T14:15:14.845 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4196864 len 2048 2024-04-25T14:15:14.845 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4196864 len 2048 2024-04-25T14:15:14.845 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4196864 len 2048 2024-04-25T14:15:14.845 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4196864 len 4096 2024-04-25T14:15:14.845 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4196864 len 4096 2024-04-25T14:15:14.846 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4196864 len 4096 2024-04-25T14:15:14.846 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4196864 len 4096 2024-04-25T14:15:14.846 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4196864 len 8192 2024-04-25T14:15:14.846 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4196864 len 8192 2024-04-25T14:15:14.846 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4196864 len 8192 2024-04-25T14:15:14.846 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4196864 len 8192 2024-04-25T14:15:14.846 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4196864 len 16384 2024-04-25T14:15:14.846 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4196864 len 16384 2024-04-25T14:15:14.846 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4196864 len 16384 2024-04-25T14:15:14.846 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4196864 len 16384 2024-04-25T14:15:14.846 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4197376 len 1024 2024-04-25T14:15:14.846 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4197376 len 1024 2024-04-25T14:15:14.846 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4197376 len 1024 2024-04-25T14:15:14.846 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4197376 len 1024 2024-04-25T14:15:14.846 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4197376 len 2048 2024-04-25T14:15:14.846 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4197376 len 2048 2024-04-25T14:15:14.847 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4197376 len 2048 2024-04-25T14:15:14.847 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4197376 len 2048 2024-04-25T14:15:14.847 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4197376 len 4096 2024-04-25T14:15:14.847 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4197376 len 4096 2024-04-25T14:15:14.847 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4197376 len 4096 2024-04-25T14:15:14.847 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4197376 len 4096 2024-04-25T14:15:14.847 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4197376 len 8192 2024-04-25T14:15:14.847 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4197376 len 8192 2024-04-25T14:15:14.847 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4197376 len 8192 2024-04-25T14:15:14.847 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4197376 len 8192 2024-04-25T14:15:14.847 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4197376 len 16384 2024-04-25T14:15:14.847 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4197376 len 16384 2024-04-25T14:15:14.847 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4197376 len 16384 2024-04-25T14:15:14.847 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4197376 len 16384 2024-04-25T14:15:14.847 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4197888 len 1024 2024-04-25T14:15:14.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:14 smithi096 ceph-mon[29207]: pgmap v394: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1.4 MiB/s wr, 1.97k op/s 2024-04-25T14:15:14.861 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4197888 len 1024 2024-04-25T14:15:14.861 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4197888 len 1024 2024-04-25T14:15:14.861 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4197888 len 1024 2024-04-25T14:15:14.861 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4197888 len 2048 2024-04-25T14:15:14.861 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4197888 len 2048 2024-04-25T14:15:14.861 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4197888 len 2048 2024-04-25T14:15:14.861 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4197888 len 2048 2024-04-25T14:15:14.861 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4197888 len 4096 2024-04-25T14:15:14.861 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4197888 len 4096 2024-04-25T14:15:14.861 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4197888 len 4096 2024-04-25T14:15:14.861 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4197888 len 4096 2024-04-25T14:15:14.861 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4197888 len 8192 2024-04-25T14:15:14.861 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4197888 len 8192 2024-04-25T14:15:14.862 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4197888 len 8192 2024-04-25T14:15:14.862 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4197888 len 8192 2024-04-25T14:15:14.862 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 1536 offset 4197888 len 16384 2024-04-25T14:15:14.862 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4197888 len 16384 2024-04-25T14:15:14.862 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 1536 offset 4197888 len 16384 2024-04-25T14:15:14.862 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 1536 offset 4197888 len 16384 2024-04-25T14:15:14.862 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4190208 len 1024 2024-04-25T14:15:14.862 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4190208 len 1024 2024-04-25T14:15:14.862 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4190208 len 1024 2024-04-25T14:15:14.862 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4190208 len 1024 2024-04-25T14:15:14.862 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4190208 len 2048 2024-04-25T14:15:14.862 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4190208 len 2048 2024-04-25T14:15:14.862 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4190208 len 2048 2024-04-25T14:15:14.862 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4190208 len 2048 2024-04-25T14:15:14.863 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4190208 len 4096 2024-04-25T14:15:14.863 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4190208 len 4096 2024-04-25T14:15:14.863 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4190208 len 4096 2024-04-25T14:15:14.863 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4190208 len 4096 2024-04-25T14:15:14.863 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4190208 len 8192 2024-04-25T14:15:14.863 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4190208 len 8192 2024-04-25T14:15:14.863 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4190208 len 8192 2024-04-25T14:15:14.863 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4190208 len 8192 2024-04-25T14:15:14.863 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4190208 len 16384 2024-04-25T14:15:14.863 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4190208 len 16384 2024-04-25T14:15:14.863 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4190208 len 16384 2024-04-25T14:15:14.863 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4190208 len 16384 2024-04-25T14:15:14.863 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4190720 len 1024 2024-04-25T14:15:14.863 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4190720 len 1024 2024-04-25T14:15:14.863 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4190720 len 1024 2024-04-25T14:15:14.864 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4190720 len 1024 2024-04-25T14:15:14.864 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4190720 len 2048 2024-04-25T14:15:14.864 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4190720 len 2048 2024-04-25T14:15:14.864 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4190720 len 2048 2024-04-25T14:15:14.864 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4190720 len 2048 2024-04-25T14:15:14.864 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4190720 len 4096 2024-04-25T14:15:14.864 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4190720 len 4096 2024-04-25T14:15:14.864 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4190720 len 4096 2024-04-25T14:15:14.864 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4190720 len 4096 2024-04-25T14:15:14.864 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4190720 len 8192 2024-04-25T14:15:14.864 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4190720 len 8192 2024-04-25T14:15:14.864 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4190720 len 8192 2024-04-25T14:15:14.864 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4190720 len 8192 2024-04-25T14:15:14.864 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4190720 len 16384 2024-04-25T14:15:14.864 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4190720 len 16384 2024-04-25T14:15:14.864 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4190720 len 16384 2024-04-25T14:15:14.865 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4190720 len 16384 2024-04-25T14:15:14.865 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4191232 len 1024 2024-04-25T14:15:14.865 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4191232 len 1024 2024-04-25T14:15:14.865 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4191232 len 1024 2024-04-25T14:15:14.865 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4191232 len 1024 2024-04-25T14:15:14.865 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4191232 len 2048 2024-04-25T14:15:14.865 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4191232 len 2048 2024-04-25T14:15:14.865 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4191232 len 2048 2024-04-25T14:15:14.865 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4191232 len 2048 2024-04-25T14:15:14.865 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4191232 len 4096 2024-04-25T14:15:14.865 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4191232 len 4096 2024-04-25T14:15:14.865 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4191232 len 4096 2024-04-25T14:15:14.865 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4191232 len 4096 2024-04-25T14:15:14.865 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4191232 len 8192 2024-04-25T14:15:14.865 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4191232 len 8192 2024-04-25T14:15:14.866 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4191232 len 8192 2024-04-25T14:15:14.866 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4191232 len 8192 2024-04-25T14:15:14.866 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4191232 len 16384 2024-04-25T14:15:14.866 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4191232 len 16384 2024-04-25T14:15:14.866 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4191232 len 16384 2024-04-25T14:15:14.866 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4191232 len 16384 2024-04-25T14:15:14.866 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4191744 len 1024 2024-04-25T14:15:14.866 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4191744 len 1024 2024-04-25T14:15:14.866 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4191744 len 1024 2024-04-25T14:15:14.866 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4191744 len 1024 2024-04-25T14:15:14.883 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4191744 len 2048 2024-04-25T14:15:14.883 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4191744 len 2048 2024-04-25T14:15:14.883 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4191744 len 2048 2024-04-25T14:15:14.884 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4191744 len 2048 2024-04-25T14:15:14.884 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4191744 len 4096 2024-04-25T14:15:14.884 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4191744 len 4096 2024-04-25T14:15:14.884 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4191744 len 4096 2024-04-25T14:15:14.884 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4191744 len 4096 2024-04-25T14:15:14.884 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4191744 len 8192 2024-04-25T14:15:14.884 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4191744 len 8192 2024-04-25T14:15:14.884 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4191744 len 8192 2024-04-25T14:15:14.884 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4191744 len 8192 2024-04-25T14:15:14.884 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4191744 len 16384 2024-04-25T14:15:14.884 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4191744 len 16384 2024-04-25T14:15:14.884 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4191744 len 16384 2024-04-25T14:15:14.884 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4191744 len 16384 2024-04-25T14:15:14.884 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4192256 len 1024 2024-04-25T14:15:14.885 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4192256 len 1024 2024-04-25T14:15:14.885 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4192256 len 1024 2024-04-25T14:15:14.885 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4192256 len 1024 2024-04-25T14:15:14.885 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4192256 len 2048 2024-04-25T14:15:14.885 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4192256 len 2048 2024-04-25T14:15:14.885 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4192256 len 2048 2024-04-25T14:15:14.885 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4192256 len 2048 2024-04-25T14:15:14.885 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4192256 len 4096 2024-04-25T14:15:14.885 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4192256 len 4096 2024-04-25T14:15:14.885 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4192256 len 4096 2024-04-25T14:15:14.885 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4192256 len 4096 2024-04-25T14:15:14.885 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4192256 len 8192 2024-04-25T14:15:14.885 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4192256 len 8192 2024-04-25T14:15:14.885 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4192256 len 8192 2024-04-25T14:15:14.885 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4192256 len 8192 2024-04-25T14:15:14.885 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4192256 len 16384 2024-04-25T14:15:14.886 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4192256 len 16384 2024-04-25T14:15:14.886 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4192256 len 16384 2024-04-25T14:15:14.886 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4192256 len 16384 2024-04-25T14:15:14.886 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4192768 len 1024 2024-04-25T14:15:14.886 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4192768 len 1024 2024-04-25T14:15:14.886 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4192768 len 1024 2024-04-25T14:15:14.886 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4192768 len 1024 2024-04-25T14:15:14.886 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4192768 len 2048 2024-04-25T14:15:14.886 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4192768 len 2048 2024-04-25T14:15:14.886 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4192768 len 2048 2024-04-25T14:15:14.886 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4192768 len 2048 2024-04-25T14:15:14.886 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4192768 len 4096 2024-04-25T14:15:14.886 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4192768 len 4096 2024-04-25T14:15:14.886 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4192768 len 4096 2024-04-25T14:15:14.886 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4192768 len 4096 2024-04-25T14:15:14.887 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4192768 len 8192 2024-04-25T14:15:14.887 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4192768 len 8192 2024-04-25T14:15:14.887 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4192768 len 8192 2024-04-25T14:15:14.887 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4192768 len 8192 2024-04-25T14:15:14.887 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4192768 len 16384 2024-04-25T14:15:14.887 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4192768 len 16384 2024-04-25T14:15:14.887 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4192768 len 16384 2024-04-25T14:15:14.887 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4192768 len 16384 2024-04-25T14:15:14.887 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4193280 len 1024 2024-04-25T14:15:14.887 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4193280 len 1024 2024-04-25T14:15:14.887 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4193280 len 1024 2024-04-25T14:15:14.887 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4193280 len 1024 2024-04-25T14:15:14.887 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4193280 len 2048 2024-04-25T14:15:14.887 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4193280 len 2048 2024-04-25T14:15:14.887 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4193280 len 2048 2024-04-25T14:15:14.887 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4193280 len 2048 2024-04-25T14:15:14.888 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4193280 len 4096 2024-04-25T14:15:14.888 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4193280 len 4096 2024-04-25T14:15:14.888 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4193280 len 4096 2024-04-25T14:15:14.888 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4193280 len 4096 2024-04-25T14:15:14.888 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4193280 len 8192 2024-04-25T14:15:14.888 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4193280 len 8192 2024-04-25T14:15:14.888 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4193280 len 8192 2024-04-25T14:15:14.888 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4193280 len 8192 2024-04-25T14:15:14.888 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4193280 len 16384 2024-04-25T14:15:14.888 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4193280 len 16384 2024-04-25T14:15:14.888 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4193280 len 16384 2024-04-25T14:15:14.888 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4193280 len 16384 2024-04-25T14:15:14.888 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4193792 len 1024 2024-04-25T14:15:14.888 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4193792 len 1024 2024-04-25T14:15:14.888 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4193792 len 1024 2024-04-25T14:15:14.888 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4193792 len 1024 2024-04-25T14:15:14.889 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4193792 len 2048 2024-04-25T14:15:14.889 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4193792 len 2048 2024-04-25T14:15:14.896 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4193792 len 2048 2024-04-25T14:15:14.896 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4193792 len 2048 2024-04-25T14:15:14.896 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4193792 len 4096 2024-04-25T14:15:14.896 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4193792 len 4096 2024-04-25T14:15:14.897 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4193792 len 4096 2024-04-25T14:15:14.897 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4193792 len 4096 2024-04-25T14:15:14.897 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4193792 len 8192 2024-04-25T14:15:14.897 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4193792 len 8192 2024-04-25T14:15:14.897 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4193792 len 8192 2024-04-25T14:15:14.897 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4193792 len 8192 2024-04-25T14:15:14.897 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4193792 len 16384 2024-04-25T14:15:14.897 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4193792 len 16384 2024-04-25T14:15:14.897 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4193792 len 16384 2024-04-25T14:15:14.897 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4193792 len 16384 2024-04-25T14:15:14.897 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4194304 len 1024 2024-04-25T14:15:14.897 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4194304 len 1024 2024-04-25T14:15:14.897 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4194304 len 1024 2024-04-25T14:15:14.897 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4194304 len 1024 2024-04-25T14:15:14.898 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4194304 len 2048 2024-04-25T14:15:14.898 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4194304 len 2048 2024-04-25T14:15:14.898 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4194304 len 2048 2024-04-25T14:15:14.898 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4194304 len 2048 2024-04-25T14:15:14.898 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4194304 len 4096 2024-04-25T14:15:14.898 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4194304 len 4096 2024-04-25T14:15:14.898 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4194304 len 4096 2024-04-25T14:15:14.898 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4194304 len 4096 2024-04-25T14:15:14.898 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4194304 len 8192 2024-04-25T14:15:14.898 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4194304 len 8192 2024-04-25T14:15:14.898 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4194304 len 8192 2024-04-25T14:15:14.898 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4194304 len 8192 2024-04-25T14:15:14.898 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4194304 len 16384 2024-04-25T14:15:14.899 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4194304 len 16384 2024-04-25T14:15:14.899 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4194304 len 16384 2024-04-25T14:15:14.899 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4194304 len 16384 2024-04-25T14:15:14.899 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4194816 len 1024 2024-04-25T14:15:14.899 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4194816 len 1024 2024-04-25T14:15:14.899 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4194816 len 1024 2024-04-25T14:15:14.899 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4194816 len 1024 2024-04-25T14:15:14.899 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4194816 len 2048 2024-04-25T14:15:14.899 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4194816 len 2048 2024-04-25T14:15:14.899 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4194816 len 2048 2024-04-25T14:15:14.899 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4194816 len 2048 2024-04-25T14:15:14.899 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4194816 len 4096 2024-04-25T14:15:14.899 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4194816 len 4096 2024-04-25T14:15:14.899 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4194816 len 4096 2024-04-25T14:15:14.900 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4194816 len 4096 2024-04-25T14:15:14.900 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4194816 len 8192 2024-04-25T14:15:14.900 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4194816 len 8192 2024-04-25T14:15:14.900 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4194816 len 8192 2024-04-25T14:15:14.900 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4194816 len 8192 2024-04-25T14:15:14.900 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4194816 len 16384 2024-04-25T14:15:14.900 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4194816 len 16384 2024-04-25T14:15:14.900 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4194816 len 16384 2024-04-25T14:15:14.900 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4194816 len 16384 2024-04-25T14:15:14.900 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4195328 len 1024 2024-04-25T14:15:14.900 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4195328 len 1024 2024-04-25T14:15:14.900 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4195328 len 1024 2024-04-25T14:15:14.900 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4195328 len 1024 2024-04-25T14:15:14.900 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4195328 len 2048 2024-04-25T14:15:14.900 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4195328 len 2048 2024-04-25T14:15:14.901 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4195328 len 2048 2024-04-25T14:15:14.901 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4195328 len 2048 2024-04-25T14:15:14.901 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4195328 len 4096 2024-04-25T14:15:14.901 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4195328 len 4096 2024-04-25T14:15:14.901 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4195328 len 4096 2024-04-25T14:15:14.901 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4195328 len 4096 2024-04-25T14:15:14.901 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4195328 len 8192 2024-04-25T14:15:14.901 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4195328 len 8192 2024-04-25T14:15:14.901 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4195328 len 8192 2024-04-25T14:15:14.901 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4195328 len 8192 2024-04-25T14:15:14.901 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4195328 len 16384 2024-04-25T14:15:14.901 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4195328 len 16384 2024-04-25T14:15:14.901 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4195328 len 16384 2024-04-25T14:15:14.902 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4195328 len 16384 2024-04-25T14:15:14.902 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4195840 len 1024 2024-04-25T14:15:14.902 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4195840 len 1024 2024-04-25T14:15:14.902 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4195840 len 1024 2024-04-25T14:15:14.902 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4195840 len 1024 2024-04-25T14:15:14.902 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4195840 len 2048 2024-04-25T14:15:14.902 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4195840 len 2048 2024-04-25T14:15:14.902 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4195840 len 2048 2024-04-25T14:15:14.902 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4195840 len 2048 2024-04-25T14:15:14.905 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4195840 len 4096 2024-04-25T14:15:14.906 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4195840 len 4096 2024-04-25T14:15:14.906 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4195840 len 4096 2024-04-25T14:15:14.906 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4195840 len 4096 2024-04-25T14:15:14.906 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4195840 len 8192 2024-04-25T14:15:14.906 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4195840 len 8192 2024-04-25T14:15:14.906 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4195840 len 8192 2024-04-25T14:15:14.906 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4195840 len 8192 2024-04-25T14:15:14.906 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4195840 len 16384 2024-04-25T14:15:14.906 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4195840 len 16384 2024-04-25T14:15:14.906 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4195840 len 16384 2024-04-25T14:15:14.906 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4195840 len 16384 2024-04-25T14:15:14.906 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4196352 len 1024 2024-04-25T14:15:14.906 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4196352 len 1024 2024-04-25T14:15:14.907 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4196352 len 1024 2024-04-25T14:15:14.907 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4196352 len 1024 2024-04-25T14:15:14.907 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4196352 len 2048 2024-04-25T14:15:14.907 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4196352 len 2048 2024-04-25T14:15:14.907 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4196352 len 2048 2024-04-25T14:15:14.907 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4196352 len 2048 2024-04-25T14:15:14.907 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4196352 len 4096 2024-04-25T14:15:14.907 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4196352 len 4096 2024-04-25T14:15:14.907 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4196352 len 4096 2024-04-25T14:15:14.907 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4196352 len 4096 2024-04-25T14:15:14.907 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4196352 len 8192 2024-04-25T14:15:14.907 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4196352 len 8192 2024-04-25T14:15:14.907 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4196352 len 8192 2024-04-25T14:15:14.907 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4196352 len 8192 2024-04-25T14:15:14.907 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4196352 len 16384 2024-04-25T14:15:14.908 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4196352 len 16384 2024-04-25T14:15:14.908 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4196352 len 16384 2024-04-25T14:15:14.908 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4196352 len 16384 2024-04-25T14:15:14.908 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4196864 len 1024 2024-04-25T14:15:14.908 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4196864 len 1024 2024-04-25T14:15:14.908 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4196864 len 1024 2024-04-25T14:15:14.908 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4196864 len 1024 2024-04-25T14:15:14.908 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4196864 len 2048 2024-04-25T14:15:14.908 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4196864 len 2048 2024-04-25T14:15:14.908 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4196864 len 2048 2024-04-25T14:15:14.908 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4196864 len 2048 2024-04-25T14:15:14.908 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4196864 len 4096 2024-04-25T14:15:14.908 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4196864 len 4096 2024-04-25T14:15:14.909 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4196864 len 4096 2024-04-25T14:15:14.909 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4196864 len 4096 2024-04-25T14:15:14.910 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4196864 len 8192 2024-04-25T14:15:14.910 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4196864 len 8192 2024-04-25T14:15:14.910 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4196864 len 8192 2024-04-25T14:15:14.910 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4196864 len 8192 2024-04-25T14:15:14.910 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4196864 len 16384 2024-04-25T14:15:14.910 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4196864 len 16384 2024-04-25T14:15:14.910 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4196864 len 16384 2024-04-25T14:15:14.910 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4196864 len 16384 2024-04-25T14:15:14.910 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4197376 len 1024 2024-04-25T14:15:14.910 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4197376 len 1024 2024-04-25T14:15:14.910 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4197376 len 1024 2024-04-25T14:15:14.910 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4197376 len 1024 2024-04-25T14:15:14.910 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4197376 len 2048 2024-04-25T14:15:14.911 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4197376 len 2048 2024-04-25T14:15:14.911 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4197376 len 2048 2024-04-25T14:15:14.911 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4197376 len 2048 2024-04-25T14:15:14.911 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4197376 len 4096 2024-04-25T14:15:14.911 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4197376 len 4096 2024-04-25T14:15:14.911 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4197376 len 4096 2024-04-25T14:15:14.911 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4197376 len 4096 2024-04-25T14:15:14.911 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4197376 len 8192 2024-04-25T14:15:14.911 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4197376 len 8192 2024-04-25T14:15:14.911 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4197376 len 8192 2024-04-25T14:15:14.911 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4197376 len 8192 2024-04-25T14:15:14.911 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4197376 len 16384 2024-04-25T14:15:14.911 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4197376 len 16384 2024-04-25T14:15:14.911 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4197376 len 16384 2024-04-25T14:15:14.911 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4197376 len 16384 2024-04-25T14:15:14.911 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4197888 len 1024 2024-04-25T14:15:14.912 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4197888 len 1024 2024-04-25T14:15:14.912 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4197888 len 1024 2024-04-25T14:15:14.912 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4197888 len 1024 2024-04-25T14:15:14.912 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4197888 len 2048 2024-04-25T14:15:14.912 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4197888 len 2048 2024-04-25T14:15:14.912 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4197888 len 2048 2024-04-25T14:15:14.912 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4197888 len 2048 2024-04-25T14:15:14.912 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4197888 len 4096 2024-04-25T14:15:14.912 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4197888 len 4096 2024-04-25T14:15:14.912 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4197888 len 4096 2024-04-25T14:15:14.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:14 smithi097 ceph-mon[32720]: pgmap v394: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1.4 MiB/s wr, 1.97k op/s 2024-04-25T14:15:14.923 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4197888 len 4096 2024-04-25T14:15:14.923 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4197888 len 8192 2024-04-25T14:15:14.923 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4197888 len 8192 2024-04-25T14:15:14.923 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4197888 len 8192 2024-04-25T14:15:14.923 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4197888 len 8192 2024-04-25T14:15:14.923 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2048 offset 4197888 len 16384 2024-04-25T14:15:14.923 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4197888 len 16384 2024-04-25T14:15:14.923 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2048 offset 4197888 len 16384 2024-04-25T14:15:14.923 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2048 offset 4197888 len 16384 2024-04-25T14:15:14.923 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4190208 len 1024 2024-04-25T14:15:14.923 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4190208 len 1024 2024-04-25T14:15:14.924 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4190208 len 1024 2024-04-25T14:15:14.924 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4190208 len 1024 2024-04-25T14:15:14.924 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4190208 len 2048 2024-04-25T14:15:14.924 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4190208 len 2048 2024-04-25T14:15:14.924 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4190208 len 2048 2024-04-25T14:15:14.924 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4190208 len 2048 2024-04-25T14:15:14.924 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4190208 len 4096 2024-04-25T14:15:14.924 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4190208 len 4096 2024-04-25T14:15:14.924 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4190208 len 4096 2024-04-25T14:15:14.924 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4190208 len 4096 2024-04-25T14:15:14.924 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4190208 len 8192 2024-04-25T14:15:14.924 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4190208 len 8192 2024-04-25T14:15:14.924 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4190208 len 8192 2024-04-25T14:15:14.924 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4190208 len 8192 2024-04-25T14:15:14.924 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4190208 len 16384 2024-04-25T14:15:14.925 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4190208 len 16384 2024-04-25T14:15:14.925 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4190208 len 16384 2024-04-25T14:15:14.925 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4190208 len 16384 2024-04-25T14:15:14.925 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4190720 len 1024 2024-04-25T14:15:14.925 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4190720 len 1024 2024-04-25T14:15:14.925 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4190720 len 1024 2024-04-25T14:15:14.925 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4190720 len 1024 2024-04-25T14:15:14.925 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4190720 len 2048 2024-04-25T14:15:14.925 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4190720 len 2048 2024-04-25T14:15:14.925 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4190720 len 2048 2024-04-25T14:15:14.925 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4190720 len 2048 2024-04-25T14:15:14.925 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4190720 len 4096 2024-04-25T14:15:14.925 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4190720 len 4096 2024-04-25T14:15:14.925 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4190720 len 4096 2024-04-25T14:15:14.925 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4190720 len 4096 2024-04-25T14:15:14.926 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4190720 len 8192 2024-04-25T14:15:14.926 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4190720 len 8192 2024-04-25T14:15:14.926 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4190720 len 8192 2024-04-25T14:15:14.926 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4190720 len 8192 2024-04-25T14:15:14.926 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4190720 len 16384 2024-04-25T14:15:14.926 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4190720 len 16384 2024-04-25T14:15:14.926 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4190720 len 16384 2024-04-25T14:15:14.926 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4190720 len 16384 2024-04-25T14:15:14.926 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4191232 len 1024 2024-04-25T14:15:14.926 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4191232 len 1024 2024-04-25T14:15:14.926 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4191232 len 1024 2024-04-25T14:15:14.926 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4191232 len 1024 2024-04-25T14:15:14.926 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4191232 len 2048 2024-04-25T14:15:14.927 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4191232 len 2048 2024-04-25T14:15:14.927 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4191232 len 2048 2024-04-25T14:15:14.927 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4191232 len 2048 2024-04-25T14:15:14.927 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4191232 len 4096 2024-04-25T14:15:14.927 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4191232 len 4096 2024-04-25T14:15:14.927 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4191232 len 4096 2024-04-25T14:15:14.927 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4191232 len 4096 2024-04-25T14:15:14.927 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4191232 len 8192 2024-04-25T14:15:14.927 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4191232 len 8192 2024-04-25T14:15:14.927 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4191232 len 8192 2024-04-25T14:15:14.927 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4191232 len 8192 2024-04-25T14:15:14.927 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4191232 len 16384 2024-04-25T14:15:14.927 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4191232 len 16384 2024-04-25T14:15:14.927 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4191232 len 16384 2024-04-25T14:15:14.927 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4191232 len 16384 2024-04-25T14:15:14.928 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4191744 len 1024 2024-04-25T14:15:14.928 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4191744 len 1024 2024-04-25T14:15:14.928 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4191744 len 1024 2024-04-25T14:15:14.928 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4191744 len 1024 2024-04-25T14:15:14.928 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4191744 len 2048 2024-04-25T14:15:14.928 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4191744 len 2048 2024-04-25T14:15:14.928 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4191744 len 2048 2024-04-25T14:15:14.928 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4191744 len 2048 2024-04-25T14:15:14.928 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4191744 len 4096 2024-04-25T14:15:14.928 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4191744 len 4096 2024-04-25T14:15:14.928 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4191744 len 4096 2024-04-25T14:15:14.928 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4191744 len 4096 2024-04-25T14:15:14.928 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4191744 len 8192 2024-04-25T14:15:14.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:14 smithi151 ceph-mon[32990]: pgmap v394: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1.4 MiB/s wr, 1.97k op/s 2024-04-25T14:15:14.945 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4191744 len 8192 2024-04-25T14:15:14.945 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4191744 len 8192 2024-04-25T14:15:14.946 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4191744 len 8192 2024-04-25T14:15:14.946 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4191744 len 16384 2024-04-25T14:15:14.946 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4191744 len 16384 2024-04-25T14:15:14.946 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4191744 len 16384 2024-04-25T14:15:14.946 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4191744 len 16384 2024-04-25T14:15:14.946 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4192256 len 1024 2024-04-25T14:15:14.946 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4192256 len 1024 2024-04-25T14:15:14.946 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4192256 len 1024 2024-04-25T14:15:14.946 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4192256 len 1024 2024-04-25T14:15:14.946 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4192256 len 2048 2024-04-25T14:15:14.946 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4192256 len 2048 2024-04-25T14:15:14.946 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4192256 len 2048 2024-04-25T14:15:14.946 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4192256 len 2048 2024-04-25T14:15:14.946 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4192256 len 4096 2024-04-25T14:15:14.946 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4192256 len 4096 2024-04-25T14:15:14.947 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4192256 len 4096 2024-04-25T14:15:14.947 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4192256 len 4096 2024-04-25T14:15:14.947 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4192256 len 8192 2024-04-25T14:15:14.947 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4192256 len 8192 2024-04-25T14:15:14.947 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4192256 len 8192 2024-04-25T14:15:14.947 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4192256 len 8192 2024-04-25T14:15:14.947 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4192256 len 16384 2024-04-25T14:15:14.947 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4192256 len 16384 2024-04-25T14:15:14.947 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4192256 len 16384 2024-04-25T14:15:14.947 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4192256 len 16384 2024-04-25T14:15:14.947 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4192768 len 1024 2024-04-25T14:15:14.947 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4192768 len 1024 2024-04-25T14:15:14.947 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4192768 len 1024 2024-04-25T14:15:14.947 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4192768 len 1024 2024-04-25T14:15:14.948 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4192768 len 2048 2024-04-25T14:15:14.948 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4192768 len 2048 2024-04-25T14:15:14.948 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4192768 len 2048 2024-04-25T14:15:14.948 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4192768 len 2048 2024-04-25T14:15:14.948 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4192768 len 4096 2024-04-25T14:15:14.948 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4192768 len 4096 2024-04-25T14:15:14.948 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4192768 len 4096 2024-04-25T14:15:14.948 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4192768 len 4096 2024-04-25T14:15:14.948 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4192768 len 8192 2024-04-25T14:15:14.948 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4192768 len 8192 2024-04-25T14:15:14.948 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4192768 len 8192 2024-04-25T14:15:14.948 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4192768 len 8192 2024-04-25T14:15:14.948 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4192768 len 16384 2024-04-25T14:15:14.948 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4192768 len 16384 2024-04-25T14:15:14.948 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4192768 len 16384 2024-04-25T14:15:14.949 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4192768 len 16384 2024-04-25T14:15:14.949 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4193280 len 1024 2024-04-25T14:15:14.949 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4193280 len 1024 2024-04-25T14:15:14.949 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4193280 len 1024 2024-04-25T14:15:14.949 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4193280 len 1024 2024-04-25T14:15:14.949 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4193280 len 2048 2024-04-25T14:15:14.949 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4193280 len 2048 2024-04-25T14:15:14.949 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4193280 len 2048 2024-04-25T14:15:14.949 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4193280 len 2048 2024-04-25T14:15:14.949 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4193280 len 4096 2024-04-25T14:15:14.949 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4193280 len 4096 2024-04-25T14:15:14.949 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4193280 len 4096 2024-04-25T14:15:14.949 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4193280 len 4096 2024-04-25T14:15:14.950 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4193280 len 8192 2024-04-25T14:15:14.950 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4193280 len 8192 2024-04-25T14:15:14.950 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4193280 len 8192 2024-04-25T14:15:14.950 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4193280 len 8192 2024-04-25T14:15:14.950 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4193280 len 16384 2024-04-25T14:15:14.950 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4193280 len 16384 2024-04-25T14:15:14.950 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4193280 len 16384 2024-04-25T14:15:14.950 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4193280 len 16384 2024-04-25T14:15:14.950 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4193792 len 1024 2024-04-25T14:15:14.950 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4193792 len 1024 2024-04-25T14:15:14.950 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4193792 len 1024 2024-04-25T14:15:14.950 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4193792 len 1024 2024-04-25T14:15:14.950 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4193792 len 2048 2024-04-25T14:15:14.950 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4193792 len 2048 2024-04-25T14:15:14.950 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4193792 len 2048 2024-04-25T14:15:14.951 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4193792 len 2048 2024-04-25T14:15:14.951 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4193792 len 4096 2024-04-25T14:15:14.951 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4193792 len 4096 2024-04-25T14:15:14.951 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4193792 len 4096 2024-04-25T14:15:14.951 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4193792 len 4096 2024-04-25T14:15:14.951 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4193792 len 8192 2024-04-25T14:15:14.951 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4193792 len 8192 2024-04-25T14:15:14.951 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4193792 len 8192 2024-04-25T14:15:14.951 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4193792 len 8192 2024-04-25T14:15:14.956 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4193792 len 16384 2024-04-25T14:15:14.956 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4193792 len 16384 2024-04-25T14:15:14.956 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4193792 len 16384 2024-04-25T14:15:14.956 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4193792 len 16384 2024-04-25T14:15:14.956 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4194304 len 1024 2024-04-25T14:15:14.956 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4194304 len 1024 2024-04-25T14:15:14.956 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4194304 len 1024 2024-04-25T14:15:14.956 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4194304 len 1024 2024-04-25T14:15:14.957 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4194304 len 2048 2024-04-25T14:15:14.957 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4194304 len 2048 2024-04-25T14:15:14.957 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4194304 len 2048 2024-04-25T14:15:14.957 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4194304 len 2048 2024-04-25T14:15:14.957 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4194304 len 4096 2024-04-25T14:15:14.957 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4194304 len 4096 2024-04-25T14:15:14.957 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4194304 len 4096 2024-04-25T14:15:14.957 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4194304 len 4096 2024-04-25T14:15:14.957 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4194304 len 8192 2024-04-25T14:15:14.957 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4194304 len 8192 2024-04-25T14:15:14.957 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4194304 len 8192 2024-04-25T14:15:14.957 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4194304 len 8192 2024-04-25T14:15:14.957 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4194304 len 16384 2024-04-25T14:15:14.957 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4194304 len 16384 2024-04-25T14:15:14.957 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4194304 len 16384 2024-04-25T14:15:14.957 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4194304 len 16384 2024-04-25T14:15:14.958 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4194816 len 1024 2024-04-25T14:15:14.958 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4194816 len 1024 2024-04-25T14:15:14.958 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4194816 len 1024 2024-04-25T14:15:14.958 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4194816 len 1024 2024-04-25T14:15:14.958 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4194816 len 2048 2024-04-25T14:15:14.958 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4194816 len 2048 2024-04-25T14:15:14.958 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4194816 len 2048 2024-04-25T14:15:14.958 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4194816 len 2048 2024-04-25T14:15:14.958 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4194816 len 4096 2024-04-25T14:15:14.958 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4194816 len 4096 2024-04-25T14:15:14.958 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4194816 len 4096 2024-04-25T14:15:14.958 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4194816 len 4096 2024-04-25T14:15:14.958 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4194816 len 8192 2024-04-25T14:15:14.958 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4194816 len 8192 2024-04-25T14:15:14.958 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4194816 len 8192 2024-04-25T14:15:14.959 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4194816 len 8192 2024-04-25T14:15:14.959 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4194816 len 16384 2024-04-25T14:15:14.959 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4194816 len 16384 2024-04-25T14:15:14.959 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4194816 len 16384 2024-04-25T14:15:14.959 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4194816 len 16384 2024-04-25T14:15:14.959 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4195328 len 1024 2024-04-25T14:15:14.959 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4195328 len 1024 2024-04-25T14:15:14.959 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4195328 len 1024 2024-04-25T14:15:14.959 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4195328 len 1024 2024-04-25T14:15:14.959 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4195328 len 2048 2024-04-25T14:15:14.959 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4195328 len 2048 2024-04-25T14:15:14.959 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4195328 len 2048 2024-04-25T14:15:14.959 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4195328 len 2048 2024-04-25T14:15:14.959 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4195328 len 4096 2024-04-25T14:15:14.959 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4195328 len 4096 2024-04-25T14:15:14.960 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4195328 len 4096 2024-04-25T14:15:14.960 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4195328 len 4096 2024-04-25T14:15:14.960 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4195328 len 8192 2024-04-25T14:15:14.960 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4195328 len 8192 2024-04-25T14:15:14.960 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4195328 len 8192 2024-04-25T14:15:14.960 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4195328 len 8192 2024-04-25T14:15:14.960 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4195328 len 16384 2024-04-25T14:15:14.960 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4195328 len 16384 2024-04-25T14:15:14.960 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4195328 len 16384 2024-04-25T14:15:14.960 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4195328 len 16384 2024-04-25T14:15:14.960 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4195840 len 1024 2024-04-25T14:15:14.960 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4195840 len 1024 2024-04-25T14:15:14.960 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4195840 len 1024 2024-04-25T14:15:14.960 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4195840 len 1024 2024-04-25T14:15:14.960 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4195840 len 2048 2024-04-25T14:15:14.960 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4195840 len 2048 2024-04-25T14:15:14.961 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4195840 len 2048 2024-04-25T14:15:14.961 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4195840 len 2048 2024-04-25T14:15:14.961 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4195840 len 4096 2024-04-25T14:15:14.961 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4195840 len 4096 2024-04-25T14:15:14.961 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4195840 len 4096 2024-04-25T14:15:14.961 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4195840 len 4096 2024-04-25T14:15:14.961 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4195840 len 8192 2024-04-25T14:15:14.961 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4195840 len 8192 2024-04-25T14:15:14.961 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4195840 len 8192 2024-04-25T14:15:14.961 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4195840 len 8192 2024-04-25T14:15:14.961 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4195840 len 16384 2024-04-25T14:15:14.961 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4195840 len 16384 2024-04-25T14:15:14.967 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4195840 len 16384 2024-04-25T14:15:14.967 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4195840 len 16384 2024-04-25T14:15:14.967 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4196352 len 1024 2024-04-25T14:15:14.967 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4196352 len 1024 2024-04-25T14:15:14.967 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4196352 len 1024 2024-04-25T14:15:14.967 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4196352 len 1024 2024-04-25T14:15:14.968 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4196352 len 2048 2024-04-25T14:15:14.968 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4196352 len 2048 2024-04-25T14:15:14.968 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4196352 len 2048 2024-04-25T14:15:14.968 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4196352 len 2048 2024-04-25T14:15:14.968 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4196352 len 4096 2024-04-25T14:15:14.968 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4196352 len 4096 2024-04-25T14:15:14.968 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4196352 len 4096 2024-04-25T14:15:14.968 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4196352 len 4096 2024-04-25T14:15:14.968 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4196352 len 8192 2024-04-25T14:15:14.968 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4196352 len 8192 2024-04-25T14:15:14.968 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4196352 len 8192 2024-04-25T14:15:14.968 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4196352 len 8192 2024-04-25T14:15:14.968 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4196352 len 16384 2024-04-25T14:15:14.968 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4196352 len 16384 2024-04-25T14:15:14.968 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4196352 len 16384 2024-04-25T14:15:14.968 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4196352 len 16384 2024-04-25T14:15:14.969 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4196864 len 1024 2024-04-25T14:15:14.969 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4196864 len 1024 2024-04-25T14:15:14.969 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4196864 len 1024 2024-04-25T14:15:14.969 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4196864 len 1024 2024-04-25T14:15:14.969 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4196864 len 2048 2024-04-25T14:15:14.969 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4196864 len 2048 2024-04-25T14:15:14.969 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4196864 len 2048 2024-04-25T14:15:14.969 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4196864 len 2048 2024-04-25T14:15:14.969 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4196864 len 4096 2024-04-25T14:15:14.969 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4196864 len 4096 2024-04-25T14:15:14.969 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4196864 len 4096 2024-04-25T14:15:14.969 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4196864 len 4096 2024-04-25T14:15:14.969 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4196864 len 8192 2024-04-25T14:15:14.969 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4196864 len 8192 2024-04-25T14:15:14.969 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4196864 len 8192 2024-04-25T14:15:14.969 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4196864 len 8192 2024-04-25T14:15:14.970 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4196864 len 16384 2024-04-25T14:15:14.970 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4196864 len 16384 2024-04-25T14:15:14.970 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4196864 len 16384 2024-04-25T14:15:14.970 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4196864 len 16384 2024-04-25T14:15:14.970 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4197376 len 1024 2024-04-25T14:15:14.970 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4197376 len 1024 2024-04-25T14:15:14.970 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4197376 len 1024 2024-04-25T14:15:14.970 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4197376 len 1024 2024-04-25T14:15:14.970 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4197376 len 2048 2024-04-25T14:15:14.970 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4197376 len 2048 2024-04-25T14:15:14.970 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4197376 len 2048 2024-04-25T14:15:14.970 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4197376 len 2048 2024-04-25T14:15:14.970 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4197376 len 4096 2024-04-25T14:15:14.970 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4197376 len 4096 2024-04-25T14:15:14.971 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4197376 len 4096 2024-04-25T14:15:14.971 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4197376 len 4096 2024-04-25T14:15:14.971 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4197376 len 8192 2024-04-25T14:15:14.971 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4197376 len 8192 2024-04-25T14:15:14.971 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4197376 len 8192 2024-04-25T14:15:14.971 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4197376 len 8192 2024-04-25T14:15:14.971 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4197376 len 16384 2024-04-25T14:15:14.971 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4197376 len 16384 2024-04-25T14:15:14.971 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4197376 len 16384 2024-04-25T14:15:14.971 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4197376 len 16384 2024-04-25T14:15:14.971 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4197888 len 1024 2024-04-25T14:15:14.971 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4197888 len 1024 2024-04-25T14:15:14.971 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4197888 len 1024 2024-04-25T14:15:14.971 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4197888 len 1024 2024-04-25T14:15:14.971 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4197888 len 2048 2024-04-25T14:15:14.971 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4197888 len 2048 2024-04-25T14:15:14.971 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4197888 len 2048 2024-04-25T14:15:14.972 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4197888 len 2048 2024-04-25T14:15:14.972 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4197888 len 4096 2024-04-25T14:15:14.972 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4197888 len 4096 2024-04-25T14:15:14.972 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4197888 len 4096 2024-04-25T14:15:14.972 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4197888 len 4096 2024-04-25T14:15:14.972 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4197888 len 8192 2024-04-25T14:15:14.972 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4197888 len 8192 2024-04-25T14:15:14.972 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4197888 len 8192 2024-04-25T14:15:14.972 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4197888 len 8192 2024-04-25T14:15:14.972 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 2560 offset 4197888 len 16384 2024-04-25T14:15:14.972 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4197888 len 16384 2024-04-25T14:15:14.972 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 2560 offset 4197888 len 16384 2024-04-25T14:15:14.972 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 2560 offset 4197888 len 16384 2024-04-25T14:15:14.991 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4190208 len 1024 2024-04-25T14:15:14.991 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4190208 len 1024 2024-04-25T14:15:14.991 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4190208 len 1024 2024-04-25T14:15:14.991 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4190208 len 1024 2024-04-25T14:15:14.991 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4190208 len 2048 2024-04-25T14:15:14.991 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4190208 len 2048 2024-04-25T14:15:14.991 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4190208 len 2048 2024-04-25T14:15:14.991 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4190208 len 2048 2024-04-25T14:15:14.991 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4190208 len 4096 2024-04-25T14:15:14.991 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4190208 len 4096 2024-04-25T14:15:14.992 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4190208 len 4096 2024-04-25T14:15:14.992 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4190208 len 4096 2024-04-25T14:15:14.992 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4190208 len 8192 2024-04-25T14:15:14.992 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4190208 len 8192 2024-04-25T14:15:14.992 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4190208 len 8192 2024-04-25T14:15:14.992 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4190208 len 8192 2024-04-25T14:15:14.992 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4190208 len 16384 2024-04-25T14:15:14.992 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4190208 len 16384 2024-04-25T14:15:14.992 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4190208 len 16384 2024-04-25T14:15:14.992 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4190208 len 16384 2024-04-25T14:15:14.992 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4190720 len 1024 2024-04-25T14:15:14.992 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4190720 len 1024 2024-04-25T14:15:14.992 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4190720 len 1024 2024-04-25T14:15:14.992 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4190720 len 1024 2024-04-25T14:15:14.992 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4190720 len 2048 2024-04-25T14:15:14.993 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4190720 len 2048 2024-04-25T14:15:14.993 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4190720 len 2048 2024-04-25T14:15:14.993 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4190720 len 2048 2024-04-25T14:15:14.993 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4190720 len 4096 2024-04-25T14:15:14.993 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4190720 len 4096 2024-04-25T14:15:14.993 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4190720 len 4096 2024-04-25T14:15:14.993 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4190720 len 4096 2024-04-25T14:15:14.993 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4190720 len 8192 2024-04-25T14:15:14.993 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4190720 len 8192 2024-04-25T14:15:14.993 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4190720 len 8192 2024-04-25T14:15:14.993 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4190720 len 8192 2024-04-25T14:15:14.993 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4190720 len 16384 2024-04-25T14:15:14.993 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4190720 len 16384 2024-04-25T14:15:14.993 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4190720 len 16384 2024-04-25T14:15:14.993 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4190720 len 16384 2024-04-25T14:15:14.994 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4191232 len 1024 2024-04-25T14:15:14.994 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4191232 len 1024 2024-04-25T14:15:14.994 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4191232 len 1024 2024-04-25T14:15:14.994 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4191232 len 1024 2024-04-25T14:15:14.994 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4191232 len 2048 2024-04-25T14:15:14.994 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4191232 len 2048 2024-04-25T14:15:14.994 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4191232 len 2048 2024-04-25T14:15:14.994 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4191232 len 2048 2024-04-25T14:15:14.994 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4191232 len 4096 2024-04-25T14:15:14.994 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4191232 len 4096 2024-04-25T14:15:14.994 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4191232 len 4096 2024-04-25T14:15:14.994 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4191232 len 4096 2024-04-25T14:15:14.994 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4191232 len 8192 2024-04-25T14:15:14.994 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4191232 len 8192 2024-04-25T14:15:14.995 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4191232 len 8192 2024-04-25T14:15:14.995 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4191232 len 8192 2024-04-25T14:15:14.995 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4191232 len 16384 2024-04-25T14:15:14.995 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4191232 len 16384 2024-04-25T14:15:14.995 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4191232 len 16384 2024-04-25T14:15:14.995 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4191232 len 16384 2024-04-25T14:15:14.995 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4191744 len 1024 2024-04-25T14:15:14.995 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4191744 len 1024 2024-04-25T14:15:14.995 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4191744 len 1024 2024-04-25T14:15:14.995 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4191744 len 1024 2024-04-25T14:15:14.995 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4191744 len 2048 2024-04-25T14:15:14.995 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4191744 len 2048 2024-04-25T14:15:14.995 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4191744 len 2048 2024-04-25T14:15:14.995 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4191744 len 2048 2024-04-25T14:15:14.995 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4191744 len 4096 2024-04-25T14:15:14.996 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4191744 len 4096 2024-04-25T14:15:14.996 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4191744 len 4096 2024-04-25T14:15:14.996 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4191744 len 4096 2024-04-25T14:15:14.996 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4191744 len 8192 2024-04-25T14:15:14.996 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4191744 len 8192 2024-04-25T14:15:14.996 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4191744 len 8192 2024-04-25T14:15:14.996 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4191744 len 8192 2024-04-25T14:15:14.996 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4191744 len 16384 2024-04-25T14:15:14.996 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4191744 len 16384 2024-04-25T14:15:14.996 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4191744 len 16384 2024-04-25T14:15:14.996 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4191744 len 16384 2024-04-25T14:15:14.996 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4192256 len 1024 2024-04-25T14:15:14.996 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4192256 len 1024 2024-04-25T14:15:14.996 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4192256 len 1024 2024-04-25T14:15:15.016 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4192256 len 1024 2024-04-25T14:15:15.016 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4192256 len 2048 2024-04-25T14:15:15.016 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4192256 len 2048 2024-04-25T14:15:15.017 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4192256 len 2048 2024-04-25T14:15:15.017 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4192256 len 2048 2024-04-25T14:15:15.017 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4192256 len 4096 2024-04-25T14:15:15.017 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4192256 len 4096 2024-04-25T14:15:15.017 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4192256 len 4096 2024-04-25T14:15:15.017 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4192256 len 4096 2024-04-25T14:15:15.017 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4192256 len 8192 2024-04-25T14:15:15.017 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4192256 len 8192 2024-04-25T14:15:15.017 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4192256 len 8192 2024-04-25T14:15:15.017 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4192256 len 8192 2024-04-25T14:15:15.017 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4192256 len 16384 2024-04-25T14:15:15.017 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4192256 len 16384 2024-04-25T14:15:15.017 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4192256 len 16384 2024-04-25T14:15:15.017 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4192256 len 16384 2024-04-25T14:15:15.017 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4192768 len 1024 2024-04-25T14:15:15.018 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4192768 len 1024 2024-04-25T14:15:15.018 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4192768 len 1024 2024-04-25T14:15:15.018 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4192768 len 1024 2024-04-25T14:15:15.018 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4192768 len 2048 2024-04-25T14:15:15.018 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4192768 len 2048 2024-04-25T14:15:15.018 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4192768 len 2048 2024-04-25T14:15:15.018 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4192768 len 2048 2024-04-25T14:15:15.018 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4192768 len 4096 2024-04-25T14:15:15.018 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4192768 len 4096 2024-04-25T14:15:15.018 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4192768 len 4096 2024-04-25T14:15:15.018 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4192768 len 4096 2024-04-25T14:15:15.018 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4192768 len 8192 2024-04-25T14:15:15.018 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4192768 len 8192 2024-04-25T14:15:15.018 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4192768 len 8192 2024-04-25T14:15:15.019 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4192768 len 8192 2024-04-25T14:15:15.019 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4192768 len 16384 2024-04-25T14:15:15.019 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4192768 len 16384 2024-04-25T14:15:15.019 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4192768 len 16384 2024-04-25T14:15:15.019 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4192768 len 16384 2024-04-25T14:15:15.019 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4193280 len 1024 2024-04-25T14:15:15.019 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4193280 len 1024 2024-04-25T14:15:15.019 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4193280 len 1024 2024-04-25T14:15:15.019 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4193280 len 1024 2024-04-25T14:15:15.019 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4193280 len 2048 2024-04-25T14:15:15.019 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4193280 len 2048 2024-04-25T14:15:15.019 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4193280 len 2048 2024-04-25T14:15:15.019 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4193280 len 2048 2024-04-25T14:15:15.019 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4193280 len 4096 2024-04-25T14:15:15.019 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4193280 len 4096 2024-04-25T14:15:15.020 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4193280 len 4096 2024-04-25T14:15:15.020 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4193280 len 4096 2024-04-25T14:15:15.020 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4193280 len 8192 2024-04-25T14:15:15.020 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4193280 len 8192 2024-04-25T14:15:15.020 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4193280 len 8192 2024-04-25T14:15:15.020 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4193280 len 8192 2024-04-25T14:15:15.020 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4193280 len 16384 2024-04-25T14:15:15.020 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4193280 len 16384 2024-04-25T14:15:15.020 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4193280 len 16384 2024-04-25T14:15:15.020 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4193280 len 16384 2024-04-25T14:15:15.020 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4193792 len 1024 2024-04-25T14:15:15.020 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4193792 len 1024 2024-04-25T14:15:15.020 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4193792 len 1024 2024-04-25T14:15:15.020 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4193792 len 1024 2024-04-25T14:15:15.020 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4193792 len 2048 2024-04-25T14:15:15.021 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4193792 len 2048 2024-04-25T14:15:15.021 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4193792 len 2048 2024-04-25T14:15:15.021 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4193792 len 2048 2024-04-25T14:15:15.021 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4193792 len 4096 2024-04-25T14:15:15.021 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4193792 len 4096 2024-04-25T14:15:15.021 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4193792 len 4096 2024-04-25T14:15:15.021 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4193792 len 4096 2024-04-25T14:15:15.021 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4193792 len 8192 2024-04-25T14:15:15.021 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4193792 len 8192 2024-04-25T14:15:15.021 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4193792 len 8192 2024-04-25T14:15:15.021 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4193792 len 8192 2024-04-25T14:15:15.021 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4193792 len 16384 2024-04-25T14:15:15.021 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4193792 len 16384 2024-04-25T14:15:15.021 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4193792 len 16384 2024-04-25T14:15:15.021 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4193792 len 16384 2024-04-25T14:15:15.022 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4194304 len 1024 2024-04-25T14:15:15.022 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4194304 len 1024 2024-04-25T14:15:15.022 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4194304 len 1024 2024-04-25T14:15:15.022 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4194304 len 1024 2024-04-25T14:15:15.022 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4194304 len 2048 2024-04-25T14:15:15.027 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4194304 len 2048 2024-04-25T14:15:15.027 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4194304 len 2048 2024-04-25T14:15:15.027 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4194304 len 2048 2024-04-25T14:15:15.027 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4194304 len 4096 2024-04-25T14:15:15.027 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4194304 len 4096 2024-04-25T14:15:15.027 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4194304 len 4096 2024-04-25T14:15:15.027 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4194304 len 4096 2024-04-25T14:15:15.027 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4194304 len 8192 2024-04-25T14:15:15.027 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4194304 len 8192 2024-04-25T14:15:15.027 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4194304 len 8192 2024-04-25T14:15:15.027 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4194304 len 8192 2024-04-25T14:15:15.027 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4194304 len 16384 2024-04-25T14:15:15.027 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4194304 len 16384 2024-04-25T14:15:15.028 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4194304 len 16384 2024-04-25T14:15:15.028 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4194304 len 16384 2024-04-25T14:15:15.028 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4194816 len 1024 2024-04-25T14:15:15.028 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4194816 len 1024 2024-04-25T14:15:15.028 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4194816 len 1024 2024-04-25T14:15:15.028 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4194816 len 1024 2024-04-25T14:15:15.028 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4194816 len 2048 2024-04-25T14:15:15.028 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4194816 len 2048 2024-04-25T14:15:15.028 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4194816 len 2048 2024-04-25T14:15:15.028 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4194816 len 2048 2024-04-25T14:15:15.028 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4194816 len 4096 2024-04-25T14:15:15.028 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4194816 len 4096 2024-04-25T14:15:15.028 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4194816 len 4096 2024-04-25T14:15:15.028 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4194816 len 4096 2024-04-25T14:15:15.028 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4194816 len 8192 2024-04-25T14:15:15.029 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4194816 len 8192 2024-04-25T14:15:15.029 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4194816 len 8192 2024-04-25T14:15:15.029 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4194816 len 8192 2024-04-25T14:15:15.029 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4194816 len 16384 2024-04-25T14:15:15.029 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4194816 len 16384 2024-04-25T14:15:15.029 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4194816 len 16384 2024-04-25T14:15:15.029 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4194816 len 16384 2024-04-25T14:15:15.029 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4195328 len 1024 2024-04-25T14:15:15.029 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4195328 len 1024 2024-04-25T14:15:15.029 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4195328 len 1024 2024-04-25T14:15:15.029 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4195328 len 1024 2024-04-25T14:15:15.029 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4195328 len 2048 2024-04-25T14:15:15.029 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4195328 len 2048 2024-04-25T14:15:15.029 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4195328 len 2048 2024-04-25T14:15:15.029 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4195328 len 2048 2024-04-25T14:15:15.029 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4195328 len 4096 2024-04-25T14:15:15.030 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4195328 len 4096 2024-04-25T14:15:15.030 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4195328 len 4096 2024-04-25T14:15:15.030 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4195328 len 4096 2024-04-25T14:15:15.030 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4195328 len 8192 2024-04-25T14:15:15.030 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4195328 len 8192 2024-04-25T14:15:15.030 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4195328 len 8192 2024-04-25T14:15:15.030 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4195328 len 8192 2024-04-25T14:15:15.030 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4195328 len 16384 2024-04-25T14:15:15.030 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4195328 len 16384 2024-04-25T14:15:15.030 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4195328 len 16384 2024-04-25T14:15:15.030 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4195328 len 16384 2024-04-25T14:15:15.030 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4195840 len 1024 2024-04-25T14:15:15.030 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4195840 len 1024 2024-04-25T14:15:15.030 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4195840 len 1024 2024-04-25T14:15:15.031 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4195840 len 1024 2024-04-25T14:15:15.031 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4195840 len 2048 2024-04-25T14:15:15.031 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4195840 len 2048 2024-04-25T14:15:15.031 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4195840 len 2048 2024-04-25T14:15:15.031 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4195840 len 2048 2024-04-25T14:15:15.031 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4195840 len 4096 2024-04-25T14:15:15.031 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4195840 len 4096 2024-04-25T14:15:15.031 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4195840 len 4096 2024-04-25T14:15:15.031 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4195840 len 4096 2024-04-25T14:15:15.031 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4195840 len 8192 2024-04-25T14:15:15.031 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4195840 len 8192 2024-04-25T14:15:15.031 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4195840 len 8192 2024-04-25T14:15:15.031 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4195840 len 8192 2024-04-25T14:15:15.031 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4195840 len 16384 2024-04-25T14:15:15.031 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4195840 len 16384 2024-04-25T14:15:15.031 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4195840 len 16384 2024-04-25T14:15:15.032 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4195840 len 16384 2024-04-25T14:15:15.032 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4196352 len 1024 2024-04-25T14:15:15.032 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4196352 len 1024 2024-04-25T14:15:15.032 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4196352 len 1024 2024-04-25T14:15:15.032 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4196352 len 1024 2024-04-25T14:15:15.032 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4196352 len 2048 2024-04-25T14:15:15.032 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4196352 len 2048 2024-04-25T14:15:15.032 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4196352 len 2048 2024-04-25T14:15:15.032 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4196352 len 2048 2024-04-25T14:15:15.037 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4196352 len 4096 2024-04-25T14:15:15.037 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4196352 len 4096 2024-04-25T14:15:15.037 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4196352 len 4096 2024-04-25T14:15:15.037 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4196352 len 4096 2024-04-25T14:15:15.037 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4196352 len 8192 2024-04-25T14:15:15.037 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4196352 len 8192 2024-04-25T14:15:15.037 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4196352 len 8192 2024-04-25T14:15:15.037 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4196352 len 8192 2024-04-25T14:15:15.037 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4196352 len 16384 2024-04-25T14:15:15.038 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4196352 len 16384 2024-04-25T14:15:15.038 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4196352 len 16384 2024-04-25T14:15:15.038 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4196352 len 16384 2024-04-25T14:15:15.038 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4196864 len 1024 2024-04-25T14:15:15.038 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4196864 len 1024 2024-04-25T14:15:15.038 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4196864 len 1024 2024-04-25T14:15:15.038 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4196864 len 1024 2024-04-25T14:15:15.038 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4196864 len 2048 2024-04-25T14:15:15.038 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4196864 len 2048 2024-04-25T14:15:15.038 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4196864 len 2048 2024-04-25T14:15:15.038 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4196864 len 2048 2024-04-25T14:15:15.038 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4196864 len 4096 2024-04-25T14:15:15.038 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4196864 len 4096 2024-04-25T14:15:15.039 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4196864 len 4096 2024-04-25T14:15:15.039 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4196864 len 4096 2024-04-25T14:15:15.039 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4196864 len 8192 2024-04-25T14:15:15.039 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4196864 len 8192 2024-04-25T14:15:15.039 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4196864 len 8192 2024-04-25T14:15:15.039 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4196864 len 8192 2024-04-25T14:15:15.039 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4196864 len 16384 2024-04-25T14:15:15.039 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4196864 len 16384 2024-04-25T14:15:15.039 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4196864 len 16384 2024-04-25T14:15:15.039 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4196864 len 16384 2024-04-25T14:15:15.039 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4197376 len 1024 2024-04-25T14:15:15.039 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4197376 len 1024 2024-04-25T14:15:15.039 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4197376 len 1024 2024-04-25T14:15:15.039 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4197376 len 1024 2024-04-25T14:15:15.039 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4197376 len 2048 2024-04-25T14:15:15.039 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4197376 len 2048 2024-04-25T14:15:15.040 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4197376 len 2048 2024-04-25T14:15:15.040 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4197376 len 2048 2024-04-25T14:15:15.040 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4197376 len 4096 2024-04-25T14:15:15.040 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4197376 len 4096 2024-04-25T14:15:15.040 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4197376 len 4096 2024-04-25T14:15:15.040 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4197376 len 4096 2024-04-25T14:15:15.040 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4197376 len 8192 2024-04-25T14:15:15.040 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4197376 len 8192 2024-04-25T14:15:15.040 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4197376 len 8192 2024-04-25T14:15:15.040 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4197376 len 8192 2024-04-25T14:15:15.040 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4197376 len 16384 2024-04-25T14:15:15.040 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4197376 len 16384 2024-04-25T14:15:15.040 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4197376 len 16384 2024-04-25T14:15:15.040 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4197376 len 16384 2024-04-25T14:15:15.041 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4197888 len 1024 2024-04-25T14:15:15.041 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4197888 len 1024 2024-04-25T14:15:15.041 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4197888 len 1024 2024-04-25T14:15:15.041 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4197888 len 1024 2024-04-25T14:15:15.041 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4197888 len 2048 2024-04-25T14:15:15.041 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4197888 len 2048 2024-04-25T14:15:15.041 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4197888 len 2048 2024-04-25T14:15:15.041 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4197888 len 2048 2024-04-25T14:15:15.041 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4197888 len 4096 2024-04-25T14:15:15.041 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4197888 len 4096 2024-04-25T14:15:15.041 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4197888 len 4096 2024-04-25T14:15:15.041 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4197888 len 4096 2024-04-25T14:15:15.041 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4197888 len 8192 2024-04-25T14:15:15.041 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4197888 len 8192 2024-04-25T14:15:15.042 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4197888 len 8192 2024-04-25T14:15:15.042 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4197888 len 8192 2024-04-25T14:15:15.042 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3072 offset 4197888 len 16384 2024-04-25T14:15:15.042 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4197888 len 16384 2024-04-25T14:15:15.042 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3072 offset 4197888 len 16384 2024-04-25T14:15:15.042 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3072 offset 4197888 len 16384 2024-04-25T14:15:15.042 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4190208 len 1024 2024-04-25T14:15:15.042 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4190208 len 1024 2024-04-25T14:15:15.042 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4190208 len 1024 2024-04-25T14:15:15.042 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4190208 len 1024 2024-04-25T14:15:15.042 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4190208 len 2048 2024-04-25T14:15:15.042 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4190208 len 2048 2024-04-25T14:15:15.042 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4190208 len 2048 2024-04-25T14:15:15.042 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4190208 len 2048 2024-04-25T14:15:15.043 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4190208 len 4096 2024-04-25T14:15:15.043 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4190208 len 4096 2024-04-25T14:15:15.056 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4190208 len 4096 2024-04-25T14:15:15.056 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4190208 len 4096 2024-04-25T14:15:15.056 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4190208 len 8192 2024-04-25T14:15:15.056 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4190208 len 8192 2024-04-25T14:15:15.057 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4190208 len 8192 2024-04-25T14:15:15.057 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4190208 len 8192 2024-04-25T14:15:15.057 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4190208 len 16384 2024-04-25T14:15:15.057 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4190208 len 16384 2024-04-25T14:15:15.057 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4190208 len 16384 2024-04-25T14:15:15.057 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4190208 len 16384 2024-04-25T14:15:15.057 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4190720 len 1024 2024-04-25T14:15:15.057 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4190720 len 1024 2024-04-25T14:15:15.057 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4190720 len 1024 2024-04-25T14:15:15.057 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4190720 len 1024 2024-04-25T14:15:15.057 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4190720 len 2048 2024-04-25T14:15:15.057 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4190720 len 2048 2024-04-25T14:15:15.057 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4190720 len 2048 2024-04-25T14:15:15.057 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4190720 len 2048 2024-04-25T14:15:15.058 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4190720 len 4096 2024-04-25T14:15:15.058 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4190720 len 4096 2024-04-25T14:15:15.058 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4190720 len 4096 2024-04-25T14:15:15.058 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4190720 len 4096 2024-04-25T14:15:15.058 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4190720 len 8192 2024-04-25T14:15:15.058 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4190720 len 8192 2024-04-25T14:15:15.058 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4190720 len 8192 2024-04-25T14:15:15.058 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4190720 len 8192 2024-04-25T14:15:15.058 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4190720 len 16384 2024-04-25T14:15:15.058 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4190720 len 16384 2024-04-25T14:15:15.058 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4190720 len 16384 2024-04-25T14:15:15.058 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4190720 len 16384 2024-04-25T14:15:15.058 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4191232 len 1024 2024-04-25T14:15:15.058 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4191232 len 1024 2024-04-25T14:15:15.059 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4191232 len 1024 2024-04-25T14:15:15.059 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4191232 len 1024 2024-04-25T14:15:15.059 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4191232 len 2048 2024-04-25T14:15:15.059 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4191232 len 2048 2024-04-25T14:15:15.059 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4191232 len 2048 2024-04-25T14:15:15.059 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4191232 len 2048 2024-04-25T14:15:15.059 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4191232 len 4096 2024-04-25T14:15:15.059 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4191232 len 4096 2024-04-25T14:15:15.059 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4191232 len 4096 2024-04-25T14:15:15.059 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4191232 len 4096 2024-04-25T14:15:15.059 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4191232 len 8192 2024-04-25T14:15:15.059 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4191232 len 8192 2024-04-25T14:15:15.059 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4191232 len 8192 2024-04-25T14:15:15.059 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4191232 len 8192 2024-04-25T14:15:15.059 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4191232 len 16384 2024-04-25T14:15:15.060 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4191232 len 16384 2024-04-25T14:15:15.060 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4191232 len 16384 2024-04-25T14:15:15.060 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4191232 len 16384 2024-04-25T14:15:15.060 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4191744 len 1024 2024-04-25T14:15:15.060 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4191744 len 1024 2024-04-25T14:15:15.060 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4191744 len 1024 2024-04-25T14:15:15.060 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4191744 len 1024 2024-04-25T14:15:15.060 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4191744 len 2048 2024-04-25T14:15:15.060 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4191744 len 2048 2024-04-25T14:15:15.060 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4191744 len 2048 2024-04-25T14:15:15.060 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4191744 len 2048 2024-04-25T14:15:15.060 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4191744 len 4096 2024-04-25T14:15:15.060 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4191744 len 4096 2024-04-25T14:15:15.060 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4191744 len 4096 2024-04-25T14:15:15.060 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4191744 len 4096 2024-04-25T14:15:15.061 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4191744 len 8192 2024-04-25T14:15:15.061 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4191744 len 8192 2024-04-25T14:15:15.061 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4191744 len 8192 2024-04-25T14:15:15.061 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4191744 len 8192 2024-04-25T14:15:15.061 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4191744 len 16384 2024-04-25T14:15:15.061 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4191744 len 16384 2024-04-25T14:15:15.061 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4191744 len 16384 2024-04-25T14:15:15.061 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4191744 len 16384 2024-04-25T14:15:15.061 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4192256 len 1024 2024-04-25T14:15:15.061 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4192256 len 1024 2024-04-25T14:15:15.061 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4192256 len 1024 2024-04-25T14:15:15.061 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4192256 len 1024 2024-04-25T14:15:15.061 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4192256 len 2048 2024-04-25T14:15:15.061 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4192256 len 2048 2024-04-25T14:15:15.061 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4192256 len 2048 2024-04-25T14:15:15.062 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4192256 len 2048 2024-04-25T14:15:15.062 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4192256 len 4096 2024-04-25T14:15:15.062 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4192256 len 4096 2024-04-25T14:15:15.062 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4192256 len 4096 2024-04-25T14:15:15.062 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4192256 len 4096 2024-04-25T14:15:15.062 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4192256 len 8192 2024-04-25T14:15:15.078 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4192256 len 8192 2024-04-25T14:15:15.078 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4192256 len 8192 2024-04-25T14:15:15.078 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4192256 len 8192 2024-04-25T14:15:15.078 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4192256 len 16384 2024-04-25T14:15:15.078 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4192256 len 16384 2024-04-25T14:15:15.078 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4192256 len 16384 2024-04-25T14:15:15.078 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4192256 len 16384 2024-04-25T14:15:15.079 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4192768 len 1024 2024-04-25T14:15:15.079 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4192768 len 1024 2024-04-25T14:15:15.079 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4192768 len 1024 2024-04-25T14:15:15.079 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4192768 len 1024 2024-04-25T14:15:15.079 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4192768 len 2048 2024-04-25T14:15:15.079 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4192768 len 2048 2024-04-25T14:15:15.079 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4192768 len 2048 2024-04-25T14:15:15.079 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4192768 len 2048 2024-04-25T14:15:15.079 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4192768 len 4096 2024-04-25T14:15:15.079 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4192768 len 4096 2024-04-25T14:15:15.079 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4192768 len 4096 2024-04-25T14:15:15.079 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4192768 len 4096 2024-04-25T14:15:15.079 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4192768 len 8192 2024-04-25T14:15:15.079 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4192768 len 8192 2024-04-25T14:15:15.079 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4192768 len 8192 2024-04-25T14:15:15.079 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4192768 len 8192 2024-04-25T14:15:15.080 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4192768 len 16384 2024-04-25T14:15:15.080 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4192768 len 16384 2024-04-25T14:15:15.080 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4192768 len 16384 2024-04-25T14:15:15.080 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4192768 len 16384 2024-04-25T14:15:15.080 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4193280 len 1024 2024-04-25T14:15:15.080 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4193280 len 1024 2024-04-25T14:15:15.080 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4193280 len 1024 2024-04-25T14:15:15.080 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4193280 len 1024 2024-04-25T14:15:15.080 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4193280 len 2048 2024-04-25T14:15:15.080 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4193280 len 2048 2024-04-25T14:15:15.080 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4193280 len 2048 2024-04-25T14:15:15.080 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4193280 len 2048 2024-04-25T14:15:15.080 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4193280 len 4096 2024-04-25T14:15:15.080 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4193280 len 4096 2024-04-25T14:15:15.080 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4193280 len 4096 2024-04-25T14:15:15.080 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4193280 len 4096 2024-04-25T14:15:15.081 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4193280 len 8192 2024-04-25T14:15:15.081 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4193280 len 8192 2024-04-25T14:15:15.081 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4193280 len 8192 2024-04-25T14:15:15.081 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4193280 len 8192 2024-04-25T14:15:15.081 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4193280 len 16384 2024-04-25T14:15:15.081 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4193280 len 16384 2024-04-25T14:15:15.081 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4193280 len 16384 2024-04-25T14:15:15.081 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4193280 len 16384 2024-04-25T14:15:15.081 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4193792 len 1024 2024-04-25T14:15:15.081 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4193792 len 1024 2024-04-25T14:15:15.081 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4193792 len 1024 2024-04-25T14:15:15.081 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4193792 len 1024 2024-04-25T14:15:15.081 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4193792 len 2048 2024-04-25T14:15:15.081 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4193792 len 2048 2024-04-25T14:15:15.081 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4193792 len 2048 2024-04-25T14:15:15.081 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4193792 len 2048 2024-04-25T14:15:15.082 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4193792 len 4096 2024-04-25T14:15:15.082 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4193792 len 4096 2024-04-25T14:15:15.082 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4193792 len 4096 2024-04-25T14:15:15.082 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4193792 len 4096 2024-04-25T14:15:15.082 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4193792 len 8192 2024-04-25T14:15:15.082 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4193792 len 8192 2024-04-25T14:15:15.082 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4193792 len 8192 2024-04-25T14:15:15.082 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4193792 len 8192 2024-04-25T14:15:15.082 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4193792 len 16384 2024-04-25T14:15:15.082 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4193792 len 16384 2024-04-25T14:15:15.082 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4193792 len 16384 2024-04-25T14:15:15.082 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4193792 len 16384 2024-04-25T14:15:15.082 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4194304 len 1024 2024-04-25T14:15:15.082 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4194304 len 1024 2024-04-25T14:15:15.082 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4194304 len 1024 2024-04-25T14:15:15.083 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4194304 len 1024 2024-04-25T14:15:15.083 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4194304 len 2048 2024-04-25T14:15:15.083 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4194304 len 2048 2024-04-25T14:15:15.083 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4194304 len 2048 2024-04-25T14:15:15.083 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4194304 len 2048 2024-04-25T14:15:15.083 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4194304 len 4096 2024-04-25T14:15:15.083 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4194304 len 4096 2024-04-25T14:15:15.083 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4194304 len 4096 2024-04-25T14:15:15.083 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4194304 len 4096 2024-04-25T14:15:15.083 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4194304 len 8192 2024-04-25T14:15:15.083 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4194304 len 8192 2024-04-25T14:15:15.083 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4194304 len 8192 2024-04-25T14:15:15.088 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4194304 len 8192 2024-04-25T14:15:15.088 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4194304 len 16384 2024-04-25T14:15:15.088 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4194304 len 16384 2024-04-25T14:15:15.088 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4194304 len 16384 2024-04-25T14:15:15.088 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4194304 len 16384 2024-04-25T14:15:15.088 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4194816 len 1024 2024-04-25T14:15:15.088 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4194816 len 1024 2024-04-25T14:15:15.088 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4194816 len 1024 2024-04-25T14:15:15.088 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4194816 len 1024 2024-04-25T14:15:15.088 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4194816 len 2048 2024-04-25T14:15:15.088 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4194816 len 2048 2024-04-25T14:15:15.088 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4194816 len 2048 2024-04-25T14:15:15.088 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4194816 len 2048 2024-04-25T14:15:15.089 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4194816 len 4096 2024-04-25T14:15:15.089 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4194816 len 4096 2024-04-25T14:15:15.089 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4194816 len 4096 2024-04-25T14:15:15.089 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4194816 len 4096 2024-04-25T14:15:15.089 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4194816 len 8192 2024-04-25T14:15:15.089 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4194816 len 8192 2024-04-25T14:15:15.089 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4194816 len 8192 2024-04-25T14:15:15.089 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4194816 len 8192 2024-04-25T14:15:15.089 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4194816 len 16384 2024-04-25T14:15:15.089 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4194816 len 16384 2024-04-25T14:15:15.089 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4194816 len 16384 2024-04-25T14:15:15.089 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4194816 len 16384 2024-04-25T14:15:15.089 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4195328 len 1024 2024-04-25T14:15:15.089 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4195328 len 1024 2024-04-25T14:15:15.089 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4195328 len 1024 2024-04-25T14:15:15.090 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4195328 len 1024 2024-04-25T14:15:15.090 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4195328 len 2048 2024-04-25T14:15:15.090 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4195328 len 2048 2024-04-25T14:15:15.090 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4195328 len 2048 2024-04-25T14:15:15.090 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4195328 len 2048 2024-04-25T14:15:15.090 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4195328 len 4096 2024-04-25T14:15:15.090 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4195328 len 4096 2024-04-25T14:15:15.090 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4195328 len 4096 2024-04-25T14:15:15.090 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4195328 len 4096 2024-04-25T14:15:15.090 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4195328 len 8192 2024-04-25T14:15:15.090 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4195328 len 8192 2024-04-25T14:15:15.090 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4195328 len 8192 2024-04-25T14:15:15.090 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4195328 len 8192 2024-04-25T14:15:15.090 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4195328 len 16384 2024-04-25T14:15:15.091 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4195328 len 16384 2024-04-25T14:15:15.091 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4195328 len 16384 2024-04-25T14:15:15.091 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4195328 len 16384 2024-04-25T14:15:15.091 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4195840 len 1024 2024-04-25T14:15:15.091 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4195840 len 1024 2024-04-25T14:15:15.091 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4195840 len 1024 2024-04-25T14:15:15.091 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4195840 len 1024 2024-04-25T14:15:15.091 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4195840 len 2048 2024-04-25T14:15:15.091 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4195840 len 2048 2024-04-25T14:15:15.091 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4195840 len 2048 2024-04-25T14:15:15.091 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4195840 len 2048 2024-04-25T14:15:15.091 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4195840 len 4096 2024-04-25T14:15:15.091 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4195840 len 4096 2024-04-25T14:15:15.091 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4195840 len 4096 2024-04-25T14:15:15.091 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4195840 len 4096 2024-04-25T14:15:15.092 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4195840 len 8192 2024-04-25T14:15:15.092 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4195840 len 8192 2024-04-25T14:15:15.092 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4195840 len 8192 2024-04-25T14:15:15.092 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4195840 len 8192 2024-04-25T14:15:15.092 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4195840 len 16384 2024-04-25T14:15:15.092 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4195840 len 16384 2024-04-25T14:15:15.092 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4195840 len 16384 2024-04-25T14:15:15.092 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4195840 len 16384 2024-04-25T14:15:15.092 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4196352 len 1024 2024-04-25T14:15:15.092 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4196352 len 1024 2024-04-25T14:15:15.092 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4196352 len 1024 2024-04-25T14:15:15.092 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4196352 len 1024 2024-04-25T14:15:15.092 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4196352 len 2048 2024-04-25T14:15:15.092 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4196352 len 2048 2024-04-25T14:15:15.092 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4196352 len 2048 2024-04-25T14:15:15.093 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4196352 len 2048 2024-04-25T14:15:15.093 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4196352 len 4096 2024-04-25T14:15:15.093 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4196352 len 4096 2024-04-25T14:15:15.093 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4196352 len 4096 2024-04-25T14:15:15.093 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4196352 len 4096 2024-04-25T14:15:15.093 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4196352 len 8192 2024-04-25T14:15:15.093 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4196352 len 8192 2024-04-25T14:15:15.093 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4196352 len 8192 2024-04-25T14:15:15.093 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4196352 len 8192 2024-04-25T14:15:15.093 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4196352 len 16384 2024-04-25T14:15:15.282 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4196352 len 16384 2024-04-25T14:15:15.283 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4196352 len 16384 2024-04-25T14:15:15.283 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4196352 len 16384 2024-04-25T14:15:15.283 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4196864 len 1024 2024-04-25T14:15:15.283 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4196864 len 1024 2024-04-25T14:15:15.283 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4196864 len 1024 2024-04-25T14:15:15.283 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4196864 len 1024 2024-04-25T14:15:15.283 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4196864 len 2048 2024-04-25T14:15:15.283 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4196864 len 2048 2024-04-25T14:15:15.283 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4196864 len 2048 2024-04-25T14:15:15.283 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4196864 len 2048 2024-04-25T14:15:15.283 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4196864 len 4096 2024-04-25T14:15:15.283 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4196864 len 4096 2024-04-25T14:15:15.283 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4196864 len 4096 2024-04-25T14:15:15.284 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4196864 len 4096 2024-04-25T14:15:15.284 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4196864 len 8192 2024-04-25T14:15:15.284 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4196864 len 8192 2024-04-25T14:15:15.284 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4196864 len 8192 2024-04-25T14:15:15.284 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4196864 len 8192 2024-04-25T14:15:15.284 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4196864 len 16384 2024-04-25T14:15:15.284 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4196864 len 16384 2024-04-25T14:15:15.284 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4196864 len 16384 2024-04-25T14:15:15.284 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4196864 len 16384 2024-04-25T14:15:15.284 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4197376 len 1024 2024-04-25T14:15:15.284 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4197376 len 1024 2024-04-25T14:15:15.284 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4197376 len 1024 2024-04-25T14:15:15.284 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4197376 len 1024 2024-04-25T14:15:15.284 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4197376 len 2048 2024-04-25T14:15:15.284 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4197376 len 2048 2024-04-25T14:15:15.285 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4197376 len 2048 2024-04-25T14:15:15.285 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4197376 len 2048 2024-04-25T14:15:15.285 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4197376 len 4096 2024-04-25T14:15:15.285 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4197376 len 4096 2024-04-25T14:15:15.285 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4197376 len 4096 2024-04-25T14:15:15.285 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4197376 len 4096 2024-04-25T14:15:15.285 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4197376 len 8192 2024-04-25T14:15:15.285 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4197376 len 8192 2024-04-25T14:15:15.285 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4197376 len 8192 2024-04-25T14:15:15.285 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4197376 len 8192 2024-04-25T14:15:15.285 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4197376 len 16384 2024-04-25T14:15:15.285 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4197376 len 16384 2024-04-25T14:15:15.285 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4197376 len 16384 2024-04-25T14:15:15.285 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4197376 len 16384 2024-04-25T14:15:15.285 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4197888 len 1024 2024-04-25T14:15:15.285 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4197888 len 1024 2024-04-25T14:15:15.286 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4197888 len 1024 2024-04-25T14:15:15.286 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4197888 len 1024 2024-04-25T14:15:15.286 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4197888 len 2048 2024-04-25T14:15:15.286 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4197888 len 2048 2024-04-25T14:15:15.286 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4197888 len 2048 2024-04-25T14:15:15.286 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4197888 len 2048 2024-04-25T14:15:15.286 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4197888 len 4096 2024-04-25T14:15:15.286 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4197888 len 4096 2024-04-25T14:15:15.286 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4197888 len 4096 2024-04-25T14:15:15.286 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4197888 len 4096 2024-04-25T14:15:15.286 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4197888 len 8192 2024-04-25T14:15:15.286 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4197888 len 8192 2024-04-25T14:15:15.286 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4197888 len 8192 2024-04-25T14:15:15.287 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4197888 len 8192 2024-04-25T14:15:15.287 INFO:tasks.workunit.client.0.smithi096.stdout:read_direct buf_align 3584 offset 4197888 len 16384 2024-04-25T14:15:15.287 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4197888 len 16384 2024-04-25T14:15:15.287 INFO:tasks.workunit.client.0.smithi096.stdout:read_sync buf_align 3584 offset 4197888 len 16384 2024-04-25T14:15:15.287 INFO:tasks.workunit.client.0.smithi096.stdout:read_file buf_align 3584 offset 4197888 len 16384 2024-04-25T14:15:15.287 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4190720 len 1024 2024-04-25T14:15:15.287 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4190720 len 1024 2024-04-25T14:15:15.287 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4190720 len 1024 2024-04-25T14:15:15.287 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4190720 len 1024 2024-04-25T14:15:15.287 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4190720 len 2048 2024-04-25T14:15:15.287 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4190720 len 2048 2024-04-25T14:15:15.287 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4190720 len 2048 2024-04-25T14:15:15.287 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4190720 len 2048 2024-04-25T14:15:15.287 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4190720 len 4096 2024-04-25T14:15:15.287 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4190720 len 4096 2024-04-25T14:15:15.288 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4190720 len 4096 2024-04-25T14:15:15.288 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4190720 len 4096 2024-04-25T14:15:15.288 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4190720 len 8192 2024-04-25T14:15:15.288 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4190720 len 8192 2024-04-25T14:15:15.288 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4190720 len 8192 2024-04-25T14:15:15.288 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4190720 len 8192 2024-04-25T14:15:15.288 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4190720 len 16384 2024-04-25T14:15:15.288 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4190720 len 16384 2024-04-25T14:15:15.288 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4190720 len 16384 2024-04-25T14:15:15.288 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4190720 len 16384 2024-04-25T14:15:15.288 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4191232 len 1024 2024-04-25T14:15:16.015 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4191232 len 1024 2024-04-25T14:15:16.016 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4191232 len 1024 2024-04-25T14:15:16.016 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4191232 len 1024 2024-04-25T14:15:16.016 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4191232 len 2048 2024-04-25T14:15:16.016 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4191232 len 2048 2024-04-25T14:15:16.016 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4191232 len 2048 2024-04-25T14:15:16.016 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4191232 len 2048 2024-04-25T14:15:16.016 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4191232 len 4096 2024-04-25T14:15:16.016 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4191232 len 4096 2024-04-25T14:15:16.016 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4191232 len 4096 2024-04-25T14:15:16.016 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4191232 len 4096 2024-04-25T14:15:16.016 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4191232 len 8192 2024-04-25T14:15:16.016 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4191232 len 8192 2024-04-25T14:15:16.016 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4191232 len 8192 2024-04-25T14:15:16.016 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4191232 len 8192 2024-04-25T14:15:16.016 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4191232 len 16384 2024-04-25T14:15:16.017 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4191232 len 16384 2024-04-25T14:15:16.017 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4191232 len 16384 2024-04-25T14:15:16.017 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4191232 len 16384 2024-04-25T14:15:16.017 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4191744 len 1024 2024-04-25T14:15:16.017 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4191744 len 1024 2024-04-25T14:15:16.017 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4191744 len 1024 2024-04-25T14:15:16.017 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4191744 len 1024 2024-04-25T14:15:16.017 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4191744 len 2048 2024-04-25T14:15:16.017 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4191744 len 2048 2024-04-25T14:15:16.017 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4191744 len 2048 2024-04-25T14:15:16.017 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4191744 len 2048 2024-04-25T14:15:16.017 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4191744 len 4096 2024-04-25T14:15:16.017 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4191744 len 4096 2024-04-25T14:15:16.017 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4191744 len 4096 2024-04-25T14:15:16.018 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4191744 len 4096 2024-04-25T14:15:16.018 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4191744 len 8192 2024-04-25T14:15:16.018 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4191744 len 8192 2024-04-25T14:15:16.018 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4191744 len 8192 2024-04-25T14:15:16.018 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4191744 len 8192 2024-04-25T14:15:16.018 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4191744 len 16384 2024-04-25T14:15:16.018 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4191744 len 16384 2024-04-25T14:15:16.018 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4191744 len 16384 2024-04-25T14:15:16.018 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4191744 len 16384 2024-04-25T14:15:16.018 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4192256 len 1024 2024-04-25T14:15:16.018 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4192256 len 1024 2024-04-25T14:15:16.018 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4192256 len 1024 2024-04-25T14:15:16.018 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4192256 len 1024 2024-04-25T14:15:16.018 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4192256 len 2048 2024-04-25T14:15:16.019 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4192256 len 2048 2024-04-25T14:15:16.019 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4192256 len 2048 2024-04-25T14:15:16.019 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4192256 len 2048 2024-04-25T14:15:16.019 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4192256 len 4096 2024-04-25T14:15:16.019 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4192256 len 4096 2024-04-25T14:15:16.019 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4192256 len 4096 2024-04-25T14:15:16.019 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4192256 len 4096 2024-04-25T14:15:16.019 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4192256 len 8192 2024-04-25T14:15:16.019 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4192256 len 8192 2024-04-25T14:15:16.019 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4192256 len 8192 2024-04-25T14:15:16.019 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4192256 len 8192 2024-04-25T14:15:16.019 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4192256 len 16384 2024-04-25T14:15:16.019 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4192256 len 16384 2024-04-25T14:15:16.019 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4192256 len 16384 2024-04-25T14:15:16.019 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4192256 len 16384 2024-04-25T14:15:16.020 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4192768 len 1024 2024-04-25T14:15:16.020 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4192768 len 1024 2024-04-25T14:15:16.020 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4192768 len 1024 2024-04-25T14:15:16.020 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4192768 len 1024 2024-04-25T14:15:16.020 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4192768 len 2048 2024-04-25T14:15:16.020 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4192768 len 2048 2024-04-25T14:15:16.020 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4192768 len 2048 2024-04-25T14:15:16.020 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4192768 len 2048 2024-04-25T14:15:16.020 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4192768 len 4096 2024-04-25T14:15:16.020 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4192768 len 4096 2024-04-25T14:15:16.020 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4192768 len 4096 2024-04-25T14:15:16.020 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4192768 len 4096 2024-04-25T14:15:16.020 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4192768 len 8192 2024-04-25T14:15:16.020 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4192768 len 8192 2024-04-25T14:15:16.020 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4192768 len 8192 2024-04-25T14:15:16.020 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4192768 len 8192 2024-04-25T14:15:16.021 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4192768 len 16384 2024-04-25T14:15:16.021 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4192768 len 16384 2024-04-25T14:15:16.021 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4192768 len 16384 2024-04-25T14:15:16.021 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4192768 len 16384 2024-04-25T14:15:16.021 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4193280 len 1024 2024-04-25T14:15:16.021 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4193280 len 1024 2024-04-25T14:15:16.021 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4193280 len 1024 2024-04-25T14:15:16.021 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4193280 len 1024 2024-04-25T14:15:16.021 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4193280 len 2048 2024-04-25T14:15:16.021 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4193280 len 2048 2024-04-25T14:15:16.021 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4193280 len 2048 2024-04-25T14:15:16.601 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4193280 len 2048 2024-04-25T14:15:16.602 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4193280 len 4096 2024-04-25T14:15:16.602 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4193280 len 4096 2024-04-25T14:15:16.602 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4193280 len 4096 2024-04-25T14:15:16.602 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4193280 len 4096 2024-04-25T14:15:16.602 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4193280 len 8192 2024-04-25T14:15:16.602 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4193280 len 8192 2024-04-25T14:15:16.602 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4193280 len 8192 2024-04-25T14:15:16.602 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4193280 len 8192 2024-04-25T14:15:16.602 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4193280 len 16384 2024-04-25T14:15:16.602 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4193280 len 16384 2024-04-25T14:15:16.602 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4193280 len 16384 2024-04-25T14:15:16.602 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4193280 len 16384 2024-04-25T14:15:16.603 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4193792 len 1024 2024-04-25T14:15:16.603 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4193792 len 1024 2024-04-25T14:15:16.603 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4193792 len 1024 2024-04-25T14:15:16.603 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4193792 len 1024 2024-04-25T14:15:16.603 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4193792 len 2048 2024-04-25T14:15:16.603 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4193792 len 2048 2024-04-25T14:15:16.603 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4193792 len 2048 2024-04-25T14:15:16.603 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4193792 len 2048 2024-04-25T14:15:16.603 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4193792 len 4096 2024-04-25T14:15:16.603 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4193792 len 4096 2024-04-25T14:15:16.603 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4193792 len 4096 2024-04-25T14:15:16.603 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4193792 len 4096 2024-04-25T14:15:16.603 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4193792 len 8192 2024-04-25T14:15:16.603 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4193792 len 8192 2024-04-25T14:15:16.604 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4193792 len 8192 2024-04-25T14:15:16.604 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4193792 len 8192 2024-04-25T14:15:16.604 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4193792 len 16384 2024-04-25T14:15:16.604 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4193792 len 16384 2024-04-25T14:15:16.604 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4193792 len 16384 2024-04-25T14:15:16.604 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4193792 len 16384 2024-04-25T14:15:16.604 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4194304 len 1024 2024-04-25T14:15:16.604 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4194304 len 1024 2024-04-25T14:15:16.604 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4194304 len 1024 2024-04-25T14:15:16.604 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4194304 len 1024 2024-04-25T14:15:16.604 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4194304 len 2048 2024-04-25T14:15:16.604 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4194304 len 2048 2024-04-25T14:15:16.604 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4194304 len 2048 2024-04-25T14:15:16.604 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4194304 len 2048 2024-04-25T14:15:16.604 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4194304 len 4096 2024-04-25T14:15:16.605 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4194304 len 4096 2024-04-25T14:15:16.605 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4194304 len 4096 2024-04-25T14:15:16.605 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4194304 len 4096 2024-04-25T14:15:16.605 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4194304 len 8192 2024-04-25T14:15:16.605 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4194304 len 8192 2024-04-25T14:15:16.605 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4194304 len 8192 2024-04-25T14:15:16.605 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4194304 len 8192 2024-04-25T14:15:16.605 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4194304 len 16384 2024-04-25T14:15:16.605 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4194304 len 16384 2024-04-25T14:15:16.605 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4194304 len 16384 2024-04-25T14:15:16.605 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4194304 len 16384 2024-04-25T14:15:16.605 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4194816 len 1024 2024-04-25T14:15:16.605 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4194816 len 1024 2024-04-25T14:15:16.606 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4194816 len 1024 2024-04-25T14:15:16.606 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4194816 len 1024 2024-04-25T14:15:16.606 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4194816 len 2048 2024-04-25T14:15:16.606 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4194816 len 2048 2024-04-25T14:15:16.606 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4194816 len 2048 2024-04-25T14:15:16.606 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4194816 len 2048 2024-04-25T14:15:16.606 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4194816 len 4096 2024-04-25T14:15:16.606 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4194816 len 4096 2024-04-25T14:15:16.606 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4194816 len 4096 2024-04-25T14:15:16.606 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4194816 len 4096 2024-04-25T14:15:16.606 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4194816 len 8192 2024-04-25T14:15:16.606 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4194816 len 8192 2024-04-25T14:15:16.606 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4194816 len 8192 2024-04-25T14:15:16.606 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4194816 len 8192 2024-04-25T14:15:16.607 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4194816 len 16384 2024-04-25T14:15:16.607 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4194816 len 16384 2024-04-25T14:15:16.607 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4194816 len 16384 2024-04-25T14:15:16.607 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4194816 len 16384 2024-04-25T14:15:16.607 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4195328 len 1024 2024-04-25T14:15:16.607 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4195328 len 1024 2024-04-25T14:15:16.607 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4195328 len 1024 2024-04-25T14:15:16.607 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4195328 len 1024 2024-04-25T14:15:16.607 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4195328 len 2048 2024-04-25T14:15:16.607 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4195328 len 2048 2024-04-25T14:15:16.607 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4195328 len 2048 2024-04-25T14:15:16.607 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4195328 len 2048 2024-04-25T14:15:16.607 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4195328 len 4096 2024-04-25T14:15:16.607 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4195328 len 4096 2024-04-25T14:15:16.607 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4195328 len 4096 2024-04-25T14:15:16.608 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4195328 len 4096 2024-04-25T14:15:16.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:16 smithi096 ceph-mon[29207]: pgmap v395: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1.4 MiB/s wr, 1.97k op/s 2024-04-25T14:15:16.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:16 smithi097 ceph-mon[32720]: pgmap v395: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1.4 MiB/s wr, 1.97k op/s 2024-04-25T14:15:16.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:16 smithi151 ceph-mon[32990]: pgmap v395: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1.4 MiB/s wr, 1.97k op/s 2024-04-25T14:15:16.938 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:15:16.938 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:15:17.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.054+0000 7f6f53051640 1 -- 172.21.15.96:0/2516086584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f4c1033d0 msgr2=0x7f6f4c1037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:17.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.054+0000 7f6f53051640 1 --2- 172.21.15.96:0/2516086584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f4c1033d0 0x7f6f4c1037e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f6f38009900 tx=0x7f6f3802a410 comp rx=0 tx=0).stop 2024-04-25T14:15:17.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.054+0000 7f6f53051640 1 -- 172.21.15.96:0/2516086584 shutdown_connections 2024-04-25T14:15:17.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.054+0000 7f6f53051640 1 --2- 172.21.15.96:0/2516086584 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f4c1080d0 0x7f6f4c10a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.054+0000 7f6f53051640 1 --2- 172.21.15.96:0/2516086584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f4c1033d0 0x7f6f4c1037e0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.054+0000 7f6f53051640 1 --2- 172.21.15.96:0/2516086584 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6f4c102190 0x7f6f4c102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.054+0000 7f6f53051640 1 -- 172.21.15.96:0/2516086584 >> 172.21.15.96:0/2516086584 conn(0x7f6f4c0fd380 msgr2=0x7f6f4c0ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:17.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.054+0000 7f6f53051640 1 -- 172.21.15.96:0/2516086584 shutdown_connections 2024-04-25T14:15:17.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.054+0000 7f6f53051640 1 -- 172.21.15.96:0/2516086584 wait complete. 2024-04-25T14:15:17.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.055+0000 7f6f53051640 1 Processor -- start 2024-04-25T14:15:17.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.055+0000 7f6f53051640 1 -- start start 2024-04-25T14:15:17.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.055+0000 7f6f53051640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f4c102190 0x7f6f4c198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.055+0000 7f6f53051640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f4c1033d0 0x7f6f4c198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.055+0000 7f6f53051640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6f4c1080d0 0x7f6f4c19f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.055+0000 7f6f53051640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c10d420 con 0x7f6f4c102190 2024-04-25T14:15:17.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.055+0000 7f6f53051640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c10d2a0 con 0x7f6f4c1033d0 2024-04-25T14:15:17.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.055+0000 7f6f53051640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c10d5a0 con 0x7f6f4c1080d0 2024-04-25T14:15:17.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.055+0000 7f6f50ded640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f4c102190 0x7f6f4c198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.055+0000 7f6f50ded640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f4c102190 0x7f6f4c198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41354/0 (socket says 172.21.15.96:41354) 2024-04-25T14:15:17.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.055+0000 7f6f50ded640 1 -- 172.21.15.96:0/1470346154 learned_addr learned my addr 172.21.15.96:0/1470346154 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:17.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.055+0000 7f6f515ee640 1 --2- 172.21.15.96:0/1470346154 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6f4c1080d0 0x7f6f4c19f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.055+0000 7f6f4bfff640 1 --2- 172.21.15.96:0/1470346154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f4c1033d0 0x7f6f4c198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.056+0000 7f6f50ded640 1 -- 172.21.15.96:0/1470346154 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6f4c1080d0 msgr2=0x7f6f4c19f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:17.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.056+0000 7f6f50ded640 1 --2- 172.21.15.96:0/1470346154 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6f4c1080d0 0x7f6f4c19f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.056+0000 7f6f50ded640 1 -- 172.21.15.96:0/1470346154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f4c1033d0 msgr2=0x7f6f4c198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:17.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.056+0000 7f6f50ded640 1 --2- 172.21.15.96:0/1470346154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f4c1033d0 0x7f6f4c198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.056+0000 7f6f50ded640 1 -- 172.21.15.96:0/1470346154 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f4c19fb90 con 0x7f6f4c102190 2024-04-25T14:15:17.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.056+0000 7f6f50ded640 1 --2- 172.21.15.96:0/1470346154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f4c102190 0x7f6f4c198050 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f6f4000e9f0 tx=0x7f6f4000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:17.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.056+0000 7f6f49ffb640 1 -- 172.21.15.96:0/1470346154 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f4000ccc0 con 0x7f6f4c102190 2024-04-25T14:15:17.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.056+0000 7f6f49ffb640 1 -- 172.21.15.96:0/1470346154 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f400035f0 con 0x7f6f4c102190 2024-04-25T14:15:17.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.056+0000 7f6f53051640 1 -- 172.21.15.96:0/1470346154 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f4c19fe80 con 0x7f6f4c102190 2024-04-25T14:15:17.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.056+0000 7f6f49ffb640 1 -- 172.21.15.96:0/1470346154 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f4000ce60 con 0x7f6f4c102190 2024-04-25T14:15:17.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.056+0000 7f6f53051640 1 -- 172.21.15.96:0/1470346154 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f4c1a02e0 con 0x7f6f4c102190 2024-04-25T14:15:17.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.058+0000 7f6f49ffb640 1 -- 172.21.15.96:0/1470346154 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f6f40003aa0 con 0x7f6f4c102190 2024-04-25T14:15:17.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.059+0000 7f6f59fe7740 1 Processor -- start 2024-04-25T14:15:17.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.059+0000 7f6f59fe7740 1 -- start start 2024-04-25T14:15:17.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.059+0000 7f6f59fe7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e36cf60390 0x55e36d2fe7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.059+0000 7f6f59fe7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e36cf374d0 0x55e36d2fccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.059+0000 7f6f59fe7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e36cf3f6b0 0x55e36d2fd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.059+0000 7f6f59fe7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e36d266d80 con 0x55e36cf60390 2024-04-25T14:15:17.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.059+0000 7f6f59fe7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e36d2208b0 con 0x55e36cf3f6b0 2024-04-25T14:15:17.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.059+0000 7f6f59fe7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e36d262d40 con 0x55e36cf374d0 2024-04-25T14:15:17.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.059+0000 7f6f515ee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e36cf60390 0x55e36d2fe7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.059+0000 7f6f515ee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e36cf60390 0x55e36d2fe7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41356/0 (socket says 172.21.15.96:41356) 2024-04-25T14:15:17.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.059+0000 7f6f515ee640 1 -- 172.21.15.96:0/2762747993 learned_addr learned my addr 172.21.15.96:0/2762747993 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:17.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.059+0000 7f6f4bfff640 1 --2- 172.21.15.96:0/2762747993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e36cf3f6b0 0x55e36d2fd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.059+0000 7f6f49ffb640 1 --2- 172.21.15.96:0/1470346154 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f6f3c0afaf0 0x7f6f3c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.059+0000 7f6f50ded640 1 --2- 172.21.15.96:0/2762747993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e36cf374d0 0x55e36d2fccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.059+0000 7f6f515ee640 1 --2- 172.21.15.96:0/1470346154 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f6f3c0afaf0 0x7f6f3c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.059+0000 7f6f4bfff640 1 -- 172.21.15.96:0/2762747993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e36cf374d0 msgr2=0x55e36d2fccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:17.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.059+0000 7f6f4bfff640 1 --2- 172.21.15.96:0/2762747993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e36cf374d0 0x55e36d2fccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.059+0000 7f6f4bfff640 1 -- 172.21.15.96:0/2762747993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e36cf60390 msgr2=0x55e36d2fe7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:17.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.059+0000 7f6f4bfff640 1 --2- 172.21.15.96:0/2762747993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e36cf60390 0x55e36d2fe7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.059+0000 7f6f4bfff640 1 -- 172.21.15.96:0/2762747993 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e36d1f1a40 con 0x55e36cf3f6b0 2024-04-25T14:15:17.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.059+0000 7f6f49ffb640 1 -- 172.21.15.96:0/1470346154 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f6f400def70 con 0x7f6f4c102190 2024-04-25T14:15:17.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.060+0000 7f6f4bfff640 1 --2- 172.21.15.96:0/2762747993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e36cf3f6b0 0x55e36d2fd410 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6f38005ab0 tx=0x7f6f38009600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34f716bc400fc19e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:17.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.060+0000 7f6f515ee640 1 --2- 172.21.15.96:0/1470346154 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f6f3c0afaf0 0x7f6f3c0b2000 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6f44005c40 tx=0x7f6f44004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:17.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.060+0000 7f6f36ffd640 1 -- 172.21.15.96:0/2762747993 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f38035030 con 0x55e36cf3f6b0 2024-04-25T14:15:17.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.060+0000 7f6f36ffd640 1 -- 172.21.15.96:0/2762747993 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f38040070 con 0x55e36cf3f6b0 2024-04-25T14:15:17.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.060+0000 7f6f36ffd640 1 -- 172.21.15.96:0/2762747993 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f38003850 con 0x55e36cf3f6b0 2024-04-25T14:15:17.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.061+0000 7f6f59fe7740 1 -- 172.21.15.96:0/2762747993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e36cf3f6b0 msgr2=0x55e36d2fd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:17.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.061+0000 7f6f59fe7740 1 --2- 172.21.15.96:0/2762747993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e36cf3f6b0 0x55e36d2fd410 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6f38005ab0 tx=0x7f6f38009600 comp rx=0 tx=0).stop 2024-04-25T14:15:17.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.061+0000 7f6f59fe7740 1 -- 172.21.15.96:0/2762747993 shutdown_connections 2024-04-25T14:15:17.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.061+0000 7f6f59fe7740 1 --2- 172.21.15.96:0/2762747993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e36cf3f6b0 0x55e36d2fd410 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.061+0000 7f6f59fe7740 1 --2- 172.21.15.96:0/2762747993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e36cf374d0 0x55e36d2fccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.061+0000 7f6f59fe7740 1 --2- 172.21.15.96:0/2762747993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e36cf60390 0x55e36d2fe7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.061+0000 7f6f59fe7740 1 -- 172.21.15.96:0/2762747993 >> 172.21.15.96:0/2762747993 conn(0x55e36cf544e0 msgr2=0x55e36d2ff480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:17.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.061+0000 7f6f59fe7740 1 -- 172.21.15.96:0/2762747993 shutdown_connections 2024-04-25T14:15:17.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.061+0000 7f6f59fe7740 1 -- 172.21.15.96:0/2762747993 wait complete. 2024-04-25T14:15:17.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.061+0000 7f6f59fe7740 1 -- start start 2024-04-25T14:15:17.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.061+0000 7f6f59fe7740 1 Processor -- start 2024-04-25T14:15:17.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.062+0000 7f6f59fe7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e36cf374d0 0x55e36d30fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.062+0000 7f6f59fe7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e36cf3f6b0 0x55e36d3101e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.062+0000 7f6f59fe7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e36cf60390 0x55e36d317140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.062+0000 7f6f59fe7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e36d30dd80 con 0x55e36cf60390 2024-04-25T14:15:17.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.062+0000 7f6f59fe7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e36d1edd10 con 0x55e36cf3f6b0 2024-04-25T14:15:17.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.062+0000 7f6f59fe7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e36d30df00 con 0x55e36cf374d0 2024-04-25T14:15:17.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.062+0000 7f6f59fe7740 10 client.0 _ll_register_callbacks cb 0x55e36d21a210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:15:17.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.062+0000 7f6f4bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e36cf3f6b0 0x55e36d3101e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.062+0000 7f6f50ded640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e36cf374d0 0x55e36d30fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.062+0000 7f6f50ded640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e36cf374d0 0x55e36d30fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.96:50910/0 (socket says 172.21.15.96:50910) 2024-04-25T14:15:17.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.062+0000 7f6f50ded640 1 -- 172.21.15.96:0/1197395311 learned_addr learned my addr 172.21.15.96:0/1197395311 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:17.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.062+0000 7f6f4bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e36cf3f6b0 0x55e36d3101e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:41760/0 (socket says 172.21.15.96:41760) 2024-04-25T14:15:17.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.062+0000 7f6f515ee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e36cf60390 0x55e36d317140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.062+0000 7f6f4bfff640 1 -- 172.21.15.96:0/1197395311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e36cf374d0 msgr2=0x55e36d30fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:17.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.062+0000 7f6f4bfff640 1 --2- 172.21.15.96:0/1197395311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e36cf374d0 0x55e36d30fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.062+0000 7f6f4bfff640 1 -- 172.21.15.96:0/1197395311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e36cf60390 msgr2=0x55e36d317140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:17.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.062+0000 7f6f4bfff640 1 --2- 172.21.15.96:0/1197395311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e36cf60390 0x55e36d317140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.062+0000 7f6f4bfff640 1 -- 172.21.15.96:0/1197395311 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e36d0427d0 con 0x55e36cf3f6b0 2024-04-25T14:15:17.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.062+0000 7f6f50ded640 1 --2- 172.21.15.96:0/1197395311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e36cf374d0 0x55e36d30fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:15:17.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.063+0000 7f6f4bfff640 1 --2- 172.21.15.96:0/1197395311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e36cf3f6b0 0x55e36d3101e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6f38006010 tx=0x7f6f38031f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a16d82f8b3afb81b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:17.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.063+0000 7f6f2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T14:15:17.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.063+0000 7f6f2bfff640 1 -- 172.21.15.96:0/1197395311 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f38035040 con 0x55e36cf3f6b0 2024-04-25T14:15:17.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.063+0000 7f6f2bfff640 1 -- 172.21.15.96:0/1197395311 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f6f380315a0 con 0x55e36cf3f6b0 2024-04-25T14:15:17.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.063+0000 7f6f59fe7740 1 -- 172.21.15.96:0/1197395311 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f38040070 con 0x55e36cf3f6b0 2024-04-25T14:15:17.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.063+0000 7f6f2bfff640 1 -- 172.21.15.96:0/1197395311 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6f380317a0 con 0x55e36cf3f6b0 2024-04-25T14:15:17.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.063+0000 7f6f59fe7740 10 client.25562 fetch_fsmap learned FSMap version 26 2024-04-25T14:15:17.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.063+0000 7f6f59fe7740 1 -- 172.21.15.96:0/1197395311 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e36cf783f0 con 0x55e36cf3f6b0 2024-04-25T14:15:17.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.063+0000 7f6f2bfff640 1 -- 172.21.15.96:0/1197395311 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f38040070 con 0x55e36cf3f6b0 2024-04-25T14:15:17.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.063+0000 7f6f2bfff640 1 -- 172.21.15.96:0/1197395311 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6f38045470 con 0x55e36cf3f6b0 2024-04-25T14:15:17.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.063+0000 7f6f59fe7740 10 client.25562 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:15:17.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.063+0000 7f6f59fe7740 10 client.25562 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:15:17.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.063+0000 7f6f59fe7740 20 client.25562 populate_metadata read hostname 'smithi096' 2024-04-25T14:15:17.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.063+0000 7f6f59fe7740 1 --2- 172.21.15.96:0/1197395311 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55e36cf1ef60 0x55e36d31d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.063+0000 7f6f59fe7740 4 client.25562 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:15:17.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.063+0000 7f6f59fe7740 1 -- 172.21.15.96:0/1197395311 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e36d274990 con 0x55e36cf1ef60 2024-04-25T14:15:17.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.064+0000 7f6f50ded640 1 --2- 172.21.15.96:0/1197395311 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55e36cf1ef60 0x55e36d31d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.064+0000 7f6f2bfff640 10 client.25562 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:15:17.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.064+0000 7f6f50ded640 1 --2- 172.21.15.96:0/1197395311 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55e36cf1ef60 0x55e36d31d200 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39717005271df050 server_cookie=adb6d837c781a5eb in_seq=0 out_seq=0 2024-04-25T14:15:17.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.065+0000 7f6f2bfff640 1 -- 172.21.15.96:0/1197395311 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e36d274990 con 0x55e36cf1ef60 2024-04-25T14:15:17.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.065+0000 7f6f2bfff640 10 client.25562 handle_command_reply: tid=0 2024-04-25T14:15:17.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.065+0000 7f6f59fe7740 1 client.25562 shutdown 2024-04-25T14:15:17.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.065+0000 7f6f59fe7740 1 -- 172.21.15.96:0/1197395311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e36cf3f6b0 msgr2=0x55e36d3101e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:17.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.065+0000 7f6f59fe7740 1 --2- 172.21.15.96:0/1197395311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e36cf3f6b0 0x55e36d3101e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6f38006010 tx=0x7f6f38031f00 comp rx=0 tx=0).stop 2024-04-25T14:15:17.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.065+0000 7f6f59fe7740 1 -- 172.21.15.96:0/1197395311 shutdown_connections 2024-04-25T14:15:17.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.065+0000 7f6f59fe7740 1 --2- 172.21.15.96:0/1197395311 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55e36cf1ef60 0x55e36d31d200 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.065+0000 7f6f59fe7740 1 --2- 172.21.15.96:0/1197395311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e36cf60390 0x55e36d317140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.065+0000 7f6f59fe7740 1 --2- 172.21.15.96:0/1197395311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e36cf3f6b0 0x55e36d3101e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.065+0000 7f6f2bfff640 0 client.25562 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:15:17.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.065+0000 7f6f59fe7740 1 --2- 172.21.15.96:0/1197395311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e36cf374d0 0x55e36d30fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.065+0000 7f6f59fe7740 1 -- 172.21.15.96:0/1197395311 >> 172.21.15.96:0/1197395311 conn(0x55e36cf544e0 msgr2=0x55e36d2fc3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:17.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.065+0000 7f6f59fe7740 1 -- 172.21.15.96:0/1197395311 shutdown_connections 2024-04-25T14:15:17.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.065+0000 7f6f59fe7740 1 -- 172.21.15.96:0/1197395311 wait complete. 2024-04-25T14:15:17.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.065+0000 7f6f59fe7740 20 client.25562 trim_cache size 0 max 16384 2024-04-25T14:15:17.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.083+0000 7f6f59fe7740 1 Processor -- start 2024-04-25T14:15:17.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.083+0000 7f6f59fe7740 1 -- start start 2024-04-25T14:15:17.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.083+0000 7f6f59fe7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e36cf374d0 0x55e36d30e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.083+0000 7f6f59fe7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e36cf3f6b0 0x55e36d317060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.083+0000 7f6f59fe7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e36cf60390 0x55e36d3175a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.083+0000 7f6f59fe7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c1a9160 con 0x55e36cf3f6b0 2024-04-25T14:15:17.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.083+0000 7f6f59fe7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c1aaca0 con 0x55e36cf60390 2024-04-25T14:15:17.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.083+0000 7f6f59fe7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c1a9830 con 0x55e36cf374d0 2024-04-25T14:15:17.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.084+0000 7f6f4bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e36cf3f6b0 0x55e36d317060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.084+0000 7f6f50ded640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e36cf374d0 0x55e36d30e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.084+0000 7f6f4bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e36cf3f6b0 0x55e36d317060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41386/0 (socket says 172.21.15.96:41386) 2024-04-25T14:15:17.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.084+0000 7f6f515ee640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e36cf60390 0x55e36d3175a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.084+0000 7f6f4bfff640 1 -- 172.21.15.96:0/4262648542 learned_addr learned my addr 172.21.15.96:0/4262648542 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:17.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.084+0000 7f6f4bfff640 1 -- 172.21.15.96:0/4262648542 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e36cf374d0 msgr2=0x55e36d30e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:17.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.084+0000 7f6f4bfff640 1 --2- 172.21.15.96:0/4262648542 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e36cf374d0 0x55e36d30e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.084+0000 7f6f4bfff640 1 -- 172.21.15.96:0/4262648542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e36cf60390 msgr2=0x55e36d3175a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:17.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.084+0000 7f6f4bfff640 1 --2- 172.21.15.96:0/4262648542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e36cf60390 0x55e36d3175a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.084+0000 7f6f4bfff640 1 -- 172.21.15.96:0/4262648542 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e36cf380f0 con 0x55e36cf3f6b0 2024-04-25T14:15:17.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.084+0000 7f6f50ded640 1 --2- 172.21.15.96:0/4262648542 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e36cf374d0 0x55e36d30e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:15:17.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.084+0000 7f6f4bfff640 1 --2- 172.21.15.96:0/4262648542 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e36cf3f6b0 0x55e36d317060 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f6f38000c00 tx=0x7f6f380095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e421016fd54a34de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:17.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.084+0000 7f6f367fc640 1 -- 172.21.15.96:0/4262648542 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3805e070 con 0x55e36cf3f6b0 2024-04-25T14:15:17.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.084+0000 7f6f367fc640 1 -- 172.21.15.96:0/4262648542 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f38040030 con 0x55e36cf3f6b0 2024-04-25T14:15:17.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.084+0000 7f6f367fc640 1 -- 172.21.15.96:0/4262648542 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f38035040 con 0x55e36cf3f6b0 2024-04-25T14:15:17.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.085+0000 7f6f59fe7740 1 -- 172.21.15.96:0/4262648542 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e36cf3f6b0 msgr2=0x55e36d317060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:17.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.085+0000 7f6f59fe7740 1 --2- 172.21.15.96:0/4262648542 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e36cf3f6b0 0x55e36d317060 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f6f38000c00 tx=0x7f6f380095b0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.085+0000 7f6f59fe7740 1 -- 172.21.15.96:0/4262648542 shutdown_connections 2024-04-25T14:15:17.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.085+0000 7f6f59fe7740 1 --2- 172.21.15.96:0/4262648542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e36cf60390 0x55e36d3175a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.085+0000 7f6f59fe7740 1 --2- 172.21.15.96:0/4262648542 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e36cf3f6b0 0x55e36d317060 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.085+0000 7f6f59fe7740 1 --2- 172.21.15.96:0/4262648542 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e36cf374d0 0x55e36d30e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.085+0000 7f6f59fe7740 1 -- 172.21.15.96:0/4262648542 >> 172.21.15.96:0/4262648542 conn(0x55e36cf544e0 msgr2=0x55e36d302cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:17.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.085+0000 7f6f59fe7740 1 -- 172.21.15.96:0/4262648542 shutdown_connections 2024-04-25T14:15:17.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.085+0000 7f6f59fe7740 1 -- 172.21.15.96:0/4262648542 wait complete. 2024-04-25T14:15:17.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.085+0000 7f6f59fe7740 1 -- start start 2024-04-25T14:15:17.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.086+0000 7f6f59fe7740 1 Processor -- start 2024-04-25T14:15:17.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.086+0000 7f6f59fe7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e36cf374d0 0x55e36d314780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.086+0000 7f6f59fe7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e36cf3f6b0 0x55e36d314cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.086+0000 7f6f59fe7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e36cf60390 0x55e36d383430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.086+0000 7f6f59fe7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c1aa5d0 con 0x55e36cf374d0 2024-04-25T14:15:17.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.086+0000 7f6f59fe7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c1a9f00 con 0x55e36cf60390 2024-04-25T14:15:17.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.086+0000 7f6f59fe7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c1a7140 con 0x55e36cf3f6b0 2024-04-25T14:15:17.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.086+0000 7f6f59fe7740 10 client.0 _ll_register_callbacks cb 0x7f6f4400a250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:15:17.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.086+0000 7f6f50ded640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e36cf374d0 0x55e36d314780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.086+0000 7f6f4bfff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e36cf3f6b0 0x55e36d314cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.086+0000 7f6f50ded640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e36cf374d0 0x55e36d314780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41398/0 (socket says 172.21.15.96:41398) 2024-04-25T14:15:17.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.086+0000 7f6f50ded640 1 -- 172.21.15.96:0/4112076348 learned_addr learned my addr 172.21.15.96:0/4112076348 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:17.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.086+0000 7f6f515ee640 1 --2- 172.21.15.96:0/4112076348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e36cf60390 0x55e36d383430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.086+0000 7f6f4bfff640 1 -- 172.21.15.96:0/4112076348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e36cf60390 msgr2=0x55e36d383430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:17.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.086+0000 7f6f4bfff640 1 --2- 172.21.15.96:0/4112076348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e36cf60390 0x55e36d383430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.086+0000 7f6f4bfff640 1 -- 172.21.15.96:0/4112076348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e36cf374d0 msgr2=0x55e36d314780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:17.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.086+0000 7f6f4bfff640 1 --2- 172.21.15.96:0/4112076348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e36cf374d0 0x55e36d314780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.086+0000 7f6f4bfff640 1 -- 172.21.15.96:0/4112076348 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e36d2274b0 con 0x55e36cf3f6b0 2024-04-25T14:15:17.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.086+0000 7f6f50ded640 1 --2- 172.21.15.96:0/4112076348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e36cf374d0 0x55e36d314780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:15:17.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.087+0000 7f6f4bfff640 1 --2- 172.21.15.96:0/4112076348 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e36cf3f6b0 0x55e36d314cc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6f38006010 tx=0x7f6f38045d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bb83531c678548a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:17.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.087+0000 7f6f2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-25T14:15:17.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.087+0000 7f6f2bfff640 1 -- 172.21.15.96:0/4112076348 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f38035070 con 0x55e36cf3f6b0 2024-04-25T14:15:17.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.087+0000 7f6f2bfff640 1 -- 172.21.15.96:0/4112076348 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f6f380315a0 con 0x55e36cf3f6b0 2024-04-25T14:15:17.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.087+0000 7f6f59fe7740 1 -- 172.21.15.96:0/4112076348 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f38040030 con 0x55e36cf3f6b0 2024-04-25T14:15:17.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.087+0000 7f6f2bfff640 1 -- 172.21.15.96:0/4112076348 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6f3805e040 con 0x55e36cf3f6b0 2024-04-25T14:15:17.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.087+0000 7f6f59fe7740 10 client.25507 fetch_fsmap learned FSMap version 26 2024-04-25T14:15:17.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.087+0000 7f6f59fe7740 1 -- 172.21.15.96:0/4112076348 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e36d384780 con 0x55e36cf3f6b0 2024-04-25T14:15:17.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.087+0000 7f6f2bfff640 1 -- 172.21.15.96:0/4112076348 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f38040030 con 0x55e36cf3f6b0 2024-04-25T14:15:17.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.087+0000 7f6f2bfff640 1 -- 172.21.15.96:0/4112076348 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6f38061030 con 0x55e36cf3f6b0 2024-04-25T14:15:17.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.087+0000 7f6f59fe7740 10 client.25507 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:15:17.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.087+0000 7f6f59fe7740 10 client.25507 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:15:17.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.087+0000 7f6f59fe7740 20 client.25507 populate_metadata read hostname 'smithi096' 2024-04-25T14:15:17.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.087+0000 7f6f59fe7740 1 --2- 172.21.15.96:0/4112076348 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55e36cef43a0 0x55e36d386ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.088+0000 7f6f59fe7740 4 client.25507 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:15:17.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.088+0000 7f6f59fe7740 1 -- 172.21.15.96:0/4112076348 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e36d26e800 con 0x55e36cef43a0 2024-04-25T14:15:17.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.088+0000 7f6f50ded640 1 --2- 172.21.15.96:0/4112076348 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55e36cef43a0 0x55e36d386ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.088+0000 7f6f2bfff640 10 client.25507 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:15:17.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.088+0000 7f6f50ded640 1 --2- 172.21.15.96:0/4112076348 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55e36cef43a0 0x55e36d386ea0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8725d07f6aa46a1 server_cookie=3f102c99017ff336 in_seq=0 out_seq=0 2024-04-25T14:15:17.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.088+0000 7f6f2bfff640 1 -- 172.21.15.96:0/4112076348 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e36d26e800 con 0x55e36cef43a0 2024-04-25T14:15:17.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.088+0000 7f6f2bfff640 10 client.25507 handle_command_reply: tid=0 2024-04-25T14:15:17.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.088+0000 7f6f59fe7740 1 client.25507 shutdown 2024-04-25T14:15:17.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.088+0000 7f6f59fe7740 1 -- 172.21.15.96:0/4112076348 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e36cf3f6b0 msgr2=0x55e36d314cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:17.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.088+0000 7f6f59fe7740 1 --2- 172.21.15.96:0/4112076348 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e36cf3f6b0 0x55e36d314cc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6f38006010 tx=0x7f6f38045d10 comp rx=0 tx=0).stop 2024-04-25T14:15:17.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.088+0000 7f6f59fe7740 1 -- 172.21.15.96:0/4112076348 shutdown_connections 2024-04-25T14:15:17.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.088+0000 7f6f59fe7740 1 --2- 172.21.15.96:0/4112076348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e36cf60390 0x55e36d383430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.088+0000 7f6f59fe7740 1 --2- 172.21.15.96:0/4112076348 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e36cf3f6b0 0x55e36d314cc0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.088+0000 7f6f59fe7740 1 --2- 172.21.15.96:0/4112076348 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55e36cef43a0 0x55e36d386ea0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.088+0000 7f6f59fe7740 1 --2- 172.21.15.96:0/4112076348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e36cf374d0 0x55e36d314780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.088+0000 7f6f2bfff640 0 client.25507 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:15:17.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.088+0000 7f6f59fe7740 1 -- 172.21.15.96:0/4112076348 >> 172.21.15.96:0/4112076348 conn(0x55e36cf544e0 msgr2=0x55e36d302cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:17.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.089+0000 7f6f59fe7740 1 -- 172.21.15.96:0/4112076348 shutdown_connections 2024-04-25T14:15:17.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.089+0000 7f6f59fe7740 1 -- 172.21.15.96:0/4112076348 wait complete. 2024-04-25T14:15:17.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.089+0000 7f6f59fe7740 20 client.25507 trim_cache size 0 max 16384 2024-04-25T14:15:17.094 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T14:15:17.094 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "no active scrubs running", 2024-04-25T14:15:17.094 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": {} 2024-04-25T14:15:17.095 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T14:15:17.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.089+0000 7f6f53051640 1 -- 172.21.15.96:0/1470346154 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f6f3c0afaf0 msgr2=0x7f6f3c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:17.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.089+0000 7f6f53051640 1 --2- 172.21.15.96:0/1470346154 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f6f3c0afaf0 0x7f6f3c0b2000 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6f44005c40 tx=0x7f6f44004ab0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.089+0000 7f6f53051640 1 -- 172.21.15.96:0/1470346154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f4c102190 msgr2=0x7f6f4c198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:17.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.089+0000 7f6f53051640 1 --2- 172.21.15.96:0/1470346154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f4c102190 0x7f6f4c198050 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f6f4000e9f0 tx=0x7f6f4000eeb0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.089+0000 7f6f53051640 1 -- 172.21.15.96:0/1470346154 shutdown_connections 2024-04-25T14:15:17.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.089+0000 7f6f53051640 1 --2- 172.21.15.96:0/1470346154 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6f4c1080d0 0x7f6f4c19f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.089+0000 7f6f53051640 1 --2- 172.21.15.96:0/1470346154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f4c1033d0 0x7f6f4c198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.089+0000 7f6f53051640 1 --2- 172.21.15.96:0/1470346154 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f6f3c0afaf0 0x7f6f3c0b2000 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.089+0000 7f6f53051640 1 --2- 172.21.15.96:0/1470346154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f4c102190 0x7f6f4c198050 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.089+0000 7f6f53051640 1 -- 172.21.15.96:0/1470346154 >> 172.21.15.96:0/1470346154 conn(0x7f6f4c0fd380 msgr2=0x7f6f4c1040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:17.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.089+0000 7f6f53051640 1 -- 172.21.15.96:0/1470346154 shutdown_connections 2024-04-25T14:15:17.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.089+0000 7f6f53051640 1 -- 172.21.15.96:0/1470346154 wait complete. 2024-04-25T14:15:17.096 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4195328 len 8192 2024-04-25T14:15:17.096 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4195328 len 8192 2024-04-25T14:15:17.097 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4195328 len 8192 2024-04-25T14:15:17.097 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4195328 len 8192 2024-04-25T14:15:17.097 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4195328 len 16384 2024-04-25T14:15:17.097 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4195328 len 16384 2024-04-25T14:15:17.097 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4195328 len 16384 2024-04-25T14:15:17.097 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4195328 len 16384 2024-04-25T14:15:17.097 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4195840 len 1024 2024-04-25T14:15:17.097 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4195840 len 1024 2024-04-25T14:15:17.097 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4195840 len 1024 2024-04-25T14:15:17.097 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4195840 len 1024 2024-04-25T14:15:17.097 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4195840 len 2048 2024-04-25T14:15:17.097 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4195840 len 2048 2024-04-25T14:15:17.097 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4195840 len 2048 2024-04-25T14:15:17.097 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4195840 len 2048 2024-04-25T14:15:17.097 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4195840 len 4096 2024-04-25T14:15:17.098 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4195840 len 4096 2024-04-25T14:15:17.098 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4195840 len 4096 2024-04-25T14:15:17.098 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4195840 len 4096 2024-04-25T14:15:17.098 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4195840 len 8192 2024-04-25T14:15:17.098 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4195840 len 8192 2024-04-25T14:15:17.098 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4195840 len 8192 2024-04-25T14:15:17.098 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4195840 len 8192 2024-04-25T14:15:17.098 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4195840 len 16384 2024-04-25T14:15:17.098 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4195840 len 16384 2024-04-25T14:15:17.098 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4195840 len 16384 2024-04-25T14:15:17.098 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4195840 len 16384 2024-04-25T14:15:17.098 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4196352 len 1024 2024-04-25T14:15:17.098 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4196352 len 1024 2024-04-25T14:15:17.099 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4196352 len 1024 2024-04-25T14:15:17.099 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4196352 len 1024 2024-04-25T14:15:17.099 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4196352 len 2048 2024-04-25T14:15:17.099 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4196352 len 2048 2024-04-25T14:15:17.099 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4196352 len 2048 2024-04-25T14:15:17.099 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4196352 len 2048 2024-04-25T14:15:17.099 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4196352 len 4096 2024-04-25T14:15:17.099 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4196352 len 4096 2024-04-25T14:15:17.099 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4196352 len 4096 2024-04-25T14:15:17.099 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4196352 len 4096 2024-04-25T14:15:17.099 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4196352 len 8192 2024-04-25T14:15:17.099 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4196352 len 8192 2024-04-25T14:15:17.099 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4196352 len 8192 2024-04-25T14:15:17.099 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4196352 len 8192 2024-04-25T14:15:17.099 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4196352 len 16384 2024-04-25T14:15:17.099 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4196352 len 16384 2024-04-25T14:15:17.100 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4196352 len 16384 2024-04-25T14:15:17.100 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4196352 len 16384 2024-04-25T14:15:17.100 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4196864 len 1024 2024-04-25T14:15:17.100 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4196864 len 1024 2024-04-25T14:15:17.100 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4196864 len 1024 2024-04-25T14:15:17.100 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4196864 len 1024 2024-04-25T14:15:17.100 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4196864 len 2048 2024-04-25T14:15:17.100 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4196864 len 2048 2024-04-25T14:15:17.100 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4196864 len 2048 2024-04-25T14:15:17.100 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4196864 len 2048 2024-04-25T14:15:17.100 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4196864 len 4096 2024-04-25T14:15:17.101 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4196864 len 4096 2024-04-25T14:15:17.101 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4196864 len 4096 2024-04-25T14:15:17.101 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4196864 len 4096 2024-04-25T14:15:17.101 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4196864 len 8192 2024-04-25T14:15:17.101 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4196864 len 8192 2024-04-25T14:15:17.101 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4196864 len 8192 2024-04-25T14:15:17.101 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4196864 len 8192 2024-04-25T14:15:17.101 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4196864 len 16384 2024-04-25T14:15:17.101 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4196864 len 16384 2024-04-25T14:15:17.101 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4196864 len 16384 2024-04-25T14:15:17.101 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4196864 len 16384 2024-04-25T14:15:17.101 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4197376 len 1024 2024-04-25T14:15:17.101 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4197376 len 1024 2024-04-25T14:15:17.101 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4197376 len 1024 2024-04-25T14:15:17.101 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4197376 len 1024 2024-04-25T14:15:17.102 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4197376 len 2048 2024-04-25T14:15:17.102 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4197376 len 2048 2024-04-25T14:15:17.102 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4197376 len 2048 2024-04-25T14:15:17.102 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4197376 len 2048 2024-04-25T14:15:17.102 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4197376 len 4096 2024-04-25T14:15:17.102 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4197376 len 4096 2024-04-25T14:15:17.102 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4197376 len 4096 2024-04-25T14:15:17.102 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4197376 len 4096 2024-04-25T14:15:17.102 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4197376 len 8192 2024-04-25T14:15:17.102 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4197376 len 8192 2024-04-25T14:15:17.102 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4197376 len 8192 2024-04-25T14:15:17.102 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4197376 len 8192 2024-04-25T14:15:17.102 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4197376 len 16384 2024-04-25T14:15:17.102 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4197376 len 16384 2024-04-25T14:15:17.104 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T14:15:17.104 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:15:17.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.216+0000 7f815255e640 1 -- 172.21.15.96:0/383583450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f814c104d10 msgr2=0x7f814c105120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:17.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.216+0000 7f815255e640 1 --2- 172.21.15.96:0/383583450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f814c104d10 0x7f814c105120 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f8140009900 tx=0x7f814002a410 comp rx=0 tx=0).stop 2024-04-25T14:15:17.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.216+0000 7f815255e640 1 -- 172.21.15.96:0/383583450 shutdown_connections 2024-04-25T14:15:17.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.216+0000 7f815255e640 1 --2- 172.21.15.96:0/383583450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f814c109a10 0x7f814c10be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.216+0000 7f815255e640 1 --2- 172.21.15.96:0/383583450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f814c104d10 0x7f814c105120 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.216+0000 7f815255e640 1 --2- 172.21.15.96:0/383583450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f814c103ad0 0x7f814c1047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.216+0000 7f815255e640 1 -- 172.21.15.96:0/383583450 >> 172.21.15.96:0/383583450 conn(0x7f814c0fecc0 msgr2=0x7f814c101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:17.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.216+0000 7f815255e640 1 -- 172.21.15.96:0/383583450 shutdown_connections 2024-04-25T14:15:17.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.216+0000 7f815255e640 1 -- 172.21.15.96:0/383583450 wait complete. 2024-04-25T14:15:17.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.216+0000 7f815255e640 1 Processor -- start 2024-04-25T14:15:17.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.216+0000 7f815255e640 1 -- start start 2024-04-25T14:15:17.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.217+0000 7f815255e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f814c103ad0 0x7f814c19a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.217+0000 7f815255e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f814c104d10 0x7f814c19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.217+0000 7f815255e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f814c109a10 0x7f814c1a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.217+0000 7f815255e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f814c10f600 con 0x7f814c109a10 2024-04-25T14:15:17.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.217+0000 7f815255e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f814c10f480 con 0x7f814c104d10 2024-04-25T14:15:17.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.217+0000 7f815255e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f814c10f780 con 0x7f814c103ad0 2024-04-25T14:15:17.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.217+0000 7f8150afb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f814c109a10 0x7f814c1a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.217+0000 7f8150afb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f814c109a10 0x7f814c1a1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41420/0 (socket says 172.21.15.96:41420) 2024-04-25T14:15:17.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.217+0000 7f8150afb640 1 -- 172.21.15.96:0/3324102411 learned_addr learned my addr 172.21.15.96:0/3324102411 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:17.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.217+0000 7f814b7fe640 1 --2- 172.21.15.96:0/3324102411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f814c104d10 0x7f814c19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.217+0000 7f814bfff640 1 --2- 172.21.15.96:0/3324102411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f814c103ad0 0x7f814c19a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.217+0000 7f8150afb640 1 -- 172.21.15.96:0/3324102411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f814c103ad0 msgr2=0x7f814c19a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:17.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.217+0000 7f8150afb640 1 --2- 172.21.15.96:0/3324102411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f814c103ad0 0x7f814c19a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.217+0000 7f8150afb640 1 -- 172.21.15.96:0/3324102411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f814c104d10 msgr2=0x7f814c19a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:17.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.217+0000 7f8150afb640 1 --2- 172.21.15.96:0/3324102411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f814c104d10 0x7f814c19a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.217+0000 7f8150afb640 1 -- 172.21.15.96:0/3324102411 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f814c1a1d00 con 0x7f814c109a10 2024-04-25T14:15:17.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.217+0000 7f8150afb640 1 --2- 172.21.15.96:0/3324102411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f814c109a10 0x7f814c1a1690 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f814400eb10 tx=0x7f814400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:17.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.217+0000 7f81497fa640 1 -- 172.21.15.96:0/3324102411 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8144004b20 con 0x7f814c109a10 2024-04-25T14:15:17.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.218+0000 7f81497fa640 1 -- 172.21.15.96:0/3324102411 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8144004cc0 con 0x7f814c109a10 2024-04-25T14:15:17.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.218+0000 7f815255e640 1 -- 172.21.15.96:0/3324102411 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f814c1a1ff0 con 0x7f814c109a10 2024-04-25T14:15:17.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.218+0000 7f81497fa640 1 -- 172.21.15.96:0/3324102411 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8144010670 con 0x7f814c109a10 2024-04-25T14:15:17.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.218+0000 7f815255e640 1 -- 172.21.15.96:0/3324102411 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f814c1a2450 con 0x7f814c109a10 2024-04-25T14:15:17.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.219+0000 7f815255e640 1 -- 172.21.15.96:0/3324102411 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f814c06afa0 con 0x7f814c109a10 2024-04-25T14:15:17.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.220+0000 7f81497fa640 1 -- 172.21.15.96:0/3324102411 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f8144005000 con 0x7f814c109a10 2024-04-25T14:15:17.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.221+0000 7f81497fa640 1 --2- 172.21.15.96:0/3324102411 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f81340afaf0 0x7f81340b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.221+0000 7f814bfff640 1 --2- 172.21.15.96:0/3324102411 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f81340afaf0 0x7f81340b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.221+0000 7f81497fa640 1 -- 172.21.15.96:0/3324102411 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f81440df320 con 0x7f814c109a10 2024-04-25T14:15:17.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.222+0000 7f814bfff640 1 --2- 172.21.15.96:0/3324102411 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f81340afaf0 0x7f81340b2000 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f814c1046d0 tx=0x7f813c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:17.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.225+0000 7f81497fa640 1 -- 172.21.15.96:0/3324102411 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8144053ff0 con 0x7f814c109a10 2024-04-25T14:15:17.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.467+0000 7f815255e640 1 -- 172.21.15.96:0/3324102411 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f814c02dc10 con 0x7f814c109a10 2024-04-25T14:15:17.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.468+0000 7f81497fa640 1 -- 172.21.15.96:0/3324102411 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11927 (secure 0 0 0) 0x7f81440a7ee0 con 0x7f814c109a10 2024-04-25T14:15:17.468 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:15:17.468 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/606227917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":606227917},{"type":"v1","addr":"172.21.15.96:6835","nonce":606227917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/3265300710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3265300710},{"type":"v1","addr":"172.21.15.96:6837","nonce":3265300710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T14:11:28.886961+0000","modified":"2024-04-25T14:13:32.120297+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24517,"mds_2":24478,"mds_3":24455,"mds_4":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/4080308509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":4080308509},{"type":"v1","addr":"172.21.15.96:6839","nonce":4080308509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/1606748971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":1606748971},{"type":"v1","addr":"172.21.15.96:6841","nonce":1606748971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6833/1995324831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1995324831},{"type":"v1","addr":"172.21.15.97:6833","nonce":1995324831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":4,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.151:6833/1382721608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1382721608},{"type":"v1","addr":"172.21.15.151:6833","nonce":1382721608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6835/3872301726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3872301726},{"type":"v1","addr":"172.21.15.97:6835","nonce":3872301726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.151:6835/1313823638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":1313823638},{"type":"v1","addr":"172.21.15.151:6835","nonce":1313823638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/2691348366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2691348366},{"type":"v1","addr":"172.21.15.151:6837","nonce":2691348366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3306780081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3306780081},{"type":"v1","addr":"172.21.15.97:6837","nonce":3306780081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.151:6839/245699981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":245699981},{"type":"v1","addr":"172.21.15.151:6839","nonce":245699981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6839/2850442269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":2850442269},{"type":"v1","addr":"172.21.15.97:6839","nonce":2850442269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24427,24455,24478,24517,24536]},"id":1}]} 2024-04-25T14:15:17.468 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 26 2024-04-25T14:15:17.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.471+0000 7f815255e640 1 -- 172.21.15.96:0/3324102411 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f81340afaf0 msgr2=0x7f81340b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:17.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.471+0000 7f815255e640 1 --2- 172.21.15.96:0/3324102411 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f81340afaf0 0x7f81340b2000 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f814c1046d0 tx=0x7f813c005f90 comp rx=0 tx=0).stop 2024-04-25T14:15:17.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.471+0000 7f815255e640 1 -- 172.21.15.96:0/3324102411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f814c109a10 msgr2=0x7f814c1a1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:17.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.471+0000 7f815255e640 1 --2- 172.21.15.96:0/3324102411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f814c109a10 0x7f814c1a1690 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f814400eb10 tx=0x7f814400c2d0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.471+0000 7f815255e640 1 -- 172.21.15.96:0/3324102411 shutdown_connections 2024-04-25T14:15:17.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.471+0000 7f815255e640 1 --2- 172.21.15.96:0/3324102411 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f81340afaf0 0x7f81340b2000 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.471+0000 7f815255e640 1 --2- 172.21.15.96:0/3324102411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f814c109a10 0x7f814c1a1690 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.471+0000 7f815255e640 1 --2- 172.21.15.96:0/3324102411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f814c104d10 0x7f814c19a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.471+0000 7f815255e640 1 --2- 172.21.15.96:0/3324102411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f814c103ad0 0x7f814c19a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.471+0000 7f815255e640 1 -- 172.21.15.96:0/3324102411 >> 172.21.15.96:0/3324102411 conn(0x7f814c0fecc0 msgr2=0x7f814c1059f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:17.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.471+0000 7f815255e640 1 -- 172.21.15.96:0/3324102411 shutdown_connections 2024-04-25T14:15:17.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.471+0000 7f815255e640 1 -- 172.21.15.96:0/3324102411 wait complete. 2024-04-25T14:15:17.484 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T14:15:17.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.605+0000 7f962b706640 1 -- 172.21.15.96:0/835738884 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f962410bc30 msgr2=0x7f962410e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:17.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.605+0000 7f962b706640 1 --2- 172.21.15.96:0/835738884 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f962410bc30 0x7f962410e050 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f961c009ca0 tx=0x7f961c02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.605+0000 7f962b706640 1 -- 172.21.15.96:0/835738884 shutdown_connections 2024-04-25T14:15:17.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.605+0000 7f962b706640 1 --2- 172.21.15.96:0/835738884 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f962410bc30 0x7f962410e050 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.605+0000 7f962b706640 1 --2- 172.21.15.96:0/835738884 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9624106f30 0x7f9624107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.605+0000 7f962b706640 1 --2- 172.21.15.96:0/835738884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9624105cf0 0x7f96241069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.605+0000 7f962b706640 1 -- 172.21.15.96:0/835738884 >> 172.21.15.96:0/835738884 conn(0x7f9624100f20 msgr2=0x7f9624103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:17.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.605+0000 7f962b706640 1 -- 172.21.15.96:0/835738884 shutdown_connections 2024-04-25T14:15:17.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.605+0000 7f962b706640 1 -- 172.21.15.96:0/835738884 wait complete. 2024-04-25T14:15:17.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.606+0000 7f962b706640 1 Processor -- start 2024-04-25T14:15:17.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.606+0000 7f962b706640 1 -- start start 2024-04-25T14:15:17.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.606+0000 7f962b706640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9624105cf0 0x7f962419c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.606+0000 7f962b706640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9624106f30 0x7f962419c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.606+0000 7f962b706640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f962410bc30 0x7f96241a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.606+0000 7f962b706640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9624111820 con 0x7f9624106f30 2024-04-25T14:15:17.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.606+0000 7f962b706640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f96241116a0 con 0x7f962410bc30 2024-04-25T14:15:17.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.606+0000 7f962b706640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f96241119a0 con 0x7f9624105cf0 2024-04-25T14:15:17.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.606+0000 7f9628ca1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9624106f30 0x7f962419c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.606+0000 7f9628ca1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9624106f30 0x7f962419c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41448/0 (socket says 172.21.15.96:41448) 2024-04-25T14:15:17.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.606+0000 7f9628ca1640 1 -- 172.21.15.96:0/2490403033 learned_addr learned my addr 172.21.15.96:0/2490403033 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:17.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.606+0000 7f96294a2640 1 --2- 172.21.15.96:0/2490403033 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9624105cf0 0x7f962419c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.606+0000 7f9629ca3640 1 --2- 172.21.15.96:0/2490403033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f962410bc30 0x7f96241a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.607+0000 7f9628ca1640 1 -- 172.21.15.96:0/2490403033 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9624105cf0 msgr2=0x7f962419c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:17.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.607+0000 7f9628ca1640 1 --2- 172.21.15.96:0/2490403033 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9624105cf0 0x7f962419c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.607+0000 7f9628ca1640 1 -- 172.21.15.96:0/2490403033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f962410bc30 msgr2=0x7f96241a38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:17.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.607+0000 7f9628ca1640 1 --2- 172.21.15.96:0/2490403033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f962410bc30 0x7f96241a38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.607+0000 7f9628ca1640 1 -- 172.21.15.96:0/2490403033 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96241a3f30 con 0x7f9624106f30 2024-04-25T14:15:17.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.607+0000 7f9628ca1640 1 --2- 172.21.15.96:0/2490403033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9624106f30 0x7f962419c930 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f961400d950 tx=0x7f961400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:17.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.607+0000 7f961a7fc640 1 -- 172.21.15.96:0/2490403033 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9614017070 con 0x7f9624106f30 2024-04-25T14:15:17.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.607+0000 7f962b706640 1 -- 172.21.15.96:0/2490403033 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96241a4220 con 0x7f9624106f30 2024-04-25T14:15:17.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.607+0000 7f961a7fc640 1 -- 172.21.15.96:0/2490403033 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9614009c30 con 0x7f9624106f30 2024-04-25T14:15:17.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.607+0000 7f961a7fc640 1 -- 172.21.15.96:0/2490403033 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961400f620 con 0x7f9624106f30 2024-04-25T14:15:17.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.607+0000 7f962b706640 1 -- 172.21.15.96:0/2490403033 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96241a47c0 con 0x7f9624106f30 2024-04-25T14:15:17.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.610+0000 7f96325b9740 1 Processor -- start 2024-04-25T14:15:17.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.610+0000 7f96325b9740 1 -- start start 2024-04-25T14:15:17.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.610+0000 7f96325b9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c29143390 0x557c294e5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.610+0000 7f96325b9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c2911a4d0 0x557c294e5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.610+0000 7f96325b9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557c291226b0 0x557c294e6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.610+0000 7f96325b9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557c29449d80 con 0x557c29143390 2024-04-25T14:15:17.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.610+0000 7f96325b9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557c29403a20 con 0x557c2911a4d0 2024-04-25T14:15:17.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.610+0000 7f96325b9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557c29445d40 con 0x557c291226b0 2024-04-25T14:15:17.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.610+0000 7f961a7fc640 1 -- 172.21.15.96:0/2490403033 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f961400f7c0 con 0x7f9624106f30 2024-04-25T14:15:17.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.610+0000 7f9629ca3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c29143390 0x557c294e5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.610+0000 7f9629ca3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c29143390 0x557c294e5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41464/0 (socket says 172.21.15.96:41464) 2024-04-25T14:15:17.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.610+0000 7f9629ca3640 1 -- 172.21.15.96:0/199830044 learned_addr learned my addr 172.21.15.96:0/199830044 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:17.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.610+0000 7f96294a2640 1 --2- 172.21.15.96:0/199830044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c2911a4d0 0x557c294e5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.610+0000 7f9628ca1640 1 --2- 172.21.15.96:0/199830044 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557c291226b0 0x557c294e6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.610+0000 7f9629ca3640 1 -- 172.21.15.96:0/199830044 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557c291226b0 msgr2=0x557c294e6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:17.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.610+0000 7f9629ca3640 1 --2- 172.21.15.96:0/199830044 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557c291226b0 0x557c294e6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.610+0000 7f9629ca3640 1 -- 172.21.15.96:0/199830044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c2911a4d0 msgr2=0x557c294e5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:17.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.610+0000 7f9629ca3640 1 --2- 172.21.15.96:0/199830044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c2911a4d0 0x557c294e5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.610+0000 7f9629ca3640 1 -- 172.21.15.96:0/199830044 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c2940ec30 con 0x557c29143390 2024-04-25T14:15:17.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.610+0000 7f96294a2640 1 --2- 172.21.15.96:0/199830044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c2911a4d0 0x557c294e5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:15:17.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.611+0000 7f9629ca3640 1 --2- 172.21.15.96:0/199830044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c29143390 0x557c294e5810 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f961c035000 tx=0x7f961c02f4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a238bba1ec6ddd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:17.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.611+0000 7f961a7fc640 1 --2- 172.21.15.96:0/2490403033 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f96100afaf0 0x7f96100b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.611+0000 7f960f7fe640 1 -- 172.21.15.96:0/199830044 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961c031650 con 0x557c29143390 2024-04-25T14:15:17.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.611+0000 7f96294a2640 1 --2- 172.21.15.96:0/2490403033 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f96100afaf0 0x7f96100b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.611+0000 7f960f7fe640 1 -- 172.21.15.96:0/199830044 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f961c036030 con 0x557c29143390 2024-04-25T14:15:17.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.611+0000 7f960f7fe640 1 -- 172.21.15.96:0/199830044 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961c004800 con 0x557c29143390 2024-04-25T14:15:17.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.611+0000 7f961a7fc640 1 -- 172.21.15.96:0/2490403033 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f96140de7b0 con 0x7f9624106f30 2024-04-25T14:15:17.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.612+0000 7f96325b9740 1 -- 172.21.15.96:0/199830044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c29143390 msgr2=0x557c294e5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:17.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.612+0000 7f96294a2640 1 --2- 172.21.15.96:0/2490403033 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f96100afaf0 0x7f96100b2000 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x557c294e5530 tx=0x7f9620005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:17.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.612+0000 7f96325b9740 1 --2- 172.21.15.96:0/199830044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c29143390 0x557c294e5810 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f961c035000 tx=0x7f961c02f4c0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.612+0000 7f96325b9740 1 -- 172.21.15.96:0/199830044 shutdown_connections 2024-04-25T14:15:17.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.612+0000 7f96325b9740 1 --2- 172.21.15.96:0/199830044 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557c291226b0 0x557c294e6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.612+0000 7f96325b9740 1 --2- 172.21.15.96:0/199830044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c2911a4d0 0x557c294e5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.612+0000 7f96325b9740 1 --2- 172.21.15.96:0/199830044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c29143390 0x557c294e5810 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.612+0000 7f96325b9740 1 -- 172.21.15.96:0/199830044 >> 172.21.15.96:0/199830044 conn(0x557c291374e0 msgr2=0x557c294e1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:17.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.612+0000 7f96325b9740 1 -- 172.21.15.96:0/199830044 shutdown_connections 2024-04-25T14:15:17.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.612+0000 7f96325b9740 1 -- 172.21.15.96:0/199830044 wait complete. 2024-04-25T14:15:17.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.612+0000 7f96325b9740 1 -- start start 2024-04-25T14:15:17.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.612+0000 7f96325b9740 1 Processor -- start 2024-04-25T14:15:17.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.613+0000 7f96325b9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c2911a4d0 0x557c294ef470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.613+0000 7f96325b9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c291226b0 0x557c294ef9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.613+0000 7f96325b9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557c29143390 0x557c294f6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.613+0000 7f96325b9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557c294e7350 con 0x557c291226b0 2024-04-25T14:15:17.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.613+0000 7f96325b9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557c293d0dc0 con 0x557c2911a4d0 2024-04-25T14:15:17.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.613+0000 7f96325b9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557c294e74d0 con 0x557c29143390 2024-04-25T14:15:17.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.613+0000 7f96325b9740 10 client.0 _ll_register_callbacks cb 0x557c293ec600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:15:17.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.613+0000 7f9628ca1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c291226b0 0x557c294ef9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.613+0000 7f96294a2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c2911a4d0 0x557c294ef470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.613+0000 7f96294a2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c2911a4d0 0x557c294ef470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:41830/0 (socket says 172.21.15.96:41830) 2024-04-25T14:15:17.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.613+0000 7f9629ca3640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557c29143390 0x557c294f6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.613+0000 7f96294a2640 1 -- 172.21.15.96:0/2222840786 learned_addr learned my addr 172.21.15.96:0/2222840786 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:17.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.613+0000 7f9628ca1640 1 --2- 172.21.15.96:0/2222840786 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c291226b0 0x557c294ef9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41480/0 (socket says 172.21.15.96:41480) 2024-04-25T14:15:17.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.613+0000 7f9628ca1640 1 -- 172.21.15.96:0/2222840786 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557c29143390 msgr2=0x557c294f6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:17.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.613+0000 7f9628ca1640 1 --2- 172.21.15.96:0/2222840786 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557c29143390 0x557c294f6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.613+0000 7f9628ca1640 1 -- 172.21.15.96:0/2222840786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c2911a4d0 msgr2=0x557c294ef470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:17.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.613+0000 7f9628ca1640 1 --2- 172.21.15.96:0/2222840786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c2911a4d0 0x557c294ef470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.613+0000 7f9628ca1640 1 -- 172.21.15.96:0/2222840786 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c2915b3f0 con 0x557c291226b0 2024-04-25T14:15:17.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.613+0000 7f96294a2640 1 --2- 172.21.15.96:0/2222840786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c2911a4d0 0x557c294ef470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:15:17.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.613+0000 7f9629ca3640 1 --2- 172.21.15.96:0/2222840786 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557c29143390 0x557c294f6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:15:17.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.614+0000 7f9628ca1640 1 --2- 172.21.15.96:0/2222840786 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c291226b0 0x557c294ef9b0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f96140dc2e0 tx=0x7f961400bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afcb403ed2a390b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:17.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.614+0000 7f960cff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:15:17.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.614+0000 7f960cff9640 1 -- 172.21.15.96:0/2222840786 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9614020070 con 0x557c291226b0 2024-04-25T14:15:17.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.614+0000 7f960cff9640 1 -- 172.21.15.96:0/2222840786 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f96140a5f80 con 0x557c291226b0 2024-04-25T14:15:17.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.614+0000 7f96325b9740 1 -- 172.21.15.96:0/2222840786 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f961c036030 con 0x557c291226b0 2024-04-25T14:15:17.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.614+0000 7f960cff9640 1 -- 172.21.15.96:0/2222840786 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f96140a6830 con 0x557c291226b0 2024-04-25T14:15:17.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.614+0000 7f96325b9740 10 client.16092 fetch_fsmap learned FSMap version 26 2024-04-25T14:15:17.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.614+0000 7f96325b9740 1 -- 172.21.15.96:0/2222840786 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557c292257d0 con 0x557c291226b0 2024-04-25T14:15:17.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.614+0000 7f960cff9640 1 -- 172.21.15.96:0/2222840786 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f961c036030 con 0x557c291226b0 2024-04-25T14:15:17.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.614+0000 7f960cff9640 1 -- 172.21.15.96:0/2222840786 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f96140a59f0 con 0x557c291226b0 2024-04-25T14:15:17.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.614+0000 7f96325b9740 10 client.16092 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:15:17.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.614+0000 7f96325b9740 10 client.16092 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-25T14:15:17.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.614+0000 7f96325b9740 20 client.16092 populate_metadata read hostname 'smithi096' 2024-04-25T14:15:17.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.614+0000 7f96325b9740 1 --2- 172.21.15.96:0/2222840786 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x557c29101f60 0x557c294fd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.614+0000 7f96325b9740 4 client.16092 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:15:17.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.614+0000 7f96325b9740 1 -- 172.21.15.96:0/2222840786 --> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c29457990 con 0x557c29101f60 2024-04-25T14:15:17.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.614+0000 7f96294a2640 1 --2- 172.21.15.96:0/2222840786 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x557c29101f60 0x557c294fd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.615+0000 7f960cff9640 10 client.16092 ms_handle_connect on v2:172.21.15.151:6832/1382721608 2024-04-25T14:15:17.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.615+0000 7f96294a2640 1 --2- 172.21.15.96:0/2222840786 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x557c29101f60 0x557c294fd160 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4e50d7278178bed9 server_cookie=61d25fcdedd7867c in_seq=0 out_seq=0 2024-04-25T14:15:17.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.615+0000 7f960cff9640 1 -- 172.21.15.96:0/2222840786 <== mds.4 v2:172.21.15.151:6832/1382721608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557c29457990 con 0x557c29101f60 2024-04-25T14:15:17.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.615+0000 7f960cff9640 10 client.16092 handle_command_reply: tid=0 2024-04-25T14:15:17.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.616+0000 7f96325b9740 1 client.16092 shutdown 2024-04-25T14:15:17.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.616+0000 7f96325b9740 1 -- 172.21.15.96:0/2222840786 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c291226b0 msgr2=0x557c294ef9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:17.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.616+0000 7f96325b9740 1 --2- 172.21.15.96:0/2222840786 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c291226b0 0x557c294ef9b0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f96140dc2e0 tx=0x7f961400bf60 comp rx=0 tx=0).stop 2024-04-25T14:15:17.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.616+0000 7f96325b9740 1 -- 172.21.15.96:0/2222840786 shutdown_connections 2024-04-25T14:15:17.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.616+0000 7f96325b9740 1 --2- 172.21.15.96:0/2222840786 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x557c29101f60 0x557c294fd160 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.616+0000 7f96325b9740 1 --2- 172.21.15.96:0/2222840786 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557c29143390 0x557c294f6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.616+0000 7f960cff9640 0 client.16092 ms_handle_reset on v2:172.21.15.151:6832/1382721608 2024-04-25T14:15:17.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.616+0000 7f96325b9740 1 --2- 172.21.15.96:0/2222840786 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c291226b0 0x557c294ef9b0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.616+0000 7f96325b9740 1 --2- 172.21.15.96:0/2222840786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c2911a4d0 0x557c294ef470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.616+0000 7f96325b9740 1 -- 172.21.15.96:0/2222840786 >> 172.21.15.96:0/2222840786 conn(0x557c291374e0 msgr2=0x557c294e0040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:17.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.616+0000 7f96325b9740 1 -- 172.21.15.96:0/2222840786 shutdown_connections 2024-04-25T14:15:17.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.616+0000 7f96325b9740 1 -- 172.21.15.96:0/2222840786 wait complete. 2024-04-25T14:15:17.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.616+0000 7f96325b9740 20 client.16092 trim_cache size 0 max 16384 2024-04-25T14:15:17.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.632+0000 7f96325b9740 1 Processor -- start 2024-04-25T14:15:17.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.632+0000 7f96325b9740 1 -- start start 2024-04-25T14:15:17.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.633+0000 7f96325b9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c2911a4d0 0x557c294eda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.633+0000 7f96325b9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557c291226b0 0x557c294f6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.633+0000 7f96325b9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c29143390 0x557c294edfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.633+0000 7f96325b9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f96241ad430 con 0x557c2911a4d0 2024-04-25T14:15:17.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.633+0000 7f96325b9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f96241aef70 con 0x557c29143390 2024-04-25T14:15:17.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.633+0000 7f96325b9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f96241adb00 con 0x557c291226b0 2024-04-25T14:15:17.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.633+0000 7f96294a2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c2911a4d0 0x557c294eda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.633+0000 7f96294a2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c2911a4d0 0x557c294eda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41492/0 (socket says 172.21.15.96:41492) 2024-04-25T14:15:17.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.633+0000 7f9628ca1640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557c291226b0 0x557c294f6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.633+0000 7f96294a2640 1 -- 172.21.15.96:0/1584279618 learned_addr learned my addr 172.21.15.96:0/1584279618 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:17.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.633+0000 7f9629ca3640 1 --2- 172.21.15.96:0/1584279618 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c29143390 0x557c294edfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.633+0000 7f9628ca1640 1 -- 172.21.15.96:0/1584279618 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c29143390 msgr2=0x557c294edfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:17.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.633+0000 7f9628ca1640 1 --2- 172.21.15.96:0/1584279618 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c29143390 0x557c294edfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.633+0000 7f9628ca1640 1 -- 172.21.15.96:0/1584279618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c2911a4d0 msgr2=0x557c294eda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:17.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.633+0000 7f9628ca1640 1 --2- 172.21.15.96:0/1584279618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c2911a4d0 0x557c294eda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.633+0000 7f9628ca1640 1 -- 172.21.15.96:0/1584279618 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c294ed710 con 0x557c291226b0 2024-04-25T14:15:17.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.633+0000 7f9628ca1640 1 --2- 172.21.15.96:0/1584279618 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557c291226b0 0x557c294f6830 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f96140a6a20 tx=0x7f9614005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27932265bb228dbb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:17.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.633+0000 7f960effd640 1 -- 172.21.15.96:0/1584279618 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9614014020 con 0x557c291226b0 2024-04-25T14:15:17.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.634+0000 7f960effd640 1 -- 172.21.15.96:0/1584279618 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96140c1330 con 0x557c291226b0 2024-04-25T14:15:17.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.634+0000 7f960effd640 1 -- 172.21.15.96:0/1584279618 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9614020070 con 0x557c291226b0 2024-04-25T14:15:17.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.634+0000 7f96325b9740 1 -- 172.21.15.96:0/1584279618 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557c291226b0 msgr2=0x557c294f6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:17.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.634+0000 7f96325b9740 1 --2- 172.21.15.96:0/1584279618 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557c291226b0 0x557c294f6830 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f96140a6a20 tx=0x7f9614005880 comp rx=0 tx=0).stop 2024-04-25T14:15:17.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.634+0000 7f96325b9740 1 -- 172.21.15.96:0/1584279618 shutdown_connections 2024-04-25T14:15:17.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.634+0000 7f96325b9740 1 --2- 172.21.15.96:0/1584279618 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c29143390 0x557c294edfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.634+0000 7f96325b9740 1 --2- 172.21.15.96:0/1584279618 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557c291226b0 0x557c294f6830 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.634+0000 7f96325b9740 1 --2- 172.21.15.96:0/1584279618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c2911a4d0 0x557c294eda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.634+0000 7f96325b9740 1 -- 172.21.15.96:0/1584279618 >> 172.21.15.96:0/1584279618 conn(0x557c291374e0 msgr2=0x557c294ea350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:17.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.634+0000 7f96325b9740 1 -- 172.21.15.96:0/1584279618 shutdown_connections 2024-04-25T14:15:17.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.634+0000 7f96325b9740 1 -- 172.21.15.96:0/1584279618 wait complete. 2024-04-25T14:15:17.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.635+0000 7f96325b9740 1 -- start start 2024-04-25T14:15:17.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.635+0000 7f96325b9740 1 Processor -- start 2024-04-25T14:15:17.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.635+0000 7f96325b9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c2911a4d0 0x557c294f3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.635+0000 7f96325b9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557c291226b0 0x557c294f3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.635+0000 7f96325b9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c29143390 0x557c295634c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.635+0000 7f96325b9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f96241ae8a0 con 0x557c2911a4d0 2024-04-25T14:15:17.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.635+0000 7f96325b9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f96241ae1d0 con 0x557c29143390 2024-04-25T14:15:17.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.635+0000 7f96325b9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f96241ab410 con 0x557c291226b0 2024-04-25T14:15:17.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.635+0000 7f96325b9740 10 client.0 _ll_register_callbacks cb 0x7f961400df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:15:17.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.635+0000 7f96294a2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c2911a4d0 0x557c294f3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.635+0000 7f96294a2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c2911a4d0 0x557c294f3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41500/0 (socket says 172.21.15.96:41500) 2024-04-25T14:15:17.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.635+0000 7f96294a2640 1 -- 172.21.15.96:0/1395565746 learned_addr learned my addr 172.21.15.96:0/1395565746 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:17.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.635+0000 7f9628ca1640 1 --2- 172.21.15.96:0/1395565746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557c291226b0 0x557c294f3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.635+0000 7f9629ca3640 1 --2- 172.21.15.96:0/1395565746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c29143390 0x557c295634c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.636+0000 7f96294a2640 1 -- 172.21.15.96:0/1395565746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557c291226b0 msgr2=0x557c294f3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:17.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.636+0000 7f96294a2640 1 --2- 172.21.15.96:0/1395565746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557c291226b0 0x557c294f3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.636+0000 7f96294a2640 1 -- 172.21.15.96:0/1395565746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c29143390 msgr2=0x557c295634c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:17.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.636+0000 7f96294a2640 1 --2- 172.21.15.96:0/1395565746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c29143390 0x557c295634c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.636+0000 7f96294a2640 1 -- 172.21.15.96:0/1395565746 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c294f6a30 con 0x557c2911a4d0 2024-04-25T14:15:17.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.636+0000 7f96294a2640 1 --2- 172.21.15.96:0/1395565746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c2911a4d0 0x557c294f3530 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x557c294fd8c0 tx=0x7f9620004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f940894892eb9a43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:17.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.636+0000 7f960cff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:15:17.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.636+0000 7f960cff9640 1 -- 172.21.15.96:0/1395565746 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9620003850 con 0x557c2911a4d0 2024-04-25T14:15:17.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.636+0000 7f960cff9640 1 -- 172.21.15.96:0/1395565746 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f96200039f0 con 0x557c2911a4d0 2024-04-25T14:15:17.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.636+0000 7f96325b9740 1 -- 172.21.15.96:0/1395565746 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96140c1330 con 0x557c2911a4d0 2024-04-25T14:15:17.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.636+0000 7f960cff9640 1 -- 172.21.15.96:0/1395565746 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9620003bf0 con 0x557c2911a4d0 2024-04-25T14:15:17.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.636+0000 7f96325b9740 10 client.16098 fetch_fsmap learned FSMap version 26 2024-04-25T14:15:17.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.636+0000 7f96325b9740 1 -- 172.21.15.96:0/1395565746 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557c29564a50 con 0x557c2911a4d0 2024-04-25T14:15:17.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.636+0000 7f960cff9640 1 -- 172.21.15.96:0/1395565746 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96140c1330 con 0x557c2911a4d0 2024-04-25T14:15:17.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.636+0000 7f960cff9640 1 -- 172.21.15.96:0/1395565746 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f962003f420 con 0x557c2911a4d0 2024-04-25T14:15:17.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.636+0000 7f96325b9740 10 client.16098 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:15:17.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.636+0000 7f96325b9740 10 client.16098 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-25T14:15:17.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.636+0000 7f96325b9740 20 client.16098 populate_metadata read hostname 'smithi096' 2024-04-25T14:15:17.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.637+0000 7f96325b9740 1 --2- 172.21.15.96:0/1395565746 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x557c290d73a0 0x557c29567450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.637+0000 7f96325b9740 4 client.16098 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:15:17.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.637+0000 7f96325b9740 1 -- 172.21.15.96:0/1395565746 --> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557c29451800 con 0x557c290d73a0 2024-04-25T14:15:17.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.637+0000 7f9628ca1640 1 --2- 172.21.15.96:0/1395565746 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x557c290d73a0 0x557c29567450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.637+0000 7f960cff9640 10 client.16098 ms_handle_connect on v2:172.21.15.151:6832/1382721608 2024-04-25T14:15:17.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.637+0000 7f9628ca1640 1 --2- 172.21.15.96:0/1395565746 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x557c290d73a0 0x557c29567450 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c34f771e1ef02180 server_cookie=8b4d1e05baa9d885 in_seq=0 out_seq=0 2024-04-25T14:15:17.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.637+0000 7f960cff9640 1 -- 172.21.15.96:0/1395565746 <== mds.4 v2:172.21.15.151:6832/1382721608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557c29451800 con 0x557c290d73a0 2024-04-25T14:15:17.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.637+0000 7f960cff9640 10 client.16098 handle_command_reply: tid=0 2024-04-25T14:15:17.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.637+0000 7f96325b9740 1 client.16098 shutdown 2024-04-25T14:15:17.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.637+0000 7f96325b9740 1 -- 172.21.15.96:0/1395565746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c2911a4d0 msgr2=0x557c294f3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:17.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.637+0000 7f96325b9740 1 --2- 172.21.15.96:0/1395565746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c2911a4d0 0x557c294f3530 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x557c294fd8c0 tx=0x7f9620004730 comp rx=0 tx=0).stop 2024-04-25T14:15:17.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.637+0000 7f96325b9740 1 -- 172.21.15.96:0/1395565746 shutdown_connections 2024-04-25T14:15:17.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.638+0000 7f96325b9740 1 --2- 172.21.15.96:0/1395565746 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x557c290d73a0 0x557c29567450 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.638+0000 7f96325b9740 1 --2- 172.21.15.96:0/1395565746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c29143390 0x557c295634c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.638+0000 7f960cff9640 0 client.16098 ms_handle_reset on v2:172.21.15.151:6832/1382721608 2024-04-25T14:15:17.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.638+0000 7f96325b9740 1 --2- 172.21.15.96:0/1395565746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557c291226b0 0x557c294f3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.638+0000 7f96325b9740 1 --2- 172.21.15.96:0/1395565746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c2911a4d0 0x557c294f3530 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.638+0000 7f96325b9740 1 -- 172.21.15.96:0/1395565746 >> 172.21.15.96:0/1395565746 conn(0x557c291374e0 msgr2=0x557c294ed920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:17.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.638+0000 7f96325b9740 1 -- 172.21.15.96:0/1395565746 shutdown_connections 2024-04-25T14:15:17.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.638+0000 7f96325b9740 1 -- 172.21.15.96:0/1395565746 wait complete. 2024-04-25T14:15:17.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.638+0000 7f96325b9740 20 client.16098 trim_cache size 0 max 16384 2024-04-25T14:15:17.644 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-04-25T14:15:17.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.638+0000 7f962b706640 1 -- 172.21.15.96:0/2490403033 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f96100afaf0 msgr2=0x7f96100b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:17.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.638+0000 7f962b706640 1 --2- 172.21.15.96:0/2490403033 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f96100afaf0 0x7f96100b2000 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x557c294e5530 tx=0x7f9620005f90 comp rx=0 tx=0).stop 2024-04-25T14:15:17.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.638+0000 7f962b706640 1 -- 172.21.15.96:0/2490403033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9624106f30 msgr2=0x7f962419c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:17.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.638+0000 7f962b706640 1 --2- 172.21.15.96:0/2490403033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9624106f30 0x7f962419c930 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f961400d950 tx=0x7f961400de10 comp rx=0 tx=0).stop 2024-04-25T14:15:17.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.639+0000 7f962b706640 1 -- 172.21.15.96:0/2490403033 shutdown_connections 2024-04-25T14:15:17.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.639+0000 7f962b706640 1 --2- 172.21.15.96:0/2490403033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f962410bc30 0x7f96241a38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.639+0000 7f962b706640 1 --2- 172.21.15.96:0/2490403033 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f96100afaf0 0x7f96100b2000 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.639+0000 7f962b706640 1 --2- 172.21.15.96:0/2490403033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9624106f30 0x7f962419c930 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.639+0000 7f962b706640 1 --2- 172.21.15.96:0/2490403033 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9624105cf0 0x7f962419c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.639+0000 7f962b706640 1 -- 172.21.15.96:0/2490403033 >> 172.21.15.96:0/2490403033 conn(0x7f9624100f20 msgr2=0x7f9624107c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:17.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.639+0000 7f962b706640 1 -- 172.21.15.96:0/2490403033 shutdown_connections 2024-04-25T14:15:17.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.639+0000 7f962b706640 1 -- 172.21.15.96:0/2490403033 wait complete. 2024-04-25T14:15:17.651 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T14:15:17.761 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4197376 len 16384 2024-04-25T14:15:17.761 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4197376 len 16384 2024-04-25T14:15:17.761 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4197888 len 1024 2024-04-25T14:15:17.761 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4197888 len 1024 2024-04-25T14:15:17.761 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4197888 len 1024 2024-04-25T14:15:17.762 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4197888 len 1024 2024-04-25T14:15:17.762 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4197888 len 2048 2024-04-25T14:15:17.762 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4197888 len 2048 2024-04-25T14:15:17.762 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4197888 len 2048 2024-04-25T14:15:17.762 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4197888 len 2048 2024-04-25T14:15:17.762 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4197888 len 4096 2024-04-25T14:15:17.762 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4197888 len 4096 2024-04-25T14:15:17.762 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4197888 len 4096 2024-04-25T14:15:17.762 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4197888 len 4096 2024-04-25T14:15:17.762 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4197888 len 8192 2024-04-25T14:15:17.762 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4197888 len 8192 2024-04-25T14:15:17.762 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4197888 len 8192 2024-04-25T14:15:17.762 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4197888 len 8192 2024-04-25T14:15:17.763 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 0 offset 4197888 len 16384 2024-04-25T14:15:17.763 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4197888 len 16384 2024-04-25T14:15:17.763 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 0 offset 4197888 len 16384 2024-04-25T14:15:17.763 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 0 offset 4197888 len 16384 2024-04-25T14:15:17.763 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4190720 len 1024 2024-04-25T14:15:17.763 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4190720 len 1024 2024-04-25T14:15:17.763 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4190720 len 1024 2024-04-25T14:15:17.763 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4190720 len 1024 2024-04-25T14:15:17.763 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4190720 len 2048 2024-04-25T14:15:17.763 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4190720 len 2048 2024-04-25T14:15:17.763 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4190720 len 2048 2024-04-25T14:15:17.763 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4190720 len 2048 2024-04-25T14:15:17.763 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4190720 len 4096 2024-04-25T14:15:17.763 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4190720 len 4096 2024-04-25T14:15:17.763 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4190720 len 4096 2024-04-25T14:15:17.764 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4190720 len 4096 2024-04-25T14:15:17.764 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4190720 len 8192 2024-04-25T14:15:17.764 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4190720 len 8192 2024-04-25T14:15:17.764 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4190720 len 8192 2024-04-25T14:15:17.764 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4190720 len 8192 2024-04-25T14:15:17.764 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4190720 len 16384 2024-04-25T14:15:17.764 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4190720 len 16384 2024-04-25T14:15:17.764 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4190720 len 16384 2024-04-25T14:15:17.764 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4190720 len 16384 2024-04-25T14:15:17.764 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4191232 len 1024 2024-04-25T14:15:17.764 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4191232 len 1024 2024-04-25T14:15:17.764 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4191232 len 1024 2024-04-25T14:15:17.764 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4191232 len 1024 2024-04-25T14:15:17.764 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4191232 len 2048 2024-04-25T14:15:17.765 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4191232 len 2048 2024-04-25T14:15:17.765 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4191232 len 2048 2024-04-25T14:15:17.765 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4191232 len 2048 2024-04-25T14:15:17.765 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4191232 len 4096 2024-04-25T14:15:17.765 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4191232 len 4096 2024-04-25T14:15:17.765 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4191232 len 4096 2024-04-25T14:15:17.765 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4191232 len 4096 2024-04-25T14:15:17.765 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4191232 len 8192 2024-04-25T14:15:17.765 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4191232 len 8192 2024-04-25T14:15:17.765 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4191232 len 8192 2024-04-25T14:15:17.765 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4191232 len 8192 2024-04-25T14:15:17.765 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4191232 len 16384 2024-04-25T14:15:17.765 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4191232 len 16384 2024-04-25T14:15:17.765 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4191232 len 16384 2024-04-25T14:15:17.765 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4191232 len 16384 2024-04-25T14:15:17.766 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4191744 len 1024 2024-04-25T14:15:17.766 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4191744 len 1024 2024-04-25T14:15:17.766 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4191744 len 1024 2024-04-25T14:15:17.766 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4191744 len 1024 2024-04-25T14:15:17.766 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4191744 len 2048 2024-04-25T14:15:17.766 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4191744 len 2048 2024-04-25T14:15:17.766 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4191744 len 2048 2024-04-25T14:15:17.766 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4191744 len 2048 2024-04-25T14:15:17.766 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4191744 len 4096 2024-04-25T14:15:17.766 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4191744 len 4096 2024-04-25T14:15:17.766 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4191744 len 4096 2024-04-25T14:15:17.766 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4191744 len 4096 2024-04-25T14:15:17.766 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4191744 len 8192 2024-04-25T14:15:17.766 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4191744 len 8192 2024-04-25T14:15:17.767 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4191744 len 8192 2024-04-25T14:15:17.767 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4191744 len 8192 2024-04-25T14:15:17.767 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4191744 len 16384 2024-04-25T14:15:17.767 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4191744 len 16384 2024-04-25T14:15:17.767 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4191744 len 16384 2024-04-25T14:15:17.767 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4191744 len 16384 2024-04-25T14:15:17.767 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4192256 len 1024 2024-04-25T14:15:17.767 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4192256 len 1024 2024-04-25T14:15:17.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.771+0000 7f1ce7ad1640 1 -- 172.21.15.96:0/1339060042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ce0101ef0 msgr2=0x7f1ce0106b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:17.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.771+0000 7f1ce7ad1640 1 --2- 172.21.15.96:0/1339060042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ce0101ef0 0x7f1ce0106b60 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc009900 tx=0x7f1cdc02a410 comp rx=0 tx=0).stop 2024-04-25T14:15:17.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.771+0000 7f1ce7ad1640 1 -- 172.21.15.96:0/1339060042 shutdown_connections 2024-04-25T14:15:17.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.771+0000 7f1ce7ad1640 1 --2- 172.21.15.96:0/1339060042 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ce0109a00 0x7f1ce010be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.771+0000 7f1ce7ad1640 1 --2- 172.21.15.96:0/1339060042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ce01070a0 0x7f1ce01094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.771+0000 7f1ce7ad1640 1 --2- 172.21.15.96:0/1339060042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ce0101ef0 0x7f1ce0106b60 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.771+0000 7f1ce7ad1640 1 -- 172.21.15.96:0/1339060042 >> 172.21.15.96:0/1339060042 conn(0x7f1ce00fd350 msgr2=0x7f1ce00ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:17.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.772+0000 7f1ce7ad1640 1 -- 172.21.15.96:0/1339060042 shutdown_connections 2024-04-25T14:15:17.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.772+0000 7f1ce7ad1640 1 -- 172.21.15.96:0/1339060042 wait complete. 2024-04-25T14:15:17.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.772+0000 7f1ce7ad1640 1 Processor -- start 2024-04-25T14:15:17.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.772+0000 7f1ce7ad1640 1 -- start start 2024-04-25T14:15:17.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.772+0000 7f1ce7ad1640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ce0101ef0 0x7f1ce019a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.772+0000 7f1ce7ad1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ce01070a0 0x7f1ce019a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.772+0000 7f1ce7ad1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ce0109a00 0x7f1ce01a1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.772+0000 7f1ce7ad1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce010f670 con 0x7f1ce01070a0 2024-04-25T14:15:17.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.772+0000 7f1ce7ad1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce010f4f0 con 0x7f1ce0109a00 2024-04-25T14:15:17.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.772+0000 7f1ce7ad1640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce010f7f0 con 0x7f1ce0101ef0 2024-04-25T14:15:17.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.773+0000 7f1ce506c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ce01070a0 0x7f1ce019a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.773+0000 7f1ce506c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ce01070a0 0x7f1ce019a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41516/0 (socket says 172.21.15.96:41516) 2024-04-25T14:15:17.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.773+0000 7f1ce586d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ce0101ef0 0x7f1ce019a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.773+0000 7f1ce506c640 1 -- 172.21.15.96:0/3641870526 learned_addr learned my addr 172.21.15.96:0/3641870526 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:17.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.773+0000 7f1ce606e640 1 --2- 172.21.15.96:0/3641870526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ce0109a00 0x7f1ce01a1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.773+0000 7f1ce506c640 1 -- 172.21.15.96:0/3641870526 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ce0101ef0 msgr2=0x7f1ce019a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:17.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.773+0000 7f1ce506c640 1 --2- 172.21.15.96:0/3641870526 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ce0101ef0 0x7f1ce019a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.773+0000 7f1ce506c640 1 -- 172.21.15.96:0/3641870526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ce0109a00 msgr2=0x7f1ce01a1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:17.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.773+0000 7f1ce506c640 1 --2- 172.21.15.96:0/3641870526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ce0109a00 0x7f1ce01a1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.773+0000 7f1ce506c640 1 -- 172.21.15.96:0/3641870526 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ce01a1de0 con 0x7f1ce01070a0 2024-04-25T14:15:17.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.773+0000 7f1ce506c640 1 --2- 172.21.15.96:0/3641870526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ce01070a0 0x7f1ce019a7e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f1cd000e9f0 tx=0x7f1cd000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:17.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.773+0000 7f1cd6ffd640 1 -- 172.21.15.96:0/3641870526 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd000cd80 con 0x7f1ce01070a0 2024-04-25T14:15:17.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.773+0000 7f1ce7ad1640 1 -- 172.21.15.96:0/3641870526 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ce01a20d0 con 0x7f1ce01070a0 2024-04-25T14:15:17.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.773+0000 7f1ce7ad1640 1 -- 172.21.15.96:0/3641870526 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ce01a2670 con 0x7f1ce01070a0 2024-04-25T14:15:17.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.773+0000 7f1cd6ffd640 1 -- 172.21.15.96:0/3641870526 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cd00108e0 con 0x7f1ce01070a0 2024-04-25T14:15:17.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.773+0000 7f1cd6ffd640 1 -- 172.21.15.96:0/3641870526 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd0018980 con 0x7f1ce01070a0 2024-04-25T14:15:17.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.776+0000 7f1cee97d740 1 Processor -- start 2024-04-25T14:15:17.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.776+0000 7f1cee97d740 1 -- start start 2024-04-25T14:15:17.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.777+0000 7f1cd6ffd640 1 -- 172.21.15.96:0/3641870526 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f1cd0018b20 con 0x7f1ce01070a0 2024-04-25T14:15:17.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.777+0000 7f1cee97d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d166920390 0x55d166cc2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.777+0000 7f1cee97d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1668f74d0 0x55d166cc2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.777+0000 7f1cee97d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1668ff6b0 0x55d166cc3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.777+0000 7f1cee97d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d166c26d80 con 0x55d1668f74d0 2024-04-25T14:15:17.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.777+0000 7f1cee97d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d166be0a20 con 0x55d1668ff6b0 2024-04-25T14:15:17.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.777+0000 7f1cee97d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d166c22d40 con 0x55d166920390 2024-04-25T14:15:17.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.777+0000 7f1ce606e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d166920390 0x55d166cc2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.777+0000 7f1ce586d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1668f74d0 0x55d166cc2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.777+0000 7f1ce606e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d166920390 0x55d166cc2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.96:51046/0 (socket says 172.21.15.96:51046) 2024-04-25T14:15:17.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.777+0000 7f1ce586d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1668f74d0 0x55d166cc2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41520/0 (socket says 172.21.15.96:41520) 2024-04-25T14:15:17.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.777+0000 7f1ce606e640 1 -- 172.21.15.96:0/402375793 learned_addr learned my addr 172.21.15.96:0/402375793 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:17.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.777+0000 7f1ce506c640 1 --2- 172.21.15.96:0/402375793 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1668ff6b0 0x55d166cc3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.777+0000 7f1ce606e640 1 -- 172.21.15.96:0/402375793 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1668ff6b0 msgr2=0x55d166cc3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:17.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.777+0000 7f1ce606e640 1 --2- 172.21.15.96:0/402375793 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1668ff6b0 0x55d166cc3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.777+0000 7f1ce606e640 1 -- 172.21.15.96:0/402375793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1668f74d0 msgr2=0x55d166cc2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:17.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.777+0000 7f1ce606e640 1 --2- 172.21.15.96:0/402375793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1668f74d0 0x55d166cc2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.777+0000 7f1ce606e640 1 -- 172.21.15.96:0/402375793 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d166bebc30 con 0x55d166920390 2024-04-25T14:15:17.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.777+0000 7f1ce606e640 1 --2- 172.21.15.96:0/402375793 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d166920390 0x55d166cc2810 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1cd8010370 tx=0x7f1cd8010830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d48b754f4d5b01fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:17.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.777+0000 7f1cd6ffd640 1 --2- 172.21.15.96:0/3641870526 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1ccc0afaf0 0x7f1ccc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.778+0000 7f1ccbfff640 1 -- 172.21.15.96:0/402375793 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd8016070 con 0x55d166920390 2024-04-25T14:15:17.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.778+0000 7f1ce586d640 1 --2- 172.21.15.96:0/3641870526 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1ccc0afaf0 0x7f1ccc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.778+0000 7f1ccbfff640 1 -- 172.21.15.96:0/402375793 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cd8004d20 con 0x55d166920390 2024-04-25T14:15:17.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.778+0000 7f1ccbfff640 1 -- 172.21.15.96:0/402375793 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd8005350 con 0x55d166920390 2024-04-25T14:15:17.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.778+0000 7f1cd6ffd640 1 -- 172.21.15.96:0/3641870526 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f1cd00defc0 con 0x7f1ce01070a0 2024-04-25T14:15:17.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.779+0000 7f1cee97d740 1 -- 172.21.15.96:0/402375793 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d166920390 msgr2=0x55d166cc2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:17.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.779+0000 7f1cee97d740 1 --2- 172.21.15.96:0/402375793 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d166920390 0x55d166cc2810 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1cd8010370 tx=0x7f1cd8010830 comp rx=0 tx=0).stop 2024-04-25T14:15:17.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.779+0000 7f1ce586d640 1 --2- 172.21.15.96:0/3641870526 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1ccc0afaf0 0x7f1ccc0b2000 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc009230 tx=0x7f1cdc031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:17.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.779+0000 7f1cee97d740 1 -- 172.21.15.96:0/402375793 shutdown_connections 2024-04-25T14:15:17.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.779+0000 7f1cee97d740 1 --2- 172.21.15.96:0/402375793 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1668ff6b0 0x55d166cc3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.779+0000 7f1cee97d740 1 --2- 172.21.15.96:0/402375793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1668f74d0 0x55d166cc2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.779+0000 7f1cee97d740 1 --2- 172.21.15.96:0/402375793 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d166920390 0x55d166cc2810 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.779+0000 7f1cee97d740 1 -- 172.21.15.96:0/402375793 >> 172.21.15.96:0/402375793 conn(0x55d1669144e0 msgr2=0x55d166cbe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:17.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.779+0000 7f1cee97d740 1 -- 172.21.15.96:0/402375793 shutdown_connections 2024-04-25T14:15:17.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.779+0000 7f1cee97d740 1 -- 172.21.15.96:0/402375793 wait complete. 2024-04-25T14:15:17.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.779+0000 7f1cee97d740 1 -- start start 2024-04-25T14:15:17.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.779+0000 7f1cee97d740 1 Processor -- start 2024-04-25T14:15:17.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.779+0000 7f1cee97d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1668f74d0 0x55d166ccc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.780+0000 7f1cee97d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1668ff6b0 0x55d166cccba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.780+0000 7f1cee97d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d166920390 0x55d166cd3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.780+0000 7f1cee97d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d166cc4530 con 0x55d1668f74d0 2024-04-25T14:15:17.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.780+0000 7f1cee97d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d166baddc0 con 0x55d1668ff6b0 2024-04-25T14:15:17.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.780+0000 7f1cee97d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d166cc46b0 con 0x55d166920390 2024-04-25T14:15:17.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.780+0000 7f1cee97d740 10 client.0 _ll_register_callbacks cb 0x55d166bc9600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:15:17.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.780+0000 7f1ce586d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1668f74d0 0x55d166ccc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.780+0000 7f1ce586d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1668f74d0 0x55d166ccc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41526/0 (socket says 172.21.15.96:41526) 2024-04-25T14:15:17.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.780+0000 7f1ce586d640 1 -- 172.21.15.96:0/4155854145 learned_addr learned my addr 172.21.15.96:0/4155854145 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:17.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.780+0000 7f1ce606e640 1 --2- 172.21.15.96:0/4155854145 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d166920390 0x55d166cd3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.780+0000 7f1ce586d640 1 -- 172.21.15.96:0/4155854145 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d166920390 msgr2=0x55d166cd3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:17.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.780+0000 7f1ce586d640 1 --2- 172.21.15.96:0/4155854145 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d166920390 0x55d166cd3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.780+0000 7f1ce586d640 1 -- 172.21.15.96:0/4155854145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1668ff6b0 msgr2=0x55d166cccba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:17.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.780+0000 7f1ce586d640 1 --2- 172.21.15.96:0/4155854145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1668ff6b0 0x55d166cccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.780+0000 7f1ce586d640 1 -- 172.21.15.96:0/4155854145 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1669383f0 con 0x55d1668f74d0 2024-04-25T14:15:17.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.780+0000 7f1ce586d640 1 --2- 172.21.15.96:0/4155854145 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1668f74d0 0x55d166ccc660 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc005f70 tx=0x7f1cdc02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e80a47605a5f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:17.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.780+0000 7f1cc97fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:15:17.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.780+0000 7f1cc97fa640 1 -- 172.21.15.96:0/4155854145 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc035040 con 0x55d1668f74d0 2024-04-25T14:15:17.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.780+0000 7f1cc97fa640 1 -- 172.21.15.96:0/4155854145 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f1cdc004300 con 0x55d1668f74d0 2024-04-25T14:15:17.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.780+0000 7f1cee97d740 1 -- 172.21.15.96:0/4155854145 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cd8004d20 con 0x55d1668f74d0 2024-04-25T14:15:17.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.781+0000 7f1cc97fa640 1 -- 172.21.15.96:0/4155854145 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1cdc0035f0 con 0x55d1668f74d0 2024-04-25T14:15:17.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.781+0000 7f1cee97d740 10 client.16110 fetch_fsmap learned FSMap version 26 2024-04-25T14:15:17.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.781+0000 7f1cee97d740 1 -- 172.21.15.96:0/4155854145 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d166a027d0 con 0x55d1668f74d0 2024-04-25T14:15:17.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.781+0000 7f1cc97fa640 1 -- 172.21.15.96:0/4155854145 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cd8004d20 con 0x55d1668f74d0 2024-04-25T14:15:17.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.781+0000 7f1cc97fa640 1 -- 172.21.15.96:0/4155854145 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1cdc044070 con 0x55d1668f74d0 2024-04-25T14:15:17.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.781+0000 7f1cee97d740 10 client.16110 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:15:17.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.781+0000 7f1cee97d740 10 client.16110 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T14:15:17.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.781+0000 7f1cee97d740 20 client.16110 populate_metadata read hostname 'smithi096' 2024-04-25T14:15:17.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.781+0000 7f1cee97d740 1 --2- 172.21.15.96:0/4155854145 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x55d1668def60 0x55d166cda440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.781+0000 7f1cee97d740 4 client.16110 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:15:17.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.781+0000 7f1cee97d740 1 -- 172.21.15.96:0/4155854145 --> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d166c34990 con 0x55d1668def60 2024-04-25T14:15:17.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.782+0000 7f1ce506c640 1 --2- 172.21.15.96:0/4155854145 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x55d1668def60 0x55d166cda440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.782+0000 7f1cc97fa640 10 client.16110 ms_handle_connect on v2:172.21.15.97:6834/3872301726 2024-04-25T14:15:17.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.782+0000 7f1ce506c640 1 --2- 172.21.15.96:0/4155854145 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x55d1668def60 0x55d166cda440 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=24ed477718293693 server_cookie=8dbc672d61447895 in_seq=0 out_seq=0 2024-04-25T14:15:17.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.783+0000 7f1cc97fa640 1 -- 172.21.15.96:0/4155854145 <== mds.3 v2:172.21.15.97:6834/3872301726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d166c34990 con 0x55d1668def60 2024-04-25T14:15:17.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.783+0000 7f1cc97fa640 10 client.16110 handle_command_reply: tid=0 2024-04-25T14:15:17.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.783+0000 7f1cee97d740 1 client.16110 shutdown 2024-04-25T14:15:17.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.783+0000 7f1cee97d740 1 -- 172.21.15.96:0/4155854145 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1668f74d0 msgr2=0x55d166ccc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:17.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.783+0000 7f1cee97d740 1 --2- 172.21.15.96:0/4155854145 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1668f74d0 0x55d166ccc660 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc005f70 tx=0x7f1cdc02a410 comp rx=0 tx=0).stop 2024-04-25T14:15:17.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.783+0000 7f1cee97d740 1 -- 172.21.15.96:0/4155854145 shutdown_connections 2024-04-25T14:15:17.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.783+0000 7f1cee97d740 1 --2- 172.21.15.96:0/4155854145 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x55d1668def60 0x55d166cda440 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.783+0000 7f1cee97d740 1 --2- 172.21.15.96:0/4155854145 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d166920390 0x55d166cd3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.783+0000 7f1cee97d740 1 --2- 172.21.15.96:0/4155854145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1668ff6b0 0x55d166cccba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.783+0000 7f1cee97d740 1 --2- 172.21.15.96:0/4155854145 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1668f74d0 0x55d166ccc660 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.783+0000 7f1cc97fa640 0 client.16110 ms_handle_reset on v2:172.21.15.97:6834/3872301726 2024-04-25T14:15:17.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.783+0000 7f1cee97d740 1 -- 172.21.15.96:0/4155854145 >> 172.21.15.96:0/4155854145 conn(0x55d1669144e0 msgr2=0x55d166cbd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:17.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.783+0000 7f1cee97d740 1 -- 172.21.15.96:0/4155854145 shutdown_connections 2024-04-25T14:15:17.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.784+0000 7f1cee97d740 1 -- 172.21.15.96:0/4155854145 wait complete. 2024-04-25T14:15:17.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.784+0000 7f1cee97d740 20 client.16110 trim_cache size 0 max 16384 2024-04-25T14:15:17.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.797+0000 7f1cee97d740 1 Processor -- start 2024-04-25T14:15:17.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.797+0000 7f1cee97d740 1 -- start start 2024-04-25T14:15:17.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.798+0000 7f1cee97d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1669144e0 0x55d166ccc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.798+0000 7f1cee97d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1668f74d0 0x55d166cd3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.798+0000 7f1cee97d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1668ff6b0 0x55d166cbc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.798+0000 7f1cee97d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce01ab3f0 con 0x55d1669144e0 2024-04-25T14:15:17.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.798+0000 7f1cee97d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce01acf30 con 0x55d1668f74d0 2024-04-25T14:15:17.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.798+0000 7f1cee97d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce01abac0 con 0x55d1668ff6b0 2024-04-25T14:15:17.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.798+0000 7f1ce586d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1669144e0 0x55d166ccc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.798+0000 7f1ce586d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1669144e0 0x55d166ccc940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41530/0 (socket says 172.21.15.96:41530) 2024-04-25T14:15:17.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.798+0000 7f1ce586d640 1 -- 172.21.15.96:0/848871794 learned_addr learned my addr 172.21.15.96:0/848871794 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:17.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.798+0000 7f1ce506c640 1 --2- 172.21.15.96:0/848871794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1668f74d0 0x55d166cd3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.798+0000 7f1ce606e640 1 --2- 172.21.15.96:0/848871794 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1668ff6b0 0x55d166cbc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.798+0000 7f1ce586d640 1 -- 172.21.15.96:0/848871794 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1668ff6b0 msgr2=0x55d166cbc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:17.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.798+0000 7f1ce586d640 1 --2- 172.21.15.96:0/848871794 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1668ff6b0 0x55d166cbc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.798+0000 7f1ce586d640 1 -- 172.21.15.96:0/848871794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1668f74d0 msgr2=0x55d166cd3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:17.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.798+0000 7f1ce586d640 1 --2- 172.21.15.96:0/848871794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1668f74d0 0x55d166cd3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.798+0000 7f1ce586d640 1 -- 172.21.15.96:0/848871794 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d166cbebb0 con 0x55d1669144e0 2024-04-25T14:15:17.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.798+0000 7f1ce586d640 1 --2- 172.21.15.96:0/848871794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1669144e0 0x55d166ccc940 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc009690 tx=0x7f1cdc03de00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee4a8fc28921dc49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:17.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.799+0000 7f1ccb7fe640 1 -- 172.21.15.96:0/848871794 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc035030 con 0x55d1669144e0 2024-04-25T14:15:17.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.799+0000 7f1ccb7fe640 1 -- 172.21.15.96:0/848871794 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cdc05d070 con 0x55d1669144e0 2024-04-25T14:15:17.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.799+0000 7f1ccb7fe640 1 -- 172.21.15.96:0/848871794 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc031e60 con 0x55d1669144e0 2024-04-25T14:15:17.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.799+0000 7f1cee97d740 1 -- 172.21.15.96:0/848871794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1669144e0 msgr2=0x55d166ccc940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:17.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.799+0000 7f1cee97d740 1 --2- 172.21.15.96:0/848871794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1669144e0 0x55d166ccc940 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc009690 tx=0x7f1cdc03de00 comp rx=0 tx=0).stop 2024-04-25T14:15:17.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.799+0000 7f1cee97d740 1 -- 172.21.15.96:0/848871794 shutdown_connections 2024-04-25T14:15:17.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.799+0000 7f1cee97d740 1 --2- 172.21.15.96:0/848871794 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1668ff6b0 0x55d166cbc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.799+0000 7f1cee97d740 1 --2- 172.21.15.96:0/848871794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1668f74d0 0x55d166cd3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.799+0000 7f1cee97d740 1 --2- 172.21.15.96:0/848871794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1669144e0 0x55d166ccc940 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.799+0000 7f1cee97d740 1 -- 172.21.15.96:0/848871794 >> 172.21.15.96:0/848871794 conn(0x55d166cbe1b0 msgr2=0x55d166ccc230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:17.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.800+0000 7f1cee97d740 1 -- 172.21.15.96:0/848871794 shutdown_connections 2024-04-25T14:15:17.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.800+0000 7f1cee97d740 1 -- 172.21.15.96:0/848871794 wait complete. 2024-04-25T14:15:17.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.800+0000 7f1cee97d740 1 -- start start 2024-04-25T14:15:17.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.800+0000 7f1cee97d740 1 Processor -- start 2024-04-25T14:15:17.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.800+0000 7f1cee97d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1668f74d0 0x55d166cd1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.800+0000 7f1cee97d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1668ff6b0 0x55d166cd1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.800+0000 7f1cee97d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1669144e0 0x55d166cd2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.800+0000 7f1cee97d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce01ac860 con 0x55d1669144e0 2024-04-25T14:15:17.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.800+0000 7f1cee97d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce01ac190 con 0x55d1668f74d0 2024-04-25T14:15:17.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.800+0000 7f1cee97d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce01a93d0 con 0x55d1668ff6b0 2024-04-25T14:15:17.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.800+0000 7f1cee97d740 10 client.0 _ll_register_callbacks cb 0x7f1cdc009580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:15:17.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.800+0000 7f1ce606e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1669144e0 0x55d166cd2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.800+0000 7f1ce586d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1668f74d0 0x55d166cd1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.801+0000 7f1ce606e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1669144e0 0x55d166cd2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41538/0 (socket says 172.21.15.96:41538) 2024-04-25T14:15:17.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.801+0000 7f1ce586d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1668f74d0 0x55d166cd1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:41912/0 (socket says 172.21.15.96:41912) 2024-04-25T14:15:17.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.801+0000 7f1ce606e640 1 -- 172.21.15.96:0/1352097075 learned_addr learned my addr 172.21.15.96:0/1352097075 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:17.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.801+0000 7f1ce506c640 1 --2- 172.21.15.96:0/1352097075 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1668ff6b0 0x55d166cd1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.801+0000 7f1ce606e640 1 -- 172.21.15.96:0/1352097075 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1668ff6b0 msgr2=0x55d166cd1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:17.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.801+0000 7f1ce606e640 1 --2- 172.21.15.96:0/1352097075 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1668ff6b0 0x55d166cd1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.801+0000 7f1ce606e640 1 -- 172.21.15.96:0/1352097075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1668f74d0 msgr2=0x55d166cd1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:17.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.801+0000 7f1ce606e640 1 --2- 172.21.15.96:0/1352097075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1668f74d0 0x55d166cd1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.801+0000 7f1ce606e640 1 -- 172.21.15.96:0/1352097075 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d166cd3070 con 0x55d1669144e0 2024-04-25T14:15:17.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.801+0000 7f1ce586d640 1 --2- 172.21.15.96:0/1352097075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1668f74d0 0x55d166cd1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:15:17.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.801+0000 7f1ce606e640 1 --2- 172.21.15.96:0/1352097075 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1669144e0 0x55d166cd2ab0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f1cd8010370 tx=0x7f1cd8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=315ee742d5c6d84f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:17.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.801+0000 7f1cc97fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:15:17.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.801+0000 7f1cc97fa640 1 -- 172.21.15.96:0/1352097075 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd802b060 con 0x55d1669144e0 2024-04-25T14:15:17.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.801+0000 7f1cc97fa640 1 -- 172.21.15.96:0/1352097075 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f1cd80052b0 con 0x55d1669144e0 2024-04-25T14:15:17.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.801+0000 7f1cee97d740 1 -- 172.21.15.96:0/1352097075 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cdc05d070 con 0x55d1669144e0 2024-04-25T14:15:17.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.801+0000 7f1cc97fa640 1 -- 172.21.15.96:0/1352097075 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1cd8016070 con 0x55d1669144e0 2024-04-25T14:15:17.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.801+0000 7f1cee97d740 10 client.16116 fetch_fsmap learned FSMap version 26 2024-04-25T14:15:17.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.801+0000 7f1cee97d740 1 -- 172.21.15.96:0/1352097075 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d166d42ad0 con 0x55d1669144e0 2024-04-25T14:15:17.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.801+0000 7f1cc97fa640 1 -- 172.21.15.96:0/1352097075 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cdc05d070 con 0x55d1669144e0 2024-04-25T14:15:17.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.802+0000 7f1cc97fa640 1 -- 172.21.15.96:0/1352097075 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1cd800f040 con 0x55d1669144e0 2024-04-25T14:15:17.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.802+0000 7f1cee97d740 10 client.16116 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:15:17.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.802+0000 7f1cee97d740 10 client.16116 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T14:15:17.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.802+0000 7f1cee97d740 20 client.16116 populate_metadata read hostname 'smithi096' 2024-04-25T14:15:17.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.802+0000 7f1cee97d740 1 --2- 172.21.15.96:0/1352097075 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x55d166920390 0x55d166d451e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.802+0000 7f1cee97d740 4 client.16116 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:15:17.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.802+0000 7f1cee97d740 1 -- 172.21.15.96:0/1352097075 --> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d166cc6f00 con 0x55d166920390 2024-04-25T14:15:17.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.802+0000 7f1ce586d640 1 --2- 172.21.15.96:0/1352097075 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x55d166920390 0x55d166d451e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.802+0000 7f1ce586d640 1 --2- 172.21.15.96:0/1352097075 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x55d166920390 0x55d166d451e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=db8db6777b5956bd server_cookie=92814af282bd2a1 in_seq=0 out_seq=0 2024-04-25T14:15:17.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.802+0000 7f1cc97fa640 10 client.16116 ms_handle_connect on v2:172.21.15.97:6834/3872301726 2024-04-25T14:15:17.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.802+0000 7f1cc97fa640 1 -- 172.21.15.96:0/1352097075 <== mds.3 v2:172.21.15.97:6834/3872301726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d166cc6f00 con 0x55d166920390 2024-04-25T14:15:17.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.802+0000 7f1cc97fa640 10 client.16116 handle_command_reply: tid=0 2024-04-25T14:15:17.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.802+0000 7f1cee97d740 1 client.16116 shutdown 2024-04-25T14:15:17.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.802+0000 7f1cee97d740 1 -- 172.21.15.96:0/1352097075 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1669144e0 msgr2=0x55d166cd2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:17.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.802+0000 7f1cee97d740 1 --2- 172.21.15.96:0/1352097075 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1669144e0 0x55d166cd2ab0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f1cd8010370 tx=0x7f1cd8003ba0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.803+0000 7f1cee97d740 1 -- 172.21.15.96:0/1352097075 shutdown_connections 2024-04-25T14:15:17.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.803+0000 7f1cee97d740 1 --2- 172.21.15.96:0/1352097075 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x55d166920390 0x55d166d451e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.803+0000 7f1cee97d740 1 --2- 172.21.15.96:0/1352097075 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1669144e0 0x55d166cd2ab0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.803+0000 7f1cee97d740 1 --2- 172.21.15.96:0/1352097075 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1668ff6b0 0x55d166cd1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.803+0000 7f1cee97d740 1 --2- 172.21.15.96:0/1352097075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1668f74d0 0x55d166cd1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.803+0000 7f1cc97fa640 0 client.16116 ms_handle_reset on v2:172.21.15.97:6834/3872301726 2024-04-25T14:15:17.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.803+0000 7f1cee97d740 1 -- 172.21.15.96:0/1352097075 >> 172.21.15.96:0/1352097075 conn(0x55d166cbe1b0 msgr2=0x55d166ccc1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:17.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.803+0000 7f1cee97d740 1 -- 172.21.15.96:0/1352097075 shutdown_connections 2024-04-25T14:15:17.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.803+0000 7f1cee97d740 1 -- 172.21.15.96:0/1352097075 wait complete. 2024-04-25T14:15:17.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.803+0000 7f1cee97d740 20 client.16116 trim_cache size 0 max 16384 2024-04-25T14:15:17.809 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-04-25T14:15:17.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.803+0000 7f1ce7ad1640 1 -- 172.21.15.96:0/3641870526 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1ccc0afaf0 msgr2=0x7f1ccc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:17.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.803+0000 7f1ce7ad1640 1 --2- 172.21.15.96:0/3641870526 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1ccc0afaf0 0x7f1ccc0b2000 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc009230 tx=0x7f1cdc031400 comp rx=0 tx=0).stop 2024-04-25T14:15:17.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.803+0000 7f1ce7ad1640 1 -- 172.21.15.96:0/3641870526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ce01070a0 msgr2=0x7f1ce019a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:17.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.803+0000 7f1ce7ad1640 1 --2- 172.21.15.96:0/3641870526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ce01070a0 0x7f1ce019a7e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f1cd000e9f0 tx=0x7f1cd000eeb0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.803+0000 7f1ce7ad1640 1 -- 172.21.15.96:0/3641870526 shutdown_connections 2024-04-25T14:15:17.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.803+0000 7f1ce7ad1640 1 --2- 172.21.15.96:0/3641870526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ce0109a00 0x7f1ce01a1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.803+0000 7f1ce7ad1640 1 --2- 172.21.15.96:0/3641870526 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1ccc0afaf0 0x7f1ccc0b2000 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.803+0000 7f1ce7ad1640 1 --2- 172.21.15.96:0/3641870526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ce01070a0 0x7f1ce019a7e0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.803+0000 7f1ce7ad1640 1 --2- 172.21.15.96:0/3641870526 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ce0101ef0 0x7f1ce019a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.803+0000 7f1ce7ad1640 1 -- 172.21.15.96:0/3641870526 >> 172.21.15.96:0/3641870526 conn(0x7f1ce00fd350 msgr2=0x7f1ce0105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:17.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.803+0000 7f1ce7ad1640 1 -- 172.21.15.96:0/3641870526 shutdown_connections 2024-04-25T14:15:17.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.803+0000 7f1ce7ad1640 1 -- 172.21.15.96:0/3641870526 wait complete. 2024-04-25T14:15:17.814 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:17 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/3324102411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:15:17.816 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T14:15:17.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:17 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/3324102411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:15:17.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.934+0000 7f9f3da42640 1 -- 172.21.15.96:0/1352212483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f381033c0 msgr2=0x7f9f381037d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:17.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.934+0000 7f9f3da42640 1 --2- 172.21.15.96:0/1352212483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f381033c0 0x7f9f381037d0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c009900 tx=0x7f9f2c02a410 comp rx=0 tx=0).stop 2024-04-25T14:15:17.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.934+0000 7f9f3da42640 1 -- 172.21.15.96:0/1352212483 shutdown_connections 2024-04-25T14:15:17.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.934+0000 7f9f3da42640 1 --2- 172.21.15.96:0/1352212483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f381080c0 0x7f9f3810a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.934+0000 7f9f3da42640 1 --2- 172.21.15.96:0/1352212483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f381033c0 0x7f9f381037d0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.934+0000 7f9f3da42640 1 --2- 172.21.15.96:0/1352212483 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f38102180 0x7f9f38102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.934+0000 7f9f3da42640 1 -- 172.21.15.96:0/1352212483 >> 172.21.15.96:0/1352212483 conn(0x7f9f380fd350 msgr2=0x7f9f380ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:17.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.934+0000 7f9f3da42640 1 -- 172.21.15.96:0/1352212483 shutdown_connections 2024-04-25T14:15:17.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.934+0000 7f9f3da42640 1 -- 172.21.15.96:0/1352212483 wait complete. 2024-04-25T14:15:17.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.935+0000 7f9f3da42640 1 Processor -- start 2024-04-25T14:15:17.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.935+0000 7f9f3da42640 1 -- start start 2024-04-25T14:15:17.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.935+0000 7f9f3da42640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f38102180 0x7f9f381a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.935+0000 7f9f3da42640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f381033c0 0x7f9f381a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.935+0000 7f9f3da42640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f381080c0 0x7f9f381a7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.935+0000 7f9f3da42640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3810cf60 con 0x7f9f381080c0 2024-04-25T14:15:17.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.935+0000 7f9f3da42640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3810cde0 con 0x7f9f381033c0 2024-04-25T14:15:17.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.935+0000 7f9f3da42640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3810d0e0 con 0x7f9f38102180 2024-04-25T14:15:17.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.935+0000 7f9f377fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f381080c0 0x7f9f381a7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.935+0000 7f9f367fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f381033c0 0x7f9f381a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.935+0000 7f9f377fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f381080c0 0x7f9f381a7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41552/0 (socket says 172.21.15.96:41552) 2024-04-25T14:15:17.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.935+0000 7f9f377fe640 1 -- 172.21.15.96:0/2968145063 learned_addr learned my addr 172.21.15.96:0/2968145063 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:17.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.935+0000 7f9f36ffd640 1 --2- 172.21.15.96:0/2968145063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f38102180 0x7f9f381a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.936+0000 7f9f367fc640 1 -- 172.21.15.96:0/2968145063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f38102180 msgr2=0x7f9f381a0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:17.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.936+0000 7f9f367fc640 1 --2- 172.21.15.96:0/2968145063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f38102180 0x7f9f381a0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.936+0000 7f9f367fc640 1 -- 172.21.15.96:0/2968145063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f381080c0 msgr2=0x7f9f381a7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:17.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.936+0000 7f9f367fc640 1 --2- 172.21.15.96:0/2968145063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f381080c0 0x7f9f381a7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.936+0000 7f9f367fc640 1 -- 172.21.15.96:0/2968145063 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f381a8350 con 0x7f9f381033c0 2024-04-25T14:15:17.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.936+0000 7f9f36ffd640 1 --2- 172.21.15.96:0/2968145063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f38102180 0x7f9f381a0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:15:17.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.936+0000 7f9f367fc640 1 --2- 172.21.15.96:0/2968145063 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f381033c0 0x7f9f381a0d50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c006010 tx=0x7f9f2c004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:17.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.936+0000 7f9f3ca40640 1 -- 172.21.15.96:0/2968145063 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c03f070 con 0x7f9f381033c0 2024-04-25T14:15:17.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.936+0000 7f9f3da42640 1 -- 172.21.15.96:0/2968145063 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f381a85e0 con 0x7f9f381033c0 2024-04-25T14:15:17.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.936+0000 7f9f3ca40640 1 -- 172.21.15.96:0/2968145063 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f2c031910 con 0x7f9f381033c0 2024-04-25T14:15:17.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.936+0000 7f9f3ca40640 1 -- 172.21.15.96:0/2968145063 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c035070 con 0x7f9f381033c0 2024-04-25T14:15:17.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.936+0000 7f9f3da42640 1 -- 172.21.15.96:0/2968145063 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f381a8ad0 con 0x7f9f381033c0 2024-04-25T14:15:17.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:17 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/3324102411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:15:17.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.939+0000 7f9f3ca40640 1 -- 172.21.15.96:0/2968145063 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f9f2c0037f0 con 0x7f9f381033c0 2024-04-25T14:15:17.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.939+0000 7f9f449cf740 1 Processor -- start 2024-04-25T14:15:17.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.939+0000 7f9f449cf740 1 -- start start 2024-04-25T14:15:17.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.939+0000 7f9f449cf740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e88a5b390 0x559e88dfd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.939+0000 7f9f449cf740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559e88a324d0 0x559e88dfdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.939+0000 7f9f449cf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e88a3a6b0 0x559e88dfea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.939+0000 7f9f449cf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559e88d61d80 con 0x559e88a3a6b0 2024-04-25T14:15:17.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.939+0000 7f9f449cf740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559e88d1ba20 con 0x559e88a324d0 2024-04-25T14:15:17.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.939+0000 7f9f449cf740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559e88d5dd40 con 0x559e88a5b390 2024-04-25T14:15:17.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.939+0000 7f9f367fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e88a3a6b0 0x559e88dfea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.939+0000 7f9f377fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e88a5b390 0x559e88dfd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.939+0000 7f9f36ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559e88a324d0 0x559e88dfdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.939+0000 7f9f367fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e88a3a6b0 0x559e88dfea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41560/0 (socket says 172.21.15.96:41560) 2024-04-25T14:15:17.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.939+0000 7f9f367fc640 1 -- 172.21.15.96:0/4076763496 learned_addr learned my addr 172.21.15.96:0/4076763496 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:17.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.939+0000 7f9f3ca40640 1 --2- 172.21.15.96:0/2968145063 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f9f200afaf0 0x7f9f200b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.939+0000 7f9f367fc640 1 -- 172.21.15.96:0/4076763496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e88a5b390 msgr2=0x559e88dfd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:17.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.939+0000 7f9f367fc640 1 --2- 172.21.15.96:0/4076763496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e88a5b390 0x559e88dfd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.939+0000 7f9f367fc640 1 -- 172.21.15.96:0/4076763496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559e88a324d0 msgr2=0x559e88dfdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:17.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.939+0000 7f9f367fc640 1 --2- 172.21.15.96:0/4076763496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559e88a324d0 0x559e88dfdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.939+0000 7f9f367fc640 1 -- 172.21.15.96:0/4076763496 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e88d26c30 con 0x559e88a3a6b0 2024-04-25T14:15:17.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.939+0000 7f9f36ffd640 1 --2- 172.21.15.96:0/4076763496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559e88a324d0 0x559e88dfdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:15:17.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.939+0000 7f9f377fe640 1 --2- 172.21.15.96:0/4076763496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e88a5b390 0x559e88dfd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:15:17.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.939+0000 7f9f377fe640 1 --2- 172.21.15.96:0/2968145063 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f9f200afaf0 0x7f9f200b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.940+0000 7f9f3ca40640 1 -- 172.21.15.96:0/2968145063 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f9f2c0ff100 con 0x7f9f381033c0 2024-04-25T14:15:17.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.940+0000 7f9f367fc640 1 --2- 172.21.15.96:0/4076763496 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e88a3a6b0 0x559e88dfea20 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c04ade0 tx=0x7f9f2c049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1b4a50a8eaf7c21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:17.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.940+0000 7f9f1dffb640 1 -- 172.21.15.96:0/4076763496 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c0c7440 con 0x559e88a3a6b0 2024-04-25T14:15:17.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.940+0000 7f9f377fe640 1 --2- 172.21.15.96:0/2968145063 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f9f200afaf0 0x7f9f200b2000 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x559e88dfe860 tx=0x7f9f30004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:17.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.940+0000 7f9f1dffb640 1 -- 172.21.15.96:0/4076763496 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f2c042360 con 0x559e88a3a6b0 2024-04-25T14:15:17.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.940+0000 7f9f1dffb640 1 -- 172.21.15.96:0/4076763496 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c0425b0 con 0x559e88a3a6b0 2024-04-25T14:15:17.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.941+0000 7f9f449cf740 1 -- 172.21.15.96:0/4076763496 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e88a3a6b0 msgr2=0x559e88dfea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:17.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.941+0000 7f9f449cf740 1 --2- 172.21.15.96:0/4076763496 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e88a3a6b0 0x559e88dfea20 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c04ade0 tx=0x7f9f2c049000 comp rx=0 tx=0).stop 2024-04-25T14:15:17.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.941+0000 7f9f449cf740 1 -- 172.21.15.96:0/4076763496 shutdown_connections 2024-04-25T14:15:17.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.941+0000 7f9f449cf740 1 --2- 172.21.15.96:0/4076763496 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e88a3a6b0 0x559e88dfea20 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.941+0000 7f9f449cf740 1 --2- 172.21.15.96:0/4076763496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559e88a324d0 0x559e88dfdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.941+0000 7f9f449cf740 1 --2- 172.21.15.96:0/4076763496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e88a5b390 0x559e88dfd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.941+0000 7f9f449cf740 1 -- 172.21.15.96:0/4076763496 >> 172.21.15.96:0/4076763496 conn(0x559e88a4f4e0 msgr2=0x559e88df9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:17.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.941+0000 7f9f449cf740 1 -- 172.21.15.96:0/4076763496 shutdown_connections 2024-04-25T14:15:17.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.941+0000 7f9f449cf740 1 -- 172.21.15.96:0/4076763496 wait complete. 2024-04-25T14:15:17.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.941+0000 7f9f449cf740 1 -- start start 2024-04-25T14:15:17.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.941+0000 7f9f449cf740 1 Processor -- start 2024-04-25T14:15:17.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.942+0000 7f9f449cf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e88a324d0 0x559e88e07230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.942+0000 7f9f449cf740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e88a3a6b0 0x559e88e07770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.942+0000 7f9f449cf740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559e88a5b390 0x559e88e0e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.942+0000 7f9f449cf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559e88dff2f0 con 0x559e88a324d0 2024-04-25T14:15:17.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.942+0000 7f9f449cf740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559e88ce8dc0 con 0x559e88a5b390 2024-04-25T14:15:17.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.942+0000 7f9f449cf740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559e88dff470 con 0x559e88a3a6b0 2024-04-25T14:15:17.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.942+0000 7f9f449cf740 10 client.0 _ll_register_callbacks cb 0x559e88d04600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:15:17.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.942+0000 7f9f36ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e88a324d0 0x559e88e07230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.942+0000 7f9f36ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e88a324d0 0x559e88e07230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41572/0 (socket says 172.21.15.96:41572) 2024-04-25T14:15:17.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.942+0000 7f9f36ffd640 1 -- 172.21.15.96:0/1987309068 learned_addr learned my addr 172.21.15.96:0/1987309068 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:17.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.942+0000 7f9f367fc640 1 --2- 172.21.15.96:0/1987309068 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e88a3a6b0 0x559e88e07770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.942+0000 7f9f377fe640 1 --2- 172.21.15.96:0/1987309068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559e88a5b390 0x559e88e0e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.942+0000 7f9f36ffd640 1 -- 172.21.15.96:0/1987309068 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e88a3a6b0 msgr2=0x559e88e07770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:17.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.942+0000 7f9f36ffd640 1 --2- 172.21.15.96:0/1987309068 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e88a3a6b0 0x559e88e07770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.942+0000 7f9f36ffd640 1 -- 172.21.15.96:0/1987309068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559e88a5b390 msgr2=0x559e88e0e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:17.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.942+0000 7f9f36ffd640 1 --2- 172.21.15.96:0/1987309068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559e88a5b390 0x559e88e0e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.942+0000 7f9f36ffd640 1 -- 172.21.15.96:0/1987309068 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e88a733f0 con 0x559e88a324d0 2024-04-25T14:15:17.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.942+0000 7f9f36ffd640 1 --2- 172.21.15.96:0/1987309068 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e88a324d0 0x559e88e07230 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f9f28012d20 tx=0x7f9f2800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b56b85ca7a1db8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:17.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.942+0000 7f9f16ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:15:17.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.942+0000 7f9f16ffd640 1 -- 172.21.15.96:0/1987309068 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2800fe60 con 0x559e88a324d0 2024-04-25T14:15:17.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.942+0000 7f9f449cf740 1 -- 172.21.15.96:0/1987309068 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559e88d6f990 con 0x559e88a324d0 2024-04-25T14:15:17.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.942+0000 7f9f16ffd640 1 -- 172.21.15.96:0/1987309068 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f9f280120d0 con 0x559e88a324d0 2024-04-25T14:15:17.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.942+0000 7f9f16ffd640 1 -- 172.21.15.96:0/1987309068 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9f28004030 con 0x559e88a324d0 2024-04-25T14:15:17.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.943+0000 7f9f449cf740 10 client.16128 fetch_fsmap learned FSMap version 26 2024-04-25T14:15:17.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.943+0000 7f9f449cf740 1 -- 172.21.15.96:0/1987309068 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559e88b3d7d0 con 0x559e88a324d0 2024-04-25T14:15:17.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.943+0000 7f9f16ffd640 1 -- 172.21.15.96:0/1987309068 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559e88d6f990 con 0x559e88a324d0 2024-04-25T14:15:17.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.943+0000 7f9f16ffd640 1 -- 172.21.15.96:0/1987309068 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9f28010430 con 0x559e88a324d0 2024-04-25T14:15:17.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.943+0000 7f9f449cf740 10 client.16128 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:15:17.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.943+0000 7f9f449cf740 10 client.16128 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:15:17.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.943+0000 7f9f449cf740 20 client.16128 populate_metadata read hostname 'smithi096' 2024-04-25T14:15:17.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.943+0000 7f9f449cf740 1 --2- 172.21.15.96:0/1987309068 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x559e88a19f60 0x559e88e15390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.943+0000 7f9f449cf740 4 client.16128 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:15:17.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.943+0000 7f9f449cf740 1 -- 172.21.15.96:0/1987309068 --> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e88d69800 con 0x559e88a19f60 2024-04-25T14:15:17.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.943+0000 7f9f367fc640 1 --2- 172.21.15.96:0/1987309068 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x559e88a19f60 0x559e88e15390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.943+0000 7f9f16ffd640 10 client.16128 ms_handle_connect on v2:172.21.15.151:6834/1313823638 2024-04-25T14:15:17.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.944+0000 7f9f367fc640 1 --2- 172.21.15.96:0/1987309068 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x559e88a19f60 0x559e88e15390 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=27f7e61d686a4953 server_cookie=919caf3e44fa0479 in_seq=0 out_seq=0 2024-04-25T14:15:17.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.944+0000 7f9f16ffd640 1 -- 172.21.15.96:0/1987309068 <== mds.2 v2:172.21.15.151:6834/1313823638 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559e88d69800 con 0x559e88a19f60 2024-04-25T14:15:17.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.945+0000 7f9f16ffd640 10 client.16128 handle_command_reply: tid=0 2024-04-25T14:15:17.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.945+0000 7f9f449cf740 1 client.16128 shutdown 2024-04-25T14:15:17.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.945+0000 7f9f449cf740 1 -- 172.21.15.96:0/1987309068 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e88a324d0 msgr2=0x559e88e07230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:17.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.945+0000 7f9f449cf740 1 --2- 172.21.15.96:0/1987309068 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e88a324d0 0x559e88e07230 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f9f28012d20 tx=0x7f9f2800f790 comp rx=0 tx=0).stop 2024-04-25T14:15:17.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.945+0000 7f9f449cf740 1 -- 172.21.15.96:0/1987309068 shutdown_connections 2024-04-25T14:15:17.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.945+0000 7f9f449cf740 1 --2- 172.21.15.96:0/1987309068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559e88a5b390 0x559e88e0e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.945+0000 7f9f449cf740 1 --2- 172.21.15.96:0/1987309068 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e88a3a6b0 0x559e88e07770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.945+0000 7f9f449cf740 1 --2- 172.21.15.96:0/1987309068 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x559e88a19f60 0x559e88e15390 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.945+0000 7f9f449cf740 1 --2- 172.21.15.96:0/1987309068 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e88a324d0 0x559e88e07230 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.945+0000 7f9f16ffd640 0 client.16128 ms_handle_reset on v2:172.21.15.151:6834/1313823638 2024-04-25T14:15:17.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.945+0000 7f9f449cf740 1 -- 172.21.15.96:0/1987309068 >> 172.21.15.96:0/1987309068 conn(0x559e88a4f4e0 msgr2=0x559e88dfbf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:17.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.945+0000 7f9f449cf740 1 -- 172.21.15.96:0/1987309068 shutdown_connections 2024-04-25T14:15:17.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.945+0000 7f9f449cf740 1 -- 172.21.15.96:0/1987309068 wait complete. 2024-04-25T14:15:17.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.945+0000 7f9f449cf740 20 client.16128 trim_cache size 0 max 16384 2024-04-25T14:15:17.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.959+0000 7f9f449cf740 1 Processor -- start 2024-04-25T14:15:17.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.959+0000 7f9f449cf740 1 -- start start 2024-04-25T14:15:17.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.959+0000 7f9f449cf740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559e88a324d0 0x559e88df8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.959+0000 7f9f449cf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e88a3a6b0 0x559e88e0e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.959+0000 7f9f449cf740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e88a5b390 0x559e88df8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.959+0000 7f9f449cf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9f381b18f0 con 0x559e88a3a6b0 2024-04-25T14:15:17.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.959+0000 7f9f449cf740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9f381b3430 con 0x559e88a324d0 2024-04-25T14:15:17.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.959+0000 7f9f449cf740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9f381b1fc0 con 0x559e88a5b390 2024-04-25T14:15:17.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.960+0000 7f9f36ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559e88a324d0 0x559e88df8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.960+0000 7f9f367fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e88a3a6b0 0x559e88e0e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.960+0000 7f9f36ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559e88a324d0 0x559e88df8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:41962/0 (socket says 172.21.15.96:41962) 2024-04-25T14:15:17.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.960+0000 7f9f36ffd640 1 -- 172.21.15.96:0/2873363062 learned_addr learned my addr 172.21.15.96:0/2873363062 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:17.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.960+0000 7f9f367fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e88a3a6b0 0x559e88e0e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41588/0 (socket says 172.21.15.96:41588) 2024-04-25T14:15:17.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.960+0000 7f9f377fe640 1 --2- 172.21.15.96:0/2873363062 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e88a5b390 0x559e88df8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.960+0000 7f9f36ffd640 1 -- 172.21.15.96:0/2873363062 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e88a5b390 msgr2=0x559e88df8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:17.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.960+0000 7f9f36ffd640 1 --2- 172.21.15.96:0/2873363062 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e88a5b390 0x559e88df8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.960+0000 7f9f36ffd640 1 -- 172.21.15.96:0/2873363062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e88a3a6b0 msgr2=0x559e88e0e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:17.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.960+0000 7f9f36ffd640 1 --2- 172.21.15.96:0/2873363062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e88a3a6b0 0x559e88e0e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.960+0000 7f9f36ffd640 1 -- 172.21.15.96:0/2873363062 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e88e02b40 con 0x559e88a324d0 2024-04-25T14:15:17.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.960+0000 7f9f367fc640 1 --2- 172.21.15.96:0/2873363062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e88a3a6b0 0x559e88e0e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:15:17.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.960+0000 7f9f36ffd640 1 --2- 172.21.15.96:0/2873363062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559e88a324d0 0x559e88df8a50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9f28010850 tx=0x7f9f28009f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35c61ab26aedc98e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:17.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.960+0000 7f9f1d7fa640 1 -- 172.21.15.96:0/2873363062 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f280120d0 con 0x559e88a324d0 2024-04-25T14:15:17.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.960+0000 7f9f1d7fa640 1 -- 172.21.15.96:0/2873363062 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f28012270 con 0x559e88a324d0 2024-04-25T14:15:17.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.961+0000 7f9f1d7fa640 1 -- 172.21.15.96:0/2873363062 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f280035f0 con 0x559e88a324d0 2024-04-25T14:15:17.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.961+0000 7f9f449cf740 1 -- 172.21.15.96:0/2873363062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559e88a324d0 msgr2=0x559e88df8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:17.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.961+0000 7f9f449cf740 1 --2- 172.21.15.96:0/2873363062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559e88a324d0 0x559e88df8a50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9f28010850 tx=0x7f9f28009f80 comp rx=0 tx=0).stop 2024-04-25T14:15:17.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.961+0000 7f9f449cf740 1 -- 172.21.15.96:0/2873363062 shutdown_connections 2024-04-25T14:15:17.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.962+0000 7f9f449cf740 1 --2- 172.21.15.96:0/2873363062 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e88a5b390 0x559e88df8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.962+0000 7f9f449cf740 1 --2- 172.21.15.96:0/2873363062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e88a3a6b0 0x559e88e0e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.962+0000 7f9f449cf740 1 --2- 172.21.15.96:0/2873363062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559e88a324d0 0x559e88df8a50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.962+0000 7f9f449cf740 1 -- 172.21.15.96:0/2873363062 >> 172.21.15.96:0/2873363062 conn(0x559e88a4f4e0 msgr2=0x559e88e05850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:17.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.962+0000 7f9f449cf740 1 -- 172.21.15.96:0/2873363062 shutdown_connections 2024-04-25T14:15:17.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.962+0000 7f9f449cf740 1 -- 172.21.15.96:0/2873363062 wait complete. 2024-04-25T14:15:17.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.962+0000 7f9f449cf740 1 -- start start 2024-04-25T14:15:17.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.962+0000 7f9f449cf740 1 Processor -- start 2024-04-25T14:15:17.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.962+0000 7f9f449cf740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e88a324d0 0x559e88e0b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.962+0000 7f9f449cf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e88a3a6b0 0x559e88e0b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.962+0000 7f9f449cf740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559e88a5b390 0x559e88e7b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.962+0000 7f9f449cf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9f381b2d60 con 0x559e88a3a6b0 2024-04-25T14:15:17.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.962+0000 7f9f449cf740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9f381b2690 con 0x559e88a5b390 2024-04-25T14:15:17.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.962+0000 7f9f449cf740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9f381af8d0 con 0x559e88a324d0 2024-04-25T14:15:17.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.963+0000 7f9f449cf740 10 client.0 _ll_register_callbacks cb 0x7f9f28009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:15:17.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.963+0000 7f9f367fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e88a3a6b0 0x559e88e0b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.963+0000 7f9f36ffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e88a324d0 0x559e88e0b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.963+0000 7f9f367fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e88a3a6b0 0x559e88e0b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41594/0 (socket says 172.21.15.96:41594) 2024-04-25T14:15:17.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.963+0000 7f9f377fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559e88a5b390 0x559e88e7b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.963+0000 7f9f367fc640 1 -- 172.21.15.96:0/4249872865 learned_addr learned my addr 172.21.15.96:0/4249872865 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:17.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.963+0000 7f9f367fc640 1 -- 172.21.15.96:0/4249872865 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e88a324d0 msgr2=0x559e88e0b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:17.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.963+0000 7f9f367fc640 1 --2- 172.21.15.96:0/4249872865 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e88a324d0 0x559e88e0b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.963+0000 7f9f367fc640 1 -- 172.21.15.96:0/4249872865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559e88a5b390 msgr2=0x559e88e7b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:17.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.963+0000 7f9f367fc640 1 --2- 172.21.15.96:0/4249872865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559e88a5b390 0x559e88e7b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.963+0000 7f9f367fc640 1 -- 172.21.15.96:0/4249872865 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e88e0eb30 con 0x559e88a3a6b0 2024-04-25T14:15:17.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.963+0000 7f9f367fc640 1 --2- 172.21.15.96:0/4249872865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e88a3a6b0 0x559e88e0b990 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c0fcc30 tx=0x7f9f2c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9845b2702e778dc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:17.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.963+0000 7f9f16ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:15:17.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.963+0000 7f9f16ffd640 1 -- 172.21.15.96:0/4249872865 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c0a0e60 con 0x559e88a3a6b0 2024-04-25T14:15:17.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.963+0000 7f9f16ffd640 1 -- 172.21.15.96:0/4249872865 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f9f2c03b460 con 0x559e88a3a6b0 2024-04-25T14:15:17.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.963+0000 7f9f449cf740 1 -- 172.21.15.96:0/4249872865 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f28012270 con 0x559e88a3a6b0 2024-04-25T14:15:17.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.963+0000 7f9f16ffd640 1 -- 172.21.15.96:0/4249872865 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9f2c0c6de0 con 0x559e88a3a6b0 2024-04-25T14:15:17.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.963+0000 7f9f449cf740 10 client.16134 fetch_fsmap learned FSMap version 26 2024-04-25T14:15:17.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.963+0000 7f9f449cf740 1 -- 172.21.15.96:0/4249872865 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559e88e7cb60 con 0x559e88a3a6b0 2024-04-25T14:15:17.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.964+0000 7f9f16ffd640 1 -- 172.21.15.96:0/4249872865 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f28012270 con 0x559e88a3a6b0 2024-04-25T14:15:17.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.964+0000 7f9f16ffd640 1 -- 172.21.15.96:0/4249872865 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9f2c04b060 con 0x559e88a3a6b0 2024-04-25T14:15:17.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.964+0000 7f9f449cf740 10 client.16134 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:15:17.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.964+0000 7f9f449cf740 10 client.16134 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:15:17.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.964+0000 7f9f449cf740 20 client.16134 populate_metadata read hostname 'smithi096' 2024-04-25T14:15:17.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.964+0000 7f9f449cf740 1 --2- 172.21.15.96:0/4249872865 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x559e889ef3a0 0x559e88e7f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.964+0000 7f9f449cf740 4 client.16134 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:15:17.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.964+0000 7f9f449cf740 1 -- 172.21.15.96:0/4249872865 --> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559e88ce4610 con 0x559e889ef3a0 2024-04-25T14:15:17.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.964+0000 7f9f36ffd640 1 --2- 172.21.15.96:0/4249872865 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x559e889ef3a0 0x559e88e7f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.964+0000 7f9f16ffd640 10 client.16134 ms_handle_connect on v2:172.21.15.151:6834/1313823638 2024-04-25T14:15:17.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.965+0000 7f9f36ffd640 1 --2- 172.21.15.96:0/4249872865 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x559e889ef3a0 0x559e88e7f3f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=437cee43f00ab5d5 server_cookie=29c55d0711a8272b in_seq=0 out_seq=0 2024-04-25T14:15:17.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.965+0000 7f9f16ffd640 1 -- 172.21.15.96:0/4249872865 <== mds.2 v2:172.21.15.151:6834/1313823638 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559e88ce4610 con 0x559e889ef3a0 2024-04-25T14:15:17.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.965+0000 7f9f16ffd640 10 client.16134 handle_command_reply: tid=0 2024-04-25T14:15:17.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.965+0000 7f9f449cf740 1 client.16134 shutdown 2024-04-25T14:15:17.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.965+0000 7f9f449cf740 1 -- 172.21.15.96:0/4249872865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e88a3a6b0 msgr2=0x559e88e0b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:17.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.965+0000 7f9f449cf740 1 --2- 172.21.15.96:0/4249872865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e88a3a6b0 0x559e88e0b990 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c0fcc30 tx=0x7f9f2c000bc0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.965+0000 7f9f449cf740 1 -- 172.21.15.96:0/4249872865 shutdown_connections 2024-04-25T14:15:17.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.965+0000 7f9f449cf740 1 --2- 172.21.15.96:0/4249872865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559e88a5b390 0x559e88e7b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.965+0000 7f9f449cf740 1 --2- 172.21.15.96:0/4249872865 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x559e889ef3a0 0x559e88e7f3f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.965+0000 7f9f16ffd640 0 client.16134 ms_handle_reset on v2:172.21.15.151:6834/1313823638 2024-04-25T14:15:17.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.965+0000 7f9f449cf740 1 --2- 172.21.15.96:0/4249872865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e88a3a6b0 0x559e88e0b990 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.965+0000 7f9f449cf740 1 --2- 172.21.15.96:0/4249872865 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e88a324d0 0x559e88e0b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.965+0000 7f9f449cf740 1 -- 172.21.15.96:0/4249872865 >> 172.21.15.96:0/4249872865 conn(0x559e88a4f4e0 msgr2=0x559e88e05810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:17.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.965+0000 7f9f449cf740 1 -- 172.21.15.96:0/4249872865 shutdown_connections 2024-04-25T14:15:17.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.965+0000 7f9f449cf740 1 -- 172.21.15.96:0/4249872865 wait complete. 2024-04-25T14:15:17.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.965+0000 7f9f449cf740 20 client.16134 trim_cache size 0 max 16384 2024-04-25T14:15:17.971 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-04-25T14:15:17.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.966+0000 7f9f3da42640 1 -- 172.21.15.96:0/2968145063 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f9f200afaf0 msgr2=0x7f9f200b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:17.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.966+0000 7f9f3da42640 1 --2- 172.21.15.96:0/2968145063 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f9f200afaf0 0x7f9f200b2000 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x559e88dfe860 tx=0x7f9f30004a40 comp rx=0 tx=0).stop 2024-04-25T14:15:17.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.966+0000 7f9f3da42640 1 -- 172.21.15.96:0/2968145063 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f381033c0 msgr2=0x7f9f381a0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:17.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.966+0000 7f9f3da42640 1 --2- 172.21.15.96:0/2968145063 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f381033c0 0x7f9f381a0d50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c006010 tx=0x7f9f2c004270 comp rx=0 tx=0).stop 2024-04-25T14:15:17.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.966+0000 7f9f3da42640 1 -- 172.21.15.96:0/2968145063 shutdown_connections 2024-04-25T14:15:17.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.966+0000 7f9f3da42640 1 --2- 172.21.15.96:0/2968145063 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f9f200afaf0 0x7f9f200b2000 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.966+0000 7f9f3da42640 1 --2- 172.21.15.96:0/2968145063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f381080c0 0x7f9f381a7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.966+0000 7f9f3da42640 1 --2- 172.21.15.96:0/2968145063 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f381033c0 0x7f9f381a0d50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.966+0000 7f9f3da42640 1 --2- 172.21.15.96:0/2968145063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f38102180 0x7f9f381a0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.966+0000 7f9f3da42640 1 -- 172.21.15.96:0/2968145063 >> 172.21.15.96:0/2968145063 conn(0x7f9f380fd350 msgr2=0x7f9f380ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:17.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.966+0000 7f9f3da42640 1 -- 172.21.15.96:0/2968145063 shutdown_connections 2024-04-25T14:15:17.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:17.966+0000 7f9f3da42640 1 -- 172.21.15.96:0/2968145063 wait complete. 2024-04-25T14:15:17.978 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T14:15:18.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.098+0000 7f0c58396640 1 -- 172.21.15.96:0/4040292108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c50108930 msgr2=0x7f0c5010ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:18.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.098+0000 7f0c58396640 1 --2- 172.21.15.96:0/4040292108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c50108930 0x7f0c5010ad50 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f0c48009900 tx=0x7f0c4802a410 comp rx=0 tx=0).stop 2024-04-25T14:15:18.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.098+0000 7f0c58396640 1 -- 172.21.15.96:0/4040292108 shutdown_connections 2024-04-25T14:15:18.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.098+0000 7f0c58396640 1 --2- 172.21.15.96:0/4040292108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c50108930 0x7f0c5010ad50 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.098+0000 7f0c58396640 1 --2- 172.21.15.96:0/4040292108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c50103c30 0x7f0c50104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.098+0000 7f0c58396640 1 --2- 172.21.15.96:0/4040292108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c501029f0 0x7f0c501036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.098+0000 7f0c58396640 1 -- 172.21.15.96:0/4040292108 >> 172.21.15.96:0/4040292108 conn(0x7f0c500fdc00 msgr2=0x7f0c50100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:18.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.098+0000 7f0c58396640 1 -- 172.21.15.96:0/4040292108 shutdown_connections 2024-04-25T14:15:18.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.098+0000 7f0c58396640 1 -- 172.21.15.96:0/4040292108 wait complete. 2024-04-25T14:15:18.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.099+0000 7f0c58396640 1 Processor -- start 2024-04-25T14:15:18.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.099+0000 7f0c58396640 1 -- start start 2024-04-25T14:15:18.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.099+0000 7f0c58396640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c501029f0 0x7f0c501990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:18.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.099+0000 7f0c58396640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c50103c30 0x7f0c50199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:18.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.099+0000 7f0c58396640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c50108930 0x7f0c501a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:18.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.099+0000 7f0c58396640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0c5010e520 con 0x7f0c50103c30 2024-04-25T14:15:18.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.099+0000 7f0c58396640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0c5010e3a0 con 0x7f0c50108930 2024-04-25T14:15:18.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.099+0000 7f0c58396640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0c5010e6a0 con 0x7f0c501029f0 2024-04-25T14:15:18.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.099+0000 7f0c55931640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c50103c30 0x7f0c50199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:18.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.099+0000 7f0c55931640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c50103c30 0x7f0c50199610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41606/0 (socket says 172.21.15.96:41606) 2024-04-25T14:15:18.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.099+0000 7f0c55931640 1 -- 172.21.15.96:0/1840200291 learned_addr learned my addr 172.21.15.96:0/1840200291 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:18.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.099+0000 7f0c56132640 1 --2- 172.21.15.96:0/1840200291 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c501029f0 0x7f0c501990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:18.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.099+0000 7f0c56933640 1 --2- 172.21.15.96:0/1840200291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c50108930 0x7f0c501a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:18.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.100+0000 7f0c55931640 1 -- 172.21.15.96:0/1840200291 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c501029f0 msgr2=0x7f0c501990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:18.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.100+0000 7f0c55931640 1 --2- 172.21.15.96:0/1840200291 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c501029f0 0x7f0c501990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.100+0000 7f0c55931640 1 -- 172.21.15.96:0/1840200291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c50108930 msgr2=0x7f0c501a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:18.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.100+0000 7f0c55931640 1 --2- 172.21.15.96:0/1840200291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c50108930 0x7f0c501a05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.100+0000 7f0c55931640 1 -- 172.21.15.96:0/1840200291 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c501a0c10 con 0x7f0c50103c30 2024-04-25T14:15:18.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.100+0000 7f0c55931640 1 --2- 172.21.15.96:0/1840200291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c50103c30 0x7f0c50199610 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f0c4000e9f0 tx=0x7f0c4000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:18.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.100+0000 7f0c477fe640 1 -- 172.21.15.96:0/1840200291 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4000cd80 con 0x7f0c50103c30 2024-04-25T14:15:18.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.100+0000 7f0c477fe640 1 -- 172.21.15.96:0/1840200291 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c400108e0 con 0x7f0c50103c30 2024-04-25T14:15:18.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.100+0000 7f0c58396640 1 -- 172.21.15.96:0/1840200291 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c501a0f00 con 0x7f0c50103c30 2024-04-25T14:15:18.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.100+0000 7f0c477fe640 1 -- 172.21.15.96:0/1840200291 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c40018980 con 0x7f0c50103c30 2024-04-25T14:15:18.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.100+0000 7f0c58396640 1 -- 172.21.15.96:0/1840200291 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c501a1370 con 0x7f0c50103c30 2024-04-25T14:15:18.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.103+0000 7f0c5f24a740 1 Processor -- start 2024-04-25T14:15:18.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.103+0000 7f0c477fe640 1 -- 172.21.15.96:0/1840200291 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f0c40010430 con 0x7f0c50103c30 2024-04-25T14:15:18.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.103+0000 7f0c5f24a740 1 -- start start 2024-04-25T14:15:18.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.103+0000 7f0c5f24a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c5bfbd390 0x558c5c35f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:18.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.103+0000 7f0c5f24a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c5bf944d0 0x558c5c35fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:18.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.103+0000 7f0c5f24a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c5bf9c6b0 0x558c5c360a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:18.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.103+0000 7f0c5f24a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558c5c2c3d80 con 0x558c5bfbd390 2024-04-25T14:15:18.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.103+0000 7f0c5f24a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558c5c27da20 con 0x558c5bf9c6b0 2024-04-25T14:15:18.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.103+0000 7f0c5f24a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558c5c2bfd40 con 0x558c5bf944d0 2024-04-25T14:15:18.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.103+0000 7f0c56933640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c5bfbd390 0x558c5c35f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:18.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.103+0000 7f0c56933640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c5bfbd390 0x558c5c35f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41622/0 (socket says 172.21.15.96:41622) 2024-04-25T14:15:18.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.103+0000 7f0c55931640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c5bf9c6b0 0x558c5c360a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:18.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.103+0000 7f0c56933640 1 -- 172.21.15.96:0/2788820600 learned_addr learned my addr 172.21.15.96:0/2788820600 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:18.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.103+0000 7f0c56132640 1 --2- 172.21.15.96:0/2788820600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c5bf944d0 0x558c5c35fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:18.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.104+0000 7f0c56933640 1 -- 172.21.15.96:0/2788820600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c5bf944d0 msgr2=0x558c5c35fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:18.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.104+0000 7f0c477fe640 1 --2- 172.21.15.96:0/1840200291 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f0c3c0afaf0 0x7f0c3c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:18.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.104+0000 7f0c56933640 1 --2- 172.21.15.96:0/2788820600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c5bf944d0 0x558c5c35fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.104+0000 7f0c56933640 1 -- 172.21.15.96:0/2788820600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c5bf9c6b0 msgr2=0x558c5c360a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:18.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.104+0000 7f0c56933640 1 --2- 172.21.15.96:0/2788820600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c5bf9c6b0 0x558c5c360a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.104+0000 7f0c56933640 1 -- 172.21.15.96:0/2788820600 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c5c288c30 con 0x558c5bfbd390 2024-04-25T14:15:18.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.104+0000 7f0c56132640 1 --2- 172.21.15.96:0/2788820600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c5bf944d0 0x558c5c35fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:15:18.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.104+0000 7f0c477fe640 1 -- 172.21.15.96:0/1840200291 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f0c400def10 con 0x7f0c50103c30 2024-04-25T14:15:18.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.104+0000 7f0c56933640 1 --2- 172.21.15.96:0/1840200291 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f0c3c0afaf0 0x7f0c3c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:18.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.104+0000 7f0c56933640 1 --2- 172.21.15.96:0/2788820600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c5bfbd390 0x558c5c35f810 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f0c48009900 tx=0x7f0c48009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1971128c372d9ddd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:18.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.104+0000 7f0c44bfe640 1 -- 172.21.15.96:0/2788820600 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4804d070 con 0x558c5bfbd390 2024-04-25T14:15:18.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.104+0000 7f0c44bfe640 1 -- 172.21.15.96:0/2788820600 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c48040070 con 0x558c5bfbd390 2024-04-25T14:15:18.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.104+0000 7f0c44bfe640 1 -- 172.21.15.96:0/2788820600 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c48035030 con 0x558c5bfbd390 2024-04-25T14:15:18.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.105+0000 7f0c5f24a740 1 -- 172.21.15.96:0/2788820600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c5bfbd390 msgr2=0x558c5c35f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:18.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.105+0000 7f0c56933640 1 --2- 172.21.15.96:0/1840200291 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f0c3c0afaf0 0x7f0c3c0b2000 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0c48031fd0 tx=0x7f0c4804b000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:18.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.105+0000 7f0c5f24a740 1 --2- 172.21.15.96:0/2788820600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c5bfbd390 0x558c5c35f810 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f0c48009900 tx=0x7f0c48009620 comp rx=0 tx=0).stop 2024-04-25T14:15:18.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.105+0000 7f0c5f24a740 1 -- 172.21.15.96:0/2788820600 shutdown_connections 2024-04-25T14:15:18.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.105+0000 7f0c5f24a740 1 --2- 172.21.15.96:0/2788820600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c5bf9c6b0 0x558c5c360a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.105+0000 7f0c5f24a740 1 --2- 172.21.15.96:0/2788820600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c5bf944d0 0x558c5c35fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.105+0000 7f0c5f24a740 1 --2- 172.21.15.96:0/2788820600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c5bfbd390 0x558c5c35f810 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.105+0000 7f0c5f24a740 1 -- 172.21.15.96:0/2788820600 >> 172.21.15.96:0/2788820600 conn(0x558c5bfb14e0 msgr2=0x558c5c35b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:18.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.106+0000 7f0c5f24a740 1 -- 172.21.15.96:0/2788820600 shutdown_connections 2024-04-25T14:15:18.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.106+0000 7f0c5f24a740 1 -- 172.21.15.96:0/2788820600 wait complete. 2024-04-25T14:15:18.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.106+0000 7f0c5f24a740 1 -- start start 2024-04-25T14:15:18.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.106+0000 7f0c5f24a740 1 Processor -- start 2024-04-25T14:15:18.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.106+0000 7f0c5f24a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c5bf944d0 0x558c5c3695c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:18.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.106+0000 7f0c5f24a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c5bf9c6b0 0x558c5c369b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:18.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.106+0000 7f0c5f24a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c5bfbd390 0x558c5c370a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:18.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.106+0000 7f0c5f24a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558c5c361530 con 0x558c5bf944d0 2024-04-25T14:15:18.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.106+0000 7f0c5f24a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558c5c24adc0 con 0x558c5bfbd390 2024-04-25T14:15:18.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.106+0000 7f0c5f24a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558c5c3616b0 con 0x558c5bf9c6b0 2024-04-25T14:15:18.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.106+0000 7f0c5f24a740 10 client.0 _ll_register_callbacks cb 0x558c5c266600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:15:18.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.106+0000 7f0c56132640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c5bf944d0 0x558c5c3695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:18.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.106+0000 7f0c56132640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c5bf944d0 0x558c5c3695c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41626/0 (socket says 172.21.15.96:41626) 2024-04-25T14:15:18.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.106+0000 7f0c56132640 1 -- 172.21.15.96:0/3116197769 learned_addr learned my addr 172.21.15.96:0/3116197769 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:18.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.106+0000 7f0c55931640 1 --2- 172.21.15.96:0/3116197769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c5bf9c6b0 0x558c5c369b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:18.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.107+0000 7f0c56132640 1 -- 172.21.15.96:0/3116197769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c5bf9c6b0 msgr2=0x558c5c369b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:18.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.107+0000 7f0c56933640 1 --2- 172.21.15.96:0/3116197769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c5bfbd390 0x558c5c370a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:18.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.107+0000 7f0c56132640 1 --2- 172.21.15.96:0/3116197769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c5bf9c6b0 0x558c5c369b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.107+0000 7f0c56132640 1 -- 172.21.15.96:0/3116197769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c5bfbd390 msgr2=0x558c5c370a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:18.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.107+0000 7f0c56132640 1 --2- 172.21.15.96:0/3116197769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c5bfbd390 0x558c5c370a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.107+0000 7f0c56132640 1 -- 172.21.15.96:0/3116197769 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c5bfd53f0 con 0x558c5bf944d0 2024-04-25T14:15:18.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.107+0000 7f0c55931640 1 --2- 172.21.15.96:0/3116197769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c5bf9c6b0 0x558c5c369b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:15:18.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.107+0000 7f0c56933640 1 --2- 172.21.15.96:0/3116197769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c5bfbd390 0x558c5c370a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:15:18.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.107+0000 7f0c56132640 1 --2- 172.21.15.96:0/3116197769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c5bf944d0 0x558c5c3695c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c011ce0 tx=0x7f0c4c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a98eb96d0c3701e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:18.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.107+0000 7f0c39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:15:18.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.107+0000 7f0c39ffb640 1 -- 172.21.15.96:0/3116197769 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c004b20 con 0x558c5bf944d0 2024-04-25T14:15:18.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.107+0000 7f0c5f24a740 1 -- 172.21.15.96:0/3116197769 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c48040070 con 0x558c5bf944d0 2024-04-25T14:15:18.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.107+0000 7f0c39ffb640 1 -- 172.21.15.96:0/3116197769 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f0c4c004cc0 con 0x558c5bf944d0 2024-04-25T14:15:18.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.108+0000 7f0c39ffb640 1 -- 172.21.15.96:0/3116197769 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0c4c01a430 con 0x558c5bf944d0 2024-04-25T14:15:18.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.108+0000 7f0c5f24a740 10 client.16146 fetch_fsmap learned FSMap version 26 2024-04-25T14:15:18.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.108+0000 7f0c5f24a740 1 -- 172.21.15.96:0/3116197769 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558c5c09f7d0 con 0x558c5bf944d0 2024-04-25T14:15:18.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.108+0000 7f0c39ffb640 1 -- 172.21.15.96:0/3116197769 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c48040070 con 0x558c5bf944d0 2024-04-25T14:15:18.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.108+0000 7f0c39ffb640 1 -- 172.21.15.96:0/3116197769 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0c4c019070 con 0x558c5bf944d0 2024-04-25T14:15:18.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.108+0000 7f0c5f24a740 10 client.16146 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:15:18.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.108+0000 7f0c5f24a740 10 client.16146 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:15:18.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.108+0000 7f0c5f24a740 20 client.16146 populate_metadata read hostname 'smithi096' 2024-04-25T14:15:18.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.108+0000 7f0c5f24a740 1 --2- 172.21.15.96:0/3116197769 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x558c5bf7bf60 0x558c5c3772d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:18.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.108+0000 7f0c5f24a740 4 client.16146 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:15:18.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.108+0000 7f0c5f24a740 1 -- 172.21.15.96:0/3116197769 --> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c5c2d1990 con 0x558c5bf7bf60 2024-04-25T14:15:18.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.108+0000 7f0c55931640 1 --2- 172.21.15.96:0/3116197769 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x558c5bf7bf60 0x558c5c3772d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:18.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.109+0000 7f0c39ffb640 10 client.16146 ms_handle_connect on v2:172.21.15.151:6838/245699981 2024-04-25T14:15:18.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.109+0000 7f0c55931640 1 --2- 172.21.15.96:0/3116197769 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x558c5bf7bf60 0x558c5c3772d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=21b937cb0717a792 server_cookie=9c572aca4c1a69f2 in_seq=0 out_seq=0 2024-04-25T14:15:18.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.110+0000 7f0c39ffb640 1 -- 172.21.15.96:0/3116197769 <== mds.1 v2:172.21.15.151:6838/245699981 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558c5c2d1990 con 0x558c5bf7bf60 2024-04-25T14:15:18.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.110+0000 7f0c39ffb640 10 client.16146 handle_command_reply: tid=0 2024-04-25T14:15:18.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.110+0000 7f0c5f24a740 1 client.16146 shutdown 2024-04-25T14:15:18.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.110+0000 7f0c5f24a740 1 -- 172.21.15.96:0/3116197769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c5bf944d0 msgr2=0x558c5c3695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:18.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.110+0000 7f0c5f24a740 1 --2- 172.21.15.96:0/3116197769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c5bf944d0 0x558c5c3695c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c011ce0 tx=0x7f0c4c00f790 comp rx=0 tx=0).stop 2024-04-25T14:15:18.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.110+0000 7f0c5f24a740 1 -- 172.21.15.96:0/3116197769 shutdown_connections 2024-04-25T14:15:18.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.110+0000 7f0c5f24a740 1 --2- 172.21.15.96:0/3116197769 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x558c5bf7bf60 0x558c5c3772d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.110+0000 7f0c5f24a740 1 --2- 172.21.15.96:0/3116197769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c5bfbd390 0x558c5c370a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.110+0000 7f0c39ffb640 0 client.16146 ms_handle_reset on v2:172.21.15.151:6838/245699981 2024-04-25T14:15:18.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.110+0000 7f0c5f24a740 1 --2- 172.21.15.96:0/3116197769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c5bf9c6b0 0x558c5c369b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.110+0000 7f0c5f24a740 1 --2- 172.21.15.96:0/3116197769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c5bf944d0 0x558c5c3695c0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.110+0000 7f0c5f24a740 1 -- 172.21.15.96:0/3116197769 >> 172.21.15.96:0/3116197769 conn(0x558c5bfb14e0 msgr2=0x558c5c35a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:18.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.110+0000 7f0c5f24a740 1 -- 172.21.15.96:0/3116197769 shutdown_connections 2024-04-25T14:15:18.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.110+0000 7f0c5f24a740 1 -- 172.21.15.96:0/3116197769 wait complete. 2024-04-25T14:15:18.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.110+0000 7f0c5f24a740 20 client.16146 trim_cache size 0 max 16384 2024-04-25T14:15:18.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.124+0000 7f0c5f24a740 1 Processor -- start 2024-04-25T14:15:18.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.124+0000 7f0c5f24a740 1 -- start start 2024-04-25T14:15:18.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.124+0000 7f0c5f24a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c5bf7bf60 0x558c5c35e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:18.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.124+0000 7f0c5f24a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c5bf944d0 0x558c5c370980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:18.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.124+0000 7f0c5f24a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c5bf9c6b0 0x558c5c37ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:18.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.124+0000 7f0c5f24a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0c501aa2e0 con 0x558c5bf7bf60 2024-04-25T14:15:18.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.124+0000 7f0c5f24a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0c501abe20 con 0x558c5bf9c6b0 2024-04-25T14:15:18.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.124+0000 7f0c5f24a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0c501aa9b0 con 0x558c5bf944d0 2024-04-25T14:15:18.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.124+0000 7f0c56132640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c5bf7bf60 0x558c5c35e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:18.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.124+0000 7f0c56132640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c5bf7bf60 0x558c5c35e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41628/0 (socket says 172.21.15.96:41628) 2024-04-25T14:15:18.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.124+0000 7f0c56132640 1 -- 172.21.15.96:0/487238722 learned_addr learned my addr 172.21.15.96:0/487238722 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:18.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.124+0000 7f0c56933640 1 --2- 172.21.15.96:0/487238722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c5bf9c6b0 0x558c5c37ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:18.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.124+0000 7f0c55931640 1 --2- 172.21.15.96:0/487238722 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c5bf944d0 0x558c5c370980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:18.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.124+0000 7f0c56132640 1 -- 172.21.15.96:0/487238722 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c5bf944d0 msgr2=0x558c5c370980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:18.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.124+0000 7f0c56132640 1 --2- 172.21.15.96:0/487238722 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c5bf944d0 0x558c5c370980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.124+0000 7f0c56132640 1 -- 172.21.15.96:0/487238722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c5bf9c6b0 msgr2=0x558c5c37ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:18.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.124+0000 7f0c56132640 1 --2- 172.21.15.96:0/487238722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c5bf9c6b0 0x558c5c37ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.124+0000 7f0c56132640 1 -- 172.21.15.96:0/487238722 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c5c370ec0 con 0x558c5bf7bf60 2024-04-25T14:15:18.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.124+0000 7f0c56132640 1 --2- 172.21.15.96:0/487238722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c5bf7bf60 0x558c5c35e850 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c000f80 tx=0x7f0c4c005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea99004f1b66469c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:18.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.125+0000 7f0c3bfff640 1 -- 172.21.15.96:0/487238722 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c032070 con 0x558c5bf7bf60 2024-04-25T14:15:18.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.125+0000 7f0c3bfff640 1 -- 172.21.15.96:0/487238722 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c4c003e20 con 0x558c5bf7bf60 2024-04-25T14:15:18.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.125+0000 7f0c3bfff640 1 -- 172.21.15.96:0/487238722 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c029070 con 0x558c5bf7bf60 2024-04-25T14:15:18.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.126+0000 7f0c5f24a740 1 -- 172.21.15.96:0/487238722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c5bf7bf60 msgr2=0x558c5c35e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:18.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.126+0000 7f0c5f24a740 1 --2- 172.21.15.96:0/487238722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c5bf7bf60 0x558c5c35e850 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c000f80 tx=0x7f0c4c005d30 comp rx=0 tx=0).stop 2024-04-25T14:15:18.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.126+0000 7f0c5f24a740 1 -- 172.21.15.96:0/487238722 shutdown_connections 2024-04-25T14:15:18.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.126+0000 7f0c5f24a740 1 --2- 172.21.15.96:0/487238722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c5bf9c6b0 0x558c5c37ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.126+0000 7f0c5f24a740 1 --2- 172.21.15.96:0/487238722 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c5bf944d0 0x558c5c370980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.126+0000 7f0c5f24a740 1 --2- 172.21.15.96:0/487238722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c5bf7bf60 0x558c5c35e850 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.126+0000 7f0c5f24a740 1 -- 172.21.15.96:0/487238722 >> 172.21.15.96:0/487238722 conn(0x558c5bfb14e0 msgr2=0x558c5c367cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:18.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.126+0000 7f0c5f24a740 1 -- 172.21.15.96:0/487238722 shutdown_connections 2024-04-25T14:15:18.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.126+0000 7f0c5f24a740 1 -- 172.21.15.96:0/487238722 wait complete. 2024-04-25T14:15:18.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.127+0000 7f0c5f24a740 1 -- start start 2024-04-25T14:15:18.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.127+0000 7f0c5f24a740 1 Processor -- start 2024-04-25T14:15:18.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.127+0000 7f0c5f24a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c5bf7bf60 0x558c5c36d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:18.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.127+0000 7f0c5f24a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c5bf944d0 0x558c5c36dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:18.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.127+0000 7f0c5f24a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c5bf9c6b0 0x558c5c3dd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:18.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.127+0000 7f0c5f24a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0c501ab750 con 0x558c5bf7bf60 2024-04-25T14:15:18.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.127+0000 7f0c56132640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c5bf7bf60 0x558c5c36d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:18.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.127+0000 7f0c5f24a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0c501ab080 con 0x558c5bf9c6b0 2024-04-25T14:15:18.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.127+0000 7f0c5f24a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0c501a82c0 con 0x558c5bf944d0 2024-04-25T14:15:18.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.127+0000 7f0c56132640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c5bf7bf60 0x558c5c36d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41644/0 (socket says 172.21.15.96:41644) 2024-04-25T14:15:18.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.127+0000 7f0c56132640 1 -- 172.21.15.96:0/24686583 learned_addr learned my addr 172.21.15.96:0/24686583 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:18.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.127+0000 7f0c55931640 1 --2- 172.21.15.96:0/24686583 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c5bf944d0 0x558c5c36dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:18.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.127+0000 7f0c5f24a740 10 client.0 _ll_register_callbacks cb 0x7f0c4c009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:15:18.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.127+0000 7f0c56933640 1 --2- 172.21.15.96:0/24686583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c5bf9c6b0 0x558c5c3dd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:18.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.128+0000 7f0c56132640 1 -- 172.21.15.96:0/24686583 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c5bf944d0 msgr2=0x558c5c36dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:18.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.128+0000 7f0c56132640 1 --2- 172.21.15.96:0/24686583 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c5bf944d0 0x558c5c36dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.128+0000 7f0c56132640 1 -- 172.21.15.96:0/24686583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c5bf9c6b0 msgr2=0x558c5c3dd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:18.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.128+0000 7f0c56132640 1 --2- 172.21.15.96:0/24686583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c5bf9c6b0 0x558c5c3dd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.128+0000 7f0c56132640 1 -- 172.21.15.96:0/24686583 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c5c36e2a0 con 0x558c5bf7bf60 2024-04-25T14:15:18.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.128+0000 7f0c56933640 1 --2- 172.21.15.96:0/24686583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c5bf9c6b0 0x558c5c3dd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:15:18.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.128+0000 7f0c55931640 1 --2- 172.21.15.96:0/24686583 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c5bf944d0 0x558c5c36dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:15:18.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.128+0000 7f0c56132640 1 --2- 172.21.15.96:0/24686583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c5bf7bf60 0x558c5c36d770 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c0247c0 tx=0x7f0c4c005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5293f3d99490739 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:18.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.128+0000 7f0c39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:15:18.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.128+0000 7f0c39ffb640 1 -- 172.21.15.96:0/24686583 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c032020 con 0x558c5bf7bf60 2024-04-25T14:15:18.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.128+0000 7f0c5f24a740 1 -- 172.21.15.96:0/24686583 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c4c003e20 con 0x558c5bf7bf60 2024-04-25T14:15:18.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.128+0000 7f0c39ffb640 1 -- 172.21.15.96:0/24686583 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f0c4c003b80 con 0x558c5bf7bf60 2024-04-25T14:15:18.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.128+0000 7f0c39ffb640 1 -- 172.21.15.96:0/24686583 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0c4c03d050 con 0x558c5bf7bf60 2024-04-25T14:15:18.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.128+0000 7f0c5f24a740 10 client.16152 fetch_fsmap learned FSMap version 26 2024-04-25T14:15:18.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.128+0000 7f0c5f24a740 1 -- 172.21.15.96:0/24686583 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558c5c3debb0 con 0x558c5bf7bf60 2024-04-25T14:15:18.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.129+0000 7f0c39ffb640 1 -- 172.21.15.96:0/24686583 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c4c003e20 con 0x558c5bf7bf60 2024-04-25T14:15:18.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.129+0000 7f0c39ffb640 1 -- 172.21.15.96:0/24686583 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0c4c019040 con 0x558c5bf7bf60 2024-04-25T14:15:18.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.129+0000 7f0c5f24a740 10 client.16152 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:15:18.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.129+0000 7f0c5f24a740 10 client.16152 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:15:18.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.129+0000 7f0c5f24a740 20 client.16152 populate_metadata read hostname 'smithi096' 2024-04-25T14:15:18.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.129+0000 7f0c5f24a740 1 --2- 172.21.15.96:0/24686583 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x558c5bfbd390 0x558c5c3e1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:18.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.129+0000 7f0c5f24a740 4 client.16152 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:15:18.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.129+0000 7f0c5f24a740 1 -- 172.21.15.96:0/24686583 --> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558c5c2cb800 con 0x558c5bfbd390 2024-04-25T14:15:18.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.129+0000 7f0c55931640 1 --2- 172.21.15.96:0/24686583 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x558c5bfbd390 0x558c5c3e1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:18.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.129+0000 7f0c39ffb640 10 client.16152 ms_handle_connect on v2:172.21.15.151:6838/245699981 2024-04-25T14:15:18.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.130+0000 7f0c55931640 1 --2- 172.21.15.96:0/24686583 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x558c5bfbd390 0x558c5c3e1590 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1058c33a258f6f2f server_cookie=e609686fcf2cfcd3 in_seq=0 out_seq=0 2024-04-25T14:15:18.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.130+0000 7f0c39ffb640 1 -- 172.21.15.96:0/24686583 <== mds.1 v2:172.21.15.151:6838/245699981 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558c5c2cb800 con 0x558c5bfbd390 2024-04-25T14:15:18.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.130+0000 7f0c39ffb640 10 client.16152 handle_command_reply: tid=0 2024-04-25T14:15:18.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.130+0000 7f0c5f24a740 1 client.16152 shutdown 2024-04-25T14:15:18.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.130+0000 7f0c5f24a740 1 -- 172.21.15.96:0/24686583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c5bf7bf60 msgr2=0x558c5c36d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:18.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.130+0000 7f0c5f24a740 1 --2- 172.21.15.96:0/24686583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c5bf7bf60 0x558c5c36d770 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c0247c0 tx=0x7f0c4c005df0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.131+0000 7f0c5f24a740 1 -- 172.21.15.96:0/24686583 shutdown_connections 2024-04-25T14:15:18.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.131+0000 7f0c5f24a740 1 --2- 172.21.15.96:0/24686583 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x558c5bfbd390 0x558c5c3e1590 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.131+0000 7f0c5f24a740 1 --2- 172.21.15.96:0/24686583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c5bf9c6b0 0x558c5c3dd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.131+0000 7f0c5f24a740 1 --2- 172.21.15.96:0/24686583 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558c5bf944d0 0x558c5c36dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.131+0000 7f0c5f24a740 1 --2- 172.21.15.96:0/24686583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c5bf7bf60 0x558c5c36d770 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.131+0000 7f0c39ffb640 0 client.16152 ms_handle_reset on v2:172.21.15.151:6838/245699981 2024-04-25T14:15:18.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.131+0000 7f0c5f24a740 1 -- 172.21.15.96:0/24686583 >> 172.21.15.96:0/24686583 conn(0x558c5bfb14e0 msgr2=0x558c5c35e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:18.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.131+0000 7f0c5f24a740 1 -- 172.21.15.96:0/24686583 shutdown_connections 2024-04-25T14:15:18.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.131+0000 7f0c5f24a740 1 -- 172.21.15.96:0/24686583 wait complete. 2024-04-25T14:15:18.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.131+0000 7f0c5f24a740 20 client.16152 trim_cache size 0 max 16384 2024-04-25T14:15:18.135 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-04-25T14:15:18.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.131+0000 7f0c58396640 1 -- 172.21.15.96:0/1840200291 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f0c3c0afaf0 msgr2=0x7f0c3c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:18.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.131+0000 7f0c58396640 1 --2- 172.21.15.96:0/1840200291 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f0c3c0afaf0 0x7f0c3c0b2000 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0c48031fd0 tx=0x7f0c4804b000 comp rx=0 tx=0).stop 2024-04-25T14:15:18.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.131+0000 7f0c58396640 1 -- 172.21.15.96:0/1840200291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c50103c30 msgr2=0x7f0c50199610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:18.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.131+0000 7f0c58396640 1 --2- 172.21.15.96:0/1840200291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c50103c30 0x7f0c50199610 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f0c4000e9f0 tx=0x7f0c4000eeb0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.131+0000 7f0c58396640 1 -- 172.21.15.96:0/1840200291 shutdown_connections 2024-04-25T14:15:18.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.131+0000 7f0c58396640 1 --2- 172.21.15.96:0/1840200291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c50108930 0x7f0c501a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.131+0000 7f0c58396640 1 --2- 172.21.15.96:0/1840200291 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f0c3c0afaf0 0x7f0c3c0b2000 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.131+0000 7f0c58396640 1 --2- 172.21.15.96:0/1840200291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c50103c30 0x7f0c50199610 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.131+0000 7f0c58396640 1 --2- 172.21.15.96:0/1840200291 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c501029f0 0x7f0c501990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.131+0000 7f0c58396640 1 -- 172.21.15.96:0/1840200291 >> 172.21.15.96:0/1840200291 conn(0x7f0c500fdc00 msgr2=0x7f0c50104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:18.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.131+0000 7f0c58396640 1 -- 172.21.15.96:0/1840200291 shutdown_connections 2024-04-25T14:15:18.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.131+0000 7f0c58396640 1 -- 172.21.15.96:0/1840200291 wait complete. 2024-04-25T14:15:18.142 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T14:15:18.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.264+0000 7f8ba9261640 1 -- 172.21.15.96:0/2950585996 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ba41033b0 msgr2=0x7f8ba41037c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:18.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.264+0000 7f8ba9261640 1 --2- 172.21.15.96:0/2950585996 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ba41033b0 0x7f8ba41037c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f8b98009900 tx=0x7f8b9802a410 comp rx=0 tx=0).stop 2024-04-25T14:15:18.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.264+0000 7f8ba9261640 1 -- 172.21.15.96:0/2950585996 shutdown_connections 2024-04-25T14:15:18.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.264+0000 7f8ba9261640 1 --2- 172.21.15.96:0/2950585996 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ba41080b0 0x7f8ba410a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.264+0000 7f8ba9261640 1 --2- 172.21.15.96:0/2950585996 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ba41033b0 0x7f8ba41037c0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.264+0000 7f8ba9261640 1 --2- 172.21.15.96:0/2950585996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ba4102170 0x7f8ba4102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.264+0000 7f8ba9261640 1 -- 172.21.15.96:0/2950585996 >> 172.21.15.96:0/2950585996 conn(0x7f8ba40fd340 msgr2=0x7f8ba40ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:18.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.265+0000 7f8ba9261640 1 -- 172.21.15.96:0/2950585996 shutdown_connections 2024-04-25T14:15:18.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.265+0000 7f8ba9261640 1 -- 172.21.15.96:0/2950585996 wait complete. 2024-04-25T14:15:18.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.265+0000 7f8ba9261640 1 Processor -- start 2024-04-25T14:15:18.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.266+0000 7f8ba9261640 1 -- start start 2024-04-25T14:15:18.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.266+0000 7f8ba9261640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ba4102170 0x7f8ba419c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:18.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.266+0000 7f8ba9261640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ba41033b0 0x7f8ba419c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:18.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.266+0000 7f8ba9261640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ba41080b0 0x7f8ba41a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:18.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.266+0000 7f8ba9261640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba410d400 con 0x7f8ba41033b0 2024-04-25T14:15:18.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.266+0000 7f8ba9261640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba410d280 con 0x7f8ba4102170 2024-04-25T14:15:18.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.266+0000 7f8ba9261640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba410d580 con 0x7f8ba41080b0 2024-04-25T14:15:18.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.266+0000 7f8ba27fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ba41033b0 0x7f8ba419c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:18.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.266+0000 7f8ba27fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ba41033b0 0x7f8ba419c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41656/0 (socket says 172.21.15.96:41656) 2024-04-25T14:15:18.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.266+0000 7f8ba27fc640 1 -- 172.21.15.96:0/2657837588 learned_addr learned my addr 172.21.15.96:0/2657837588 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:18.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.266+0000 7f8ba2ffd640 1 --2- 172.21.15.96:0/2657837588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ba4102170 0x7f8ba419c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:18.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.266+0000 7f8ba37fe640 1 --2- 172.21.15.96:0/2657837588 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ba41080b0 0x7f8ba41a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:18.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.266+0000 7f8ba27fc640 1 -- 172.21.15.96:0/2657837588 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ba41080b0 msgr2=0x7f8ba41a38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:18.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.266+0000 7f8ba27fc640 1 --2- 172.21.15.96:0/2657837588 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ba41080b0 0x7f8ba41a38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.266+0000 7f8ba27fc640 1 -- 172.21.15.96:0/2657837588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ba4102170 msgr2=0x7f8ba419c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:18.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.266+0000 7f8ba27fc640 1 --2- 172.21.15.96:0/2657837588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ba4102170 0x7f8ba419c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.266+0000 7f8ba27fc640 1 -- 172.21.15.96:0/2657837588 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba41a3f50 con 0x7f8ba41033b0 2024-04-25T14:15:18.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.266+0000 7f8ba2ffd640 1 --2- 172.21.15.96:0/2657837588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ba4102170 0x7f8ba419c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:15:18.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.267+0000 7f8ba27fc640 1 --2- 172.21.15.96:0/2657837588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ba41033b0 0x7f8ba419c950 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f8b98006010 tx=0x7f8b980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:18.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.267+0000 7f8b8bfff640 1 -- 172.21.15.96:0/2657837588 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9803f030 con 0x7f8ba41033b0 2024-04-25T14:15:18.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.267+0000 7f8b8bfff640 1 -- 172.21.15.96:0/2657837588 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b98031910 con 0x7f8ba41033b0 2024-04-25T14:15:18.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.267+0000 7f8ba9261640 1 -- 172.21.15.96:0/2657837588 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ba41a41e0 con 0x7f8ba41033b0 2024-04-25T14:15:18.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.267+0000 7f8b8bfff640 1 -- 172.21.15.96:0/2657837588 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b98035070 con 0x7f8ba41033b0 2024-04-25T14:15:18.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.267+0000 7f8ba9261640 1 -- 172.21.15.96:0/2657837588 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ba41a46d0 con 0x7f8ba41033b0 2024-04-25T14:15:18.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.269+0000 7f8bb01ee740 1 Processor -- start 2024-04-25T14:15:18.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.269+0000 7f8bb01ee740 1 -- start start 2024-04-25T14:15:18.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.269+0000 7f8b8bfff640 1 -- 172.21.15.96:0/2657837588 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f8b98042030 con 0x7f8ba41033b0 2024-04-25T14:15:18.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.270+0000 7f8bb01ee740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581485c3390 0x558148965810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:18.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.270+0000 7f8bb01ee740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55814859a4d0 0x558148965d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:18.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.270+0000 7f8bb01ee740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581485a26b0 0x558148966a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:18.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.270+0000 7f8bb01ee740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5581488c9d80 con 0x5581485a26b0 2024-04-25T14:15:18.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.270+0000 7f8bb01ee740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558148883a20 con 0x5581485c3390 2024-04-25T14:15:18.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.270+0000 7f8bb01ee740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5581488c5d40 con 0x55814859a4d0 2024-04-25T14:15:18.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.270+0000 7f8ba27fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581485a26b0 0x558148966a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:18.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.270+0000 7f8ba37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581485c3390 0x558148965810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:18.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.270+0000 7f8ba37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581485c3390 0x558148965810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:42064/0 (socket says 172.21.15.96:42064) 2024-04-25T14:15:18.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.270+0000 7f8ba37fe640 1 -- 172.21.15.96:0/267428987 learned_addr learned my addr 172.21.15.96:0/267428987 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:18.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.270+0000 7f8ba2ffd640 1 --2- 172.21.15.96:0/267428987 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55814859a4d0 0x558148965d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:18.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.270+0000 7f8ba27fc640 1 -- 172.21.15.96:0/267428987 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55814859a4d0 msgr2=0x558148965d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:18.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.270+0000 7f8ba27fc640 1 --2- 172.21.15.96:0/267428987 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55814859a4d0 0x558148965d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.270+0000 7f8ba27fc640 1 -- 172.21.15.96:0/267428987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581485c3390 msgr2=0x558148965810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:18.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.270+0000 7f8ba27fc640 1 --2- 172.21.15.96:0/267428987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581485c3390 0x558148965810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.270+0000 7f8ba27fc640 1 -- 172.21.15.96:0/267428987 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55814888ec30 con 0x5581485a26b0 2024-04-25T14:15:18.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.270+0000 7f8ba37fe640 1 --2- 172.21.15.96:0/267428987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581485c3390 0x558148965810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:15:18.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.270+0000 7f8b8bfff640 1 --2- 172.21.15.96:0/2657837588 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f8b8c0afaf0 0x7f8b8c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:18.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.270+0000 7f8ba27fc640 1 --2- 172.21.15.96:0/267428987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581485a26b0 0x558148966a20 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f8b980fcb80 tx=0x7f8b981073b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29a77e9540318418 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:18.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.270+0000 7f8ba37fe640 1 --2- 172.21.15.96:0/2657837588 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f8b8c0afaf0 0x7f8b8c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:18.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.270+0000 7f8b8bfff640 1 -- 172.21.15.96:0/2657837588 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f8b980ff0d0 con 0x7f8ba41033b0 2024-04-25T14:15:18.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.271+0000 7f8b897fa640 1 -- 172.21.15.96:0/267428987 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b980c3500 con 0x5581485a26b0 2024-04-25T14:15:18.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.271+0000 7f8b897fa640 1 -- 172.21.15.96:0/267428987 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b980c7150 con 0x5581485a26b0 2024-04-25T14:15:18.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.271+0000 7f8b897fa640 1 -- 172.21.15.96:0/267428987 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b980c6db0 con 0x5581485a26b0 2024-04-25T14:15:18.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.271+0000 7f8bb01ee740 1 -- 172.21.15.96:0/267428987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581485a26b0 msgr2=0x558148966a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:18.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.271+0000 7f8bb01ee740 1 --2- 172.21.15.96:0/267428987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581485a26b0 0x558148966a20 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f8b980fcb80 tx=0x7f8b981073b0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.271+0000 7f8ba37fe640 1 --2- 172.21.15.96:0/2657837588 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f8b8c0afaf0 0x7f8b8c0b2000 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x558148966860 tx=0x7f8b9c00d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:18.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.272+0000 7f8bb01ee740 1 -- 172.21.15.96:0/267428987 shutdown_connections 2024-04-25T14:15:18.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.272+0000 7f8bb01ee740 1 --2- 172.21.15.96:0/267428987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581485a26b0 0x558148966a20 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.272+0000 7f8bb01ee740 1 --2- 172.21.15.96:0/267428987 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55814859a4d0 0x558148965d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.272+0000 7f8bb01ee740 1 --2- 172.21.15.96:0/267428987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581485c3390 0x558148965810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.272+0000 7f8bb01ee740 1 -- 172.21.15.96:0/267428987 >> 172.21.15.96:0/267428987 conn(0x5581485b74e0 msgr2=0x558148961020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:18.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.272+0000 7f8bb01ee740 1 -- 172.21.15.96:0/267428987 shutdown_connections 2024-04-25T14:15:18.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.272+0000 7f8bb01ee740 1 -- 172.21.15.96:0/267428987 wait complete. 2024-04-25T14:15:18.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.272+0000 7f8bb01ee740 1 -- start start 2024-04-25T14:15:18.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.272+0000 7f8bb01ee740 1 Processor -- start 2024-04-25T14:15:18.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.272+0000 7f8bb01ee740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55814859a4d0 0x55814896f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:18.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.272+0000 7f8bb01ee740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5581485a26b0 0x55814896f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:18.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.272+0000 7f8bb01ee740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581485c3390 0x5581489766d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:18.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.272+0000 7f8bb01ee740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5581489672f0 con 0x5581485c3390 2024-04-25T14:15:18.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.272+0000 7f8bb01ee740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558148850dc0 con 0x55814859a4d0 2024-04-25T14:15:18.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.272+0000 7f8bb01ee740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558148967470 con 0x5581485a26b0 2024-04-25T14:15:18.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.272+0000 7f8bb01ee740 10 client.0 _ll_register_callbacks cb 0x55814886c600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:15:18.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.272+0000 7f8ba37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581485c3390 0x5581489766d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:18.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.272+0000 7f8ba2ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55814859a4d0 0x55814896f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:18.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.272+0000 7f8ba27fc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5581485a26b0 0x55814896f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:18.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.272+0000 7f8ba37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581485c3390 0x5581489766d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41672/0 (socket says 172.21.15.96:41672) 2024-04-25T14:15:18.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.272+0000 7f8ba37fe640 1 -- 172.21.15.96:0/757620256 learned_addr learned my addr 172.21.15.96:0/757620256 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:18.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.273+0000 7f8ba37fe640 1 -- 172.21.15.96:0/757620256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5581485a26b0 msgr2=0x55814896f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:18.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.273+0000 7f8ba37fe640 1 --2- 172.21.15.96:0/757620256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5581485a26b0 0x55814896f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.273+0000 7f8ba37fe640 1 -- 172.21.15.96:0/757620256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55814859a4d0 msgr2=0x55814896f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:18.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.273+0000 7f8ba37fe640 1 --2- 172.21.15.96:0/757620256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55814859a4d0 0x55814896f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.273+0000 7f8ba37fe640 1 -- 172.21.15.96:0/757620256 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581485db3f0 con 0x5581485c3390 2024-04-25T14:15:18.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.273+0000 7f8ba2ffd640 1 --2- 172.21.15.96:0/757620256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55814859a4d0 0x55814896f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:15:18.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.273+0000 7f8ba37fe640 1 --2- 172.21.15.96:0/757620256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581485c3390 0x5581489766d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c016ba0 tx=0x7f8b9c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4785426a5d82c099 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:18.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.273+0000 7f8b86ff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:15:18.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.273+0000 7f8b86ff5640 1 -- 172.21.15.96:0/757620256 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c028030 con 0x5581485c3390 2024-04-25T14:15:18.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.273+0000 7f8b86ff5640 1 -- 172.21.15.96:0/757620256 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f8b9c009050 con 0x5581485c3390 2024-04-25T14:15:18.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.273+0000 7f8bb01ee740 1 -- 172.21.15.96:0/757620256 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b980c7150 con 0x5581485c3390 2024-04-25T14:15:18.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.273+0000 7f8b86ff5640 1 -- 172.21.15.96:0/757620256 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8b9c004b20 con 0x5581485c3390 2024-04-25T14:15:18.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.273+0000 7f8bb01ee740 10 client.16164 fetch_fsmap learned FSMap version 26 2024-04-25T14:15:18.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.273+0000 7f8bb01ee740 1 -- 172.21.15.96:0/757620256 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5581486a57d0 con 0x5581485c3390 2024-04-25T14:15:18.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.273+0000 7f8b86ff5640 1 -- 172.21.15.96:0/757620256 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b980c7150 con 0x5581485c3390 2024-04-25T14:15:18.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.274+0000 7f8b86ff5640 1 -- 172.21.15.96:0/757620256 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8b9c005400 con 0x5581485c3390 2024-04-25T14:15:18.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.274+0000 7f8bb01ee740 10 client.16164 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:15:18.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.274+0000 7f8bb01ee740 10 client.16164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:15:18.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.274+0000 7f8bb01ee740 20 client.16164 populate_metadata read hostname 'smithi096' 2024-04-25T14:15:18.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.274+0000 7f8bb01ee740 1 --2- 172.21.15.96:0/757620256 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x558148581f60 0x55814897cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:18.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.274+0000 7f8bb01ee740 4 client.16164 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:15:18.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.274+0000 7f8bb01ee740 1 -- 172.21.15.96:0/757620256 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581488d7990 con 0x558148581f60 2024-04-25T14:15:18.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.274+0000 7f8ba2ffd640 1 --2- 172.21.15.96:0/757620256 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x558148581f60 0x55814897cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:18.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.274+0000 7f8b86ff5640 10 client.16164 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:15:18.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.274+0000 7f8ba2ffd640 1 --2- 172.21.15.96:0/757620256 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x558148581f60 0x55814897cf70 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9fa141e7a98dc65 server_cookie=a92bd27b53628345 in_seq=0 out_seq=0 2024-04-25T14:15:18.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.276+0000 7f8b86ff5640 1 -- 172.21.15.96:0/757620256 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5581488d7990 con 0x558148581f60 2024-04-25T14:15:18.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.276+0000 7f8b86ff5640 10 client.16164 handle_command_reply: tid=0 2024-04-25T14:15:18.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.276+0000 7f8bb01ee740 1 client.16164 shutdown 2024-04-25T14:15:18.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.276+0000 7f8bb01ee740 1 -- 172.21.15.96:0/757620256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581485c3390 msgr2=0x5581489766d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:18.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.276+0000 7f8bb01ee740 1 --2- 172.21.15.96:0/757620256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581485c3390 0x5581489766d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c016ba0 tx=0x7f8b9c004ab0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.276+0000 7f8bb01ee740 1 -- 172.21.15.96:0/757620256 shutdown_connections 2024-04-25T14:15:18.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.276+0000 7f8bb01ee740 1 --2- 172.21.15.96:0/757620256 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x558148581f60 0x55814897cf70 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.276+0000 7f8bb01ee740 1 --2- 172.21.15.96:0/757620256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581485c3390 0x5581489766d0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.276+0000 7f8bb01ee740 1 --2- 172.21.15.96:0/757620256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5581485a26b0 0x55814896f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.276+0000 7f8bb01ee740 1 --2- 172.21.15.96:0/757620256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55814859a4d0 0x55814896f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.276+0000 7f8bb01ee740 1 -- 172.21.15.96:0/757620256 >> 172.21.15.96:0/757620256 conn(0x5581485b74e0 msgr2=0x558148963f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:18.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.276+0000 7f8b86ff5640 0 client.16164 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:15:18.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.276+0000 7f8bb01ee740 1 -- 172.21.15.96:0/757620256 shutdown_connections 2024-04-25T14:15:18.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.276+0000 7f8bb01ee740 1 -- 172.21.15.96:0/757620256 wait complete. 2024-04-25T14:15:18.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.276+0000 7f8bb01ee740 20 client.16164 trim_cache size 0 max 16384 2024-04-25T14:15:18.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.293+0000 7f8bb01ee740 1 Processor -- start 2024-04-25T14:15:18.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.293+0000 7f8bb01ee740 1 -- start start 2024-04-25T14:15:18.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.293+0000 7f8bb01ee740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55814859a4d0 0x558148965790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:18.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.293+0000 7f8bb01ee740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581485a26b0 0x558148960a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:18.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.293+0000 7f8bb01ee740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581485c3390 0x558148960f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:18.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.293+0000 7f8bb01ee740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba41ad570 con 0x5581485c3390 2024-04-25T14:15:18.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.293+0000 7f8bb01ee740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba41af0b0 con 0x5581485a26b0 2024-04-25T14:15:18.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.293+0000 7f8bb01ee740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba41adc40 con 0x55814859a4d0 2024-04-25T14:15:18.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.293+0000 7f8ba37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581485c3390 0x558148960f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:18.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.293+0000 7f8ba27fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581485a26b0 0x558148960a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:18.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.293+0000 7f8ba37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581485c3390 0x558148960f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41688/0 (socket says 172.21.15.96:41688) 2024-04-25T14:15:18.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.293+0000 7f8ba37fe640 1 -- 172.21.15.96:0/2319948782 learned_addr learned my addr 172.21.15.96:0/2319948782 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:18.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.293+0000 7f8ba2ffd640 1 --2- 172.21.15.96:0/2319948782 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55814859a4d0 0x558148965790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:18.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.294+0000 7f8ba37fe640 1 -- 172.21.15.96:0/2319948782 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55814859a4d0 msgr2=0x558148965790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:18.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.294+0000 7f8ba37fe640 1 --2- 172.21.15.96:0/2319948782 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55814859a4d0 0x558148965790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.294+0000 7f8ba37fe640 1 -- 172.21.15.96:0/2319948782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581485a26b0 msgr2=0x558148960a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:18.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.294+0000 7f8ba37fe640 1 --2- 172.21.15.96:0/2319948782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581485a26b0 0x558148960a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.294+0000 7f8ba37fe640 1 -- 172.21.15.96:0/2319948782 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558148965e70 con 0x5581485c3390 2024-04-25T14:15:18.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.294+0000 7f8ba27fc640 1 --2- 172.21.15.96:0/2319948782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581485a26b0 0x558148960a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:15:18.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.294+0000 7f8ba37fe640 1 --2- 172.21.15.96:0/2319948782 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581485c3390 0x558148960f90 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c0048a0 tx=0x7f8b9c02d450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88dcd761120f3cf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:18.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.294+0000 7f8b88ff9640 1 -- 172.21.15.96:0/2319948782 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c028040 con 0x5581485c3390 2024-04-25T14:15:18.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.294+0000 7f8b88ff9640 1 -- 172.21.15.96:0/2319948782 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b9c01d030 con 0x5581485c3390 2024-04-25T14:15:18.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.294+0000 7f8b88ff9640 1 -- 172.21.15.96:0/2319948782 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c017c90 con 0x5581485c3390 2024-04-25T14:15:18.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.295+0000 7f8bb01ee740 1 -- 172.21.15.96:0/2319948782 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581485c3390 msgr2=0x558148960f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:18.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.295+0000 7f8bb01ee740 1 --2- 172.21.15.96:0/2319948782 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581485c3390 0x558148960f90 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c0048a0 tx=0x7f8b9c02d450 comp rx=0 tx=0).stop 2024-04-25T14:15:18.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.295+0000 7f8bb01ee740 1 -- 172.21.15.96:0/2319948782 shutdown_connections 2024-04-25T14:15:18.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.295+0000 7f8bb01ee740 1 --2- 172.21.15.96:0/2319948782 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581485c3390 0x558148960f90 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.295+0000 7f8bb01ee740 1 --2- 172.21.15.96:0/2319948782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581485a26b0 0x558148960a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.295+0000 7f8bb01ee740 1 --2- 172.21.15.96:0/2319948782 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55814859a4d0 0x558148965790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.295+0000 7f8bb01ee740 1 -- 172.21.15.96:0/2319948782 >> 172.21.15.96:0/2319948782 conn(0x5581485b74e0 msgr2=0x55814896a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:18.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.295+0000 7f8bb01ee740 1 -- 172.21.15.96:0/2319948782 shutdown_connections 2024-04-25T14:15:18.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.295+0000 7f8bb01ee740 1 -- 172.21.15.96:0/2319948782 wait complete. 2024-04-25T14:15:18.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.295+0000 7f8bb01ee740 1 -- start start 2024-04-25T14:15:18.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.295+0000 7f8bb01ee740 1 Processor -- start 2024-04-25T14:15:18.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.295+0000 7f8bb01ee740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55814859a4d0 0x558148975c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:18.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.295+0000 7f8bb01ee740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581485a26b0 0x5581489727b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:18.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.295+0000 7f8bb01ee740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581485c3390 0x558148972d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:18.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.295+0000 7f8bb01ee740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba41ae9e0 con 0x5581485a26b0 2024-04-25T14:15:18.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.295+0000 7f8bb01ee740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba41ae310 con 0x5581485c3390 2024-04-25T14:15:18.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.295+0000 7f8bb01ee740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba41ab550 con 0x55814859a4d0 2024-04-25T14:15:18.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.296+0000 7f8bb01ee740 10 client.0 _ll_register_callbacks cb 0x7f8b9c003ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:15:18.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.296+0000 7f8ba27fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581485a26b0 0x5581489727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:18.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.296+0000 7f8ba27fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581485a26b0 0x5581489727b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41692/0 (socket says 172.21.15.96:41692) 2024-04-25T14:15:18.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.296+0000 7f8ba27fc640 1 -- 172.21.15.96:0/1309637854 learned_addr learned my addr 172.21.15.96:0/1309637854 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:18.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.296+0000 7f8ba37fe640 1 --2- 172.21.15.96:0/1309637854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581485c3390 0x558148972d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:18.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.296+0000 7f8ba2ffd640 1 --2- 172.21.15.96:0/1309637854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55814859a4d0 0x558148975c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:18.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.296+0000 7f8ba27fc640 1 -- 172.21.15.96:0/1309637854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55814859a4d0 msgr2=0x558148975c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:18.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.296+0000 7f8ba27fc640 1 --2- 172.21.15.96:0/1309637854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55814859a4d0 0x558148975c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.296+0000 7f8ba27fc640 1 -- 172.21.15.96:0/1309637854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581485c3390 msgr2=0x558148972d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:18.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.296+0000 7f8ba27fc640 1 --2- 172.21.15.96:0/1309637854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581485c3390 0x558148972d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.296+0000 7f8ba27fc640 1 -- 172.21.15.96:0/1309637854 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581489ebae0 con 0x5581485a26b0 2024-04-25T14:15:18.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.296+0000 7f8ba27fc640 1 --2- 172.21.15.96:0/1309637854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581485a26b0 0x5581489727b0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f8b980c48b0 tx=0x7f8b98005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe0bb4116c8b5ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:18.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.296+0000 7f8b86ff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:15:18.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.296+0000 7f8b86ff5640 1 -- 172.21.15.96:0/1309637854 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b98049040 con 0x5581485a26b0 2024-04-25T14:15:18.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.296+0000 7f8b86ff5640 1 -- 172.21.15.96:0/1309637854 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f8b980fcc30 con 0x5581485a26b0 2024-04-25T14:15:18.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.296+0000 7f8bb01ee740 1 -- 172.21.15.96:0/1309637854 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b9c01d030 con 0x5581485a26b0 2024-04-25T14:15:18.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.296+0000 7f8b86ff5640 1 -- 172.21.15.96:0/1309637854 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8b980c6f90 con 0x5581485a26b0 2024-04-25T14:15:18.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.296+0000 7f8bb01ee740 10 client.16170 fetch_fsmap learned FSMap version 26 2024-04-25T14:15:18.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.296+0000 7f8bb01ee740 1 -- 172.21.15.96:0/1309637854 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5581489ecd70 con 0x5581485a26b0 2024-04-25T14:15:18.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.297+0000 7f8b86ff5640 1 -- 172.21.15.96:0/1309637854 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b9c01d030 con 0x5581485a26b0 2024-04-25T14:15:18.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.297+0000 7f8b86ff5640 1 -- 172.21.15.96:0/1309637854 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8b980c78a0 con 0x5581485a26b0 2024-04-25T14:15:18.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.297+0000 7f8bb01ee740 10 client.16170 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:15:18.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.297+0000 7f8bb01ee740 10 client.16170 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:15:18.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.297+0000 7f8bb01ee740 20 client.16170 populate_metadata read hostname 'smithi096' 2024-04-25T14:15:18.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.297+0000 7f8bb01ee740 1 --2- 172.21.15.96:0/1309637854 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5581485573a0 0x5581489ef390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:18.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.297+0000 7f8bb01ee740 4 client.16170 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:15:18.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.297+0000 7f8bb01ee740 1 -- 172.21.15.96:0/1309637854 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5581488d1800 con 0x5581485573a0 2024-04-25T14:15:18.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.297+0000 7f8ba2ffd640 1 --2- 172.21.15.96:0/1309637854 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5581485573a0 0x5581489ef390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:18.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.297+0000 7f8b86ff5640 10 client.16170 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:15:18.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.297+0000 7f8ba2ffd640 1 --2- 172.21.15.96:0/1309637854 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5581485573a0 0x5581489ef390 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54ea708f035f1409 server_cookie=f02f6098ac23be16 in_seq=0 out_seq=0 2024-04-25T14:15:18.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.298+0000 7f8b86ff5640 1 -- 172.21.15.96:0/1309637854 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5581488d1800 con 0x5581485573a0 2024-04-25T14:15:18.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.298+0000 7f8b86ff5640 10 client.16170 handle_command_reply: tid=0 2024-04-25T14:15:18.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.298+0000 7f8bb01ee740 1 client.16170 shutdown 2024-04-25T14:15:18.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.298+0000 7f8bb01ee740 1 -- 172.21.15.96:0/1309637854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581485a26b0 msgr2=0x5581489727b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:18.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.298+0000 7f8bb01ee740 1 --2- 172.21.15.96:0/1309637854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581485a26b0 0x5581489727b0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f8b980c48b0 tx=0x7f8b98005c90 comp rx=0 tx=0).stop 2024-04-25T14:15:18.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.298+0000 7f8bb01ee740 1 -- 172.21.15.96:0/1309637854 shutdown_connections 2024-04-25T14:15:18.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.298+0000 7f8bb01ee740 1 --2- 172.21.15.96:0/1309637854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581485c3390 0x558148972d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.298+0000 7f8bb01ee740 1 --2- 172.21.15.96:0/1309637854 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5581485573a0 0x5581489ef390 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.298+0000 7f8b86ff5640 0 client.16170 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:15:18.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.298+0000 7f8bb01ee740 1 --2- 172.21.15.96:0/1309637854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581485a26b0 0x5581489727b0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.298+0000 7f8bb01ee740 1 --2- 172.21.15.96:0/1309637854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55814859a4d0 0x558148975c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.298+0000 7f8bb01ee740 1 -- 172.21.15.96:0/1309637854 >> 172.21.15.96:0/1309637854 conn(0x5581485b74e0 msgr2=0x55814896d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:18.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.298+0000 7f8bb01ee740 1 -- 172.21.15.96:0/1309637854 shutdown_connections 2024-04-25T14:15:18.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.298+0000 7f8bb01ee740 1 -- 172.21.15.96:0/1309637854 wait complete. 2024-04-25T14:15:18.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.298+0000 7f8bb01ee740 20 client.16170 trim_cache size 0 max 16384 2024-04-25T14:15:18.304 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-04-25T14:15:18.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.298+0000 7f8ba9261640 1 -- 172.21.15.96:0/2657837588 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f8b8c0afaf0 msgr2=0x7f8b8c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:18.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.298+0000 7f8ba9261640 1 --2- 172.21.15.96:0/2657837588 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f8b8c0afaf0 0x7f8b8c0b2000 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x558148966860 tx=0x7f8b9c00d040 comp rx=0 tx=0).stop 2024-04-25T14:15:18.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.298+0000 7f8ba9261640 1 -- 172.21.15.96:0/2657837588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ba41033b0 msgr2=0x7f8ba419c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:18.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.298+0000 7f8ba9261640 1 --2- 172.21.15.96:0/2657837588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ba41033b0 0x7f8ba419c950 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f8b98006010 tx=0x7f8b980042f0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.299+0000 7f8ba9261640 1 -- 172.21.15.96:0/2657837588 shutdown_connections 2024-04-25T14:15:18.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.299+0000 7f8ba9261640 1 --2- 172.21.15.96:0/2657837588 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ba41080b0 0x7f8ba41a38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.299+0000 7f8ba9261640 1 --2- 172.21.15.96:0/2657837588 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f8b8c0afaf0 0x7f8b8c0b2000 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.299+0000 7f8ba9261640 1 --2- 172.21.15.96:0/2657837588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ba41033b0 0x7f8ba419c950 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.299+0000 7f8ba9261640 1 --2- 172.21.15.96:0/2657837588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ba4102170 0x7f8ba419c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.299+0000 7f8ba9261640 1 -- 172.21.15.96:0/2657837588 >> 172.21.15.96:0/2657837588 conn(0x7f8ba40fd340 msgr2=0x7f8ba4104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:18.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.299+0000 7f8ba9261640 1 -- 172.21.15.96:0/2657837588 shutdown_connections 2024-04-25T14:15:18.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:18.299+0000 7f8ba9261640 1 -- 172.21.15.96:0/2657837588 wait complete. 2024-04-25T14:15:18.483 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4192256 len 1024 2024-04-25T14:15:18.484 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4192256 len 1024 2024-04-25T14:15:18.484 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4192256 len 2048 2024-04-25T14:15:18.484 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4192256 len 2048 2024-04-25T14:15:18.484 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4192256 len 2048 2024-04-25T14:15:18.484 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4192256 len 2048 2024-04-25T14:15:18.484 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4192256 len 4096 2024-04-25T14:15:18.484 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4192256 len 4096 2024-04-25T14:15:18.484 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4192256 len 4096 2024-04-25T14:15:18.484 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4192256 len 4096 2024-04-25T14:15:18.484 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4192256 len 8192 2024-04-25T14:15:18.484 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4192256 len 8192 2024-04-25T14:15:18.484 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4192256 len 8192 2024-04-25T14:15:18.485 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4192256 len 8192 2024-04-25T14:15:18.485 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4192256 len 16384 2024-04-25T14:15:18.485 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4192256 len 16384 2024-04-25T14:15:18.485 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4192256 len 16384 2024-04-25T14:15:18.485 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4192256 len 16384 2024-04-25T14:15:18.485 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4192768 len 1024 2024-04-25T14:15:18.485 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4192768 len 1024 2024-04-25T14:15:18.485 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4192768 len 1024 2024-04-25T14:15:18.485 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4192768 len 1024 2024-04-25T14:15:18.485 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4192768 len 2048 2024-04-25T14:15:18.485 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4192768 len 2048 2024-04-25T14:15:18.485 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4192768 len 2048 2024-04-25T14:15:18.485 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4192768 len 2048 2024-04-25T14:15:18.485 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4192768 len 4096 2024-04-25T14:15:18.485 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4192768 len 4096 2024-04-25T14:15:18.485 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4192768 len 4096 2024-04-25T14:15:18.486 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4192768 len 4096 2024-04-25T14:15:18.486 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4192768 len 8192 2024-04-25T14:15:18.486 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4192768 len 8192 2024-04-25T14:15:18.486 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4192768 len 8192 2024-04-25T14:15:18.486 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4192768 len 8192 2024-04-25T14:15:18.486 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4192768 len 16384 2024-04-25T14:15:18.486 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4192768 len 16384 2024-04-25T14:15:18.486 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4192768 len 16384 2024-04-25T14:15:18.486 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4192768 len 16384 2024-04-25T14:15:18.486 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4193280 len 1024 2024-04-25T14:15:18.486 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4193280 len 1024 2024-04-25T14:15:18.487 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4193280 len 1024 2024-04-25T14:15:18.487 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4193280 len 1024 2024-04-25T14:15:18.487 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4193280 len 2048 2024-04-25T14:15:18.487 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4193280 len 2048 2024-04-25T14:15:18.487 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4193280 len 2048 2024-04-25T14:15:18.487 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4193280 len 2048 2024-04-25T14:15:18.487 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4193280 len 4096 2024-04-25T14:15:18.487 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4193280 len 4096 2024-04-25T14:15:18.487 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4193280 len 4096 2024-04-25T14:15:18.487 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4193280 len 4096 2024-04-25T14:15:18.487 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4193280 len 8192 2024-04-25T14:15:18.487 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4193280 len 8192 2024-04-25T14:15:18.487 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4193280 len 8192 2024-04-25T14:15:18.487 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4193280 len 8192 2024-04-25T14:15:18.487 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4193280 len 16384 2024-04-25T14:15:18.488 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4193280 len 16384 2024-04-25T14:15:18.488 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4193280 len 16384 2024-04-25T14:15:18.488 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4193280 len 16384 2024-04-25T14:15:18.488 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4193792 len 1024 2024-04-25T14:15:18.488 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4193792 len 1024 2024-04-25T14:15:18.488 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4193792 len 1024 2024-04-25T14:15:18.488 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4193792 len 1024 2024-04-25T14:15:18.488 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4193792 len 2048 2024-04-25T14:15:18.488 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4193792 len 2048 2024-04-25T14:15:18.488 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4193792 len 2048 2024-04-25T14:15:18.488 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4193792 len 2048 2024-04-25T14:15:18.488 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4193792 len 4096 2024-04-25T14:15:18.488 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4193792 len 4096 2024-04-25T14:15:18.488 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4193792 len 4096 2024-04-25T14:15:18.489 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4193792 len 4096 2024-04-25T14:15:18.489 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4193792 len 8192 2024-04-25T14:15:18.489 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4193792 len 8192 2024-04-25T14:15:18.489 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4193792 len 8192 2024-04-25T14:15:18.489 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4193792 len 8192 2024-04-25T14:15:18.489 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4193792 len 16384 2024-04-25T14:15:18.489 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4193792 len 16384 2024-04-25T14:15:18.489 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4193792 len 16384 2024-04-25T14:15:18.489 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4193792 len 16384 2024-04-25T14:15:18.489 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4194304 len 1024 2024-04-25T14:15:18.489 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4194304 len 1024 2024-04-25T14:15:18.490 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4194304 len 1024 2024-04-25T14:15:18.490 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4194304 len 1024 2024-04-25T14:15:18.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:18 smithi096 ceph-mon[29207]: pgmap v396: 97 pgs: 97 active+clean; 126 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 1.4 MiB/s wr, 2.66k op/s 2024-04-25T14:15:18.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:18 smithi097 ceph-mon[32720]: pgmap v396: 97 pgs: 97 active+clean; 126 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 1.4 MiB/s wr, 2.66k op/s 2024-04-25T14:15:18.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:18 smithi151 ceph-mon[32990]: pgmap v396: 97 pgs: 97 active+clean; 126 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 1.4 MiB/s wr, 2.66k op/s 2024-04-25T14:15:18.950 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4194304 len 2048 2024-04-25T14:15:18.950 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4194304 len 2048 2024-04-25T14:15:18.950 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4194304 len 2048 2024-04-25T14:15:18.950 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4194304 len 2048 2024-04-25T14:15:18.950 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4194304 len 4096 2024-04-25T14:15:18.950 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4194304 len 4096 2024-04-25T14:15:18.950 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4194304 len 4096 2024-04-25T14:15:18.950 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4194304 len 4096 2024-04-25T14:15:18.951 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4194304 len 8192 2024-04-25T14:15:18.951 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4194304 len 8192 2024-04-25T14:15:18.951 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4194304 len 8192 2024-04-25T14:15:18.951 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4194304 len 8192 2024-04-25T14:15:18.951 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4194304 len 16384 2024-04-25T14:15:18.951 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4194304 len 16384 2024-04-25T14:15:18.951 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4194304 len 16384 2024-04-25T14:15:18.951 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4194304 len 16384 2024-04-25T14:15:18.951 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4194816 len 1024 2024-04-25T14:15:18.951 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4194816 len 1024 2024-04-25T14:15:18.951 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4194816 len 1024 2024-04-25T14:15:18.951 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4194816 len 1024 2024-04-25T14:15:18.951 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4194816 len 2048 2024-04-25T14:15:18.951 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4194816 len 2048 2024-04-25T14:15:18.952 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4194816 len 2048 2024-04-25T14:15:18.952 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4194816 len 2048 2024-04-25T14:15:18.952 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4194816 len 4096 2024-04-25T14:15:18.952 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4194816 len 4096 2024-04-25T14:15:18.952 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4194816 len 4096 2024-04-25T14:15:18.952 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4194816 len 4096 2024-04-25T14:15:18.952 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4194816 len 8192 2024-04-25T14:15:18.952 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4194816 len 8192 2024-04-25T14:15:18.952 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4194816 len 8192 2024-04-25T14:15:18.952 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4194816 len 8192 2024-04-25T14:15:18.952 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4194816 len 16384 2024-04-25T14:15:18.952 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4194816 len 16384 2024-04-25T14:15:18.952 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4194816 len 16384 2024-04-25T14:15:18.952 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4194816 len 16384 2024-04-25T14:15:18.953 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4195328 len 1024 2024-04-25T14:15:18.953 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4195328 len 1024 2024-04-25T14:15:18.953 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4195328 len 1024 2024-04-25T14:15:18.953 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4195328 len 1024 2024-04-25T14:15:18.953 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4195328 len 2048 2024-04-25T14:15:18.953 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4195328 len 2048 2024-04-25T14:15:18.953 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4195328 len 2048 2024-04-25T14:15:18.953 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4195328 len 2048 2024-04-25T14:15:18.953 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4195328 len 4096 2024-04-25T14:15:18.953 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4195328 len 4096 2024-04-25T14:15:18.953 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4195328 len 4096 2024-04-25T14:15:18.953 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4195328 len 4096 2024-04-25T14:15:18.953 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4195328 len 8192 2024-04-25T14:15:18.953 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4195328 len 8192 2024-04-25T14:15:18.953 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4195328 len 8192 2024-04-25T14:15:18.954 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4195328 len 8192 2024-04-25T14:15:18.954 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4195328 len 16384 2024-04-25T14:15:18.954 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4195328 len 16384 2024-04-25T14:15:18.954 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4195328 len 16384 2024-04-25T14:15:18.954 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4195328 len 16384 2024-04-25T14:15:18.954 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4195840 len 1024 2024-04-25T14:15:18.954 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4195840 len 1024 2024-04-25T14:15:18.954 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4195840 len 1024 2024-04-25T14:15:18.954 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4195840 len 1024 2024-04-25T14:15:18.954 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4195840 len 2048 2024-04-25T14:15:18.954 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4195840 len 2048 2024-04-25T14:15:18.954 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4195840 len 2048 2024-04-25T14:15:18.954 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4195840 len 2048 2024-04-25T14:15:18.954 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4195840 len 4096 2024-04-25T14:15:18.955 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4195840 len 4096 2024-04-25T14:15:18.955 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4195840 len 4096 2024-04-25T14:15:18.955 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4195840 len 4096 2024-04-25T14:15:18.955 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4195840 len 8192 2024-04-25T14:15:18.955 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4195840 len 8192 2024-04-25T14:15:18.955 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4195840 len 8192 2024-04-25T14:15:18.955 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4195840 len 8192 2024-04-25T14:15:18.955 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4195840 len 16384 2024-04-25T14:15:18.955 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4195840 len 16384 2024-04-25T14:15:18.955 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4195840 len 16384 2024-04-25T14:15:18.955 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4195840 len 16384 2024-04-25T14:15:18.955 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4196352 len 1024 2024-04-25T14:15:18.955 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4196352 len 1024 2024-04-25T14:15:18.955 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4196352 len 1024 2024-04-25T14:15:18.955 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4196352 len 1024 2024-04-25T14:15:18.956 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4196352 len 2048 2024-04-25T14:15:18.956 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4196352 len 2048 2024-04-25T14:15:19.311 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T14:15:19.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:15:19.312 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T14:15:19.381 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4196352 len 2048 2024-04-25T14:15:19.381 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4196352 len 2048 2024-04-25T14:15:19.381 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4196352 len 4096 2024-04-25T14:15:19.381 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4196352 len 4096 2024-04-25T14:15:19.381 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4196352 len 4096 2024-04-25T14:15:19.381 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4196352 len 4096 2024-04-25T14:15:19.381 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4196352 len 8192 2024-04-25T14:15:19.381 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4196352 len 8192 2024-04-25T14:15:19.381 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4196352 len 8192 2024-04-25T14:15:19.381 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4196352 len 8192 2024-04-25T14:15:19.381 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4196352 len 16384 2024-04-25T14:15:19.382 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4196352 len 16384 2024-04-25T14:15:19.382 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4196352 len 16384 2024-04-25T14:15:19.382 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4196352 len 16384 2024-04-25T14:15:19.382 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4196864 len 1024 2024-04-25T14:15:19.382 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4196864 len 1024 2024-04-25T14:15:19.382 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4196864 len 1024 2024-04-25T14:15:19.382 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4196864 len 1024 2024-04-25T14:15:19.382 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4196864 len 2048 2024-04-25T14:15:19.382 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4196864 len 2048 2024-04-25T14:15:19.382 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4196864 len 2048 2024-04-25T14:15:19.382 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4196864 len 2048 2024-04-25T14:15:19.382 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4196864 len 4096 2024-04-25T14:15:19.382 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4196864 len 4096 2024-04-25T14:15:19.382 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4196864 len 4096 2024-04-25T14:15:19.383 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4196864 len 4096 2024-04-25T14:15:19.383 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4196864 len 8192 2024-04-25T14:15:19.383 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4196864 len 8192 2024-04-25T14:15:19.383 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4196864 len 8192 2024-04-25T14:15:19.383 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4196864 len 8192 2024-04-25T14:15:19.383 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4196864 len 16384 2024-04-25T14:15:19.383 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4196864 len 16384 2024-04-25T14:15:19.383 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4196864 len 16384 2024-04-25T14:15:19.383 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4196864 len 16384 2024-04-25T14:15:19.383 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4197376 len 1024 2024-04-25T14:15:19.383 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4197376 len 1024 2024-04-25T14:15:19.383 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4197376 len 1024 2024-04-25T14:15:19.383 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4197376 len 1024 2024-04-25T14:15:19.383 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4197376 len 2048 2024-04-25T14:15:19.383 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4197376 len 2048 2024-04-25T14:15:19.384 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4197376 len 2048 2024-04-25T14:15:19.384 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4197376 len 2048 2024-04-25T14:15:19.384 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4197376 len 4096 2024-04-25T14:15:19.384 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4197376 len 4096 2024-04-25T14:15:19.384 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4197376 len 4096 2024-04-25T14:15:19.384 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4197376 len 4096 2024-04-25T14:15:19.384 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4197376 len 8192 2024-04-25T14:15:19.384 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4197376 len 8192 2024-04-25T14:15:19.384 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4197376 len 8192 2024-04-25T14:15:19.384 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4197376 len 8192 2024-04-25T14:15:19.384 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4197376 len 16384 2024-04-25T14:15:19.384 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4197376 len 16384 2024-04-25T14:15:19.384 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4197376 len 16384 2024-04-25T14:15:19.384 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4197376 len 16384 2024-04-25T14:15:19.385 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4197888 len 1024 2024-04-25T14:15:19.385 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4197888 len 1024 2024-04-25T14:15:19.385 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4197888 len 1024 2024-04-25T14:15:19.385 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4197888 len 1024 2024-04-25T14:15:19.385 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4197888 len 2048 2024-04-25T14:15:19.385 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4197888 len 2048 2024-04-25T14:15:19.385 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4197888 len 2048 2024-04-25T14:15:19.385 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4197888 len 2048 2024-04-25T14:15:19.385 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4197888 len 4096 2024-04-25T14:15:19.385 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4197888 len 4096 2024-04-25T14:15:19.385 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4197888 len 4096 2024-04-25T14:15:19.385 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4197888 len 4096 2024-04-25T14:15:19.385 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4197888 len 8192 2024-04-25T14:15:19.385 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4197888 len 8192 2024-04-25T14:15:19.385 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4197888 len 8192 2024-04-25T14:15:19.386 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4197888 len 8192 2024-04-25T14:15:19.386 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 512 offset 4197888 len 16384 2024-04-25T14:15:19.386 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4197888 len 16384 2024-04-25T14:15:19.386 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 512 offset 4197888 len 16384 2024-04-25T14:15:19.386 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 512 offset 4197888 len 16384 2024-04-25T14:15:19.386 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4190720 len 1024 2024-04-25T14:15:19.386 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4190720 len 1024 2024-04-25T14:15:19.386 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4190720 len 1024 2024-04-25T14:15:19.386 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4190720 len 1024 2024-04-25T14:15:19.386 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4190720 len 2048 2024-04-25T14:15:19.386 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4190720 len 2048 2024-04-25T14:15:19.386 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4190720 len 2048 2024-04-25T14:15:19.386 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4190720 len 2048 2024-04-25T14:15:19.386 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4190720 len 4096 2024-04-25T14:15:19.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.436+0000 7f4d59ad9640 1 -- 172.21.15.96:0/1286928237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d54101e50 msgr2=0x7f4d54106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:19.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.436+0000 7f4d59ad9640 1 --2- 172.21.15.96:0/1286928237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d54101e50 0x7f4d54106b10 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f4d44009900 tx=0x7f4d4402a410 comp rx=0 tx=0).stop 2024-04-25T14:15:19.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.436+0000 7f4d59ad9640 1 -- 172.21.15.96:0/1286928237 shutdown_connections 2024-04-25T14:15:19.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.436+0000 7f4d59ad9640 1 --2- 172.21.15.96:0/1286928237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d54109770 0x7f4d5410bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.436+0000 7f4d59ad9640 1 --2- 172.21.15.96:0/1286928237 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d54102350 0x7f4d541090e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.436+0000 7f4d59ad9640 1 --2- 172.21.15.96:0/1286928237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d54101e50 0x7f4d54106b10 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.436+0000 7f4d59ad9640 1 -- 172.21.15.96:0/1286928237 >> 172.21.15.96:0/1286928237 conn(0x7f4d540fd340 msgr2=0x7f4d540ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:19.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.436+0000 7f4d59ad9640 1 -- 172.21.15.96:0/1286928237 shutdown_connections 2024-04-25T14:15:19.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.436+0000 7f4d59ad9640 1 -- 172.21.15.96:0/1286928237 wait complete. 2024-04-25T14:15:19.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.437+0000 7f4d59ad9640 1 Processor -- start 2024-04-25T14:15:19.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.437+0000 7f4d59ad9640 1 -- start start 2024-04-25T14:15:19.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.437+0000 7f4d59ad9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d54101e50 0x7f4d5419a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:19.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.437+0000 7f4d59ad9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d54102350 0x7f4d5419a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:19.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.437+0000 7f4d59ad9640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d54109770 0x7f4d541a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:19.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.437+0000 7f4d59ad9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5410f630 con 0x7f4d54102350 2024-04-25T14:15:19.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.437+0000 7f4d59ad9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5410f4b0 con 0x7f4d54101e50 2024-04-25T14:15:19.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.437+0000 7f4d59ad9640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5410f7b0 con 0x7f4d54109770 2024-04-25T14:15:19.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.438+0000 7f4d52ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d54102350 0x7f4d5419a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:19.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.438+0000 7f4d52ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d54102350 0x7f4d5419a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41714/0 (socket says 172.21.15.96:41714) 2024-04-25T14:15:19.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.438+0000 7f4d537fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d54101e50 0x7f4d5419a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:19.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.438+0000 7f4d53fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d54109770 0x7f4d541a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:19.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.438+0000 7f4d52ffd640 1 -- 172.21.15.96:0/3493180985 learned_addr learned my addr 172.21.15.96:0/3493180985 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:19.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.438+0000 7f4d52ffd640 1 -- 172.21.15.96:0/3493180985 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d54109770 msgr2=0x7f4d541a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:19.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.438+0000 7f4d52ffd640 1 --2- 172.21.15.96:0/3493180985 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d54109770 0x7f4d541a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.438+0000 7f4d52ffd640 1 -- 172.21.15.96:0/3493180985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d54101e50 msgr2=0x7f4d5419a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:19.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.438+0000 7f4d52ffd640 1 --2- 172.21.15.96:0/3493180985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d54101e50 0x7f4d5419a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.438+0000 7f4d52ffd640 1 -- 172.21.15.96:0/3493180985 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d541a1db0 con 0x7f4d54102350 2024-04-25T14:15:19.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.438+0000 7f4d537fe640 1 --2- 172.21.15.96:0/3493180985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d54101e50 0x7f4d5419a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:15:19.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.438+0000 7f4d52ffd640 1 --2- 172.21.15.96:0/3493180985 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d54102350 0x7f4d5419a7b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f4d4800b4a0 tx=0x7f4d4800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:19.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.438+0000 7f4d50ff9640 1 -- 172.21.15.96:0/3493180985 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d48014070 con 0x7f4d54102350 2024-04-25T14:15:19.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.438+0000 7f4d50ff9640 1 -- 172.21.15.96:0/3493180985 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d48010ac0 con 0x7f4d54102350 2024-04-25T14:15:19.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.438+0000 7f4d59ad9640 1 -- 172.21.15.96:0/3493180985 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d541a20a0 con 0x7f4d54102350 2024-04-25T14:15:19.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.438+0000 7f4d50ff9640 1 -- 172.21.15.96:0/3493180985 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d48017c30 con 0x7f4d54102350 2024-04-25T14:15:19.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.438+0000 7f4d59ad9640 1 -- 172.21.15.96:0/3493180985 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d541a2620 con 0x7f4d54102350 2024-04-25T14:15:19.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.441+0000 7f4d60982740 1 Processor -- start 2024-04-25T14:15:19.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.441+0000 7f4d60982740 1 -- start start 2024-04-25T14:15:19.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.441+0000 7f4d50ff9640 1 -- 172.21.15.96:0/3493180985 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f4d48017070 con 0x7f4d54102350 2024-04-25T14:15:19.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.441+0000 7f4d60982740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddd43f5410 0x55ddd47964a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:19.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.441+0000 7f4d60982740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddd43cc550 0x55ddd4796a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:19.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.441+0000 7f4d60982740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddd43d4730 0x55ddd479d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:19.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.441+0000 7f4d60982740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ddd46fbb00 con 0x55ddd43d4730 2024-04-25T14:15:19.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.441+0000 7f4d60982740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ddd4697920 con 0x55ddd43cc550 2024-04-25T14:15:19.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.441+0000 7f4d60982740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ddd46f7ac0 con 0x55ddd43f5410 2024-04-25T14:15:19.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.441+0000 7f4d52ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddd43d4730 0x55ddd479d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:19.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.442+0000 7f4d53fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddd43f5410 0x55ddd47964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:19.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.442+0000 7f4d52ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddd43d4730 0x55ddd479d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41726/0 (socket says 172.21.15.96:41726) 2024-04-25T14:15:19.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.442+0000 7f4d537fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddd43cc550 0x55ddd4796a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:19.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.442+0000 7f4d52ffd640 1 -- 172.21.15.96:0/4196945455 learned_addr learned my addr 172.21.15.96:0/4196945455 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:19.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.442+0000 7f4d52ffd640 1 -- 172.21.15.96:0/4196945455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddd43f5410 msgr2=0x55ddd47964a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:19.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.442+0000 7f4d52ffd640 1 --2- 172.21.15.96:0/4196945455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddd43f5410 0x55ddd47964a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.442+0000 7f4d52ffd640 1 -- 172.21.15.96:0/4196945455 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddd43cc550 msgr2=0x55ddd4796a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:19.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.442+0000 7f4d52ffd640 1 --2- 172.21.15.96:0/4196945455 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddd43cc550 0x55ddd4796a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.442+0000 7f4d52ffd640 1 -- 172.21.15.96:0/4196945455 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddd44d57e0 con 0x55ddd43d4730 2024-04-25T14:15:19.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.442+0000 7f4d50ff9640 1 --2- 172.21.15.96:0/3493180985 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f4d3c0afaf0 0x7f4d3c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:19.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.442+0000 7f4d53fff640 1 --2- 172.21.15.96:0/3493180985 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f4d3c0afaf0 0x7f4d3c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:19.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.442+0000 7f4d52ffd640 1 --2- 172.21.15.96:0/4196945455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddd43d4730 0x55ddd479d920 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f4d480e6160 tx=0x7f4d480e6620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=286c65e5277a4ff5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:19.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.442+0000 7f4d50ff9640 1 -- 172.21.15.96:0/3493180985 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f4d480ddf50 con 0x7f4d54102350 2024-04-25T14:15:19.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.442+0000 7f4d39ffb640 1 -- 172.21.15.96:0/4196945455 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d480a2380 con 0x55ddd43d4730 2024-04-25T14:15:19.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.442+0000 7f4d39ffb640 1 -- 172.21.15.96:0/4196945455 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d48017360 con 0x55ddd43d4730 2024-04-25T14:15:19.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.442+0000 7f4d39ffb640 1 -- 172.21.15.96:0/4196945455 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d480a5c30 con 0x55ddd43d4730 2024-04-25T14:15:19.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.443+0000 7f4d60982740 1 -- 172.21.15.96:0/4196945455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddd43d4730 msgr2=0x55ddd479d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:19.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.443+0000 7f4d53fff640 1 --2- 172.21.15.96:0/3493180985 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f4d3c0afaf0 0x7f4d3c0b2000 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55ddd4797670 tx=0x7f4d4c009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:19.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.443+0000 7f4d60982740 1 --2- 172.21.15.96:0/4196945455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddd43d4730 0x55ddd479d920 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f4d480e6160 tx=0x7f4d480e6620 comp rx=0 tx=0).stop 2024-04-25T14:15:19.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.444+0000 7f4d60982740 1 -- 172.21.15.96:0/4196945455 shutdown_connections 2024-04-25T14:15:19.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.444+0000 7f4d60982740 1 --2- 172.21.15.96:0/4196945455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddd43d4730 0x55ddd479d920 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.444+0000 7f4d60982740 1 --2- 172.21.15.96:0/4196945455 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddd43cc550 0x55ddd4796a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.444+0000 7f4d60982740 1 --2- 172.21.15.96:0/4196945455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddd43f5410 0x55ddd47964a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.444+0000 7f4d60982740 1 -- 172.21.15.96:0/4196945455 >> 172.21.15.96:0/4196945455 conn(0x55ddd43e9560 msgr2=0x55ddd4791e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:19.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.444+0000 7f4d60982740 1 -- 172.21.15.96:0/4196945455 shutdown_connections 2024-04-25T14:15:19.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.444+0000 7f4d60982740 1 -- 172.21.15.96:0/4196945455 wait complete. 2024-04-25T14:15:19.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.444+0000 7f4d60982740 1 -- start start 2024-04-25T14:15:19.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.444+0000 7f4d60982740 1 Processor -- start 2024-04-25T14:15:19.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.444+0000 7f4d60982740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddd43cc550 0x55ddd47a0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:19.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.444+0000 7f4d60982740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddd43d4730 0x55ddd47a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:19.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.444+0000 7f4d60982740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddd43f5410 0x55ddd47a7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:19.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.445+0000 7f4d60982740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ddd479e310 con 0x55ddd43f5410 2024-04-25T14:15:19.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.445+0000 7f4d60982740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ddd46955d0 con 0x55ddd43d4730 2024-04-25T14:15:19.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.445+0000 7f4d60982740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ddd479e490 con 0x55ddd43cc550 2024-04-25T14:15:19.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.445+0000 7f4d60982740 10 client.0 _ll_register_callbacks cb 0x55ddd4683080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:15:19.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.445+0000 7f4d53fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddd43f5410 0x55ddd47a7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:19.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.445+0000 7f4d52ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddd43d4730 0x55ddd47a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:19.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.445+0000 7f4d537fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddd43cc550 0x55ddd47a0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:19.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.445+0000 7f4d53fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddd43f5410 0x55ddd47a7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41740/0 (socket says 172.21.15.96:41740) 2024-04-25T14:15:19.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.445+0000 7f4d53fff640 1 -- 172.21.15.96:0/3683820841 learned_addr learned my addr 172.21.15.96:0/3683820841 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:19.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.445+0000 7f4d53fff640 1 -- 172.21.15.96:0/3683820841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddd43cc550 msgr2=0x55ddd47a0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:19.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.445+0000 7f4d53fff640 1 --2- 172.21.15.96:0/3683820841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddd43cc550 0x55ddd47a0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.445+0000 7f4d53fff640 1 -- 172.21.15.96:0/3683820841 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddd43d4730 msgr2=0x55ddd47a07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:19.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.445+0000 7f4d53fff640 1 --2- 172.21.15.96:0/3683820841 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddd43d4730 0x55ddd47a07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.445+0000 7f4d53fff640 1 -- 172.21.15.96:0/3683820841 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddd4535f70 con 0x55ddd43f5410 2024-04-25T14:15:19.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.445+0000 7f4d52ffd640 1 --2- 172.21.15.96:0/3683820841 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddd43d4730 0x55ddd47a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:15:19.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.445+0000 7f4d53fff640 1 --2- 172.21.15.96:0/3683820841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddd43f5410 0x55ddd47a7730 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c0042e0 tx=0x7f4d4c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f49a9a7ed1c372c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:19.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.445+0000 7f4d32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:15:19.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.445+0000 7f4d32ffd640 1 -- 172.21.15.96:0/3683820841 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c025070 con 0x55ddd43f5410 2024-04-25T14:15:19.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.445+0000 7f4d32ffd640 1 -- 172.21.15.96:0/3683820841 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f4d4c011d60 con 0x55ddd43f5410 2024-04-25T14:15:19.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.445+0000 7f4d60982740 1 -- 172.21.15.96:0/3683820841 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ddd4709710 con 0x55ddd43f5410 2024-04-25T14:15:19.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.446+0000 7f4d32ffd640 1 -- 172.21.15.96:0/3683820841 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4d4c031050 con 0x55ddd43f5410 2024-04-25T14:15:19.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.446+0000 7f4d60982740 10 client.16182 fetch_fsmap learned FSMap version 26 2024-04-25T14:15:19.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.446+0000 7f4d60982740 1 -- 172.21.15.96:0/3683820841 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ddd43cd170 con 0x55ddd43f5410 2024-04-25T14:15:19.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.446+0000 7f4d32ffd640 1 -- 172.21.15.96:0/3683820841 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ddd4709710 con 0x55ddd43f5410 2024-04-25T14:15:19.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.446+0000 7f4d32ffd640 1 -- 172.21.15.96:0/3683820841 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4d4c00f040 con 0x55ddd43f5410 2024-04-25T14:15:19.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.446+0000 7f4d60982740 10 client.16182 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:15:19.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.446+0000 7f4d60982740 10 client.16182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:15:19.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.446+0000 7f4d60982740 20 client.16182 populate_metadata read hostname 'smithi096' 2024-04-25T14:15:19.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.446+0000 7f4d60982740 1 --2- 172.21.15.96:0/3683820841 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55ddd43b41f0 0x55ddd47adc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:19.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.446+0000 7f4d60982740 4 client.16182 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:15:19.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.446+0000 7f4d60982740 1 -- 172.21.15.96:0/3683820841 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ddd4703580 con 0x55ddd43b41f0 2024-04-25T14:15:19.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.446+0000 7f4d537fe640 1 --2- 172.21.15.96:0/3683820841 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55ddd43b41f0 0x55ddd47adc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:19.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.447+0000 7f4d537fe640 1 --2- 172.21.15.96:0/3683820841 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55ddd43b41f0 0x55ddd47adc10 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee2a5eb386b900d2 server_cookie=b3813e738653ed34 in_seq=0 out_seq=0 2024-04-25T14:15:19.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.447+0000 7f4d32ffd640 10 client.16182 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:15:19.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.447+0000 7f4d32ffd640 1 -- 172.21.15.96:0/3683820841 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ddd4703580 con 0x55ddd43b41f0 2024-04-25T14:15:19.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.447+0000 7f4d32ffd640 10 client.16182 handle_command_reply: tid=0 2024-04-25T14:15:19.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.448+0000 7f4d60982740 1 client.16182 shutdown 2024-04-25T14:15:19.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.448+0000 7f4d60982740 1 -- 172.21.15.96:0/3683820841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddd43f5410 msgr2=0x55ddd47a7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:19.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.448+0000 7f4d60982740 1 --2- 172.21.15.96:0/3683820841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddd43f5410 0x55ddd47a7730 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c0042e0 tx=0x7f4d4c004a40 comp rx=0 tx=0).stop 2024-04-25T14:15:19.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.448+0000 7f4d60982740 1 -- 172.21.15.96:0/3683820841 shutdown_connections 2024-04-25T14:15:19.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.448+0000 7f4d60982740 1 --2- 172.21.15.96:0/3683820841 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55ddd43b41f0 0x55ddd47adc10 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.448+0000 7f4d60982740 1 --2- 172.21.15.96:0/3683820841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddd43f5410 0x55ddd47a7730 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.448+0000 7f4d60982740 1 --2- 172.21.15.96:0/3683820841 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddd43d4730 0x55ddd47a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.448+0000 7f4d32ffd640 0 client.16182 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:15:19.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.448+0000 7f4d60982740 1 --2- 172.21.15.96:0/3683820841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddd43cc550 0x55ddd47a0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.448+0000 7f4d60982740 1 -- 172.21.15.96:0/3683820841 >> 172.21.15.96:0/3683820841 conn(0x55ddd43e9560 msgr2=0x55ddd4790ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:19.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.448+0000 7f4d60982740 1 -- 172.21.15.96:0/3683820841 shutdown_connections 2024-04-25T14:15:19.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.448+0000 7f4d60982740 1 -- 172.21.15.96:0/3683820841 wait complete. 2024-04-25T14:15:19.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.448+0000 7f4d60982740 20 client.16182 trim_cache size 0 max 16384 2024-04-25T14:15:19.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.467+0000 7f4d60982740 1 Processor -- start 2024-04-25T14:15:19.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.467+0000 7f4d60982740 1 -- start start 2024-04-25T14:15:19.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.467+0000 7f4d60982740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddd43cc550 0x55ddd4799dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:19.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.467+0000 7f4d60982740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddd43d4730 0x55ddd4791620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:19.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.467+0000 7f4d60982740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddd43f5410 0x55ddd4791b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:19.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.467+0000 7f4d60982740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4d541ab3e0 con 0x55ddd43d4730 2024-04-25T14:15:19.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.467+0000 7f4d60982740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4d541acf20 con 0x55ddd43cc550 2024-04-25T14:15:19.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.467+0000 7f4d60982740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4d541abab0 con 0x55ddd43f5410 2024-04-25T14:15:19.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.468+0000 7f4d52ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddd43d4730 0x55ddd4791620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:19.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.468+0000 7f4d52ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddd43d4730 0x55ddd4791620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41756/0 (socket says 172.21.15.96:41756) 2024-04-25T14:15:19.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.468+0000 7f4d52ffd640 1 -- 172.21.15.96:0/2444917514 learned_addr learned my addr 172.21.15.96:0/2444917514 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:19.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.468+0000 7f4d53fff640 1 --2- 172.21.15.96:0/2444917514 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddd43f5410 0x55ddd4791b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:19.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.468+0000 7f4d537fe640 1 --2- 172.21.15.96:0/2444917514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddd43cc550 0x55ddd4799dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:19.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.468+0000 7f4d52ffd640 1 -- 172.21.15.96:0/2444917514 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddd43f5410 msgr2=0x55ddd4791b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:19.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.468+0000 7f4d52ffd640 1 --2- 172.21.15.96:0/2444917514 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddd43f5410 0x55ddd4791b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.468+0000 7f4d52ffd640 1 -- 172.21.15.96:0/2444917514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddd43cc550 msgr2=0x55ddd4799dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:19.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.468+0000 7f4d52ffd640 1 --2- 172.21.15.96:0/2444917514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddd43cc550 0x55ddd4799dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.468+0000 7f4d52ffd640 1 -- 172.21.15.96:0/2444917514 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddd479b190 con 0x55ddd43d4730 2024-04-25T14:15:19.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.468+0000 7f4d53fff640 1 --2- 172.21.15.96:0/2444917514 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddd43f5410 0x55ddd4791b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:15:19.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.468+0000 7f4d537fe640 1 --2- 172.21.15.96:0/2444917514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddd43cc550 0x55ddd4799dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:15:19.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.468+0000 7f4d52ffd640 1 --2- 172.21.15.96:0/2444917514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddd43d4730 0x55ddd4791620 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f4d480e6b20 tx=0x7f4d48017550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3db8658933788e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:19.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.468+0000 7f4d397fa640 1 -- 172.21.15.96:0/2444917514 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d48081070 con 0x55ddd43d4730 2024-04-25T14:15:19.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.468+0000 7f4d397fa640 1 -- 172.21.15.96:0/2444917514 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d480a5fd0 con 0x55ddd43d4730 2024-04-25T14:15:19.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.468+0000 7f4d397fa640 1 -- 172.21.15.96:0/2444917514 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4801e040 con 0x55ddd43d4730 2024-04-25T14:15:19.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.469+0000 7f4d60982740 1 -- 172.21.15.96:0/2444917514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddd43d4730 msgr2=0x55ddd4791620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:19.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.469+0000 7f4d60982740 1 --2- 172.21.15.96:0/2444917514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddd43d4730 0x55ddd4791620 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f4d480e6b20 tx=0x7f4d48017550 comp rx=0 tx=0).stop 2024-04-25T14:15:19.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.469+0000 7f4d60982740 1 -- 172.21.15.96:0/2444917514 shutdown_connections 2024-04-25T14:15:19.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.469+0000 7f4d60982740 1 --2- 172.21.15.96:0/2444917514 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddd43f5410 0x55ddd4791b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.469+0000 7f4d60982740 1 --2- 172.21.15.96:0/2444917514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddd43d4730 0x55ddd4791620 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.469+0000 7f4d60982740 1 --2- 172.21.15.96:0/2444917514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddd43cc550 0x55ddd4799dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.469+0000 7f4d60982740 1 -- 172.21.15.96:0/2444917514 >> 172.21.15.96:0/2444917514 conn(0x55ddd43e9560 msgr2=0x55ddd47996b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:19.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.469+0000 7f4d60982740 1 -- 172.21.15.96:0/2444917514 shutdown_connections 2024-04-25T14:15:19.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.469+0000 7f4d60982740 1 -- 172.21.15.96:0/2444917514 wait complete. 2024-04-25T14:15:19.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.470+0000 7f4d60982740 1 -- start start 2024-04-25T14:15:19.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.470+0000 7f4d60982740 1 Processor -- start 2024-04-25T14:15:19.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.470+0000 7f4d60982740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddd43cc550 0x55ddd47a4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:19.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.470+0000 7f4d60982740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddd43d4730 0x55ddd47a4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:19.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.470+0000 7f4d60982740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddd43f5410 0x55ddd47a5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:19.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.470+0000 7f4d60982740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4d541ac850 con 0x55ddd43d4730 2024-04-25T14:15:19.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.470+0000 7f4d60982740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4d541ac180 con 0x55ddd43f5410 2024-04-25T14:15:19.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.470+0000 7f4d60982740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4d541a93c0 con 0x55ddd43cc550 2024-04-25T14:15:19.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.470+0000 7f4d60982740 10 client.0 _ll_register_callbacks cb 0x7f4d4c003db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:15:19.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.471+0000 7f4d52ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddd43d4730 0x55ddd47a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:19.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.471+0000 7f4d53fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddd43f5410 0x55ddd47a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:19.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.471+0000 7f4d52ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddd43d4730 0x55ddd47a4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41760/0 (socket says 172.21.15.96:41760) 2024-04-25T14:15:19.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.471+0000 7f4d53fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddd43f5410 0x55ddd47a5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:42160/0 (socket says 172.21.15.96:42160) 2024-04-25T14:15:19.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.471+0000 7f4d52ffd640 1 -- 172.21.15.96:0/1373173404 learned_addr learned my addr 172.21.15.96:0/1373173404 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:19.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.471+0000 7f4d537fe640 1 --2- 172.21.15.96:0/1373173404 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddd43cc550 0x55ddd47a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:19.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.471+0000 7f4d52ffd640 1 -- 172.21.15.96:0/1373173404 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddd43cc550 msgr2=0x55ddd47a4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:19.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.471+0000 7f4d52ffd640 1 --2- 172.21.15.96:0/1373173404 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddd43cc550 0x55ddd47a4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.471+0000 7f4d52ffd640 1 -- 172.21.15.96:0/1373173404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddd43f5410 msgr2=0x55ddd47a5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:19.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.471+0000 7f4d52ffd640 1 --2- 172.21.15.96:0/1373173404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddd43f5410 0x55ddd47a5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.471+0000 7f4d52ffd640 1 -- 172.21.15.96:0/1373173404 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddd47a5df0 con 0x55ddd43d4730 2024-04-25T14:15:19.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.471+0000 7f4d53fff640 1 --2- 172.21.15.96:0/1373173404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddd43f5410 0x55ddd47a5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:15:19.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.471+0000 7f4d537fe640 1 --2- 172.21.15.96:0/1373173404 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddd43cc550 0x55ddd47a4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:15:19.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.471+0000 7f4d52ffd640 1 --2- 172.21.15.96:0/1373173404 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddd43d4730 0x55ddd47a4950 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f4d480a61c0 tx=0x7f4d480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95feafd02f98e92c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:19.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.471+0000 7f4d32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:15:19.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.471+0000 7f4d32ffd640 1 -- 172.21.15.96:0/1373173404 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4801e070 con 0x55ddd43d4730 2024-04-25T14:15:19.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.471+0000 7f4d32ffd640 1 -- 172.21.15.96:0/1373173404 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f4d480e1b60 con 0x55ddd43d4730 2024-04-25T14:15:19.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.471+0000 7f4d60982740 1 -- 172.21.15.96:0/1373173404 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d480a5fd0 con 0x55ddd43d4730 2024-04-25T14:15:19.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.471+0000 7f4d32ffd640 1 -- 172.21.15.96:0/1373173404 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4d48081070 con 0x55ddd43d4730 2024-04-25T14:15:19.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.471+0000 7f4d60982740 10 client.16188 fetch_fsmap learned FSMap version 26 2024-04-25T14:15:19.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.471+0000 7f4d60982740 1 -- 172.21.15.96:0/1373173404 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ddd47a70b0 con 0x55ddd43d4730 2024-04-25T14:15:19.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.472+0000 7f4d32ffd640 1 -- 172.21.15.96:0/1373173404 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d480a5fd0 con 0x55ddd43d4730 2024-04-25T14:15:19.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.472+0000 7f4d32ffd640 1 -- 172.21.15.96:0/1373173404 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4d480a6720 con 0x55ddd43d4730 2024-04-25T14:15:19.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.472+0000 7f4d60982740 10 client.16188 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:15:19.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.472+0000 7f4d60982740 10 client.16188 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:15:19.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.472+0000 7f4d60982740 20 client.16188 populate_metadata read hostname 'smithi096' 2024-04-25T14:15:19.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.472+0000 7f4d60982740 1 --2- 172.21.15.96:0/1373173404 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55ddd4389630 0x55ddd4818210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:19.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.472+0000 7f4d60982740 4 client.16188 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T14:15:19.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.472+0000 7f4d60982740 1 -- 172.21.15.96:0/1373173404 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ddd467eac0 con 0x55ddd4389630 2024-04-25T14:15:19.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.472+0000 7f4d537fe640 1 --2- 172.21.15.96:0/1373173404 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55ddd4389630 0x55ddd4818210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:19.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.472+0000 7f4d32ffd640 10 client.16188 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:15:19.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.473+0000 7f4d537fe640 1 --2- 172.21.15.96:0/1373173404 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55ddd4389630 0x55ddd4818210 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce8a1e91e194f08d server_cookie=3d954151ad929a2c in_seq=0 out_seq=0 2024-04-25T14:15:19.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.475+0000 7f4d32ffd640 1 -- 172.21.15.96:0/1373173404 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ddd467eac0 con 0x55ddd4389630 2024-04-25T14:15:19.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.475+0000 7f4d32ffd640 10 client.16188 handle_command_reply: tid=0 2024-04-25T14:15:19.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.476+0000 7f4d60982740 1 client.16188 shutdown 2024-04-25T14:15:19.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.476+0000 7f4d60982740 1 -- 172.21.15.96:0/1373173404 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddd43d4730 msgr2=0x55ddd47a4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:19.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.476+0000 7f4d60982740 1 --2- 172.21.15.96:0/1373173404 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddd43d4730 0x55ddd47a4950 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f4d480a61c0 tx=0x7f4d480046c0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.476+0000 7f4d60982740 1 -- 172.21.15.96:0/1373173404 shutdown_connections 2024-04-25T14:15:19.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.476+0000 7f4d60982740 1 --2- 172.21.15.96:0/1373173404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddd43f5410 0x55ddd47a5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.476+0000 7f4d60982740 1 --2- 172.21.15.96:0/1373173404 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55ddd4389630 0x55ddd4818210 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.476+0000 7f4d60982740 1 --2- 172.21.15.96:0/1373173404 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddd43d4730 0x55ddd47a4950 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.476+0000 7f4d32ffd640 0 client.16188 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:15:19.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.476+0000 7f4d60982740 1 --2- 172.21.15.96:0/1373173404 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddd43cc550 0x55ddd47a4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.476+0000 7f4d60982740 1 -- 172.21.15.96:0/1373173404 >> 172.21.15.96:0/1373173404 conn(0x55ddd43e9560 msgr2=0x55ddd4794c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:19.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.476+0000 7f4d60982740 1 -- 172.21.15.96:0/1373173404 shutdown_connections 2024-04-25T14:15:19.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.476+0000 7f4d60982740 1 -- 172.21.15.96:0/1373173404 wait complete. 2024-04-25T14:15:19.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.476+0000 7f4d60982740 20 client.16188 trim_cache size 0 max 16384 2024-04-25T14:15:19.480 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T14:15:19.480 INFO:teuthology.orchestra.run.smithi096.stdout: "return_code": 0, 2024-04-25T14:15:19.480 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_tag": "3d74358d-1e7c-4b2d-bf36-a5ada163014d", 2024-04-25T14:15:19.480 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": "asynchronous" 2024-04-25T14:15:19.481 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T14:15:19.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.476+0000 7f4d59ad9640 1 -- 172.21.15.96:0/3493180985 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f4d3c0afaf0 msgr2=0x7f4d3c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:19.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.476+0000 7f4d59ad9640 1 --2- 172.21.15.96:0/3493180985 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f4d3c0afaf0 0x7f4d3c0b2000 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55ddd4797670 tx=0x7f4d4c009600 comp rx=0 tx=0).stop 2024-04-25T14:15:19.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.477+0000 7f4d59ad9640 1 -- 172.21.15.96:0/3493180985 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d54102350 msgr2=0x7f4d5419a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:19.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.477+0000 7f4d59ad9640 1 --2- 172.21.15.96:0/3493180985 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d54102350 0x7f4d5419a7b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f4d4800b4a0 tx=0x7f4d4800b960 comp rx=0 tx=0).stop 2024-04-25T14:15:19.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.477+0000 7f4d59ad9640 1 -- 172.21.15.96:0/3493180985 shutdown_connections 2024-04-25T14:15:19.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.477+0000 7f4d59ad9640 1 --2- 172.21.15.96:0/3493180985 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d54109770 0x7f4d541a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.477+0000 7f4d59ad9640 1 --2- 172.21.15.96:0/3493180985 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f4d3c0afaf0 0x7f4d3c0b2000 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.477+0000 7f4d59ad9640 1 --2- 172.21.15.96:0/3493180985 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d54102350 0x7f4d5419a7b0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.477+0000 7f4d59ad9640 1 --2- 172.21.15.96:0/3493180985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d54101e50 0x7f4d5419a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.477+0000 7f4d59ad9640 1 -- 172.21.15.96:0/3493180985 >> 172.21.15.96:0/3493180985 conn(0x7f4d540fd340 msgr2=0x7f4d54106740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:19.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.477+0000 7f4d59ad9640 1 -- 172.21.15.96:0/3493180985 shutdown_connections 2024-04-25T14:15:19.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.477+0000 7f4d59ad9640 1 -- 172.21.15.96:0/3493180985 wait complete. 2024-04-25T14:15:19.489 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:15:19.489 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:15:19.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.612+0000 7fcf1cb9d640 1 -- 172.21.15.96:0/443233268 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf1810ba70 msgr2=0x7fcf1810de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:19.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.612+0000 7fcf1cb9d640 1 --2- 172.21.15.96:0/443233268 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf1810ba70 0x7fcf1810de90 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fcf10009ca0 tx=0x7fcf1002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.612+0000 7fcf1cb9d640 1 -- 172.21.15.96:0/443233268 shutdown_connections 2024-04-25T14:15:19.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.612+0000 7fcf1cb9d640 1 --2- 172.21.15.96:0/443233268 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf1810ba70 0x7fcf1810de90 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.612+0000 7fcf1cb9d640 1 --2- 172.21.15.96:0/443233268 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf18106d70 0x7fcf18107180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.612+0000 7fcf1cb9d640 1 --2- 172.21.15.96:0/443233268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf18105b30 0x7fcf18106830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.612+0000 7fcf1cb9d640 1 -- 172.21.15.96:0/443233268 >> 172.21.15.96:0/443233268 conn(0x7fcf18100d20 msgr2=0x7fcf181031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:19.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.612+0000 7fcf1cb9d640 1 -- 172.21.15.96:0/443233268 shutdown_connections 2024-04-25T14:15:19.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.612+0000 7fcf1cb9d640 1 -- 172.21.15.96:0/443233268 wait complete. 2024-04-25T14:15:19.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.613+0000 7fcf1cb9d640 1 Processor -- start 2024-04-25T14:15:19.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.613+0000 7fcf1cb9d640 1 -- start start 2024-04-25T14:15:19.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.613+0000 7fcf1cb9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf18105b30 0x7fcf1819c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:19.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.613+0000 7fcf1cb9d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf18106d70 0x7fcf1819c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:19.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.613+0000 7fcf1cb9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf1810ba70 0x7fcf181a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:19.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.613+0000 7fcf1cb9d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcf18111660 con 0x7fcf18105b30 2024-04-25T14:15:19.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.613+0000 7fcf1cb9d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcf181114e0 con 0x7fcf1810ba70 2024-04-25T14:15:19.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.613+0000 7fcf1cb9d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcf181117e0 con 0x7fcf18106d70 2024-04-25T14:15:19.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.613+0000 7fcf177fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf18105b30 0x7fcf1819c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:19.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.613+0000 7fcf177fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf18105b30 0x7fcf1819c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41784/0 (socket says 172.21.15.96:41784) 2024-04-25T14:15:19.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.613+0000 7fcf177fe640 1 -- 172.21.15.96:0/1989906658 learned_addr learned my addr 172.21.15.96:0/1989906658 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:19.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.613+0000 7fcf17fff640 1 --2- 172.21.15.96:0/1989906658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf1810ba70 0x7fcf181a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:19.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.613+0000 7fcf16ffd640 1 --2- 172.21.15.96:0/1989906658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf18106d70 0x7fcf1819c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:19.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.613+0000 7fcf177fe640 1 -- 172.21.15.96:0/1989906658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf18106d70 msgr2=0x7fcf1819c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:19.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.613+0000 7fcf177fe640 1 --2- 172.21.15.96:0/1989906658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf18106d70 0x7fcf1819c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.613+0000 7fcf177fe640 1 -- 172.21.15.96:0/1989906658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf1810ba70 msgr2=0x7fcf181a36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:19.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.613+0000 7fcf177fe640 1 --2- 172.21.15.96:0/1989906658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf1810ba70 0x7fcf181a36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.613+0000 7fcf177fe640 1 -- 172.21.15.96:0/1989906658 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf181a3d40 con 0x7fcf18105b30 2024-04-25T14:15:19.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.614+0000 7fcf177fe640 1 --2- 172.21.15.96:0/1989906658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf18105b30 0x7fcf1819c200 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fcf0800e9f0 tx=0x7fcf0800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:19.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.614+0000 7fcf14ff9640 1 -- 172.21.15.96:0/1989906658 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0800ccc0 con 0x7fcf18105b30 2024-04-25T14:15:19.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.614+0000 7fcf14ff9640 1 -- 172.21.15.96:0/1989906658 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf080035f0 con 0x7fcf18105b30 2024-04-25T14:15:19.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.614+0000 7fcf1cb9d640 1 -- 172.21.15.96:0/1989906658 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf181a3fd0 con 0x7fcf18105b30 2024-04-25T14:15:19.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.614+0000 7fcf14ff9640 1 -- 172.21.15.96:0/1989906658 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0800ce60 con 0x7fcf18105b30 2024-04-25T14:15:19.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.614+0000 7fcf1cb9d640 1 -- 172.21.15.96:0/1989906658 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf181a4490 con 0x7fcf18105b30 2024-04-25T14:15:19.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.617+0000 7fcf14ff9640 1 -- 172.21.15.96:0/1989906658 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fcf08010690 con 0x7fcf18105b30 2024-04-25T14:15:19.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.617+0000 7fcf24d87740 1 Processor -- start 2024-04-25T14:15:19.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.617+0000 7fcf24d87740 1 -- start start 2024-04-25T14:15:19.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.617+0000 7fcf24d87740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595e0ffd390 0x5595e139ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:19.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.617+0000 7fcf24d87740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595e0fd44d0 0x5595e139f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:19.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.617+0000 7fcf24d87740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595e0fdc6b0 0x5595e139fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:19.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.617+0000 7fcf24d87740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5595e1303d80 con 0x5595e0fd44d0 2024-04-25T14:15:19.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.617+0000 7fcf24d87740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5595e12bd8b0 con 0x5595e0ffd390 2024-04-25T14:15:19.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.617+0000 7fcf24d87740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5595e12ffd40 con 0x5595e0fdc6b0 2024-04-25T14:15:19.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.617+0000 7fcf14ff9640 1 --2- 172.21.15.96:0/1989906658 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fcf000afae0 0x7fcf000b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:19.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.617+0000 7fcf16ffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595e0fdc6b0 0x5595e139fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:19.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.617+0000 7fcf16ffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595e0fdc6b0 0x5595e139fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.96:51276/0 (socket says 172.21.15.96:51276) 2024-04-25T14:15:19.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.617+0000 7fcf17fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595e0ffd390 0x5595e139ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:19.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.617+0000 7fcf16ffd640 1 -- 172.21.15.96:0/1191896409 learned_addr learned my addr 172.21.15.96:0/1191896409 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:19.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.617+0000 7fcf14ff9640 1 -- 172.21.15.96:0/1989906658 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fcf080defb0 con 0x7fcf18105b30 2024-04-25T14:15:19.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.617+0000 7fcf17fff640 1 --2- 172.21.15.96:0/1989906658 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fcf000afae0 0x7fcf000b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:19.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.617+0000 7fcf177fe640 1 --2- 172.21.15.96:0/1191896409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595e0fd44d0 0x5595e139f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:19.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.618+0000 7fcf16ffd640 1 -- 172.21.15.96:0/1191896409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595e0ffd390 msgr2=0x5595e139ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:19.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.618+0000 7fcf16ffd640 1 --2- 172.21.15.96:0/1191896409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595e0ffd390 0x5595e139ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.618+0000 7fcf16ffd640 1 -- 172.21.15.96:0/1191896409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595e0fd44d0 msgr2=0x5595e139f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:19.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.618+0000 7fcf16ffd640 1 --2- 172.21.15.96:0/1191896409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595e0fd44d0 0x5595e139f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.618+0000 7fcf16ffd640 1 -- 172.21.15.96:0/1191896409 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595e128ea40 con 0x5595e0fdc6b0 2024-04-25T14:15:19.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.618+0000 7fcf16ffd640 1 --2- 172.21.15.96:0/1191896409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595e0fdc6b0 0x5595e139fe60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c009c20 tx=0x7fcf0c0117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=492000fd03c311bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:19.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.618+0000 7fcf17fff640 1 --2- 172.21.15.96:0/1989906658 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fcf000afae0 0x7fcf000b1ff0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fcf100094e0 tx=0x7fcf1002a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:19.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.618+0000 7fcefdffb640 1 -- 172.21.15.96:0/1191896409 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c004030 con 0x5595e0fdc6b0 2024-04-25T14:15:19.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.618+0000 7fcefdffb640 1 -- 172.21.15.96:0/1191896409 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf0c0035f0 con 0x5595e0fdc6b0 2024-04-25T14:15:19.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.618+0000 7fcefdffb640 1 -- 172.21.15.96:0/1191896409 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c012650 con 0x5595e0fdc6b0 2024-04-25T14:15:19.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.619+0000 7fcf24d87740 1 -- 172.21.15.96:0/1191896409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595e0fdc6b0 msgr2=0x5595e139fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:19.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.619+0000 7fcf24d87740 1 --2- 172.21.15.96:0/1191896409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595e0fdc6b0 0x5595e139fe60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c009c20 tx=0x7fcf0c0117b0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.619+0000 7fcf24d87740 1 -- 172.21.15.96:0/1191896409 shutdown_connections 2024-04-25T14:15:19.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.619+0000 7fcf24d87740 1 --2- 172.21.15.96:0/1191896409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595e0fdc6b0 0x5595e139fe60 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.619+0000 7fcf24d87740 1 --2- 172.21.15.96:0/1191896409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595e0fd44d0 0x5595e139f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.619+0000 7fcf24d87740 1 --2- 172.21.15.96:0/1191896409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595e0ffd390 0x5595e139ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.619+0000 7fcf24d87740 1 -- 172.21.15.96:0/1191896409 >> 172.21.15.96:0/1191896409 conn(0x5595e0ff14e0 msgr2=0x5595e139a460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:19.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.619+0000 7fcf24d87740 1 -- 172.21.15.96:0/1191896409 shutdown_connections 2024-04-25T14:15:19.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.619+0000 7fcf24d87740 1 -- 172.21.15.96:0/1191896409 wait complete. 2024-04-25T14:15:19.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.620+0000 7fcf24d87740 1 -- start start 2024-04-25T14:15:19.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.620+0000 7fcf24d87740 1 Processor -- start 2024-04-25T14:15:19.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.620+0000 7fcf24d87740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595e0fd44d0 0x5595e13a8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:19.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.620+0000 7fcf24d87740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595e0fdc6b0 0x5595e13a8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:19.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.620+0000 7fcf24d87740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595e0ffd390 0x5595e13aff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:19.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.620+0000 7fcf24d87740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5595e13a0970 con 0x5595e0ffd390 2024-04-25T14:15:19.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.620+0000 7fcf24d87740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5595e128ad10 con 0x5595e0fd44d0 2024-04-25T14:15:19.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.620+0000 7fcf24d87740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5595e13a0af0 con 0x5595e0fdc6b0 2024-04-25T14:15:19.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.620+0000 7fcf24d87740 10 client.0 _ll_register_callbacks cb 0x5595e12b7210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:15:19.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.620+0000 7fcf16ffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595e0fdc6b0 0x5595e13a8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:19.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.620+0000 7fcf17fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595e0ffd390 0x5595e13aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:19.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.620+0000 7fcf16ffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595e0fdc6b0 0x5595e13a8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.96:51286/0 (socket says 172.21.15.96:51286) 2024-04-25T14:15:19.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.620+0000 7fcf177fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595e0fd44d0 0x5595e13a8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:19.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.620+0000 7fcf16ffd640 1 -- 172.21.15.96:0/4096994053 learned_addr learned my addr 172.21.15.96:0/4096994053 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:19.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.621+0000 7fcf177fe640 1 -- 172.21.15.96:0/4096994053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595e0fdc6b0 msgr2=0x5595e13a8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:19.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.621+0000 7fcf177fe640 1 --2- 172.21.15.96:0/4096994053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595e0fdc6b0 0x5595e13a8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.621+0000 7fcf177fe640 1 -- 172.21.15.96:0/4096994053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595e0ffd390 msgr2=0x5595e13aff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:19.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.621+0000 7fcf177fe640 1 --2- 172.21.15.96:0/4096994053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595e0ffd390 0x5595e13aff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.621+0000 7fcf177fe640 1 -- 172.21.15.96:0/4096994053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595e10df7d0 con 0x5595e0fd44d0 2024-04-25T14:15:19.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.621+0000 7fcf177fe640 1 --2- 172.21.15.96:0/4096994053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595e0fd44d0 0x5595e13a8a60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcf080dcae0 tx=0x7fcf080a7b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f05511d000b85ec8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:19.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.621+0000 7fcef6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T14:15:19.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.621+0000 7fcef6ffd640 1 -- 172.21.15.96:0/4096994053 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf080b02d0 con 0x5595e0fd44d0 2024-04-25T14:15:19.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.621+0000 7fcf24d87740 1 -- 172.21.15.96:0/4096994053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf0c0035f0 con 0x5595e0fd44d0 2024-04-25T14:15:19.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.621+0000 7fcef6ffd640 1 -- 172.21.15.96:0/4096994053 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fcf08003aa0 con 0x5595e0fd44d0 2024-04-25T14:15:19.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.621+0000 7fcef6ffd640 1 -- 172.21.15.96:0/4096994053 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcf08003ca0 con 0x5595e0fd44d0 2024-04-25T14:15:19.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.621+0000 7fcf24d87740 10 client.25688 fetch_fsmap learned FSMap version 26 2024-04-25T14:15:19.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.621+0000 7fcf24d87740 1 -- 172.21.15.96:0/4096994053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5595e10153f0 con 0x5595e0fd44d0 2024-04-25T14:15:19.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.622+0000 7fcef6ffd640 1 -- 172.21.15.96:0/4096994053 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf0c0035f0 con 0x5595e0fd44d0 2024-04-25T14:15:19.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.622+0000 7fcef6ffd640 1 -- 172.21.15.96:0/4096994053 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fcf08004140 con 0x5595e0fd44d0 2024-04-25T14:15:19.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.622+0000 7fcf24d87740 10 client.25688 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:15:19.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.622+0000 7fcf24d87740 10 client.25688 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:15:19.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.622+0000 7fcf24d87740 20 client.25688 populate_metadata read hostname 'smithi096' 2024-04-25T14:15:19.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.622+0000 7fcf24d87740 1 --2- 172.21.15.96:0/4096994053 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5595e0fbbf60 0x5595e13b6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:19.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.622+0000 7fcf24d87740 4 client.25688 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:15:19.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.622+0000 7fcf24d87740 1 -- 172.21.15.96:0/4096994053 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595e1311990 con 0x5595e0fbbf60 2024-04-25T14:15:19.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.622+0000 7fcf16ffd640 1 --2- 172.21.15.96:0/4096994053 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5595e0fbbf60 0x5595e13b6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:19.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.622+0000 7fcef6ffd640 10 client.25688 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:15:19.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.623+0000 7fcf16ffd640 1 --2- 172.21.15.96:0/4096994053 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5595e0fbbf60 0x5595e13b6430 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bd6d7e9fc988c71 server_cookie=bc4572e075dee4b4 in_seq=0 out_seq=0 2024-04-25T14:15:19.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.624+0000 7fcef6ffd640 1 -- 172.21.15.96:0/4096994053 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5595e1311990 con 0x5595e0fbbf60 2024-04-25T14:15:19.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.624+0000 7fcef6ffd640 10 client.25688 handle_command_reply: tid=0 2024-04-25T14:15:19.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.624+0000 7fcf24d87740 1 client.25688 shutdown 2024-04-25T14:15:19.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.624+0000 7fcf24d87740 1 -- 172.21.15.96:0/4096994053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595e0fd44d0 msgr2=0x5595e13a8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:19.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.624+0000 7fcf24d87740 1 --2- 172.21.15.96:0/4096994053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595e0fd44d0 0x5595e13a8a60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcf080dcae0 tx=0x7fcf080a7b70 comp rx=0 tx=0).stop 2024-04-25T14:15:19.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.624+0000 7fcf24d87740 1 -- 172.21.15.96:0/4096994053 shutdown_connections 2024-04-25T14:15:19.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.624+0000 7fcf24d87740 1 --2- 172.21.15.96:0/4096994053 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5595e0fbbf60 0x5595e13b6430 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.624+0000 7fcf24d87740 1 --2- 172.21.15.96:0/4096994053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595e0ffd390 0x5595e13aff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.624+0000 7fcf24d87740 1 --2- 172.21.15.96:0/4096994053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595e0fdc6b0 0x5595e13a8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.624+0000 7fcf24d87740 1 --2- 172.21.15.96:0/4096994053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595e0fd44d0 0x5595e13a8a60 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.624+0000 7fcef6ffd640 0 client.25688 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:15:19.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.624+0000 7fcf24d87740 1 -- 172.21.15.96:0/4096994053 >> 172.21.15.96:0/4096994053 conn(0x5595e0ff14e0 msgr2=0x5595e1399420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:19.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.624+0000 7fcf24d87740 1 -- 172.21.15.96:0/4096994053 shutdown_connections 2024-04-25T14:15:19.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.624+0000 7fcf24d87740 1 -- 172.21.15.96:0/4096994053 wait complete. 2024-04-25T14:15:19.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.624+0000 7fcf24d87740 20 client.25688 trim_cache size 0 max 16384 2024-04-25T14:15:19.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.643+0000 7fcf24d87740 1 Processor -- start 2024-04-25T14:15:19.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.643+0000 7fcf24d87740 1 -- start start 2024-04-25T14:15:19.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.643+0000 7fcf24d87740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595e0fd44d0 0x5595e13a70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:19.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.643+0000 7fcf24d87740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595e0fdc6b0 0x5595e13a7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:19.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.643+0000 7fcf24d87740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595e0ffd390 0x5595e13a8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:19.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.643+0000 7fcf24d87740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcf181ad320 con 0x5595e0ffd390 2024-04-25T14:15:19.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.643+0000 7fcf24d87740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcf181aee60 con 0x5595e0fd44d0 2024-04-25T14:15:19.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.643+0000 7fcf24d87740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcf181ad9f0 con 0x5595e0fdc6b0 2024-04-25T14:15:19.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.644+0000 7fcf17fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595e0ffd390 0x5595e13a8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:19.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.644+0000 7fcf177fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595e0fd44d0 0x5595e13a70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:19.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.644+0000 7fcf17fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595e0ffd390 0x5595e13a8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41798/0 (socket says 172.21.15.96:41798) 2024-04-25T14:15:19.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.644+0000 7fcf17fff640 1 -- 172.21.15.96:0/3396300689 learned_addr learned my addr 172.21.15.96:0/3396300689 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:19.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.644+0000 7fcf16ffd640 1 --2- 172.21.15.96:0/3396300689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595e0fdc6b0 0x5595e13a7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:19.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.644+0000 7fcf177fe640 1 -- 172.21.15.96:0/3396300689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595e0fdc6b0 msgr2=0x5595e13a7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:19.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.644+0000 7fcf177fe640 1 --2- 172.21.15.96:0/3396300689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595e0fdc6b0 0x5595e13a7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.644+0000 7fcf177fe640 1 -- 172.21.15.96:0/3396300689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595e0ffd390 msgr2=0x5595e13a8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:19.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.644+0000 7fcf177fe640 1 --2- 172.21.15.96:0/3396300689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595e0ffd390 0x5595e13a8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.644+0000 7fcf177fe640 1 -- 172.21.15.96:0/3396300689 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595e0fd50f0 con 0x5595e0fd44d0 2024-04-25T14:15:19.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.644+0000 7fcf177fe640 1 --2- 172.21.15.96:0/3396300689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595e0fd44d0 0x5595e13a70f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcf080a3a10 tx=0x7fcf08003a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=822ae642143ca624 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:19.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.644+0000 7fcefd7fa640 1 -- 172.21.15.96:0/3396300689 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf080a2fa0 con 0x5595e0fd44d0 2024-04-25T14:15:19.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.644+0000 7fcefd7fa640 1 -- 172.21.15.96:0/3396300689 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf080a3140 con 0x5595e0fd44d0 2024-04-25T14:15:19.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.644+0000 7fcefd7fa640 1 -- 172.21.15.96:0/3396300689 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf08003fd0 con 0x5595e0fd44d0 2024-04-25T14:15:19.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.645+0000 7fcf24d87740 1 -- 172.21.15.96:0/3396300689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595e0fd44d0 msgr2=0x5595e13a70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:19.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.645+0000 7fcf24d87740 1 --2- 172.21.15.96:0/3396300689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595e0fd44d0 0x5595e13a70f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcf080a3a10 tx=0x7fcf08003a10 comp rx=0 tx=0).stop 2024-04-25T14:15:19.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.645+0000 7fcf24d87740 1 -- 172.21.15.96:0/3396300689 shutdown_connections 2024-04-25T14:15:19.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.645+0000 7fcf24d87740 1 --2- 172.21.15.96:0/3396300689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595e0ffd390 0x5595e13a8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.645+0000 7fcf24d87740 1 --2- 172.21.15.96:0/3396300689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595e0fdc6b0 0x5595e13a7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.645+0000 7fcf24d87740 1 --2- 172.21.15.96:0/3396300689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595e0fd44d0 0x5595e13a70f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.645+0000 7fcf24d87740 1 -- 172.21.15.96:0/3396300689 >> 172.21.15.96:0/3396300689 conn(0x5595e0ff14e0 msgr2=0x5595e13a37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:19.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.645+0000 7fcf24d87740 1 -- 172.21.15.96:0/3396300689 shutdown_connections 2024-04-25T14:15:19.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.645+0000 7fcf24d87740 1 -- 172.21.15.96:0/3396300689 wait complete. 2024-04-25T14:15:19.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.646+0000 7fcf24d87740 1 -- start start 2024-04-25T14:15:19.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.646+0000 7fcf24d87740 1 Processor -- start 2024-04-25T14:15:19.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.646+0000 7fcf24d87740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595e0fd44d0 0x5595e13ac750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:19.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.646+0000 7fcf24d87740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595e0fdc6b0 0x5595e13acc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:19.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.646+0000 7fcf24d87740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595e0ffd390 0x5595e13adbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:19.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.646+0000 7fcf24d87740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcf181ae790 con 0x5595e0ffd390 2024-04-25T14:15:19.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.646+0000 7fcf24d87740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcf181ae0c0 con 0x5595e0fdc6b0 2024-04-25T14:15:19.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.646+0000 7fcf24d87740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcf181ab300 con 0x5595e0fd44d0 2024-04-25T14:15:19.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.646+0000 7fcf24d87740 10 client.0 _ll_register_callbacks cb 0x7fcf080a6e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:15:19.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.646+0000 7fcf17fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595e0ffd390 0x5595e13adbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:19.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.646+0000 7fcf16ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595e0fdc6b0 0x5595e13acc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:19.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.646+0000 7fcf177fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595e0fd44d0 0x5595e13ac750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:19.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.646+0000 7fcf17fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595e0ffd390 0x5595e13adbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41810/0 (socket says 172.21.15.96:41810) 2024-04-25T14:15:19.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.646+0000 7fcf17fff640 1 -- 172.21.15.96:0/3238552109 learned_addr learned my addr 172.21.15.96:0/3238552109 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:19.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.646+0000 7fcf16ffd640 1 -- 172.21.15.96:0/3238552109 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595e0fd44d0 msgr2=0x5595e13ac750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:19.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.646+0000 7fcf16ffd640 1 --2- 172.21.15.96:0/3238552109 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595e0fd44d0 0x5595e13ac750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.646+0000 7fcf16ffd640 1 -- 172.21.15.96:0/3238552109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595e0ffd390 msgr2=0x5595e13adbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:19.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.646+0000 7fcf16ffd640 1 --2- 172.21.15.96:0/3238552109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595e0ffd390 0x5595e13adbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.646+0000 7fcf16ffd640 1 -- 172.21.15.96:0/3238552109 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595e12c44b0 con 0x5595e0fdc6b0 2024-04-25T14:15:19.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.647+0000 7fcf17fff640 1 --2- 172.21.15.96:0/3238552109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595e0ffd390 0x5595e13adbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:15:19.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.647+0000 7fcf16ffd640 1 --2- 172.21.15.96:0/3238552109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595e0fdc6b0 0x5595e13acc90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x5595e13b6b30 tx=0x7fcf0c0036e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7645ab9a56f56bb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:19.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.647+0000 7fcef6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T14:15:19.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.647+0000 7fcef6ffd640 1 -- 172.21.15.96:0/3238552109 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c01f040 con 0x5595e0fdc6b0 2024-04-25T14:15:19.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.647+0000 7fcef6ffd640 1 -- 172.21.15.96:0/3238552109 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fcf0c0038f0 con 0x5595e0fdc6b0 2024-04-25T14:15:19.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.647+0000 7fcf24d87740 1 -- 172.21.15.96:0/3238552109 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5595e130b800 con 0x5595e0fdc6b0 2024-04-25T14:15:19.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.647+0000 7fcef6ffd640 1 -- 172.21.15.96:0/3238552109 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcf0c003b10 con 0x5595e0fdc6b0 2024-04-25T14:15:19.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.647+0000 7fcf24d87740 10 client.25694 fetch_fsmap learned FSMap version 26 2024-04-25T14:15:19.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.647+0000 7fcf24d87740 1 -- 172.21.15.96:0/3238552109 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5595e13af2b0 con 0x5595e0fdc6b0 2024-04-25T14:15:19.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.647+0000 7fcef6ffd640 1 -- 172.21.15.96:0/3238552109 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5595e130b800 con 0x5595e0fdc6b0 2024-04-25T14:15:19.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.647+0000 7fcef6ffd640 1 -- 172.21.15.96:0/3238552109 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fcf0c034030 con 0x5595e0fdc6b0 2024-04-25T14:15:19.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.647+0000 7fcf24d87740 10 client.25694 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:15:19.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.647+0000 7fcf24d87740 10 client.25694 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:15:19.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.647+0000 7fcf24d87740 20 client.25694 populate_metadata read hostname 'smithi096' 2024-04-25T14:15:19.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.647+0000 7fcf24d87740 1 --2- 172.21.15.96:0/3238552109 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5595e0f913a0 0x5595e14207b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:19.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.647+0000 7fcf24d87740 4 client.25694 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:15:19.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.647+0000 7fcf24d87740 1 -- 172.21.15.96:0/3238552109 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5595e12864a0 con 0x5595e0f913a0 2024-04-25T14:15:19.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.648+0000 7fcf177fe640 1 --2- 172.21.15.96:0/3238552109 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5595e0f913a0 0x5595e14207b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:19.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.648+0000 7fcef6ffd640 10 client.25694 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:15:19.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.648+0000 7fcf177fe640 1 --2- 172.21.15.96:0/3238552109 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5595e0f913a0 0x5595e14207b0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55bc7027b5de421b server_cookie=9fa9a95ef8b901c7 in_seq=0 out_seq=0 2024-04-25T14:15:19.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.648+0000 7fcef6ffd640 1 -- 172.21.15.96:0/3238552109 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5595e12864a0 con 0x5595e0f913a0 2024-04-25T14:15:19.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.648+0000 7fcef6ffd640 10 client.25694 handle_command_reply: tid=0 2024-04-25T14:15:19.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.648+0000 7fcf24d87740 1 client.25694 shutdown 2024-04-25T14:15:19.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.649+0000 7fcf24d87740 1 -- 172.21.15.96:0/3238552109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595e0fdc6b0 msgr2=0x5595e13acc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:19.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.649+0000 7fcf24d87740 1 --2- 172.21.15.96:0/3238552109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595e0fdc6b0 0x5595e13acc90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x5595e13b6b30 tx=0x7fcf0c0036e0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.649+0000 7fcf24d87740 1 -- 172.21.15.96:0/3238552109 shutdown_connections 2024-04-25T14:15:19.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.649+0000 7fcf24d87740 1 --2- 172.21.15.96:0/3238552109 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5595e0f913a0 0x5595e14207b0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.649+0000 7fcf24d87740 1 --2- 172.21.15.96:0/3238552109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5595e0ffd390 0x5595e13adbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.649+0000 7fcf24d87740 1 --2- 172.21.15.96:0/3238552109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595e0fdc6b0 0x5595e13acc90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.649+0000 7fcef6ffd640 0 client.25694 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:15:19.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.649+0000 7fcf24d87740 1 --2- 172.21.15.96:0/3238552109 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595e0fd44d0 0x5595e13ac750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.649+0000 7fcf24d87740 1 -- 172.21.15.96:0/3238552109 >> 172.21.15.96:0/3238552109 conn(0x5595e0ff14e0 msgr2=0x5595e13a70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:19.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.649+0000 7fcf24d87740 1 -- 172.21.15.96:0/3238552109 shutdown_connections 2024-04-25T14:15:19.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.649+0000 7fcf24d87740 1 -- 172.21.15.96:0/3238552109 wait complete. 2024-04-25T14:15:19.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.649+0000 7fcf24d87740 20 client.25694 trim_cache size 0 max 16384 2024-04-25T14:15:19.655 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T14:15:19.655 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T14:15:19.655 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": { 2024-04-25T14:15:19.655 INFO:teuthology.orchestra.run.smithi096.stdout: "3d74358d-1e7c-4b2d-bf36-a5ada163014d": { 2024-04-25T14:15:19.655 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-04-25T14:15:19.655 INFO:teuthology.orchestra.run.smithi096.stdout: "tag": "3d74358d-1e7c-4b2d-bf36-a5ada163014d", 2024-04-25T14:15:19.655 INFO:teuthology.orchestra.run.smithi096.stdout: "options": "recursive,force" 2024-04-25T14:15:19.655 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:19.656 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:19.656 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T14:15:19.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.649+0000 7fcf1cb9d640 1 -- 172.21.15.96:0/1989906658 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fcf000afae0 msgr2=0x7fcf000b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:19.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.649+0000 7fcf1cb9d640 1 --2- 172.21.15.96:0/1989906658 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fcf000afae0 0x7fcf000b1ff0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fcf100094e0 tx=0x7fcf1002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.649+0000 7fcf1cb9d640 1 -- 172.21.15.96:0/1989906658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf18105b30 msgr2=0x7fcf1819c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:19.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.649+0000 7fcf1cb9d640 1 --2- 172.21.15.96:0/1989906658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf18105b30 0x7fcf1819c200 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fcf0800e9f0 tx=0x7fcf0800eeb0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.649+0000 7fcf1cb9d640 1 -- 172.21.15.96:0/1989906658 shutdown_connections 2024-04-25T14:15:19.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.649+0000 7fcf1cb9d640 1 --2- 172.21.15.96:0/1989906658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf1810ba70 0x7fcf181a36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.649+0000 7fcf1cb9d640 1 --2- 172.21.15.96:0/1989906658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf18106d70 0x7fcf1819c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.649+0000 7fcf1cb9d640 1 --2- 172.21.15.96:0/1989906658 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fcf000afae0 0x7fcf000b1ff0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.649+0000 7fcf1cb9d640 1 --2- 172.21.15.96:0/1989906658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf18105b30 0x7fcf1819c200 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.649+0000 7fcf1cb9d640 1 -- 172.21.15.96:0/1989906658 >> 172.21.15.96:0/1989906658 conn(0x7fcf18100d20 msgr2=0x7fcf18107a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:19.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.650+0000 7fcf1cb9d640 1 -- 172.21.15.96:0/1989906658 shutdown_connections 2024-04-25T14:15:19.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:19.650+0000 7fcf1cb9d640 1 -- 172.21.15.96:0/1989906658 wait complete. 2024-04-25T14:15:19.662 INFO:tasks.cephfs.filesystem:scrub status for tag:3d74358d-1e7c-4b2d-bf36-a5ada163014d - {'path': '/', 'tag': '3d74358d-1e7c-4b2d-bf36-a5ada163014d', 'options': 'recursive,force'} 2024-04-25T14:15:20.042 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4190720 len 4096 2024-04-25T14:15:20.042 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4190720 len 4096 2024-04-25T14:15:20.042 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4190720 len 4096 2024-04-25T14:15:20.043 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4190720 len 8192 2024-04-25T14:15:20.043 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4190720 len 8192 2024-04-25T14:15:20.043 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4190720 len 8192 2024-04-25T14:15:20.043 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4190720 len 8192 2024-04-25T14:15:20.043 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4190720 len 16384 2024-04-25T14:15:20.043 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4190720 len 16384 2024-04-25T14:15:20.043 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4190720 len 16384 2024-04-25T14:15:20.043 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4190720 len 16384 2024-04-25T14:15:20.043 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4191232 len 1024 2024-04-25T14:15:20.043 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4191232 len 1024 2024-04-25T14:15:20.043 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4191232 len 1024 2024-04-25T14:15:20.043 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4191232 len 1024 2024-04-25T14:15:20.043 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4191232 len 2048 2024-04-25T14:15:20.043 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4191232 len 2048 2024-04-25T14:15:20.044 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4191232 len 2048 2024-04-25T14:15:20.044 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4191232 len 2048 2024-04-25T14:15:20.044 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4191232 len 4096 2024-04-25T14:15:20.044 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4191232 len 4096 2024-04-25T14:15:20.044 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4191232 len 4096 2024-04-25T14:15:20.044 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4191232 len 4096 2024-04-25T14:15:20.044 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4191232 len 8192 2024-04-25T14:15:20.044 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4191232 len 8192 2024-04-25T14:15:20.044 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4191232 len 8192 2024-04-25T14:15:20.044 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4191232 len 8192 2024-04-25T14:15:20.044 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4191232 len 16384 2024-04-25T14:15:20.044 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4191232 len 16384 2024-04-25T14:15:20.044 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4191232 len 16384 2024-04-25T14:15:20.044 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4191232 len 16384 2024-04-25T14:15:20.044 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4191744 len 1024 2024-04-25T14:15:20.045 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4191744 len 1024 2024-04-25T14:15:20.045 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4191744 len 1024 2024-04-25T14:15:20.045 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4191744 len 1024 2024-04-25T14:15:20.045 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4191744 len 2048 2024-04-25T14:15:20.045 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4191744 len 2048 2024-04-25T14:15:20.045 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4191744 len 2048 2024-04-25T14:15:20.045 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4191744 len 2048 2024-04-25T14:15:20.045 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4191744 len 4096 2024-04-25T14:15:20.045 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4191744 len 4096 2024-04-25T14:15:20.045 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4191744 len 4096 2024-04-25T14:15:20.045 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4191744 len 4096 2024-04-25T14:15:20.045 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4191744 len 8192 2024-04-25T14:15:20.045 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4191744 len 8192 2024-04-25T14:15:20.045 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4191744 len 8192 2024-04-25T14:15:20.046 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4191744 len 8192 2024-04-25T14:15:20.046 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4191744 len 16384 2024-04-25T14:15:20.046 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4191744 len 16384 2024-04-25T14:15:20.046 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4191744 len 16384 2024-04-25T14:15:20.046 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4191744 len 16384 2024-04-25T14:15:20.046 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4192256 len 1024 2024-04-25T14:15:20.046 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4192256 len 1024 2024-04-25T14:15:20.046 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4192256 len 1024 2024-04-25T14:15:20.046 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4192256 len 1024 2024-04-25T14:15:20.046 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4192256 len 2048 2024-04-25T14:15:20.046 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4192256 len 2048 2024-04-25T14:15:20.046 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4192256 len 2048 2024-04-25T14:15:20.046 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4192256 len 2048 2024-04-25T14:15:20.046 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4192256 len 4096 2024-04-25T14:15:20.047 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4192256 len 4096 2024-04-25T14:15:20.047 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4192256 len 4096 2024-04-25T14:15:20.047 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4192256 len 4096 2024-04-25T14:15:20.047 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4192256 len 8192 2024-04-25T14:15:20.047 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4192256 len 8192 2024-04-25T14:15:20.047 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4192256 len 8192 2024-04-25T14:15:20.047 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4192256 len 8192 2024-04-25T14:15:20.047 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4192256 len 16384 2024-04-25T14:15:20.047 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4192256 len 16384 2024-04-25T14:15:20.047 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4192256 len 16384 2024-04-25T14:15:20.047 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4192256 len 16384 2024-04-25T14:15:20.047 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4192768 len 1024 2024-04-25T14:15:20.047 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4192768 len 1024 2024-04-25T14:15:20.047 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4192768 len 1024 2024-04-25T14:15:20.048 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4192768 len 1024 2024-04-25T14:15:20.048 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4192768 len 2048 2024-04-25T14:15:20.048 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4192768 len 2048 2024-04-25T14:15:20.048 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4192768 len 2048 2024-04-25T14:15:20.048 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4192768 len 2048 2024-04-25T14:15:20.048 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4192768 len 4096 2024-04-25T14:15:20.638 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4192768 len 4096 2024-04-25T14:15:20.638 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4192768 len 4096 2024-04-25T14:15:20.638 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4192768 len 4096 2024-04-25T14:15:20.638 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4192768 len 8192 2024-04-25T14:15:20.638 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4192768 len 8192 2024-04-25T14:15:20.639 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4192768 len 8192 2024-04-25T14:15:20.639 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4192768 len 8192 2024-04-25T14:15:20.639 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4192768 len 16384 2024-04-25T14:15:20.639 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4192768 len 16384 2024-04-25T14:15:20.639 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4192768 len 16384 2024-04-25T14:15:20.639 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4192768 len 16384 2024-04-25T14:15:20.639 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4193280 len 1024 2024-04-25T14:15:20.639 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4193280 len 1024 2024-04-25T14:15:20.639 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4193280 len 1024 2024-04-25T14:15:20.639 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4193280 len 1024 2024-04-25T14:15:20.639 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4193280 len 2048 2024-04-25T14:15:20.639 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4193280 len 2048 2024-04-25T14:15:20.639 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4193280 len 2048 2024-04-25T14:15:20.639 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4193280 len 2048 2024-04-25T14:15:20.640 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4193280 len 4096 2024-04-25T14:15:20.640 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4193280 len 4096 2024-04-25T14:15:20.640 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4193280 len 4096 2024-04-25T14:15:20.640 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4193280 len 4096 2024-04-25T14:15:20.640 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4193280 len 8192 2024-04-25T14:15:20.640 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4193280 len 8192 2024-04-25T14:15:20.640 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4193280 len 8192 2024-04-25T14:15:20.640 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4193280 len 8192 2024-04-25T14:15:20.640 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4193280 len 16384 2024-04-25T14:15:20.640 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4193280 len 16384 2024-04-25T14:15:20.640 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4193280 len 16384 2024-04-25T14:15:20.640 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4193280 len 16384 2024-04-25T14:15:20.640 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4193792 len 1024 2024-04-25T14:15:20.640 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4193792 len 1024 2024-04-25T14:15:20.640 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4193792 len 1024 2024-04-25T14:15:20.641 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4193792 len 1024 2024-04-25T14:15:20.641 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4193792 len 2048 2024-04-25T14:15:20.641 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4193792 len 2048 2024-04-25T14:15:20.641 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4193792 len 2048 2024-04-25T14:15:20.641 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4193792 len 2048 2024-04-25T14:15:20.641 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4193792 len 4096 2024-04-25T14:15:20.641 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4193792 len 4096 2024-04-25T14:15:20.641 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4193792 len 4096 2024-04-25T14:15:20.641 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4193792 len 4096 2024-04-25T14:15:20.641 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4193792 len 8192 2024-04-25T14:15:20.641 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4193792 len 8192 2024-04-25T14:15:20.641 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4193792 len 8192 2024-04-25T14:15:20.641 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4193792 len 8192 2024-04-25T14:15:20.641 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4193792 len 16384 2024-04-25T14:15:20.641 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4193792 len 16384 2024-04-25T14:15:20.642 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4193792 len 16384 2024-04-25T14:15:20.642 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4193792 len 16384 2024-04-25T14:15:20.642 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4194304 len 1024 2024-04-25T14:15:20.642 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4194304 len 1024 2024-04-25T14:15:20.642 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4194304 len 1024 2024-04-25T14:15:20.642 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4194304 len 1024 2024-04-25T14:15:20.642 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4194304 len 2048 2024-04-25T14:15:20.642 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4194304 len 2048 2024-04-25T14:15:20.642 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4194304 len 2048 2024-04-25T14:15:20.642 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4194304 len 2048 2024-04-25T14:15:20.642 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4194304 len 4096 2024-04-25T14:15:20.642 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4194304 len 4096 2024-04-25T14:15:20.642 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4194304 len 4096 2024-04-25T14:15:20.642 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4194304 len 4096 2024-04-25T14:15:20.643 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4194304 len 8192 2024-04-25T14:15:20.643 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4194304 len 8192 2024-04-25T14:15:20.643 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4194304 len 8192 2024-04-25T14:15:20.643 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4194304 len 8192 2024-04-25T14:15:20.643 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4194304 len 16384 2024-04-25T14:15:20.643 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4194304 len 16384 2024-04-25T14:15:20.643 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4194304 len 16384 2024-04-25T14:15:20.643 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4194304 len 16384 2024-04-25T14:15:20.643 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4194816 len 1024 2024-04-25T14:15:20.643 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4194816 len 1024 2024-04-25T14:15:20.643 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4194816 len 1024 2024-04-25T14:15:20.643 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4194816 len 1024 2024-04-25T14:15:20.643 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4194816 len 2048 2024-04-25T14:15:20.643 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4194816 len 2048 2024-04-25T14:15:20.644 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4194816 len 2048 2024-04-25T14:15:20.644 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4194816 len 2048 2024-04-25T14:15:20.644 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4194816 len 4096 2024-04-25T14:15:20.644 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4194816 len 4096 2024-04-25T14:15:20.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:20 smithi096 ceph-mon[29207]: pgmap v397: 97 pgs: 97 active+clean; 182 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 31 MiB/s rd, 781 KiB/s wr, 2.09k op/s 2024-04-25T14:15:20.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:20 smithi096 ceph-mon[29207]: scrub queued for path: / 2024-04-25T14:15:20.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:20 smithi096 ceph-mon[29207]: scrub summary: idle+waiting paths [/] 2024-04-25T14:15:20.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:20 smithi096 ceph-mon[29207]: scrub summary: active paths [/] 2024-04-25T14:15:20.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:20 smithi096 ceph-mon[29207]: scrub summary: idle+waiting paths [/] 2024-04-25T14:15:20.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:20 smithi097 ceph-mon[32720]: pgmap v397: 97 pgs: 97 active+clean; 182 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 31 MiB/s rd, 781 KiB/s wr, 2.09k op/s 2024-04-25T14:15:20.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:20 smithi097 ceph-mon[32720]: scrub queued for path: / 2024-04-25T14:15:20.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:20 smithi097 ceph-mon[32720]: scrub summary: idle+waiting paths [/] 2024-04-25T14:15:20.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:20 smithi097 ceph-mon[32720]: scrub summary: active paths [/] 2024-04-25T14:15:20.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:20 smithi097 ceph-mon[32720]: scrub summary: idle+waiting paths [/] 2024-04-25T14:15:20.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:20 smithi151 ceph-mon[32990]: pgmap v397: 97 pgs: 97 active+clean; 182 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 31 MiB/s rd, 781 KiB/s wr, 2.09k op/s 2024-04-25T14:15:20.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:20 smithi151 ceph-mon[32990]: scrub queued for path: / 2024-04-25T14:15:20.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:20 smithi151 ceph-mon[32990]: scrub summary: idle+waiting paths [/] 2024-04-25T14:15:20.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:20 smithi151 ceph-mon[32990]: scrub summary: active paths [/] 2024-04-25T14:15:20.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:20 smithi151 ceph-mon[32990]: scrub summary: idle+waiting paths [/] 2024-04-25T14:15:21.052 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4194816 len 4096 2024-04-25T14:15:21.053 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4194816 len 4096 2024-04-25T14:15:21.053 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4194816 len 8192 2024-04-25T14:15:21.053 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4194816 len 8192 2024-04-25T14:15:21.053 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4194816 len 8192 2024-04-25T14:15:21.053 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4194816 len 8192 2024-04-25T14:15:21.053 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4194816 len 16384 2024-04-25T14:15:21.053 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4194816 len 16384 2024-04-25T14:15:21.053 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4194816 len 16384 2024-04-25T14:15:21.053 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4194816 len 16384 2024-04-25T14:15:21.053 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4195328 len 1024 2024-04-25T14:15:21.053 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4195328 len 1024 2024-04-25T14:15:21.053 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4195328 len 1024 2024-04-25T14:15:21.054 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4195328 len 1024 2024-04-25T14:15:21.054 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4195328 len 2048 2024-04-25T14:15:21.054 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4195328 len 2048 2024-04-25T14:15:21.054 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4195328 len 2048 2024-04-25T14:15:21.054 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4195328 len 2048 2024-04-25T14:15:21.054 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4195328 len 4096 2024-04-25T14:15:21.054 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4195328 len 4096 2024-04-25T14:15:21.054 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4195328 len 4096 2024-04-25T14:15:21.054 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4195328 len 4096 2024-04-25T14:15:21.054 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4195328 len 8192 2024-04-25T14:15:21.054 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4195328 len 8192 2024-04-25T14:15:21.054 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4195328 len 8192 2024-04-25T14:15:21.054 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4195328 len 8192 2024-04-25T14:15:21.054 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4195328 len 16384 2024-04-25T14:15:21.054 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4195328 len 16384 2024-04-25T14:15:21.055 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4195328 len 16384 2024-04-25T14:15:21.055 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4195328 len 16384 2024-04-25T14:15:21.055 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4195840 len 1024 2024-04-25T14:15:21.055 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4195840 len 1024 2024-04-25T14:15:21.055 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4195840 len 1024 2024-04-25T14:15:21.055 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4195840 len 1024 2024-04-25T14:15:21.055 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4195840 len 2048 2024-04-25T14:15:21.055 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4195840 len 2048 2024-04-25T14:15:21.055 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4195840 len 2048 2024-04-25T14:15:21.055 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4195840 len 2048 2024-04-25T14:15:21.055 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4195840 len 4096 2024-04-25T14:15:21.055 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4195840 len 4096 2024-04-25T14:15:21.055 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4195840 len 4096 2024-04-25T14:15:21.056 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4195840 len 4096 2024-04-25T14:15:21.056 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4195840 len 8192 2024-04-25T14:15:21.056 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4195840 len 8192 2024-04-25T14:15:21.056 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4195840 len 8192 2024-04-25T14:15:21.056 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4195840 len 8192 2024-04-25T14:15:21.056 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4195840 len 16384 2024-04-25T14:15:21.056 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4195840 len 16384 2024-04-25T14:15:21.056 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4195840 len 16384 2024-04-25T14:15:21.056 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4195840 len 16384 2024-04-25T14:15:21.056 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4196352 len 1024 2024-04-25T14:15:21.056 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4196352 len 1024 2024-04-25T14:15:21.056 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4196352 len 1024 2024-04-25T14:15:21.056 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4196352 len 1024 2024-04-25T14:15:21.056 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4196352 len 2048 2024-04-25T14:15:21.056 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4196352 len 2048 2024-04-25T14:15:21.057 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4196352 len 2048 2024-04-25T14:15:21.057 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4196352 len 2048 2024-04-25T14:15:21.057 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4196352 len 4096 2024-04-25T14:15:21.057 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4196352 len 4096 2024-04-25T14:15:21.057 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4196352 len 4096 2024-04-25T14:15:21.057 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4196352 len 4096 2024-04-25T14:15:21.057 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4196352 len 8192 2024-04-25T14:15:21.057 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4196352 len 8192 2024-04-25T14:15:21.057 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4196352 len 8192 2024-04-25T14:15:21.057 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4196352 len 8192 2024-04-25T14:15:21.057 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4196352 len 16384 2024-04-25T14:15:21.057 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4196352 len 16384 2024-04-25T14:15:21.057 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4196352 len 16384 2024-04-25T14:15:21.057 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4196352 len 16384 2024-04-25T14:15:21.057 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4196864 len 1024 2024-04-25T14:15:21.058 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4196864 len 1024 2024-04-25T14:15:21.058 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4196864 len 1024 2024-04-25T14:15:21.058 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4196864 len 1024 2024-04-25T14:15:21.058 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4196864 len 2048 2024-04-25T14:15:21.058 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4196864 len 2048 2024-04-25T14:15:21.058 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4196864 len 2048 2024-04-25T14:15:21.058 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4196864 len 2048 2024-04-25T14:15:21.058 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4196864 len 4096 2024-04-25T14:15:21.059 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4196864 len 4096 2024-04-25T14:15:21.059 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4196864 len 4096 2024-04-25T14:15:21.519 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4196864 len 4096 2024-04-25T14:15:21.519 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4196864 len 8192 2024-04-25T14:15:21.519 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4196864 len 8192 2024-04-25T14:15:21.520 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4196864 len 8192 2024-04-25T14:15:21.520 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4196864 len 8192 2024-04-25T14:15:21.520 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4196864 len 16384 2024-04-25T14:15:21.520 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4196864 len 16384 2024-04-25T14:15:21.520 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4196864 len 16384 2024-04-25T14:15:21.520 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4196864 len 16384 2024-04-25T14:15:21.520 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4197376 len 1024 2024-04-25T14:15:21.520 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4197376 len 1024 2024-04-25T14:15:21.520 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4197376 len 1024 2024-04-25T14:15:21.520 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4197376 len 1024 2024-04-25T14:15:21.520 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4197376 len 2048 2024-04-25T14:15:21.520 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4197376 len 2048 2024-04-25T14:15:21.520 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4197376 len 2048 2024-04-25T14:15:21.520 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4197376 len 2048 2024-04-25T14:15:21.521 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4197376 len 4096 2024-04-25T14:15:21.521 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4197376 len 4096 2024-04-25T14:15:21.521 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4197376 len 4096 2024-04-25T14:15:21.521 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4197376 len 4096 2024-04-25T14:15:21.521 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4197376 len 8192 2024-04-25T14:15:21.521 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4197376 len 8192 2024-04-25T14:15:21.521 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4197376 len 8192 2024-04-25T14:15:21.521 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4197376 len 8192 2024-04-25T14:15:21.521 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4197376 len 16384 2024-04-25T14:15:21.521 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4197376 len 16384 2024-04-25T14:15:21.521 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4197376 len 16384 2024-04-25T14:15:21.521 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4197376 len 16384 2024-04-25T14:15:21.521 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4197888 len 1024 2024-04-25T14:15:21.521 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4197888 len 1024 2024-04-25T14:15:21.521 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4197888 len 1024 2024-04-25T14:15:21.522 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4197888 len 1024 2024-04-25T14:15:21.522 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4197888 len 2048 2024-04-25T14:15:21.522 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4197888 len 2048 2024-04-25T14:15:21.522 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4197888 len 2048 2024-04-25T14:15:21.522 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4197888 len 2048 2024-04-25T14:15:21.522 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4197888 len 4096 2024-04-25T14:15:21.522 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4197888 len 4096 2024-04-25T14:15:21.522 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4197888 len 4096 2024-04-25T14:15:21.522 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4197888 len 4096 2024-04-25T14:15:21.522 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4197888 len 8192 2024-04-25T14:15:21.522 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4197888 len 8192 2024-04-25T14:15:21.522 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4197888 len 8192 2024-04-25T14:15:21.522 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4197888 len 8192 2024-04-25T14:15:21.522 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1024 offset 4197888 len 16384 2024-04-25T14:15:21.523 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4197888 len 16384 2024-04-25T14:15:21.523 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1024 offset 4197888 len 16384 2024-04-25T14:15:21.523 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1024 offset 4197888 len 16384 2024-04-25T14:15:21.523 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4190720 len 1024 2024-04-25T14:15:21.523 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4190720 len 1024 2024-04-25T14:15:21.523 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4190720 len 1024 2024-04-25T14:15:21.523 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4190720 len 1024 2024-04-25T14:15:21.523 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4190720 len 2048 2024-04-25T14:15:21.523 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4190720 len 2048 2024-04-25T14:15:21.523 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4190720 len 2048 2024-04-25T14:15:21.523 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4190720 len 2048 2024-04-25T14:15:21.523 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4190720 len 4096 2024-04-25T14:15:21.523 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4190720 len 4096 2024-04-25T14:15:21.523 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4190720 len 4096 2024-04-25T14:15:21.523 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4190720 len 4096 2024-04-25T14:15:21.524 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4190720 len 8192 2024-04-25T14:15:21.524 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4190720 len 8192 2024-04-25T14:15:21.524 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4190720 len 8192 2024-04-25T14:15:21.524 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4190720 len 8192 2024-04-25T14:15:21.524 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4190720 len 16384 2024-04-25T14:15:21.524 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4190720 len 16384 2024-04-25T14:15:21.524 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4190720 len 16384 2024-04-25T14:15:21.524 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4190720 len 16384 2024-04-25T14:15:21.524 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4191232 len 1024 2024-04-25T14:15:21.524 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4191232 len 1024 2024-04-25T14:15:21.524 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4191232 len 1024 2024-04-25T14:15:21.524 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4191232 len 1024 2024-04-25T14:15:21.524 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4191232 len 2048 2024-04-25T14:15:21.524 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4191232 len 2048 2024-04-25T14:15:21.525 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4191232 len 2048 2024-04-25T14:15:21.525 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4191232 len 2048 2024-04-25T14:15:21.525 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4191232 len 4096 2024-04-25T14:15:21.525 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4191232 len 4096 2024-04-25T14:15:21.525 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4191232 len 4096 2024-04-25T14:15:21.525 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4191232 len 4096 2024-04-25T14:15:22.170 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4191232 len 8192 2024-04-25T14:15:22.170 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4191232 len 8192 2024-04-25T14:15:22.170 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4191232 len 8192 2024-04-25T14:15:22.171 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4191232 len 8192 2024-04-25T14:15:22.171 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4191232 len 16384 2024-04-25T14:15:22.171 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4191232 len 16384 2024-04-25T14:15:22.171 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4191232 len 16384 2024-04-25T14:15:22.171 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4191232 len 16384 2024-04-25T14:15:22.171 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4191744 len 1024 2024-04-25T14:15:22.171 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4191744 len 1024 2024-04-25T14:15:22.171 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4191744 len 1024 2024-04-25T14:15:22.171 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4191744 len 1024 2024-04-25T14:15:22.171 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4191744 len 2048 2024-04-25T14:15:22.171 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4191744 len 2048 2024-04-25T14:15:22.171 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4191744 len 2048 2024-04-25T14:15:22.171 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4191744 len 2048 2024-04-25T14:15:22.172 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4191744 len 4096 2024-04-25T14:15:22.172 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4191744 len 4096 2024-04-25T14:15:22.172 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4191744 len 4096 2024-04-25T14:15:22.172 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4191744 len 4096 2024-04-25T14:15:22.172 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4191744 len 8192 2024-04-25T14:15:22.172 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4191744 len 8192 2024-04-25T14:15:22.172 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4191744 len 8192 2024-04-25T14:15:22.172 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4191744 len 8192 2024-04-25T14:15:22.172 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4191744 len 16384 2024-04-25T14:15:22.172 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4191744 len 16384 2024-04-25T14:15:22.172 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4191744 len 16384 2024-04-25T14:15:22.172 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4191744 len 16384 2024-04-25T14:15:22.172 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4192256 len 1024 2024-04-25T14:15:22.172 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4192256 len 1024 2024-04-25T14:15:22.173 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4192256 len 1024 2024-04-25T14:15:22.173 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4192256 len 1024 2024-04-25T14:15:22.173 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4192256 len 2048 2024-04-25T14:15:22.173 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4192256 len 2048 2024-04-25T14:15:22.173 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4192256 len 2048 2024-04-25T14:15:22.173 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4192256 len 2048 2024-04-25T14:15:22.173 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4192256 len 4096 2024-04-25T14:15:22.173 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4192256 len 4096 2024-04-25T14:15:22.173 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4192256 len 4096 2024-04-25T14:15:22.173 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4192256 len 4096 2024-04-25T14:15:22.173 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4192256 len 8192 2024-04-25T14:15:22.173 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4192256 len 8192 2024-04-25T14:15:22.173 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4192256 len 8192 2024-04-25T14:15:22.173 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4192256 len 8192 2024-04-25T14:15:22.173 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4192256 len 16384 2024-04-25T14:15:22.174 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4192256 len 16384 2024-04-25T14:15:22.174 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4192256 len 16384 2024-04-25T14:15:22.174 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4192256 len 16384 2024-04-25T14:15:22.174 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4192768 len 1024 2024-04-25T14:15:22.174 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4192768 len 1024 2024-04-25T14:15:22.174 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4192768 len 1024 2024-04-25T14:15:22.174 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4192768 len 1024 2024-04-25T14:15:22.174 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4192768 len 2048 2024-04-25T14:15:22.174 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4192768 len 2048 2024-04-25T14:15:22.174 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4192768 len 2048 2024-04-25T14:15:22.174 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4192768 len 2048 2024-04-25T14:15:22.174 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4192768 len 4096 2024-04-25T14:15:22.174 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4192768 len 4096 2024-04-25T14:15:22.174 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4192768 len 4096 2024-04-25T14:15:22.174 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4192768 len 4096 2024-04-25T14:15:22.175 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4192768 len 8192 2024-04-25T14:15:22.175 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4192768 len 8192 2024-04-25T14:15:22.175 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4192768 len 8192 2024-04-25T14:15:22.175 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4192768 len 8192 2024-04-25T14:15:22.175 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4192768 len 16384 2024-04-25T14:15:22.175 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4192768 len 16384 2024-04-25T14:15:22.175 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4192768 len 16384 2024-04-25T14:15:22.175 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4192768 len 16384 2024-04-25T14:15:22.175 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4193280 len 1024 2024-04-25T14:15:22.175 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4193280 len 1024 2024-04-25T14:15:22.175 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4193280 len 1024 2024-04-25T14:15:22.175 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4193280 len 1024 2024-04-25T14:15:22.175 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4193280 len 2048 2024-04-25T14:15:22.175 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4193280 len 2048 2024-04-25T14:15:22.176 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4193280 len 2048 2024-04-25T14:15:22.176 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4193280 len 2048 2024-04-25T14:15:22.176 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4193280 len 4096 2024-04-25T14:15:22.176 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4193280 len 4096 2024-04-25T14:15:22.176 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4193280 len 4096 2024-04-25T14:15:22.176 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4193280 len 4096 2024-04-25T14:15:22.176 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4193280 len 8192 2024-04-25T14:15:22.646 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4193280 len 8192 2024-04-25T14:15:22.646 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4193280 len 8192 2024-04-25T14:15:22.646 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4193280 len 8192 2024-04-25T14:15:22.646 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4193280 len 16384 2024-04-25T14:15:22.646 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4193280 len 16384 2024-04-25T14:15:22.646 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4193280 len 16384 2024-04-25T14:15:22.646 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4193280 len 16384 2024-04-25T14:15:22.646 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4193792 len 1024 2024-04-25T14:15:22.646 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4193792 len 1024 2024-04-25T14:15:22.646 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4193792 len 1024 2024-04-25T14:15:22.646 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4193792 len 1024 2024-04-25T14:15:22.647 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4193792 len 2048 2024-04-25T14:15:22.647 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4193792 len 2048 2024-04-25T14:15:22.647 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4193792 len 2048 2024-04-25T14:15:22.647 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4193792 len 2048 2024-04-25T14:15:22.647 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4193792 len 4096 2024-04-25T14:15:22.647 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4193792 len 4096 2024-04-25T14:15:22.647 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4193792 len 4096 2024-04-25T14:15:22.647 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4193792 len 4096 2024-04-25T14:15:22.647 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4193792 len 8192 2024-04-25T14:15:22.647 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4193792 len 8192 2024-04-25T14:15:22.647 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4193792 len 8192 2024-04-25T14:15:22.647 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4193792 len 8192 2024-04-25T14:15:22.647 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4193792 len 16384 2024-04-25T14:15:22.648 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4193792 len 16384 2024-04-25T14:15:22.648 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4193792 len 16384 2024-04-25T14:15:22.648 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4193792 len 16384 2024-04-25T14:15:22.648 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4194304 len 1024 2024-04-25T14:15:22.648 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4194304 len 1024 2024-04-25T14:15:22.648 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4194304 len 1024 2024-04-25T14:15:22.648 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4194304 len 1024 2024-04-25T14:15:22.648 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4194304 len 2048 2024-04-25T14:15:22.648 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4194304 len 2048 2024-04-25T14:15:22.648 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4194304 len 2048 2024-04-25T14:15:22.648 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4194304 len 2048 2024-04-25T14:15:22.648 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4194304 len 4096 2024-04-25T14:15:22.648 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4194304 len 4096 2024-04-25T14:15:22.649 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4194304 len 4096 2024-04-25T14:15:22.649 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4194304 len 4096 2024-04-25T14:15:22.649 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4194304 len 8192 2024-04-25T14:15:22.649 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4194304 len 8192 2024-04-25T14:15:22.649 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4194304 len 8192 2024-04-25T14:15:22.649 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4194304 len 8192 2024-04-25T14:15:22.649 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4194304 len 16384 2024-04-25T14:15:22.649 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4194304 len 16384 2024-04-25T14:15:22.649 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4194304 len 16384 2024-04-25T14:15:22.649 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4194304 len 16384 2024-04-25T14:15:22.649 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4194816 len 1024 2024-04-25T14:15:22.649 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4194816 len 1024 2024-04-25T14:15:22.649 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4194816 len 1024 2024-04-25T14:15:22.649 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4194816 len 1024 2024-04-25T14:15:22.649 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4194816 len 2048 2024-04-25T14:15:22.650 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4194816 len 2048 2024-04-25T14:15:22.650 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4194816 len 2048 2024-04-25T14:15:22.650 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4194816 len 2048 2024-04-25T14:15:22.650 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4194816 len 4096 2024-04-25T14:15:22.650 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4194816 len 4096 2024-04-25T14:15:22.650 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4194816 len 4096 2024-04-25T14:15:22.650 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4194816 len 4096 2024-04-25T14:15:22.650 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4194816 len 8192 2024-04-25T14:15:22.650 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4194816 len 8192 2024-04-25T14:15:22.650 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4194816 len 8192 2024-04-25T14:15:22.650 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4194816 len 8192 2024-04-25T14:15:22.650 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4194816 len 16384 2024-04-25T14:15:22.650 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4194816 len 16384 2024-04-25T14:15:22.650 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4194816 len 16384 2024-04-25T14:15:22.650 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4194816 len 16384 2024-04-25T14:15:22.651 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4195328 len 1024 2024-04-25T14:15:22.651 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4195328 len 1024 2024-04-25T14:15:22.651 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4195328 len 1024 2024-04-25T14:15:22.651 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4195328 len 1024 2024-04-25T14:15:22.651 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4195328 len 2048 2024-04-25T14:15:22.651 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4195328 len 2048 2024-04-25T14:15:22.651 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4195328 len 2048 2024-04-25T14:15:22.651 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4195328 len 2048 2024-04-25T14:15:22.651 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4195328 len 4096 2024-04-25T14:15:22.651 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4195328 len 4096 2024-04-25T14:15:22.651 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4195328 len 4096 2024-04-25T14:15:22.651 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4195328 len 4096 2024-04-25T14:15:22.651 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4195328 len 8192 2024-04-25T14:15:22.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:22 smithi096 ceph-mon[29207]: pgmap v398: 97 pgs: 97 active+clean; 181 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 52 MiB/s rd, 1.2 MiB/s wr, 2.19k op/s 2024-04-25T14:15:22.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:22 smithi097 ceph-mon[32720]: pgmap v398: 97 pgs: 97 active+clean; 181 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 52 MiB/s rd, 1.2 MiB/s wr, 2.19k op/s 2024-04-25T14:15:22.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:22 smithi151 ceph-mon[32990]: pgmap v398: 97 pgs: 97 active+clean; 181 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 52 MiB/s rd, 1.2 MiB/s wr, 2.19k op/s 2024-04-25T14:15:23.050 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4195328 len 8192 2024-04-25T14:15:23.050 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4195328 len 8192 2024-04-25T14:15:23.050 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4195328 len 8192 2024-04-25T14:15:23.050 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4195328 len 16384 2024-04-25T14:15:23.050 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4195328 len 16384 2024-04-25T14:15:23.050 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4195328 len 16384 2024-04-25T14:15:23.051 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4195328 len 16384 2024-04-25T14:15:23.051 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4195840 len 1024 2024-04-25T14:15:23.051 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4195840 len 1024 2024-04-25T14:15:23.051 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4195840 len 1024 2024-04-25T14:15:23.051 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4195840 len 1024 2024-04-25T14:15:23.051 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4195840 len 2048 2024-04-25T14:15:23.051 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4195840 len 2048 2024-04-25T14:15:23.051 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4195840 len 2048 2024-04-25T14:15:23.051 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4195840 len 2048 2024-04-25T14:15:23.051 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4195840 len 4096 2024-04-25T14:15:23.051 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4195840 len 4096 2024-04-25T14:15:23.051 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4195840 len 4096 2024-04-25T14:15:23.051 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4195840 len 4096 2024-04-25T14:15:23.052 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4195840 len 8192 2024-04-25T14:15:23.052 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4195840 len 8192 2024-04-25T14:15:23.052 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4195840 len 8192 2024-04-25T14:15:23.052 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4195840 len 8192 2024-04-25T14:15:23.052 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4195840 len 16384 2024-04-25T14:15:23.052 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4195840 len 16384 2024-04-25T14:15:23.052 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4195840 len 16384 2024-04-25T14:15:23.052 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4195840 len 16384 2024-04-25T14:15:23.052 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4196352 len 1024 2024-04-25T14:15:23.052 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4196352 len 1024 2024-04-25T14:15:23.052 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4196352 len 1024 2024-04-25T14:15:23.052 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4196352 len 1024 2024-04-25T14:15:23.052 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4196352 len 2048 2024-04-25T14:15:23.052 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4196352 len 2048 2024-04-25T14:15:23.052 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4196352 len 2048 2024-04-25T14:15:23.053 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4196352 len 2048 2024-04-25T14:15:23.053 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4196352 len 4096 2024-04-25T14:15:23.053 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4196352 len 4096 2024-04-25T14:15:23.053 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4196352 len 4096 2024-04-25T14:15:23.053 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4196352 len 4096 2024-04-25T14:15:23.053 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4196352 len 8192 2024-04-25T14:15:23.053 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4196352 len 8192 2024-04-25T14:15:23.053 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4196352 len 8192 2024-04-25T14:15:23.053 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4196352 len 8192 2024-04-25T14:15:23.053 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4196352 len 16384 2024-04-25T14:15:23.053 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4196352 len 16384 2024-04-25T14:15:23.053 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4196352 len 16384 2024-04-25T14:15:23.053 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4196352 len 16384 2024-04-25T14:15:23.053 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4196864 len 1024 2024-04-25T14:15:23.053 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4196864 len 1024 2024-04-25T14:15:23.053 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4196864 len 1024 2024-04-25T14:15:23.054 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4196864 len 1024 2024-04-25T14:15:23.054 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4196864 len 2048 2024-04-25T14:15:23.054 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4196864 len 2048 2024-04-25T14:15:23.054 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4196864 len 2048 2024-04-25T14:15:23.054 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4196864 len 2048 2024-04-25T14:15:23.054 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4196864 len 4096 2024-04-25T14:15:23.054 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4196864 len 4096 2024-04-25T14:15:23.054 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4196864 len 4096 2024-04-25T14:15:23.054 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4196864 len 4096 2024-04-25T14:15:23.054 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4196864 len 8192 2024-04-25T14:15:23.055 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4196864 len 8192 2024-04-25T14:15:23.055 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4196864 len 8192 2024-04-25T14:15:23.055 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4196864 len 8192 2024-04-25T14:15:23.055 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4196864 len 16384 2024-04-25T14:15:23.055 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4196864 len 16384 2024-04-25T14:15:23.055 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4196864 len 16384 2024-04-25T14:15:23.055 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4196864 len 16384 2024-04-25T14:15:23.055 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4197376 len 1024 2024-04-25T14:15:23.055 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4197376 len 1024 2024-04-25T14:15:23.055 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4197376 len 1024 2024-04-25T14:15:23.055 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4197376 len 1024 2024-04-25T14:15:23.055 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4197376 len 2048 2024-04-25T14:15:23.055 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4197376 len 2048 2024-04-25T14:15:23.055 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4197376 len 2048 2024-04-25T14:15:23.055 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4197376 len 2048 2024-04-25T14:15:23.056 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4197376 len 4096 2024-04-25T14:15:23.056 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4197376 len 4096 2024-04-25T14:15:23.056 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4197376 len 4096 2024-04-25T14:15:23.056 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4197376 len 4096 2024-04-25T14:15:23.056 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4197376 len 8192 2024-04-25T14:15:23.056 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4197376 len 8192 2024-04-25T14:15:23.613 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4197376 len 8192 2024-04-25T14:15:23.613 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4197376 len 8192 2024-04-25T14:15:23.613 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4197376 len 16384 2024-04-25T14:15:23.613 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4197376 len 16384 2024-04-25T14:15:23.613 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4197376 len 16384 2024-04-25T14:15:23.613 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4197376 len 16384 2024-04-25T14:15:23.613 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4197888 len 1024 2024-04-25T14:15:23.614 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4197888 len 1024 2024-04-25T14:15:23.614 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4197888 len 1024 2024-04-25T14:15:23.614 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4197888 len 1024 2024-04-25T14:15:23.614 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4197888 len 2048 2024-04-25T14:15:23.614 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4197888 len 2048 2024-04-25T14:15:23.614 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4197888 len 2048 2024-04-25T14:15:23.614 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4197888 len 2048 2024-04-25T14:15:23.614 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4197888 len 4096 2024-04-25T14:15:23.614 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4197888 len 4096 2024-04-25T14:15:23.614 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4197888 len 4096 2024-04-25T14:15:23.614 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4197888 len 4096 2024-04-25T14:15:23.614 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4197888 len 8192 2024-04-25T14:15:23.614 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4197888 len 8192 2024-04-25T14:15:23.614 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4197888 len 8192 2024-04-25T14:15:23.615 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4197888 len 8192 2024-04-25T14:15:23.615 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 1536 offset 4197888 len 16384 2024-04-25T14:15:23.615 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4197888 len 16384 2024-04-25T14:15:23.615 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 1536 offset 4197888 len 16384 2024-04-25T14:15:23.615 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 1536 offset 4197888 len 16384 2024-04-25T14:15:23.615 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4190720 len 1024 2024-04-25T14:15:23.615 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4190720 len 1024 2024-04-25T14:15:23.615 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4190720 len 1024 2024-04-25T14:15:23.615 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4190720 len 1024 2024-04-25T14:15:23.615 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4190720 len 2048 2024-04-25T14:15:23.615 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4190720 len 2048 2024-04-25T14:15:23.615 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4190720 len 2048 2024-04-25T14:15:23.615 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4190720 len 2048 2024-04-25T14:15:23.615 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4190720 len 4096 2024-04-25T14:15:23.616 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4190720 len 4096 2024-04-25T14:15:23.616 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4190720 len 4096 2024-04-25T14:15:23.616 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4190720 len 4096 2024-04-25T14:15:23.616 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4190720 len 8192 2024-04-25T14:15:23.616 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4190720 len 8192 2024-04-25T14:15:23.616 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4190720 len 8192 2024-04-25T14:15:23.616 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4190720 len 8192 2024-04-25T14:15:23.616 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4190720 len 16384 2024-04-25T14:15:23.616 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4190720 len 16384 2024-04-25T14:15:23.616 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4190720 len 16384 2024-04-25T14:15:23.616 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4190720 len 16384 2024-04-25T14:15:23.616 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4191232 len 1024 2024-04-25T14:15:23.616 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4191232 len 1024 2024-04-25T14:15:23.616 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4191232 len 1024 2024-04-25T14:15:23.616 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4191232 len 1024 2024-04-25T14:15:23.617 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4191232 len 2048 2024-04-25T14:15:23.617 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4191232 len 2048 2024-04-25T14:15:23.617 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4191232 len 2048 2024-04-25T14:15:23.617 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4191232 len 2048 2024-04-25T14:15:23.617 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4191232 len 4096 2024-04-25T14:15:23.617 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4191232 len 4096 2024-04-25T14:15:23.617 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4191232 len 4096 2024-04-25T14:15:23.617 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4191232 len 4096 2024-04-25T14:15:23.617 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4191232 len 8192 2024-04-25T14:15:23.617 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4191232 len 8192 2024-04-25T14:15:23.617 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4191232 len 8192 2024-04-25T14:15:23.617 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4191232 len 8192 2024-04-25T14:15:23.617 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4191232 len 16384 2024-04-25T14:15:23.617 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4191232 len 16384 2024-04-25T14:15:23.618 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4191232 len 16384 2024-04-25T14:15:23.618 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4191232 len 16384 2024-04-25T14:15:23.618 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4191744 len 1024 2024-04-25T14:15:23.618 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4191744 len 1024 2024-04-25T14:15:23.618 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4191744 len 1024 2024-04-25T14:15:23.618 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4191744 len 1024 2024-04-25T14:15:23.618 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4191744 len 2048 2024-04-25T14:15:23.618 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4191744 len 2048 2024-04-25T14:15:23.618 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4191744 len 2048 2024-04-25T14:15:23.618 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4191744 len 2048 2024-04-25T14:15:23.618 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4191744 len 4096 2024-04-25T14:15:23.618 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4191744 len 4096 2024-04-25T14:15:23.619 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4191744 len 4096 2024-04-25T14:15:23.619 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4191744 len 4096 2024-04-25T14:15:23.619 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4191744 len 8192 2024-04-25T14:15:23.619 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4191744 len 8192 2024-04-25T14:15:23.619 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4191744 len 8192 2024-04-25T14:15:23.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.875+0000 7fd25a892640 1 -- 172.21.15.96:0/3456174442 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd254103400 msgr2=0x7fd254103810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:23.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.875+0000 7fd25a892640 1 --2- 172.21.15.96:0/3456174442 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd254103400 0x7fd254103810 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fd248009900 tx=0x7fd24802a410 comp rx=0 tx=0).stop 2024-04-25T14:15:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.875+0000 7fd25a892640 1 -- 172.21.15.96:0/3456174442 shutdown_connections 2024-04-25T14:15:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.875+0000 7fd25a892640 1 --2- 172.21.15.96:0/3456174442 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd254108100 0x7fd25410a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.875+0000 7fd25a892640 1 --2- 172.21.15.96:0/3456174442 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd254103400 0x7fd254103810 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.875+0000 7fd25a892640 1 --2- 172.21.15.96:0/3456174442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2541021c0 0x7fd254102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.875+0000 7fd25a892640 1 -- 172.21.15.96:0/3456174442 >> 172.21.15.96:0/3456174442 conn(0x7fd2540fd390 msgr2=0x7fd2540ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.875+0000 7fd25a892640 1 -- 172.21.15.96:0/3456174442 shutdown_connections 2024-04-25T14:15:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.875+0000 7fd25a892640 1 -- 172.21.15.96:0/3456174442 wait complete. 2024-04-25T14:15:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.875+0000 7fd25a892640 1 Processor -- start 2024-04-25T14:15:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.875+0000 7fd25a892640 1 -- start start 2024-04-25T14:15:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.876+0000 7fd25a892640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2541021c0 0x7fd25410f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:23.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.876+0000 7fd25a892640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd254103400 0x7fd25419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:23.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.876+0000 7fd25a892640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd254108100 0x7fd2541a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:23.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.876+0000 7fd25a892640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd25410dce0 con 0x7fd2541021c0 2024-04-25T14:15:23.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.876+0000 7fd25a892640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd25410db60 con 0x7fd254103400 2024-04-25T14:15:23.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.876+0000 7fd25a892640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd25410de60 con 0x7fd254108100 2024-04-25T14:15:23.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.876+0000 7fd253fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2541021c0 0x7fd25410f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:23.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.876+0000 7fd253fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2541021c0 0x7fd25410f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39714/0 (socket says 172.21.15.96:39714) 2024-04-25T14:15:23.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.876+0000 7fd253fff640 1 -- 172.21.15.96:0/183993461 learned_addr learned my addr 172.21.15.96:0/183993461 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:23.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.876+0000 7fd258e2f640 1 --2- 172.21.15.96:0/183993461 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd254108100 0x7fd2541a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:23.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.876+0000 7fd253fff640 1 -- 172.21.15.96:0/183993461 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd254108100 msgr2=0x7fd2541a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.876+0000 7fd253fff640 1 --2- 172.21.15.96:0/183993461 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd254108100 0x7fd2541a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.876+0000 7fd253fff640 1 -- 172.21.15.96:0/183993461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd254103400 msgr2=0x7fd25419efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.876+0000 7fd253fff640 1 --2- 172.21.15.96:0/183993461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd254103400 0x7fd25419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.876+0000 7fd253fff640 1 -- 172.21.15.96:0/183993461 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2541a1d30 con 0x7fd2541021c0 2024-04-25T14:15:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.876+0000 7fd253fff640 1 --2- 172.21.15.96:0/183993461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2541021c0 0x7fd25410f120 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fd24400e980 tx=0x7fd24400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.876+0000 7fd2517fa640 1 -- 172.21.15.96:0/183993461 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd24400cd80 con 0x7fd2541021c0 2024-04-25T14:15:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.877+0000 7fd2517fa640 1 -- 172.21.15.96:0/183993461 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2440108e0 con 0x7fd2541021c0 2024-04-25T14:15:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.877+0000 7fd25a892640 1 -- 172.21.15.96:0/183993461 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2541a1fc0 con 0x7fd2541021c0 2024-04-25T14:15:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.877+0000 7fd2517fa640 1 -- 172.21.15.96:0/183993461 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd244018990 con 0x7fd2541021c0 2024-04-25T14:15:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.877+0000 7fd25a892640 1 -- 172.21.15.96:0/183993461 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2541a24e0 con 0x7fd2541021c0 2024-04-25T14:15:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.879+0000 7fd261819740 1 Processor -- start 2024-04-25T14:15:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.879+0000 7fd261819740 1 -- start start 2024-04-25T14:15:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.879+0000 7fd261819740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a575cc11b0 0x55a576061970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.879+0000 7fd261819740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a575c982f0 0x55a576061ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.879+0000 7fd261819740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a575ca04d0 0x55a576068df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:23.889 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.879+0000 7fd261819740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a575fc7460 con 0x55a575ca04d0 2024-04-25T14:15:23.889 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.879+0000 7fd261819740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a575f81590 con 0x55a575c982f0 2024-04-25T14:15:23.889 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.879+0000 7fd261819740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a575fc3420 con 0x55a575cc11b0 2024-04-25T14:15:23.889 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.880+0000 7fd2537fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a575ca04d0 0x55a576068df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:23.889 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.880+0000 7fd2537fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a575ca04d0 0x55a576068df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39718/0 (socket says 172.21.15.96:39718) 2024-04-25T14:15:23.889 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.880+0000 7fd2537fe640 1 -- 172.21.15.96:0/3945434581 learned_addr learned my addr 172.21.15.96:0/3945434581 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:23.889 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.880+0000 7fd258e2f640 1 --2- 172.21.15.96:0/3945434581 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a575cc11b0 0x55a576061970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:23.889 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.880+0000 7fd2537fe640 1 -- 172.21.15.96:0/3945434581 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a575cc11b0 msgr2=0x55a576061970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:23.889 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.880+0000 7fd2537fe640 1 --2- 172.21.15.96:0/3945434581 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a575cc11b0 0x55a576061970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:23.889 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.880+0000 7fd2537fe640 1 -- 172.21.15.96:0/3945434581 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a575c982f0 msgr2=0x55a576061ef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T14:15:23.889 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.880+0000 7fd2537fe640 1 --2- 172.21.15.96:0/3945434581 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a575c982f0 0x55a576061ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:23.890 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.880+0000 7fd2537fe640 1 -- 172.21.15.96:0/3945434581 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a575f8e0d0 con 0x55a575ca04d0 2024-04-25T14:15:23.890 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.880+0000 7fd2517fa640 1 -- 172.21.15.96:0/183993461 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fd24401f020 con 0x7fd2541021c0 2024-04-25T14:15:23.890 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.880+0000 7fd2537fe640 1 --2- 172.21.15.96:0/3945434581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a575ca04d0 0x55a576068df0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fd248005f70 tx=0x7fd248000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a98d6c4261561b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:23.890 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.880+0000 7fd23a7fc640 1 -- 172.21.15.96:0/3945434581 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd24803e030 con 0x55a575ca04d0 2024-04-25T14:15:23.890 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.881+0000 7fd23a7fc640 1 -- 172.21.15.96:0/3945434581 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd248032d60 con 0x55a575ca04d0 2024-04-25T14:15:23.890 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.881+0000 7fd23a7fc640 1 -- 172.21.15.96:0/3945434581 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd248035040 con 0x55a575ca04d0 2024-04-25T14:15:23.890 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.881+0000 7fd2517fa640 1 --2- 172.21.15.96:0/183993461 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd23c0afae0 0x7fd23c0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:23.890 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.881+0000 7fd258e2f640 1 --2- 172.21.15.96:0/183993461 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd23c0afae0 0x7fd23c0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:23.890 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.881+0000 7fd2517fa640 1 -- 172.21.15.96:0/183993461 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fd2440deea0 con 0x7fd2541021c0 2024-04-25T14:15:23.890 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.881+0000 7fd261819740 1 -- 172.21.15.96:0/3945434581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a575ca04d0 msgr2=0x55a576068df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:23.890 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.881+0000 7fd258e2f640 1 --2- 172.21.15.96:0/183993461 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd23c0afae0 0x7fd23c0b1ff0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55a576062b40 tx=0x7fd24c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:23.890 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.881+0000 7fd261819740 1 --2- 172.21.15.96:0/3945434581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a575ca04d0 0x55a576068df0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fd248005f70 tx=0x7fd248000bc0 comp rx=0 tx=0).stop 2024-04-25T14:15:23.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.882+0000 7fd261819740 1 -- 172.21.15.96:0/3945434581 shutdown_connections 2024-04-25T14:15:23.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.882+0000 7fd261819740 1 --2- 172.21.15.96:0/3945434581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a575ca04d0 0x55a576068df0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:23.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.882+0000 7fd261819740 1 --2- 172.21.15.96:0/3945434581 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a575c982f0 0x55a576061ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:23.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.882+0000 7fd261819740 1 --2- 172.21.15.96:0/3945434581 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a575cc11b0 0x55a576061970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:23.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.882+0000 7fd261819740 1 -- 172.21.15.96:0/3945434581 >> 172.21.15.96:0/3945434581 conn(0x55a575cb5300 msgr2=0x55a57605d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:23.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.882+0000 7fd261819740 1 -- 172.21.15.96:0/3945434581 shutdown_connections 2024-04-25T14:15:23.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.882+0000 7fd261819740 1 -- 172.21.15.96:0/3945434581 wait complete. 2024-04-25T14:15:23.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.882+0000 7fd261819740 1 -- start start 2024-04-25T14:15:23.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.882+0000 7fd261819740 1 Processor -- start 2024-04-25T14:15:23.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.882+0000 7fd261819740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a575c982f0 0x55a57606b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:23.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.882+0000 7fd261819740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a575ca04d0 0x55a57606bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:23.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.882+0000 7fd261819740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a575cc11b0 0x55a576072ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:23.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.882+0000 7fd261819740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a5760697e0 con 0x55a575cc11b0 2024-04-25T14:15:23.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.882+0000 7fd261819740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a575f4eb60 con 0x55a575c982f0 2024-04-25T14:15:23.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.883+0000 7fd261819740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a576069960 con 0x55a575ca04d0 2024-04-25T14:15:23.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.883+0000 7fd261819740 10 client.0 _ll_register_callbacks cb 0x55a575f732b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:15:23.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.883+0000 7fd258e2f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a575cc11b0 0x55a576072ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:23.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.883+0000 7fd258e2f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a575cc11b0 0x55a576072ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39734/0 (socket says 172.21.15.96:39734) 2024-04-25T14:15:23.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.883+0000 7fd258e2f640 1 -- 172.21.15.96:0/4202276824 learned_addr learned my addr 172.21.15.96:0/4202276824 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:23.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.883+0000 7fd2537fe640 1 --2- 172.21.15.96:0/4202276824 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a575ca04d0 0x55a57606bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:23.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.883+0000 7fd253fff640 1 --2- 172.21.15.96:0/4202276824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a575c982f0 0x55a57606b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:23.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.883+0000 7fd258e2f640 1 -- 172.21.15.96:0/4202276824 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a575ca04d0 msgr2=0x55a57606bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:23.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.883+0000 7fd258e2f640 1 --2- 172.21.15.96:0/4202276824 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a575ca04d0 0x55a57606bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:23.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.883+0000 7fd258e2f640 1 -- 172.21.15.96:0/4202276824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a575c982f0 msgr2=0x55a57606b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:23.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.883+0000 7fd258e2f640 1 --2- 172.21.15.96:0/4202276824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a575c982f0 0x55a57606b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.883+0000 7fd258e2f640 1 -- 172.21.15.96:0/4202276824 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a575cd9210 con 0x55a575cc11b0 2024-04-25T14:15:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.883+0000 7fd258e2f640 1 --2- 172.21.15.96:0/4202276824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a575cc11b0 0x55a576072ab0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fd24c003fd0 tx=0x7fd24c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e5290bc83eeacbe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.883+0000 7fd2337fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:15:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.883+0000 7fd2337fe640 1 -- 172.21.15.96:0/4202276824 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd24c00f8f0 con 0x55a575cc11b0 2024-04-25T14:15:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.883+0000 7fd2337fe640 1 -- 172.21.15.96:0/4202276824 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fd24c00fa90 con 0x55a575cc11b0 2024-04-25T14:15:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.883+0000 7fd261819740 1 -- 172.21.15.96:0/4202276824 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd248032d60 con 0x55a575cc11b0 2024-04-25T14:15:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.884+0000 7fd2337fe640 1 -- 172.21.15.96:0/4202276824 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd24c00fc90 con 0x55a575cc11b0 2024-04-25T14:15:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.884+0000 7fd261819740 10 client.16218 fetch_fsmap learned FSMap version 26 2024-04-25T14:15:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.884+0000 7fd261819740 1 -- 172.21.15.96:0/4202276824 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a575da5620 con 0x55a575cc11b0 2024-04-25T14:15:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.884+0000 7fd2337fe640 1 -- 172.21.15.96:0/4202276824 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd248032d60 con 0x55a575cc11b0 2024-04-25T14:15:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.884+0000 7fd2337fe640 1 -- 172.21.15.96:0/4202276824 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd24c026070 con 0x55a575cc11b0 2024-04-25T14:15:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.884+0000 7fd261819740 10 client.16218 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:15:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.884+0000 7fd261819740 10 client.16218 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:15:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.884+0000 7fd261819740 20 client.16218 populate_metadata read hostname 'smithi096' 2024-04-25T14:15:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.884+0000 7fd261819740 1 --2- 172.21.15.96:0/4202276824 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55a575c7fd80 0x55a576079330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.884+0000 7fd261819740 4 client.16218 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:15:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.884+0000 7fd261819740 1 -- 172.21.15.96:0/4202276824 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a575fd5070 con 0x55a575c7fd80 2024-04-25T14:15:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.884+0000 7fd253fff640 1 --2- 172.21.15.96:0/4202276824 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55a575c7fd80 0x55a576079330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.884+0000 7fd253fff640 1 --2- 172.21.15.96:0/4202276824 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55a575c7fd80 0x55a576079330 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=920c1a8769156305 server_cookie=52ac9db13a98c536 in_seq=0 out_seq=0 2024-04-25T14:15:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.884+0000 7fd2337fe640 10 client.16218 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:15:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.885+0000 7fd2337fe640 1 -- 172.21.15.96:0/4202276824 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a575fd5070 con 0x55a575c7fd80 2024-04-25T14:15:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.885+0000 7fd2337fe640 10 client.16218 handle_command_reply: tid=0 2024-04-25T14:15:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.885+0000 7fd261819740 1 client.16218 shutdown 2024-04-25T14:15:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.886+0000 7fd261819740 1 -- 172.21.15.96:0/4202276824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a575cc11b0 msgr2=0x55a576072ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.886+0000 7fd261819740 1 --2- 172.21.15.96:0/4202276824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a575cc11b0 0x55a576072ab0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fd24c003fd0 tx=0x7fd24c004ab0 comp rx=0 tx=0).stop 2024-04-25T14:15:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.886+0000 7fd261819740 1 -- 172.21.15.96:0/4202276824 shutdown_connections 2024-04-25T14:15:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.886+0000 7fd261819740 1 --2- 172.21.15.96:0/4202276824 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55a575c7fd80 0x55a576079330 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.886+0000 7fd261819740 1 --2- 172.21.15.96:0/4202276824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a575cc11b0 0x55a576072ab0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.886+0000 7fd261819740 1 --2- 172.21.15.96:0/4202276824 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a575ca04d0 0x55a57606bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.886+0000 7fd2337fe640 0 client.16218 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:15:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.886+0000 7fd261819740 1 --2- 172.21.15.96:0/4202276824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a575c982f0 0x55a57606b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.886+0000 7fd261819740 1 -- 172.21.15.96:0/4202276824 >> 172.21.15.96:0/4202276824 conn(0x55a575cb5300 msgr2=0x55a57605c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.886+0000 7fd261819740 1 -- 172.21.15.96:0/4202276824 shutdown_connections 2024-04-25T14:15:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.886+0000 7fd261819740 1 -- 172.21.15.96:0/4202276824 wait complete. 2024-04-25T14:15:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.886+0000 7fd261819740 20 client.16218 trim_cache size 0 max 16384 2024-04-25T14:15:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.902+0000 7fd261819740 1 Processor -- start 2024-04-25T14:15:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.902+0000 7fd261819740 1 -- start start 2024-04-25T14:15:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.902+0000 7fd261819740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a575c7fd80 0x55a57605cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.902+0000 7fd261819740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a575c982f0 0x55a5760729d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.902+0000 7fd261819740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a575ca04d0 0x55a57605d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.902+0000 7fd261819740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd2541ab310 con 0x55a575c982f0 2024-04-25T14:15:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.902+0000 7fd261819740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd2541ace50 con 0x55a575ca04d0 2024-04-25T14:15:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.902+0000 7fd261819740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd2541ab9e0 con 0x55a575c7fd80 2024-04-25T14:15:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.903+0000 7fd253fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a575c7fd80 0x55a57605cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.903+0000 7fd2537fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a575c982f0 0x55a5760729d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.903+0000 7fd258e2f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a575ca04d0 0x55a57605d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.903+0000 7fd253fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a575c7fd80 0x55a57605cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.96:55774/0 (socket says 172.21.15.96:55774) 2024-04-25T14:15:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.903+0000 7fd253fff640 1 -- 172.21.15.96:0/3228150772 learned_addr learned my addr 172.21.15.96:0/3228150772 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.903+0000 7fd2537fe640 1 -- 172.21.15.96:0/3228150772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a575c7fd80 msgr2=0x55a57605cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.903+0000 7fd2537fe640 1 --2- 172.21.15.96:0/3228150772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a575c7fd80 0x55a57605cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.903+0000 7fd2537fe640 1 -- 172.21.15.96:0/3228150772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a575ca04d0 msgr2=0x55a57605d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.903+0000 7fd2537fe640 1 --2- 172.21.15.96:0/3228150772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a575ca04d0 0x55a57605d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.903+0000 7fd2537fe640 1 -- 172.21.15.96:0/3228150772 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a576072f10 con 0x55a575c982f0 2024-04-25T14:15:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.903+0000 7fd2537fe640 1 --2- 172.21.15.96:0/3228150772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a575c982f0 0x55a5760729d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fd248009900 tx=0x7fd248032f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=482004653b8244c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.903+0000 7fd239ffb640 1 -- 172.21.15.96:0/3228150772 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd24803e040 con 0x55a575c982f0 2024-04-25T14:15:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.903+0000 7fd239ffb640 1 -- 172.21.15.96:0/3228150772 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd24804b070 con 0x55a575c982f0 2024-04-25T14:15:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.903+0000 7fd239ffb640 1 -- 172.21.15.96:0/3228150772 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd248035040 con 0x55a575c982f0 2024-04-25T14:15:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.904+0000 7fd261819740 1 -- 172.21.15.96:0/3228150772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a575c982f0 msgr2=0x55a5760729d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.904+0000 7fd261819740 1 --2- 172.21.15.96:0/3228150772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a575c982f0 0x55a5760729d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fd248009900 tx=0x7fd248032f10 comp rx=0 tx=0).stop 2024-04-25T14:15:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.904+0000 7fd261819740 1 -- 172.21.15.96:0/3228150772 shutdown_connections 2024-04-25T14:15:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.904+0000 7fd261819740 1 --2- 172.21.15.96:0/3228150772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a575ca04d0 0x55a57605d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.904+0000 7fd261819740 1 --2- 172.21.15.96:0/3228150772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a575c982f0 0x55a5760729d0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.904+0000 7fd261819740 1 --2- 172.21.15.96:0/3228150772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a575c7fd80 0x55a57605cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.904+0000 7fd261819740 1 -- 172.21.15.96:0/3228150772 >> 172.21.15.96:0/3228150772 conn(0x55a575cb5300 msgr2=0x55a576065ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.904+0000 7fd261819740 1 -- 172.21.15.96:0/3228150772 shutdown_connections 2024-04-25T14:15:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.904+0000 7fd261819740 1 -- 172.21.15.96:0/3228150772 wait complete. 2024-04-25T14:15:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.905+0000 7fd261819740 1 -- start start 2024-04-25T14:15:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.905+0000 7fd261819740 1 Processor -- start 2024-04-25T14:15:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.905+0000 7fd261819740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a575c7fd80 0x55a576070020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.905+0000 7fd261819740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a575c982f0 0x55a576070560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.905+0000 7fd261819740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a575ca04d0 0x55a576071440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.905+0000 7fd261819740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd2541ac780 con 0x55a575c982f0 2024-04-25T14:15:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.905+0000 7fd261819740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd2541ac0b0 con 0x55a575c7fd80 2024-04-25T14:15:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.905+0000 7fd261819740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd2541a92f0 con 0x55a575ca04d0 2024-04-25T14:15:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.905+0000 7fd261819740 10 client.0 _ll_register_callbacks cb 0x7fd24c009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:15:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.905+0000 7fd2537fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a575c982f0 0x55a576070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.905+0000 7fd2537fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a575c982f0 0x55a576070560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39758/0 (socket says 172.21.15.96:39758) 2024-04-25T14:15:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.905+0000 7fd2537fe640 1 -- 172.21.15.96:0/1845862494 learned_addr learned my addr 172.21.15.96:0/1845862494 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.905+0000 7fd258e2f640 1 --2- 172.21.15.96:0/1845862494 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a575ca04d0 0x55a576071440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.905+0000 7fd253fff640 1 --2- 172.21.15.96:0/1845862494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a575c7fd80 0x55a576070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.905+0000 7fd2537fe640 1 -- 172.21.15.96:0/1845862494 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a575ca04d0 msgr2=0x55a576071440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.905+0000 7fd2537fe640 1 --2- 172.21.15.96:0/1845862494 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a575ca04d0 0x55a576071440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.905+0000 7fd2537fe640 1 -- 172.21.15.96:0/1845862494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a575c7fd80 msgr2=0x55a576070020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.905+0000 7fd2537fe640 1 --2- 172.21.15.96:0/1845862494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a575c7fd80 0x55a576070020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.905+0000 7fd2537fe640 1 -- 172.21.15.96:0/1845862494 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a576071a00 con 0x55a575c982f0 2024-04-25T14:15:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.906+0000 7fd2537fe640 1 --2- 172.21.15.96:0/1845862494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a575c982f0 0x55a576070560 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fd24802a930 tx=0x7fd248005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8af8043195d8e4e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.906+0000 7fd2337fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:15:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.906+0000 7fd2337fe640 1 -- 172.21.15.96:0/1845862494 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd248056060 con 0x55a575c982f0 2024-04-25T14:15:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.906+0000 7fd2337fe640 1 -- 172.21.15.96:0/1845862494 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fd2480317a0 con 0x55a575c982f0 2024-04-25T14:15:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.906+0000 7fd261819740 1 -- 172.21.15.96:0/1845862494 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd24804b070 con 0x55a575c982f0 2024-04-25T14:15:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.906+0000 7fd2337fe640 1 -- 172.21.15.96:0/1845862494 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd24803e050 con 0x55a575c982f0 2024-04-25T14:15:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.906+0000 7fd261819740 10 client.16224 fetch_fsmap learned FSMap version 26 2024-04-25T14:15:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.906+0000 7fd261819740 1 -- 172.21.15.96:0/1845862494 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a576072c90 con 0x55a575c982f0 2024-04-25T14:15:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.906+0000 7fd2337fe640 1 -- 172.21.15.96:0/1845862494 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd24804b070 con 0x55a575c982f0 2024-04-25T14:15:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.906+0000 7fd2337fe640 1 -- 172.21.15.96:0/1845862494 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd248004bb0 con 0x55a575c982f0 2024-04-25T14:15:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.906+0000 7fd261819740 10 client.16224 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:15:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.906+0000 7fd261819740 10 client.16224 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:15:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.906+0000 7fd261819740 20 client.16224 populate_metadata read hostname 'smithi096' 2024-04-25T14:15:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.906+0000 7fd261819740 1 --2- 172.21.15.96:0/1845862494 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55a575cc11b0 0x55a5760e3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.906+0000 7fd261819740 4 client.16224 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:15:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.906+0000 7fd261819740 1 -- 172.21.15.96:0/1845862494 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a575fceee0 con 0x55a575cc11b0 2024-04-25T14:15:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.907+0000 7fd253fff640 1 --2- 172.21.15.96:0/1845862494 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55a575cc11b0 0x55a5760e3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.907+0000 7fd2337fe640 10 client.16224 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:15:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.907+0000 7fd253fff640 1 --2- 172.21.15.96:0/1845862494 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55a575cc11b0 0x55a5760e3a60 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9311216a474d165 server_cookie=180edfdb90da6247 in_seq=0 out_seq=0 2024-04-25T14:15:23.962 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.961+0000 7fd2337fe640 1 -- 172.21.15.96:0/1845862494 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a575fceee0 con 0x55a575cc11b0 2024-04-25T14:15:23.962 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.961+0000 7fd2337fe640 10 client.16224 handle_command_reply: tid=0 2024-04-25T14:15:23.962 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.961+0000 7fd261819740 1 client.16224 shutdown 2024-04-25T14:15:23.962 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.961+0000 7fd261819740 1 -- 172.21.15.96:0/1845862494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a575c982f0 msgr2=0x55a576070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:23.962 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.961+0000 7fd261819740 1 --2- 172.21.15.96:0/1845862494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a575c982f0 0x55a576070560 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fd24802a930 tx=0x7fd248005dd0 comp rx=0 tx=0).stop 2024-04-25T14:15:23.963 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.961+0000 7fd261819740 1 -- 172.21.15.96:0/1845862494 shutdown_connections 2024-04-25T14:15:23.963 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.961+0000 7fd261819740 1 --2- 172.21.15.96:0/1845862494 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a575ca04d0 0x55a576071440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:23.963 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.962+0000 7fd261819740 1 --2- 172.21.15.96:0/1845862494 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x55a575cc11b0 0x55a5760e3a60 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:23.963 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.962+0000 7fd261819740 1 --2- 172.21.15.96:0/1845862494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a575c982f0 0x55a576070560 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:23.963 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.962+0000 7fd261819740 1 --2- 172.21.15.96:0/1845862494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a575c7fd80 0x55a576070020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:23.963 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.962+0000 7fd2337fe640 0 client.16224 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:15:23.963 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.962+0000 7fd261819740 1 -- 172.21.15.96:0/1845862494 >> 172.21.15.96:0/1845862494 conn(0x55a575cb5300 msgr2=0x55a576065ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:23.963 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.962+0000 7fd261819740 1 -- 172.21.15.96:0/1845862494 shutdown_connections 2024-04-25T14:15:23.963 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.962+0000 7fd261819740 1 -- 172.21.15.96:0/1845862494 wait complete. 2024-04-25T14:15:23.963 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.962+0000 7fd261819740 20 client.16224 trim_cache size 0 max 16384 2024-04-25T14:15:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T14:15:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T14:15:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T14:15:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T14:15:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.962+0000 7fd25a892640 1 -- 172.21.15.96:0/183993461 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd23c0afae0 msgr2=0x7fd23c0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.962+0000 7fd25a892640 1 --2- 172.21.15.96:0/183993461 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd23c0afae0 0x7fd23c0b1ff0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55a576062b40 tx=0x7fd24c009500 comp rx=0 tx=0).stop 2024-04-25T14:15:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.962+0000 7fd25a892640 1 -- 172.21.15.96:0/183993461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2541021c0 msgr2=0x7fd25410f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.962+0000 7fd25a892640 1 --2- 172.21.15.96:0/183993461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2541021c0 0x7fd25410f120 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fd24400e980 tx=0x7fd24400ee40 comp rx=0 tx=0).stop 2024-04-25T14:15:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.962+0000 7fd25a892640 1 -- 172.21.15.96:0/183993461 shutdown_connections 2024-04-25T14:15:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.962+0000 7fd25a892640 1 --2- 172.21.15.96:0/183993461 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd254108100 0x7fd2541a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.962+0000 7fd25a892640 1 --2- 172.21.15.96:0/183993461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd254103400 0x7fd25419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.962+0000 7fd25a892640 1 --2- 172.21.15.96:0/183993461 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fd23c0afae0 0x7fd23c0b1ff0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.962+0000 7fd25a892640 1 --2- 172.21.15.96:0/183993461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2541021c0 0x7fd25410f120 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.962+0000 7fd25a892640 1 -- 172.21.15.96:0/183993461 >> 172.21.15.96:0/183993461 conn(0x7fd2540fd390 msgr2=0x7fd2540ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.962+0000 7fd25a892640 1 -- 172.21.15.96:0/183993461 shutdown_connections 2024-04-25T14:15:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T14:15:23.963+0000 7fd25a892640 1 -- 172.21.15.96:0/183993461 wait complete. 2024-04-25T14:15:24.301 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4191744 len 8192 2024-04-25T14:15:24.301 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4191744 len 16384 2024-04-25T14:15:24.301 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4191744 len 16384 2024-04-25T14:15:24.301 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4191744 len 16384 2024-04-25T14:15:24.301 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4191744 len 16384 2024-04-25T14:15:24.301 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4192256 len 1024 2024-04-25T14:15:24.301 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4192256 len 1024 2024-04-25T14:15:24.301 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4192256 len 1024 2024-04-25T14:15:24.301 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4192256 len 1024 2024-04-25T14:15:24.301 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4192256 len 2048 2024-04-25T14:15:24.301 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4192256 len 2048 2024-04-25T14:15:24.301 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4192256 len 2048 2024-04-25T14:15:24.302 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4192256 len 2048 2024-04-25T14:15:24.302 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4192256 len 4096 2024-04-25T14:15:24.302 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4192256 len 4096 2024-04-25T14:15:24.302 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4192256 len 4096 2024-04-25T14:15:24.302 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4192256 len 4096 2024-04-25T14:15:24.302 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4192256 len 8192 2024-04-25T14:15:24.302 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4192256 len 8192 2024-04-25T14:15:24.302 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4192256 len 8192 2024-04-25T14:15:24.302 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4192256 len 8192 2024-04-25T14:15:24.302 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4192256 len 16384 2024-04-25T14:15:24.302 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4192256 len 16384 2024-04-25T14:15:24.302 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4192256 len 16384 2024-04-25T14:15:24.302 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4192256 len 16384 2024-04-25T14:15:24.303 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4192768 len 1024 2024-04-25T14:15:24.303 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4192768 len 1024 2024-04-25T14:15:24.303 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4192768 len 1024 2024-04-25T14:15:24.303 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4192768 len 1024 2024-04-25T14:15:24.303 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4192768 len 2048 2024-04-25T14:15:24.303 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4192768 len 2048 2024-04-25T14:15:24.303 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4192768 len 2048 2024-04-25T14:15:24.303 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4192768 len 2048 2024-04-25T14:15:24.303 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4192768 len 4096 2024-04-25T14:15:24.303 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4192768 len 4096 2024-04-25T14:15:24.303 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4192768 len 4096 2024-04-25T14:15:24.303 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4192768 len 4096 2024-04-25T14:15:24.303 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4192768 len 8192 2024-04-25T14:15:24.303 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4192768 len 8192 2024-04-25T14:15:24.304 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4192768 len 8192 2024-04-25T14:15:24.304 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4192768 len 8192 2024-04-25T14:15:24.304 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4192768 len 16384 2024-04-25T14:15:24.304 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4192768 len 16384 2024-04-25T14:15:24.304 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4192768 len 16384 2024-04-25T14:15:24.304 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4192768 len 16384 2024-04-25T14:15:24.304 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4193280 len 1024 2024-04-25T14:15:24.304 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4193280 len 1024 2024-04-25T14:15:24.304 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4193280 len 1024 2024-04-25T14:15:24.304 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4193280 len 1024 2024-04-25T14:15:24.304 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4193280 len 2048 2024-04-25T14:15:24.304 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4193280 len 2048 2024-04-25T14:15:24.304 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4193280 len 2048 2024-04-25T14:15:24.304 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4193280 len 2048 2024-04-25T14:15:24.305 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4193280 len 4096 2024-04-25T14:15:24.305 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4193280 len 4096 2024-04-25T14:15:24.305 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4193280 len 4096 2024-04-25T14:15:24.305 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4193280 len 4096 2024-04-25T14:15:24.305 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4193280 len 8192 2024-04-25T14:15:24.305 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4193280 len 8192 2024-04-25T14:15:24.305 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4193280 len 8192 2024-04-25T14:15:24.305 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4193280 len 8192 2024-04-25T14:15:24.305 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4193280 len 16384 2024-04-25T14:15:24.305 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4193280 len 16384 2024-04-25T14:15:24.305 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4193280 len 16384 2024-04-25T14:15:24.305 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4193280 len 16384 2024-04-25T14:15:24.305 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4193792 len 1024 2024-04-25T14:15:24.305 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4193792 len 1024 2024-04-25T14:15:24.305 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4193792 len 1024 2024-04-25T14:15:24.306 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4193792 len 1024 2024-04-25T14:15:24.306 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4193792 len 2048 2024-04-25T14:15:24.306 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4193792 len 2048 2024-04-25T14:15:24.306 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4193792 len 2048 2024-04-25T14:15:24.306 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4193792 len 2048 2024-04-25T14:15:24.306 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4193792 len 4096 2024-04-25T14:15:24.306 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4193792 len 4096 2024-04-25T14:15:24.306 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4193792 len 4096 2024-04-25T14:15:24.306 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4193792 len 4096 2024-04-25T14:15:24.306 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4193792 len 8192 2024-04-25T14:15:24.306 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4193792 len 8192 2024-04-25T14:15:24.306 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4193792 len 8192 2024-04-25T14:15:24.306 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4193792 len 8192 2024-04-25T14:15:24.717 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4193792 len 16384 2024-04-25T14:15:24.718 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4193792 len 16384 2024-04-25T14:15:24.718 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4193792 len 16384 2024-04-25T14:15:24.718 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4193792 len 16384 2024-04-25T14:15:24.718 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4194304 len 1024 2024-04-25T14:15:24.718 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4194304 len 1024 2024-04-25T14:15:24.718 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4194304 len 1024 2024-04-25T14:15:24.718 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4194304 len 1024 2024-04-25T14:15:24.718 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4194304 len 2048 2024-04-25T14:15:24.718 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4194304 len 2048 2024-04-25T14:15:24.718 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4194304 len 2048 2024-04-25T14:15:24.718 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4194304 len 2048 2024-04-25T14:15:24.718 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4194304 len 4096 2024-04-25T14:15:24.719 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4194304 len 4096 2024-04-25T14:15:24.719 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4194304 len 4096 2024-04-25T14:15:24.719 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4194304 len 4096 2024-04-25T14:15:24.719 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4194304 len 8192 2024-04-25T14:15:24.719 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4194304 len 8192 2024-04-25T14:15:24.719 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4194304 len 8192 2024-04-25T14:15:24.719 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4194304 len 8192 2024-04-25T14:15:24.719 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4194304 len 16384 2024-04-25T14:15:24.719 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4194304 len 16384 2024-04-25T14:15:24.719 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4194304 len 16384 2024-04-25T14:15:24.719 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4194304 len 16384 2024-04-25T14:15:24.719 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4194816 len 1024 2024-04-25T14:15:24.719 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4194816 len 1024 2024-04-25T14:15:24.719 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4194816 len 1024 2024-04-25T14:15:24.719 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4194816 len 1024 2024-04-25T14:15:24.720 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4194816 len 2048 2024-04-25T14:15:24.720 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4194816 len 2048 2024-04-25T14:15:24.720 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4194816 len 2048 2024-04-25T14:15:24.720 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4194816 len 2048 2024-04-25T14:15:24.720 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4194816 len 4096 2024-04-25T14:15:24.720 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4194816 len 4096 2024-04-25T14:15:24.720 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4194816 len 4096 2024-04-25T14:15:24.720 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4194816 len 4096 2024-04-25T14:15:24.720 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4194816 len 8192 2024-04-25T14:15:24.720 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4194816 len 8192 2024-04-25T14:15:24.720 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4194816 len 8192 2024-04-25T14:15:24.720 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4194816 len 8192 2024-04-25T14:15:24.720 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4194816 len 16384 2024-04-25T14:15:24.721 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4194816 len 16384 2024-04-25T14:15:24.721 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4194816 len 16384 2024-04-25T14:15:24.721 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4194816 len 16384 2024-04-25T14:15:24.721 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4195328 len 1024 2024-04-25T14:15:24.721 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4195328 len 1024 2024-04-25T14:15:24.721 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4195328 len 1024 2024-04-25T14:15:24.721 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4195328 len 1024 2024-04-25T14:15:24.721 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4195328 len 2048 2024-04-25T14:15:24.721 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4195328 len 2048 2024-04-25T14:15:24.721 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4195328 len 2048 2024-04-25T14:15:24.721 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4195328 len 2048 2024-04-25T14:15:24.721 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4195328 len 4096 2024-04-25T14:15:24.721 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4195328 len 4096 2024-04-25T14:15:24.721 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4195328 len 4096 2024-04-25T14:15:24.722 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4195328 len 4096 2024-04-25T14:15:24.722 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4195328 len 8192 2024-04-25T14:15:24.722 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4195328 len 8192 2024-04-25T14:15:24.722 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4195328 len 8192 2024-04-25T14:15:24.722 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4195328 len 8192 2024-04-25T14:15:24.722 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4195328 len 16384 2024-04-25T14:15:24.722 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4195328 len 16384 2024-04-25T14:15:24.722 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4195328 len 16384 2024-04-25T14:15:24.722 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4195328 len 16384 2024-04-25T14:15:24.722 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4195840 len 1024 2024-04-25T14:15:24.722 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4195840 len 1024 2024-04-25T14:15:24.722 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4195840 len 1024 2024-04-25T14:15:24.723 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4195840 len 1024 2024-04-25T14:15:24.723 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4195840 len 2048 2024-04-25T14:15:24.723 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4195840 len 2048 2024-04-25T14:15:24.726 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4195840 len 2048 2024-04-25T14:15:24.726 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4195840 len 2048 2024-04-25T14:15:24.726 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4195840 len 4096 2024-04-25T14:15:24.726 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4195840 len 4096 2024-04-25T14:15:24.726 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4195840 len 4096 2024-04-25T14:15:24.726 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4195840 len 4096 2024-04-25T14:15:24.726 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4195840 len 8192 2024-04-25T14:15:24.726 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4195840 len 8192 2024-04-25T14:15:24.727 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4195840 len 8192 2024-04-25T14:15:24.727 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4195840 len 8192 2024-04-25T14:15:24.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:24 smithi096 ceph-mon[29207]: pgmap v399: 97 pgs: 97 active+clean; 109 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 73 MiB/s rd, 1.3 MiB/s wr, 975 op/s 2024-04-25T14:15:24.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:24 smithi096 ceph-mon[29207]: scrub summary: idle 2024-04-25T14:15:24.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:24 smithi097 ceph-mon[32720]: pgmap v399: 97 pgs: 97 active+clean; 109 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 73 MiB/s rd, 1.3 MiB/s wr, 975 op/s 2024-04-25T14:15:24.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:24 smithi097 ceph-mon[32720]: scrub summary: idle 2024-04-25T14:15:24.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:24 smithi151 ceph-mon[32990]: pgmap v399: 97 pgs: 97 active+clean; 109 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 73 MiB/s rd, 1.3 MiB/s wr, 975 op/s 2024-04-25T14:15:24.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:24 smithi151 ceph-mon[32990]: scrub summary: idle 2024-04-25T14:15:25.090 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4195840 len 16384 2024-04-25T14:15:25.090 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4195840 len 16384 2024-04-25T14:15:25.090 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4195840 len 16384 2024-04-25T14:15:25.090 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4195840 len 16384 2024-04-25T14:15:25.090 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4196352 len 1024 2024-04-25T14:15:25.091 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4196352 len 1024 2024-04-25T14:15:25.091 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4196352 len 1024 2024-04-25T14:15:25.091 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4196352 len 1024 2024-04-25T14:15:25.091 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4196352 len 2048 2024-04-25T14:15:25.091 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4196352 len 2048 2024-04-25T14:15:25.091 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4196352 len 2048 2024-04-25T14:15:25.091 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4196352 len 2048 2024-04-25T14:15:25.092 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4196352 len 4096 2024-04-25T14:15:25.092 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4196352 len 4096 2024-04-25T14:15:25.092 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4196352 len 4096 2024-04-25T14:15:25.092 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4196352 len 4096 2024-04-25T14:15:25.092 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4196352 len 8192 2024-04-25T14:15:25.092 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4196352 len 8192 2024-04-25T14:15:25.092 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4196352 len 8192 2024-04-25T14:15:25.093 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4196352 len 8192 2024-04-25T14:15:25.093 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4196352 len 16384 2024-04-25T14:15:25.093 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4196352 len 16384 2024-04-25T14:15:25.093 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4196352 len 16384 2024-04-25T14:15:25.093 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4196352 len 16384 2024-04-25T14:15:25.093 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4196864 len 1024 2024-04-25T14:15:25.093 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4196864 len 1024 2024-04-25T14:15:25.093 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4196864 len 1024 2024-04-25T14:15:25.094 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4196864 len 1024 2024-04-25T14:15:25.094 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4196864 len 2048 2024-04-25T14:15:25.094 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4196864 len 2048 2024-04-25T14:15:25.094 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4196864 len 2048 2024-04-25T14:15:25.094 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4196864 len 2048 2024-04-25T14:15:25.094 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4196864 len 4096 2024-04-25T14:15:25.094 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4196864 len 4096 2024-04-25T14:15:25.095 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4196864 len 4096 2024-04-25T14:15:25.095 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4196864 len 4096 2024-04-25T14:15:25.095 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4196864 len 8192 2024-04-25T14:15:25.095 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4196864 len 8192 2024-04-25T14:15:25.095 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4196864 len 8192 2024-04-25T14:15:25.095 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4196864 len 8192 2024-04-25T14:15:25.096 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4196864 len 16384 2024-04-25T14:15:25.096 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4196864 len 16384 2024-04-25T14:15:25.096 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4196864 len 16384 2024-04-25T14:15:25.096 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4196864 len 16384 2024-04-25T14:15:25.096 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4197376 len 1024 2024-04-25T14:15:25.096 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4197376 len 1024 2024-04-25T14:15:25.097 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4197376 len 1024 2024-04-25T14:15:25.097 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4197376 len 1024 2024-04-25T14:15:25.097 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4197376 len 2048 2024-04-25T14:15:25.097 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4197376 len 2048 2024-04-25T14:15:25.097 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4197376 len 2048 2024-04-25T14:15:25.097 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4197376 len 2048 2024-04-25T14:15:25.097 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4197376 len 4096 2024-04-25T14:15:25.097 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4197376 len 4096 2024-04-25T14:15:25.097 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4197376 len 4096 2024-04-25T14:15:25.097 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4197376 len 4096 2024-04-25T14:15:25.098 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4197376 len 8192 2024-04-25T14:15:25.098 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4197376 len 8192 2024-04-25T14:15:25.098 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4197376 len 8192 2024-04-25T14:15:25.098 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4197376 len 8192 2024-04-25T14:15:25.098 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4197376 len 16384 2024-04-25T14:15:25.098 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4197376 len 16384 2024-04-25T14:15:25.098 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4197376 len 16384 2024-04-25T14:15:25.098 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4197376 len 16384 2024-04-25T14:15:25.098 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4197888 len 1024 2024-04-25T14:15:25.098 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4197888 len 1024 2024-04-25T14:15:25.098 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4197888 len 1024 2024-04-25T14:15:25.098 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4197888 len 1024 2024-04-25T14:15:25.098 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4197888 len 2048 2024-04-25T14:15:25.099 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4197888 len 2048 2024-04-25T14:15:25.099 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4197888 len 2048 2024-04-25T14:15:25.099 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4197888 len 2048 2024-04-25T14:15:25.099 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4197888 len 4096 2024-04-25T14:15:25.099 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4197888 len 4096 2024-04-25T14:15:25.099 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4197888 len 4096 2024-04-25T14:15:25.099 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4197888 len 4096 2024-04-25T14:15:25.099 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4197888 len 8192 2024-04-25T14:15:25.099 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4197888 len 8192 2024-04-25T14:15:25.099 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4197888 len 8192 2024-04-25T14:15:25.099 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4197888 len 8192 2024-04-25T14:15:25.099 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2048 offset 4197888 len 16384 2024-04-25T14:15:25.745 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4197888 len 16384 2024-04-25T14:15:25.745 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2048 offset 4197888 len 16384 2024-04-25T14:15:25.745 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2048 offset 4197888 len 16384 2024-04-25T14:15:25.745 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4190720 len 1024 2024-04-25T14:15:25.745 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4190720 len 1024 2024-04-25T14:15:25.745 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4190720 len 1024 2024-04-25T14:15:25.746 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4190720 len 1024 2024-04-25T14:15:25.746 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4190720 len 2048 2024-04-25T14:15:25.746 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4190720 len 2048 2024-04-25T14:15:25.746 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4190720 len 2048 2024-04-25T14:15:25.746 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4190720 len 2048 2024-04-25T14:15:25.746 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4190720 len 4096 2024-04-25T14:15:25.746 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4190720 len 4096 2024-04-25T14:15:25.746 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4190720 len 4096 2024-04-25T14:15:25.746 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4190720 len 4096 2024-04-25T14:15:25.746 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4190720 len 8192 2024-04-25T14:15:25.746 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4190720 len 8192 2024-04-25T14:15:25.746 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4190720 len 8192 2024-04-25T14:15:25.746 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4190720 len 8192 2024-04-25T14:15:25.746 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4190720 len 16384 2024-04-25T14:15:25.747 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4190720 len 16384 2024-04-25T14:15:25.747 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4190720 len 16384 2024-04-25T14:15:25.747 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4190720 len 16384 2024-04-25T14:15:25.747 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4191232 len 1024 2024-04-25T14:15:25.747 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4191232 len 1024 2024-04-25T14:15:25.747 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4191232 len 1024 2024-04-25T14:15:25.747 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4191232 len 1024 2024-04-25T14:15:25.747 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4191232 len 2048 2024-04-25T14:15:25.747 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4191232 len 2048 2024-04-25T14:15:25.747 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4191232 len 2048 2024-04-25T14:15:25.747 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4191232 len 2048 2024-04-25T14:15:25.747 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4191232 len 4096 2024-04-25T14:15:25.747 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4191232 len 4096 2024-04-25T14:15:25.747 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4191232 len 4096 2024-04-25T14:15:25.748 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4191232 len 4096 2024-04-25T14:15:25.748 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4191232 len 8192 2024-04-25T14:15:25.748 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4191232 len 8192 2024-04-25T14:15:25.748 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4191232 len 8192 2024-04-25T14:15:25.748 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4191232 len 8192 2024-04-25T14:15:25.748 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4191232 len 16384 2024-04-25T14:15:25.748 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4191232 len 16384 2024-04-25T14:15:25.748 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4191232 len 16384 2024-04-25T14:15:25.748 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4191232 len 16384 2024-04-25T14:15:25.748 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4191744 len 1024 2024-04-25T14:15:25.748 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4191744 len 1024 2024-04-25T14:15:25.748 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4191744 len 1024 2024-04-25T14:15:25.748 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4191744 len 1024 2024-04-25T14:15:25.748 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4191744 len 2048 2024-04-25T14:15:25.749 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4191744 len 2048 2024-04-25T14:15:25.749 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4191744 len 2048 2024-04-25T14:15:25.749 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4191744 len 2048 2024-04-25T14:15:25.749 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4191744 len 4096 2024-04-25T14:15:25.749 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4191744 len 4096 2024-04-25T14:15:25.749 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4191744 len 4096 2024-04-25T14:15:25.749 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4191744 len 4096 2024-04-25T14:15:25.749 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4191744 len 8192 2024-04-25T14:15:25.749 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4191744 len 8192 2024-04-25T14:15:25.749 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4191744 len 8192 2024-04-25T14:15:25.749 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4191744 len 8192 2024-04-25T14:15:25.749 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4191744 len 16384 2024-04-25T14:15:25.749 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4191744 len 16384 2024-04-25T14:15:25.749 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4191744 len 16384 2024-04-25T14:15:25.749 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4191744 len 16384 2024-04-25T14:15:25.750 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4192256 len 1024 2024-04-25T14:15:25.750 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4192256 len 1024 2024-04-25T14:15:25.750 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4192256 len 1024 2024-04-25T14:15:25.750 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4192256 len 1024 2024-04-25T14:15:25.750 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4192256 len 2048 2024-04-25T14:15:25.750 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4192256 len 2048 2024-04-25T14:15:25.750 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4192256 len 2048 2024-04-25T14:15:25.750 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4192256 len 2048 2024-04-25T14:15:25.750 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4192256 len 4096 2024-04-25T14:15:25.750 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4192256 len 4096 2024-04-25T14:15:25.750 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4192256 len 4096 2024-04-25T14:15:25.750 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4192256 len 4096 2024-04-25T14:15:25.750 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4192256 len 8192 2024-04-25T14:15:25.750 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4192256 len 8192 2024-04-25T14:15:25.750 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4192256 len 8192 2024-04-25T14:15:25.751 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4192256 len 8192 2024-04-25T14:15:25.751 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4192256 len 16384 2024-04-25T14:15:25.751 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4192256 len 16384 2024-04-25T14:15:26.376 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4192256 len 16384 2024-04-25T14:15:26.376 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4192256 len 16384 2024-04-25T14:15:26.377 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4192768 len 1024 2024-04-25T14:15:26.377 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4192768 len 1024 2024-04-25T14:15:26.377 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4192768 len 1024 2024-04-25T14:15:26.377 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4192768 len 1024 2024-04-25T14:15:26.377 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4192768 len 2048 2024-04-25T14:15:26.377 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4192768 len 2048 2024-04-25T14:15:26.377 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4192768 len 2048 2024-04-25T14:15:26.377 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4192768 len 2048 2024-04-25T14:15:26.377 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4192768 len 4096 2024-04-25T14:15:26.377 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4192768 len 4096 2024-04-25T14:15:26.377 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4192768 len 4096 2024-04-25T14:15:26.377 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4192768 len 4096 2024-04-25T14:15:26.377 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4192768 len 8192 2024-04-25T14:15:26.377 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4192768 len 8192 2024-04-25T14:15:26.378 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4192768 len 8192 2024-04-25T14:15:26.378 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4192768 len 8192 2024-04-25T14:15:26.378 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4192768 len 16384 2024-04-25T14:15:26.378 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4192768 len 16384 2024-04-25T14:15:26.378 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4192768 len 16384 2024-04-25T14:15:26.378 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4192768 len 16384 2024-04-25T14:15:26.378 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4193280 len 1024 2024-04-25T14:15:26.378 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4193280 len 1024 2024-04-25T14:15:26.378 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4193280 len 1024 2024-04-25T14:15:26.378 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4193280 len 1024 2024-04-25T14:15:26.378 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4193280 len 2048 2024-04-25T14:15:26.378 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4193280 len 2048 2024-04-25T14:15:26.378 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4193280 len 2048 2024-04-25T14:15:26.378 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4193280 len 2048 2024-04-25T14:15:26.379 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4193280 len 4096 2024-04-25T14:15:26.379 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4193280 len 4096 2024-04-25T14:15:26.379 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4193280 len 4096 2024-04-25T14:15:26.379 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4193280 len 4096 2024-04-25T14:15:26.379 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4193280 len 8192 2024-04-25T14:15:26.379 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4193280 len 8192 2024-04-25T14:15:26.379 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4193280 len 8192 2024-04-25T14:15:26.379 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4193280 len 8192 2024-04-25T14:15:26.379 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4193280 len 16384 2024-04-25T14:15:26.379 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4193280 len 16384 2024-04-25T14:15:26.379 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4193280 len 16384 2024-04-25T14:15:26.379 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4193280 len 16384 2024-04-25T14:15:26.379 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4193792 len 1024 2024-04-25T14:15:26.379 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4193792 len 1024 2024-04-25T14:15:26.380 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4193792 len 1024 2024-04-25T14:15:26.380 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4193792 len 1024 2024-04-25T14:15:26.380 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4193792 len 2048 2024-04-25T14:15:26.380 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4193792 len 2048 2024-04-25T14:15:26.380 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4193792 len 2048 2024-04-25T14:15:26.380 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4193792 len 2048 2024-04-25T14:15:26.380 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4193792 len 4096 2024-04-25T14:15:26.380 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4193792 len 4096 2024-04-25T14:15:26.380 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4193792 len 4096 2024-04-25T14:15:26.380 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4193792 len 4096 2024-04-25T14:15:26.380 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4193792 len 8192 2024-04-25T14:15:26.380 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4193792 len 8192 2024-04-25T14:15:26.380 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4193792 len 8192 2024-04-25T14:15:26.380 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4193792 len 8192 2024-04-25T14:15:26.381 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4193792 len 16384 2024-04-25T14:15:26.381 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4193792 len 16384 2024-04-25T14:15:26.381 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4193792 len 16384 2024-04-25T14:15:26.381 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4193792 len 16384 2024-04-25T14:15:26.381 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4194304 len 1024 2024-04-25T14:15:26.381 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4194304 len 1024 2024-04-25T14:15:26.381 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4194304 len 1024 2024-04-25T14:15:26.381 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4194304 len 1024 2024-04-25T14:15:26.381 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4194304 len 2048 2024-04-25T14:15:26.381 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4194304 len 2048 2024-04-25T14:15:26.381 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4194304 len 2048 2024-04-25T14:15:26.381 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4194304 len 2048 2024-04-25T14:15:26.381 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4194304 len 4096 2024-04-25T14:15:26.382 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4194304 len 4096 2024-04-25T14:15:26.382 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4194304 len 4096 2024-04-25T14:15:26.382 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4194304 len 4096 2024-04-25T14:15:26.382 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4194304 len 8192 2024-04-25T14:15:26.382 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4194304 len 8192 2024-04-25T14:15:26.382 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4194304 len 8192 2024-04-25T14:15:26.382 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4194304 len 8192 2024-04-25T14:15:26.382 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4194304 len 16384 2024-04-25T14:15:26.382 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4194304 len 16384 2024-04-25T14:15:26.382 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4194304 len 16384 2024-04-25T14:15:26.792 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4194304 len 16384 2024-04-25T14:15:26.792 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4194816 len 1024 2024-04-25T14:15:26.792 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4194816 len 1024 2024-04-25T14:15:26.792 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4194816 len 1024 2024-04-25T14:15:26.792 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4194816 len 1024 2024-04-25T14:15:26.792 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4194816 len 2048 2024-04-25T14:15:26.792 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4194816 len 2048 2024-04-25T14:15:26.793 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4194816 len 2048 2024-04-25T14:15:26.793 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4194816 len 2048 2024-04-25T14:15:26.793 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4194816 len 4096 2024-04-25T14:15:26.793 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4194816 len 4096 2024-04-25T14:15:26.793 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4194816 len 4096 2024-04-25T14:15:26.793 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4194816 len 4096 2024-04-25T14:15:26.793 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4194816 len 8192 2024-04-25T14:15:26.793 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4194816 len 8192 2024-04-25T14:15:26.793 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4194816 len 8192 2024-04-25T14:15:26.793 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4194816 len 8192 2024-04-25T14:15:26.793 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4194816 len 16384 2024-04-25T14:15:26.793 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4194816 len 16384 2024-04-25T14:15:26.793 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4194816 len 16384 2024-04-25T14:15:26.793 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4194816 len 16384 2024-04-25T14:15:26.794 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4195328 len 1024 2024-04-25T14:15:26.794 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4195328 len 1024 2024-04-25T14:15:26.794 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4195328 len 1024 2024-04-25T14:15:26.794 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4195328 len 1024 2024-04-25T14:15:26.794 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4195328 len 2048 2024-04-25T14:15:26.794 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4195328 len 2048 2024-04-25T14:15:26.794 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4195328 len 2048 2024-04-25T14:15:26.794 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4195328 len 2048 2024-04-25T14:15:26.794 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4195328 len 4096 2024-04-25T14:15:26.794 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4195328 len 4096 2024-04-25T14:15:26.794 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4195328 len 4096 2024-04-25T14:15:26.794 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4195328 len 4096 2024-04-25T14:15:26.794 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4195328 len 8192 2024-04-25T14:15:26.795 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4195328 len 8192 2024-04-25T14:15:26.795 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4195328 len 8192 2024-04-25T14:15:26.795 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4195328 len 8192 2024-04-25T14:15:26.795 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4195328 len 16384 2024-04-25T14:15:26.795 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4195328 len 16384 2024-04-25T14:15:26.795 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4195328 len 16384 2024-04-25T14:15:26.795 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4195328 len 16384 2024-04-25T14:15:26.795 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4195840 len 1024 2024-04-25T14:15:26.795 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4195840 len 1024 2024-04-25T14:15:26.795 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4195840 len 1024 2024-04-25T14:15:26.795 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4195840 len 1024 2024-04-25T14:15:26.795 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4195840 len 2048 2024-04-25T14:15:26.795 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4195840 len 2048 2024-04-25T14:15:26.795 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4195840 len 2048 2024-04-25T14:15:26.795 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4195840 len 2048 2024-04-25T14:15:26.796 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4195840 len 4096 2024-04-25T14:15:26.796 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4195840 len 4096 2024-04-25T14:15:26.796 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4195840 len 4096 2024-04-25T14:15:26.796 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4195840 len 4096 2024-04-25T14:15:26.796 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4195840 len 8192 2024-04-25T14:15:26.796 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4195840 len 8192 2024-04-25T14:15:26.796 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4195840 len 8192 2024-04-25T14:15:26.796 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4195840 len 8192 2024-04-25T14:15:26.796 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4195840 len 16384 2024-04-25T14:15:26.796 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4195840 len 16384 2024-04-25T14:15:26.796 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4195840 len 16384 2024-04-25T14:15:26.796 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4195840 len 16384 2024-04-25T14:15:26.796 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4196352 len 1024 2024-04-25T14:15:26.796 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4196352 len 1024 2024-04-25T14:15:26.796 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4196352 len 1024 2024-04-25T14:15:26.797 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4196352 len 1024 2024-04-25T14:15:26.797 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4196352 len 2048 2024-04-25T14:15:26.797 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4196352 len 2048 2024-04-25T14:15:26.797 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4196352 len 2048 2024-04-25T14:15:26.797 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4196352 len 2048 2024-04-25T14:15:26.797 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4196352 len 4096 2024-04-25T14:15:26.797 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4196352 len 4096 2024-04-25T14:15:26.797 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4196352 len 4096 2024-04-25T14:15:26.797 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4196352 len 4096 2024-04-25T14:15:26.797 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4196352 len 8192 2024-04-25T14:15:26.797 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4196352 len 8192 2024-04-25T14:15:26.797 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4196352 len 8192 2024-04-25T14:15:26.797 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4196352 len 8192 2024-04-25T14:15:26.797 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4196352 len 16384 2024-04-25T14:15:26.798 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4196352 len 16384 2024-04-25T14:15:26.798 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4196352 len 16384 2024-04-25T14:15:26.798 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4196352 len 16384 2024-04-25T14:15:26.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:26 smithi096 ceph-mon[29207]: pgmap v400: 97 pgs: 97 active+clean; 154 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 87 MiB/s rd, 1.3 MiB/s wr, 1.00k op/s 2024-04-25T14:15:26.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:26 smithi097 ceph-mon[32720]: pgmap v400: 97 pgs: 97 active+clean; 154 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 87 MiB/s rd, 1.3 MiB/s wr, 1.00k op/s 2024-04-25T14:15:26.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:26 smithi151 ceph-mon[32990]: pgmap v400: 97 pgs: 97 active+clean; 154 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 87 MiB/s rd, 1.3 MiB/s wr, 1.00k op/s 2024-04-25T14:15:27.264 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4196864 len 1024 2024-04-25T14:15:27.264 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4196864 len 1024 2024-04-25T14:15:27.264 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4196864 len 1024 2024-04-25T14:15:27.265 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4196864 len 1024 2024-04-25T14:15:27.265 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4196864 len 2048 2024-04-25T14:15:27.265 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4196864 len 2048 2024-04-25T14:15:27.265 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4196864 len 2048 2024-04-25T14:15:27.265 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4196864 len 2048 2024-04-25T14:15:27.265 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4196864 len 4096 2024-04-25T14:15:27.265 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4196864 len 4096 2024-04-25T14:15:27.265 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4196864 len 4096 2024-04-25T14:15:27.266 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4196864 len 4096 2024-04-25T14:15:27.266 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4196864 len 8192 2024-04-25T14:15:27.266 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4196864 len 8192 2024-04-25T14:15:27.266 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4196864 len 8192 2024-04-25T14:15:27.266 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4196864 len 8192 2024-04-25T14:15:27.266 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4196864 len 16384 2024-04-25T14:15:27.266 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4196864 len 16384 2024-04-25T14:15:27.266 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4196864 len 16384 2024-04-25T14:15:27.266 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4196864 len 16384 2024-04-25T14:15:27.266 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4197376 len 1024 2024-04-25T14:15:27.266 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4197376 len 1024 2024-04-25T14:15:27.266 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4197376 len 1024 2024-04-25T14:15:27.266 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4197376 len 1024 2024-04-25T14:15:27.266 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4197376 len 2048 2024-04-25T14:15:27.266 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4197376 len 2048 2024-04-25T14:15:27.267 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4197376 len 2048 2024-04-25T14:15:27.267 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4197376 len 2048 2024-04-25T14:15:27.267 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4197376 len 4096 2024-04-25T14:15:27.267 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4197376 len 4096 2024-04-25T14:15:27.267 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4197376 len 4096 2024-04-25T14:15:27.267 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4197376 len 4096 2024-04-25T14:15:27.267 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4197376 len 8192 2024-04-25T14:15:27.267 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4197376 len 8192 2024-04-25T14:15:27.267 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4197376 len 8192 2024-04-25T14:15:27.267 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4197376 len 8192 2024-04-25T14:15:27.267 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4197376 len 16384 2024-04-25T14:15:27.267 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4197376 len 16384 2024-04-25T14:15:27.267 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4197376 len 16384 2024-04-25T14:15:27.267 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4197376 len 16384 2024-04-25T14:15:27.268 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4197888 len 1024 2024-04-25T14:15:27.268 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4197888 len 1024 2024-04-25T14:15:27.268 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4197888 len 1024 2024-04-25T14:15:27.268 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4197888 len 1024 2024-04-25T14:15:27.268 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4197888 len 2048 2024-04-25T14:15:27.268 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4197888 len 2048 2024-04-25T14:15:27.268 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4197888 len 2048 2024-04-25T14:15:27.268 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4197888 len 2048 2024-04-25T14:15:27.268 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4197888 len 4096 2024-04-25T14:15:27.268 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4197888 len 4096 2024-04-25T14:15:27.269 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4197888 len 4096 2024-04-25T14:15:27.269 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4197888 len 4096 2024-04-25T14:15:27.269 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4197888 len 8192 2024-04-25T14:15:27.269 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4197888 len 8192 2024-04-25T14:15:27.269 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4197888 len 8192 2024-04-25T14:15:27.269 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4197888 len 8192 2024-04-25T14:15:27.269 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 2560 offset 4197888 len 16384 2024-04-25T14:15:27.269 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4197888 len 16384 2024-04-25T14:15:27.269 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 2560 offset 4197888 len 16384 2024-04-25T14:15:27.269 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 2560 offset 4197888 len 16384 2024-04-25T14:15:27.269 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4190720 len 1024 2024-04-25T14:15:27.269 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4190720 len 1024 2024-04-25T14:15:27.269 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4190720 len 1024 2024-04-25T14:15:27.269 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4190720 len 1024 2024-04-25T14:15:27.270 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4190720 len 2048 2024-04-25T14:15:27.270 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4190720 len 2048 2024-04-25T14:15:27.270 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4190720 len 2048 2024-04-25T14:15:27.270 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4190720 len 2048 2024-04-25T14:15:27.270 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4190720 len 4096 2024-04-25T14:15:27.270 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4190720 len 4096 2024-04-25T14:15:27.270 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4190720 len 4096 2024-04-25T14:15:27.270 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4190720 len 4096 2024-04-25T14:15:27.270 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4190720 len 8192 2024-04-25T14:15:27.270 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4190720 len 8192 2024-04-25T14:15:27.270 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4190720 len 8192 2024-04-25T14:15:27.270 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4190720 len 8192 2024-04-25T14:15:27.270 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4190720 len 16384 2024-04-25T14:15:27.270 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4190720 len 16384 2024-04-25T14:15:27.271 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4190720 len 16384 2024-04-25T14:15:27.271 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4190720 len 16384 2024-04-25T14:15:27.924 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4191232 len 1024 2024-04-25T14:15:27.924 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4191232 len 1024 2024-04-25T14:15:27.925 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4191232 len 1024 2024-04-25T14:15:27.925 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4191232 len 1024 2024-04-25T14:15:27.925 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4191232 len 2048 2024-04-25T14:15:27.925 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4191232 len 2048 2024-04-25T14:15:27.925 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4191232 len 2048 2024-04-25T14:15:27.925 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4191232 len 2048 2024-04-25T14:15:27.925 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4191232 len 4096 2024-04-25T14:15:27.925 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4191232 len 4096 2024-04-25T14:15:27.925 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4191232 len 4096 2024-04-25T14:15:27.925 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4191232 len 4096 2024-04-25T14:15:27.925 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4191232 len 8192 2024-04-25T14:15:27.926 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4191232 len 8192 2024-04-25T14:15:27.926 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4191232 len 8192 2024-04-25T14:15:27.926 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4191232 len 8192 2024-04-25T14:15:27.926 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4191232 len 16384 2024-04-25T14:15:27.926 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4191232 len 16384 2024-04-25T14:15:27.926 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4191232 len 16384 2024-04-25T14:15:27.926 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4191232 len 16384 2024-04-25T14:15:27.926 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4191744 len 1024 2024-04-25T14:15:27.926 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4191744 len 1024 2024-04-25T14:15:27.926 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4191744 len 1024 2024-04-25T14:15:27.926 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4191744 len 1024 2024-04-25T14:15:27.926 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4191744 len 2048 2024-04-25T14:15:27.926 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4191744 len 2048 2024-04-25T14:15:27.926 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4191744 len 2048 2024-04-25T14:15:27.927 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4191744 len 2048 2024-04-25T14:15:27.927 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4191744 len 4096 2024-04-25T14:15:27.927 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4191744 len 4096 2024-04-25T14:15:27.927 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4191744 len 4096 2024-04-25T14:15:27.927 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4191744 len 4096 2024-04-25T14:15:27.927 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4191744 len 8192 2024-04-25T14:15:27.927 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4191744 len 8192 2024-04-25T14:15:27.927 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4191744 len 8192 2024-04-25T14:15:27.927 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4191744 len 8192 2024-04-25T14:15:27.927 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4191744 len 16384 2024-04-25T14:15:27.927 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4191744 len 16384 2024-04-25T14:15:27.927 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4191744 len 16384 2024-04-25T14:15:27.928 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4191744 len 16384 2024-04-25T14:15:27.928 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4192256 len 1024 2024-04-25T14:15:27.928 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4192256 len 1024 2024-04-25T14:15:27.928 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4192256 len 1024 2024-04-25T14:15:27.928 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4192256 len 1024 2024-04-25T14:15:27.928 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4192256 len 2048 2024-04-25T14:15:27.928 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4192256 len 2048 2024-04-25T14:15:27.928 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4192256 len 2048 2024-04-25T14:15:27.928 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4192256 len 2048 2024-04-25T14:15:27.928 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4192256 len 4096 2024-04-25T14:15:27.928 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4192256 len 4096 2024-04-25T14:15:27.928 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4192256 len 4096 2024-04-25T14:15:27.928 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4192256 len 4096 2024-04-25T14:15:27.929 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4192256 len 8192 2024-04-25T14:15:27.929 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4192256 len 8192 2024-04-25T14:15:27.929 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4192256 len 8192 2024-04-25T14:15:27.929 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4192256 len 8192 2024-04-25T14:15:27.929 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4192256 len 16384 2024-04-25T14:15:27.929 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4192256 len 16384 2024-04-25T14:15:27.929 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4192256 len 16384 2024-04-25T14:15:27.929 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4192256 len 16384 2024-04-25T14:15:27.929 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4192768 len 1024 2024-04-25T14:15:27.929 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4192768 len 1024 2024-04-25T14:15:27.929 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4192768 len 1024 2024-04-25T14:15:27.929 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4192768 len 1024 2024-04-25T14:15:27.929 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4192768 len 2048 2024-04-25T14:15:27.929 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4192768 len 2048 2024-04-25T14:15:27.929 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4192768 len 2048 2024-04-25T14:15:27.930 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4192768 len 2048 2024-04-25T14:15:27.930 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4192768 len 4096 2024-04-25T14:15:27.930 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4192768 len 4096 2024-04-25T14:15:27.930 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4192768 len 4096 2024-04-25T14:15:27.930 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4192768 len 4096 2024-04-25T14:15:27.930 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4192768 len 8192 2024-04-25T14:15:27.930 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4192768 len 8192 2024-04-25T14:15:27.930 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4192768 len 8192 2024-04-25T14:15:27.930 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4192768 len 8192 2024-04-25T14:15:27.930 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4192768 len 16384 2024-04-25T14:15:27.930 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4192768 len 16384 2024-04-25T14:15:27.930 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4192768 len 16384 2024-04-25T14:15:27.931 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4192768 len 16384 2024-04-25T14:15:27.931 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4193280 len 1024 2024-04-25T14:15:28.453 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4193280 len 1024 2024-04-25T14:15:28.453 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4193280 len 1024 2024-04-25T14:15:28.453 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4193280 len 1024 2024-04-25T14:15:28.454 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4193280 len 2048 2024-04-25T14:15:28.454 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4193280 len 2048 2024-04-25T14:15:28.454 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4193280 len 2048 2024-04-25T14:15:28.454 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4193280 len 2048 2024-04-25T14:15:28.454 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4193280 len 4096 2024-04-25T14:15:28.454 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4193280 len 4096 2024-04-25T14:15:28.454 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4193280 len 4096 2024-04-25T14:15:28.454 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4193280 len 4096 2024-04-25T14:15:28.454 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4193280 len 8192 2024-04-25T14:15:28.454 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4193280 len 8192 2024-04-25T14:15:28.454 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4193280 len 8192 2024-04-25T14:15:28.454 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4193280 len 8192 2024-04-25T14:15:28.454 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4193280 len 16384 2024-04-25T14:15:28.454 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4193280 len 16384 2024-04-25T14:15:28.455 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4193280 len 16384 2024-04-25T14:15:28.455 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4193280 len 16384 2024-04-25T14:15:28.455 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4193792 len 1024 2024-04-25T14:15:28.455 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4193792 len 1024 2024-04-25T14:15:28.455 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4193792 len 1024 2024-04-25T14:15:28.455 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4193792 len 1024 2024-04-25T14:15:28.455 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4193792 len 2048 2024-04-25T14:15:28.455 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4193792 len 2048 2024-04-25T14:15:28.455 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4193792 len 2048 2024-04-25T14:15:28.455 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4193792 len 2048 2024-04-25T14:15:28.455 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4193792 len 4096 2024-04-25T14:15:28.455 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4193792 len 4096 2024-04-25T14:15:28.455 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4193792 len 4096 2024-04-25T14:15:28.455 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4193792 len 4096 2024-04-25T14:15:28.456 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4193792 len 8192 2024-04-25T14:15:28.456 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4193792 len 8192 2024-04-25T14:15:28.456 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4193792 len 8192 2024-04-25T14:15:28.456 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4193792 len 8192 2024-04-25T14:15:28.456 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4193792 len 16384 2024-04-25T14:15:28.456 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4193792 len 16384 2024-04-25T14:15:28.456 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4193792 len 16384 2024-04-25T14:15:28.456 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4193792 len 16384 2024-04-25T14:15:28.456 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4194304 len 1024 2024-04-25T14:15:28.456 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4194304 len 1024 2024-04-25T14:15:28.456 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4194304 len 1024 2024-04-25T14:15:28.456 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4194304 len 1024 2024-04-25T14:15:28.456 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4194304 len 2048 2024-04-25T14:15:28.456 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4194304 len 2048 2024-04-25T14:15:28.456 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4194304 len 2048 2024-04-25T14:15:28.457 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4194304 len 2048 2024-04-25T14:15:28.457 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4194304 len 4096 2024-04-25T14:15:28.457 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4194304 len 4096 2024-04-25T14:15:28.457 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4194304 len 4096 2024-04-25T14:15:28.457 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4194304 len 4096 2024-04-25T14:15:28.457 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4194304 len 8192 2024-04-25T14:15:28.457 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4194304 len 8192 2024-04-25T14:15:28.457 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4194304 len 8192 2024-04-25T14:15:28.457 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4194304 len 8192 2024-04-25T14:15:28.457 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4194304 len 16384 2024-04-25T14:15:28.457 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4194304 len 16384 2024-04-25T14:15:28.457 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4194304 len 16384 2024-04-25T14:15:28.457 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4194304 len 16384 2024-04-25T14:15:28.457 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4194816 len 1024 2024-04-25T14:15:28.457 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4194816 len 1024 2024-04-25T14:15:28.457 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4194816 len 1024 2024-04-25T14:15:28.458 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4194816 len 1024 2024-04-25T14:15:28.458 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4194816 len 2048 2024-04-25T14:15:28.458 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4194816 len 2048 2024-04-25T14:15:28.458 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4194816 len 2048 2024-04-25T14:15:28.458 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4194816 len 2048 2024-04-25T14:15:28.458 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4194816 len 4096 2024-04-25T14:15:28.458 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4194816 len 4096 2024-04-25T14:15:28.458 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4194816 len 4096 2024-04-25T14:15:28.458 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4194816 len 4096 2024-04-25T14:15:28.458 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4194816 len 8192 2024-04-25T14:15:28.458 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4194816 len 8192 2024-04-25T14:15:28.458 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4194816 len 8192 2024-04-25T14:15:28.458 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4194816 len 8192 2024-04-25T14:15:28.459 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4194816 len 16384 2024-04-25T14:15:28.459 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4194816 len 16384 2024-04-25T14:15:28.459 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4194816 len 16384 2024-04-25T14:15:28.459 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4194816 len 16384 2024-04-25T14:15:28.459 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4195328 len 1024 2024-04-25T14:15:28.459 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4195328 len 1024 2024-04-25T14:15:28.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:28 smithi096 ceph-mon[29207]: pgmap v401: 97 pgs: 97 active+clean; 226 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 128 MiB/s rd, 1.8 MiB/s wr, 1.18k op/s 2024-04-25T14:15:28.899 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4195328 len 1024 2024-04-25T14:15:28.899 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4195328 len 1024 2024-04-25T14:15:28.899 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4195328 len 2048 2024-04-25T14:15:28.899 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4195328 len 2048 2024-04-25T14:15:28.899 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4195328 len 2048 2024-04-25T14:15:28.899 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4195328 len 2048 2024-04-25T14:15:28.899 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4195328 len 4096 2024-04-25T14:15:28.899 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4195328 len 4096 2024-04-25T14:15:28.899 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4195328 len 4096 2024-04-25T14:15:28.899 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4195328 len 4096 2024-04-25T14:15:28.899 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4195328 len 8192 2024-04-25T14:15:28.900 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4195328 len 8192 2024-04-25T14:15:28.900 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4195328 len 8192 2024-04-25T14:15:28.900 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4195328 len 8192 2024-04-25T14:15:28.900 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4195328 len 16384 2024-04-25T14:15:28.900 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4195328 len 16384 2024-04-25T14:15:28.900 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4195328 len 16384 2024-04-25T14:15:28.900 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4195328 len 16384 2024-04-25T14:15:28.900 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4195840 len 1024 2024-04-25T14:15:28.900 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4195840 len 1024 2024-04-25T14:15:28.900 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4195840 len 1024 2024-04-25T14:15:28.900 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4195840 len 1024 2024-04-25T14:15:28.900 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4195840 len 2048 2024-04-25T14:15:28.900 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4195840 len 2048 2024-04-25T14:15:28.900 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4195840 len 2048 2024-04-25T14:15:28.900 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4195840 len 2048 2024-04-25T14:15:28.901 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4195840 len 4096 2024-04-25T14:15:28.901 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4195840 len 4096 2024-04-25T14:15:28.901 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4195840 len 4096 2024-04-25T14:15:28.901 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4195840 len 4096 2024-04-25T14:15:28.901 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4195840 len 8192 2024-04-25T14:15:28.901 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4195840 len 8192 2024-04-25T14:15:28.901 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4195840 len 8192 2024-04-25T14:15:28.901 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4195840 len 8192 2024-04-25T14:15:28.901 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4195840 len 16384 2024-04-25T14:15:28.901 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4195840 len 16384 2024-04-25T14:15:28.901 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4195840 len 16384 2024-04-25T14:15:28.901 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4195840 len 16384 2024-04-25T14:15:28.901 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4196352 len 1024 2024-04-25T14:15:28.902 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4196352 len 1024 2024-04-25T14:15:28.902 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4196352 len 1024 2024-04-25T14:15:28.902 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4196352 len 1024 2024-04-25T14:15:28.902 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4196352 len 2048 2024-04-25T14:15:28.902 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4196352 len 2048 2024-04-25T14:15:28.902 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4196352 len 2048 2024-04-25T14:15:28.902 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4196352 len 2048 2024-04-25T14:15:28.902 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4196352 len 4096 2024-04-25T14:15:28.902 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4196352 len 4096 2024-04-25T14:15:28.902 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4196352 len 4096 2024-04-25T14:15:28.902 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4196352 len 4096 2024-04-25T14:15:28.902 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4196352 len 8192 2024-04-25T14:15:28.902 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4196352 len 8192 2024-04-25T14:15:28.902 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4196352 len 8192 2024-04-25T14:15:28.903 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4196352 len 8192 2024-04-25T14:15:28.903 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4196352 len 16384 2024-04-25T14:15:28.903 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4196352 len 16384 2024-04-25T14:15:28.903 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4196352 len 16384 2024-04-25T14:15:28.903 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4196352 len 16384 2024-04-25T14:15:28.903 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4196864 len 1024 2024-04-25T14:15:28.903 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4196864 len 1024 2024-04-25T14:15:28.903 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4196864 len 1024 2024-04-25T14:15:28.903 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4196864 len 1024 2024-04-25T14:15:28.903 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4196864 len 2048 2024-04-25T14:15:28.903 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4196864 len 2048 2024-04-25T14:15:28.903 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4196864 len 2048 2024-04-25T14:15:28.903 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4196864 len 2048 2024-04-25T14:15:28.903 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4196864 len 4096 2024-04-25T14:15:28.904 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4196864 len 4096 2024-04-25T14:15:28.904 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4196864 len 4096 2024-04-25T14:15:28.904 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4196864 len 4096 2024-04-25T14:15:28.904 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4196864 len 8192 2024-04-25T14:15:28.904 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4196864 len 8192 2024-04-25T14:15:28.904 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4196864 len 8192 2024-04-25T14:15:28.904 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4196864 len 8192 2024-04-25T14:15:28.904 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4196864 len 16384 2024-04-25T14:15:28.904 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4196864 len 16384 2024-04-25T14:15:28.904 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4196864 len 16384 2024-04-25T14:15:28.904 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4196864 len 16384 2024-04-25T14:15:28.904 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4197376 len 1024 2024-04-25T14:15:28.904 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4197376 len 1024 2024-04-25T14:15:28.904 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4197376 len 1024 2024-04-25T14:15:28.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:28 smithi097 ceph-mon[32720]: pgmap v401: 97 pgs: 97 active+clean; 226 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 128 MiB/s rd, 1.8 MiB/s wr, 1.18k op/s 2024-04-25T14:15:28.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:28 smithi151 ceph-mon[32990]: pgmap v401: 97 pgs: 97 active+clean; 226 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 128 MiB/s rd, 1.8 MiB/s wr, 1.18k op/s 2024-04-25T14:15:29.439 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4197376 len 1024 2024-04-25T14:15:29.439 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4197376 len 2048 2024-04-25T14:15:29.439 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4197376 len 2048 2024-04-25T14:15:29.439 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4197376 len 2048 2024-04-25T14:15:29.439 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4197376 len 2048 2024-04-25T14:15:29.439 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4197376 len 4096 2024-04-25T14:15:29.439 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4197376 len 4096 2024-04-25T14:15:29.439 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4197376 len 4096 2024-04-25T14:15:29.439 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4197376 len 4096 2024-04-25T14:15:29.439 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4197376 len 8192 2024-04-25T14:15:29.439 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4197376 len 8192 2024-04-25T14:15:29.440 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4197376 len 8192 2024-04-25T14:15:29.440 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4197376 len 8192 2024-04-25T14:15:29.440 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4197376 len 16384 2024-04-25T14:15:29.440 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4197376 len 16384 2024-04-25T14:15:29.440 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4197376 len 16384 2024-04-25T14:15:29.440 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4197376 len 16384 2024-04-25T14:15:29.440 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4197888 len 1024 2024-04-25T14:15:29.440 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4197888 len 1024 2024-04-25T14:15:29.440 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4197888 len 1024 2024-04-25T14:15:29.440 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4197888 len 1024 2024-04-25T14:15:29.440 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4197888 len 2048 2024-04-25T14:15:29.440 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4197888 len 2048 2024-04-25T14:15:29.440 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4197888 len 2048 2024-04-25T14:15:29.440 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4197888 len 2048 2024-04-25T14:15:29.441 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4197888 len 4096 2024-04-25T14:15:29.441 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4197888 len 4096 2024-04-25T14:15:29.441 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4197888 len 4096 2024-04-25T14:15:29.441 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4197888 len 4096 2024-04-25T14:15:29.441 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4197888 len 8192 2024-04-25T14:15:29.441 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4197888 len 8192 2024-04-25T14:15:29.441 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4197888 len 8192 2024-04-25T14:15:29.441 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4197888 len 8192 2024-04-25T14:15:29.441 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3072 offset 4197888 len 16384 2024-04-25T14:15:29.441 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4197888 len 16384 2024-04-25T14:15:29.441 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3072 offset 4197888 len 16384 2024-04-25T14:15:29.441 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3072 offset 4197888 len 16384 2024-04-25T14:15:29.441 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4190720 len 1024 2024-04-25T14:15:29.441 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4190720 len 1024 2024-04-25T14:15:29.442 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4190720 len 1024 2024-04-25T14:15:29.442 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4190720 len 1024 2024-04-25T14:15:29.442 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4190720 len 2048 2024-04-25T14:15:29.442 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4190720 len 2048 2024-04-25T14:15:29.442 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4190720 len 2048 2024-04-25T14:15:29.442 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4190720 len 2048 2024-04-25T14:15:29.442 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4190720 len 4096 2024-04-25T14:15:29.442 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4190720 len 4096 2024-04-25T14:15:29.442 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4190720 len 4096 2024-04-25T14:15:29.442 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4190720 len 4096 2024-04-25T14:15:29.442 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4190720 len 8192 2024-04-25T14:15:29.442 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4190720 len 8192 2024-04-25T14:15:29.442 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4190720 len 8192 2024-04-25T14:15:29.442 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4190720 len 8192 2024-04-25T14:15:29.443 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4190720 len 16384 2024-04-25T14:15:29.443 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4190720 len 16384 2024-04-25T14:15:29.443 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4190720 len 16384 2024-04-25T14:15:29.443 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4190720 len 16384 2024-04-25T14:15:29.443 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4191232 len 1024 2024-04-25T14:15:29.443 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4191232 len 1024 2024-04-25T14:15:29.443 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4191232 len 1024 2024-04-25T14:15:29.443 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4191232 len 1024 2024-04-25T14:15:29.443 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4191232 len 2048 2024-04-25T14:15:29.443 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4191232 len 2048 2024-04-25T14:15:29.443 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4191232 len 2048 2024-04-25T14:15:29.443 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4191232 len 2048 2024-04-25T14:15:29.443 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4191232 len 4096 2024-04-25T14:15:29.443 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4191232 len 4096 2024-04-25T14:15:29.443 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4191232 len 4096 2024-04-25T14:15:29.444 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4191232 len 4096 2024-04-25T14:15:29.444 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4191232 len 8192 2024-04-25T14:15:29.444 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4191232 len 8192 2024-04-25T14:15:29.444 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4191232 len 8192 2024-04-25T14:15:29.444 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4191232 len 8192 2024-04-25T14:15:29.444 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4191232 len 16384 2024-04-25T14:15:29.444 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4191232 len 16384 2024-04-25T14:15:29.444 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4191232 len 16384 2024-04-25T14:15:29.444 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4191232 len 16384 2024-04-25T14:15:29.444 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4191744 len 1024 2024-04-25T14:15:29.444 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4191744 len 1024 2024-04-25T14:15:29.444 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4191744 len 1024 2024-04-25T14:15:29.444 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4191744 len 1024 2024-04-25T14:15:30.092 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4191744 len 2048 2024-04-25T14:15:30.093 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4191744 len 2048 2024-04-25T14:15:30.093 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4191744 len 2048 2024-04-25T14:15:30.093 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4191744 len 2048 2024-04-25T14:15:30.093 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4191744 len 4096 2024-04-25T14:15:30.093 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4191744 len 4096 2024-04-25T14:15:30.093 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4191744 len 4096 2024-04-25T14:15:30.093 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4191744 len 4096 2024-04-25T14:15:30.093 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4191744 len 8192 2024-04-25T14:15:30.093 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4191744 len 8192 2024-04-25T14:15:30.093 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4191744 len 8192 2024-04-25T14:15:30.093 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4191744 len 8192 2024-04-25T14:15:30.093 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4191744 len 16384 2024-04-25T14:15:30.093 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4191744 len 16384 2024-04-25T14:15:30.094 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4191744 len 16384 2024-04-25T14:15:30.094 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4191744 len 16384 2024-04-25T14:15:30.094 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4192256 len 1024 2024-04-25T14:15:30.094 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4192256 len 1024 2024-04-25T14:15:30.094 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4192256 len 1024 2024-04-25T14:15:30.094 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4192256 len 1024 2024-04-25T14:15:30.094 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4192256 len 2048 2024-04-25T14:15:30.094 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4192256 len 2048 2024-04-25T14:15:30.094 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4192256 len 2048 2024-04-25T14:15:30.094 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4192256 len 2048 2024-04-25T14:15:30.094 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4192256 len 4096 2024-04-25T14:15:30.094 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4192256 len 4096 2024-04-25T14:15:30.094 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4192256 len 4096 2024-04-25T14:15:30.094 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4192256 len 4096 2024-04-25T14:15:30.094 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4192256 len 8192 2024-04-25T14:15:30.095 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4192256 len 8192 2024-04-25T14:15:30.095 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4192256 len 8192 2024-04-25T14:15:30.095 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4192256 len 8192 2024-04-25T14:15:30.095 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4192256 len 16384 2024-04-25T14:15:30.095 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4192256 len 16384 2024-04-25T14:15:30.095 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4192256 len 16384 2024-04-25T14:15:30.095 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4192256 len 16384 2024-04-25T14:15:30.095 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4192768 len 1024 2024-04-25T14:15:30.095 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4192768 len 1024 2024-04-25T14:15:30.095 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4192768 len 1024 2024-04-25T14:15:30.095 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4192768 len 1024 2024-04-25T14:15:30.095 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4192768 len 2048 2024-04-25T14:15:30.095 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4192768 len 2048 2024-04-25T14:15:30.095 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4192768 len 2048 2024-04-25T14:15:30.096 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4192768 len 2048 2024-04-25T14:15:30.096 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4192768 len 4096 2024-04-25T14:15:30.096 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4192768 len 4096 2024-04-25T14:15:30.096 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4192768 len 4096 2024-04-25T14:15:30.096 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4192768 len 4096 2024-04-25T14:15:30.096 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4192768 len 8192 2024-04-25T14:15:30.096 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4192768 len 8192 2024-04-25T14:15:30.096 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4192768 len 8192 2024-04-25T14:15:30.096 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4192768 len 8192 2024-04-25T14:15:30.096 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4192768 len 16384 2024-04-25T14:15:30.096 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4192768 len 16384 2024-04-25T14:15:30.096 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4192768 len 16384 2024-04-25T14:15:30.096 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4192768 len 16384 2024-04-25T14:15:30.096 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4193280 len 1024 2024-04-25T14:15:30.096 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4193280 len 1024 2024-04-25T14:15:30.097 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4193280 len 1024 2024-04-25T14:15:30.097 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4193280 len 1024 2024-04-25T14:15:30.097 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4193280 len 2048 2024-04-25T14:15:30.097 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4193280 len 2048 2024-04-25T14:15:30.097 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4193280 len 2048 2024-04-25T14:15:30.097 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4193280 len 2048 2024-04-25T14:15:30.097 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4193280 len 4096 2024-04-25T14:15:30.097 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4193280 len 4096 2024-04-25T14:15:30.097 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4193280 len 4096 2024-04-25T14:15:30.097 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4193280 len 4096 2024-04-25T14:15:30.097 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4193280 len 8192 2024-04-25T14:15:30.097 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4193280 len 8192 2024-04-25T14:15:30.097 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4193280 len 8192 2024-04-25T14:15:30.098 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4193280 len 8192 2024-04-25T14:15:30.098 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4193280 len 16384 2024-04-25T14:15:30.098 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4193280 len 16384 2024-04-25T14:15:30.098 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4193280 len 16384 2024-04-25T14:15:30.098 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4193280 len 16384 2024-04-25T14:15:30.098 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4193792 len 1024 2024-04-25T14:15:30.098 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4193792 len 1024 2024-04-25T14:15:30.098 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4193792 len 1024 2024-04-25T14:15:30.098 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4193792 len 1024 2024-04-25T14:15:30.522 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4193792 len 2048 2024-04-25T14:15:30.522 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4193792 len 2048 2024-04-25T14:15:30.522 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4193792 len 2048 2024-04-25T14:15:30.522 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4193792 len 2048 2024-04-25T14:15:30.522 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4193792 len 4096 2024-04-25T14:15:30.522 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4193792 len 4096 2024-04-25T14:15:30.522 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4193792 len 4096 2024-04-25T14:15:30.523 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4193792 len 4096 2024-04-25T14:15:30.523 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4193792 len 8192 2024-04-25T14:15:30.523 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4193792 len 8192 2024-04-25T14:15:30.523 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4193792 len 8192 2024-04-25T14:15:30.523 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4193792 len 8192 2024-04-25T14:15:30.523 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4193792 len 16384 2024-04-25T14:15:30.523 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4193792 len 16384 2024-04-25T14:15:30.523 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4193792 len 16384 2024-04-25T14:15:30.523 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4193792 len 16384 2024-04-25T14:15:30.523 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4194304 len 1024 2024-04-25T14:15:30.523 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4194304 len 1024 2024-04-25T14:15:30.523 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4194304 len 1024 2024-04-25T14:15:30.523 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4194304 len 1024 2024-04-25T14:15:30.523 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4194304 len 2048 2024-04-25T14:15:30.524 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4194304 len 2048 2024-04-25T14:15:30.524 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4194304 len 2048 2024-04-25T14:15:30.524 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4194304 len 2048 2024-04-25T14:15:30.524 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4194304 len 4096 2024-04-25T14:15:30.524 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4194304 len 4096 2024-04-25T14:15:30.524 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4194304 len 4096 2024-04-25T14:15:30.524 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4194304 len 4096 2024-04-25T14:15:30.524 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4194304 len 8192 2024-04-25T14:15:30.524 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4194304 len 8192 2024-04-25T14:15:30.524 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4194304 len 8192 2024-04-25T14:15:30.524 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4194304 len 8192 2024-04-25T14:15:30.524 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4194304 len 16384 2024-04-25T14:15:30.524 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4194304 len 16384 2024-04-25T14:15:30.524 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4194304 len 16384 2024-04-25T14:15:30.525 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4194304 len 16384 2024-04-25T14:15:30.525 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4194816 len 1024 2024-04-25T14:15:30.525 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4194816 len 1024 2024-04-25T14:15:30.525 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4194816 len 1024 2024-04-25T14:15:30.525 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4194816 len 1024 2024-04-25T14:15:30.525 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4194816 len 2048 2024-04-25T14:15:30.525 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4194816 len 2048 2024-04-25T14:15:30.525 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4194816 len 2048 2024-04-25T14:15:30.525 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4194816 len 2048 2024-04-25T14:15:30.525 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4194816 len 4096 2024-04-25T14:15:30.525 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4194816 len 4096 2024-04-25T14:15:30.525 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4194816 len 4096 2024-04-25T14:15:30.525 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4194816 len 4096 2024-04-25T14:15:30.525 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4194816 len 8192 2024-04-25T14:15:30.526 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4194816 len 8192 2024-04-25T14:15:30.526 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4194816 len 8192 2024-04-25T14:15:30.526 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4194816 len 8192 2024-04-25T14:15:30.526 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4194816 len 16384 2024-04-25T14:15:30.526 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4194816 len 16384 2024-04-25T14:15:30.526 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4194816 len 16384 2024-04-25T14:15:30.526 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4194816 len 16384 2024-04-25T14:15:30.526 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4195328 len 1024 2024-04-25T14:15:30.526 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4195328 len 1024 2024-04-25T14:15:30.526 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4195328 len 1024 2024-04-25T14:15:30.526 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4195328 len 1024 2024-04-25T14:15:30.526 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4195328 len 2048 2024-04-25T14:15:30.526 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4195328 len 2048 2024-04-25T14:15:30.526 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4195328 len 2048 2024-04-25T14:15:30.527 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4195328 len 2048 2024-04-25T14:15:30.527 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4195328 len 4096 2024-04-25T14:15:30.527 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4195328 len 4096 2024-04-25T14:15:30.527 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4195328 len 4096 2024-04-25T14:15:30.527 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4195328 len 4096 2024-04-25T14:15:30.527 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4195328 len 8192 2024-04-25T14:15:30.527 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4195328 len 8192 2024-04-25T14:15:30.527 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4195328 len 8192 2024-04-25T14:15:30.527 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4195328 len 8192 2024-04-25T14:15:30.527 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4195328 len 16384 2024-04-25T14:15:30.527 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4195328 len 16384 2024-04-25T14:15:30.527 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4195328 len 16384 2024-04-25T14:15:30.527 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4195328 len 16384 2024-04-25T14:15:30.527 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4195840 len 1024 2024-04-25T14:15:30.527 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4195840 len 1024 2024-04-25T14:15:30.527 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4195840 len 1024 2024-04-25T14:15:30.528 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4195840 len 1024 2024-04-25T14:15:30.528 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4195840 len 2048 2024-04-25T14:15:30.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:30 smithi096 ceph-mon[29207]: pgmap v402: 97 pgs: 97 active+clean; 215 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 134 MiB/s rd, 1.7 MiB/s wr, 560 op/s 2024-04-25T14:15:30.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:30 smithi097 ceph-mon[32720]: pgmap v402: 97 pgs: 97 active+clean; 215 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 134 MiB/s rd, 1.7 MiB/s wr, 560 op/s 2024-04-25T14:15:30.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:30 smithi151 ceph-mon[32990]: pgmap v402: 97 pgs: 97 active+clean; 215 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 134 MiB/s rd, 1.7 MiB/s wr, 560 op/s 2024-04-25T14:15:30.961 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4195840 len 2048 2024-04-25T14:15:30.961 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4195840 len 2048 2024-04-25T14:15:30.961 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4195840 len 2048 2024-04-25T14:15:30.961 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4195840 len 4096 2024-04-25T14:15:30.961 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4195840 len 4096 2024-04-25T14:15:30.961 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4195840 len 4096 2024-04-25T14:15:30.961 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4195840 len 4096 2024-04-25T14:15:30.961 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4195840 len 8192 2024-04-25T14:15:30.961 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4195840 len 8192 2024-04-25T14:15:30.962 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4195840 len 8192 2024-04-25T14:15:30.962 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4195840 len 8192 2024-04-25T14:15:30.962 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4195840 len 16384 2024-04-25T14:15:30.962 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4195840 len 16384 2024-04-25T14:15:30.962 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4195840 len 16384 2024-04-25T14:15:30.962 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4195840 len 16384 2024-04-25T14:15:30.962 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4196352 len 1024 2024-04-25T14:15:30.962 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4196352 len 1024 2024-04-25T14:15:30.962 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4196352 len 1024 2024-04-25T14:15:30.962 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4196352 len 1024 2024-04-25T14:15:30.962 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4196352 len 2048 2024-04-25T14:15:30.962 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4196352 len 2048 2024-04-25T14:15:30.962 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4196352 len 2048 2024-04-25T14:15:30.962 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4196352 len 2048 2024-04-25T14:15:30.962 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4196352 len 4096 2024-04-25T14:15:30.963 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4196352 len 4096 2024-04-25T14:15:30.963 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4196352 len 4096 2024-04-25T14:15:30.963 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4196352 len 4096 2024-04-25T14:15:30.963 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4196352 len 8192 2024-04-25T14:15:30.963 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4196352 len 8192 2024-04-25T14:15:30.963 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4196352 len 8192 2024-04-25T14:15:30.963 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4196352 len 8192 2024-04-25T14:15:30.963 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4196352 len 16384 2024-04-25T14:15:30.963 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4196352 len 16384 2024-04-25T14:15:30.963 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4196352 len 16384 2024-04-25T14:15:30.963 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4196352 len 16384 2024-04-25T14:15:30.963 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4196864 len 1024 2024-04-25T14:15:30.963 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4196864 len 1024 2024-04-25T14:15:30.963 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4196864 len 1024 2024-04-25T14:15:30.964 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4196864 len 1024 2024-04-25T14:15:30.964 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4196864 len 2048 2024-04-25T14:15:30.964 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4196864 len 2048 2024-04-25T14:15:30.964 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4196864 len 2048 2024-04-25T14:15:30.964 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4196864 len 2048 2024-04-25T14:15:30.964 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4196864 len 4096 2024-04-25T14:15:30.964 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4196864 len 4096 2024-04-25T14:15:30.964 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4196864 len 4096 2024-04-25T14:15:30.964 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4196864 len 4096 2024-04-25T14:15:30.964 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4196864 len 8192 2024-04-25T14:15:30.964 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4196864 len 8192 2024-04-25T14:15:30.964 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4196864 len 8192 2024-04-25T14:15:30.964 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4196864 len 8192 2024-04-25T14:15:30.964 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4196864 len 16384 2024-04-25T14:15:30.965 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4196864 len 16384 2024-04-25T14:15:30.965 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4196864 len 16384 2024-04-25T14:15:30.965 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4196864 len 16384 2024-04-25T14:15:30.965 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4197376 len 1024 2024-04-25T14:15:30.965 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4197376 len 1024 2024-04-25T14:15:30.965 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4197376 len 1024 2024-04-25T14:15:30.965 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4197376 len 1024 2024-04-25T14:15:30.965 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4197376 len 2048 2024-04-25T14:15:30.965 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4197376 len 2048 2024-04-25T14:15:30.965 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4197376 len 2048 2024-04-25T14:15:30.965 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4197376 len 2048 2024-04-25T14:15:30.965 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4197376 len 4096 2024-04-25T14:15:30.965 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4197376 len 4096 2024-04-25T14:15:30.965 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4197376 len 4096 2024-04-25T14:15:30.965 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4197376 len 4096 2024-04-25T14:15:30.966 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4197376 len 8192 2024-04-25T14:15:30.966 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4197376 len 8192 2024-04-25T14:15:30.966 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4197376 len 8192 2024-04-25T14:15:30.966 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4197376 len 8192 2024-04-25T14:15:30.966 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4197376 len 16384 2024-04-25T14:15:30.966 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4197376 len 16384 2024-04-25T14:15:30.966 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4197376 len 16384 2024-04-25T14:15:30.966 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4197376 len 16384 2024-04-25T14:15:30.966 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4197888 len 1024 2024-04-25T14:15:30.966 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4197888 len 1024 2024-04-25T14:15:30.966 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4197888 len 1024 2024-04-25T14:15:30.966 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4197888 len 1024 2024-04-25T14:15:30.966 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4197888 len 2048 2024-04-25T14:15:30.966 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4197888 len 2048 2024-04-25T14:15:31.027 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4197888 len 2048 2024-04-25T14:15:31.027 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4197888 len 2048 2024-04-25T14:15:31.027 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4197888 len 4096 2024-04-25T14:15:31.027 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4197888 len 4096 2024-04-25T14:15:31.027 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4197888 len 4096 2024-04-25T14:15:31.027 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4197888 len 4096 2024-04-25T14:15:31.027 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4197888 len 8192 2024-04-25T14:15:31.027 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4197888 len 8192 2024-04-25T14:15:31.027 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4197888 len 8192 2024-04-25T14:15:31.027 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4197888 len 8192 2024-04-25T14:15:31.027 INFO:tasks.workunit.client.0.smithi096.stdout:write_direct buf_align 3584 offset 4197888 len 16384 2024-04-25T14:15:31.027 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4197888 len 16384 2024-04-25T14:15:31.028 INFO:tasks.workunit.client.0.smithi096.stdout:write_sync buf_align 3584 offset 4197888 len 16384 2024-04-25T14:15:31.028 INFO:tasks.workunit.client.0.smithi096.stdout:write_file buf_align 3584 offset 4197888 len 16384 2024-04-25T14:15:31.028 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-25T14:15:31.028 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-25T14:15:31.056 INFO:tasks.workunit:Stopping ['direct_io'] on client.0... 2024-04-25T14:15:31.056 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-25T14:15:31.362 DEBUG:teuthology.parallel:result is None 2024-04-25T14:15:31.363 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T14:15:31.396 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T14:15:31.396 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-25T14:15:31.407 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi096.front.sepia.ceph.com 2024-04-25T14:15:31.444 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-04-25T14:15:31.454 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-04-25T14:15:32.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:32 smithi096 ceph-mon[29207]: pgmap v403: 97 pgs: 97 active+clean; 199 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 140 MiB/s rd, 1.4 MiB/s wr, 502 op/s 2024-04-25T14:15:32.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:32 smithi097 ceph-mon[32720]: pgmap v403: 97 pgs: 97 active+clean; 199 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 140 MiB/s rd, 1.4 MiB/s wr, 502 op/s 2024-04-25T14:15:32.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:32 smithi151 ceph-mon[32990]: pgmap v403: 97 pgs: 97 active+clean; 199 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 140 MiB/s rd, 1.4 MiB/s wr, 502 op/s 2024-04-25T14:15:34.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:34 smithi096 ceph-mon[29207]: pgmap v404: 97 pgs: 97 active+clean; 169 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 141 MiB/s rd, 1.3 MiB/s wr, 493 op/s 2024-04-25T14:15:34.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:34 smithi097 ceph-mon[32720]: pgmap v404: 97 pgs: 97 active+clean; 169 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 141 MiB/s rd, 1.3 MiB/s wr, 493 op/s 2024-04-25T14:15:34.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:34 smithi151 ceph-mon[32990]: pgmap v404: 97 pgs: 97 active+clean; 169 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 141 MiB/s rd, 1.3 MiB/s wr, 493 op/s 2024-04-25T14:15:36.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:36 smithi096 ceph-mon[29207]: pgmap v405: 97 pgs: 97 active+clean; 129 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 118 MiB/s rd, 1.2 MiB/s wr, 439 op/s 2024-04-25T14:15:36.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:36 smithi097 ceph-mon[32720]: pgmap v405: 97 pgs: 97 active+clean; 129 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 118 MiB/s rd, 1.2 MiB/s wr, 439 op/s 2024-04-25T14:15:36.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:36 smithi151 ceph-mon[32990]: pgmap v405: 97 pgs: 97 active+clean; 129 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 118 MiB/s rd, 1.2 MiB/s wr, 439 op/s 2024-04-25T14:15:38.863 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:38 smithi096 ceph-mon[29207]: pgmap v406: 97 pgs: 97 active+clean; 13 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 104 MiB/s rd, 1.1 MiB/s wr, 425 op/s 2024-04-25T14:15:38.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:38 smithi097 ceph-mon[32720]: pgmap v406: 97 pgs: 97 active+clean; 13 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 104 MiB/s rd, 1.1 MiB/s wr, 425 op/s 2024-04-25T14:15:38.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:38 smithi151 ceph-mon[32990]: pgmap v406: 97 pgs: 97 active+clean; 13 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 104 MiB/s rd, 1.1 MiB/s wr, 425 op/s 2024-04-25T14:15:40.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:40 smithi096 ceph-mon[29207]: pgmap v407: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 64 MiB/s rd, 647 KiB/s wr, 252 op/s 2024-04-25T14:15:40.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:40 smithi097 ceph-mon[32720]: pgmap v407: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 64 MiB/s rd, 647 KiB/s wr, 252 op/s 2024-04-25T14:15:40.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:40 smithi151 ceph-mon[32990]: pgmap v407: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 64 MiB/s rd, 647 KiB/s wr, 252 op/s 2024-04-25T14:15:42.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:42 smithi096 ceph-mon[29207]: pgmap v408: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 44 MiB/s rd, 432 KiB/s wr, 181 op/s 2024-04-25T14:15:42.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:42 smithi097 ceph-mon[32720]: pgmap v408: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 44 MiB/s rd, 432 KiB/s wr, 181 op/s 2024-04-25T14:15:42.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:42 smithi151 ceph-mon[32990]: pgmap v408: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 44 MiB/s rd, 432 KiB/s wr, 181 op/s 2024-04-25T14:15:44.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:44 smithi096 ceph-mon[29207]: pgmap v409: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 351 KiB/s wr, 132 op/s 2024-04-25T14:15:44.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:44 smithi097 ceph-mon[32720]: pgmap v409: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 351 KiB/s wr, 132 op/s 2024-04-25T14:15:44.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:44 smithi151 ceph-mon[32990]: pgmap v409: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 351 KiB/s wr, 132 op/s 2024-04-25T14:15:46.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:46 smithi096 ceph-mon[29207]: pgmap v410: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 29 KiB/s wr, 36 op/s 2024-04-25T14:15:46.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:46 smithi097 ceph-mon[32720]: pgmap v410: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 29 KiB/s wr, 36 op/s 2024-04-25T14:15:46.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:46 smithi151 ceph-mon[32990]: pgmap v410: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 29 KiB/s wr, 36 op/s 2024-04-25T14:15:48.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:48 smithi096 ceph-mon[29207]: pgmap v411: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 KiB/s wr, 26 op/s 2024-04-25T14:15:48.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:48 smithi097 ceph-mon[32720]: pgmap v411: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 KiB/s wr, 26 op/s 2024-04-25T14:15:48.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:48 smithi151 ceph-mon[32990]: pgmap v411: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 KiB/s wr, 26 op/s 2024-04-25T14:15:49.663 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:15:49.664 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:15:49.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.787+0000 7fb5c636c640 1 -- 172.21.15.96:0/2956149844 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5c01080d0 msgr2=0x7fb5c010a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:49.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.787+0000 7fb5c636c640 1 --2- 172.21.15.96:0/2956149844 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5c01080d0 0x7fb5c010a4f0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fb5b800adb0 tx=0x7fb5b802b7a0 comp rx=0 tx=0).stop 2024-04-25T14:15:49.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.787+0000 7fb5c636c640 1 -- 172.21.15.96:0/2956149844 shutdown_connections 2024-04-25T14:15:49.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.787+0000 7fb5c636c640 1 --2- 172.21.15.96:0/2956149844 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5c01080d0 0x7fb5c010a4f0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:49.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.787+0000 7fb5c636c640 1 --2- 172.21.15.96:0/2956149844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5c01033d0 0x7fb5c01037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:49.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.787+0000 7fb5c636c640 1 --2- 172.21.15.96:0/2956149844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5c0102190 0x7fb5c0102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:49.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.787+0000 7fb5c636c640 1 -- 172.21.15.96:0/2956149844 >> 172.21.15.96:0/2956149844 conn(0x7fb5c00fd380 msgr2=0x7fb5c00ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:49.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.787+0000 7fb5c636c640 1 -- 172.21.15.96:0/2956149844 shutdown_connections 2024-04-25T14:15:49.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.787+0000 7fb5c636c640 1 -- 172.21.15.96:0/2956149844 wait complete. 2024-04-25T14:15:49.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.788+0000 7fb5c636c640 1 Processor -- start 2024-04-25T14:15:49.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.788+0000 7fb5c636c640 1 -- start start 2024-04-25T14:15:49.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.788+0000 7fb5c636c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5c0102190 0x7fb5c019a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:49.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.788+0000 7fb5c636c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5c01033d0 0x7fb5c019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:49.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.788+0000 7fb5c636c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5c01080d0 0x7fb5c01a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:49.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.788+0000 7fb5c636c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c010d420 con 0x7fb5c01033d0 2024-04-25T14:15:49.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.788+0000 7fb5c636c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c010d2a0 con 0x7fb5c01080d0 2024-04-25T14:15:49.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.788+0000 7fb5c636c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c010d5a0 con 0x7fb5c0102190 2024-04-25T14:15:49.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.788+0000 7fb5bf7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5c01033d0 0x7fb5c019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:49.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.788+0000 7fb5bf7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5c01033d0 0x7fb5c019a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50836/0 (socket says 172.21.15.96:50836) 2024-04-25T14:15:49.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.788+0000 7fb5bf7fe640 1 -- 172.21.15.96:0/1715378334 learned_addr learned my addr 172.21.15.96:0/1715378334 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:49.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.789+0000 7fb5bffff640 1 --2- 172.21.15.96:0/1715378334 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5c0102190 0x7fb5c019a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:49.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.789+0000 7fb5c4909640 1 --2- 172.21.15.96:0/1715378334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5c01080d0 0x7fb5c01a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:49.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.789+0000 7fb5bf7fe640 1 -- 172.21.15.96:0/1715378334 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5c0102190 msgr2=0x7fb5c019a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:49.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.789+0000 7fb5bf7fe640 1 --2- 172.21.15.96:0/1715378334 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5c0102190 0x7fb5c019a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:49.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.789+0000 7fb5bf7fe640 1 -- 172.21.15.96:0/1715378334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5c01080d0 msgr2=0x7fb5c01a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:49.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.789+0000 7fb5bf7fe640 1 --2- 172.21.15.96:0/1715378334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5c01080d0 0x7fb5c01a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:49.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.789+0000 7fb5bf7fe640 1 -- 172.21.15.96:0/1715378334 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c01a1c40 con 0x7fb5c01033d0 2024-04-25T14:15:49.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.789+0000 7fb5c4909640 1 --2- 172.21.15.96:0/1715378334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5c01080d0 0x7fb5c01a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:15:49.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.789+0000 7fb5bf7fe640 1 --2- 172.21.15.96:0/1715378334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5c01033d0 0x7fb5c019a6f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fb5b400c920 tx=0x7fb5b400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:49.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.789+0000 7fb5bd7fa640 1 -- 172.21.15.96:0/1715378334 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b4007c70 con 0x7fb5c01033d0 2024-04-25T14:15:49.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.789+0000 7fb5bd7fa640 1 -- 172.21.15.96:0/1715378334 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5b4007e10 con 0x7fb5c01033d0 2024-04-25T14:15:49.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.789+0000 7fb5c636c640 1 -- 172.21.15.96:0/1715378334 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5c01a1f30 con 0x7fb5c01033d0 2024-04-25T14:15:49.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.789+0000 7fb5bd7fa640 1 -- 172.21.15.96:0/1715378334 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b4018820 con 0x7fb5c01033d0 2024-04-25T14:15:49.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.789+0000 7fb5c636c640 1 -- 172.21.15.96:0/1715378334 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5c01a23a0 con 0x7fb5c01033d0 2024-04-25T14:15:49.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.792+0000 7fb5cd21c740 1 Processor -- start 2024-04-25T14:15:49.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.792+0000 7fb5cd21c740 1 -- start start 2024-04-25T14:15:49.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.792+0000 7fb5cd21c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592706b3390 0x559270a54a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:49.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.792+0000 7fb5cd21c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55927068a4d0 0x559270a54fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:49.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.792+0000 7fb5cd21c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5592706926b0 0x559270a55c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:49.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.792+0000 7fb5cd21c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5592709b9d80 con 0x5592706b3390 2024-04-25T14:15:49.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.792+0000 7fb5cd21c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5592709738b0 con 0x5592706926b0 2024-04-25T14:15:49.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.792+0000 7fb5cd21c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5592709b5d40 con 0x55927068a4d0 2024-04-25T14:15:49.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.793+0000 7fb5c4909640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592706b3390 0x559270a54a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:49.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.793+0000 7fb5c4909640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592706b3390 0x559270a54a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50842/0 (socket says 172.21.15.96:50842) 2024-04-25T14:15:49.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.793+0000 7fb5c4909640 1 -- 172.21.15.96:0/1726199720 learned_addr learned my addr 172.21.15.96:0/1726199720 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:49.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.793+0000 7fb5bd7fa640 1 -- 172.21.15.96:0/1715378334 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fb5b400f450 con 0x7fb5c01033d0 2024-04-25T14:15:49.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.793+0000 7fb5bffff640 1 --2- 172.21.15.96:0/1726199720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55927068a4d0 0x559270a54fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:49.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.793+0000 7fb5c4909640 1 -- 172.21.15.96:0/1726199720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55927068a4d0 msgr2=0x559270a54fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:49.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.793+0000 7fb5c4909640 1 --2- 172.21.15.96:0/1726199720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55927068a4d0 0x559270a54fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:49.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.793+0000 7fb5c4909640 1 -- 172.21.15.96:0/1726199720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5592706926b0 msgr2=0x559270a55c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:49.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.793+0000 7fb5c4909640 1 --2- 172.21.15.96:0/1726199720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5592706926b0 0x559270a55c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:49.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.793+0000 7fb5c4909640 1 -- 172.21.15.96:0/1726199720 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559270944a40 con 0x5592706b3390 2024-04-25T14:15:49.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.793+0000 7fb5c4909640 1 --2- 172.21.15.96:0/1726199720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592706b3390 0x559270a54a60 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8032ab0 tx=0x7fb5b800a910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a50baec2b260fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:49.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.793+0000 7fb5a67fc640 1 -- 172.21.15.96:0/1726199720 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b8032600 con 0x5592706b3390 2024-04-25T14:15:49.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.793+0000 7fb5a67fc640 1 -- 172.21.15.96:0/1726199720 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5b8003e20 con 0x5592706b3390 2024-04-25T14:15:49.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.793+0000 7fb5a67fc640 1 -- 172.21.15.96:0/1726199720 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b8004850 con 0x5592706b3390 2024-04-25T14:15:49.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.793+0000 7fb5bd7fa640 1 --2- 172.21.15.96:0/1715378334 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb5a80afaf0 0x7fb5a80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:49.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.794+0000 7fb5bffff640 1 --2- 172.21.15.96:0/1715378334 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb5a80afaf0 0x7fb5a80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:49.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.794+0000 7fb5bd7fa640 1 -- 172.21.15.96:0/1715378334 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fb5b40ded70 con 0x7fb5c01033d0 2024-04-25T14:15:49.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.794+0000 7fb5cd21c740 1 -- 172.21.15.96:0/1726199720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592706b3390 msgr2=0x559270a54a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:49.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.794+0000 7fb5bffff640 1 --2- 172.21.15.96:0/1715378334 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb5a80afaf0 0x7fb5a80b2000 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x559270a54780 tx=0x7fb5b000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:49.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.794+0000 7fb5cd21c740 1 --2- 172.21.15.96:0/1726199720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592706b3390 0x559270a54a60 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8032ab0 tx=0x7fb5b800a910 comp rx=0 tx=0).stop 2024-04-25T14:15:49.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.794+0000 7fb5cd21c740 1 -- 172.21.15.96:0/1726199720 shutdown_connections 2024-04-25T14:15:49.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.794+0000 7fb5cd21c740 1 --2- 172.21.15.96:0/1726199720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5592706926b0 0x559270a55c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:49.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.794+0000 7fb5cd21c740 1 --2- 172.21.15.96:0/1726199720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55927068a4d0 0x559270a54fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:49.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.794+0000 7fb5cd21c740 1 --2- 172.21.15.96:0/1726199720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592706b3390 0x559270a54a60 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:49.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.794+0000 7fb5cd21c740 1 -- 172.21.15.96:0/1726199720 >> 172.21.15.96:0/1726199720 conn(0x5592706a74e0 msgr2=0x559270a50270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:49.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.794+0000 7fb5cd21c740 1 -- 172.21.15.96:0/1726199720 shutdown_connections 2024-04-25T14:15:49.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.794+0000 7fb5cd21c740 1 -- 172.21.15.96:0/1726199720 wait complete. 2024-04-25T14:15:49.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.795+0000 7fb5cd21c740 1 -- start start 2024-04-25T14:15:49.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.795+0000 7fb5cd21c740 1 Processor -- start 2024-04-25T14:15:49.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.795+0000 7fb5cd21c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55927068a4d0 0x559270a5e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:49.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.795+0000 7fb5cd21c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5592706926b0 0x559270a5ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:49.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.795+0000 7fb5cd21c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592706b3390 0x559270a65e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:49.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.795+0000 7fb5cd21c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559270a56720 con 0x5592706b3390 2024-04-25T14:15:49.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.795+0000 7fb5cd21c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559270940d10 con 0x5592706926b0 2024-04-25T14:15:49.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.795+0000 7fb5cd21c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559270a568a0 con 0x55927068a4d0 2024-04-25T14:15:49.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.795+0000 7fb5cd21c740 10 client.0 _ll_register_callbacks cb 0x55927096d210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:15:49.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.795+0000 7fb5c4909640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592706b3390 0x559270a65e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:49.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.795+0000 7fb5c4909640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592706b3390 0x559270a65e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50850/0 (socket says 172.21.15.96:50850) 2024-04-25T14:15:49.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.795+0000 7fb5c4909640 1 -- 172.21.15.96:0/723186980 learned_addr learned my addr 172.21.15.96:0/723186980 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:49.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.795+0000 7fb5bffff640 1 --2- 172.21.15.96:0/723186980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55927068a4d0 0x559270a5e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:49.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.795+0000 7fb5bf7fe640 1 --2- 172.21.15.96:0/723186980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5592706926b0 0x559270a5ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:49.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.796+0000 7fb5c4909640 1 -- 172.21.15.96:0/723186980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55927068a4d0 msgr2=0x559270a5e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:49.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.796+0000 7fb5c4909640 1 --2- 172.21.15.96:0/723186980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55927068a4d0 0x559270a5e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:49.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.796+0000 7fb5c4909640 1 -- 172.21.15.96:0/723186980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5592706926b0 msgr2=0x559270a5ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:49.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.796+0000 7fb5c4909640 1 --2- 172.21.15.96:0/723186980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5592706926b0 0x559270a5ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:49.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.796+0000 7fb5c4909640 1 -- 172.21.15.96:0/723186980 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592707957d0 con 0x5592706b3390 2024-04-25T14:15:49.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.796+0000 7fb5bf7fe640 1 --2- 172.21.15.96:0/723186980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5592706926b0 0x559270a5ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:15:49.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.796+0000 7fb5c4909640 1 --2- 172.21.15.96:0/723186980 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592706b3390 0x559270a65e90 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8005e30 tx=0x7fb5b80048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47ef1782a425e39b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:49.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.796+0000 7fb59f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:15:49.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.796+0000 7fb59f7fe640 1 -- 172.21.15.96:0/723186980 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b8054060 con 0x5592706b3390 2024-04-25T14:15:49.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.796+0000 7fb59f7fe640 1 -- 172.21.15.96:0/723186980 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fb5b8032400 con 0x5592706b3390 2024-04-25T14:15:49.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.796+0000 7fb5cd21c740 1 -- 172.21.15.96:0/723186980 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5b8003e20 con 0x5592706b3390 2024-04-25T14:15:49.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.796+0000 7fb59f7fe640 1 -- 172.21.15.96:0/723186980 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb5b8032600 con 0x5592706b3390 2024-04-25T14:15:49.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.796+0000 7fb5cd21c740 10 client.16236 fetch_fsmap learned FSMap version 26 2024-04-25T14:15:49.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.796+0000 7fb5cd21c740 1 -- 172.21.15.96:0/723186980 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5592706cb3f0 con 0x5592706b3390 2024-04-25T14:15:49.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.796+0000 7fb59f7fe640 1 -- 172.21.15.96:0/723186980 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5b8003e20 con 0x5592706b3390 2024-04-25T14:15:49.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.796+0000 7fb59f7fe640 1 -- 172.21.15.96:0/723186980 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb5b8030510 con 0x5592706b3390 2024-04-25T14:15:49.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.796+0000 7fb5cd21c740 10 client.16236 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:15:49.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.796+0000 7fb5cd21c740 10 client.16236 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:15:49.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.797+0000 7fb5cd21c740 20 client.16236 populate_metadata read hostname 'smithi096' 2024-04-25T14:15:49.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.797+0000 7fb5cd21c740 1 --2- 172.21.15.96:0/723186980 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x559270671f60 0x559270a6bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:49.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.797+0000 7fb5cd21c740 4 client.16236 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:15:49.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.797+0000 7fb5cd21c740 1 -- 172.21.15.96:0/723186980 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592709c7990 con 0x559270671f60 2024-04-25T14:15:49.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.797+0000 7fb5bffff640 1 --2- 172.21.15.96:0/723186980 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x559270671f60 0x559270a6bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:49.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.797+0000 7fb59f7fe640 10 client.16236 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:15:49.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.797+0000 7fb5bffff640 1 --2- 172.21.15.96:0/723186980 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x559270671f60 0x559270a6bf50 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbd67ded0e737b5f server_cookie=fb753f1a2f32a13b in_seq=0 out_seq=0 2024-04-25T14:15:49.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.798+0000 7fb59f7fe640 1 -- 172.21.15.96:0/723186980 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5592709c7990 con 0x559270671f60 2024-04-25T14:15:49.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.798+0000 7fb59f7fe640 10 client.16236 handle_command_reply: tid=0 2024-04-25T14:15:49.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.798+0000 7fb5cd21c740 1 client.16236 shutdown 2024-04-25T14:15:49.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.798+0000 7fb5cd21c740 1 -- 172.21.15.96:0/723186980 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592706b3390 msgr2=0x559270a65e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:49.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.798+0000 7fb5cd21c740 1 --2- 172.21.15.96:0/723186980 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592706b3390 0x559270a65e90 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8005e30 tx=0x7fb5b80048a0 comp rx=0 tx=0).stop 2024-04-25T14:15:49.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.798+0000 7fb5cd21c740 1 -- 172.21.15.96:0/723186980 shutdown_connections 2024-04-25T14:15:49.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.798+0000 7fb5cd21c740 1 --2- 172.21.15.96:0/723186980 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x559270671f60 0x559270a6bf50 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:49.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.798+0000 7fb5cd21c740 1 --2- 172.21.15.96:0/723186980 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592706b3390 0x559270a65e90 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:49.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.798+0000 7fb5cd21c740 1 --2- 172.21.15.96:0/723186980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5592706926b0 0x559270a5ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:49.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.798+0000 7fb59f7fe640 0 client.16236 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:15:49.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.798+0000 7fb5cd21c740 1 --2- 172.21.15.96:0/723186980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55927068a4d0 0x559270a5e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:49.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.798+0000 7fb5cd21c740 1 -- 172.21.15.96:0/723186980 >> 172.21.15.96:0/723186980 conn(0x5592706a74e0 msgr2=0x559270a4f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:49.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.798+0000 7fb5cd21c740 1 -- 172.21.15.96:0/723186980 shutdown_connections 2024-04-25T14:15:49.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.798+0000 7fb5cd21c740 1 -- 172.21.15.96:0/723186980 wait complete. 2024-04-25T14:15:49.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.798+0000 7fb5cd21c740 20 client.16236 trim_cache size 0 max 16384 2024-04-25T14:15:49.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.816+0000 7fb5cd21c740 1 Processor -- start 2024-04-25T14:15:49.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.816+0000 7fb5cd21c740 1 -- start start 2024-04-25T14:15:49.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.816+0000 7fb5cd21c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55927068a4d0 0x559270a4fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:49.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.816+0000 7fb5cd21c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592706926b0 0x559270a501e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:49.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.816+0000 7fb5cd21c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592706b3390 0x559270a5d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:49.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.816+0000 7fb5cd21c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c01ab1b0 con 0x5592706b3390 2024-04-25T14:15:49.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.816+0000 7fb5cd21c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c01accf0 con 0x55927068a4d0 2024-04-25T14:15:49.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.816+0000 7fb5cd21c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c01ab880 con 0x5592706926b0 2024-04-25T14:15:49.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.816+0000 7fb5c4909640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592706b3390 0x559270a5d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:49.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.817+0000 7fb5c4909640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592706b3390 0x559270a5d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50858/0 (socket says 172.21.15.96:50858) 2024-04-25T14:15:49.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.817+0000 7fb5c4909640 1 -- 172.21.15.96:0/299310473 learned_addr learned my addr 172.21.15.96:0/299310473 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:49.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.817+0000 7fb5bffff640 1 --2- 172.21.15.96:0/299310473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55927068a4d0 0x559270a4fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:49.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.817+0000 7fb5c4909640 1 -- 172.21.15.96:0/299310473 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592706926b0 msgr2=0x559270a501e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:49.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.817+0000 7fb5c4909640 1 --2- 172.21.15.96:0/299310473 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592706926b0 0x559270a501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:49.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.817+0000 7fb5c4909640 1 -- 172.21.15.96:0/299310473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55927068a4d0 msgr2=0x559270a4fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:49.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.817+0000 7fb5c4909640 1 --2- 172.21.15.96:0/299310473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55927068a4d0 0x559270a4fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:49.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.817+0000 7fb5c4909640 1 -- 172.21.15.96:0/299310473 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55927068b0f0 con 0x5592706b3390 2024-04-25T14:15:49.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.817+0000 7fb5bffff640 1 --2- 172.21.15.96:0/299310473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55927068a4d0 0x559270a4fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:15:49.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.817+0000 7fb5c4909640 1 --2- 172.21.15.96:0/299310473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592706b3390 0x559270a5d010 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8030960 tx=0x7fb5b800ab30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27ac5bd0aacee7c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:49.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.817+0000 7fb5a5ffb640 1 -- 172.21.15.96:0/299310473 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b8054040 con 0x5592706b3390 2024-04-25T14:15:49.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.817+0000 7fb5a5ffb640 1 -- 172.21.15.96:0/299310473 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5b8032400 con 0x5592706b3390 2024-04-25T14:15:49.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.817+0000 7fb5a5ffb640 1 -- 172.21.15.96:0/299310473 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b8004400 con 0x5592706b3390 2024-04-25T14:15:49.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.818+0000 7fb5cd21c740 1 -- 172.21.15.96:0/299310473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592706b3390 msgr2=0x559270a5d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:49.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.818+0000 7fb5cd21c740 1 --2- 172.21.15.96:0/299310473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592706b3390 0x559270a5d010 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8030960 tx=0x7fb5b800ab30 comp rx=0 tx=0).stop 2024-04-25T14:15:49.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.818+0000 7fb5cd21c740 1 -- 172.21.15.96:0/299310473 shutdown_connections 2024-04-25T14:15:49.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.818+0000 7fb5cd21c740 1 --2- 172.21.15.96:0/299310473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592706b3390 0x559270a5d010 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:49.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.818+0000 7fb5cd21c740 1 --2- 172.21.15.96:0/299310473 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592706926b0 0x559270a501e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:49.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.818+0000 7fb5cd21c740 1 --2- 172.21.15.96:0/299310473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55927068a4d0 0x559270a4fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:49.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.818+0000 7fb5cd21c740 1 -- 172.21.15.96:0/299310473 >> 172.21.15.96:0/299310473 conn(0x5592706a74e0 msgr2=0x559270a59790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:49.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.818+0000 7fb5cd21c740 1 -- 172.21.15.96:0/299310473 shutdown_connections 2024-04-25T14:15:49.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.818+0000 7fb5cd21c740 1 -- 172.21.15.96:0/299310473 wait complete. 2024-04-25T14:15:49.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.818+0000 7fb5cd21c740 1 -- start start 2024-04-25T14:15:49.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.819+0000 7fb5cd21c740 1 Processor -- start 2024-04-25T14:15:49.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.819+0000 7fb5cd21c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55927068a4d0 0x559270a603a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:49.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.819+0000 7fb5cd21c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592706926b0 0x559270a608e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:49.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.819+0000 7fb5cd21c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5592706b3390 0x559270ad21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:49.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.819+0000 7fb5cd21c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c01ac620 con 0x5592706926b0 2024-04-25T14:15:49.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.819+0000 7fb5cd21c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c01abf50 con 0x5592706b3390 2024-04-25T14:15:49.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.819+0000 7fb5cd21c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c01a9190 con 0x55927068a4d0 2024-04-25T14:15:49.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.819+0000 7fb5cd21c740 10 client.0 _ll_register_callbacks cb 0x7fb5b8031430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:15:49.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.819+0000 7fb5bf7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592706926b0 0x559270a608e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:49.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.819+0000 7fb5bf7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592706926b0 0x559270a608e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50868/0 (socket says 172.21.15.96:50868) 2024-04-25T14:15:49.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.819+0000 7fb5bf7fe640 1 -- 172.21.15.96:0/610350720 learned_addr learned my addr 172.21.15.96:0/610350720 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:49.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.819+0000 7fb5c4909640 1 --2- 172.21.15.96:0/610350720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5592706b3390 0x559270ad21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:49.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.819+0000 7fb5bffff640 1 --2- 172.21.15.96:0/610350720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55927068a4d0 0x559270a603a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:49.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.819+0000 7fb5bf7fe640 1 -- 172.21.15.96:0/610350720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55927068a4d0 msgr2=0x559270a603a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:49.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.819+0000 7fb5bf7fe640 1 --2- 172.21.15.96:0/610350720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55927068a4d0 0x559270a603a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:49.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.819+0000 7fb5bf7fe640 1 -- 172.21.15.96:0/610350720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5592706b3390 msgr2=0x559270ad21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:49.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.819+0000 7fb5bf7fe640 1 --2- 172.21.15.96:0/610350720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5592706b3390 0x559270ad21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:49.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.819+0000 7fb5bf7fe640 1 -- 172.21.15.96:0/610350720 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55927097a4b0 con 0x5592706926b0 2024-04-25T14:15:49.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.819+0000 7fb5c4909640 1 --2- 172.21.15.96:0/610350720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5592706b3390 0x559270ad21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:15:49.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.819+0000 7fb5bf7fe640 1 --2- 172.21.15.96:0/610350720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592706926b0 0x559270a608e0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fb5b40dc8a0 tx=0x7fb5b40a6e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=579b146e2e49d3e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:49.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.819+0000 7fb59f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:15:49.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.819+0000 7fb59f7fe640 1 -- 172.21.15.96:0/610350720 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b40c0020 con 0x5592706926b0 2024-04-25T14:15:49.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.820+0000 7fb59f7fe640 1 -- 172.21.15.96:0/610350720 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fb5b40a34c0 con 0x5592706926b0 2024-04-25T14:15:49.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.820+0000 7fb5cd21c740 1 -- 172.21.15.96:0/610350720 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5b8032400 con 0x5592706926b0 2024-04-25T14:15:49.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.820+0000 7fb59f7fe640 1 -- 172.21.15.96:0/610350720 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb5b40ab5e0 con 0x5592706926b0 2024-04-25T14:15:49.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.820+0000 7fb5cd21c740 10 client.16242 fetch_fsmap learned FSMap version 26 2024-04-25T14:15:49.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.820+0000 7fb5cd21c740 1 -- 172.21.15.96:0/610350720 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559270ad3610 con 0x5592706926b0 2024-04-25T14:15:49.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.820+0000 7fb59f7fe640 1 -- 172.21.15.96:0/610350720 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5b8032400 con 0x5592706926b0 2024-04-25T14:15:49.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.820+0000 7fb59f7fe640 1 -- 172.21.15.96:0/610350720 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb5b40a6520 con 0x5592706926b0 2024-04-25T14:15:49.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.820+0000 7fb5cd21c740 10 client.16242 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:15:49.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.820+0000 7fb5cd21c740 10 client.16242 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:15:49.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.820+0000 7fb5cd21c740 20 client.16242 populate_metadata read hostname 'smithi096' 2024-04-25T14:15:49.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.820+0000 7fb5cd21c740 1 --2- 172.21.15.96:0/610350720 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5592706473a0 0x559270ad6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:49.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.820+0000 7fb5cd21c740 4 client.16242 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:15:49.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.820+0000 7fb5cd21c740 1 -- 172.21.15.96:0/610350720 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5592709c1800 con 0x5592706473a0 2024-04-25T14:15:49.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.820+0000 7fb5bffff640 1 --2- 172.21.15.96:0/610350720 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5592706473a0 0x559270ad6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:49.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.821+0000 7fb59f7fe640 10 client.16242 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:15:49.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.821+0000 7fb5bffff640 1 --2- 172.21.15.96:0/610350720 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5592706473a0 0x559270ad6000 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8aa416090ab7535b server_cookie=caacc6a2ff9a3480 in_seq=0 out_seq=0 2024-04-25T14:15:49.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.821+0000 7fb59f7fe640 1 -- 172.21.15.96:0/610350720 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5592709c1800 con 0x5592706473a0 2024-04-25T14:15:49.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.821+0000 7fb59f7fe640 10 client.16242 handle_command_reply: tid=0 2024-04-25T14:15:49.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.821+0000 7fb5cd21c740 1 client.16242 shutdown 2024-04-25T14:15:49.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.821+0000 7fb5cd21c740 1 -- 172.21.15.96:0/610350720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592706926b0 msgr2=0x559270a608e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:49.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.821+0000 7fb5cd21c740 1 --2- 172.21.15.96:0/610350720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592706926b0 0x559270a608e0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fb5b40dc8a0 tx=0x7fb5b40a6e00 comp rx=0 tx=0).stop 2024-04-25T14:15:49.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.821+0000 7fb5cd21c740 1 -- 172.21.15.96:0/610350720 shutdown_connections 2024-04-25T14:15:49.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.821+0000 7fb5cd21c740 1 --2- 172.21.15.96:0/610350720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5592706b3390 0x559270ad21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:49.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.821+0000 7fb5cd21c740 1 --2- 172.21.15.96:0/610350720 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x5592706473a0 0x559270ad6000 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:49.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.821+0000 7fb5cd21c740 1 --2- 172.21.15.96:0/610350720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592706926b0 0x559270a608e0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:49.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.821+0000 7fb5cd21c740 1 --2- 172.21.15.96:0/610350720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55927068a4d0 0x559270a603a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:49.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.821+0000 7fb5cd21c740 1 -- 172.21.15.96:0/610350720 >> 172.21.15.96:0/610350720 conn(0x5592706a74e0 msgr2=0x559270a5d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:49.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.821+0000 7fb59f7fe640 0 client.16242 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:15:49.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.821+0000 7fb5cd21c740 1 -- 172.21.15.96:0/610350720 shutdown_connections 2024-04-25T14:15:49.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.821+0000 7fb5cd21c740 1 -- 172.21.15.96:0/610350720 wait complete. 2024-04-25T14:15:49.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.821+0000 7fb5cd21c740 20 client.16242 trim_cache size 0 max 16384 2024-04-25T14:15:49.829 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T14:15:49.829 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "no active scrubs running", 2024-04-25T14:15:49.829 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": {} 2024-04-25T14:15:49.829 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T14:15:49.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.822+0000 7fb5c636c640 1 -- 172.21.15.96:0/1715378334 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb5a80afaf0 msgr2=0x7fb5a80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:49.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.822+0000 7fb5c636c640 1 --2- 172.21.15.96:0/1715378334 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb5a80afaf0 0x7fb5a80b2000 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x559270a54780 tx=0x7fb5b000b390 comp rx=0 tx=0).stop 2024-04-25T14:15:49.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.822+0000 7fb5c636c640 1 -- 172.21.15.96:0/1715378334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5c01033d0 msgr2=0x7fb5c019a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:49.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.822+0000 7fb5c636c640 1 --2- 172.21.15.96:0/1715378334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5c01033d0 0x7fb5c019a6f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fb5b400c920 tx=0x7fb5b400cde0 comp rx=0 tx=0).stop 2024-04-25T14:15:49.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.822+0000 7fb5c636c640 1 -- 172.21.15.96:0/1715378334 shutdown_connections 2024-04-25T14:15:49.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.822+0000 7fb5c636c640 1 --2- 172.21.15.96:0/1715378334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5c01080d0 0x7fb5c01a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:49.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.822+0000 7fb5c636c640 1 --2- 172.21.15.96:0/1715378334 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb5a80afaf0 0x7fb5a80b2000 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:49.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.822+0000 7fb5c636c640 1 --2- 172.21.15.96:0/1715378334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5c01033d0 0x7fb5c019a6f0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:49.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.822+0000 7fb5c636c640 1 --2- 172.21.15.96:0/1715378334 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5c0102190 0x7fb5c019a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:49.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.822+0000 7fb5c636c640 1 -- 172.21.15.96:0/1715378334 >> 172.21.15.96:0/1715378334 conn(0x7fb5c00fd380 msgr2=0x7fb5c01040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:49.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.822+0000 7fb5c636c640 1 -- 172.21.15.96:0/1715378334 shutdown_connections 2024-04-25T14:15:49.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.822+0000 7fb5c636c640 1 -- 172.21.15.96:0/1715378334 wait complete. 2024-04-25T14:15:49.833 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T14:15:49.833 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:15:49.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.941+0000 7f530aa4c640 1 -- 172.21.15.96:0/2856858128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5304102250 msgr2=0x7f530410bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:49.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.941+0000 7f530aa4c640 1 --2- 172.21.15.96:0/2856858128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5304102250 0x7f530410bcb0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f52fc009ca0 tx=0x7f52fc02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:15:49.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.941+0000 7f530aa4c640 1 -- 172.21.15.96:0/2856858128 shutdown_connections 2024-04-25T14:15:49.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.941+0000 7f530aa4c640 1 --2- 172.21.15.96:0/2856858128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5304102250 0x7f530410bcb0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:49.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.941+0000 7f530aa4c640 1 --2- 172.21.15.96:0/2856858128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5304101750 0x7f5304101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:49.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.941+0000 7f530aa4c640 1 --2- 172.21.15.96:0/2856858128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53041050a0 0x7f53041054b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:49.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.941+0000 7f530aa4c640 1 -- 172.21.15.96:0/2856858128 >> 172.21.15.96:0/2856858128 conn(0x7f53040fd370 msgr2=0x7f53040ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:49.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.941+0000 7f530aa4c640 1 -- 172.21.15.96:0/2856858128 shutdown_connections 2024-04-25T14:15:49.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.941+0000 7f530aa4c640 1 -- 172.21.15.96:0/2856858128 wait complete. 2024-04-25T14:15:49.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.942+0000 7f530aa4c640 1 Processor -- start 2024-04-25T14:15:49.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.942+0000 7f530aa4c640 1 -- start start 2024-04-25T14:15:49.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.942+0000 7f530aa4c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5304101750 0x7f530419a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:49.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.942+0000 7f530aa4c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5304102250 0x7f530419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:49.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.942+0000 7f530aa4c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53041050a0 0x7f53041a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:49.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.942+0000 7f530aa4c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f530410f610 con 0x7f5304101750 2024-04-25T14:15:49.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.942+0000 7f530aa4c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f530410f490 con 0x7f53041050a0 2024-04-25T14:15:49.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.942+0000 7f530aa4c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f530410f790 con 0x7f5304102250 2024-04-25T14:15:49.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.942+0000 7f5303fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5304101750 0x7f530419a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:49.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.942+0000 7f5303fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5304101750 0x7f530419a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50890/0 (socket says 172.21.15.96:50890) 2024-04-25T14:15:49.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.942+0000 7f5303fff640 1 -- 172.21.15.96:0/4040883412 learned_addr learned my addr 172.21.15.96:0/4040883412 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:49.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.942+0000 7f5308fe9640 1 --2- 172.21.15.96:0/4040883412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53041050a0 0x7f53041a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:49.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.943+0000 7f53037fe640 1 --2- 172.21.15.96:0/4040883412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5304102250 0x7f530419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:49.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.943+0000 7f5303fff640 1 -- 172.21.15.96:0/4040883412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5304102250 msgr2=0x7f530419a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:49.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.943+0000 7f5303fff640 1 --2- 172.21.15.96:0/4040883412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5304102250 0x7f530419a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:49.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.943+0000 7f5303fff640 1 -- 172.21.15.96:0/4040883412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53041050a0 msgr2=0x7f53041a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:49.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.943+0000 7f5303fff640 1 --2- 172.21.15.96:0/4040883412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53041050a0 0x7f53041a1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:49.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.943+0000 7f5303fff640 1 -- 172.21.15.96:0/4040883412 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53041a1c90 con 0x7f5304101750 2024-04-25T14:15:49.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.943+0000 7f53037fe640 1 --2- 172.21.15.96:0/4040883412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5304102250 0x7f530419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:15:49.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.943+0000 7f5303fff640 1 --2- 172.21.15.96:0/4040883412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5304101750 0x7f530419a150 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f52f4005fd0 tx=0x7f52f400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:49.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.943+0000 7f53017fa640 1 -- 172.21.15.96:0/4040883412 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f4017070 con 0x7f5304101750 2024-04-25T14:15:49.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.943+0000 7f530aa4c640 1 -- 172.21.15.96:0/4040883412 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53041a1f80 con 0x7f5304101750 2024-04-25T14:15:49.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.943+0000 7f53017fa640 1 -- 172.21.15.96:0/4040883412 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52f4009c30 con 0x7f5304101750 2024-04-25T14:15:49.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.943+0000 7f53017fa640 1 -- 172.21.15.96:0/4040883412 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f4003d10 con 0x7f5304101750 2024-04-25T14:15:49.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.943+0000 7f530aa4c640 1 -- 172.21.15.96:0/4040883412 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53041a23e0 con 0x7f5304101750 2024-04-25T14:15:49.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.944+0000 7f530aa4c640 1 -- 172.21.15.96:0/4040883412 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f530406a1c0 con 0x7f5304101750 2024-04-25T14:15:49.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.946+0000 7f53017fa640 1 -- 172.21.15.96:0/4040883412 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f52f4003eb0 con 0x7f5304101750 2024-04-25T14:15:49.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.946+0000 7f53017fa640 1 --2- 172.21.15.96:0/4040883412 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f52ec0afaf0 0x7f52ec0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:49.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.946+0000 7f53037fe640 1 --2- 172.21.15.96:0/4040883412 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f52ec0afaf0 0x7f52ec0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:49.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.946+0000 7f53017fa640 1 -- 172.21.15.96:0/4040883412 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f52f40df030 con 0x7f5304101750 2024-04-25T14:15:49.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.947+0000 7f53037fe640 1 --2- 172.21.15.96:0/4040883412 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f52ec0afaf0 0x7f52ec0b2000 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f52f8005fd0 tx=0x7f52f8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:49.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:49.950+0000 7f53017fa640 1 -- 172.21.15.96:0/4040883412 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f530406a1c0 con 0x7f5304101750 2024-04-25T14:15:50.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.166+0000 7f530aa4c640 1 -- 172.21.15.96:0/4040883412 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f530402dc10 con 0x7f5304101750 2024-04-25T14:15:50.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.166+0000 7f53017fa640 1 -- 172.21.15.96:0/4040883412 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11927 (secure 0 0 0) 0x7f52f4020030 con 0x7f5304101750 2024-04-25T14:15:50.166 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:15:50.166 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/606227917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":606227917},{"type":"v1","addr":"172.21.15.96:6835","nonce":606227917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/3265300710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3265300710},{"type":"v1","addr":"172.21.15.96:6837","nonce":3265300710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T14:11:28.886961+0000","modified":"2024-04-25T14:13:32.120297+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24517,"mds_2":24478,"mds_3":24455,"mds_4":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/4080308509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":4080308509},{"type":"v1","addr":"172.21.15.96:6839","nonce":4080308509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/1606748971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":1606748971},{"type":"v1","addr":"172.21.15.96:6841","nonce":1606748971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6833/1995324831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1995324831},{"type":"v1","addr":"172.21.15.97:6833","nonce":1995324831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":4,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.151:6833/1382721608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1382721608},{"type":"v1","addr":"172.21.15.151:6833","nonce":1382721608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6835/3872301726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3872301726},{"type":"v1","addr":"172.21.15.97:6835","nonce":3872301726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.151:6835/1313823638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":1313823638},{"type":"v1","addr":"172.21.15.151:6835","nonce":1313823638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/2691348366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2691348366},{"type":"v1","addr":"172.21.15.151:6837","nonce":2691348366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3306780081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3306780081},{"type":"v1","addr":"172.21.15.97:6837","nonce":3306780081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.151:6839/245699981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":245699981},{"type":"v1","addr":"172.21.15.151:6839","nonce":245699981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6839/2850442269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":2850442269},{"type":"v1","addr":"172.21.15.97:6839","nonce":2850442269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24427,24455,24478,24517,24536]},"id":1}]} 2024-04-25T14:15:50.167 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 26 2024-04-25T14:15:50.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.168+0000 7f530aa4c640 1 -- 172.21.15.96:0/4040883412 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f52ec0afaf0 msgr2=0x7f52ec0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.168+0000 7f530aa4c640 1 --2- 172.21.15.96:0/4040883412 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f52ec0afaf0 0x7f52ec0b2000 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f52f8005fd0 tx=0x7f52f8005850 comp rx=0 tx=0).stop 2024-04-25T14:15:50.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.168+0000 7f530aa4c640 1 -- 172.21.15.96:0/4040883412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5304101750 msgr2=0x7f530419a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.168+0000 7f530aa4c640 1 --2- 172.21.15.96:0/4040883412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5304101750 0x7f530419a150 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f52f4005fd0 tx=0x7f52f400dde0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.168+0000 7f530aa4c640 1 -- 172.21.15.96:0/4040883412 shutdown_connections 2024-04-25T14:15:50.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.168+0000 7f530aa4c640 1 --2- 172.21.15.96:0/4040883412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53041050a0 0x7f53041a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.169+0000 7f530aa4c640 1 --2- 172.21.15.96:0/4040883412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5304102250 0x7f530419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.169+0000 7f530aa4c640 1 --2- 172.21.15.96:0/4040883412 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f52ec0afaf0 0x7f52ec0b2000 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.169+0000 7f530aa4c640 1 --2- 172.21.15.96:0/4040883412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5304101750 0x7f530419a150 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.169+0000 7f530aa4c640 1 -- 172.21.15.96:0/4040883412 >> 172.21.15.96:0/4040883412 conn(0x7f53040fd370 msgr2=0x7f5304107c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:50.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.169+0000 7f530aa4c640 1 -- 172.21.15.96:0/4040883412 shutdown_connections 2024-04-25T14:15:50.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.169+0000 7f530aa4c640 1 -- 172.21.15.96:0/4040883412 wait complete. 2024-04-25T14:15:50.180 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T14:15:50.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.291+0000 7f612e020640 1 -- 172.21.15.96:0/2309546595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6128103c40 msgr2=0x7f6128104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.291+0000 7f612e020640 1 --2- 172.21.15.96:0/2309546595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6128103c40 0x7f6128104050 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f6110009900 tx=0x7f611002a410 comp rx=0 tx=0).stop 2024-04-25T14:15:50.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.291+0000 7f612e020640 1 -- 172.21.15.96:0/2309546595 shutdown_connections 2024-04-25T14:15:50.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.291+0000 7f612e020640 1 --2- 172.21.15.96:0/2309546595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6128108940 0x7f612810ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.291+0000 7f612e020640 1 --2- 172.21.15.96:0/2309546595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6128103c40 0x7f6128104050 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.291+0000 7f612e020640 1 --2- 172.21.15.96:0/2309546595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6128102a00 0x7f6128103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.291+0000 7f612e020640 1 -- 172.21.15.96:0/2309546595 >> 172.21.15.96:0/2309546595 conn(0x7f61280fdbf0 msgr2=0x7f61281000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:50.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.291+0000 7f612e020640 1 -- 172.21.15.96:0/2309546595 shutdown_connections 2024-04-25T14:15:50.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.291+0000 7f612e020640 1 -- 172.21.15.96:0/2309546595 wait complete. 2024-04-25T14:15:50.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.291+0000 7f612e020640 1 Processor -- start 2024-04-25T14:15:50.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.292+0000 7f612e020640 1 -- start start 2024-04-25T14:15:50.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.292+0000 7f612e020640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6128102a00 0x7f612819a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.292+0000 7f612e020640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6128103c40 0x7f612819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.292+0000 7f612e020640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6128108940 0x7f61281a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.292+0000 7f612e020640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f612810e500 con 0x7f6128102a00 2024-04-25T14:15:50.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.292+0000 7f612e020640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f612810e380 con 0x7f6128108940 2024-04-25T14:15:50.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.292+0000 7f612e020640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f612810e680 con 0x7f6128103c40 2024-04-25T14:15:50.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.292+0000 7f61277fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6128102a00 0x7f612819a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.292+0000 7f61277fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6128102a00 0x7f612819a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50908/0 (socket says 172.21.15.96:50908) 2024-04-25T14:15:50.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.292+0000 7f61277fe640 1 -- 172.21.15.96:0/1086164990 learned_addr learned my addr 172.21.15.96:0/1086164990 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:50.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.292+0000 7f611effd640 1 --2- 172.21.15.96:0/1086164990 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6128103c40 0x7f612819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.292+0000 7f6127fff640 1 --2- 172.21.15.96:0/1086164990 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6128108940 0x7f61281a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.292+0000 7f61277fe640 1 -- 172.21.15.96:0/1086164990 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6128103c40 msgr2=0x7f612819a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.292+0000 7f61277fe640 1 --2- 172.21.15.96:0/1086164990 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6128103c40 0x7f612819a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.292+0000 7f61277fe640 1 -- 172.21.15.96:0/1086164990 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6128108940 msgr2=0x7f61281a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.292+0000 7f61277fe640 1 --2- 172.21.15.96:0/1086164990 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6128108940 0x7f61281a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.292+0000 7f61277fe640 1 -- 172.21.15.96:0/1086164990 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61281a1c80 con 0x7f6128102a00 2024-04-25T14:15:50.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.292+0000 7f6127fff640 1 --2- 172.21.15.96:0/1086164990 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6128108940 0x7f61281a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:15:50.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.292+0000 7f61277fe640 1 --2- 172.21.15.96:0/1086164990 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6128102a00 0x7f612819a1f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f6118005fd0 tx=0x7f611800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:50.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.293+0000 7f61257fa640 1 -- 172.21.15.96:0/1086164990 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6118017070 con 0x7f6128102a00 2024-04-25T14:15:50.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.293+0000 7f61257fa640 1 -- 172.21.15.96:0/1086164990 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6118009c30 con 0x7f6128102a00 2024-04-25T14:15:50.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.293+0000 7f612e020640 1 -- 172.21.15.96:0/1086164990 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61281a1f70 con 0x7f6128102a00 2024-04-25T14:15:50.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.293+0000 7f61257fa640 1 -- 172.21.15.96:0/1086164990 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6118003d10 con 0x7f6128102a00 2024-04-25T14:15:50.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.293+0000 7f612e020640 1 -- 172.21.15.96:0/1086164990 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61281a23d0 con 0x7f6128102a00 2024-04-25T14:15:50.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.295+0000 7f61257fa640 1 -- 172.21.15.96:0/1086164990 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f6118003eb0 con 0x7f6128102a00 2024-04-25T14:15:50.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.295+0000 7f6134fac740 1 Processor -- start 2024-04-25T14:15:50.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.295+0000 7f6134fac740 1 -- start start 2024-04-25T14:15:50.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.296+0000 7f6134fac740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577818ae390 0x557781c50810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.296+0000 7f6134fac740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5577818854d0 0x557781c50d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.296+0000 7f6134fac740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55778188d6b0 0x557781c51a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.296+0000 7f6134fac740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557781bb4d80 con 0x5577818ae390 2024-04-25T14:15:50.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.296+0000 7f6134fac740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557781b6ea20 con 0x5577818854d0 2024-04-25T14:15:50.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.296+0000 7f6134fac740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557781bb0d40 con 0x55778188d6b0 2024-04-25T14:15:50.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.296+0000 7f6127fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577818ae390 0x557781c50810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.296+0000 7f6127fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577818ae390 0x557781c50810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50924/0 (socket says 172.21.15.96:50924) 2024-04-25T14:15:50.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.296+0000 7f6127fff640 1 -- 172.21.15.96:0/756249508 learned_addr learned my addr 172.21.15.96:0/756249508 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:50.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.296+0000 7f61257fa640 1 --2- 172.21.15.96:0/1086164990 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f61140afaf0 0x7f61140b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.296+0000 7f61277fe640 1 --2- 172.21.15.96:0/756249508 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5577818854d0 0x557781c50d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.296+0000 7f611effd640 1 --2- 172.21.15.96:0/756249508 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55778188d6b0 0x557781c51a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.296+0000 7f6127fff640 1 -- 172.21.15.96:0/756249508 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55778188d6b0 msgr2=0x557781c51a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:50.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.296+0000 7f6127fff640 1 --2- 172.21.15.96:0/756249508 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55778188d6b0 0x557781c51a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.296+0000 7f6127fff640 1 -- 172.21.15.96:0/756249508 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5577818854d0 msgr2=0x557781c50d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:50.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.296+0000 7f6127fff640 1 --2- 172.21.15.96:0/756249508 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5577818854d0 0x557781c50d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.296+0000 7f6127fff640 1 -- 172.21.15.96:0/756249508 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557781b79c30 con 0x5577818ae390 2024-04-25T14:15:50.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.296+0000 7f6127fff640 1 --2- 172.21.15.96:0/1086164990 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f61140afaf0 0x7f61140b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.296+0000 7f61257fa640 1 -- 172.21.15.96:0/1086164990 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f61180de950 con 0x7f6128102a00 2024-04-25T14:15:50.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.296+0000 7f6127fff640 1 --2- 172.21.15.96:0/756249508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577818ae390 0x557781c50810 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f6120008f60 tx=0x7f6120008f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c41b9edfd24ba4c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:50.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.296+0000 7f611dffb640 1 -- 172.21.15.96:0/756249508 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612001f070 con 0x5577818ae390 2024-04-25T14:15:50.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.297+0000 7f611dffb640 1 -- 172.21.15.96:0/756249508 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6120018390 con 0x5577818ae390 2024-04-25T14:15:50.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.297+0000 7f611dffb640 1 -- 172.21.15.96:0/756249508 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612000b5a0 con 0x5577818ae390 2024-04-25T14:15:50.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.297+0000 7f6134fac740 1 -- 172.21.15.96:0/756249508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577818ae390 msgr2=0x557781c50810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.297+0000 7f6127fff640 1 --2- 172.21.15.96:0/1086164990 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f61140afaf0 0x7f61140b2000 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f612001c000 tx=0x7f612001d000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:50.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.297+0000 7f6134fac740 1 --2- 172.21.15.96:0/756249508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577818ae390 0x557781c50810 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f6120008f60 tx=0x7f6120008f90 comp rx=0 tx=0).stop 2024-04-25T14:15:50.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.297+0000 7f6134fac740 1 -- 172.21.15.96:0/756249508 shutdown_connections 2024-04-25T14:15:50.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.297+0000 7f6134fac740 1 --2- 172.21.15.96:0/756249508 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55778188d6b0 0x557781c51a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.297+0000 7f6134fac740 1 --2- 172.21.15.96:0/756249508 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5577818854d0 0x557781c50d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.297+0000 7f6134fac740 1 --2- 172.21.15.96:0/756249508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577818ae390 0x557781c50810 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.297+0000 7f6134fac740 1 -- 172.21.15.96:0/756249508 >> 172.21.15.96:0/756249508 conn(0x5577818a24e0 msgr2=0x557781c4c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:50.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.297+0000 7f6134fac740 1 -- 172.21.15.96:0/756249508 shutdown_connections 2024-04-25T14:15:50.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.297+0000 7f6134fac740 1 -- 172.21.15.96:0/756249508 wait complete. 2024-04-25T14:15:50.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.298+0000 7f6134fac740 1 -- start start 2024-04-25T14:15:50.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.298+0000 7f6134fac740 1 Processor -- start 2024-04-25T14:15:50.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.298+0000 7f6134fac740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577818854d0 0x557781c5a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.298+0000 7f6134fac740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55778188d6b0 0x557781c5aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.298+0000 7f6134fac740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577818ae390 0x557781c61a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.298+0000 7f6134fac740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557781c52530 con 0x5577818854d0 2024-04-25T14:15:50.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.298+0000 7f6134fac740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557781b3bdc0 con 0x55778188d6b0 2024-04-25T14:15:50.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.298+0000 7f6134fac740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557781c526b0 con 0x5577818ae390 2024-04-25T14:15:50.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.298+0000 7f6134fac740 10 client.0 _ll_register_callbacks cb 0x557781b57600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:15:50.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.298+0000 7f61277fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577818854d0 0x557781c5a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.298+0000 7f61277fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577818854d0 0x557781c5a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50938/0 (socket says 172.21.15.96:50938) 2024-04-25T14:15:50.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.298+0000 7f61277fe640 1 -- 172.21.15.96:0/3475795544 learned_addr learned my addr 172.21.15.96:0/3475795544 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:50.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.299+0000 7f6127fff640 1 --2- 172.21.15.96:0/3475795544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577818ae390 0x557781c61a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.299+0000 7f611effd640 1 --2- 172.21.15.96:0/3475795544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55778188d6b0 0x557781c5aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.299+0000 7f61277fe640 1 -- 172.21.15.96:0/3475795544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577818ae390 msgr2=0x557781c61a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:50.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.299+0000 7f61277fe640 1 --2- 172.21.15.96:0/3475795544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577818ae390 0x557781c61a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.299+0000 7f61277fe640 1 -- 172.21.15.96:0/3475795544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55778188d6b0 msgr2=0x557781c5aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:50.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.299+0000 7f61277fe640 1 --2- 172.21.15.96:0/3475795544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55778188d6b0 0x557781c5aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.299+0000 7f61277fe640 1 -- 172.21.15.96:0/3475795544 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577818c63f0 con 0x5577818854d0 2024-04-25T14:15:50.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.299+0000 7f61277fe640 1 --2- 172.21.15.96:0/3475795544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577818854d0 0x557781c5a560 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f61180058c0 tx=0x7f61180a6a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48eef335db87ff5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:50.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.299+0000 7f6106ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:15:50.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.299+0000 7f6106ffd640 1 -- 172.21.15.96:0/3475795544 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61180ab1c0 con 0x5577818854d0 2024-04-25T14:15:50.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.299+0000 7f6106ffd640 1 -- 172.21.15.96:0/3475795544 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f61180a30a0 con 0x5577818854d0 2024-04-25T14:15:50.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.299+0000 7f6134fac740 1 -- 172.21.15.96:0/3475795544 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6120018390 con 0x5577818854d0 2024-04-25T14:15:50.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.299+0000 7f6106ffd640 1 -- 172.21.15.96:0/3475795544 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f61180afc70 con 0x5577818854d0 2024-04-25T14:15:50.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.299+0000 7f6134fac740 10 client.16260 fetch_fsmap learned FSMap version 26 2024-04-25T14:15:50.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.299+0000 7f6134fac740 1 -- 172.21.15.96:0/3475795544 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5577819907d0 con 0x5577818854d0 2024-04-25T14:15:50.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.299+0000 7f6106ffd640 1 -- 172.21.15.96:0/3475795544 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6120018390 con 0x5577818854d0 2024-04-25T14:15:50.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.299+0000 7f6106ffd640 1 -- 172.21.15.96:0/3475795544 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f61180c11b0 con 0x5577818854d0 2024-04-25T14:15:50.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.299+0000 7f6134fac740 10 client.16260 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:15:50.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.299+0000 7f6134fac740 10 client.16260 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-25T14:15:50.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.300+0000 7f6134fac740 20 client.16260 populate_metadata read hostname 'smithi096' 2024-04-25T14:15:50.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.300+0000 7f6134fac740 1 --2- 172.21.15.96:0/3475795544 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x55778186cf60 0x557781c68240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.300+0000 7f6134fac740 4 client.16260 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:15:50.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.300+0000 7f6134fac740 1 -- 172.21.15.96:0/3475795544 --> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557781bc2990 con 0x55778186cf60 2024-04-25T14:15:50.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.300+0000 7f611effd640 1 --2- 172.21.15.96:0/3475795544 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x55778186cf60 0x557781c68240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.300+0000 7f6106ffd640 10 client.16260 ms_handle_connect on v2:172.21.15.151:6832/1382721608 2024-04-25T14:15:50.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.300+0000 7f611effd640 1 --2- 172.21.15.96:0/3475795544 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x55778186cf60 0x557781c68240 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=576dde767e9a107 server_cookie=a63d3c8063abb4b5 in_seq=0 out_seq=0 2024-04-25T14:15:50.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.301+0000 7f6106ffd640 1 -- 172.21.15.96:0/3475795544 <== mds.4 v2:172.21.15.151:6832/1382721608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557781bc2990 con 0x55778186cf60 2024-04-25T14:15:50.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.301+0000 7f6106ffd640 10 client.16260 handle_command_reply: tid=0 2024-04-25T14:15:50.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.301+0000 7f6134fac740 1 client.16260 shutdown 2024-04-25T14:15:50.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.301+0000 7f6134fac740 1 -- 172.21.15.96:0/3475795544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577818854d0 msgr2=0x557781c5a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.301+0000 7f6134fac740 1 --2- 172.21.15.96:0/3475795544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577818854d0 0x557781c5a560 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f61180058c0 tx=0x7f61180a6a60 comp rx=0 tx=0).stop 2024-04-25T14:15:50.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.301+0000 7f6134fac740 1 -- 172.21.15.96:0/3475795544 shutdown_connections 2024-04-25T14:15:50.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.301+0000 7f6134fac740 1 --2- 172.21.15.96:0/3475795544 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x55778186cf60 0x557781c68240 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.301+0000 7f6134fac740 1 --2- 172.21.15.96:0/3475795544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577818ae390 0x557781c61a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.301+0000 7f6134fac740 1 --2- 172.21.15.96:0/3475795544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55778188d6b0 0x557781c5aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.301+0000 7f6106ffd640 0 client.16260 ms_handle_reset on v2:172.21.15.151:6832/1382721608 2024-04-25T14:15:50.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.301+0000 7f6134fac740 1 --2- 172.21.15.96:0/3475795544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577818854d0 0x557781c5a560 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.301+0000 7f6134fac740 1 -- 172.21.15.96:0/3475795544 >> 172.21.15.96:0/3475795544 conn(0x5577818a24e0 msgr2=0x557781c4b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:50.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.302+0000 7f6134fac740 1 -- 172.21.15.96:0/3475795544 shutdown_connections 2024-04-25T14:15:50.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.302+0000 7f6134fac740 1 -- 172.21.15.96:0/3475795544 wait complete. 2024-04-25T14:15:50.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.302+0000 7f6134fac740 20 client.16260 trim_cache size 0 max 16384 2024-04-25T14:15:50.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.315+0000 7f6134fac740 1 Processor -- start 2024-04-25T14:15:50.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.315+0000 7f6134fac740 1 -- start start 2024-04-25T14:15:50.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.315+0000 7f6134fac740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577818854d0 0x557781c5a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.315+0000 7f6134fac740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55778188d6b0 0x557781c61990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.315+0000 7f6134fac740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5577818ae390 0x557781c6c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.315+0000 7f6134fac740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f61281ab2e0 con 0x55778188d6b0 2024-04-25T14:15:50.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.315+0000 7f6134fac740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f61281ace20 con 0x5577818ae390 2024-04-25T14:15:50.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.315+0000 7f6134fac740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f61281ab9b0 con 0x5577818854d0 2024-04-25T14:15:50.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.315+0000 7f611effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55778188d6b0 0x557781c61990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.315+0000 7f611effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55778188d6b0 0x557781c61990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50950/0 (socket says 172.21.15.96:50950) 2024-04-25T14:15:50.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.315+0000 7f611effd640 1 -- 172.21.15.96:0/3476654768 learned_addr learned my addr 172.21.15.96:0/3476654768 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:50.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.315+0000 7f611effd640 1 -- 172.21.15.96:0/3476654768 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577818854d0 msgr2=0x557781c5a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:50.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.315+0000 7f611effd640 1 --2- 172.21.15.96:0/3476654768 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577818854d0 0x557781c5a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.315+0000 7f611effd640 1 -- 172.21.15.96:0/3476654768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5577818ae390 msgr2=0x557781c6c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:50.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.315+0000 7f611effd640 1 --2- 172.21.15.96:0/3476654768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5577818ae390 0x557781c6c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.315+0000 7f611effd640 1 -- 172.21.15.96:0/3476654768 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557781c58680 con 0x55778188d6b0 2024-04-25T14:15:50.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.315+0000 7f611effd640 1 --2- 172.21.15.96:0/3476654768 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55778188d6b0 0x557781c61990 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f6110009850 tx=0x7f6110031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6288735bce7ad843 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:50.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.316+0000 7f611d7fa640 1 -- 172.21.15.96:0/3476654768 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6110048020 con 0x55778188d6b0 2024-04-25T14:15:50.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.316+0000 7f611d7fa640 1 -- 172.21.15.96:0/3476654768 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61100035f0 con 0x55778188d6b0 2024-04-25T14:15:50.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.316+0000 7f611d7fa640 1 -- 172.21.15.96:0/3476654768 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6110035040 con 0x55778188d6b0 2024-04-25T14:15:50.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.316+0000 7f6134fac740 1 -- 172.21.15.96:0/3476654768 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55778188d6b0 msgr2=0x557781c61990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.316+0000 7f6134fac740 1 --2- 172.21.15.96:0/3476654768 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55778188d6b0 0x557781c61990 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f6110009850 tx=0x7f6110031400 comp rx=0 tx=0).stop 2024-04-25T14:15:50.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.316+0000 7f6134fac740 1 -- 172.21.15.96:0/3476654768 shutdown_connections 2024-04-25T14:15:50.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.316+0000 7f6134fac740 1 --2- 172.21.15.96:0/3476654768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5577818ae390 0x557781c6c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.316+0000 7f6134fac740 1 --2- 172.21.15.96:0/3476654768 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55778188d6b0 0x557781c61990 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.316+0000 7f6134fac740 1 --2- 172.21.15.96:0/3476654768 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577818854d0 0x557781c5a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.316+0000 7f6134fac740 1 -- 172.21.15.96:0/3476654768 >> 172.21.15.96:0/3476654768 conn(0x5577818a24e0 msgr2=0x557781c5a130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:50.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.316+0000 7f6134fac740 1 -- 172.21.15.96:0/3476654768 shutdown_connections 2024-04-25T14:15:50.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.316+0000 7f6134fac740 1 -- 172.21.15.96:0/3476654768 wait complete. 2024-04-25T14:15:50.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.317+0000 7f6134fac740 1 -- start start 2024-04-25T14:15:50.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.317+0000 7f6134fac740 1 Processor -- start 2024-04-25T14:15:50.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.317+0000 7f6134fac740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5577818854d0 0x557781c5ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.317+0000 7f6134fac740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55778188d6b0 0x557781c5f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.317+0000 7f6134fac740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577818ae390 0x557781c600b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.317+0000 7f6134fac740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f61281ac750 con 0x55778188d6b0 2024-04-25T14:15:50.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.317+0000 7f6134fac740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f61281ac080 con 0x5577818854d0 2024-04-25T14:15:50.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.317+0000 7f6134fac740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f61281a92c0 con 0x5577818ae390 2024-04-25T14:15:50.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.317+0000 7f6134fac740 10 client.0 _ll_register_callbacks cb 0x7f61180a6850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:15:50.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.317+0000 7f611effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55778188d6b0 0x557781c5f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.317+0000 7f611effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55778188d6b0 0x557781c5f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50964/0 (socket says 172.21.15.96:50964) 2024-04-25T14:15:50.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.317+0000 7f611effd640 1 -- 172.21.15.96:0/2400181549 learned_addr learned my addr 172.21.15.96:0/2400181549 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:50.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.317+0000 7f6127fff640 1 --2- 172.21.15.96:0/2400181549 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577818ae390 0x557781c600b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.318+0000 7f61277fe640 1 --2- 172.21.15.96:0/2400181549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5577818854d0 0x557781c5ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.318+0000 7f611effd640 1 -- 172.21.15.96:0/2400181549 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577818ae390 msgr2=0x557781c600b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:50.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.318+0000 7f611effd640 1 --2- 172.21.15.96:0/2400181549 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577818ae390 0x557781c600b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.318+0000 7f611effd640 1 -- 172.21.15.96:0/2400181549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5577818854d0 msgr2=0x557781c5ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:50.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.318+0000 7f611effd640 1 --2- 172.21.15.96:0/2400181549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5577818854d0 0x557781c5ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.318+0000 7f611effd640 1 -- 172.21.15.96:0/2400181549 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557781c60670 con 0x55778188d6b0 2024-04-25T14:15:50.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.318+0000 7f6127fff640 1 --2- 172.21.15.96:0/2400181549 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577818ae390 0x557781c600b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:15:50.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.318+0000 7f61277fe640 1 --2- 172.21.15.96:0/2400181549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5577818854d0 0x557781c5ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:15:50.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.318+0000 7f611effd640 1 --2- 172.21.15.96:0/2400181549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55778188d6b0 0x557781c5f260 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f61100045b0 tx=0x7f61100045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e9f4d789eb31a1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:50.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.318+0000 7f6106ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:15:50.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.318+0000 7f6106ffd640 1 -- 172.21.15.96:0/2400181549 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6110048030 con 0x55778188d6b0 2024-04-25T14:15:50.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.318+0000 7f6106ffd640 1 -- 172.21.15.96:0/2400181549 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f61100315a0 con 0x55778188d6b0 2024-04-25T14:15:50.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.318+0000 7f6134fac740 1 -- 172.21.15.96:0/2400181549 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61100035f0 con 0x55778188d6b0 2024-04-25T14:15:50.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.318+0000 7f6106ffd640 1 -- 172.21.15.96:0/2400181549 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6110035040 con 0x55778188d6b0 2024-04-25T14:15:50.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.318+0000 7f6134fac740 10 client.16266 fetch_fsmap learned FSMap version 26 2024-04-25T14:15:50.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.318+0000 7f6134fac740 1 -- 172.21.15.96:0/2400181549 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557781c61920 con 0x55778188d6b0 2024-04-25T14:15:50.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.318+0000 7f6106ffd640 1 -- 172.21.15.96:0/2400181549 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61100035f0 con 0x55778188d6b0 2024-04-25T14:15:50.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.318+0000 7f6106ffd640 1 -- 172.21.15.96:0/2400181549 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6110044ce0 con 0x55778188d6b0 2024-04-25T14:15:50.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.319+0000 7f6134fac740 10 client.16266 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:15:50.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.319+0000 7f6134fac740 10 client.16266 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-25T14:15:50.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.319+0000 7f6134fac740 20 client.16266 populate_metadata read hostname 'smithi096' 2024-04-25T14:15:50.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.319+0000 7f6134fac740 1 --2- 172.21.15.96:0/2400181549 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x5577818423a0 0x557781cd2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.319+0000 7f6134fac740 4 client.16266 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:15:50.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.319+0000 7f6134fac740 1 -- 172.21.15.96:0/2400181549 --> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557781bbc800 con 0x5577818423a0 2024-04-25T14:15:50.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.319+0000 7f61277fe640 1 --2- 172.21.15.96:0/2400181549 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x5577818423a0 0x557781cd2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.319+0000 7f6106ffd640 10 client.16266 ms_handle_connect on v2:172.21.15.151:6832/1382721608 2024-04-25T14:15:50.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.319+0000 7f61277fe640 1 --2- 172.21.15.96:0/2400181549 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x5577818423a0 0x557781cd2a00 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=81bdd28da39f9607 server_cookie=e8975500ac429b0 in_seq=0 out_seq=0 2024-04-25T14:15:50.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.320+0000 7f6106ffd640 1 -- 172.21.15.96:0/2400181549 <== mds.4 v2:172.21.15.151:6832/1382721608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557781bbc800 con 0x5577818423a0 2024-04-25T14:15:50.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.320+0000 7f6106ffd640 10 client.16266 handle_command_reply: tid=0 2024-04-25T14:15:50.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.320+0000 7f6134fac740 1 client.16266 shutdown 2024-04-25T14:15:50.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.320+0000 7f6134fac740 1 -- 172.21.15.96:0/2400181549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55778188d6b0 msgr2=0x557781c5f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.320+0000 7f6134fac740 1 --2- 172.21.15.96:0/2400181549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55778188d6b0 0x557781c5f260 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f61100045b0 tx=0x7f61100045e0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.320+0000 7f6134fac740 1 -- 172.21.15.96:0/2400181549 shutdown_connections 2024-04-25T14:15:50.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.320+0000 7f6134fac740 1 --2- 172.21.15.96:0/2400181549 >> [v2:172.21.15.151:6832/1382721608,v1:172.21.15.151:6833/1382721608] conn(0x5577818423a0 0x557781cd2a00 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.320+0000 7f6134fac740 1 --2- 172.21.15.96:0/2400181549 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577818ae390 0x557781c600b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.320+0000 7f6106ffd640 0 client.16266 ms_handle_reset on v2:172.21.15.151:6832/1382721608 2024-04-25T14:15:50.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.320+0000 7f6134fac740 1 --2- 172.21.15.96:0/2400181549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55778188d6b0 0x557781c5f260 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.320+0000 7f6134fac740 1 --2- 172.21.15.96:0/2400181549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5577818854d0 0x557781c5ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.320+0000 7f6134fac740 1 -- 172.21.15.96:0/2400181549 >> 172.21.15.96:0/2400181549 conn(0x5577818a24e0 msgr2=0x557781c4f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:50.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.320+0000 7f6134fac740 1 -- 172.21.15.96:0/2400181549 shutdown_connections 2024-04-25T14:15:50.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.320+0000 7f6134fac740 1 -- 172.21.15.96:0/2400181549 wait complete. 2024-04-25T14:15:50.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.320+0000 7f6134fac740 20 client.16266 trim_cache size 0 max 16384 2024-04-25T14:15:50.329 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-04-25T14:15:50.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.321+0000 7f612e020640 1 -- 172.21.15.96:0/1086164990 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f61140afaf0 msgr2=0x7f61140b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.321+0000 7f612e020640 1 --2- 172.21.15.96:0/1086164990 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f61140afaf0 0x7f61140b2000 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f612001c000 tx=0x7f612001d000 comp rx=0 tx=0).stop 2024-04-25T14:15:50.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.321+0000 7f612e020640 1 -- 172.21.15.96:0/1086164990 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6128102a00 msgr2=0x7f612819a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.321+0000 7f612e020640 1 --2- 172.21.15.96:0/1086164990 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6128102a00 0x7f612819a1f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f6118005fd0 tx=0x7f611800dde0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.321+0000 7f612e020640 1 -- 172.21.15.96:0/1086164990 shutdown_connections 2024-04-25T14:15:50.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.321+0000 7f612e020640 1 --2- 172.21.15.96:0/1086164990 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6128108940 0x7f61281a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.321+0000 7f612e020640 1 --2- 172.21.15.96:0/1086164990 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6128103c40 0x7f612819a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.321+0000 7f612e020640 1 --2- 172.21.15.96:0/1086164990 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f61140afaf0 0x7f61140b2000 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.321+0000 7f612e020640 1 --2- 172.21.15.96:0/1086164990 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6128102a00 0x7f612819a1f0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.321+0000 7f612e020640 1 -- 172.21.15.96:0/1086164990 >> 172.21.15.96:0/1086164990 conn(0x7f61280fdbf0 msgr2=0x7f6128109e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:50.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.321+0000 7f612e020640 1 -- 172.21.15.96:0/1086164990 shutdown_connections 2024-04-25T14:15:50.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.321+0000 7f612e020640 1 -- 172.21.15.96:0/1086164990 wait complete. 2024-04-25T14:15:50.332 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T14:15:50.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.441+0000 7f72625b7640 1 -- 172.21.15.96:0/1899232250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f725c102a00 msgr2=0x7f725c103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.441+0000 7f72625b7640 1 --2- 172.21.15.96:0/1899232250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f725c102a00 0x7f725c103700 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f724c009900 tx=0x7f724c02a410 comp rx=0 tx=0).stop 2024-04-25T14:15:50.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.441+0000 7f72625b7640 1 -- 172.21.15.96:0/1899232250 shutdown_connections 2024-04-25T14:15:50.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.441+0000 7f72625b7640 1 --2- 172.21.15.96:0/1899232250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f725c108940 0x7f725c10ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.441+0000 7f72625b7640 1 --2- 172.21.15.96:0/1899232250 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f725c103c40 0x7f725c104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.441+0000 7f72625b7640 1 --2- 172.21.15.96:0/1899232250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f725c102a00 0x7f725c103700 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.441+0000 7f72625b7640 1 -- 172.21.15.96:0/1899232250 >> 172.21.15.96:0/1899232250 conn(0x7f725c0fdbf0 msgr2=0x7f725c1000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:50.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.441+0000 7f72625b7640 1 -- 172.21.15.96:0/1899232250 shutdown_connections 2024-04-25T14:15:50.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.441+0000 7f72625b7640 1 -- 172.21.15.96:0/1899232250 wait complete. 2024-04-25T14:15:50.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.442+0000 7f72625b7640 1 Processor -- start 2024-04-25T14:15:50.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.442+0000 7f72625b7640 1 -- start start 2024-04-25T14:15:50.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.442+0000 7f72625b7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f725c102a00 0x7f725c075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.442+0000 7f72625b7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f725c103c40 0x7f725c19f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.442+0000 7f72625b7640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f725c108940 0x7f725c1a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.442+0000 7f72625b7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f725c10e500 con 0x7f725c103c40 2024-04-25T14:15:50.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.442+0000 7f72625b7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f725c10e380 con 0x7f725c102a00 2024-04-25T14:15:50.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.442+0000 7f72625b7640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f725c10e680 con 0x7f725c108940 2024-04-25T14:15:50.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.442+0000 7f725b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f725c103c40 0x7f725c19f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.442+0000 7f725b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f725c103c40 0x7f725c19f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50978/0 (socket says 172.21.15.96:50978) 2024-04-25T14:15:50.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.442+0000 7f725b7fe640 1 -- 172.21.15.96:0/3951358968 learned_addr learned my addr 172.21.15.96:0/3951358968 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:50.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.442+0000 7f7260b54640 1 --2- 172.21.15.96:0/3951358968 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f725c108940 0x7f725c1a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.442+0000 7f725bfff640 1 --2- 172.21.15.96:0/3951358968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f725c102a00 0x7f725c075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.443+0000 7f725b7fe640 1 -- 172.21.15.96:0/3951358968 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f725c108940 msgr2=0x7f725c1a15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.443+0000 7f725b7fe640 1 --2- 172.21.15.96:0/3951358968 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f725c108940 0x7f725c1a15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.443+0000 7f725b7fe640 1 -- 172.21.15.96:0/3951358968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f725c102a00 msgr2=0x7f725c075860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.443+0000 7f725b7fe640 1 --2- 172.21.15.96:0/3951358968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f725c102a00 0x7f725c075860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.443+0000 7f725b7fe640 1 -- 172.21.15.96:0/3951358968 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f725c1a1dc0 con 0x7f725c103c40 2024-04-25T14:15:50.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.443+0000 7f725bfff640 1 --2- 172.21.15.96:0/3951358968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f725c102a00 0x7f725c075860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:15:50.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.443+0000 7f725b7fe640 1 --2- 172.21.15.96:0/3951358968 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f725c103c40 0x7f725c19f030 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f725000c970 tx=0x7f725000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:50.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.443+0000 7f72597fa640 1 -- 172.21.15.96:0/3951358968 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7250007bb0 con 0x7f725c103c40 2024-04-25T14:15:50.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.443+0000 7f72625b7640 1 -- 172.21.15.96:0/3951358968 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f725c1a20b0 con 0x7f725c103c40 2024-04-25T14:15:50.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.443+0000 7f72597fa640 1 -- 172.21.15.96:0/3951358968 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7250007d50 con 0x7f725c103c40 2024-04-25T14:15:50.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.443+0000 7f72597fa640 1 -- 172.21.15.96:0/3951358968 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725000f450 con 0x7f725c103c40 2024-04-25T14:15:50.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.443+0000 7f72625b7640 1 -- 172.21.15.96:0/3951358968 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f725c1a2600 con 0x7f725c103c40 2024-04-25T14:15:50.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.445+0000 7f726946b740 1 Processor -- start 2024-04-25T14:15:50.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.445+0000 7f726946b740 1 -- start start 2024-04-25T14:15:50.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.445+0000 7f72597fa640 1 -- 172.21.15.96:0/3951358968 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f7250004f20 con 0x7f725c103c40 2024-04-25T14:15:50.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.446+0000 7f726946b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be70182390 0x55be70528c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.446+0000 7f726946b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be701594d0 0x55be705291a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.446+0000 7f726946b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be701616b0 0x55be70529e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.446+0000 7f726946b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55be70488d80 con 0x55be701616b0 2024-04-25T14:15:50.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.446+0000 7f726946b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55be70442a20 con 0x55be701594d0 2024-04-25T14:15:50.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.446+0000 7f726946b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55be70484d40 con 0x55be70182390 2024-04-25T14:15:50.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.446+0000 7f725b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be701616b0 0x55be70529e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.446+0000 7f725b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be701616b0 0x55be70529e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50994/0 (socket says 172.21.15.96:50994) 2024-04-25T14:15:50.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.446+0000 7f725b7fe640 1 -- 172.21.15.96:0/1900726485 learned_addr learned my addr 172.21.15.96:0/1900726485 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:50.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.446+0000 7f725bfff640 1 --2- 172.21.15.96:0/1900726485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be701594d0 0x55be705291a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.446+0000 7f7260b54640 1 --2- 172.21.15.96:0/1900726485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be70182390 0x55be70528c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.446+0000 7f725b7fe640 1 -- 172.21.15.96:0/1900726485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be70182390 msgr2=0x55be70528c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:50.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.446+0000 7f725b7fe640 1 --2- 172.21.15.96:0/1900726485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be70182390 0x55be70528c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.446+0000 7f725b7fe640 1 -- 172.21.15.96:0/1900726485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be701594d0 msgr2=0x55be705291a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:50.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.446+0000 7f725b7fe640 1 --2- 172.21.15.96:0/1900726485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be701594d0 0x55be705291a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.446+0000 7f72597fa640 1 --2- 172.21.15.96:0/3951358968 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f72440afaf0 0x7f72440b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.446+0000 7f725b7fe640 1 -- 172.21.15.96:0/1900726485 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be7044dc30 con 0x55be701616b0 2024-04-25T14:15:50.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.446+0000 7f725bfff640 1 --2- 172.21.15.96:0/1900726485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be701594d0 0x55be705291a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:15:50.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.446+0000 7f7260b54640 1 --2- 172.21.15.96:0/3951358968 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f72440afaf0 0x7f72440b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.446+0000 7f725b7fe640 1 --2- 172.21.15.96:0/1900726485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be701616b0 0x55be70529e30 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f72500a5b60 tx=0x7f72500a35a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce79c518ff122882 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:50.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.446+0000 7f72597fa640 1 -- 172.21.15.96:0/3951358968 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f72500de9d0 con 0x7f725c103c40 2024-04-25T14:15:50.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.447+0000 7f72427fc640 1 -- 172.21.15.96:0/1900726485 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725001d070 con 0x55be701616b0 2024-04-25T14:15:50.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.447+0000 7f72427fc640 1 -- 172.21.15.96:0/1900726485 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72500055f0 con 0x55be701616b0 2024-04-25T14:15:50.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.447+0000 7f72427fc640 1 -- 172.21.15.96:0/1900726485 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72500a67e0 con 0x55be701616b0 2024-04-25T14:15:50.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.447+0000 7f726946b740 1 -- 172.21.15.96:0/1900726485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be701616b0 msgr2=0x55be70529e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.447+0000 7f7260b54640 1 --2- 172.21.15.96:0/3951358968 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f72440afaf0 0x7f72440b2000 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x55be70529c70 tx=0x7f7254009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:50.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.447+0000 7f726946b740 1 --2- 172.21.15.96:0/1900726485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be701616b0 0x55be70529e30 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f72500a5b60 tx=0x7f72500a35a0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.448+0000 7f726946b740 1 -- 172.21.15.96:0/1900726485 shutdown_connections 2024-04-25T14:15:50.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.448+0000 7f726946b740 1 --2- 172.21.15.96:0/1900726485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be701616b0 0x55be70529e30 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.448+0000 7f726946b740 1 --2- 172.21.15.96:0/1900726485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be701594d0 0x55be705291a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.448+0000 7f726946b740 1 --2- 172.21.15.96:0/1900726485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be70182390 0x55be70528c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.448+0000 7f726946b740 1 -- 172.21.15.96:0/1900726485 >> 172.21.15.96:0/1900726485 conn(0x55be701764e0 msgr2=0x55be70520020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:50.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.448+0000 7f726946b740 1 -- 172.21.15.96:0/1900726485 shutdown_connections 2024-04-25T14:15:50.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.448+0000 7f726946b740 1 -- 172.21.15.96:0/1900726485 wait complete. 2024-04-25T14:15:50.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.448+0000 7f726946b740 1 -- start start 2024-04-25T14:15:50.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.448+0000 7f726946b740 1 Processor -- start 2024-04-25T14:15:50.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.448+0000 7f726946b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be701594d0 0x55be705329c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.448+0000 7f726946b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be701616b0 0x55be70532f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.448+0000 7f726946b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be70182390 0x55be70539df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.448+0000 7f726946b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55be7052a940 con 0x55be70182390 2024-04-25T14:15:50.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.448+0000 7f726946b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55be7040fdc0 con 0x55be701594d0 2024-04-25T14:15:50.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.448+0000 7f726946b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55be7052aac0 con 0x55be701616b0 2024-04-25T14:15:50.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.449+0000 7f726946b740 10 client.0 _ll_register_callbacks cb 0x55be7042b600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:15:50.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.449+0000 7f7260b54640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be70182390 0x55be70539df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.449+0000 7f7260b54640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be70182390 0x55be70539df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51006/0 (socket says 172.21.15.96:51006) 2024-04-25T14:15:50.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.449+0000 7f7260b54640 1 -- 172.21.15.96:0/3370316808 learned_addr learned my addr 172.21.15.96:0/3370316808 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:50.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.449+0000 7f725bfff640 1 --2- 172.21.15.96:0/3370316808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be701594d0 0x55be705329c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.449+0000 7f725b7fe640 1 --2- 172.21.15.96:0/3370316808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be701616b0 0x55be70532f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.449+0000 7f7260b54640 1 -- 172.21.15.96:0/3370316808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be701616b0 msgr2=0x55be70532f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:50.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.449+0000 7f7260b54640 1 --2- 172.21.15.96:0/3370316808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be701616b0 0x55be70532f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.449+0000 7f7260b54640 1 -- 172.21.15.96:0/3370316808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be701594d0 msgr2=0x55be705329c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:50.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.449+0000 7f7260b54640 1 --2- 172.21.15.96:0/3370316808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be701594d0 0x55be705329c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.449+0000 7f7260b54640 1 -- 172.21.15.96:0/3370316808 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be7019a3f0 con 0x55be70182390 2024-04-25T14:15:50.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.449+0000 7f725b7fe640 1 --2- 172.21.15.96:0/3370316808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be701616b0 0x55be70532f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:15:50.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.449+0000 7f7260b54640 1 --2- 172.21.15.96:0/3370316808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be70182390 0x55be70539df0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f72540042b0 tx=0x7f7254004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fec79057c4990f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:50.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.449+0000 7f723b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:15:50.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.449+0000 7f723b7fe640 1 -- 172.21.15.96:0/3370316808 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7254025070 con 0x55be70182390 2024-04-25T14:15:50.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.449+0000 7f723b7fe640 1 -- 172.21.15.96:0/3370316808 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f7254011d60 con 0x55be70182390 2024-04-25T14:15:50.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.449+0000 7f726946b740 1 -- 172.21.15.96:0/3370316808 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55be70496990 con 0x55be70182390 2024-04-25T14:15:50.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.449+0000 7f723b7fe640 1 -- 172.21.15.96:0/3370316808 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7254031050 con 0x55be70182390 2024-04-25T14:15:50.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.449+0000 7f726946b740 10 client.16278 fetch_fsmap learned FSMap version 26 2024-04-25T14:15:50.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.449+0000 7f726946b740 1 -- 172.21.15.96:0/3370316808 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55be702647d0 con 0x55be70182390 2024-04-25T14:15:50.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.450+0000 7f723b7fe640 1 -- 172.21.15.96:0/3370316808 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55be70496990 con 0x55be70182390 2024-04-25T14:15:50.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.450+0000 7f723b7fe640 1 -- 172.21.15.96:0/3370316808 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f725400f040 con 0x55be70182390 2024-04-25T14:15:50.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.450+0000 7f726946b740 10 client.16278 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:15:50.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.450+0000 7f726946b740 10 client.16278 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T14:15:50.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.450+0000 7f726946b740 20 client.16278 populate_metadata read hostname 'smithi096' 2024-04-25T14:15:50.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.450+0000 7f726946b740 1 --2- 172.21.15.96:0/3370316808 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x55be70140f60 0x55be705406e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.450+0000 7f726946b740 4 client.16278 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:15:50.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.450+0000 7f726946b740 1 -- 172.21.15.96:0/3370316808 --> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be70490800 con 0x55be70140f60 2024-04-25T14:15:50.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.450+0000 7f725bfff640 1 --2- 172.21.15.96:0/3370316808 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x55be70140f60 0x55be705406e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.451+0000 7f725bfff640 1 --2- 172.21.15.96:0/3370316808 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x55be70140f60 0x55be705406e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=880b2cbb8b2137ca server_cookie=259c4f13dba68003 in_seq=0 out_seq=0 2024-04-25T14:15:50.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.451+0000 7f723b7fe640 10 client.16278 ms_handle_connect on v2:172.21.15.97:6834/3872301726 2024-04-25T14:15:50.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.451+0000 7f723b7fe640 1 -- 172.21.15.96:0/3370316808 <== mds.3 v2:172.21.15.97:6834/3872301726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55be70490800 con 0x55be70140f60 2024-04-25T14:15:50.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.451+0000 7f723b7fe640 10 client.16278 handle_command_reply: tid=0 2024-04-25T14:15:50.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.451+0000 7f726946b740 1 client.16278 shutdown 2024-04-25T14:15:50.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.451+0000 7f726946b740 1 -- 172.21.15.96:0/3370316808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be70182390 msgr2=0x55be70539df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.451+0000 7f726946b740 1 --2- 172.21.15.96:0/3370316808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be70182390 0x55be70539df0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f72540042b0 tx=0x7f7254004a40 comp rx=0 tx=0).stop 2024-04-25T14:15:50.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.451+0000 7f726946b740 1 -- 172.21.15.96:0/3370316808 shutdown_connections 2024-04-25T14:15:50.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.451+0000 7f726946b740 1 --2- 172.21.15.96:0/3370316808 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x55be70140f60 0x55be705406e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.451+0000 7f726946b740 1 --2- 172.21.15.96:0/3370316808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be70182390 0x55be70539df0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.452+0000 7f726946b740 1 --2- 172.21.15.96:0/3370316808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be701616b0 0x55be70532f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.452+0000 7f726946b740 1 --2- 172.21.15.96:0/3370316808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be701594d0 0x55be705329c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.452+0000 7f723b7fe640 0 client.16278 ms_handle_reset on v2:172.21.15.97:6834/3872301726 2024-04-25T14:15:50.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.452+0000 7f726946b740 1 -- 172.21.15.96:0/3370316808 >> 172.21.15.96:0/3370316808 conn(0x55be701764e0 msgr2=0x55be7051f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:50.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.452+0000 7f726946b740 1 -- 172.21.15.96:0/3370316808 shutdown_connections 2024-04-25T14:15:50.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.452+0000 7f726946b740 1 -- 172.21.15.96:0/3370316808 wait complete. 2024-04-25T14:15:50.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.452+0000 7f726946b740 20 client.16278 trim_cache size 0 max 16384 2024-04-25T14:15:50.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.465+0000 7f726946b740 1 Processor -- start 2024-04-25T14:15:50.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.465+0000 7f726946b740 1 -- start start 2024-04-25T14:15:50.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.465+0000 7f726946b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be701594d0 0x55be7052d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.465+0000 7f726946b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be701616b0 0x55be7052dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.465+0000 7f726946b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be70182390 0x55be70544e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.465+0000 7f726946b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f725c1a92a0 con 0x55be70182390 2024-04-25T14:15:50.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.465+0000 7f726946b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f725c1aade0 con 0x55be701616b0 2024-04-25T14:15:50.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.465+0000 7f726946b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f725c1a9970 con 0x55be701594d0 2024-04-25T14:15:50.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.465+0000 7f7260b54640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be70182390 0x55be70544e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.465+0000 7f7260b54640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be70182390 0x55be70544e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51008/0 (socket says 172.21.15.96:51008) 2024-04-25T14:15:50.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.465+0000 7f7260b54640 1 -- 172.21.15.96:0/588616914 learned_addr learned my addr 172.21.15.96:0/588616914 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:50.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.466+0000 7f725b7fe640 1 --2- 172.21.15.96:0/588616914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be701616b0 0x55be7052dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.466+0000 7f7260b54640 1 -- 172.21.15.96:0/588616914 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be701594d0 msgr2=0x55be7052d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:50.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.466+0000 7f725bfff640 1 --2- 172.21.15.96:0/588616914 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be701594d0 0x55be7052d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.466+0000 7f7260b54640 1 --2- 172.21.15.96:0/588616914 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be701594d0 0x55be7052d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.466+0000 7f7260b54640 1 -- 172.21.15.96:0/588616914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be701616b0 msgr2=0x55be7052dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:50.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.466+0000 7f7260b54640 1 --2- 172.21.15.96:0/588616914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be701616b0 0x55be7052dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.466+0000 7f7260b54640 1 -- 172.21.15.96:0/588616914 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be7053e470 con 0x55be70182390 2024-04-25T14:15:50.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.466+0000 7f725bfff640 1 --2- 172.21.15.96:0/588616914 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be701594d0 0x55be7052d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:15:50.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.466+0000 7f7260b54640 1 --2- 172.21.15.96:0/588616914 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be70182390 0x55be70544e90 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f72540098b0 tx=0x7f7254005ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f43ce49bc335e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:50.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.466+0000 7f7241ffb640 1 -- 172.21.15.96:0/588616914 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7254025070 con 0x55be70182390 2024-04-25T14:15:50.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.466+0000 7f7241ffb640 1 -- 172.21.15.96:0/588616914 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7254011bb0 con 0x55be70182390 2024-04-25T14:15:50.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.466+0000 7f7241ffb640 1 -- 172.21.15.96:0/588616914 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725402a710 con 0x55be70182390 2024-04-25T14:15:50.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.467+0000 7f726946b740 1 -- 172.21.15.96:0/588616914 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be70182390 msgr2=0x55be70544e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.467+0000 7f726946b740 1 --2- 172.21.15.96:0/588616914 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be70182390 0x55be70544e90 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f72540098b0 tx=0x7f7254005ac0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.467+0000 7f726946b740 1 -- 172.21.15.96:0/588616914 shutdown_connections 2024-04-25T14:15:50.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.467+0000 7f726946b740 1 --2- 172.21.15.96:0/588616914 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be70182390 0x55be70544e90 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.467+0000 7f726946b740 1 --2- 172.21.15.96:0/588616914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be701616b0 0x55be7052dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.467+0000 7f726946b740 1 --2- 172.21.15.96:0/588616914 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be701594d0 0x55be7052d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.467+0000 7f726946b740 1 -- 172.21.15.96:0/588616914 >> 172.21.15.96:0/588616914 conn(0x55be701764e0 msgr2=0x55be70528ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:50.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.467+0000 7f726946b740 1 -- 172.21.15.96:0/588616914 shutdown_connections 2024-04-25T14:15:50.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.467+0000 7f726946b740 1 -- 172.21.15.96:0/588616914 wait complete. 2024-04-25T14:15:50.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.467+0000 7f726946b740 1 -- start start 2024-04-25T14:15:50.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.467+0000 7f726946b740 1 Processor -- start 2024-04-25T14:15:50.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.468+0000 7f726946b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be701594d0 0x55be705364f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.468+0000 7f726946b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be701616b0 0x55be70536a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.468+0000 7f726946b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be70182390 0x55be705a6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.468+0000 7f726946b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f725c1aa710 con 0x55be701616b0 2024-04-25T14:15:50.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.468+0000 7f726946b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f725c1aa040 con 0x55be701594d0 2024-04-25T14:15:50.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.468+0000 7f726946b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f725c1a7280 con 0x55be70182390 2024-04-25T14:15:50.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.468+0000 7f726946b740 10 client.0 _ll_register_callbacks cb 0x7f7254009b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:15:50.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.468+0000 7f725b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be701616b0 0x55be70536a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.468+0000 7f725b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be701616b0 0x55be70536a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51016/0 (socket says 172.21.15.96:51016) 2024-04-25T14:15:50.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.468+0000 7f725b7fe640 1 -- 172.21.15.96:0/937345978 learned_addr learned my addr 172.21.15.96:0/937345978 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:50.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.468+0000 7f725bfff640 1 --2- 172.21.15.96:0/937345978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be701594d0 0x55be705364f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.468+0000 7f725b7fe640 1 -- 172.21.15.96:0/937345978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be70182390 msgr2=0x55be705a6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:50.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.468+0000 7f725b7fe640 1 --2- 172.21.15.96:0/937345978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be70182390 0x55be705a6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.468+0000 7f725b7fe640 1 -- 172.21.15.96:0/937345978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be701594d0 msgr2=0x55be705364f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:50.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.468+0000 7f725b7fe640 1 --2- 172.21.15.96:0/937345978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be701594d0 0x55be705364f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.468+0000 7f725b7fe640 1 -- 172.21.15.96:0/937345978 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be705a7000 con 0x55be701616b0 2024-04-25T14:15:50.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.468+0000 7f725b7fe640 1 --2- 172.21.15.96:0/937345978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be701616b0 0x55be70536a30 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f72500a6e20 tx=0x7f7250004980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c0e82b5e92c0b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:50.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.468+0000 7f723b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:15:50.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.468+0000 7f723b7fe640 1 -- 172.21.15.96:0/937345978 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72500bc060 con 0x55be701616b0 2024-04-25T14:15:50.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.468+0000 7f723b7fe640 1 -- 172.21.15.96:0/937345978 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f72500a2800 con 0x55be701616b0 2024-04-25T14:15:50.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.468+0000 7f726946b740 1 -- 172.21.15.96:0/937345978 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7254011bb0 con 0x55be701616b0 2024-04-25T14:15:50.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.469+0000 7f723b7fe640 1 -- 172.21.15.96:0/937345978 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f725001d040 con 0x55be701616b0 2024-04-25T14:15:50.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.469+0000 7f726946b740 10 client.16284 fetch_fsmap learned FSMap version 26 2024-04-25T14:15:50.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.469+0000 7f726946b740 1 -- 172.21.15.96:0/937345978 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55be705a81f0 con 0x55be701616b0 2024-04-25T14:15:50.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.469+0000 7f723b7fe640 1 -- 172.21.15.96:0/937345978 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7254011bb0 con 0x55be701616b0 2024-04-25T14:15:50.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.469+0000 7f723b7fe640 1 -- 172.21.15.96:0/937345978 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7250005210 con 0x55be701616b0 2024-04-25T14:15:50.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.469+0000 7f726946b740 10 client.16284 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:15:50.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.469+0000 7f726946b740 10 client.16284 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T14:15:50.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.469+0000 7f726946b740 20 client.16284 populate_metadata read hostname 'smithi096' 2024-04-25T14:15:50.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.469+0000 7f726946b740 1 --2- 172.21.15.96:0/937345978 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x55be701163a0 0x55be705aa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.469+0000 7f726946b740 4 client.16284 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:15:50.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.469+0000 7f726946b740 1 -- 172.21.15.96:0/937345978 --> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55be7040b610 con 0x55be701163a0 2024-04-25T14:15:50.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.469+0000 7f725bfff640 1 --2- 172.21.15.96:0/937345978 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x55be701163a0 0x55be705aa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.469+0000 7f723b7fe640 10 client.16284 ms_handle_connect on v2:172.21.15.97:6834/3872301726 2024-04-25T14:15:50.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.469+0000 7f725bfff640 1 --2- 172.21.15.96:0/937345978 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x55be701163a0 0x55be705aa760 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=db195608cc5cead5 server_cookie=beddf5a840e10d2e in_seq=0 out_seq=0 2024-04-25T14:15:50.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.470+0000 7f723b7fe640 1 -- 172.21.15.96:0/937345978 <== mds.3 v2:172.21.15.97:6834/3872301726 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55be7040b610 con 0x55be701163a0 2024-04-25T14:15:50.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.470+0000 7f723b7fe640 10 client.16284 handle_command_reply: tid=0 2024-04-25T14:15:50.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.470+0000 7f726946b740 1 client.16284 shutdown 2024-04-25T14:15:50.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.470+0000 7f726946b740 1 -- 172.21.15.96:0/937345978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be701616b0 msgr2=0x55be70536a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.470+0000 7f726946b740 1 --2- 172.21.15.96:0/937345978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be701616b0 0x55be70536a30 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f72500a6e20 tx=0x7f7250004980 comp rx=0 tx=0).stop 2024-04-25T14:15:50.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.470+0000 7f726946b740 1 -- 172.21.15.96:0/937345978 shutdown_connections 2024-04-25T14:15:50.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.470+0000 7f726946b740 1 --2- 172.21.15.96:0/937345978 >> [v2:172.21.15.97:6834/3872301726,v1:172.21.15.97:6835/3872301726] conn(0x55be701163a0 0x55be705aa760 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.470+0000 7f726946b740 1 --2- 172.21.15.96:0/937345978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be70182390 0x55be705a6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.470+0000 7f723b7fe640 0 client.16284 ms_handle_reset on v2:172.21.15.97:6834/3872301726 2024-04-25T14:15:50.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.470+0000 7f726946b740 1 --2- 172.21.15.96:0/937345978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be701616b0 0x55be70536a30 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.470+0000 7f726946b740 1 --2- 172.21.15.96:0/937345978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be701594d0 0x55be705364f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.470+0000 7f726946b740 1 -- 172.21.15.96:0/937345978 >> 172.21.15.96:0/937345978 conn(0x55be701764e0 msgr2=0x55be7052d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:50.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.470+0000 7f726946b740 1 -- 172.21.15.96:0/937345978 shutdown_connections 2024-04-25T14:15:50.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.470+0000 7f726946b740 1 -- 172.21.15.96:0/937345978 wait complete. 2024-04-25T14:15:50.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.470+0000 7f726946b740 20 client.16284 trim_cache size 0 max 16384 2024-04-25T14:15:50.477 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-04-25T14:15:50.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.471+0000 7f72625b7640 1 -- 172.21.15.96:0/3951358968 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f72440afaf0 msgr2=0x7f72440b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.471+0000 7f72625b7640 1 --2- 172.21.15.96:0/3951358968 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f72440afaf0 0x7f72440b2000 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x55be70529c70 tx=0x7f7254009600 comp rx=0 tx=0).stop 2024-04-25T14:15:50.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.471+0000 7f72625b7640 1 -- 172.21.15.96:0/3951358968 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f725c103c40 msgr2=0x7f725c19f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.471+0000 7f72625b7640 1 --2- 172.21.15.96:0/3951358968 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f725c103c40 0x7f725c19f030 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f725000c970 tx=0x7f725000ce30 comp rx=0 tx=0).stop 2024-04-25T14:15:50.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.471+0000 7f72625b7640 1 -- 172.21.15.96:0/3951358968 shutdown_connections 2024-04-25T14:15:50.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.471+0000 7f72625b7640 1 --2- 172.21.15.96:0/3951358968 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f725c108940 0x7f725c1a15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.471+0000 7f72625b7640 1 --2- 172.21.15.96:0/3951358968 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f72440afaf0 0x7f72440b2000 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.471+0000 7f72625b7640 1 --2- 172.21.15.96:0/3951358968 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f725c103c40 0x7f725c19f030 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.471+0000 7f72625b7640 1 --2- 172.21.15.96:0/3951358968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f725c102a00 0x7f725c075860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.471+0000 7f72625b7640 1 -- 172.21.15.96:0/3951358968 >> 172.21.15.96:0/3951358968 conn(0x7f725c0fdbf0 msgr2=0x7f725c104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:50.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.471+0000 7f72625b7640 1 -- 172.21.15.96:0/3951358968 shutdown_connections 2024-04-25T14:15:50.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.471+0000 7f72625b7640 1 -- 172.21.15.96:0/3951358968 wait complete. 2024-04-25T14:15:50.482 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T14:15:50.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.593+0000 7fe041251640 1 -- 172.21.15.96:0/4239318699 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe03c1029e0 msgr2=0x7fe03c1036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.593+0000 7fe041251640 1 --2- 172.21.15.96:0/4239318699 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe03c1029e0 0x7fe03c1036e0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe02c009900 tx=0x7fe02c02a410 comp rx=0 tx=0).stop 2024-04-25T14:15:50.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.593+0000 7fe041251640 1 -- 172.21.15.96:0/4239318699 shutdown_connections 2024-04-25T14:15:50.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.593+0000 7fe041251640 1 --2- 172.21.15.96:0/4239318699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe03c108920 0x7fe03c10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.593+0000 7fe041251640 1 --2- 172.21.15.96:0/4239318699 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe03c103c20 0x7fe03c104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.593+0000 7fe041251640 1 --2- 172.21.15.96:0/4239318699 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe03c1029e0 0x7fe03c1036e0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.593+0000 7fe041251640 1 -- 172.21.15.96:0/4239318699 >> 172.21.15.96:0/4239318699 conn(0x7fe03c0fdbd0 msgr2=0x7fe03c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:50.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.593+0000 7fe041251640 1 -- 172.21.15.96:0/4239318699 shutdown_connections 2024-04-25T14:15:50.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.593+0000 7fe041251640 1 -- 172.21.15.96:0/4239318699 wait complete. 2024-04-25T14:15:50.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.594+0000 7fe041251640 1 Processor -- start 2024-04-25T14:15:50.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.594+0000 7fe041251640 1 -- start start 2024-04-25T14:15:50.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.594+0000 7fe041251640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe03c1029e0 0x7fe03c0ffc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.594+0000 7fe041251640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe03c103c20 0x7fe03c0fe310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.594+0000 7fe041251640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe03c108920 0x7fe03c0fe850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.594+0000 7fe041251640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe03c10e4e0 con 0x7fe03c108920 2024-04-25T14:15:50.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.594+0000 7fe041251640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe03c10e360 con 0x7fe03c1029e0 2024-04-25T14:15:50.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.594+0000 7fe041251640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe03c10e660 con 0x7fe03c103c20 2024-04-25T14:15:50.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.594+0000 7fe03b59e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe03c108920 0x7fe03c0fe850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.594+0000 7fe03b59e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe03c108920 0x7fe03c0fe850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51038/0 (socket says 172.21.15.96:51038) 2024-04-25T14:15:50.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.594+0000 7fe03b59e640 1 -- 172.21.15.96:0/3523621743 learned_addr learned my addr 172.21.15.96:0/3523621743 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:50.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.594+0000 7fe03ad9d640 1 --2- 172.21.15.96:0/3523621743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe03c1029e0 0x7fe03c0ffc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.594+0000 7fe033fff640 1 --2- 172.21.15.96:0/3523621743 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe03c103c20 0x7fe03c0fe310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.595+0000 7fe03b59e640 1 -- 172.21.15.96:0/3523621743 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe03c103c20 msgr2=0x7fe03c0fe310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.595+0000 7fe03b59e640 1 --2- 172.21.15.96:0/3523621743 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe03c103c20 0x7fe03c0fe310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.595+0000 7fe03b59e640 1 -- 172.21.15.96:0/3523621743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe03c1029e0 msgr2=0x7fe03c0ffc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.595+0000 7fe03b59e640 1 --2- 172.21.15.96:0/3523621743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe03c1029e0 0x7fe03c0ffc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.595+0000 7fe03b59e640 1 -- 172.21.15.96:0/3523621743 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe03c0fef80 con 0x7fe03c108920 2024-04-25T14:15:50.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.595+0000 7fe03b59e640 1 --2- 172.21.15.96:0/3523621743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe03c108920 0x7fe03c0fe850 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fe03400eaa0 tx=0x7fe03400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:50.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.597+0000 7fe038d99640 1 -- 172.21.15.96:0/3523621743 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03400cdb0 con 0x7fe03c108920 2024-04-25T14:15:50.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.597+0000 7fe038d99640 1 -- 172.21.15.96:0/3523621743 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe034005020 con 0x7fe03c108920 2024-04-25T14:15:50.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.597+0000 7fe041251640 1 -- 172.21.15.96:0/3523621743 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe03c1a4680 con 0x7fe03c108920 2024-04-25T14:15:50.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.597+0000 7fe038d99640 1 -- 172.21.15.96:0/3523621743 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0340116d0 con 0x7fe03c108920 2024-04-25T14:15:50.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.597+0000 7fe041251640 1 -- 172.21.15.96:0/3523621743 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe03c1a4af0 con 0x7fe03c108920 2024-04-25T14:15:50.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.600+0000 7fe038d99640 1 -- 172.21.15.96:0/3523621743 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fe0340051d0 con 0x7fe03c108920 2024-04-25T14:15:50.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.600+0000 7fe0481e9740 1 Processor -- start 2024-04-25T14:15:50.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.600+0000 7fe0481e9740 1 -- start start 2024-04-25T14:15:50.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.600+0000 7fe0481e9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e887621390 0x55e8879c3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.600+0000 7fe0481e9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e8875f84d0 0x55e8879c3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.600+0000 7fe0481e9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8876006b0 0x55e8879c4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.600+0000 7fe0481e9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e887927d80 con 0x55e8876006b0 2024-04-25T14:15:50.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.600+0000 7fe0481e9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e8878e1a20 con 0x55e887621390 2024-04-25T14:15:50.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.600+0000 7fe0481e9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e887923d40 con 0x55e8875f84d0 2024-04-25T14:15:50.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.600+0000 7fe033fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8876006b0 0x55e8879c4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.600+0000 7fe033fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8876006b0 0x55e8879c4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51046/0 (socket says 172.21.15.96:51046) 2024-04-25T14:15:50.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.600+0000 7fe033fff640 1 -- 172.21.15.96:0/1520231593 learned_addr learned my addr 172.21.15.96:0/1520231593 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:50.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.600+0000 7fe03ad9d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e8875f84d0 0x55e8879c3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.600+0000 7fe033fff640 1 --2- 172.21.15.96:0/1520231593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e887621390 0x55e8879c3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.601+0000 7fe038d99640 1 --2- 172.21.15.96:0/3523621743 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fe0280afaf0 0x7fe0280b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.601+0000 7fe033fff640 1 -- 172.21.15.96:0/1520231593 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e8875f84d0 msgr2=0x55e8879c3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:50.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.601+0000 7fe033fff640 1 --2- 172.21.15.96:0/1520231593 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e8875f84d0 0x55e8879c3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.601+0000 7fe033fff640 1 -- 172.21.15.96:0/1520231593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e887621390 msgr2=0x55e8879c3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:50.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.601+0000 7fe033fff640 1 --2- 172.21.15.96:0/1520231593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e887621390 0x55e8879c3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.601+0000 7fe033fff640 1 -- 172.21.15.96:0/1520231593 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8878ecc30 con 0x55e8876006b0 2024-04-25T14:15:50.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.601+0000 7fe03b59e640 1 --2- 172.21.15.96:0/3523621743 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fe0280afaf0 0x7fe0280b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.601+0000 7fe033fff640 1 --2- 172.21.15.96:0/1520231593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8876006b0 0x55e8879c4a20 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe024012770 tx=0x7fe02400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcb1c6cec422d7b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:50.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.601+0000 7fe038d99640 1 -- 172.21.15.96:0/3523621743 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fe0340dedb0 con 0x7fe03c108920 2024-04-25T14:15:50.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.601+0000 7fe0317fa640 1 -- 172.21.15.96:0/1520231593 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe024012cf0 con 0x55e8876006b0 2024-04-25T14:15:50.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.601+0000 7fe0317fa640 1 -- 172.21.15.96:0/1520231593 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe024005020 con 0x55e8876006b0 2024-04-25T14:15:50.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.601+0000 7fe0317fa640 1 -- 172.21.15.96:0/1520231593 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0240049c0 con 0x55e8876006b0 2024-04-25T14:15:50.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.602+0000 7fe0481e9740 1 -- 172.21.15.96:0/1520231593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8876006b0 msgr2=0x55e8879c4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.602+0000 7fe03b59e640 1 --2- 172.21.15.96:0/3523621743 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fe0280afaf0 0x7fe0280b2000 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe0340dcc00 tx=0x7fe034069040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:50.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.602+0000 7fe0481e9740 1 --2- 172.21.15.96:0/1520231593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8876006b0 0x55e8879c4a20 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe024012770 tx=0x7fe02400b7c0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.602+0000 7fe0481e9740 1 -- 172.21.15.96:0/1520231593 shutdown_connections 2024-04-25T14:15:50.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.602+0000 7fe0481e9740 1 --2- 172.21.15.96:0/1520231593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8876006b0 0x55e8879c4a20 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.602+0000 7fe0481e9740 1 --2- 172.21.15.96:0/1520231593 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e8875f84d0 0x55e8879c3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.602+0000 7fe0481e9740 1 --2- 172.21.15.96:0/1520231593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e887621390 0x55e8879c3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.602+0000 7fe0481e9740 1 -- 172.21.15.96:0/1520231593 >> 172.21.15.96:0/1520231593 conn(0x55e8876154e0 msgr2=0x55e8879bf020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:50.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.602+0000 7fe0481e9740 1 -- 172.21.15.96:0/1520231593 shutdown_connections 2024-04-25T14:15:50.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.602+0000 7fe0481e9740 1 -- 172.21.15.96:0/1520231593 wait complete. 2024-04-25T14:15:50.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.602+0000 7fe0481e9740 1 -- start start 2024-04-25T14:15:50.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.603+0000 7fe0481e9740 1 Processor -- start 2024-04-25T14:15:50.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.603+0000 7fe0481e9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e8875f84d0 0x55e8879cd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.603+0000 7fe0481e9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8876006b0 0x55e8879cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.603+0000 7fe0481e9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e887621390 0x55e8879d49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.603+0000 7fe0481e9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e8879c5530 con 0x55e887621390 2024-04-25T14:15:50.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.603+0000 7fe0481e9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e8878aedc0 con 0x55e8876006b0 2024-04-25T14:15:50.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.603+0000 7fe0481e9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e8879c56b0 con 0x55e8875f84d0 2024-04-25T14:15:50.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.603+0000 7fe0481e9740 10 client.0 _ll_register_callbacks cb 0x55e8878ca600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:15:50.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.603+0000 7fe033fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e887621390 0x55e8879d49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.603+0000 7fe033fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e887621390 0x55e8879d49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51052/0 (socket says 172.21.15.96:51052) 2024-04-25T14:15:50.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.603+0000 7fe033fff640 1 -- 172.21.15.96:0/2948532971 learned_addr learned my addr 172.21.15.96:0/2948532971 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:50.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.603+0000 7fe03ad9d640 1 --2- 172.21.15.96:0/2948532971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8876006b0 0x55e8879cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.603+0000 7fe033fff640 1 -- 172.21.15.96:0/2948532971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e8875f84d0 msgr2=0x55e8879cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:50.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.603+0000 7fe033fff640 1 --2- 172.21.15.96:0/2948532971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e8875f84d0 0x55e8879cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.603+0000 7fe033fff640 1 -- 172.21.15.96:0/2948532971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8876006b0 msgr2=0x55e8879cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:50.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.603+0000 7fe033fff640 1 --2- 172.21.15.96:0/2948532971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8876006b0 0x55e8879cda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.603+0000 7fe033fff640 1 -- 172.21.15.96:0/2948532971 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8876393f0 con 0x55e887621390 2024-04-25T14:15:50.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.603+0000 7fe033fff640 1 --2- 172.21.15.96:0/2948532971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e887621390 0x55e8879d49d0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe02400b390 tx=0x7fe02400b3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2ac2fc8a23d241a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:50.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.603+0000 7fe0227fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:15:50.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.604+0000 7fe0227fc640 1 -- 172.21.15.96:0/2948532971 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe024010940 con 0x55e887621390 2024-04-25T14:15:50.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.604+0000 7fe0227fc640 1 -- 172.21.15.96:0/2948532971 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fe024010ae0 con 0x55e887621390 2024-04-25T14:15:50.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.604+0000 7fe0481e9740 1 -- 172.21.15.96:0/2948532971 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe024005020 con 0x55e887621390 2024-04-25T14:15:50.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.604+0000 7fe0227fc640 1 -- 172.21.15.96:0/2948532971 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe024010d20 con 0x55e887621390 2024-04-25T14:15:50.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.604+0000 7fe0481e9740 10 client.16296 fetch_fsmap learned FSMap version 26 2024-04-25T14:15:50.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.604+0000 7fe0481e9740 1 -- 172.21.15.96:0/2948532971 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e8877037d0 con 0x55e887621390 2024-04-25T14:15:50.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.604+0000 7fe0227fc640 1 -- 172.21.15.96:0/2948532971 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe024005020 con 0x55e887621390 2024-04-25T14:15:50.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.604+0000 7fe0227fc640 1 -- 172.21.15.96:0/2948532971 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe0240275e0 con 0x55e887621390 2024-04-25T14:15:50.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.604+0000 7fe0481e9740 10 client.16296 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:15:50.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.604+0000 7fe0481e9740 10 client.16296 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:15:50.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.604+0000 7fe0481e9740 20 client.16296 populate_metadata read hostname 'smithi096' 2024-04-25T14:15:50.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.604+0000 7fe0481e9740 1 --2- 172.21.15.96:0/2948532971 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x55e8875dff60 0x55e8879dae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.604+0000 7fe0481e9740 4 client.16296 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:15:50.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.604+0000 7fe0481e9740 1 -- 172.21.15.96:0/2948532971 --> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e887935990 con 0x55e8875dff60 2024-04-25T14:15:50.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.605+0000 7fe03ad9d640 1 --2- 172.21.15.96:0/2948532971 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x55e8875dff60 0x55e8879dae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.605+0000 7fe0227fc640 10 client.16296 ms_handle_connect on v2:172.21.15.151:6834/1313823638 2024-04-25T14:15:50.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.605+0000 7fe03ad9d640 1 --2- 172.21.15.96:0/2948532971 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x55e8875dff60 0x55e8879dae80 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dd3cfb94a8616b7f server_cookie=6315321dfcc7848d in_seq=0 out_seq=0 2024-04-25T14:15:50.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.606+0000 7fe0227fc640 1 -- 172.21.15.96:0/2948532971 <== mds.2 v2:172.21.15.151:6834/1313823638 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e887935990 con 0x55e8875dff60 2024-04-25T14:15:50.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.606+0000 7fe0227fc640 10 client.16296 handle_command_reply: tid=0 2024-04-25T14:15:50.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.606+0000 7fe0481e9740 1 client.16296 shutdown 2024-04-25T14:15:50.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.606+0000 7fe0481e9740 1 -- 172.21.15.96:0/2948532971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e887621390 msgr2=0x55e8879d49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.606+0000 7fe0481e9740 1 --2- 172.21.15.96:0/2948532971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e887621390 0x55e8879d49d0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe02400b390 tx=0x7fe02400b3c0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.606+0000 7fe0481e9740 1 -- 172.21.15.96:0/2948532971 shutdown_connections 2024-04-25T14:15:50.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.606+0000 7fe0481e9740 1 --2- 172.21.15.96:0/2948532971 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x55e8875dff60 0x55e8879dae80 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.606+0000 7fe0481e9740 1 --2- 172.21.15.96:0/2948532971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e887621390 0x55e8879d49d0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.606+0000 7fe0227fc640 0 client.16296 ms_handle_reset on v2:172.21.15.151:6834/1313823638 2024-04-25T14:15:50.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.606+0000 7fe0481e9740 1 --2- 172.21.15.96:0/2948532971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8876006b0 0x55e8879cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.606+0000 7fe0481e9740 1 --2- 172.21.15.96:0/2948532971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e8875f84d0 0x55e8879cd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.606+0000 7fe0481e9740 1 -- 172.21.15.96:0/2948532971 >> 172.21.15.96:0/2948532971 conn(0x55e8876154e0 msgr2=0x55e8879c1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:50.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.606+0000 7fe0481e9740 1 -- 172.21.15.96:0/2948532971 shutdown_connections 2024-04-25T14:15:50.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.606+0000 7fe0481e9740 1 -- 172.21.15.96:0/2948532971 wait complete. 2024-04-25T14:15:50.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.606+0000 7fe0481e9740 20 client.16296 trim_cache size 0 max 16384 2024-04-25T14:15:50.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.619+0000 7fe0481e9740 1 Processor -- start 2024-04-25T14:15:50.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.619+0000 7fe0481e9740 1 -- start start 2024-04-25T14:15:50.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.620+0000 7fe0481e9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e8875dff60 0x55e8879b9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.620+0000 7fe0481e9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8875f84d0 0x55e8879bea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.620+0000 7fe0481e9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8876006b0 0x55e8879bef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.620+0000 7fe0481e9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe03c1ad640 con 0x55e8875f84d0 2024-04-25T14:15:50.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.620+0000 7fe0481e9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe03c1af180 con 0x55e8876006b0 2024-04-25T14:15:50.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.620+0000 7fe0481e9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe03c1add10 con 0x55e8875dff60 2024-04-25T14:15:50.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.620+0000 7fe03ad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8875f84d0 0x55e8879bea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.620+0000 7fe03ad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8875f84d0 0x55e8879bea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51062/0 (socket says 172.21.15.96:51062) 2024-04-25T14:15:50.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.620+0000 7fe03ad9d640 1 -- 172.21.15.96:0/2097359358 learned_addr learned my addr 172.21.15.96:0/2097359358 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:50.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.620+0000 7fe033fff640 1 --2- 172.21.15.96:0/2097359358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e8875dff60 0x55e8879b9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.620+0000 7fe033fff640 1 --2- 172.21.15.96:0/2097359358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8876006b0 0x55e8879bef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.620+0000 7fe03ad9d640 1 -- 172.21.15.96:0/2097359358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e8875dff60 msgr2=0x55e8879b9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:50.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.620+0000 7fe03ad9d640 1 --2- 172.21.15.96:0/2097359358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e8875dff60 0x55e8879b9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.620+0000 7fe03ad9d640 1 -- 172.21.15.96:0/2097359358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8876006b0 msgr2=0x55e8879bef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:50.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.620+0000 7fe03ad9d640 1 --2- 172.21.15.96:0/2097359358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8876006b0 0x55e8879bef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.620+0000 7fe03ad9d640 1 -- 172.21.15.96:0/2097359358 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8879ce130 con 0x55e8875f84d0 2024-04-25T14:15:50.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.620+0000 7fe033fff640 1 --2- 172.21.15.96:0/2097359358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e8875dff60 0x55e8879b9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:15:50.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.620+0000 7fe03ad9d640 1 --2- 172.21.15.96:0/2097359358 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8875f84d0 0x55e8879bea50 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fe02c031500 tx=0x7fe02c031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a4ce635d4093ec6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:50.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.620+0000 7fe030ff9640 1 -- 172.21.15.96:0/2097359358 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02c035040 con 0x55e8875f84d0 2024-04-25T14:15:50.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.621+0000 7fe030ff9640 1 -- 172.21.15.96:0/2097359358 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe02c04a070 con 0x55e8875f84d0 2024-04-25T14:15:50.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.621+0000 7fe030ff9640 1 -- 172.21.15.96:0/2097359358 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02c0035f0 con 0x55e8875f84d0 2024-04-25T14:15:50.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.621+0000 7fe0481e9740 1 -- 172.21.15.96:0/2097359358 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8875f84d0 msgr2=0x55e8879bea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.621+0000 7fe0481e9740 1 --2- 172.21.15.96:0/2097359358 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8875f84d0 0x55e8879bea50 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fe02c031500 tx=0x7fe02c031400 comp rx=0 tx=0).stop 2024-04-25T14:15:50.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.621+0000 7fe0481e9740 1 -- 172.21.15.96:0/2097359358 shutdown_connections 2024-04-25T14:15:50.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.621+0000 7fe0481e9740 1 --2- 172.21.15.96:0/2097359358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8876006b0 0x55e8879bef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.621+0000 7fe0481e9740 1 --2- 172.21.15.96:0/2097359358 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8875f84d0 0x55e8879bea50 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.621+0000 7fe0481e9740 1 --2- 172.21.15.96:0/2097359358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e8875dff60 0x55e8879b9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.621+0000 7fe0481e9740 1 -- 172.21.15.96:0/2097359358 >> 172.21.15.96:0/2097359358 conn(0x55e8876154e0 msgr2=0x55e8879c8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:50.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.621+0000 7fe0481e9740 1 -- 172.21.15.96:0/2097359358 shutdown_connections 2024-04-25T14:15:50.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.621+0000 7fe0481e9740 1 -- 172.21.15.96:0/2097359358 wait complete. 2024-04-25T14:15:50.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.622+0000 7fe0481e9740 1 -- start start 2024-04-25T14:15:50.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.622+0000 7fe0481e9740 1 Processor -- start 2024-04-25T14:15:50.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.622+0000 7fe0481e9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e8875dff60 0x55e8879d1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.622+0000 7fe0481e9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8875f84d0 0x55e8879d1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.622+0000 7fe0481e9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8876006b0 0x55e8879d2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.622+0000 7fe0481e9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe03c1aeab0 con 0x55e8876006b0 2024-04-25T14:15:50.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.622+0000 7fe0481e9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe03c1ae3e0 con 0x55e8875f84d0 2024-04-25T14:15:50.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.622+0000 7fe0481e9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe03c1ab620 con 0x55e8875dff60 2024-04-25T14:15:50.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.622+0000 7fe0481e9740 10 client.0 _ll_register_callbacks cb 0x7fe02400be70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:15:50.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.622+0000 7fe033fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8876006b0 0x55e8879d2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.622+0000 7fe033fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8876006b0 0x55e8879d2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51076/0 (socket says 172.21.15.96:51076) 2024-04-25T14:15:50.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.622+0000 7fe033fff640 1 -- 172.21.15.96:0/271180398 learned_addr learned my addr 172.21.15.96:0/271180398 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:50.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.622+0000 7fe03ad9d640 1 --2- 172.21.15.96:0/271180398 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8875f84d0 0x55e8879d1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.622+0000 7fe033fff640 1 --2- 172.21.15.96:0/271180398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e8875dff60 0x55e8879d1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.622+0000 7fe033fff640 1 -- 172.21.15.96:0/271180398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e8875dff60 msgr2=0x55e8879d1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:50.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.622+0000 7fe033fff640 1 --2- 172.21.15.96:0/271180398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e8875dff60 0x55e8879d1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.622+0000 7fe033fff640 1 -- 172.21.15.96:0/271180398 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8875f84d0 msgr2=0x55e8879d1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:50.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.622+0000 7fe033fff640 1 --2- 172.21.15.96:0/271180398 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8875f84d0 0x55e8879d1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.622+0000 7fe033fff640 1 -- 172.21.15.96:0/271180398 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8879d3090 con 0x55e8876006b0 2024-04-25T14:15:50.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.623+0000 7fe033fff640 1 --2- 172.21.15.96:0/271180398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8876006b0 0x55e8879d2ad0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe024027f20 tx=0x7fe02400b3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb092e66436bbe0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:50.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.623+0000 7fe0227fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:15:50.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.623+0000 7fe0227fc640 1 -- 172.21.15.96:0/271180398 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02400ba90 con 0x55e8876006b0 2024-04-25T14:15:50.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.623+0000 7fe0227fc640 1 -- 172.21.15.96:0/271180398 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fe024027450 con 0x55e8876006b0 2024-04-25T14:15:50.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.623+0000 7fe0481e9740 1 -- 172.21.15.96:0/271180398 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe02c04a070 con 0x55e8876006b0 2024-04-25T14:15:50.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.623+0000 7fe0227fc640 1 -- 172.21.15.96:0/271180398 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe024041050 con 0x55e8876006b0 2024-04-25T14:15:50.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.623+0000 7fe0481e9740 10 client.16302 fetch_fsmap learned FSMap version 26 2024-04-25T14:15:50.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.623+0000 7fe0481e9740 1 -- 172.21.15.96:0/271180398 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e8879d42f0 con 0x55e8876006b0 2024-04-25T14:15:50.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.623+0000 7fe0227fc640 1 -- 172.21.15.96:0/271180398 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe02c04a070 con 0x55e8876006b0 2024-04-25T14:15:50.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.623+0000 7fe0227fc640 1 -- 172.21.15.96:0/271180398 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe024003960 con 0x55e8876006b0 2024-04-25T14:15:50.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.623+0000 7fe0481e9740 10 client.16302 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:15:50.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.623+0000 7fe0481e9740 10 client.16302 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:15:50.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.623+0000 7fe0481e9740 20 client.16302 populate_metadata read hostname 'smithi096' 2024-04-25T14:15:50.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.623+0000 7fe0481e9740 1 --2- 172.21.15.96:0/271180398 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x55e887621390 0x55e887a458e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.623+0000 7fe0481e9740 4 client.16302 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:15:50.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.623+0000 7fe0481e9740 1 -- 172.21.15.96:0/271180398 --> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e88792f800 con 0x55e887621390 2024-04-25T14:15:50.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.624+0000 7fe03ad9d640 1 --2- 172.21.15.96:0/271180398 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x55e887621390 0x55e887a458e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.624+0000 7fe0227fc640 10 client.16302 ms_handle_connect on v2:172.21.15.151:6834/1313823638 2024-04-25T14:15:50.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.624+0000 7fe03ad9d640 1 --2- 172.21.15.96:0/271180398 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x55e887621390 0x55e887a458e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cfb0eb4b918f0bda server_cookie=4abee73d4b8461e6 in_seq=0 out_seq=0 2024-04-25T14:15:50.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.624+0000 7fe0227fc640 1 -- 172.21.15.96:0/271180398 <== mds.2 v2:172.21.15.151:6834/1313823638 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e88792f800 con 0x55e887621390 2024-04-25T14:15:50.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.624+0000 7fe0227fc640 10 client.16302 handle_command_reply: tid=0 2024-04-25T14:15:50.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.624+0000 7fe0481e9740 1 client.16302 shutdown 2024-04-25T14:15:50.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.624+0000 7fe0481e9740 1 -- 172.21.15.96:0/271180398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8876006b0 msgr2=0x55e8879d2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.624+0000 7fe0481e9740 1 --2- 172.21.15.96:0/271180398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8876006b0 0x55e8879d2ad0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe024027f20 tx=0x7fe02400b3c0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.625+0000 7fe0481e9740 1 -- 172.21.15.96:0/271180398 shutdown_connections 2024-04-25T14:15:50.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.625+0000 7fe0481e9740 1 --2- 172.21.15.96:0/271180398 >> [v2:172.21.15.151:6834/1313823638,v1:172.21.15.151:6835/1313823638] conn(0x55e887621390 0x55e887a458e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.625+0000 7fe0481e9740 1 --2- 172.21.15.96:0/271180398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8876006b0 0x55e8879d2ad0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.625+0000 7fe0227fc640 0 client.16302 ms_handle_reset on v2:172.21.15.151:6834/1313823638 2024-04-25T14:15:50.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.625+0000 7fe0481e9740 1 --2- 172.21.15.96:0/271180398 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8875f84d0 0x55e8879d1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.625+0000 7fe0481e9740 1 --2- 172.21.15.96:0/271180398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e8875dff60 0x55e8879d1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.625+0000 7fe0481e9740 1 -- 172.21.15.96:0/271180398 >> 172.21.15.96:0/271180398 conn(0x55e8876154e0 msgr2=0x55e8879c1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:50.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.625+0000 7fe0481e9740 1 -- 172.21.15.96:0/271180398 shutdown_connections 2024-04-25T14:15:50.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.625+0000 7fe0481e9740 1 -- 172.21.15.96:0/271180398 wait complete. 2024-04-25T14:15:50.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.625+0000 7fe0481e9740 20 client.16302 trim_cache size 0 max 16384 2024-04-25T14:15:50.634 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-04-25T14:15:50.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.625+0000 7fe041251640 1 -- 172.21.15.96:0/3523621743 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fe0280afaf0 msgr2=0x7fe0280b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.625+0000 7fe041251640 1 --2- 172.21.15.96:0/3523621743 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fe0280afaf0 0x7fe0280b2000 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe0340dcc00 tx=0x7fe034069040 comp rx=0 tx=0).stop 2024-04-25T14:15:50.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.625+0000 7fe041251640 1 -- 172.21.15.96:0/3523621743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe03c108920 msgr2=0x7fe03c0fe850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.625+0000 7fe041251640 1 --2- 172.21.15.96:0/3523621743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe03c108920 0x7fe03c0fe850 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fe03400eaa0 tx=0x7fe03400ef60 comp rx=0 tx=0).stop 2024-04-25T14:15:50.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.625+0000 7fe041251640 1 -- 172.21.15.96:0/3523621743 shutdown_connections 2024-04-25T14:15:50.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.625+0000 7fe041251640 1 --2- 172.21.15.96:0/3523621743 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fe0280afaf0 0x7fe0280b2000 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.625+0000 7fe041251640 1 --2- 172.21.15.96:0/3523621743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe03c108920 0x7fe03c0fe850 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.625+0000 7fe041251640 1 --2- 172.21.15.96:0/3523621743 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe03c103c20 0x7fe03c0fe310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.625+0000 7fe041251640 1 --2- 172.21.15.96:0/3523621743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe03c1029e0 0x7fe03c0ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.625+0000 7fe041251640 1 -- 172.21.15.96:0/3523621743 >> 172.21.15.96:0/3523621743 conn(0x7fe03c0fdbd0 msgr2=0x7fe03c069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:50.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.625+0000 7fe041251640 1 -- 172.21.15.96:0/3523621743 shutdown_connections 2024-04-25T14:15:50.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.625+0000 7fe041251640 1 -- 172.21.15.96:0/3523621743 wait complete. 2024-04-25T14:15:50.637 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T14:15:50.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.747+0000 7f64b1db2640 1 -- 172.21.15.96:0/2963594471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64ac108a30 msgr2=0x7f64ac10ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.747+0000 7f64b1db2640 1 --2- 172.21.15.96:0/2963594471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64ac108a30 0x7f64ac10ae50 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f64a4009ca0 tx=0x7f64a402a8c0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.747+0000 7f64b1db2640 1 -- 172.21.15.96:0/2963594471 shutdown_connections 2024-04-25T14:15:50.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.747+0000 7f64b1db2640 1 --2- 172.21.15.96:0/2963594471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64ac108a30 0x7f64ac10ae50 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.747+0000 7f64b1db2640 1 --2- 172.21.15.96:0/2963594471 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64ac1043e0 0x7f64ac102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.747+0000 7f64b1db2640 1 --2- 172.21.15.96:0/2963594471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64ac103f00 0x7f64ac1027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.747+0000 7f64b1db2640 1 -- 172.21.15.96:0/2963594471 >> 172.21.15.96:0/2963594471 conn(0x7f64ac0fd380 msgr2=0x7f64ac0ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:50.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.747+0000 7f64b1db2640 1 -- 172.21.15.96:0/2963594471 shutdown_connections 2024-04-25T14:15:50.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.747+0000 7f64b1db2640 1 -- 172.21.15.96:0/2963594471 wait complete. 2024-04-25T14:15:50.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.748+0000 7f64b1db2640 1 Processor -- start 2024-04-25T14:15:50.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.748+0000 7f64b1db2640 1 -- start start 2024-04-25T14:15:50.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.748+0000 7f64b1db2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64ac103f00 0x7f64ac102160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.748+0000 7f64b1db2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64ac108a30 0x7f64ac19eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.748+0000 7f64b1db2640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64ac19f0b0 0x7f64ac1a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.748+0000 7f64b1db2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac10e520 con 0x7f64ac103f00 2024-04-25T14:15:50.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.748+0000 7f64b1db2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac10e3a0 con 0x7f64ac108a30 2024-04-25T14:15:50.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.748+0000 7f64b1db2640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac10e6a0 con 0x7f64ac19f0b0 2024-04-25T14:15:50.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.748+0000 7f64ab7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64ac103f00 0x7f64ac102160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.748+0000 7f64ab7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64ac103f00 0x7f64ac102160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51082/0 (socket says 172.21.15.96:51082) 2024-04-25T14:15:50.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.748+0000 7f64ab7fe640 1 -- 172.21.15.96:0/2298948849 learned_addr learned my addr 172.21.15.96:0/2298948849 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:50.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.748+0000 7f64abfff640 1 --2- 172.21.15.96:0/2298948849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64ac19f0b0 0x7f64ac1a14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.748+0000 7f64aaffd640 1 --2- 172.21.15.96:0/2298948849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64ac108a30 0x7f64ac19eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.748+0000 7f64ab7fe640 1 -- 172.21.15.96:0/2298948849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64ac19f0b0 msgr2=0x7f64ac1a14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.748+0000 7f64ab7fe640 1 --2- 172.21.15.96:0/2298948849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64ac19f0b0 0x7f64ac1a14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.748+0000 7f64ab7fe640 1 -- 172.21.15.96:0/2298948849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64ac108a30 msgr2=0x7f64ac19eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.748+0000 7f64ab7fe640 1 --2- 172.21.15.96:0/2298948849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64ac108a30 0x7f64ac19eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.748+0000 7f64ab7fe640 1 -- 172.21.15.96:0/2298948849 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64ac1a1d10 con 0x7f64ac103f00 2024-04-25T14:15:50.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.748+0000 7f64abfff640 1 --2- 172.21.15.96:0/2298948849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64ac19f0b0 0x7f64ac1a14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:15:50.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.748+0000 7f64aaffd640 1 --2- 172.21.15.96:0/2298948849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64ac108a30 0x7f64ac19eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:15:50.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.749+0000 7f64ab7fe640 1 --2- 172.21.15.96:0/2298948849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64ac103f00 0x7f64ac102160 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f649c00d770 tx=0x7f649c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:50.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.749+0000 7f64a8ff9640 1 -- 172.21.15.96:0/2298948849 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649c017070 con 0x7f64ac103f00 2024-04-25T14:15:50.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.749+0000 7f64a8ff9640 1 -- 172.21.15.96:0/2298948849 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f649c009c30 con 0x7f64ac103f00 2024-04-25T14:15:50.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.749+0000 7f64b1db2640 1 -- 172.21.15.96:0/2298948849 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64ac1a1fa0 con 0x7f64ac103f00 2024-04-25T14:15:50.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.749+0000 7f64a8ff9640 1 -- 172.21.15.96:0/2298948849 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649c00f620 con 0x7f64ac103f00 2024-04-25T14:15:50.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.749+0000 7f64b1db2640 1 -- 172.21.15.96:0/2298948849 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64ac1a2460 con 0x7f64ac103f00 2024-04-25T14:15:50.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.751+0000 7f64b8c5e740 1 Processor -- start 2024-04-25T14:15:50.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.751+0000 7f64b8c5e740 1 -- start start 2024-04-25T14:15:50.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.752+0000 7f64a8ff9640 1 -- 172.21.15.96:0/2298948849 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f649c01a070 con 0x7f64ac103f00 2024-04-25T14:15:50.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.752+0000 7f64b8c5e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b1c4a5390 0x557b1c847810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.752+0000 7f64b8c5e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b1c47c4d0 0x557b1c847d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.752+0000 7f64b8c5e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b1c4846b0 0x557b1c848a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.752+0000 7f64b8c5e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557b1c7abd80 con 0x557b1c4a5390 2024-04-25T14:15:50.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.752+0000 7f64b8c5e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557b1c765a20 con 0x557b1c47c4d0 2024-04-25T14:15:50.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.752+0000 7f64b8c5e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557b1c7a7d40 con 0x557b1c4846b0 2024-04-25T14:15:50.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.752+0000 7f64abfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b1c4a5390 0x557b1c847810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.752+0000 7f64abfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b1c4a5390 0x557b1c847810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51084/0 (socket says 172.21.15.96:51084) 2024-04-25T14:15:50.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.752+0000 7f64abfff640 1 -- 172.21.15.96:0/239257023 learned_addr learned my addr 172.21.15.96:0/239257023 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:50.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.752+0000 7f64aaffd640 1 --2- 172.21.15.96:0/239257023 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b1c4846b0 0x557b1c848a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.752+0000 7f64ab7fe640 1 --2- 172.21.15.96:0/239257023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b1c47c4d0 0x557b1c847d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.752+0000 7f64abfff640 1 -- 172.21.15.96:0/239257023 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b1c4846b0 msgr2=0x557b1c848a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:50.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.752+0000 7f64abfff640 1 --2- 172.21.15.96:0/239257023 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b1c4846b0 0x557b1c848a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.752+0000 7f64abfff640 1 -- 172.21.15.96:0/239257023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b1c47c4d0 msgr2=0x557b1c847d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:50.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.752+0000 7f64abfff640 1 --2- 172.21.15.96:0/239257023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b1c47c4d0 0x557b1c847d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.752+0000 7f64abfff640 1 -- 172.21.15.96:0/239257023 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b1c770c30 con 0x557b1c4a5390 2024-04-25T14:15:50.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.752+0000 7f64ab7fe640 1 --2- 172.21.15.96:0/239257023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b1c47c4d0 0x557b1c847d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:15:50.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.752+0000 7f64a8ff9640 1 --2- 172.21.15.96:0/2298948849 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f64940afaf0 0x7f64940b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.752+0000 7f64abfff640 1 --2- 172.21.15.96:0/239257023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b1c4a5390 0x557b1c847810 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f64a402f440 tx=0x7f64a402f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5754154b456cd04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:50.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.752+0000 7f64ab7fe640 1 --2- 172.21.15.96:0/2298948849 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f64940afaf0 0x7f64940b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.753+0000 7f6491ffb640 1 -- 172.21.15.96:0/239257023 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a4031650 con 0x557b1c4a5390 2024-04-25T14:15:50.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.753+0000 7f64a8ff9640 1 -- 172.21.15.96:0/2298948849 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f649c0de7f0 con 0x7f64ac103f00 2024-04-25T14:15:50.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.753+0000 7f6491ffb640 1 -- 172.21.15.96:0/239257023 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64a4036040 con 0x557b1c4a5390 2024-04-25T14:15:50.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.753+0000 7f6491ffb640 1 -- 172.21.15.96:0/239257023 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a4004710 con 0x557b1c4a5390 2024-04-25T14:15:50.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.753+0000 7f64b8c5e740 1 -- 172.21.15.96:0/239257023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b1c4a5390 msgr2=0x557b1c847810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.753+0000 7f64b8c5e740 1 --2- 172.21.15.96:0/239257023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b1c4a5390 0x557b1c847810 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f64a402f440 tx=0x7f64a402f470 comp rx=0 tx=0).stop 2024-04-25T14:15:50.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.753+0000 7f64ab7fe640 1 --2- 172.21.15.96:0/2298948849 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f64940afaf0 0x7f64940b2000 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x557b1c847530 tx=0x7f649c00de80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:50.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.754+0000 7f64b8c5e740 1 -- 172.21.15.96:0/239257023 shutdown_connections 2024-04-25T14:15:50.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.754+0000 7f64b8c5e740 1 --2- 172.21.15.96:0/239257023 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b1c4846b0 0x557b1c848a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.754+0000 7f64b8c5e740 1 --2- 172.21.15.96:0/239257023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b1c47c4d0 0x557b1c847d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.754+0000 7f64b8c5e740 1 --2- 172.21.15.96:0/239257023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b1c4a5390 0x557b1c847810 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.754+0000 7f64b8c5e740 1 -- 172.21.15.96:0/239257023 >> 172.21.15.96:0/239257023 conn(0x557b1c4994e0 msgr2=0x557b1c843020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:50.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.754+0000 7f64b8c5e740 1 -- 172.21.15.96:0/239257023 shutdown_connections 2024-04-25T14:15:50.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.754+0000 7f64b8c5e740 1 -- 172.21.15.96:0/239257023 wait complete. 2024-04-25T14:15:50.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.754+0000 7f64b8c5e740 1 -- start start 2024-04-25T14:15:50.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.754+0000 7f64b8c5e740 1 Processor -- start 2024-04-25T14:15:50.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.754+0000 7f64b8c5e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b1c47c4d0 0x557b1c851530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.754+0000 7f64b8c5e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b1c4846b0 0x557b1c851a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.755+0000 7f64b8c5e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b1c4a5390 0x557b1c8589d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.755+0000 7f64b8c5e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557b1c8494d0 con 0x557b1c4846b0 2024-04-25T14:15:50.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.755+0000 7f64b8c5e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557b1c732dc0 con 0x557b1c47c4d0 2024-04-25T14:15:50.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.755+0000 7f64b8c5e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557b1c849650 con 0x557b1c4a5390 2024-04-25T14:15:50.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.755+0000 7f64b8c5e740 10 client.0 _ll_register_callbacks cb 0x557b1c74e600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:15:50.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.755+0000 7f64aaffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b1c4846b0 0x557b1c851a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.755+0000 7f64abfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b1c47c4d0 0x557b1c851530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.755+0000 7f64aaffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b1c4846b0 0x557b1c851a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51096/0 (socket says 172.21.15.96:51096) 2024-04-25T14:15:50.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.755+0000 7f64aaffd640 1 -- 172.21.15.96:0/275887843 learned_addr learned my addr 172.21.15.96:0/275887843 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:50.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.755+0000 7f64abfff640 1 --2- 172.21.15.96:0/275887843 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b1c4a5390 0x557b1c8589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.755+0000 7f64aaffd640 1 -- 172.21.15.96:0/275887843 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b1c4a5390 msgr2=0x557b1c8589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:50.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.755+0000 7f64aaffd640 1 --2- 172.21.15.96:0/275887843 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b1c4a5390 0x557b1c8589d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.755+0000 7f64aaffd640 1 -- 172.21.15.96:0/275887843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b1c47c4d0 msgr2=0x557b1c851530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:50.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.755+0000 7f64aaffd640 1 --2- 172.21.15.96:0/275887843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b1c47c4d0 0x557b1c851530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.755+0000 7f64aaffd640 1 -- 172.21.15.96:0/275887843 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b1c4bd3f0 con 0x557b1c4846b0 2024-04-25T14:15:50.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.755+0000 7f64aaffd640 1 --2- 172.21.15.96:0/275887843 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b1c4846b0 0x557b1c851a70 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f64a000ba60 tx=0x7f64a000bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15dbb8bf410dbdee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:50.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.755+0000 7f648affd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:15:50.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.755+0000 7f648affd640 1 -- 172.21.15.96:0/275887843 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a001b640 con 0x557b1c4846b0 2024-04-25T14:15:50.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.755+0000 7f648affd640 1 -- 172.21.15.96:0/275887843 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f64a001b7e0 con 0x557b1c4846b0 2024-04-25T14:15:50.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.755+0000 7f64b8c5e740 1 -- 172.21.15.96:0/275887843 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64a4036040 con 0x557b1c4846b0 2024-04-25T14:15:50.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.755+0000 7f648affd640 1 -- 172.21.15.96:0/275887843 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f64a001b9e0 con 0x557b1c4846b0 2024-04-25T14:15:50.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.755+0000 7f64b8c5e740 10 client.16314 fetch_fsmap learned FSMap version 26 2024-04-25T14:15:50.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.756+0000 7f64b8c5e740 1 -- 172.21.15.96:0/275887843 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557b1c5877d0 con 0x557b1c4846b0 2024-04-25T14:15:50.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.756+0000 7f648affd640 1 -- 172.21.15.96:0/275887843 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64a4036040 con 0x557b1c4846b0 2024-04-25T14:15:50.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.756+0000 7f648affd640 1 -- 172.21.15.96:0/275887843 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f64a0019020 con 0x557b1c4846b0 2024-04-25T14:15:50.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.756+0000 7f64b8c5e740 10 client.16314 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:15:50.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.756+0000 7f64b8c5e740 10 client.16314 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:15:50.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.756+0000 7f64b8c5e740 20 client.16314 populate_metadata read hostname 'smithi096' 2024-04-25T14:15:50.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.756+0000 7f64b8c5e740 1 --2- 172.21.15.96:0/275887843 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x557b1c463f60 0x557b1c85f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.756+0000 7f64b8c5e740 4 client.16314 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:15:50.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.756+0000 7f64b8c5e740 1 -- 172.21.15.96:0/275887843 --> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b1c7b9990 con 0x557b1c463f60 2024-04-25T14:15:50.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.756+0000 7f64abfff640 1 --2- 172.21.15.96:0/275887843 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x557b1c463f60 0x557b1c85f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.757+0000 7f64abfff640 1 --2- 172.21.15.96:0/275887843 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x557b1c463f60 0x557b1c85f2e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fa1df3daa0756100 server_cookie=c99c4d95c2e54d01 in_seq=0 out_seq=0 2024-04-25T14:15:50.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.757+0000 7f648affd640 10 client.16314 ms_handle_connect on v2:172.21.15.151:6838/245699981 2024-04-25T14:15:50.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.757+0000 7f648affd640 1 -- 172.21.15.96:0/275887843 <== mds.1 v2:172.21.15.151:6838/245699981 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557b1c7b9990 con 0x557b1c463f60 2024-04-25T14:15:50.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.757+0000 7f648affd640 10 client.16314 handle_command_reply: tid=0 2024-04-25T14:15:50.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.757+0000 7f64b8c5e740 1 client.16314 shutdown 2024-04-25T14:15:50.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.758+0000 7f64b8c5e740 1 -- 172.21.15.96:0/275887843 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b1c4846b0 msgr2=0x557b1c851a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.758+0000 7f64b8c5e740 1 --2- 172.21.15.96:0/275887843 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b1c4846b0 0x557b1c851a70 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f64a000ba60 tx=0x7f64a000bf20 comp rx=0 tx=0).stop 2024-04-25T14:15:50.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.758+0000 7f64b8c5e740 1 -- 172.21.15.96:0/275887843 shutdown_connections 2024-04-25T14:15:50.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.758+0000 7f64b8c5e740 1 --2- 172.21.15.96:0/275887843 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x557b1c463f60 0x557b1c85f2e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.758+0000 7f64b8c5e740 1 --2- 172.21.15.96:0/275887843 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b1c4a5390 0x557b1c8589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.758+0000 7f64b8c5e740 1 --2- 172.21.15.96:0/275887843 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b1c4846b0 0x557b1c851a70 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.758+0000 7f64b8c5e740 1 --2- 172.21.15.96:0/275887843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b1c47c4d0 0x557b1c851530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.758+0000 7f648affd640 0 client.16314 ms_handle_reset on v2:172.21.15.151:6838/245699981 2024-04-25T14:15:50.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.758+0000 7f64b8c5e740 1 -- 172.21.15.96:0/275887843 >> 172.21.15.96:0/275887843 conn(0x557b1c4994e0 msgr2=0x557b1c842040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:50.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.758+0000 7f64b8c5e740 1 -- 172.21.15.96:0/275887843 shutdown_connections 2024-04-25T14:15:50.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.758+0000 7f64b8c5e740 1 -- 172.21.15.96:0/275887843 wait complete. 2024-04-25T14:15:50.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.758+0000 7f64b8c5e740 20 client.16314 trim_cache size 0 max 16384 2024-04-25T14:15:50.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.771+0000 7f64b8c5e740 1 Processor -- start 2024-04-25T14:15:50.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.771+0000 7f64b8c5e740 1 -- start start 2024-04-25T14:15:50.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.772+0000 7f64b8c5e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b1c47c4d0 0x557b1c8503c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.772+0000 7f64b8c5e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b1c4846b0 0x557b1c8588f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.772+0000 7f64b8c5e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b1c4a5390 0x557b1c863ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.772+0000 7f64b8c5e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac1ab3c0 con 0x557b1c4a5390 2024-04-25T14:15:50.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.772+0000 7f64b8c5e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac1acf00 con 0x557b1c4846b0 2024-04-25T14:15:50.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.772+0000 7f64b8c5e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac1aba90 con 0x557b1c47c4d0 2024-04-25T14:15:50.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.772+0000 7f64abfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b1c4a5390 0x557b1c863ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.772+0000 7f64abfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b1c4a5390 0x557b1c863ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51098/0 (socket says 172.21.15.96:51098) 2024-04-25T14:15:50.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.772+0000 7f64abfff640 1 -- 172.21.15.96:0/500131286 learned_addr learned my addr 172.21.15.96:0/500131286 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:50.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.772+0000 7f64aaffd640 1 --2- 172.21.15.96:0/500131286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b1c4846b0 0x557b1c8588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.772+0000 7f64abfff640 1 --2- 172.21.15.96:0/500131286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b1c47c4d0 0x557b1c8503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.772+0000 7f64abfff640 1 -- 172.21.15.96:0/500131286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b1c47c4d0 msgr2=0x557b1c8503c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:50.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.772+0000 7f64abfff640 1 --2- 172.21.15.96:0/500131286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b1c47c4d0 0x557b1c8503c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.772+0000 7f64abfff640 1 -- 172.21.15.96:0/500131286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b1c4846b0 msgr2=0x557b1c8588f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:50.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.772+0000 7f64abfff640 1 --2- 172.21.15.96:0/500131286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b1c4846b0 0x557b1c8588f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.772+0000 7f64abfff640 1 -- 172.21.15.96:0/500131286 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b1c847360 con 0x557b1c4a5390 2024-04-25T14:15:50.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.772+0000 7f64abfff640 1 --2- 172.21.15.96:0/500131286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b1c4a5390 0x557b1c863ad0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f64a4009bf0 tx=0x7f64a4031af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=251aa6a5dcfb85b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:50.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.772+0000 7f64917fa640 1 -- 172.21.15.96:0/500131286 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a404a040 con 0x557b1c4a5390 2024-04-25T14:15:50.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.772+0000 7f64917fa640 1 -- 172.21.15.96:0/500131286 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64a4036070 con 0x557b1c4a5390 2024-04-25T14:15:50.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.772+0000 7f64917fa640 1 -- 172.21.15.96:0/500131286 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a4003e60 con 0x557b1c4a5390 2024-04-25T14:15:50.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.773+0000 7f64b8c5e740 1 -- 172.21.15.96:0/500131286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b1c4a5390 msgr2=0x557b1c863ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.773+0000 7f64b8c5e740 1 --2- 172.21.15.96:0/500131286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b1c4a5390 0x557b1c863ad0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f64a4009bf0 tx=0x7f64a4031af0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.773+0000 7f64b8c5e740 1 -- 172.21.15.96:0/500131286 shutdown_connections 2024-04-25T14:15:50.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.773+0000 7f64b8c5e740 1 --2- 172.21.15.96:0/500131286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b1c4a5390 0x557b1c863ad0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.773+0000 7f64b8c5e740 1 --2- 172.21.15.96:0/500131286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b1c4846b0 0x557b1c8588f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.773+0000 7f64b8c5e740 1 --2- 172.21.15.96:0/500131286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b1c47c4d0 0x557b1c8503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.773+0000 7f64b8c5e740 1 -- 172.21.15.96:0/500131286 >> 172.21.15.96:0/500131286 conn(0x557b1c4994e0 msgr2=0x557b1c84fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:50.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.773+0000 7f64b8c5e740 1 -- 172.21.15.96:0/500131286 shutdown_connections 2024-04-25T14:15:50.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.773+0000 7f64b8c5e740 1 -- 172.21.15.96:0/500131286 wait complete. 2024-04-25T14:15:50.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.773+0000 7f64b8c5e740 1 -- start start 2024-04-25T14:15:50.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.774+0000 7f64b8c5e740 1 Processor -- start 2024-04-25T14:15:50.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.774+0000 7f64b8c5e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b1c47c4d0 0x557b1c855f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.774+0000 7f64b8c5e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b1c4846b0 0x557b1c8564a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.774+0000 7f64b8c5e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b1c4a5390 0x557b1c8573d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.774+0000 7f64b8c5e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac1ac830 con 0x557b1c4a5390 2024-04-25T14:15:50.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.774+0000 7f64b8c5e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac1ac160 con 0x557b1c47c4d0 2024-04-25T14:15:50.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.774+0000 7f64b8c5e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac1a93a0 con 0x557b1c4846b0 2024-04-25T14:15:50.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.774+0000 7f64b8c5e740 10 client.0 _ll_register_callbacks cb 0x7f64a0004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:15:50.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.774+0000 7f64abfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b1c4a5390 0x557b1c8573d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.774+0000 7f64abfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b1c4a5390 0x557b1c8573d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51100/0 (socket says 172.21.15.96:51100) 2024-04-25T14:15:50.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.774+0000 7f64abfff640 1 -- 172.21.15.96:0/2070634623 learned_addr learned my addr 172.21.15.96:0/2070634623 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:50.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.774+0000 7f64aaffd640 1 --2- 172.21.15.96:0/2070634623 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b1c4846b0 0x557b1c8564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.774+0000 7f64abfff640 1 --2- 172.21.15.96:0/2070634623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b1c47c4d0 0x557b1c855f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.774+0000 7f64abfff640 1 -- 172.21.15.96:0/2070634623 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b1c4846b0 msgr2=0x557b1c8564a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:50.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.774+0000 7f64abfff640 1 --2- 172.21.15.96:0/2070634623 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b1c4846b0 0x557b1c8564a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.774+0000 7f64abfff640 1 -- 172.21.15.96:0/2070634623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b1c47c4d0 msgr2=0x557b1c855f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:50.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.774+0000 7f64abfff640 1 --2- 172.21.15.96:0/2070634623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b1c47c4d0 0x557b1c855f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.774+0000 7f64abfff640 1 -- 172.21.15.96:0/2070634623 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b1c858e30 con 0x557b1c4a5390 2024-04-25T14:15:50.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.774+0000 7f64abfff640 1 --2- 172.21.15.96:0/2070634623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b1c4a5390 0x557b1c8573d0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f64a4006010 tx=0x7f64a4057040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49c64cd5cca50a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:50.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.774+0000 7f648affd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:15:50.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.775+0000 7f648affd640 1 -- 172.21.15.96:0/2070634623 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a40040c0 con 0x557b1c4a5390 2024-04-25T14:15:50.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.775+0000 7f648affd640 1 -- 172.21.15.96:0/2070634623 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f64a4004260 con 0x557b1c4a5390 2024-04-25T14:15:50.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.775+0000 7f64b8c5e740 1 -- 172.21.15.96:0/2070634623 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64a4036070 con 0x557b1c4a5390 2024-04-25T14:15:50.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.775+0000 7f648affd640 1 -- 172.21.15.96:0/2070634623 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f64a4004b40 con 0x557b1c4a5390 2024-04-25T14:15:50.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.775+0000 7f64b8c5e740 10 client.16320 fetch_fsmap learned FSMap version 26 2024-04-25T14:15:50.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.775+0000 7f64b8c5e740 1 -- 172.21.15.96:0/2070634623 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557b1c84ccc0 con 0x557b1c4a5390 2024-04-25T14:15:50.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.775+0000 7f648affd640 1 -- 172.21.15.96:0/2070634623 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64a4036070 con 0x557b1c4a5390 2024-04-25T14:15:50.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.775+0000 7f648affd640 1 -- 172.21.15.96:0/2070634623 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f64a402fa80 con 0x557b1c4a5390 2024-04-25T14:15:50.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.775+0000 7f64b8c5e740 10 client.16320 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:15:50.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.775+0000 7f64b8c5e740 10 client.16320 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:15:50.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.775+0000 7f64b8c5e740 20 client.16320 populate_metadata read hostname 'smithi096' 2024-04-25T14:15:50.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.775+0000 7f64b8c5e740 1 --2- 172.21.15.96:0/2070634623 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x557b1c4393a0 0x557b1c8c96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.775+0000 7f64b8c5e740 4 client.16320 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:15:50.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.775+0000 7f64b8c5e740 1 -- 172.21.15.96:0/2070634623 --> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557b1c7b3800 con 0x557b1c4393a0 2024-04-25T14:15:50.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.775+0000 7f64aaffd640 1 --2- 172.21.15.96:0/2070634623 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x557b1c4393a0 0x557b1c8c96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.776+0000 7f64aaffd640 1 --2- 172.21.15.96:0/2070634623 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x557b1c4393a0 0x557b1c8c96a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=48451210bc36faff server_cookie=6b699ddf185275e2 in_seq=0 out_seq=0 2024-04-25T14:15:50.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.776+0000 7f648affd640 10 client.16320 ms_handle_connect on v2:172.21.15.151:6838/245699981 2024-04-25T14:15:50.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.776+0000 7f648affd640 1 -- 172.21.15.96:0/2070634623 <== mds.1 v2:172.21.15.151:6838/245699981 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557b1c7b3800 con 0x557b1c4393a0 2024-04-25T14:15:50.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.776+0000 7f648affd640 10 client.16320 handle_command_reply: tid=0 2024-04-25T14:15:50.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.776+0000 7f64b8c5e740 1 client.16320 shutdown 2024-04-25T14:15:50.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.776+0000 7f64b8c5e740 1 -- 172.21.15.96:0/2070634623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b1c4a5390 msgr2=0x557b1c8573d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.776+0000 7f64b8c5e740 1 --2- 172.21.15.96:0/2070634623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b1c4a5390 0x557b1c8573d0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f64a4006010 tx=0x7f64a4057040 comp rx=0 tx=0).stop 2024-04-25T14:15:50.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.776+0000 7f64b8c5e740 1 -- 172.21.15.96:0/2070634623 shutdown_connections 2024-04-25T14:15:50.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.776+0000 7f64b8c5e740 1 --2- 172.21.15.96:0/2070634623 >> [v2:172.21.15.151:6838/245699981,v1:172.21.15.151:6839/245699981] conn(0x557b1c4393a0 0x557b1c8c96a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.776+0000 7f64b8c5e740 1 --2- 172.21.15.96:0/2070634623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b1c4a5390 0x557b1c8573d0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.776+0000 7f64b8c5e740 1 --2- 172.21.15.96:0/2070634623 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b1c4846b0 0x557b1c8564a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.776+0000 7f648affd640 0 client.16320 ms_handle_reset on v2:172.21.15.151:6838/245699981 2024-04-25T14:15:50.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.776+0000 7f64b8c5e740 1 --2- 172.21.15.96:0/2070634623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b1c47c4d0 0x557b1c855f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.776+0000 7f64b8c5e740 1 -- 172.21.15.96:0/2070634623 >> 172.21.15.96:0/2070634623 conn(0x557b1c4994e0 msgr2=0x557b1c845ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:50.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.776+0000 7f64b8c5e740 1 -- 172.21.15.96:0/2070634623 shutdown_connections 2024-04-25T14:15:50.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.776+0000 7f64b8c5e740 1 -- 172.21.15.96:0/2070634623 wait complete. 2024-04-25T14:15:50.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.776+0000 7f64b8c5e740 20 client.16320 trim_cache size 0 max 16384 2024-04-25T14:15:50.784 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-04-25T14:15:50.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.777+0000 7f64b1db2640 1 -- 172.21.15.96:0/2298948849 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f64940afaf0 msgr2=0x7f64940b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.777+0000 7f64b1db2640 1 --2- 172.21.15.96:0/2298948849 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f64940afaf0 0x7f64940b2000 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x557b1c847530 tx=0x7f649c00de80 comp rx=0 tx=0).stop 2024-04-25T14:15:50.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.777+0000 7f64b1db2640 1 -- 172.21.15.96:0/2298948849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64ac103f00 msgr2=0x7f64ac102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.777+0000 7f64b1db2640 1 --2- 172.21.15.96:0/2298948849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64ac103f00 0x7f64ac102160 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f649c00d770 tx=0x7f649c00dc30 comp rx=0 tx=0).stop 2024-04-25T14:15:50.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.777+0000 7f64b1db2640 1 -- 172.21.15.96:0/2298948849 shutdown_connections 2024-04-25T14:15:50.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.777+0000 7f64b1db2640 1 --2- 172.21.15.96:0/2298948849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64ac19f0b0 0x7f64ac1a14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.777+0000 7f64b1db2640 1 --2- 172.21.15.96:0/2298948849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64ac108a30 0x7f64ac19eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.777+0000 7f64b1db2640 1 --2- 172.21.15.96:0/2298948849 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f64940afaf0 0x7f64940b2000 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.777+0000 7f64b1db2640 1 --2- 172.21.15.96:0/2298948849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64ac103f00 0x7f64ac102160 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.777+0000 7f64b1db2640 1 -- 172.21.15.96:0/2298948849 >> 172.21.15.96:0/2298948849 conn(0x7f64ac0fd380 msgr2=0x7f64ac0ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:50.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.777+0000 7f64b1db2640 1 -- 172.21.15.96:0/2298948849 shutdown_connections 2024-04-25T14:15:50.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.777+0000 7f64b1db2640 1 -- 172.21.15.96:0/2298948849 wait complete. 2024-04-25T14:15:50.787 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:50 smithi096 ceph-mon[29207]: pgmap v412: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 KiB/s wr, 11 op/s 2024-04-25T14:15:50.787 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:50 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/4040883412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:15:50.788 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T14:15:50.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.897+0000 7f390512e640 1 -- 172.21.15.96:0/3888423001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3900103b00 msgr2=0x7f3900104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.897+0000 7f390512e640 1 --2- 172.21.15.96:0/3888423001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3900103b00 0x7f3900104800 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f38f0009900 tx=0x7f38f002a410 comp rx=0 tx=0).stop 2024-04-25T14:15:50.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.897+0000 7f390512e640 1 -- 172.21.15.96:0/3888423001 shutdown_connections 2024-04-25T14:15:50.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.897+0000 7f390512e640 1 --2- 172.21.15.96:0/3888423001 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3900109a40 0x7f390010be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.897+0000 7f390512e640 1 --2- 172.21.15.96:0/3888423001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3900104d40 0x7f3900105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.897+0000 7f390512e640 1 --2- 172.21.15.96:0/3888423001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3900103b00 0x7f3900104800 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.897+0000 7f390512e640 1 -- 172.21.15.96:0/3888423001 >> 172.21.15.96:0/3888423001 conn(0x7f39000fed10 msgr2=0x7f39001011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:50.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.897+0000 7f390512e640 1 -- 172.21.15.96:0/3888423001 shutdown_connections 2024-04-25T14:15:50.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.897+0000 7f390512e640 1 -- 172.21.15.96:0/3888423001 wait complete. 2024-04-25T14:15:50.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.898+0000 7f390512e640 1 Processor -- start 2024-04-25T14:15:50.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.898+0000 7f390512e640 1 -- start start 2024-04-25T14:15:50.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.898+0000 7f390512e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3900103b00 0x7f390019a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.898+0000 7f390512e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3900104d40 0x7f390019a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.898+0000 7f390512e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3900109a40 0x7f39001a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.898+0000 7f390512e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f390010f5e0 con 0x7f3900103b00 2024-04-25T14:15:50.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.898+0000 7f390512e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f390010f460 con 0x7f3900109a40 2024-04-25T14:15:50.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.898+0000 7f390512e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f390010f760 con 0x7f3900104d40 2024-04-25T14:15:50.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.898+0000 7f38fed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3900103b00 0x7f390019a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.898+0000 7f38fed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3900103b00 0x7f390019a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51110/0 (socket says 172.21.15.96:51110) 2024-04-25T14:15:50.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.898+0000 7f38fed9d640 1 -- 172.21.15.96:0/2506334988 learned_addr learned my addr 172.21.15.96:0/2506334988 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:50.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.899+0000 7f38f7fff640 1 --2- 172.21.15.96:0/2506334988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3900104d40 0x7f390019a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.899+0000 7f38ff59e640 1 --2- 172.21.15.96:0/2506334988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3900109a40 0x7f39001a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.899+0000 7f38fed9d640 1 -- 172.21.15.96:0/2506334988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3900104d40 msgr2=0x7f390019a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.899+0000 7f38fed9d640 1 --2- 172.21.15.96:0/2506334988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3900104d40 0x7f390019a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.899+0000 7f38fed9d640 1 -- 172.21.15.96:0/2506334988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3900109a40 msgr2=0x7f39001a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.899+0000 7f38fed9d640 1 --2- 172.21.15.96:0/2506334988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3900109a40 0x7f39001a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.899+0000 7f38fed9d640 1 -- 172.21.15.96:0/2506334988 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39001a1d10 con 0x7f3900103b00 2024-04-25T14:15:50.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.899+0000 7f38f7fff640 1 --2- 172.21.15.96:0/2506334988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3900104d40 0x7f390019a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:15:50.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.899+0000 7f38fed9d640 1 --2- 172.21.15.96:0/2506334988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3900103b00 0x7f390019a280 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f38f0006010 tx=0x7f38f0031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:50.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.899+0000 7f38fcd99640 1 -- 172.21.15.96:0/2506334988 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f003f030 con 0x7f3900103b00 2024-04-25T14:15:50.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.899+0000 7f390512e640 1 -- 172.21.15.96:0/2506334988 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39001a1fa0 con 0x7f3900103b00 2024-04-25T14:15:50.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.899+0000 7f38fcd99640 1 -- 172.21.15.96:0/2506334988 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38f0003aa0 con 0x7f3900103b00 2024-04-25T14:15:50.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.899+0000 7f38fcd99640 1 -- 172.21.15.96:0/2506334988 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f0035070 con 0x7f3900103b00 2024-04-25T14:15:50.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.899+0000 7f390512e640 1 -- 172.21.15.96:0/2506334988 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39001a23a0 con 0x7f3900103b00 2024-04-25T14:15:50.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.902+0000 7f38fcd99640 1 -- 172.21.15.96:0/2506334988 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f38f0042030 con 0x7f3900103b00 2024-04-25T14:15:50.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.902+0000 7f390bfd9740 1 Processor -- start 2024-04-25T14:15:50.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.902+0000 7f390bfd9740 1 -- start start 2024-04-25T14:15:50.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.902+0000 7f390bfd9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e229ad390 0x560e22d4f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.903+0000 7f390bfd9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e229844d0 0x560e22d4fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.903+0000 7f390bfd9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560e2298c6b0 0x560e22d50a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.903+0000 7f390bfd9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560e22cb3d80 con 0x560e229ad390 2024-04-25T14:15:50.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.903+0000 7f390bfd9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560e22c6da20 con 0x560e2298c6b0 2024-04-25T14:15:50.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.903+0000 7f390bfd9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560e22cafd40 con 0x560e229844d0 2024-04-25T14:15:50.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.903+0000 7f38ff59e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e229ad390 0x560e22d4f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.903+0000 7f38ff59e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e229ad390 0x560e22d4f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51118/0 (socket says 172.21.15.96:51118) 2024-04-25T14:15:50.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.903+0000 7f38ff59e640 1 -- 172.21.15.96:0/1454914030 learned_addr learned my addr 172.21.15.96:0/1454914030 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:50.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.903+0000 7f38fed9d640 1 --2- 172.21.15.96:0/1454914030 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e229844d0 0x560e22d4fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.903+0000 7f38f7fff640 1 --2- 172.21.15.96:0/1454914030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560e2298c6b0 0x560e22d50a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.903+0000 7f38fcd99640 1 --2- 172.21.15.96:0/2506334988 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f38ec0afaf0 0x7f38ec0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.903+0000 7f38f7fff640 1 -- 172.21.15.96:0/1454914030 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e229844d0 msgr2=0x560e22d4fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:50.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.903+0000 7f38f7fff640 1 --2- 172.21.15.96:0/1454914030 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e229844d0 0x560e22d4fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.903+0000 7f38f7fff640 1 -- 172.21.15.96:0/1454914030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e229ad390 msgr2=0x560e22d4f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:50.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.903+0000 7f38f7fff640 1 --2- 172.21.15.96:0/1454914030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e229ad390 0x560e22d4f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.903+0000 7f38f7fff640 1 -- 172.21.15.96:0/1454914030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e22c78c30 con 0x560e2298c6b0 2024-04-25T14:15:50.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.903+0000 7f38fed9d640 1 --2- 172.21.15.96:0/1454914030 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e229844d0 0x560e22d4fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:15:50.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.903+0000 7f38ff59e640 1 --2- 172.21.15.96:0/1454914030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e229ad390 0x560e22d4f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:15:50.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.903+0000 7f38ff59e640 1 --2- 172.21.15.96:0/2506334988 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f38ec0afaf0 0x7f38ec0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.903+0000 7f38fcd99640 1 -- 172.21.15.96:0/2506334988 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f38f00ff210 con 0x7f3900103b00 2024-04-25T14:15:50.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.903+0000 7f38f7fff640 1 --2- 172.21.15.96:0/1454914030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560e2298c6b0 0x560e22d50a20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f38e8008b80 tx=0x7f38e800e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba2b131d5b3e6b2e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:50.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.904+0000 7f38f57fa640 1 -- 172.21.15.96:0/1454914030 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e8014070 con 0x560e2298c6b0 2024-04-25T14:15:50.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.904+0000 7f38ff59e640 1 --2- 172.21.15.96:0/2506334988 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f38ec0afaf0 0x7f38ec0b2000 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x560e22d50860 tx=0x7f38f8004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:50.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.904+0000 7f38f57fa640 1 -- 172.21.15.96:0/1454914030 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38e80054d0 con 0x560e2298c6b0 2024-04-25T14:15:50.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.904+0000 7f38f57fa640 1 -- 172.21.15.96:0/1454914030 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e8004b00 con 0x560e2298c6b0 2024-04-25T14:15:50.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.904+0000 7f390bfd9740 1 -- 172.21.15.96:0/1454914030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560e2298c6b0 msgr2=0x560e22d50a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.904+0000 7f390bfd9740 1 --2- 172.21.15.96:0/1454914030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560e2298c6b0 0x560e22d50a20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f38e8008b80 tx=0x7f38e800e720 comp rx=0 tx=0).stop 2024-04-25T14:15:50.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.904+0000 7f390bfd9740 1 -- 172.21.15.96:0/1454914030 shutdown_connections 2024-04-25T14:15:50.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.904+0000 7f390bfd9740 1 --2- 172.21.15.96:0/1454914030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560e2298c6b0 0x560e22d50a20 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.905+0000 7f390bfd9740 1 --2- 172.21.15.96:0/1454914030 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e229844d0 0x560e22d4fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.905+0000 7f390bfd9740 1 --2- 172.21.15.96:0/1454914030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e229ad390 0x560e22d4f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.905+0000 7f390bfd9740 1 -- 172.21.15.96:0/1454914030 >> 172.21.15.96:0/1454914030 conn(0x560e229a14e0 msgr2=0x560e22d4b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:50.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.905+0000 7f390bfd9740 1 -- 172.21.15.96:0/1454914030 shutdown_connections 2024-04-25T14:15:50.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.905+0000 7f390bfd9740 1 -- 172.21.15.96:0/1454914030 wait complete. 2024-04-25T14:15:50.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.905+0000 7f390bfd9740 1 -- start start 2024-04-25T14:15:50.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.905+0000 7f390bfd9740 1 Processor -- start 2024-04-25T14:15:50.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.905+0000 7f390bfd9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e229844d0 0x560e22d59510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.905+0000 7f390bfd9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560e2298c6b0 0x560e22d59a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.905+0000 7f390bfd9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e229ad390 0x560e22d609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.905+0000 7f390bfd9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560e22d51530 con 0x560e229ad390 2024-04-25T14:15:50.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.905+0000 7f390bfd9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560e22c3adc0 con 0x560e2298c6b0 2024-04-25T14:15:50.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.905+0000 7f390bfd9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560e22d516b0 con 0x560e229844d0 2024-04-25T14:15:50.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.905+0000 7f390bfd9740 10 client.0 _ll_register_callbacks cb 0x560e22c56600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:15:50.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.905+0000 7f38ff59e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e229ad390 0x560e22d609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.905+0000 7f38ff59e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e229ad390 0x560e22d609b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51122/0 (socket says 172.21.15.96:51122) 2024-04-25T14:15:50.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.905+0000 7f38ff59e640 1 -- 172.21.15.96:0/3968917043 learned_addr learned my addr 172.21.15.96:0/3968917043 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:50.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.905+0000 7f38f7fff640 1 --2- 172.21.15.96:0/3968917043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560e2298c6b0 0x560e22d59a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.905+0000 7f38fed9d640 1 --2- 172.21.15.96:0/3968917043 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e229844d0 0x560e22d59510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.906+0000 7f38ff59e640 1 -- 172.21.15.96:0/3968917043 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e229844d0 msgr2=0x560e22d59510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:50.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.906+0000 7f38ff59e640 1 --2- 172.21.15.96:0/3968917043 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e229844d0 0x560e22d59510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.906+0000 7f38ff59e640 1 -- 172.21.15.96:0/3968917043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560e2298c6b0 msgr2=0x560e22d59a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:50.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.906+0000 7f38ff59e640 1 --2- 172.21.15.96:0/3968917043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560e2298c6b0 0x560e22d59a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.906+0000 7f38ff59e640 1 -- 172.21.15.96:0/3968917043 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e229c53f0 con 0x560e229ad390 2024-04-25T14:15:50.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.906+0000 7f38ff59e640 1 --2- 172.21.15.96:0/3968917043 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e229ad390 0x560e22d609b0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f38f800f8e0 tx=0x7f38f8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b185b42fbbdd6929 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:50.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.906+0000 7f38e67fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:15:50.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.906+0000 7f38e67fc640 1 -- 172.21.15.96:0/3968917043 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f8023070 con 0x560e229ad390 2024-04-25T14:15:50.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.906+0000 7f38e67fc640 1 -- 172.21.15.96:0/3968917043 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f38f8012dd0 con 0x560e229ad390 2024-04-25T14:15:50.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.906+0000 7f390bfd9740 1 -- 172.21.15.96:0/3968917043 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38e80054d0 con 0x560e229ad390 2024-04-25T14:15:50.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.906+0000 7f38e67fc640 1 -- 172.21.15.96:0/3968917043 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f38f801f420 con 0x560e229ad390 2024-04-25T14:15:50.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.906+0000 7f390bfd9740 10 client.16332 fetch_fsmap learned FSMap version 26 2024-04-25T14:15:50.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.906+0000 7f390bfd9740 1 -- 172.21.15.96:0/3968917043 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560e22a8f7d0 con 0x560e229ad390 2024-04-25T14:15:50.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.906+0000 7f38e67fc640 1 -- 172.21.15.96:0/3968917043 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38e80054d0 con 0x560e229ad390 2024-04-25T14:15:50.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.906+0000 7f38e67fc640 1 -- 172.21.15.96:0/3968917043 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f38f800f040 con 0x560e229ad390 2024-04-25T14:15:50.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.906+0000 7f390bfd9740 10 client.16332 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:15:50.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.907+0000 7f390bfd9740 10 client.16332 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:15:50.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.907+0000 7f390bfd9740 20 client.16332 populate_metadata read hostname 'smithi096' 2024-04-25T14:15:50.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.907+0000 7f390bfd9740 1 --2- 172.21.15.96:0/3968917043 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x560e2296bf60 0x560e22d67300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.907+0000 7f390bfd9740 4 client.16332 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:15:50.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.907+0000 7f390bfd9740 1 -- 172.21.15.96:0/3968917043 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e22cc1990 con 0x560e2296bf60 2024-04-25T14:15:50.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.907+0000 7f38fed9d640 1 --2- 172.21.15.96:0/3968917043 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x560e2296bf60 0x560e22d67300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.907+0000 7f38fed9d640 1 --2- 172.21.15.96:0/3968917043 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x560e2296bf60 0x560e22d67300 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d707bd3a1e14cdd2 server_cookie=bd29ae15822421f0 in_seq=0 out_seq=0 2024-04-25T14:15:50.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.907+0000 7f38e67fc640 10 client.16332 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:15:50.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.908+0000 7f38e67fc640 1 -- 172.21.15.96:0/3968917043 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560e22cc1990 con 0x560e2296bf60 2024-04-25T14:15:50.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.908+0000 7f38e67fc640 10 client.16332 handle_command_reply: tid=0 2024-04-25T14:15:50.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.908+0000 7f390bfd9740 1 client.16332 shutdown 2024-04-25T14:15:50.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.908+0000 7f390bfd9740 1 -- 172.21.15.96:0/3968917043 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e229ad390 msgr2=0x560e22d609b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.908+0000 7f390bfd9740 1 --2- 172.21.15.96:0/3968917043 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e229ad390 0x560e22d609b0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f38f800f8e0 tx=0x7f38f8004ab0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.908+0000 7f390bfd9740 1 -- 172.21.15.96:0/3968917043 shutdown_connections 2024-04-25T14:15:50.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.908+0000 7f390bfd9740 1 --2- 172.21.15.96:0/3968917043 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x560e2296bf60 0x560e22d67300 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.908+0000 7f390bfd9740 1 --2- 172.21.15.96:0/3968917043 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e229ad390 0x560e22d609b0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.908+0000 7f38e67fc640 0 client.16332 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:15:50.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.908+0000 7f390bfd9740 1 --2- 172.21.15.96:0/3968917043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560e2298c6b0 0x560e22d59a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.908+0000 7f390bfd9740 1 --2- 172.21.15.96:0/3968917043 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e229844d0 0x560e22d59510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.908+0000 7f390bfd9740 1 -- 172.21.15.96:0/3968917043 >> 172.21.15.96:0/3968917043 conn(0x560e229a14e0 msgr2=0x560e22d4a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:50.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.908+0000 7f390bfd9740 1 -- 172.21.15.96:0/3968917043 shutdown_connections 2024-04-25T14:15:50.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.908+0000 7f390bfd9740 1 -- 172.21.15.96:0/3968917043 wait complete. 2024-04-25T14:15:50.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.908+0000 7f390bfd9740 20 client.16332 trim_cache size 0 max 16384 2024-04-25T14:15:50.927 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:50 smithi097 ceph-mon[32720]: pgmap v412: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 KiB/s wr, 11 op/s 2024-04-25T14:15:50.927 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:50 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/4040883412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:15:50.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.922+0000 7f390bfd9740 1 Processor -- start 2024-04-25T14:15:50.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.922+0000 7f390bfd9740 1 -- start start 2024-04-25T14:15:50.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.922+0000 7f390bfd9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e2296bf60 0x560e22d54230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.922+0000 7f390bfd9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e229844d0 0x560e22d54770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.922+0000 7f390bfd9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560e2298c6b0 0x560e22d608d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.922+0000 7f390bfd9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f39001ab2b0 con 0x560e2296bf60 2024-04-25T14:15:50.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.922+0000 7f390bfd9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f39001acdf0 con 0x560e2298c6b0 2024-04-25T14:15:50.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.922+0000 7f390bfd9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f39001ab980 con 0x560e229844d0 2024-04-25T14:15:50.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.922+0000 7f38fed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e2296bf60 0x560e22d54230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.922+0000 7f38fed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e2296bf60 0x560e22d54230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51126/0 (socket says 172.21.15.96:51126) 2024-04-25T14:15:50.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.922+0000 7f38fed9d640 1 -- 172.21.15.96:0/2140033967 learned_addr learned my addr 172.21.15.96:0/2140033967 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:50.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.922+0000 7f38ff59e640 1 --2- 172.21.15.96:0/2140033967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560e2298c6b0 0x560e22d608d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.922+0000 7f38fed9d640 1 -- 172.21.15.96:0/2140033967 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e229844d0 msgr2=0x560e22d54770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:50.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.922+0000 7f38fed9d640 1 --2- 172.21.15.96:0/2140033967 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e229844d0 0x560e22d54770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.922+0000 7f38fed9d640 1 -- 172.21.15.96:0/2140033967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560e2298c6b0 msgr2=0x560e22d608d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:50.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.922+0000 7f38fed9d640 1 --2- 172.21.15.96:0/2140033967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560e2298c6b0 0x560e22d608d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.922+0000 7f38fed9d640 1 -- 172.21.15.96:0/2140033967 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e22d60e10 con 0x560e2296bf60 2024-04-25T14:15:50.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.922+0000 7f38ff59e640 1 --2- 172.21.15.96:0/2140033967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560e2298c6b0 0x560e22d608d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:15:50.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.922+0000 7f38fed9d640 1 --2- 172.21.15.96:0/2140033967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e2296bf60 0x560e22d54230 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f38f00c3750 tx=0x7f38f0101720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97c2b47f80a7d291 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:50.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.923+0000 7f38f4ff9640 1 -- 172.21.15.96:0/2140033967 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f0049040 con 0x560e2296bf60 2024-04-25T14:15:50.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.923+0000 7f38f4ff9640 1 -- 172.21.15.96:0/2140033967 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38f00c71e0 con 0x560e2296bf60 2024-04-25T14:15:50.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.923+0000 7f38f4ff9640 1 -- 172.21.15.96:0/2140033967 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f00c1e60 con 0x560e2296bf60 2024-04-25T14:15:50.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.923+0000 7f390bfd9740 1 -- 172.21.15.96:0/2140033967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e2296bf60 msgr2=0x560e22d54230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.923+0000 7f390bfd9740 1 --2- 172.21.15.96:0/2140033967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e2296bf60 0x560e22d54230 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f38f00c3750 tx=0x7f38f0101720 comp rx=0 tx=0).stop 2024-04-25T14:15:50.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.923+0000 7f390bfd9740 1 -- 172.21.15.96:0/2140033967 shutdown_connections 2024-04-25T14:15:50.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.923+0000 7f390bfd9740 1 --2- 172.21.15.96:0/2140033967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560e2298c6b0 0x560e22d608d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.923+0000 7f390bfd9740 1 --2- 172.21.15.96:0/2140033967 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e229844d0 0x560e22d54770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.923+0000 7f390bfd9740 1 --2- 172.21.15.96:0/2140033967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e2296bf60 0x560e22d54230 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.923+0000 7f390bfd9740 1 -- 172.21.15.96:0/2140033967 >> 172.21.15.96:0/2140033967 conn(0x560e229a14e0 msgr2=0x560e22d4f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:50.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.924+0000 7f390bfd9740 1 -- 172.21.15.96:0/2140033967 shutdown_connections 2024-04-25T14:15:50.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.924+0000 7f390bfd9740 1 -- 172.21.15.96:0/2140033967 wait complete. 2024-04-25T14:15:50.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.924+0000 7f390bfd9740 1 -- start start 2024-04-25T14:15:50.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.924+0000 7f390bfd9740 1 Processor -- start 2024-04-25T14:15:50.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.924+0000 7f390bfd9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560e2296bf60 0x560e22d5e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.924+0000 7f390bfd9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e229844d0 0x560e22d5e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.924+0000 7f390bfd9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e2298c6b0 0x560e22d5f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.924+0000 7f390bfd9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f39001ac720 con 0x560e229844d0 2024-04-25T14:15:50.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.924+0000 7f390bfd9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f39001ac050 con 0x560e2296bf60 2024-04-25T14:15:50.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.924+0000 7f390bfd9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f39001a9290 con 0x560e2298c6b0 2024-04-25T14:15:50.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.924+0000 7f390bfd9740 10 client.0 _ll_register_callbacks cb 0x7f38f8009d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:15:50.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.924+0000 7f38f7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e229844d0 0x560e22d5e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.924+0000 7f38f7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e229844d0 0x560e22d5e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51134/0 (socket says 172.21.15.96:51134) 2024-04-25T14:15:50.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.924+0000 7f38f7fff640 1 -- 172.21.15.96:0/1261210454 learned_addr learned my addr 172.21.15.96:0/1261210454 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:50.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.924+0000 7f38fed9d640 1 --2- 172.21.15.96:0/1261210454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560e2296bf60 0x560e22d5e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.924+0000 7f38ff59e640 1 --2- 172.21.15.96:0/1261210454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e2298c6b0 0x560e22d5f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.924+0000 7f38f7fff640 1 -- 172.21.15.96:0/1261210454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e2298c6b0 msgr2=0x560e22d5f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:50.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.924+0000 7f38f7fff640 1 --2- 172.21.15.96:0/1261210454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e2298c6b0 0x560e22d5f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.924+0000 7f38f7fff640 1 -- 172.21.15.96:0/1261210454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560e2296bf60 msgr2=0x560e22d5e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:50.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.924+0000 7f38f7fff640 1 --2- 172.21.15.96:0/1261210454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560e2296bf60 0x560e22d5e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.925+0000 7f38f7fff640 1 -- 172.21.15.96:0/1261210454 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e22d5fa50 con 0x560e229844d0 2024-04-25T14:15:50.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.925+0000 7f38f7fff640 1 --2- 172.21.15.96:0/1261210454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e229844d0 0x560e22d5e560 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f38e800f010 tx=0x7f38e8010af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2b59a879695ccc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:50.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.925+0000 7f38e67fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T14:15:50.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.925+0000 7f38e67fc640 1 -- 172.21.15.96:0/1261210454 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e8014040 con 0x560e229844d0 2024-04-25T14:15:50.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.925+0000 7f38e67fc640 1 -- 172.21.15.96:0/1261210454 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f38e8010c60 con 0x560e229844d0 2024-04-25T14:15:50.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.925+0000 7f390bfd9740 1 -- 172.21.15.96:0/1261210454 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38f00c71e0 con 0x560e229844d0 2024-04-25T14:15:50.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.925+0000 7f38e67fc640 1 -- 172.21.15.96:0/1261210454 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f38e8010e60 con 0x560e229844d0 2024-04-25T14:15:50.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.925+0000 7f390bfd9740 10 client.16338 fetch_fsmap learned FSMap version 26 2024-04-25T14:15:50.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.925+0000 7f390bfd9740 1 -- 172.21.15.96:0/1261210454 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560e22dcf6b0 con 0x560e229844d0 2024-04-25T14:15:50.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.925+0000 7f38e67fc640 1 -- 172.21.15.96:0/1261210454 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38f00c71e0 con 0x560e229844d0 2024-04-25T14:15:50.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.925+0000 7f38e67fc640 1 -- 172.21.15.96:0/1261210454 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f38e802e4c0 con 0x560e229844d0 2024-04-25T14:15:50.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.925+0000 7f390bfd9740 10 client.16338 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T14:15:50.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.925+0000 7f390bfd9740 10 client.16338 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:15:50.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.925+0000 7f390bfd9740 20 client.16338 populate_metadata read hostname 'smithi096' 2024-04-25T14:15:50.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.925+0000 7f390bfd9740 1 --2- 172.21.15.96:0/1261210454 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x560e229ad390 0x560e22dd1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.925+0000 7f390bfd9740 4 client.16338 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:15:50.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.925+0000 7f390bfd9740 1 -- 172.21.15.96:0/1261210454 --> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560e22cbb800 con 0x560e229ad390 2024-04-25T14:15:50.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.926+0000 7f38fed9d640 1 --2- 172.21.15.96:0/1261210454 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x560e229ad390 0x560e22dd1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.926+0000 7f38e67fc640 10 client.16338 ms_handle_connect on v2:172.21.15.97:6838/2850442269 2024-04-25T14:15:50.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.926+0000 7f38fed9d640 1 --2- 172.21.15.96:0/1261210454 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x560e229ad390 0x560e22dd1bd0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce214c539fb22e90 server_cookie=d847c110bc8cf041 in_seq=0 out_seq=0 2024-04-25T14:15:50.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.926+0000 7f38e67fc640 1 -- 172.21.15.96:0/1261210454 <== mds.0 v2:172.21.15.97:6838/2850442269 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560e22cbb800 con 0x560e229ad390 2024-04-25T14:15:50.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.926+0000 7f38e67fc640 10 client.16338 handle_command_reply: tid=0 2024-04-25T14:15:50.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.926+0000 7f390bfd9740 1 client.16338 shutdown 2024-04-25T14:15:50.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.927+0000 7f390bfd9740 1 -- 172.21.15.96:0/1261210454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e229844d0 msgr2=0x560e22d5e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.927+0000 7f390bfd9740 1 --2- 172.21.15.96:0/1261210454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e229844d0 0x560e22d5e560 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f38e800f010 tx=0x7f38e8010af0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.927+0000 7f390bfd9740 1 -- 172.21.15.96:0/1261210454 shutdown_connections 2024-04-25T14:15:50.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.927+0000 7f390bfd9740 1 --2- 172.21.15.96:0/1261210454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e2298c6b0 0x560e22d5f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.927+0000 7f390bfd9740 1 --2- 172.21.15.96:0/1261210454 >> [v2:172.21.15.97:6838/2850442269,v1:172.21.15.97:6839/2850442269] conn(0x560e229ad390 0x560e22dd1bd0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.927+0000 7f390bfd9740 1 --2- 172.21.15.96:0/1261210454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e229844d0 0x560e22d5e560 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.927+0000 7f390bfd9740 1 --2- 172.21.15.96:0/1261210454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560e2296bf60 0x560e22d5e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.927+0000 7f38e67fc640 0 client.16338 ms_handle_reset on v2:172.21.15.97:6838/2850442269 2024-04-25T14:15:50.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.927+0000 7f390bfd9740 1 -- 172.21.15.96:0/1261210454 >> 172.21.15.96:0/1261210454 conn(0x560e229a14e0 msgr2=0x560e22d59510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:50.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.927+0000 7f390bfd9740 1 -- 172.21.15.96:0/1261210454 shutdown_connections 2024-04-25T14:15:50.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.927+0000 7f390bfd9740 1 -- 172.21.15.96:0/1261210454 wait complete. 2024-04-25T14:15:50.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.927+0000 7f390bfd9740 20 client.16338 trim_cache size 0 max 16384 2024-04-25T14:15:50.947 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-04-25T14:15:50.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.927+0000 7f390512e640 1 -- 172.21.15.96:0/2506334988 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f38ec0afaf0 msgr2=0x7f38ec0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.927+0000 7f390512e640 1 --2- 172.21.15.96:0/2506334988 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f38ec0afaf0 0x7f38ec0b2000 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x560e22d50860 tx=0x7f38f8004a40 comp rx=0 tx=0).stop 2024-04-25T14:15:50.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.927+0000 7f390512e640 1 -- 172.21.15.96:0/2506334988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3900103b00 msgr2=0x7f390019a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.927+0000 7f390512e640 1 --2- 172.21.15.96:0/2506334988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3900103b00 0x7f390019a280 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f38f0006010 tx=0x7f38f0031a50 comp rx=0 tx=0).stop 2024-04-25T14:15:50.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.927+0000 7f390512e640 1 -- 172.21.15.96:0/2506334988 shutdown_connections 2024-04-25T14:15:50.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.927+0000 7f390512e640 1 --2- 172.21.15.96:0/2506334988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3900109a40 0x7f39001a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.927+0000 7f390512e640 1 --2- 172.21.15.96:0/2506334988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3900104d40 0x7f390019a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.927+0000 7f390512e640 1 --2- 172.21.15.96:0/2506334988 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f38ec0afaf0 0x7f38ec0b2000 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.927+0000 7f390512e640 1 --2- 172.21.15.96:0/2506334988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3900103b00 0x7f390019a280 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.927+0000 7f390512e640 1 -- 172.21.15.96:0/2506334988 >> 172.21.15.96:0/2506334988 conn(0x7f39000fed10 msgr2=0x7f3900105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:50.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.927+0000 7f390512e640 1 -- 172.21.15.96:0/2506334988 shutdown_connections 2024-04-25T14:15:50.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:50.928+0000 7f390512e640 1 -- 172.21.15.96:0/2506334988 wait complete. 2024-04-25T14:15:50.949 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:50 smithi151 ceph-mon[32990]: pgmap v412: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 KiB/s wr, 11 op/s 2024-04-25T14:15:50.950 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:50 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/4040883412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:15:51.950 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-04-25T14:15:51.950 INFO:tasks.fwd_scrub:done joining 2024-04-25T14:15:51.950 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-25T14:15:51.961 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:15:52.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.076+0000 7f5031bf3640 1 -- 172.21.15.96:0/21830339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f502c108940 msgr2=0x7f502c10ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:52.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.076+0000 7f5031bf3640 1 --2- 172.21.15.96:0/21830339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f502c108940 0x7f502c10ad60 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f5024009ca0 tx=0x7f502402a8c0 comp rx=0 tx=0).stop 2024-04-25T14:15:52.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.076+0000 7f5031bf3640 1 -- 172.21.15.96:0/21830339 shutdown_connections 2024-04-25T14:15:52.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.076+0000 7f5031bf3640 1 --2- 172.21.15.96:0/21830339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f502c108940 0x7f502c10ad60 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:52.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.076+0000 7f5031bf3640 1 --2- 172.21.15.96:0/21830339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f502c103c40 0x7f502c104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:52.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.076+0000 7f5031bf3640 1 --2- 172.21.15.96:0/21830339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f502c102a00 0x7f502c103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:52.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.076+0000 7f5031bf3640 1 -- 172.21.15.96:0/21830339 >> 172.21.15.96:0/21830339 conn(0x7f502c0fdbf0 msgr2=0x7f502c1000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:52.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.076+0000 7f5031bf3640 1 -- 172.21.15.96:0/21830339 shutdown_connections 2024-04-25T14:15:52.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.076+0000 7f5031bf3640 1 -- 172.21.15.96:0/21830339 wait complete. 2024-04-25T14:15:52.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.077+0000 7f5031bf3640 1 Processor -- start 2024-04-25T14:15:52.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.077+0000 7f5031bf3640 1 -- start start 2024-04-25T14:15:52.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.077+0000 7f5031bf3640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f502c102a00 0x7f502c19a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:52.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.077+0000 7f5031bf3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f502c103c40 0x7f502c19a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:52.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.077+0000 7f5031bf3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f502c108940 0x7f502c1a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:52.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.077+0000 7f5031bf3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f502c10e510 con 0x7f502c108940 2024-04-25T14:15:52.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.077+0000 7f5031bf3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f502c10e390 con 0x7f502c103c40 2024-04-25T14:15:52.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.077+0000 7f5031bf3640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f502c10e690 con 0x7f502c102a00 2024-04-25T14:15:52.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.077+0000 7f502bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f502c108940 0x7f502c1a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:52.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.077+0000 7f502bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f502c108940 0x7f502c1a1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43920/0 (socket says 172.21.15.96:43920) 2024-04-25T14:15:52.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.077+0000 7f502bfff640 1 -- 172.21.15.96:0/3340829992 learned_addr learned my addr 172.21.15.96:0/3340829992 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:52.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.077+0000 7f5023fff640 1 --2- 172.21.15.96:0/3340829992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f502c103c40 0x7f502c19a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:52.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.077+0000 7f502b7fe640 1 --2- 172.21.15.96:0/3340829992 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f502c102a00 0x7f502c19a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:52.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.078+0000 7f502bfff640 1 -- 172.21.15.96:0/3340829992 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f502c102a00 msgr2=0x7f502c19a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:52.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.078+0000 7f502bfff640 1 --2- 172.21.15.96:0/3340829992 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f502c102a00 0x7f502c19a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:52.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.078+0000 7f502bfff640 1 -- 172.21.15.96:0/3340829992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f502c103c40 msgr2=0x7f502c19a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:52.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.078+0000 7f502bfff640 1 --2- 172.21.15.96:0/3340829992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f502c103c40 0x7f502c19a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:52.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.078+0000 7f502bfff640 1 -- 172.21.15.96:0/3340829992 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f502c1a1cc0 con 0x7f502c108940 2024-04-25T14:15:52.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.078+0000 7f502bfff640 1 --2- 172.21.15.96:0/3340829992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f502c108940 0x7f502c1a1650 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f50240097b0 tx=0x7f5024031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:52.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.078+0000 7f50297fa640 1 -- 172.21.15.96:0/3340829992 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5024003bf0 con 0x7f502c108940 2024-04-25T14:15:52.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.078+0000 7f50297fa640 1 -- 172.21.15.96:0/3340829992 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5024003d90 con 0x7f502c108940 2024-04-25T14:15:52.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.078+0000 7f5031bf3640 1 -- 172.21.15.96:0/3340829992 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f502c1a1f50 con 0x7f502c108940 2024-04-25T14:15:52.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.078+0000 7f50297fa640 1 -- 172.21.15.96:0/3340829992 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502402fc30 con 0x7f502c108940 2024-04-25T14:15:52.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.078+0000 7f5031bf3640 1 -- 172.21.15.96:0/3340829992 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f502c1a2410 con 0x7f502c108940 2024-04-25T14:15:52.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.079+0000 7f5031bf3640 1 -- 172.21.15.96:0/3340829992 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f502c108040 con 0x7f502c108940 2024-04-25T14:15:52.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.080+0000 7f50297fa640 1 -- 172.21.15.96:0/3340829992 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f5024004040 con 0x7f502c108940 2024-04-25T14:15:52.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.081+0000 7f50297fa640 1 --2- 172.21.15.96:0/3340829992 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f50180afb70 0x7f50180b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:52.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.081+0000 7f502b7fe640 1 --2- 172.21.15.96:0/3340829992 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f50180afb70 0x7f50180b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:52.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.081+0000 7f50297fa640 1 -- 172.21.15.96:0/3340829992 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f5024100520 con 0x7f502c108940 2024-04-25T14:15:52.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.081+0000 7f502b7fe640 1 --2- 172.21.15.96:0/3340829992 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f50180afb70 0x7f50180b2080 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f502c103600 tx=0x7f501c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:52.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.084+0000 7f50297fa640 1 -- 172.21.15.96:0/3340829992 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f50240c90e0 con 0x7f502c108940 2024-04-25T14:15:52.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.262+0000 7f5031bf3640 1 -- 172.21.15.96:0/3340829992 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f502c02dd20 con 0x7f502c108940 2024-04-25T14:15:52.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.264+0000 7f50297fa640 1 -- 172.21.15.96:0/3340829992 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22616 (secure 0 0 0) 0x7f50240c89e0 con 0x7f502c108940 2024-04-25T14:15:52.264 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:15:52.264 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":78,"fsid":"59334182-030c-11ef-bc93-c7b262605968","created":"2024-04-25T14:03:25.680745+0000","modified":"2024-04-25T14:12:22.496143+0000","last_up_change":"2024-04-25T14:10:35.894312+0000","last_in_change":"2024-04-25T14:10:15.583440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:06:47.402927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:11:27.434331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:11:27.854004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cf4ba3ac-142e-4e8e-96d4-410d77157eb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6803","nonce":3832506168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6805","nonce":3832506168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832506168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6807","nonce":3832506168}]},"public_addr":"172.21.15.96:6803/3832506168","cluster_addr":"172.21.15.96:6805/3832506168","heartbeat_back_addr":"172.21.15.96:6809/3832506168","heartbeat_front_addr":"172.21.15.96:6807/3832506168","state":["exists","up"]},{"osd":1,"uuid":"866e73b9-4a26-48ba-badf-704fe22f71cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6801","nonce":1083311494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6803","nonce":1083311494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6807","nonce":1083311494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6805","nonce":1083311494}]},"public_addr":"172.21.15.97:6801/1083311494","cluster_addr":"172.21.15.97:6803/1083311494","heartbeat_back_addr":"172.21.15.97:6807/1083311494","heartbeat_front_addr":"172.21.15.97:6805/1083311494","state":["exists","up"]},{"osd":2,"uuid":"45c79ecb-abc4-4dba-bae8-74804e4bc1bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6801","nonce":981902966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6803","nonce":981902966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6807","nonce":981902966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6805","nonce":981902966}]},"public_addr":"172.21.15.151:6801/981902966","cluster_addr":"172.21.15.151:6803/981902966","heartbeat_back_addr":"172.21.15.151:6807/981902966","heartbeat_front_addr":"172.21.15.151:6805/981902966","state":["exists","up"]},{"osd":3,"uuid":"9fc39468-a7ea-4ccf-b365-335ad2ea4c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6811","nonce":2274158748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6813","nonce":2274158748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6817","nonce":2274158748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6815","nonce":2274158748}]},"public_addr":"172.21.15.96:6811/2274158748","cluster_addr":"172.21.15.96:6813/2274158748","heartbeat_back_addr":"172.21.15.96:6817/2274158748","heartbeat_front_addr":"172.21.15.96:6815/2274158748","state":["exists","up"]},{"osd":4,"uuid":"616582f1-12d2-456b-b7b3-e5517800e8a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6809","nonce":3727031962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6811","nonce":3727031962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6815","nonce":3727031962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6813","nonce":3727031962}]},"public_addr":"172.21.15.97:6809/3727031962","cluster_addr":"172.21.15.97:6811/3727031962","heartbeat_back_addr":"172.21.15.97:6815/3727031962","heartbeat_front_addr":"172.21.15.97:6813/3727031962","state":["exists","up"]},{"osd":5,"uuid":"3d277784-ab0f-4c90-81a1-2a4034c52a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6809","nonce":223882546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6811","nonce":223882546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6815","nonce":223882546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6813","nonce":223882546}]},"public_addr":"172.21.15.151:6809/223882546","cluster_addr":"172.21.15.151:6811/223882546","heartbeat_back_addr":"172.21.15.151:6815/223882546","heartbeat_front_addr":"172.21.15.151:6813/223882546","state":["exists","up"]},{"osd":6,"uuid":"9a9ad585-cb1a-4239-9d18-055f59164049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6819","nonce":96197407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6821","nonce":96197407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6825","nonce":96197407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6823","nonce":96197407}]},"public_addr":"172.21.15.96:6819/96197407","cluster_addr":"172.21.15.96:6821/96197407","heartbeat_back_addr":"172.21.15.96:6825/96197407","heartbeat_front_addr":"172.21.15.96:6823/96197407","state":["exists","up"]},{"osd":7,"uuid":"303551c8-6af0-42f2-844f-45bf8db5baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6817","nonce":2247216400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6819","nonce":2247216400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6823","nonce":2247216400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6821","nonce":2247216400}]},"public_addr":"172.21.15.97:6817/2247216400","cluster_addr":"172.21.15.97:6819/2247216400","heartbeat_back_addr":"172.21.15.97:6823/2247216400","heartbeat_front_addr":"172.21.15.97:6821/2247216400","state":["exists","up"]},{"osd":8,"uuid":"81f6cc66-e4ca-430c-b336-b6678d968818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6817","nonce":2549822185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6819","nonce":2549822185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6823","nonce":2549822185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6821","nonce":2549822185}]},"public_addr":"172.21.15.151:6817/2549822185","cluster_addr":"172.21.15.151:6819/2549822185","heartbeat_back_addr":"172.21.15.151:6823/2549822185","heartbeat_front_addr":"172.21.15.151:6821/2549822185","state":["exists","up"]},{"osd":9,"uuid":"29d4da16-feab-4d1c-9d1d-de4f799bd724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6827","nonce":1099399643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6829","nonce":1099399643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6833","nonce":1099399643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6831","nonce":1099399643}]},"public_addr":"172.21.15.96:6827/1099399643","cluster_addr":"172.21.15.96:6829/1099399643","heartbeat_back_addr":"172.21.15.96:6833/1099399643","heartbeat_front_addr":"172.21.15.96:6831/1099399643","state":["exists","up"]},{"osd":10,"uuid":"197fee9b-eee4-4893-be1c-3d57f42250e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6825","nonce":155754520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6827","nonce":155754520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6831","nonce":155754520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6829","nonce":155754520}]},"public_addr":"172.21.15.97:6825/155754520","cluster_addr":"172.21.15.97:6827/155754520","heartbeat_back_addr":"172.21.15.97:6831/155754520","heartbeat_front_addr":"172.21.15.97:6829/155754520","state":["exists","up"]},{"osd":11,"uuid":"9c1d0a83-fc12-41a2-8eb0-80c4abdd2770","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6825","nonce":2027958746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6827","nonce":2027958746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6831","nonce":2027958746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6829","nonce":2027958746}]},"public_addr":"172.21.15.151:6825/2027958746","cluster_addr":"172.21.15.151:6827/2027958746","heartbeat_back_addr":"172.21.15.151:6831/2027958746","heartbeat_front_addr":"172.21.15.151:6829/2027958746","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:05:56.177390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:06:19.917663+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:06:43.608052+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:08.540382+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:33.474072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:58.242306+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:08:24.120442+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:08:49.661064+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:09:15.097271+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:09:41.904505+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:10:08.233252+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:10:33.712296+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/3839807484":"2024-04-26T14:04:21.307574+0000","172.21.15.96:0/1495237551":"2024-04-26T14:04:21.307574+0000","172.21.15.96:0/2793751450":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/129575834":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/2233075139":"2024-04-26T14:04:21.307574+0000","172.21.15.96:6801/2348397481":"2024-04-26T14:03:54.249537+0000","172.21.15.96:6801/433591062":"2024-04-26T14:04:21.307574+0000","172.21.15.96:6800/2348397481":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/3517674924":"2024-04-26T14:03:54.249537+0000","172.21.15.96:6800/433591062":"2024-04-26T14:04:21.307574+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:15:52.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.266+0000 7f5031bf3640 1 -- 172.21.15.96:0/3340829992 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f50180afb70 msgr2=0x7f50180b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:52.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.266+0000 7f5031bf3640 1 --2- 172.21.15.96:0/3340829992 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f50180afb70 0x7f50180b2080 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f502c103600 tx=0x7f501c005f90 comp rx=0 tx=0).stop 2024-04-25T14:15:52.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.266+0000 7f5031bf3640 1 -- 172.21.15.96:0/3340829992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f502c108940 msgr2=0x7f502c1a1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:52.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.266+0000 7f5031bf3640 1 --2- 172.21.15.96:0/3340829992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f502c108940 0x7f502c1a1650 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f50240097b0 tx=0x7f5024031840 comp rx=0 tx=0).stop 2024-04-25T14:15:52.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.266+0000 7f5031bf3640 1 -- 172.21.15.96:0/3340829992 shutdown_connections 2024-04-25T14:15:52.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.266+0000 7f5031bf3640 1 --2- 172.21.15.96:0/3340829992 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f50180afb70 0x7f50180b2080 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:52.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.266+0000 7f5031bf3640 1 --2- 172.21.15.96:0/3340829992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f502c108940 0x7f502c1a1650 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:52.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.266+0000 7f5031bf3640 1 --2- 172.21.15.96:0/3340829992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f502c103c40 0x7f502c19a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:52.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.266+0000 7f5031bf3640 1 --2- 172.21.15.96:0/3340829992 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f502c102a00 0x7f502c19a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:52.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.266+0000 7f5031bf3640 1 -- 172.21.15.96:0/3340829992 >> 172.21.15.96:0/3340829992 conn(0x7f502c0fdbf0 msgr2=0x7f502c101080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:52.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.266+0000 7f5031bf3640 1 -- 172.21.15.96:0/3340829992 shutdown_connections 2024-04-25T14:15:52.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.266+0000 7f5031bf3640 1 -- 172.21.15.96:0/3340829992 wait complete. 2024-04-25T14:15:52.278 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:06:47.402927+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:11:27.434331+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:11:27.854004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:15:52.279 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:15:52.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.389+0000 7efe60634640 1 -- 172.21.15.96:0/1964527963 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe581050b0 msgr2=0x7efe581054c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:52.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.389+0000 7efe60634640 1 --2- 172.21.15.96:0/1964527963 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe581050b0 0x7efe581054c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7efe54009900 tx=0x7efe5402a410 comp rx=0 tx=0).stop 2024-04-25T14:15:52.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.389+0000 7efe60634640 1 -- 172.21.15.96:0/1964527963 shutdown_connections 2024-04-25T14:15:52.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.389+0000 7efe60634640 1 --2- 172.21.15.96:0/1964527963 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe58102260 0x7efe5810bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:52.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.389+0000 7efe60634640 1 --2- 172.21.15.96:0/1964527963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe58101760 0x7efe58101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:52.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.389+0000 7efe60634640 1 --2- 172.21.15.96:0/1964527963 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe581050b0 0x7efe581054c0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:52.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.389+0000 7efe60634640 1 -- 172.21.15.96:0/1964527963 >> 172.21.15.96:0/1964527963 conn(0x7efe580fd380 msgr2=0x7efe580ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:52.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.389+0000 7efe60634640 1 -- 172.21.15.96:0/1964527963 shutdown_connections 2024-04-25T14:15:52.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.389+0000 7efe60634640 1 -- 172.21.15.96:0/1964527963 wait complete. 2024-04-25T14:15:52.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.390+0000 7efe60634640 1 Processor -- start 2024-04-25T14:15:52.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.390+0000 7efe60634640 1 -- start start 2024-04-25T14:15:52.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.390+0000 7efe60634640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe58101760 0x7efe58104260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:52.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.390+0000 7efe60634640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe58102260 0x7efe58102900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:52.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.390+0000 7efe60634640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe581050b0 0x7efe58102e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:52.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.390+0000 7efe60634640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efe5810f600 con 0x7efe58101760 2024-04-25T14:15:52.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.390+0000 7efe60634640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efe5810f480 con 0x7efe58102260 2024-04-25T14:15:52.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.390+0000 7efe60634640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efe5810f780 con 0x7efe581050b0 2024-04-25T14:15:52.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.390+0000 7efe5e3d0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe58101760 0x7efe58104260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:52.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.390+0000 7efe5e3d0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe58101760 0x7efe58104260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43944/0 (socket says 172.21.15.96:43944) 2024-04-25T14:15:52.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.390+0000 7efe5e3d0640 1 -- 172.21.15.96:0/1730928912 learned_addr learned my addr 172.21.15.96:0/1730928912 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:52.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.391+0000 7efe5ebd1640 1 --2- 172.21.15.96:0/1730928912 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe581050b0 0x7efe58102e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:52.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.391+0000 7efe5dbcf640 1 --2- 172.21.15.96:0/1730928912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe58102260 0x7efe58102900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:52.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.391+0000 7efe5e3d0640 1 -- 172.21.15.96:0/1730928912 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe581050b0 msgr2=0x7efe58102e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:52.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.391+0000 7efe5e3d0640 1 --2- 172.21.15.96:0/1730928912 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe581050b0 0x7efe58102e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:52.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.391+0000 7efe5e3d0640 1 -- 172.21.15.96:0/1730928912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe58102260 msgr2=0x7efe58102900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:52.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.391+0000 7efe5e3d0640 1 --2- 172.21.15.96:0/1730928912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe58102260 0x7efe58102900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:52.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.391+0000 7efe5e3d0640 1 -- 172.21.15.96:0/1730928912 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe58103570 con 0x7efe58101760 2024-04-25T14:15:52.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.391+0000 7efe5dbcf640 1 --2- 172.21.15.96:0/1730928912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe58102260 0x7efe58102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:15:52.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.391+0000 7efe5e3d0640 1 --2- 172.21.15.96:0/1730928912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe58101760 0x7efe58104260 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7efe54006010 tx=0x7efe54004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:52.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.391+0000 7efe4f7fe640 1 -- 172.21.15.96:0/1730928912 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5403f030 con 0x7efe58101760 2024-04-25T14:15:52.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.391+0000 7efe4f7fe640 1 -- 172.21.15.96:0/1730928912 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe54003aa0 con 0x7efe58101760 2024-04-25T14:15:52.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.391+0000 7efe60634640 1 -- 172.21.15.96:0/1730928912 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe581a4310 con 0x7efe58101760 2024-04-25T14:15:52.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.391+0000 7efe4f7fe640 1 -- 172.21.15.96:0/1730928912 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe54035070 con 0x7efe58101760 2024-04-25T14:15:52.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.391+0000 7efe60634640 1 -- 172.21.15.96:0/1730928912 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe581a4710 con 0x7efe58101760 2024-04-25T14:15:52.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.392+0000 7efe60634640 1 -- 172.21.15.96:0/1730928912 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe5806a160 con 0x7efe58101760 2024-04-25T14:15:52.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.394+0000 7efe4f7fe640 1 -- 172.21.15.96:0/1730928912 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7efe54042030 con 0x7efe58101760 2024-04-25T14:15:52.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.395+0000 7efe4f7fe640 1 --2- 172.21.15.96:0/1730928912 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7efe440afaf0 0x7efe440b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:52.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.395+0000 7efe4f7fe640 1 -- 172.21.15.96:0/1730928912 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7efe540ff6f0 con 0x7efe58101760 2024-04-25T14:15:52.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.395+0000 7efe5dbcf640 1 --2- 172.21.15.96:0/1730928912 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7efe440afaf0 0x7efe440b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:52.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.395+0000 7efe5dbcf640 1 --2- 172.21.15.96:0/1730928912 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7efe440afaf0 0x7efe440b2000 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7efe58103c00 tx=0x7efe48005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:52.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.399+0000 7efe4f7fe640 1 -- 172.21.15.96:0/1730928912 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efe540c82b0 con 0x7efe58101760 2024-04-25T14:15:52.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.578+0000 7efe60634640 1 -- 172.21.15.96:0/1730928912 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efe5802db50 con 0x7efe58101760 2024-04-25T14:15:52.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.578+0000 7efe4f7fe640 1 -- 172.21.15.96:0/1730928912 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) ==== 93+0+10 (secure 0 0 0) 0x7efe540c7bb0 con 0x7efe58101760 2024-04-25T14:15:52.578 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-04-25T14:15:52.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.580+0000 7efe60634640 1 -- 172.21.15.96:0/1730928912 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7efe440afaf0 msgr2=0x7efe440b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:52.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.580+0000 7efe60634640 1 --2- 172.21.15.96:0/1730928912 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7efe440afaf0 0x7efe440b2000 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7efe58103c00 tx=0x7efe48005f90 comp rx=0 tx=0).stop 2024-04-25T14:15:52.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.580+0000 7efe60634640 1 -- 172.21.15.96:0/1730928912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe58101760 msgr2=0x7efe58104260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:52.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.580+0000 7efe60634640 1 --2- 172.21.15.96:0/1730928912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe58101760 0x7efe58104260 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7efe54006010 tx=0x7efe54004270 comp rx=0 tx=0).stop 2024-04-25T14:15:52.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.580+0000 7efe60634640 1 -- 172.21.15.96:0/1730928912 shutdown_connections 2024-04-25T14:15:52.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.580+0000 7efe60634640 1 --2- 172.21.15.96:0/1730928912 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe581050b0 0x7efe58102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:52.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.580+0000 7efe60634640 1 --2- 172.21.15.96:0/1730928912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe58102260 0x7efe58102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:52.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.580+0000 7efe60634640 1 --2- 172.21.15.96:0/1730928912 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7efe440afaf0 0x7efe440b2000 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:52.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.580+0000 7efe60634640 1 --2- 172.21.15.96:0/1730928912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe58101760 0x7efe58104260 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:52.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.580+0000 7efe60634640 1 -- 172.21.15.96:0/1730928912 >> 172.21.15.96:0/1730928912 conn(0x7efe580fd380 msgr2=0x7efe58105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:52.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.580+0000 7efe60634640 1 -- 172.21.15.96:0/1730928912 shutdown_connections 2024-04-25T14:15:52.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.580+0000 7efe60634640 1 -- 172.21.15.96:0/1730928912 wait complete. 2024-04-25T14:15:52.592 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:15:52.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.704+0000 7f493f5e3640 1 -- 172.21.15.96:0/264813155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4938103c30 msgr2=0x7f4938104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:52.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.704+0000 7f493f5e3640 1 --2- 172.21.15.96:0/264813155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4938103c30 0x7f4938104040 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f4928009900 tx=0x7f492802a410 comp rx=0 tx=0).stop 2024-04-25T14:15:52.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.704+0000 7f493f5e3640 1 -- 172.21.15.96:0/264813155 shutdown_connections 2024-04-25T14:15:52.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.704+0000 7f493f5e3640 1 --2- 172.21.15.96:0/264813155 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4938108930 0x7f493810ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:52.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.704+0000 7f493f5e3640 1 --2- 172.21.15.96:0/264813155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4938103c30 0x7f4938104040 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:52.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.704+0000 7f493f5e3640 1 --2- 172.21.15.96:0/264813155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49381029f0 0x7f49381036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:52.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.704+0000 7f493f5e3640 1 -- 172.21.15.96:0/264813155 >> 172.21.15.96:0/264813155 conn(0x7f49380fdc00 msgr2=0x7f4938100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:52.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.704+0000 7f493f5e3640 1 -- 172.21.15.96:0/264813155 shutdown_connections 2024-04-25T14:15:52.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.704+0000 7f493f5e3640 1 -- 172.21.15.96:0/264813155 wait complete. 2024-04-25T14:15:52.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.704+0000 7f493f5e3640 1 Processor -- start 2024-04-25T14:15:52.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.704+0000 7f493f5e3640 1 -- start start 2024-04-25T14:15:52.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.705+0000 7f493f5e3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49381029f0 0x7f493819a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:52.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.705+0000 7f493f5e3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4938103c30 0x7f493819a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:52.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.705+0000 7f493f5e3640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4938108930 0x7f49381a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:52.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.705+0000 7f493f5e3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f493810e520 con 0x7f49381029f0 2024-04-25T14:15:52.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.705+0000 7f493f5e3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f493810e3a0 con 0x7f4938103c30 2024-04-25T14:15:52.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.705+0000 7f493f5e3640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f493810e6a0 con 0x7f4938108930 2024-04-25T14:15:52.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.705+0000 7f493d37f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49381029f0 0x7f493819a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:52.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.705+0000 7f493d37f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49381029f0 0x7f493819a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43956/0 (socket says 172.21.15.96:43956) 2024-04-25T14:15:52.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.705+0000 7f493d37f640 1 -- 172.21.15.96:0/2085589832 learned_addr learned my addr 172.21.15.96:0/2085589832 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:52.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.705+0000 7f493cb7e640 1 --2- 172.21.15.96:0/2085589832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4938103c30 0x7f493819a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:52.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.705+0000 7f493db80640 1 --2- 172.21.15.96:0/2085589832 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4938108930 0x7f49381a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:52.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.705+0000 7f493d37f640 1 -- 172.21.15.96:0/2085589832 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4938108930 msgr2=0x7f49381a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:52.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.705+0000 7f493d37f640 1 --2- 172.21.15.96:0/2085589832 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4938108930 0x7f49381a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:52.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.705+0000 7f493d37f640 1 -- 172.21.15.96:0/2085589832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4938103c30 msgr2=0x7f493819a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:52.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.705+0000 7f493d37f640 1 --2- 172.21.15.96:0/2085589832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4938103c30 0x7f493819a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:52.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.705+0000 7f493d37f640 1 -- 172.21.15.96:0/2085589832 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49381a1d30 con 0x7f49381029f0 2024-04-25T14:15:52.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.705+0000 7f493db80640 1 --2- 172.21.15.96:0/2085589832 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4938108930 0x7f49381a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:15:52.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.705+0000 7f493d37f640 1 --2- 172.21.15.96:0/2085589832 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49381029f0 0x7f493819a240 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f493400e9f0 tx=0x7f493400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:52.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.705+0000 7f492e7fc640 1 -- 172.21.15.96:0/2085589832 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493400ccc0 con 0x7f49381029f0 2024-04-25T14:15:52.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.706+0000 7f492e7fc640 1 -- 172.21.15.96:0/2085589832 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49340035f0 con 0x7f49381029f0 2024-04-25T14:15:52.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.706+0000 7f493f5e3640 1 -- 172.21.15.96:0/2085589832 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49381a1fc0 con 0x7f49381029f0 2024-04-25T14:15:52.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.706+0000 7f492e7fc640 1 -- 172.21.15.96:0/2085589832 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493400ce60 con 0x7f49381029f0 2024-04-25T14:15:52.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.706+0000 7f493f5e3640 1 -- 172.21.15.96:0/2085589832 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49381a2480 con 0x7f49381029f0 2024-04-25T14:15:52.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.707+0000 7f493f5e3640 1 -- 172.21.15.96:0/2085589832 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49381a27a0 con 0x7f49381029f0 2024-04-25T14:15:52.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.708+0000 7f492e7fc640 1 -- 172.21.15.96:0/2085589832 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f4934003aa0 con 0x7f49381029f0 2024-04-25T14:15:52.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.709+0000 7f492e7fc640 1 --2- 172.21.15.96:0/2085589832 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f49240afaf0 0x7f49240b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:52.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.709+0000 7f493cb7e640 1 --2- 172.21.15.96:0/2085589832 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f49240afaf0 0x7f49240b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:52.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.709+0000 7f492e7fc640 1 -- 172.21.15.96:0/2085589832 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f49340df1c0 con 0x7f49381029f0 2024-04-25T14:15:52.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.709+0000 7f493cb7e640 1 --2- 172.21.15.96:0/2085589832 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f49240afaf0 0x7f49240b2000 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4928009230 tx=0x7f4928031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:52.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.712+0000 7f492e7fc640 1 -- 172.21.15.96:0/2085589832 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4934053ff0 con 0x7f49381029f0 2024-04-25T14:15:52.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:52 smithi096 ceph-mon[29207]: pgmap v413: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 853 B/s wr, 13 op/s 2024-04-25T14:15:52.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:52 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/3340829992' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:15:52.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:52 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/1730928912' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:15:52.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.898+0000 7f493f5e3640 1 -- 172.21.15.96:0/2085589832 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f49381a2970 con 0x7f49381029f0 2024-04-25T14:15:52.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.898+0000 7f492e7fc640 1 -- 172.21.15.96:0/2085589832 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v78) ==== 104+0+11 (secure 0 0 0) 0x7f49340a7d80 con 0x7f49381029f0 2024-04-25T14:15:52.898 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-04-25T14:15:52.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.900+0000 7f493f5e3640 1 -- 172.21.15.96:0/2085589832 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f49240afaf0 msgr2=0x7f49240b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:52.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.900+0000 7f493f5e3640 1 --2- 172.21.15.96:0/2085589832 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f49240afaf0 0x7f49240b2000 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4928009230 tx=0x7f4928031400 comp rx=0 tx=0).stop 2024-04-25T14:15:52.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.900+0000 7f493f5e3640 1 -- 172.21.15.96:0/2085589832 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49381029f0 msgr2=0x7f493819a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:52.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.900+0000 7f493f5e3640 1 --2- 172.21.15.96:0/2085589832 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49381029f0 0x7f493819a240 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f493400e9f0 tx=0x7f493400eeb0 comp rx=0 tx=0).stop 2024-04-25T14:15:52.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.900+0000 7f493f5e3640 1 -- 172.21.15.96:0/2085589832 shutdown_connections 2024-04-25T14:15:52.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.900+0000 7f493f5e3640 1 --2- 172.21.15.96:0/2085589832 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4938108930 0x7f49381a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:52.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.900+0000 7f493f5e3640 1 --2- 172.21.15.96:0/2085589832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4938103c30 0x7f493819a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:52.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.900+0000 7f493f5e3640 1 --2- 172.21.15.96:0/2085589832 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f49240afaf0 0x7f49240b2000 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:52.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.900+0000 7f493f5e3640 1 --2- 172.21.15.96:0/2085589832 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49381029f0 0x7f493819a240 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:52.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.900+0000 7f493f5e3640 1 -- 172.21.15.96:0/2085589832 >> 172.21.15.96:0/2085589832 conn(0x7f49380fdc00 msgr2=0x7f4938109e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:52.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.900+0000 7f493f5e3640 1 -- 172.21.15.96:0/2085589832 shutdown_connections 2024-04-25T14:15:52.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:52.900+0000 7f493f5e3640 1 -- 172.21.15.96:0/2085589832 wait complete. 2024-04-25T14:15:52.912 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:15:52.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:52 smithi097 ceph-mon[32720]: pgmap v413: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 853 B/s wr, 13 op/s 2024-04-25T14:15:52.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:52 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/3340829992' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:15:52.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:52 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/1730928912' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:15:52.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:52 smithi151 ceph-mon[32990]: pgmap v413: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 853 B/s wr, 13 op/s 2024-04-25T14:15:52.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:52 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/3340829992' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:15:52.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:52 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/1730928912' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:15:53.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.025+0000 7fb92850e640 1 -- 172.21.15.96:0/2910619036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9201080b0 msgr2=0x7fb92010a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:53.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.025+0000 7fb92850e640 1 --2- 172.21.15.96:0/2910619036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9201080b0 0x7fb92010a4d0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fb918009ca0 tx=0x7fb91802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:15:53.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.025+0000 7fb92850e640 1 -- 172.21.15.96:0/2910619036 shutdown_connections 2024-04-25T14:15:53.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.025+0000 7fb92850e640 1 --2- 172.21.15.96:0/2910619036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9201080b0 0x7fb92010a4d0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:53.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.025+0000 7fb92850e640 1 --2- 172.21.15.96:0/2910619036 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9201033b0 0x7fb9201037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:53.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.025+0000 7fb92850e640 1 --2- 172.21.15.96:0/2910619036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb920102170 0x7fb920102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:53.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.025+0000 7fb92850e640 1 -- 172.21.15.96:0/2910619036 >> 172.21.15.96:0/2910619036 conn(0x7fb9200fd340 msgr2=0x7fb9200ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:53.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.025+0000 7fb92850e640 1 -- 172.21.15.96:0/2910619036 shutdown_connections 2024-04-25T14:15:53.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.025+0000 7fb92850e640 1 -- 172.21.15.96:0/2910619036 wait complete. 2024-04-25T14:15:53.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.026+0000 7fb92850e640 1 Processor -- start 2024-04-25T14:15:53.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.026+0000 7fb92850e640 1 -- start start 2024-04-25T14:15:53.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.026+0000 7fb92850e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb920102170 0x7fb92010ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:53.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.026+0000 7fb92850e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9201033b0 0x7fb92019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:53.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.026+0000 7fb92850e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9201080b0 0x7fb9201a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:53.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.026+0000 7fb92850e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb92010e510 con 0x7fb9201033b0 2024-04-25T14:15:53.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.026+0000 7fb92850e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb92010e390 con 0x7fb9201080b0 2024-04-25T14:15:53.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.026+0000 7fb92850e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb92010e690 con 0x7fb920102170 2024-04-25T14:15:53.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.026+0000 7fb925aa9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9201033b0 0x7fb92019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:53.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.026+0000 7fb925aa9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9201033b0 0x7fb92019efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43972/0 (socket says 172.21.15.96:43972) 2024-04-25T14:15:53.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.027+0000 7fb925aa9640 1 -- 172.21.15.96:0/2613403959 learned_addr learned my addr 172.21.15.96:0/2613403959 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:53.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.027+0000 7fb926aab640 1 --2- 172.21.15.96:0/2613403959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9201080b0 0x7fb9201a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:53.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.027+0000 7fb9262aa640 1 --2- 172.21.15.96:0/2613403959 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb920102170 0x7fb92010ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:53.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.027+0000 7fb925aa9640 1 -- 172.21.15.96:0/2613403959 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb920102170 msgr2=0x7fb92010ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:53.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.027+0000 7fb925aa9640 1 --2- 172.21.15.96:0/2613403959 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb920102170 0x7fb92010ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:53.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.027+0000 7fb925aa9640 1 -- 172.21.15.96:0/2613403959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9201080b0 msgr2=0x7fb9201a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:53.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.027+0000 7fb925aa9640 1 --2- 172.21.15.96:0/2613403959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9201080b0 0x7fb9201a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:53.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.027+0000 7fb925aa9640 1 -- 172.21.15.96:0/2613403959 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9201a1d30 con 0x7fb9201033b0 2024-04-25T14:15:53.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.027+0000 7fb926aab640 1 --2- 172.21.15.96:0/2613403959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9201080b0 0x7fb9201a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:15:53.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.027+0000 7fb925aa9640 1 --2- 172.21.15.96:0/2613403959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9201033b0 0x7fb92019efc0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb91000e9f0 tx=0x7fb91000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:53.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.027+0000 7fb9177fe640 1 -- 172.21.15.96:0/2613403959 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91000cd80 con 0x7fb9201033b0 2024-04-25T14:15:53.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.027+0000 7fb9177fe640 1 -- 172.21.15.96:0/2613403959 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9100054d0 con 0x7fb9201033b0 2024-04-25T14:15:53.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.027+0000 7fb92850e640 1 -- 172.21.15.96:0/2613403959 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9201a1fc0 con 0x7fb9201033b0 2024-04-25T14:15:53.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.027+0000 7fb9177fe640 1 -- 172.21.15.96:0/2613403959 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb910010690 con 0x7fb9201033b0 2024-04-25T14:15:53.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.027+0000 7fb92850e640 1 -- 172.21.15.96:0/2613403959 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9201a2460 con 0x7fb9201033b0 2024-04-25T14:15:53.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.028+0000 7fb92850e640 1 -- 172.21.15.96:0/2613403959 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb920106ec0 con 0x7fb9201033b0 2024-04-25T14:15:53.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.030+0000 7fb9177fe640 1 -- 172.21.15.96:0/2613403959 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fb910004e30 con 0x7fb9201033b0 2024-04-25T14:15:53.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.030+0000 7fb9177fe640 1 --2- 172.21.15.96:0/2613403959 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb90c0afaf0 0x7fb90c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:53.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.031+0000 7fb9262aa640 1 --2- 172.21.15.96:0/2613403959 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb90c0afaf0 0x7fb90c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:53.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.031+0000 7fb9177fe640 1 -- 172.21.15.96:0/2613403959 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fb9100df230 con 0x7fb9201033b0 2024-04-25T14:15:53.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.031+0000 7fb9262aa640 1 --2- 172.21.15.96:0/2613403959 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb90c0afaf0 0x7fb90c0b2000 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb920102d70 tx=0x7fb91c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:53.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.034+0000 7fb9177fe640 1 -- 172.21.15.96:0/2613403959 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb910053ff0 con 0x7fb9201033b0 2024-04-25T14:15:53.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.229+0000 7fb92850e640 1 -- 172.21.15.96:0/2613403959 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb92002dce0 con 0x7fb9201033b0 2024-04-25T14:15:53.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.230+0000 7fb9177fe640 1 -- 172.21.15.96:0/2613403959 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v78) ==== 100+0+11 (secure 0 0 0) 0x7fb9100aee20 con 0x7fb9201033b0 2024-04-25T14:15:53.230 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-04-25T14:15:53.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.232+0000 7fb92850e640 1 -- 172.21.15.96:0/2613403959 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb90c0afaf0 msgr2=0x7fb90c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:53.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.232+0000 7fb92850e640 1 --2- 172.21.15.96:0/2613403959 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb90c0afaf0 0x7fb90c0b2000 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb920102d70 tx=0x7fb91c005f90 comp rx=0 tx=0).stop 2024-04-25T14:15:53.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.232+0000 7fb92850e640 1 -- 172.21.15.96:0/2613403959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9201033b0 msgr2=0x7fb92019efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:53.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.232+0000 7fb92850e640 1 --2- 172.21.15.96:0/2613403959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9201033b0 0x7fb92019efc0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb91000e9f0 tx=0x7fb91000edf0 comp rx=0 tx=0).stop 2024-04-25T14:15:53.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.232+0000 7fb92850e640 1 -- 172.21.15.96:0/2613403959 shutdown_connections 2024-04-25T14:15:53.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.232+0000 7fb92850e640 1 --2- 172.21.15.96:0/2613403959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9201080b0 0x7fb9201a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:53.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.232+0000 7fb92850e640 1 --2- 172.21.15.96:0/2613403959 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fb90c0afaf0 0x7fb90c0b2000 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:53.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.232+0000 7fb92850e640 1 --2- 172.21.15.96:0/2613403959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9201033b0 0x7fb92019efc0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:53.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.232+0000 7fb92850e640 1 --2- 172.21.15.96:0/2613403959 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb920102170 0x7fb92010ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:53.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.232+0000 7fb92850e640 1 -- 172.21.15.96:0/2613403959 >> 172.21.15.96:0/2613403959 conn(0x7fb9200fd340 msgr2=0x7fb920104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:53.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.232+0000 7fb92850e640 1 -- 172.21.15.96:0/2613403959 shutdown_connections 2024-04-25T14:15:53.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.232+0000 7fb92850e640 1 -- 172.21.15.96:0/2613403959 wait complete. 2024-04-25T14:15:53.244 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-25T14:15:53.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.355+0000 7f66ab534640 1 -- 172.21.15.96:0/2275726496 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66a410b9b0 msgr2=0x7f66a410dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:53.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.355+0000 7f66ab534640 1 --2- 172.21.15.96:0/2275726496 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66a410b9b0 0x7f66a410dec0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f669c009ca0 tx=0x7f669c02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:15:53.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.356+0000 7f66ab534640 1 -- 172.21.15.96:0/2275726496 shutdown_connections 2024-04-25T14:15:53.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.356+0000 7f66ab534640 1 --2- 172.21.15.96:0/2275726496 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66a410b9b0 0x7f66a410dec0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:53.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.356+0000 7f66ab534640 1 --2- 172.21.15.96:0/2275726496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66a4108e80 0x7f66a410b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:53.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.356+0000 7f66ab534640 1 --2- 172.21.15.96:0/2275726496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f66a4103fd0 0x7f66a41043e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:53.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.356+0000 7f66ab534640 1 -- 172.21.15.96:0/2275726496 >> 172.21.15.96:0/2275726496 conn(0x7f66a40fd3b0 msgr2=0x7f66a40ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:53.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.356+0000 7f66ab534640 1 -- 172.21.15.96:0/2275726496 shutdown_connections 2024-04-25T14:15:53.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.356+0000 7f66ab534640 1 -- 172.21.15.96:0/2275726496 wait complete. 2024-04-25T14:15:53.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.356+0000 7f66ab534640 1 Processor -- start 2024-04-25T14:15:53.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.356+0000 7f66ab534640 1 -- start start 2024-04-25T14:15:53.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.356+0000 7f66ab534640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66a4103fd0 0x7f66a419c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:53.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.356+0000 7f66ab534640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66a4108e80 0x7f66a419c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:53.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.356+0000 7f66ab534640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f66a410b9b0 0x7f66a41a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:53.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.356+0000 7f66ab534640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f66a4111850 con 0x7f66a4108e80 2024-04-25T14:15:53.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.356+0000 7f66ab534640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f66a41116d0 con 0x7f66a4103fd0 2024-04-25T14:15:53.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.356+0000 7f66ab534640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f66a41119d0 con 0x7f66a410b9b0 2024-04-25T14:15:53.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.357+0000 7f66a8acf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66a4108e80 0x7f66a419c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:53.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.357+0000 7f66a8acf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66a4108e80 0x7f66a419c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44000/0 (socket says 172.21.15.96:44000) 2024-04-25T14:15:53.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.357+0000 7f66a8acf640 1 -- 172.21.15.96:0/392973496 learned_addr learned my addr 172.21.15.96:0/392973496 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:15:53.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.357+0000 7f66a92d0640 1 --2- 172.21.15.96:0/392973496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66a4103fd0 0x7f66a419c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:53.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.357+0000 7f66a9ad1640 1 --2- 172.21.15.96:0/392973496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f66a410b9b0 0x7f66a41a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:53.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.357+0000 7f66a8acf640 1 -- 172.21.15.96:0/392973496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f66a410b9b0 msgr2=0x7f66a41a3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:53.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.357+0000 7f66a8acf640 1 --2- 172.21.15.96:0/392973496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f66a410b9b0 0x7f66a41a3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:53.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.357+0000 7f66a8acf640 1 -- 172.21.15.96:0/392973496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66a4103fd0 msgr2=0x7f66a419c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:53.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.357+0000 7f66a8acf640 1 --2- 172.21.15.96:0/392973496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66a4103fd0 0x7f66a419c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:53.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.357+0000 7f66a8acf640 1 -- 172.21.15.96:0/392973496 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a41a3ee0 con 0x7f66a4108e80 2024-04-25T14:15:53.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.357+0000 7f66a8acf640 1 --2- 172.21.15.96:0/392973496 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66a4108e80 0x7f66a419c8e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f66a000e9f0 tx=0x7f66a000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:53.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.357+0000 7f66967fc640 1 -- 172.21.15.96:0/392973496 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a000cd80 con 0x7f66a4108e80 2024-04-25T14:15:53.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.357+0000 7f66967fc640 1 -- 172.21.15.96:0/392973496 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66a00108e0 con 0x7f66a4108e80 2024-04-25T14:15:53.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.357+0000 7f66ab534640 1 -- 172.21.15.96:0/392973496 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66a41a41d0 con 0x7f66a4108e80 2024-04-25T14:15:53.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.357+0000 7f66967fc640 1 -- 172.21.15.96:0/392973496 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a0018980 con 0x7f66a4108e80 2024-04-25T14:15:53.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.357+0000 7f66ab534640 1 -- 172.21.15.96:0/392973496 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66a41a4720 con 0x7f66a4108e80 2024-04-25T14:15:53.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.358+0000 7f66ab534640 1 -- 172.21.15.96:0/392973496 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66a40feed0 con 0x7f66a4108e80 2024-04-25T14:15:53.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.360+0000 7f66967fc640 1 -- 172.21.15.96:0/392973496 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f66a0010430 con 0x7f66a4108e80 2024-04-25T14:15:53.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.360+0000 7f66967fc640 1 --2- 172.21.15.96:0/392973496 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f66900afaf0 0x7f66900b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:53.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.360+0000 7f66a92d0640 1 --2- 172.21.15.96:0/392973496 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f66900afaf0 0x7f66900b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:53.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.360+0000 7f66967fc640 1 -- 172.21.15.96:0/392973496 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f66a00df3d0 con 0x7f66a4108e80 2024-04-25T14:15:53.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.361+0000 7f66a92d0640 1 --2- 172.21.15.96:0/392973496 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f66900afaf0 0x7f66900b2000 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f66a406b520 tx=0x7f6698005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:53.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.364+0000 7f66967fc640 1 -- 172.21.15.96:0/392973496 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f66a00a7940 con 0x7f66a4108e80 2024-04-25T14:15:53.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.597+0000 7f66ab534640 1 -- 172.21.15.96:0/392973496 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f66a40637a0 con 0x7f66a4108e80 2024-04-25T14:15:53.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.606+0000 7f66967fc640 1 -- 172.21.15.96:0/392973496 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+803122 (secure 0 0 0) 0x7f66a00abc40 con 0x7f66a4108e80 2024-04-25T14:15:53.617 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:15:53.617 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T14:15:53.617 INFO:teuthology.orchestra.run.smithi096.stdout: "epoch": 18, 2024-04-25T14:15:53.617 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.617 INFO:teuthology.orchestra.run.smithi096.stdout: "active_gid": 14152, 2024-04-25T14:15:53.617 INFO:teuthology.orchestra.run.smithi096.stdout: "active_name": "x", 2024-04-25T14:15:53.617 INFO:teuthology.orchestra.run.smithi096.stdout: "active_addrs": { 2024-04-25T14:15:53.617 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-25T14:15:53.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:53.618 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-25T14:15:53.618 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6800", 2024-04-25T14:15:53.618 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 536332425 2024-04-25T14:15:53.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:53.618 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-04-25T14:15:53.618 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6801", 2024-04-25T14:15:53.618 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 536332425 2024-04-25T14:15:53.618 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.618 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-25T14:15:53.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.618 INFO:teuthology.orchestra.run.smithi096.stdout: "active_addr": "172.21.15.96:6801/536332425", 2024-04-25T14:15:53.618 INFO:teuthology.orchestra.run.smithi096.stdout: "active_change": "2024-04-25T14:04:21.307702+0000", 2024-04-25T14:15:53.618 INFO:teuthology.orchestra.run.smithi096.stdout: "active_mgr_features": 4540701547738038271, 2024-04-25T14:15:53.619 INFO:teuthology.orchestra.run.smithi096.stdout: "available": true, 2024-04-25T14:15:53.619 INFO:teuthology.orchestra.run.smithi096.stdout: "standbys": [ 2024-04-25T14:15:53.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:53.619 INFO:teuthology.orchestra.run.smithi096.stdout: "gid": 24106, 2024-04-25T14:15:53.619 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "z", 2024-04-25T14:15:53.619 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_features": 4540701547738038271, 2024-04-25T14:15:53.619 INFO:teuthology.orchestra.run.smithi096.stdout: "available_modules": [ 2024-04-25T14:15:53.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:53.619 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "alerts", 2024-04-25T14:15:53.619 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:53.619 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:53.619 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:53.619 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-04-25T14:15:53.620 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-04-25T14:15:53.620 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:53.620 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.620 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.620 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-04-25T14:15:53.620 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.620 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.620 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.620 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T14:15:53.620 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.620 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.620 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.621 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:53.621 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:53.621 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.621 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.621 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.621 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.621 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.621 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.621 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.621 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.621 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.621 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.621 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.621 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.622 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.622 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.622 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.622 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.622 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.622 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.622 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:53.622 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:53.622 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.622 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.622 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.622 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.623 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.623 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.623 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.623 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.623 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.623 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.623 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.623 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:53.623 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:53.623 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.623 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.623 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.623 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:53.624 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.624 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.624 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.624 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.624 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.624 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.624 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.624 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.624 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.624 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.624 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.624 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.624 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.624 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.625 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:53.625 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:53.625 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.625 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.625 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.625 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.625 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.625 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.625 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.625 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.625 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.625 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.625 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.626 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_destination": { 2024-04-25T14:15:53.626 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_destination", 2024-04-25T14:15:53.626 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.626 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.626 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.626 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.626 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.626 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.626 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.626 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Email address to send alerts to", 2024-04-25T14:15:53.626 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.626 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.626 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.627 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_from_name": { 2024-04-25T14:15:53.627 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_from_name", 2024-04-25T14:15:53.627 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.627 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.627 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.627 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "Ceph", 2024-04-25T14:15:53.627 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.627 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.627 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.627 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Email From: name", 2024-04-25T14:15:53.627 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.627 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.627 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.628 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_host": { 2024-04-25T14:15:53.628 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_host", 2024-04-25T14:15:53.628 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.628 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.628 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.628 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.628 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.628 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.628 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.628 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SMTP server", 2024-04-25T14:15:53.628 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.628 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.628 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.628 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_password": { 2024-04-25T14:15:53.629 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_password", 2024-04-25T14:15:53.629 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.629 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.629 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.629 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.629 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.629 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.629 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.629 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Password to authenticate with", 2024-04-25T14:15:53.629 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.629 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.630 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.630 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_port": { 2024-04-25T14:15:53.630 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_port", 2024-04-25T14:15:53.630 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.630 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.630 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.630 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "465", 2024-04-25T14:15:53.630 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.630 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.630 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.630 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SMTP port", 2024-04-25T14:15:53.630 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.631 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.631 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.631 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_sender": { 2024-04-25T14:15:53.631 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_sender", 2024-04-25T14:15:53.631 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.631 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.631 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.631 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.631 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.631 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.631 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.632 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SMTP envelope sender", 2024-04-25T14:15:53.632 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.632 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.632 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.632 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_ssl": { 2024-04-25T14:15:53.632 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_ssl", 2024-04-25T14:15:53.632 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.632 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.632 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.632 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:53.633 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.633 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.633 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.633 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T14:15:53.633 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.633 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.633 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.633 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_user": { 2024-04-25T14:15:53.633 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_user", 2024-04-25T14:15:53.633 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.633 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.633 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.634 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.634 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.634 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.634 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.634 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "User to authenticate as", 2024-04-25T14:15:53.634 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.634 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.634 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.634 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:53.634 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:53.634 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.634 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:53.635 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.635 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:53.635 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.635 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.635 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.635 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.635 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.635 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.635 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.635 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.635 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:53.636 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "balancer", 2024-04-25T14:15:53.636 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:53.636 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:53.636 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:53.636 INFO:teuthology.orchestra.run.smithi096.stdout: "active": { 2024-04-25T14:15:53.636 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "active", 2024-04-25T14:15:53.636 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.636 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.636 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.636 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:53.636 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.636 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.636 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.636 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T14:15:53.637 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.637 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.637 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.637 INFO:teuthology.orchestra.run.smithi096.stdout: "begin_time": { 2024-04-25T14:15:53.637 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "begin_time", 2024-04-25T14:15:53.637 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.637 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.637 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.637 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0000", 2024-04-25T14:15:53.637 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.637 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.637 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.637 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T14:15:53.638 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T14:15:53.638 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.638 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.638 INFO:teuthology.orchestra.run.smithi096.stdout: "begin_weekday": { 2024-04-25T14:15:53.638 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "begin_weekday", 2024-04-25T14:15:53.638 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-04-25T14:15:53.638 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.638 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.638 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:53.638 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "0", 2024-04-25T14:15:53.638 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "6", 2024-04-25T14:15:53.639 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.639 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T14:15:53.639 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T14:15:53.639 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.639 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.639 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_compat_max_iterations": { 2024-04-25T14:15:53.639 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crush_compat_max_iterations", 2024-04-25T14:15:53.639 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-04-25T14:15:53.639 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.639 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.639 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "25", 2024-04-25T14:15:53.639 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-04-25T14:15:53.640 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "250", 2024-04-25T14:15:53.640 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.640 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T14:15:53.640 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.640 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.640 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.640 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_compat_metrics": { 2024-04-25T14:15:53.640 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crush_compat_metrics", 2024-04-25T14:15:53.640 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.640 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.640 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.640 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T14:15:53.641 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.641 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.641 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.641 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T14:15:53.641 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T14:15:53.641 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.641 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.641 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_compat_step": { 2024-04-25T14:15:53.641 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crush_compat_step", 2024-04-25T14:15:53.641 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T14:15:53.641 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.642 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.642 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0.5", 2024-04-25T14:15:53.642 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "0.001", 2024-04-25T14:15:53.642 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "0.999", 2024-04-25T14:15:53.642 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.642 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "aggressiveness of optimization", 2024-04-25T14:15:53.642 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T14:15:53.642 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.642 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.642 INFO:teuthology.orchestra.run.smithi096.stdout: "end_time": { 2024-04-25T14:15:53.642 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "end_time", 2024-04-25T14:15:53.642 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.643 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.643 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.643 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "2359", 2024-04-25T14:15:53.643 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.643 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.643 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.643 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T14:15:53.643 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T14:15:53.643 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.643 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.643 INFO:teuthology.orchestra.run.smithi096.stdout: "end_weekday": { 2024-04-25T14:15:53.643 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "end_weekday", 2024-04-25T14:15:53.644 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-04-25T14:15:53.644 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.644 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.644 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:53.644 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "0", 2024-04-25T14:15:53.644 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "6", 2024-04-25T14:15:53.644 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.644 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T14:15:53.644 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T14:15:53.644 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.644 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.644 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:53.644 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:53.645 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.645 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.645 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.645 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.645 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.645 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.645 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.645 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.645 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.645 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.645 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.645 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.646 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.646 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.646 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.646 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.646 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.646 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.646 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:53.646 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:53.646 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.646 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.646 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.646 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.647 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.647 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.647 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.647 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.647 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.647 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.647 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.647 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:53.647 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:53.647 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.647 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.647 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.647 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:53.648 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.648 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.648 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.648 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.648 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.648 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.648 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.648 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.648 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.648 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.648 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.648 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.648 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.649 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.649 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:53.649 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:53.649 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.649 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.649 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.649 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.649 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.649 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.649 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.649 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.649 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.649 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.650 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.650 INFO:teuthology.orchestra.run.smithi096.stdout: "min_score": { 2024-04-25T14:15:53.650 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "min_score", 2024-04-25T14:15:53.650 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T14:15:53.650 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.650 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.650 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:53.650 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.650 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.650 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.651 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T14:15:53.651 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.651 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.651 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.651 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": { 2024-04-25T14:15:53.651 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mode", 2024-04-25T14:15:53.651 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.651 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.651 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.651 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "upmap", 2024-04-25T14:15:53.651 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.651 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.652 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.652 INFO:teuthology.orchestra.run.smithi096.stdout: "crush-compat", 2024-04-25T14:15:53.652 INFO:teuthology.orchestra.run.smithi096.stdout: "none", 2024-04-25T14:15:53.652 INFO:teuthology.orchestra.run.smithi096.stdout: "read", 2024-04-25T14:15:53.652 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap", 2024-04-25T14:15:53.652 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap-read" 2024-04-25T14:15:53.652 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.652 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Balancer mode", 2024-04-25T14:15:53.652 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.652 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.652 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.652 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_ids": { 2024-04-25T14:15:53.652 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pool_ids", 2024-04-25T14:15:53.653 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.653 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.653 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.653 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.653 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.653 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.653 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.653 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T14:15:53.653 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.653 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.653 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.653 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-04-25T14:15:53.654 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-04-25T14:15:53.654 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:53.654 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.654 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.654 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-04-25T14:15:53.654 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.654 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.654 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.654 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T14:15:53.654 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.654 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.654 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.655 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:53.655 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:53.655 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.655 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:53.655 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.655 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:53.655 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.655 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.655 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.655 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.655 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.655 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.655 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.656 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap_max_deviation": { 2024-04-25T14:15:53.656 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "upmap_max_deviation", 2024-04-25T14:15:53.656 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.656 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.656 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.656 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-04-25T14:15:53.656 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-04-25T14:15:53.656 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.656 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.656 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T14:15:53.656 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T14:15:53.656 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.656 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.657 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap_max_optimizations": { 2024-04-25T14:15:53.657 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "upmap_max_optimizations", 2024-04-25T14:15:53.657 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-04-25T14:15:53.657 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.657 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.657 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T14:15:53.657 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.657 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.657 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.657 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T14:15:53.657 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.657 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.657 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.658 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.658 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:53.658 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cephadm", 2024-04-25T14:15:53.658 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:53.658 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:53.658 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:53.658 INFO:teuthology.orchestra.run.smithi096.stdout: "agent_down_multiplier": { 2024-04-25T14:15:53.658 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "agent_down_multiplier", 2024-04-25T14:15:53.658 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T14:15:53.658 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.658 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.658 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "3.0", 2024-04-25T14:15:53.659 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.659 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.659 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.659 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T14:15:53.659 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.659 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.659 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.659 INFO:teuthology.orchestra.run.smithi096.stdout: "agent_refresh_rate": { 2024-04-25T14:15:53.659 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "agent_refresh_rate", 2024-04-25T14:15:53.659 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:53.659 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.660 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.660 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "20", 2024-04-25T14:15:53.660 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.660 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.660 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.660 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T14:15:53.660 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.660 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.660 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.660 INFO:teuthology.orchestra.run.smithi096.stdout: "agent_starting_port": { 2024-04-25T14:15:53.660 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "agent_starting_port", 2024-04-25T14:15:53.660 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.661 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.661 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.661 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "4721", 2024-04-25T14:15:53.661 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.661 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.661 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.661 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T14:15:53.661 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.661 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.661 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.661 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_ptrace": { 2024-04-25T14:15:53.661 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "allow_ptrace", 2024-04-25T14:15:53.661 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.662 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.662 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.662 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.662 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.662 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.662 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.662 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T14:15:53.662 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T14:15:53.662 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.662 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.662 INFO:teuthology.orchestra.run.smithi096.stdout: "autotune_interval": { 2024-04-25T14:15:53.662 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "autotune_interval", 2024-04-25T14:15:53.663 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:53.663 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.663 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.663 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-04-25T14:15:53.663 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.663 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.663 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.663 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T14:15:53.663 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.663 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.663 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.663 INFO:teuthology.orchestra.run.smithi096.stdout: "autotune_memory_target_ratio": { 2024-04-25T14:15:53.663 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T14:15:53.664 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T14:15:53.664 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.664 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.664 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0.7", 2024-04-25T14:15:53.664 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.664 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.664 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.664 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T14:15:53.664 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.664 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.664 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.664 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm_log_destination": { 2024-04-25T14:15:53.665 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cephadm_log_destination", 2024-04-25T14:15:53.665 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.665 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.665 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.665 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.665 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.665 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.665 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.665 INFO:teuthology.orchestra.run.smithi096.stdout: "file", 2024-04-25T14:15:53.665 INFO:teuthology.orchestra.run.smithi096.stdout: "file,syslog", 2024-04-25T14:15:53.665 INFO:teuthology.orchestra.run.smithi096.stdout: "syslog" 2024-04-25T14:15:53.665 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.665 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T14:15:53.665 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.665 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.666 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.666 INFO:teuthology.orchestra.run.smithi096.stdout: "cgroups_split": { 2024-04-25T14:15:53.666 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cgroups_split", 2024-04-25T14:15:53.666 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.666 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.666 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.666 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:53.666 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.666 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.666 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.666 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T14:15:53.666 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.666 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.667 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.667 INFO:teuthology.orchestra.run.smithi096.stdout: "config_checks_enabled": { 2024-04-25T14:15:53.667 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "config_checks_enabled", 2024-04-25T14:15:53.667 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.667 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.667 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.667 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.667 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.667 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.667 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.667 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T14:15:53.667 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.667 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.668 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.668 INFO:teuthology.orchestra.run.smithi096.stdout: "config_dashboard": { 2024-04-25T14:15:53.668 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "config_dashboard", 2024-04-25T14:15:53.668 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.668 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.668 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.668 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:53.668 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.668 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.668 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.668 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T14:15:53.668 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.668 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.669 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.669 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_alertmanager": { 2024-04-25T14:15:53.669 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_alertmanager", 2024-04-25T14:15:53.669 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.669 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.669 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.669 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T14:15:53.669 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.669 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.669 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.669 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-04-25T14:15:53.669 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.669 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.669 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.670 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_base": { 2024-04-25T14:15:53.670 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_base", 2024-04-25T14:15:53.670 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.670 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.670 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.670 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T14:15:53.670 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.670 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.670 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.670 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Container image name, without the tag", 2024-04-25T14:15:53.670 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.670 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.671 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.671 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_elasticsearch": { 2024-04-25T14:15:53.671 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_elasticsearch", 2024-04-25T14:15:53.671 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.671 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.671 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.671 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T14:15:53.671 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.671 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.671 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.671 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "elasticsearch container image", 2024-04-25T14:15:53.672 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.672 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.672 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.672 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_grafana": { 2024-04-25T14:15:53.672 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_grafana", 2024-04-25T14:15:53.672 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.672 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.672 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.672 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T14:15:53.672 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.672 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.672 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.672 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-04-25T14:15:53.673 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.673 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.673 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.673 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_haproxy": { 2024-04-25T14:15:53.673 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_haproxy", 2024-04-25T14:15:53.673 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.673 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.673 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.673 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T14:15:53.673 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.673 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.673 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.674 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "HAproxy container image", 2024-04-25T14:15:53.674 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.674 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.674 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.674 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_jaeger_agent": { 2024-04-25T14:15:53.674 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_jaeger_agent", 2024-04-25T14:15:53.674 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.674 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.674 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.674 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T14:15:53.675 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.675 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.675 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.675 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Jaeger agent container image", 2024-04-25T14:15:53.675 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.675 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.675 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.675 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_jaeger_collector": { 2024-04-25T14:15:53.675 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_jaeger_collector", 2024-04-25T14:15:53.675 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.676 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.676 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.676 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T14:15:53.676 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.676 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.676 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.676 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Jaeger collector container image", 2024-04-25T14:15:53.676 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.676 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.676 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.676 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_jaeger_query": { 2024-04-25T14:15:53.677 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_jaeger_query", 2024-04-25T14:15:53.677 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.677 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.677 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.677 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T14:15:53.677 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.677 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.677 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.677 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Jaeger query container image", 2024-04-25T14:15:53.677 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.677 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.677 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.678 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_keepalived": { 2024-04-25T14:15:53.678 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_keepalived", 2024-04-25T14:15:53.678 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.678 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.678 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.678 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T14:15:53.678 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.678 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.678 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.678 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Keepalived container image", 2024-04-25T14:15:53.679 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.679 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.679 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.679 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_loki": { 2024-04-25T14:15:53.679 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_loki", 2024-04-25T14:15:53.679 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.679 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.679 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.679 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T14:15:53.679 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.679 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.680 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.680 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Loki container image", 2024-04-25T14:15:53.680 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.680 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.680 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.680 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_node_exporter": { 2024-04-25T14:15:53.680 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_node_exporter", 2024-04-25T14:15:53.680 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.680 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.681 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.681 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T14:15:53.681 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.681 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.681 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.681 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-04-25T14:15:53.681 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.681 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.681 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.681 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_nvmeof": { 2024-04-25T14:15:53.681 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_nvmeof", 2024-04-25T14:15:53.681 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.682 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.682 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.682 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T14:15:53.682 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.682 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.682 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.682 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Nvme-of container image", 2024-04-25T14:15:53.682 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.682 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.682 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.682 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_prometheus": { 2024-04-25T14:15:53.682 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_prometheus", 2024-04-25T14:15:53.682 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.683 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.683 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.683 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T14:15:53.683 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.683 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.683 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.683 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-04-25T14:15:53.683 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.683 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.683 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.683 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_promtail": { 2024-04-25T14:15:53.683 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_promtail", 2024-04-25T14:15:53.683 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.683 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.684 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.684 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T14:15:53.684 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.684 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.684 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.684 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Promtail container image", 2024-04-25T14:15:53.684 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.684 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.684 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.684 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_samba": { 2024-04-25T14:15:53.684 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_samba", 2024-04-25T14:15:53.684 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.684 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.685 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.685 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T14:15:53.685 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.685 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.685 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.685 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Samba/SMB container image", 2024-04-25T14:15:53.685 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.685 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.685 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.685 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_snmp_gateway": { 2024-04-25T14:15:53.685 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_snmp_gateway", 2024-04-25T14:15:53.685 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.686 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.686 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.686 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T14:15:53.686 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.686 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.686 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.686 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SNMP Gateway container image", 2024-04-25T14:15:53.686 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.686 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.686 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.686 INFO:teuthology.orchestra.run.smithi096.stdout: "container_init": { 2024-04-25T14:15:53.687 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_init", 2024-04-25T14:15:53.687 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.687 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.687 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.687 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:53.687 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.687 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.687 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.687 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T14:15:53.687 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.687 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.687 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.688 INFO:teuthology.orchestra.run.smithi096.stdout: "daemon_cache_timeout": { 2024-04-25T14:15:53.688 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "daemon_cache_timeout", 2024-04-25T14:15:53.688 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:53.688 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.688 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.688 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-04-25T14:15:53.688 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.688 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.688 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.688 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T14:15:53.688 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.688 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.688 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.689 INFO:teuthology.orchestra.run.smithi096.stdout: "default_cephadm_command_timeout": { 2024-04-25T14:15:53.689 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T14:15:53.689 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.689 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.689 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.689 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "900", 2024-04-25T14:15:53.689 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.689 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.689 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.689 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T14:15:53.689 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.689 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.690 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.690 INFO:teuthology.orchestra.run.smithi096.stdout: "default_registry": { 2024-04-25T14:15:53.690 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "default_registry", 2024-04-25T14:15:53.690 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.690 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.690 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.690 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io", 2024-04-25T14:15:53.690 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.690 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.690 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.690 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T14:15:53.690 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.690 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.691 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.691 INFO:teuthology.orchestra.run.smithi096.stdout: "device_cache_timeout": { 2024-04-25T14:15:53.691 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "device_cache_timeout", 2024-04-25T14:15:53.691 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:53.691 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.691 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.691 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "1800", 2024-04-25T14:15:53.691 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.691 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.691 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.691 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "seconds to cache device inventory", 2024-04-25T14:15:53.691 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.692 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.692 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.692 INFO:teuthology.orchestra.run.smithi096.stdout: "device_enhanced_scan": { 2024-04-25T14:15:53.692 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "device_enhanced_scan", 2024-04-25T14:15:53.692 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.692 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.692 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.692 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.692 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.692 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.693 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.693 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T14:15:53.693 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.693 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.693 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.693 INFO:teuthology.orchestra.run.smithi096.stdout: "facts_cache_timeout": { 2024-04-25T14:15:53.693 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "facts_cache_timeout", 2024-04-25T14:15:53.693 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:53.693 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.693 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.694 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-04-25T14:15:53.694 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.694 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.694 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.694 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "seconds to cache host facts data", 2024-04-25T14:15:53.694 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.694 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.694 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.694 INFO:teuthology.orchestra.run.smithi096.stdout: "grafana_dashboards_path": { 2024-04-25T14:15:53.694 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "grafana_dashboards_path", 2024-04-25T14:15:53.695 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.695 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.695 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.695 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T14:15:53.695 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.695 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.695 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.695 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T14:15:53.695 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.695 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.695 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.696 INFO:teuthology.orchestra.run.smithi096.stdout: "host_check_interval": { 2024-04-25T14:15:53.696 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "host_check_interval", 2024-04-25T14:15:53.696 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:53.696 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.696 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.696 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-04-25T14:15:53.696 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.696 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.696 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.696 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to perform a host check", 2024-04-25T14:15:53.696 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.696 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.697 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.697 INFO:teuthology.orchestra.run.smithi096.stdout: "hw_monitoring": { 2024-04-25T14:15:53.697 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "hw_monitoring", 2024-04-25T14:15:53.697 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.697 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.697 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.697 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.697 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.697 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.697 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.697 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T14:15:53.697 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.698 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.698 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.698 INFO:teuthology.orchestra.run.smithi096.stdout: "inventory_list_all": { 2024-04-25T14:15:53.698 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "inventory_list_all", 2024-04-25T14:15:53.698 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.698 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.698 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.698 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.698 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.698 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.698 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.698 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T14:15:53.699 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.699 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.699 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.699 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:53.699 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:53.699 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.699 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.699 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.699 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.699 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.700 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.700 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.700 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.700 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.700 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.700 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.700 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.700 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.700 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.700 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.700 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.701 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.701 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.701 INFO:teuthology.orchestra.run.smithi096.stdout: "log_refresh_metadata": { 2024-04-25T14:15:53.701 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_refresh_metadata", 2024-04-25T14:15:53.701 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.701 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.701 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.701 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.701 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.701 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.701 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.702 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T14:15:53.702 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.702 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.702 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.702 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:53.702 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:53.702 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.702 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.702 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.702 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:53.703 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.703 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.703 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.703 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T14:15:53.703 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.703 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.703 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.703 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:53.703 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:53.703 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.703 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.704 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.704 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:53.704 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.704 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.704 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.704 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.704 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.704 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.704 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.705 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.705 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.705 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.705 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.705 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.705 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.705 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.705 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:53.705 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:53.705 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.705 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.706 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.706 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.706 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.706 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.706 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.706 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.706 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.706 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.706 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.706 INFO:teuthology.orchestra.run.smithi096.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T14:15:53.707 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T14:15:53.707 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.707 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.707 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.707 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.707 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.707 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.707 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.707 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T14:15:53.707 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.707 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.707 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.708 INFO:teuthology.orchestra.run.smithi096.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T14:15:53.708 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T14:15:53.708 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.708 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.708 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.708 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "*", 2024-04-25T14:15:53.708 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.708 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.708 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.708 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T14:15:53.708 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.708 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.708 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.709 INFO:teuthology.orchestra.run.smithi096.stdout: "max_count_per_host": { 2024-04-25T14:15:53.709 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_count_per_host", 2024-04-25T14:15:53.709 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.709 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.709 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.709 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T14:15:53.709 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.709 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.709 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.709 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "max number of daemons per service per host", 2024-04-25T14:15:53.709 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.709 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.709 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.710 INFO:teuthology.orchestra.run.smithi096.stdout: "max_osd_draining_count": { 2024-04-25T14:15:53.710 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_osd_draining_count", 2024-04-25T14:15:53.710 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.710 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.710 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.710 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T14:15:53.710 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.710 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.710 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.710 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T14:15:53.710 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.710 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.710 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.711 INFO:teuthology.orchestra.run.smithi096.stdout: "migration_current": { 2024-04-25T14:15:53.711 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "migration_current", 2024-04-25T14:15:53.711 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.711 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.711 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.711 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.711 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.711 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.711 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.711 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "internal - do not modify", 2024-04-25T14:15:53.711 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.711 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.711 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.712 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": { 2024-04-25T14:15:53.712 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mode", 2024-04-25T14:15:53.712 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.712 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.712 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.712 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "root", 2024-04-25T14:15:53.712 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.712 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.712 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.712 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm-package", 2024-04-25T14:15:53.712 INFO:teuthology.orchestra.run.smithi096.stdout: "root" 2024-04-25T14:15:53.712 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.712 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T14:15:53.712 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.713 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.713 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.713 INFO:teuthology.orchestra.run.smithi096.stdout: "oob_default_addr": { 2024-04-25T14:15:53.713 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "oob_default_addr", 2024-04-25T14:15:53.713 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.713 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.713 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.713 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "169.254.1.1", 2024-04-25T14:15:53.713 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.713 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.713 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.713 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T14:15:53.714 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.714 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.714 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.714 INFO:teuthology.orchestra.run.smithi096.stdout: "prometheus_alerts_path": { 2024-04-25T14:15:53.714 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "prometheus_alerts_path", 2024-04-25T14:15:53.714 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.714 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.714 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.714 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T14:15:53.714 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.714 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.714 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.714 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T14:15:53.715 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.715 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.715 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.715 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_insecure": { 2024-04-25T14:15:53.715 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_insecure", 2024-04-25T14:15:53.715 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.715 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.715 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.715 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.715 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.715 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.715 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.716 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T14:15:53.716 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.716 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.716 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.716 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_password": { 2024-04-25T14:15:53.716 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_password", 2024-04-25T14:15:53.716 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.716 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.716 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.716 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.716 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.716 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.717 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.717 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T14:15:53.717 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.717 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.717 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.717 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_url": { 2024-04-25T14:15:53.717 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_url", 2024-04-25T14:15:53.717 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.717 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.717 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.717 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.717 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.718 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.718 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.718 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T14:15:53.718 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.718 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.718 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.718 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_username": { 2024-04-25T14:15:53.718 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_username", 2024-04-25T14:15:53.718 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.718 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.718 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.718 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.718 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.719 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.719 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.719 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T14:15:53.719 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.719 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.719 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.719 INFO:teuthology.orchestra.run.smithi096.stdout: "secure_monitoring_stack": { 2024-04-25T14:15:53.719 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "secure_monitoring_stack", 2024-04-25T14:15:53.719 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.719 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.719 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.719 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.720 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.720 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.720 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.720 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T14:15:53.720 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.720 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.720 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.720 INFO:teuthology.orchestra.run.smithi096.stdout: "service_discovery_port": { 2024-04-25T14:15:53.720 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "service_discovery_port", 2024-04-25T14:15:53.720 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.720 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.720 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.720 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8765", 2024-04-25T14:15:53.720 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.720 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.721 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.721 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "cephadm service discovery port", 2024-04-25T14:15:53.721 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.721 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.721 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.721 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:53.721 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:53.721 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.721 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:53.721 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.721 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:53.721 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.721 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.721 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.722 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.722 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.722 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.722 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.722 INFO:teuthology.orchestra.run.smithi096.stdout: "ssh_config_file": { 2024-04-25T14:15:53.722 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssh_config_file", 2024-04-25T14:15:53.722 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.722 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.722 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.722 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.722 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.722 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.722 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.723 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T14:15:53.723 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.723 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.723 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.723 INFO:teuthology.orchestra.run.smithi096.stdout: "use_agent": { 2024-04-25T14:15:53.723 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "use_agent", 2024-04-25T14:15:53.723 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.723 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.723 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.723 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.723 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.723 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.723 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.723 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T14:15:53.724 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.724 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.724 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.724 INFO:teuthology.orchestra.run.smithi096.stdout: "use_repo_digest": { 2024-04-25T14:15:53.724 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "use_repo_digest", 2024-04-25T14:15:53.724 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.724 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.724 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.724 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:53.724 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.724 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.724 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.724 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T14:15:53.724 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.725 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.725 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.725 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_on_failed_host_check": { 2024-04-25T14:15:53.725 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_on_failed_host_check", 2024-04-25T14:15:53.725 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.725 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.725 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.725 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:53.725 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.725 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.725 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.725 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T14:15:53.726 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.726 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.726 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.726 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_on_stray_daemons": { 2024-04-25T14:15:53.726 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_on_stray_daemons", 2024-04-25T14:15:53.726 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.726 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.726 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.726 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:53.726 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.726 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.726 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.727 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T14:15:53.727 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.727 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.727 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.727 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_on_stray_hosts": { 2024-04-25T14:15:53.727 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_on_stray_hosts", 2024-04-25T14:15:53.727 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.727 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.727 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.727 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:53.727 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.727 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.727 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.727 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T14:15:53.728 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.728 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.728 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.728 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.728 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:53.728 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crash", 2024-04-25T14:15:53.728 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:53.728 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:53.728 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:53.728 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:53.728 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:53.728 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.728 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.729 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.729 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.729 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.729 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.729 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.729 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.729 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.729 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.729 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.729 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.729 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.729 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.729 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.729 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.729 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.730 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.730 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:53.730 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:53.730 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.730 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.730 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.730 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.730 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.730 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.730 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.730 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.730 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.730 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.730 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.731 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:53.731 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:53.731 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.731 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.731 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.731 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:53.731 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.731 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.731 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.731 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.731 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.731 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.732 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.732 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.732 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.732 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.732 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.732 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.732 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.732 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.732 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:53.732 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:53.732 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.732 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.732 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.733 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.733 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.733 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.733 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.733 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.733 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.733 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.733 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.733 INFO:teuthology.orchestra.run.smithi096.stdout: "retain_interval": { 2024-04-25T14:15:53.733 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "retain_interval", 2024-04-25T14:15:53.733 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:53.734 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.734 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.734 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "31536000", 2024-04-25T14:15:53.734 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.734 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.734 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.734 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T14:15:53.734 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.734 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.734 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.734 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:53.734 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:53.735 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.735 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:53.735 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.735 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:53.735 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.735 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.735 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.735 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.735 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.735 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.735 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.735 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_recent_interval": { 2024-04-25T14:15:53.735 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_recent_interval", 2024-04-25T14:15:53.735 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:53.736 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.736 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.736 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "1209600", 2024-04-25T14:15:53.736 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.736 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.736 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.736 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T14:15:53.736 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.736 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.736 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.736 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.736 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:53.737 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "dashboard", 2024-04-25T14:15:53.737 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:53.737 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:53.737 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:53.737 INFO:teuthology.orchestra.run.smithi096.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T14:15:53.737 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T14:15:53.737 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.737 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.737 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.737 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T14:15:53.737 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.737 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.737 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.737 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.738 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.738 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.738 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.738 INFO:teuthology.orchestra.run.smithi096.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T14:15:53.738 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T14:15:53.738 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.738 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.738 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.738 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.738 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.738 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.738 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.738 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.739 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.739 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.739 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.739 INFO:teuthology.orchestra.run.smithi096.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T14:15:53.739 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T14:15:53.739 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.739 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.739 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.739 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:53.739 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.739 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.739 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.740 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.740 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.740 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.740 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.740 INFO:teuthology.orchestra.run.smithi096.stdout: "AUDIT_API_ENABLED": { 2024-04-25T14:15:53.740 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T14:15:53.740 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.740 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.740 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.740 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.740 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.740 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.741 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.741 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.741 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.741 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.741 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.741 INFO:teuthology.orchestra.run.smithi096.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T14:15:53.741 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T14:15:53.741 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.741 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.741 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.741 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:53.741 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.741 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.742 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.742 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.742 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.742 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.742 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.742 INFO:teuthology.orchestra.run.smithi096.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T14:15:53.742 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T14:15:53.742 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.742 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.742 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.742 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:53.742 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.742 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.743 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.743 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.743 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.743 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.743 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.743 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T14:15:53.743 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T14:15:53.743 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.743 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.743 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.743 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:53.743 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.744 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.744 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.744 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.744 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.744 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.744 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.744 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T14:15:53.744 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T14:15:53.744 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.744 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.744 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.744 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:53.745 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.745 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.745 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.745 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.745 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.745 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.745 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.745 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T14:15:53.745 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T14:15:53.745 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.745 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.745 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.746 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:53.746 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.746 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.746 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.746 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.746 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.746 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.746 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.746 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T14:15:53.746 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T14:15:53.746 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.746 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.746 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.747 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:53.747 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.747 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.747 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.747 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.747 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.747 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.747 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.747 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T14:15:53.747 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T14:15:53.747 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.747 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.747 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.748 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:53.748 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.748 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.748 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.748 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.748 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.748 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.748 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.748 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T14:15:53.748 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T14:15:53.748 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.748 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.748 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.749 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:53.749 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.749 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.749 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.749 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.749 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.749 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.749 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.749 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T14:15:53.749 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T14:15:53.749 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.749 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.750 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.750 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:53.750 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.750 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.750 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.750 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.750 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.750 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.750 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.750 INFO:teuthology.orchestra.run.smithi096.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T14:15:53.750 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T14:15:53.750 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.750 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.751 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.751 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.751 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.751 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.751 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.751 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.751 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.751 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.751 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.751 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T14:15:53.751 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T14:15:53.751 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.752 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.752 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.752 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "admin", 2024-04-25T14:15:53.752 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.752 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.752 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.752 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.752 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.752 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.752 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.752 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T14:15:53.752 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T14:15:53.753 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.753 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.753 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.753 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:53.753 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.753 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.753 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.753 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.753 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.753 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.753 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.753 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_URL": { 2024-04-25T14:15:53.754 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_URL", 2024-04-25T14:15:53.754 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.754 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.754 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.754 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.754 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.754 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.754 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.754 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.754 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.754 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.754 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.755 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T14:15:53.755 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T14:15:53.755 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.755 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.755 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.755 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "admin", 2024-04-25T14:15:53.755 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.755 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.755 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.755 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.755 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.755 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.755 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.756 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T14:15:53.756 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T14:15:53.756 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.756 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.756 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.756 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.756 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.756 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.756 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.756 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.756 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.756 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.757 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.757 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T14:15:53.757 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T14:15:53.757 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.757 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.757 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.757 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.757 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.757 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.757 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.757 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.757 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.757 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.758 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.758 INFO:teuthology.orchestra.run.smithi096.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T14:15:53.758 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T14:15:53.758 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.758 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.758 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.758 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:53.758 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.758 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.758 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.758 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.758 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.759 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.759 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.759 INFO:teuthology.orchestra.run.smithi096.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T14:15:53.759 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T14:15:53.759 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.759 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.759 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.759 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.759 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.759 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.759 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.759 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.760 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.760 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.760 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.760 INFO:teuthology.orchestra.run.smithi096.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T14:15:53.760 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T14:15:53.760 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.760 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.760 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.760 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "[]", 2024-04-25T14:15:53.760 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.760 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.760 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.760 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.761 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.761 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.761 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.761 INFO:teuthology.orchestra.run.smithi096.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T14:15:53.761 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T14:15:53.761 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.761 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.761 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.761 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "{}", 2024-04-25T14:15:53.761 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.761 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.762 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.762 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.762 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.762 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.762 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.762 INFO:teuthology.orchestra.run.smithi096.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T14:15:53.762 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T14:15:53.762 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.762 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.762 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.762 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.763 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.763 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.763 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.763 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.763 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.763 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.763 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.763 INFO:teuthology.orchestra.run.smithi096.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T14:15:53.763 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T14:15:53.763 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.763 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.763 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.764 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:53.764 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.764 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.764 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.764 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.764 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.764 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.764 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.764 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T14:15:53.764 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T14:15:53.764 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.764 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.764 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.765 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.765 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.765 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.765 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.765 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.765 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.765 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.765 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.765 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T14:15:53.765 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T14:15:53.765 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.765 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.765 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.766 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.766 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.766 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.766 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.766 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.766 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.766 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.766 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.766 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T14:15:53.766 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T14:15:53.766 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.767 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.767 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.767 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:53.767 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.767 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.767 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.767 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.767 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.767 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.767 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.767 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T14:15:53.767 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T14:15:53.768 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.768 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.768 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.768 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:53.768 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.768 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.768 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.768 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.768 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.768 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.768 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.769 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T14:15:53.769 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T14:15:53.769 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.769 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.769 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.769 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.769 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.769 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.769 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.769 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.769 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.769 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.769 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.770 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T14:15:53.770 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T14:15:53.770 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.770 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.770 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.770 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.770 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.770 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.770 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.770 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.770 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.770 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.770 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.771 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T14:15:53.771 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T14:15:53.771 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.771 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.771 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.771 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.771 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.771 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.771 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.771 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.771 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.771 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.772 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.772 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T14:15:53.772 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T14:15:53.772 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.772 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.772 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.772 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:53.772 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.772 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.772 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.772 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.772 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.773 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.773 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.773 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T14:15:53.773 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T14:15:53.773 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.773 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.773 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.773 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T14:15:53.773 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.773 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.773 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.773 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.774 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.774 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.774 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.774 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T14:15:53.774 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T14:15:53.774 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.774 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.774 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.774 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T14:15:53.774 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.774 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.774 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.774 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.775 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.775 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.775 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.775 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T14:15:53.775 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T14:15:53.775 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.775 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.775 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.775 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8", 2024-04-25T14:15:53.775 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.775 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.775 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.775 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.776 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.776 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.776 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.776 INFO:teuthology.orchestra.run.smithi096.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T14:15:53.776 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T14:15:53.776 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.776 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.776 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.776 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "45", 2024-04-25T14:15:53.776 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.776 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.776 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.776 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.776 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.777 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.777 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.777 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T14:15:53.777 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T14:15:53.777 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.777 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.777 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.777 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.777 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.777 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.777 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.777 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.777 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.777 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.778 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.778 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T14:15:53.778 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T14:15:53.778 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.778 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.778 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.778 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "admin", 2024-04-25T14:15:53.778 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.778 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.778 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.778 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.778 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.779 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.779 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.779 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T14:15:53.779 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T14:15:53.779 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.779 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.779 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.779 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.779 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.779 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.779 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.779 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.779 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.779 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.779 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.780 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T14:15:53.780 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T14:15:53.780 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.780 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.780 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.780 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:53.780 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.780 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.780 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.780 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.780 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.780 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.780 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.781 INFO:teuthology.orchestra.run.smithi096.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T14:15:53.781 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T14:15:53.781 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.781 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.781 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.781 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.781 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.781 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.781 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.781 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.781 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.781 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.781 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.782 INFO:teuthology.orchestra.run.smithi096.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T14:15:53.782 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T14:15:53.782 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.782 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.782 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.782 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:53.782 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.782 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.782 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.782 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.782 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.782 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.782 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.783 INFO:teuthology.orchestra.run.smithi096.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T14:15:53.783 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T14:15:53.783 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.783 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.783 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.783 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T14:15:53.783 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.783 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.783 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.783 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.783 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.783 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.783 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.783 INFO:teuthology.orchestra.run.smithi096.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T14:15:53.783 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T14:15:53.784 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.784 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.784 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.784 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-04-25T14:15:53.784 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.784 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.784 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.784 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.784 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.784 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.784 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.784 INFO:teuthology.orchestra.run.smithi096.stdout: "cross_origin_url": { 2024-04-25T14:15:53.784 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cross_origin_url", 2024-04-25T14:15:53.784 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.785 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.785 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.785 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.785 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.785 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.785 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.785 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.785 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.785 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.785 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.785 INFO:teuthology.orchestra.run.smithi096.stdout: "crt_file": { 2024-04-25T14:15:53.786 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crt_file", 2024-04-25T14:15:53.786 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.786 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.786 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.786 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.786 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.786 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.786 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.786 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.786 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.786 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.786 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.786 INFO:teuthology.orchestra.run.smithi096.stdout: "debug": { 2024-04-25T14:15:53.787 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "debug", 2024-04-25T14:15:53.787 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.787 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.787 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.787 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.787 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.787 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.787 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.787 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Enable/disable debug options", 2024-04-25T14:15:53.787 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.787 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.787 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.787 INFO:teuthology.orchestra.run.smithi096.stdout: "jwt_token_ttl": { 2024-04-25T14:15:53.787 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "jwt_token_ttl", 2024-04-25T14:15:53.788 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.788 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.788 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.788 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "28800", 2024-04-25T14:15:53.788 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.788 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.788 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.788 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.788 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.788 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.788 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.788 INFO:teuthology.orchestra.run.smithi096.stdout: "key_file": { 2024-04-25T14:15:53.789 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "key_file", 2024-04-25T14:15:53.789 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.789 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.789 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.789 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.789 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.789 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.789 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.789 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.789 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.789 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.789 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.789 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:53.790 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:53.790 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.790 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.790 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.790 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.790 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.790 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.790 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.790 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.790 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.790 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.790 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.790 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.791 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.791 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.791 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.791 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.791 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.791 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.791 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:53.791 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:53.791 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.791 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.791 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.791 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.791 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.792 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.792 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.792 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.792 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.792 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.792 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.792 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:53.792 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:53.792 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.792 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.792 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.792 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:53.793 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.793 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.793 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.793 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.793 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.793 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.793 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.793 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.793 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.793 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.793 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.793 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.793 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.794 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.794 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:53.794 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:53.794 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.794 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.794 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.794 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.794 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.794 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.794 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.794 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.794 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.794 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.795 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.795 INFO:teuthology.orchestra.run.smithi096.stdout: "motd": { 2024-04-25T14:15:53.795 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "motd", 2024-04-25T14:15:53.795 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.795 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.795 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.795 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.795 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.795 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.795 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.795 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "The message of the day", 2024-04-25T14:15:53.795 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.795 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.795 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.796 INFO:teuthology.orchestra.run.smithi096.stdout: "redirect_resolve_ip_addr": { 2024-04-25T14:15:53.796 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T14:15:53.796 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.796 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.796 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.796 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.796 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.796 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.796 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.796 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.796 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.796 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.796 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.797 INFO:teuthology.orchestra.run.smithi096.stdout: "server_addr": { 2024-04-25T14:15:53.797 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_addr", 2024-04-25T14:15:53.797 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.797 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.797 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.797 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "::", 2024-04-25T14:15:53.797 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.797 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.797 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.797 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.797 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.797 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.797 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.797 INFO:teuthology.orchestra.run.smithi096.stdout: "server_port": { 2024-04-25T14:15:53.798 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_port", 2024-04-25T14:15:53.798 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.798 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.798 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.798 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8080", 2024-04-25T14:15:53.798 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.798 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.798 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.798 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.798 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.798 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.798 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.798 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:53.799 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:53.799 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.799 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:53.799 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.799 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:53.799 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.799 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.799 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.799 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.799 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.799 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.799 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.799 INFO:teuthology.orchestra.run.smithi096.stdout: "ssl": { 2024-04-25T14:15:53.800 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssl", 2024-04-25T14:15:53.800 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.800 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.800 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.800 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:53.800 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.800 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.800 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.800 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.800 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.800 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.800 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.801 INFO:teuthology.orchestra.run.smithi096.stdout: "ssl_server_port": { 2024-04-25T14:15:53.801 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssl_server_port", 2024-04-25T14:15:53.801 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.801 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.801 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.801 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8443", 2024-04-25T14:15:53.801 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.801 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.801 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.801 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.801 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.801 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.802 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.802 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_behaviour": { 2024-04-25T14:15:53.802 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_behaviour", 2024-04-25T14:15:53.802 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.802 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.802 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.802 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "redirect", 2024-04-25T14:15:53.802 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.802 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.802 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.803 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.803 INFO:teuthology.orchestra.run.smithi096.stdout: "redirect" 2024-04-25T14:15:53.803 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.803 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.803 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.803 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.803 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.803 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_error_status_code": { 2024-04-25T14:15:53.803 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_error_status_code", 2024-04-25T14:15:53.803 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.804 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.804 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.804 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "500", 2024-04-25T14:15:53.804 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "400", 2024-04-25T14:15:53.804 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "599", 2024-04-25T14:15:53.804 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.804 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.804 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.804 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.804 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.804 INFO:teuthology.orchestra.run.smithi096.stdout: "url_prefix": { 2024-04-25T14:15:53.804 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "url_prefix", 2024-04-25T14:15:53.804 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.805 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.805 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.805 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.805 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.805 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.805 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.805 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.805 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.805 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.805 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.805 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.805 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:53.806 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "devicehealth", 2024-04-25T14:15:53.806 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:53.806 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:53.806 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:53.806 INFO:teuthology.orchestra.run.smithi096.stdout: "enable_monitoring": { 2024-04-25T14:15:53.806 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enable_monitoring", 2024-04-25T14:15:53.806 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.806 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.807 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.807 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:53.807 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.807 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.807 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.807 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "monitor device health metrics", 2024-04-25T14:15:53.807 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.807 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.807 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.807 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:53.808 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:53.808 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.808 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.808 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.808 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.808 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.808 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.808 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.808 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.808 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.808 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.808 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.808 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.809 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.809 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.809 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.809 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.809 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.809 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.809 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:53.809 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:53.809 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.809 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.809 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.809 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.810 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.810 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.810 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.810 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.810 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.810 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.810 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.810 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:53.810 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:53.810 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.810 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.811 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.811 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:53.811 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.811 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.811 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.811 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.811 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.811 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.811 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.811 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.811 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.812 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.812 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.812 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.812 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.812 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.812 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:53.812 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:53.812 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.812 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.812 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.812 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.812 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.813 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.813 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.813 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.813 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.813 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.813 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.813 INFO:teuthology.orchestra.run.smithi096.stdout: "mark_out_threshold": { 2024-04-25T14:15:53.813 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mark_out_threshold", 2024-04-25T14:15:53.813 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:53.813 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.813 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.813 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "2419200", 2024-04-25T14:15:53.813 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.814 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.814 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.814 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T14:15:53.814 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.814 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.814 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.814 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_name": { 2024-04-25T14:15:53.814 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pool_name", 2024-04-25T14:15:53.814 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.814 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.814 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.815 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "device_health_metrics", 2024-04-25T14:15:53.815 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.815 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.815 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.815 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T14:15:53.815 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.815 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.815 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.815 INFO:teuthology.orchestra.run.smithi096.stdout: "retention_period": { 2024-04-25T14:15:53.815 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "retention_period", 2024-04-25T14:15:53.815 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:53.815 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.816 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.816 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "15552000", 2024-04-25T14:15:53.816 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.816 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.816 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.816 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how long to retain device health metrics", 2024-04-25T14:15:53.816 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.816 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.816 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.816 INFO:teuthology.orchestra.run.smithi096.stdout: "scrape_frequency": { 2024-04-25T14:15:53.816 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "scrape_frequency", 2024-04-25T14:15:53.816 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:53.816 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.816 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.817 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "86400", 2024-04-25T14:15:53.817 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.817 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.817 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.817 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T14:15:53.817 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.817 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.817 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.817 INFO:teuthology.orchestra.run.smithi096.stdout: "self_heal": { 2024-04-25T14:15:53.817 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "self_heal", 2024-04-25T14:15:53.817 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.817 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.817 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.817 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:53.817 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.818 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.818 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.818 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T14:15:53.818 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.818 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.818 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.818 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-04-25T14:15:53.818 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-04-25T14:15:53.818 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:53.818 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.818 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.818 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-04-25T14:15:53.819 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.819 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.819 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.819 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T14:15:53.819 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.819 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.819 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.819 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:53.819 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:53.819 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.819 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:53.819 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.819 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:53.820 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.820 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.820 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.820 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.820 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.820 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.820 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.820 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_threshold": { 2024-04-25T14:15:53.820 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_threshold", 2024-04-25T14:15:53.820 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:53.820 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.820 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.820 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "7257600", 2024-04-25T14:15:53.821 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.821 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.821 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.821 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T14:15:53.821 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.821 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.821 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.821 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.821 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:53.821 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "diskprediction_local", 2024-04-25T14:15:53.821 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:53.821 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:53.821 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:53.822 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:53.822 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:53.822 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.822 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.822 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.822 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.822 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.822 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.822 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.822 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.822 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.822 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.823 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.823 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.823 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.823 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.823 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.823 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.823 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.823 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.823 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:53.823 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:53.823 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.823 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.824 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.824 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.824 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.824 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.824 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.824 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.824 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.824 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.824 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.824 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:53.824 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:53.824 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.824 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.825 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.825 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:53.825 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.825 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.825 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.825 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.825 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.825 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.825 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.825 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.825 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.825 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.825 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.826 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.826 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.826 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.826 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:53.826 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:53.826 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.826 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.826 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.826 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.826 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.827 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.827 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.827 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.827 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.827 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.827 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.827 INFO:teuthology.orchestra.run.smithi096.stdout: "predict_interval": { 2024-04-25T14:15:53.827 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "predict_interval", 2024-04-25T14:15:53.827 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.827 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.827 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.827 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "86400", 2024-04-25T14:15:53.827 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.827 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.828 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.828 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.828 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.828 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.828 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.828 INFO:teuthology.orchestra.run.smithi096.stdout: "predictor_model": { 2024-04-25T14:15:53.828 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "predictor_model", 2024-04-25T14:15:53.828 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.828 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.828 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.828 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "prophetstor", 2024-04-25T14:15:53.828 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.829 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.829 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.829 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.829 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.829 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.829 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.829 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-04-25T14:15:53.829 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-04-25T14:15:53.829 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.829 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.829 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.829 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-04-25T14:15:53.829 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.830 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.830 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.830 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.830 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.830 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.830 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.830 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:53.830 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:53.830 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.830 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:53.830 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.830 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:53.831 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.831 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.831 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.831 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.831 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.831 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.831 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.831 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.831 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:53.831 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "influx", 2024-04-25T14:15:53.831 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": false, 2024-04-25T14:15:53.831 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "influxdb python module not found", 2024-04-25T14:15:53.831 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:53.832 INFO:teuthology.orchestra.run.smithi096.stdout: "batch_size": { 2024-04-25T14:15:53.832 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "batch_size", 2024-04-25T14:15:53.832 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.832 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.832 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.832 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5000", 2024-04-25T14:15:53.832 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.832 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.832 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.832 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T14:15:53.832 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.832 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.832 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.833 INFO:teuthology.orchestra.run.smithi096.stdout: "database": { 2024-04-25T14:15:53.833 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "database", 2024-04-25T14:15:53.833 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.833 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.833 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.833 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "ceph", 2024-04-25T14:15:53.833 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.833 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.833 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.833 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T14:15:53.833 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.833 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.833 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.834 INFO:teuthology.orchestra.run.smithi096.stdout: "hostname": { 2024-04-25T14:15:53.834 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "hostname", 2024-04-25T14:15:53.834 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.834 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.834 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.834 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.834 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.834 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.834 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.834 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "InfluxDB server hostname", 2024-04-25T14:15:53.834 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.834 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.834 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.835 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-04-25T14:15:53.835 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-04-25T14:15:53.835 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:53.835 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.835 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.835 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "30", 2024-04-25T14:15:53.835 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "5", 2024-04-25T14:15:53.835 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.835 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.835 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T14:15:53.835 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.835 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.835 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.836 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:53.836 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:53.836 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.836 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.836 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.836 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.836 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.836 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.836 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.836 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.836 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.836 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.837 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.837 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.837 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.837 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.837 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.837 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.837 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.837 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.837 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:53.837 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:53.837 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.837 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.838 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.838 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.838 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.838 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.838 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.838 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.838 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.838 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.838 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.838 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:53.838 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:53.839 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.839 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.839 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.839 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:53.839 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.839 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.839 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.839 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.839 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.839 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.839 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.839 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.839 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.839 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.840 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.840 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.840 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.840 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.840 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:53.840 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:53.840 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.840 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.840 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.840 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.840 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.840 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.840 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.841 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.841 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.841 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.841 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.841 INFO:teuthology.orchestra.run.smithi096.stdout: "password": { 2024-04-25T14:15:53.841 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "password", 2024-04-25T14:15:53.841 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.841 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.841 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.841 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.841 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.841 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.842 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.842 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "password of InfluxDB server user", 2024-04-25T14:15:53.842 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.842 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.842 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.842 INFO:teuthology.orchestra.run.smithi096.stdout: "port": { 2024-04-25T14:15:53.842 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "port", 2024-04-25T14:15:53.842 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.842 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.842 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.842 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8086", 2024-04-25T14:15:53.842 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.842 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.843 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.843 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "InfluxDB server port", 2024-04-25T14:15:53.843 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.843 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.843 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.843 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:53.843 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:53.843 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.843 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:53.843 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.843 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:53.843 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.843 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.844 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.844 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.844 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.844 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.844 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.844 INFO:teuthology.orchestra.run.smithi096.stdout: "ssl": { 2024-04-25T14:15:53.844 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssl", 2024-04-25T14:15:53.844 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.844 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.844 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.844 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "false", 2024-04-25T14:15:53.844 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.844 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.844 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.845 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T14:15:53.845 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.845 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.845 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.845 INFO:teuthology.orchestra.run.smithi096.stdout: "threads": { 2024-04-25T14:15:53.845 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "threads", 2024-04-25T14:15:53.845 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.845 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.845 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.845 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-04-25T14:15:53.845 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-04-25T14:15:53.845 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "32", 2024-04-25T14:15:53.845 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.846 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T14:15:53.846 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.846 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.846 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.846 INFO:teuthology.orchestra.run.smithi096.stdout: "username": { 2024-04-25T14:15:53.846 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "username", 2024-04-25T14:15:53.846 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.846 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.846 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.846 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.846 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.846 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.846 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.846 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "username of InfluxDB server user", 2024-04-25T14:15:53.847 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.847 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.847 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.847 INFO:teuthology.orchestra.run.smithi096.stdout: "verify_ssl": { 2024-04-25T14:15:53.847 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "verify_ssl", 2024-04-25T14:15:53.847 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.847 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.847 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.847 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "true", 2024-04-25T14:15:53.847 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.847 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.847 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.847 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T14:15:53.848 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.848 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.848 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.848 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.848 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:53.848 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "insights", 2024-04-25T14:15:53.848 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:53.848 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:53.848 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:53.848 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:53.848 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:53.848 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.848 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.849 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.849 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.849 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.849 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.849 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.849 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.849 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.849 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.849 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.849 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.849 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.849 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.849 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.850 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.850 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.850 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.850 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:53.850 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:53.850 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.850 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.850 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.850 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.850 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.850 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.850 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.850 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.851 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.851 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.851 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.851 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:53.851 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:53.851 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.851 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.851 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.851 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:53.851 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.851 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.851 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.851 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.852 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.852 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.852 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.852 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.852 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.852 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.852 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.852 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.852 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.852 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.852 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:53.852 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:53.852 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.852 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.853 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.853 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.853 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.853 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.853 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.853 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.853 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.853 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.853 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.853 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:53.853 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:53.853 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.854 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:53.854 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.854 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:53.854 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.854 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.854 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.854 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.854 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.854 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.854 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.854 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.854 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:53.855 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "iostat", 2024-04-25T14:15:53.855 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:53.855 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:53.855 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:53.855 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:53.855 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:53.855 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.855 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.855 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.855 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.855 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.855 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.855 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.856 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.856 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.856 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.856 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.856 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.856 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.856 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.856 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.856 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.856 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.856 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.856 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:53.856 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:53.856 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.857 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.857 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.857 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.857 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.857 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.857 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.857 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.857 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.857 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.857 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.857 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:53.857 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:53.857 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.858 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.858 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.858 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:53.858 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.858 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.858 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.858 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.858 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.858 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.858 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.858 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.858 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.858 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.859 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.859 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.859 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.859 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.859 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:53.859 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:53.859 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.859 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.859 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.859 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.859 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.859 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.859 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.860 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.860 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.860 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.860 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.860 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:53.860 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:53.860 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.860 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:53.860 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.860 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:53.860 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.861 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.861 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.861 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.861 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.861 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.861 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.861 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.861 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:53.861 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "k8sevents", 2024-04-25T14:15:53.861 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:53.861 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:53.862 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:53.862 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_event_retention_days": { 2024-04-25T14:15:53.862 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ceph_event_retention_days", 2024-04-25T14:15:53.862 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.862 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.862 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.862 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "7", 2024-04-25T14:15:53.862 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.862 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.862 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.862 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T14:15:53.862 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.862 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.863 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.863 INFO:teuthology.orchestra.run.smithi096.stdout: "config_check_secs": { 2024-04-25T14:15:53.863 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "config_check_secs", 2024-04-25T14:15:53.863 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.863 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.863 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.863 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T14:15:53.863 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "10", 2024-04-25T14:15:53.863 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.863 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.863 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T14:15:53.863 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.864 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.864 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.864 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:53.864 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:53.864 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.864 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.864 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.864 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.864 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.864 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.864 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.864 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.865 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.865 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.865 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.865 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.865 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.865 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.865 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.865 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.865 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.865 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.865 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:53.865 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:53.865 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.866 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.866 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.866 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.866 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.866 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.866 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.866 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.866 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.866 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.866 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.866 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:53.866 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:53.867 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.867 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.867 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.867 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:53.867 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.867 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.867 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.867 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.867 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.867 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.867 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.867 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.867 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.868 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.868 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.868 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.868 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.868 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.868 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:53.868 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:53.868 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.868 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.868 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.868 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.868 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.868 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.869 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.869 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.869 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.869 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.869 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.869 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:53.869 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:53.869 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.869 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:53.869 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.869 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:53.869 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.869 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.870 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.870 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.870 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.870 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.870 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.870 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.870 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:53.870 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "localpool", 2024-04-25T14:15:53.870 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:53.870 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:53.870 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:53.871 INFO:teuthology.orchestra.run.smithi096.stdout: "failure_domain": { 2024-04-25T14:15:53.871 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "failure_domain", 2024-04-25T14:15:53.871 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.871 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.871 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.871 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "host", 2024-04-25T14:15:53.871 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.871 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.871 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.871 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "failure domain for any created local pool", 2024-04-25T14:15:53.871 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T14:15:53.871 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.871 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.872 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:53.872 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:53.872 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.872 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.872 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.872 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.872 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.872 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.872 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.872 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.872 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.872 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.872 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.873 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.873 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.873 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.873 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.873 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.873 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.873 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.873 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:53.873 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:53.873 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.873 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.873 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.873 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.874 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.874 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.874 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.874 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.874 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.874 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.874 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.874 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:53.874 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:53.874 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.874 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.874 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.874 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:53.875 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.875 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.875 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.875 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.875 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.875 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.875 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.875 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.875 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.875 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.875 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.875 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.875 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.875 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.876 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:53.876 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:53.876 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.876 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.876 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.876 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.876 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.876 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.876 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.876 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.876 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.876 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.876 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.876 INFO:teuthology.orchestra.run.smithi096.stdout: "min_size": { 2024-04-25T14:15:53.877 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "min_size", 2024-04-25T14:15:53.877 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.877 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.877 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.877 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.877 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.877 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.877 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.877 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "default min_size for any created local pool", 2024-04-25T14:15:53.877 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T14:15:53.877 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.877 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.877 INFO:teuthology.orchestra.run.smithi096.stdout: "num_rep": { 2024-04-25T14:15:53.878 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "num_rep", 2024-04-25T14:15:53.878 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.878 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.878 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.878 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "3", 2024-04-25T14:15:53.878 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.878 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.878 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.878 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "default replica count for any created local pool", 2024-04-25T14:15:53.878 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.878 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.878 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.879 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num": { 2024-04-25T14:15:53.879 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pg_num", 2024-04-25T14:15:53.879 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.879 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.879 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.879 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "128", 2024-04-25T14:15:53.879 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.879 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.879 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.879 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T14:15:53.879 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.879 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.879 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.879 INFO:teuthology.orchestra.run.smithi096.stdout: "prefix": { 2024-04-25T14:15:53.880 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "prefix", 2024-04-25T14:15:53.880 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.880 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.880 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.880 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.880 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.880 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.880 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.880 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "name prefix for any created local pool", 2024-04-25T14:15:53.880 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.880 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.880 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.880 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:53.881 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:53.881 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.881 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:53.881 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.881 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:53.881 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.881 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.881 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.881 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.881 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.881 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.881 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.881 INFO:teuthology.orchestra.run.smithi096.stdout: "subtree": { 2024-04-25T14:15:53.882 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "subtree", 2024-04-25T14:15:53.882 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.882 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.882 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.882 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "rack", 2024-04-25T14:15:53.882 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.882 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.882 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.882 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T14:15:53.882 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T14:15:53.882 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.882 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.882 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.882 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:53.883 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mds_autoscaler", 2024-04-25T14:15:53.883 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:53.883 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:53.883 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:53.883 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:53.883 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:53.883 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.883 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.883 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.883 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.883 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.883 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.884 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.884 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.884 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.884 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.884 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.884 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.884 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.884 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.884 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.884 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.884 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.884 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.885 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:53.885 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:53.885 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.885 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.885 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.885 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.885 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.885 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.885 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.885 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.885 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.885 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.885 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.886 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:53.886 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:53.886 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.886 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.886 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.886 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:53.886 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.886 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.886 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.886 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.886 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.886 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.887 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.887 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.887 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.887 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.887 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.887 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.887 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.887 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.887 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:53.887 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:53.887 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.887 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.887 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.888 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.888 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.888 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.888 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.888 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.888 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.888 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.888 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.888 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:53.888 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:53.888 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.888 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:53.888 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.888 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:53.889 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.889 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.889 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.889 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.889 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.889 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.889 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.889 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.889 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:53.889 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mirroring", 2024-04-25T14:15:53.890 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:53.890 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:53.890 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:53.890 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:53.890 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:53.890 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.890 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.890 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.890 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.890 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.890 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.890 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.890 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.891 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.891 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.891 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.891 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.891 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.891 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.891 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.891 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.891 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.891 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.891 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:53.891 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:53.891 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.892 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.892 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.892 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.892 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.892 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.892 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.892 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.892 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.892 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.892 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.893 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:53.893 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:53.893 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.893 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.893 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.893 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:53.893 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.893 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.893 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.893 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.893 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.893 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.893 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.893 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.894 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.894 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.894 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.894 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.894 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.894 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.894 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:53.894 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:53.894 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.894 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.894 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.894 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.894 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.895 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.895 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.895 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.895 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.895 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.895 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.895 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:53.895 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:53.895 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.895 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:53.895 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.895 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:53.896 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.896 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.896 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.896 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.896 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.896 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.896 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.896 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.896 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:53.896 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "nfs", 2024-04-25T14:15:53.896 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:53.896 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:53.897 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:53.897 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:53.897 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:53.897 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.897 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.897 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.897 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.897 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.897 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.897 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.897 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.897 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.897 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.897 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.898 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.898 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.898 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.898 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.898 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.898 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.898 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.898 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:53.898 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:53.898 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.899 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.899 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.899 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.899 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.899 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.899 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.899 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.899 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.899 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.899 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.899 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:53.899 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:53.899 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.899 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.900 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.900 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:53.900 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.900 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.900 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.900 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.900 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.900 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.900 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.900 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.900 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.900 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.900 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.901 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.901 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.901 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.901 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:53.901 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:53.901 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.901 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.901 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.901 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.901 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.901 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.901 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.902 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.902 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.902 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.902 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.902 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:53.902 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:53.902 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.902 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:53.902 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.902 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:53.902 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.903 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.903 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.903 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.903 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.903 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.903 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.903 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.903 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:53.903 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "orchestrator", 2024-04-25T14:15:53.903 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:53.903 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:53.903 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:53.904 INFO:teuthology.orchestra.run.smithi096.stdout: "fail_fs": { 2024-04-25T14:15:53.904 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "fail_fs", 2024-04-25T14:15:53.904 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.904 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.904 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.904 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.904 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.904 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.904 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.904 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T14:15:53.904 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.904 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.904 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.905 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:53.905 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:53.905 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.905 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.905 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.905 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.905 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.905 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.905 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.905 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.905 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.905 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.906 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.906 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.906 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.906 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.906 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.906 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.906 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.906 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.906 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:53.906 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:53.906 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.907 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.907 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.907 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.907 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.907 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.907 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.907 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.907 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.907 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.907 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.907 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:53.907 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:53.908 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.908 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.908 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.908 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:53.908 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.908 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.908 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.908 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.908 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.908 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.908 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.908 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.908 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.908 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.909 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.909 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.909 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.909 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.909 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:53.909 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:53.909 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.909 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.909 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.909 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.909 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.909 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.909 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.910 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.910 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.910 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.910 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.910 INFO:teuthology.orchestra.run.smithi096.stdout: "orchestrator": { 2024-04-25T14:15:53.910 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "orchestrator", 2024-04-25T14:15:53.910 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.910 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.910 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.910 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.910 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.910 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.910 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.911 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm", 2024-04-25T14:15:53.911 INFO:teuthology.orchestra.run.smithi096.stdout: "rook", 2024-04-25T14:15:53.911 INFO:teuthology.orchestra.run.smithi096.stdout: "test_orchestrator" 2024-04-25T14:15:53.911 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.911 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Orchestrator backend", 2024-04-25T14:15:53.911 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.911 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.911 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.911 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:53.911 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:53.911 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.911 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:53.911 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.912 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:53.912 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.912 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.912 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.912 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.912 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.912 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.912 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.912 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.912 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:53.912 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "osd_perf_query", 2024-04-25T14:15:53.913 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:53.913 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:53.913 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:53.913 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:53.913 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:53.913 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.913 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.913 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.913 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.913 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.913 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.913 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.913 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.913 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.914 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.914 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.914 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.914 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.914 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.914 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.914 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.914 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.914 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.914 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:53.914 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:53.915 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.915 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.915 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.915 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.915 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.915 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.915 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.915 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.915 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.915 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.915 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.915 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:53.915 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:53.915 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.916 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.916 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.916 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:53.916 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.916 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.916 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.916 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.916 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.916 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.916 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.916 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.916 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.916 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.917 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.917 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.917 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.917 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.917 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:53.917 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:53.917 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.917 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.917 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.917 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.917 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.917 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.917 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.917 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.918 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.918 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.918 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.918 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:53.918 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:53.918 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.918 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:53.918 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.918 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:53.918 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.918 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.918 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.919 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.919 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.919 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.919 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.919 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.919 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:53.919 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "osd_support", 2024-04-25T14:15:53.919 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:53.919 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:53.919 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:53.919 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:53.920 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:53.920 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.920 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.920 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.920 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.920 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.920 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.920 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.920 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.920 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.920 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.920 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.920 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.920 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.921 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.921 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.921 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.921 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.921 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.921 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:53.921 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:53.921 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.921 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.921 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.921 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.921 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.922 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.922 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.922 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.922 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.922 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.922 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.922 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:53.922 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:53.922 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.922 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.922 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.922 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:53.923 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.923 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.923 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.923 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.923 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.923 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.923 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.923 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.923 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.923 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.923 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.923 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.923 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.923 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.924 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:53.924 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:53.924 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.924 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.924 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.924 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.924 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.924 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.924 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.924 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.924 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.924 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.925 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.925 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:53.925 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:53.925 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.925 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:53.925 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.925 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:53.925 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.925 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.925 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.925 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.925 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.925 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.926 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.926 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.926 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:53.926 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pg_autoscaler", 2024-04-25T14:15:53.926 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:53.926 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:53.926 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:53.926 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:53.926 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:53.926 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.926 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.927 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.927 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.927 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.927 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.927 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.927 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.927 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.927 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.927 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.927 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.927 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.927 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.927 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.927 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.928 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.928 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.928 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:53.928 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:53.928 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.928 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.928 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.928 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.928 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.928 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.928 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.928 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.928 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.929 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.929 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.929 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:53.929 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:53.929 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.929 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.929 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.929 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:53.929 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.929 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.929 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.929 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.929 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.930 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.930 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.930 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.930 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.930 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.930 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.930 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.930 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.930 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.930 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:53.930 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:53.930 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.930 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.931 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.931 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.931 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.931 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.931 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.931 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.931 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.931 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.931 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.931 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-04-25T14:15:53.931 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-04-25T14:15:53.931 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:53.932 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.932 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.932 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-04-25T14:15:53.932 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.932 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.932 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.932 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.932 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.932 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.932 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.932 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:53.932 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:53.932 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.933 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:53.933 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.933 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:53.933 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.933 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.933 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.933 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.933 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.933 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.933 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.933 INFO:teuthology.orchestra.run.smithi096.stdout: "threshold": { 2024-04-25T14:15:53.933 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "threshold", 2024-04-25T14:15:53.934 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T14:15:53.934 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.934 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.934 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "3.0", 2024-04-25T14:15:53.934 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1.0", 2024-04-25T14:15:53.934 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.934 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.934 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "scaling threshold", 2024-04-25T14:15:53.934 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T14:15:53.934 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.934 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.934 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.934 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:53.935 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "progress", 2024-04-25T14:15:53.935 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:53.935 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:53.935 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:53.935 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_pg_recovery_event": { 2024-04-25T14:15:53.935 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "allow_pg_recovery_event", 2024-04-25T14:15:53.935 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.935 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.935 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.935 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.935 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.935 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.936 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.936 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T14:15:53.936 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.936 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.936 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.936 INFO:teuthology.orchestra.run.smithi096.stdout: "enabled": { 2024-04-25T14:15:53.936 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enabled", 2024-04-25T14:15:53.936 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.936 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.936 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.936 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:53.936 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.936 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.937 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.937 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.937 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.937 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.937 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.937 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:53.937 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:53.937 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.937 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.937 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.937 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.937 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.937 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.937 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.938 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.938 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.938 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.938 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.938 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.938 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.938 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.938 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.938 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.938 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.938 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.938 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:53.939 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:53.939 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.939 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.939 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.939 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.939 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.939 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.939 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.939 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.939 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.939 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.939 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.939 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:53.939 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:53.939 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.940 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.940 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.940 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:53.940 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.940 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.940 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.940 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.940 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.940 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.940 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.940 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.940 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.940 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.940 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.941 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.941 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.941 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.941 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:53.941 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:53.941 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.941 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.941 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.941 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.941 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.941 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.941 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.941 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.941 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.942 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.942 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.942 INFO:teuthology.orchestra.run.smithi096.stdout: "max_completed_events": { 2024-04-25T14:15:53.942 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_completed_events", 2024-04-25T14:15:53.942 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.942 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.942 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.942 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "50", 2024-04-25T14:15:53.942 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.942 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.942 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.942 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "number of past completed events to remember", 2024-04-25T14:15:53.943 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.943 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.943 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.943 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-04-25T14:15:53.943 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-04-25T14:15:53.943 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:53.943 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.943 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.943 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-04-25T14:15:53.943 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.943 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.943 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.943 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how long the module is going to sleep", 2024-04-25T14:15:53.944 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.944 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.944 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.944 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:53.944 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:53.944 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.944 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:53.944 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.944 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:53.944 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.944 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.944 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.944 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.945 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.945 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.945 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.945 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.945 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:53.945 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "prometheus", 2024-04-25T14:15:53.945 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:53.945 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:53.945 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:53.945 INFO:teuthology.orchestra.run.smithi096.stdout: "cache": { 2024-04-25T14:15:53.945 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cache", 2024-04-25T14:15:53.945 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.946 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.946 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.946 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:53.946 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.946 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.946 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.946 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.946 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.946 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.946 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.946 INFO:teuthology.orchestra.run.smithi096.stdout: "exclude_perf_counters": { 2024-04-25T14:15:53.947 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "exclude_perf_counters", 2024-04-25T14:15:53.947 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.947 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.947 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.947 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:53.947 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.947 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.947 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.947 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T14:15:53.947 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T14:15:53.947 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.947 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.948 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:53.948 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:53.948 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.948 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.948 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.948 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.948 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.948 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.948 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.948 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.948 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.948 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.948 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.949 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.949 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.949 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.949 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.949 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.949 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.949 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.949 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:53.949 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:53.949 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.950 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.950 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.950 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.950 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.950 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.950 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.950 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.950 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.950 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.950 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.950 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:53.950 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:53.951 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.951 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.951 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.951 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:53.951 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.951 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.951 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.951 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.951 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.951 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.951 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.951 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.951 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.951 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.952 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.952 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.952 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.952 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.952 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:53.952 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:53.952 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.952 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.952 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.952 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.952 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.952 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.952 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.953 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.953 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.953 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.953 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.953 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_stats_pools": { 2024-04-25T14:15:53.953 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_stats_pools", 2024-04-25T14:15:53.953 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.953 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.953 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.953 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.953 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.953 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.954 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.954 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.954 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.954 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.954 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.954 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T14:15:53.954 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T14:15:53.954 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.954 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.954 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.954 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "300", 2024-04-25T14:15:53.954 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.955 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.955 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.955 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.955 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.955 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.955 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.955 INFO:teuthology.orchestra.run.smithi096.stdout: "scrape_interval": { 2024-04-25T14:15:53.955 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "scrape_interval", 2024-04-25T14:15:53.955 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T14:15:53.955 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.955 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.955 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "15.0", 2024-04-25T14:15:53.956 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.956 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.956 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.956 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.956 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.956 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.956 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.956 INFO:teuthology.orchestra.run.smithi096.stdout: "server_addr": { 2024-04-25T14:15:53.956 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_addr", 2024-04-25T14:15:53.956 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.956 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.956 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.956 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "::", 2024-04-25T14:15:53.957 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.957 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.957 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.957 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T14:15:53.957 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.957 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.957 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.957 INFO:teuthology.orchestra.run.smithi096.stdout: "server_port": { 2024-04-25T14:15:53.957 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_port", 2024-04-25T14:15:53.957 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.957 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.957 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.957 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "9283", 2024-04-25T14:15:53.958 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.958 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.958 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.958 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T14:15:53.958 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.958 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.958 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.958 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:53.958 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:53.958 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.958 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:53.959 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.959 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:53.959 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.959 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.959 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.959 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.959 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.959 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.959 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.959 INFO:teuthology.orchestra.run.smithi096.stdout: "stale_cache_strategy": { 2024-04-25T14:15:53.959 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "stale_cache_strategy", 2024-04-25T14:15:53.959 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.959 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.960 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.960 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "log", 2024-04-25T14:15:53.960 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.960 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.960 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.960 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.960 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.960 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.960 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.960 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_behaviour": { 2024-04-25T14:15:53.960 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_behaviour", 2024-04-25T14:15:53.960 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.961 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.961 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.961 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "default", 2024-04-25T14:15:53.961 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.961 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.961 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.961 INFO:teuthology.orchestra.run.smithi096.stdout: "default", 2024-04-25T14:15:53.961 INFO:teuthology.orchestra.run.smithi096.stdout: "error" 2024-04-25T14:15:53.961 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.961 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.961 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.961 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.961 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.961 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_error_status_code": { 2024-04-25T14:15:53.962 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_error_status_code", 2024-04-25T14:15:53.962 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.962 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.962 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.962 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "500", 2024-04-25T14:15:53.962 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "400", 2024-04-25T14:15:53.962 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "599", 2024-04-25T14:15:53.962 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.962 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.962 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.962 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.962 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.962 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.962 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:53.963 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_support", 2024-04-25T14:15:53.963 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:53.963 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:53.963 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:53.963 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:53.963 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:53.963 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.963 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.963 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.963 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.963 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.964 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.964 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.964 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.964 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.964 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.964 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.964 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.964 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.964 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.964 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.964 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.964 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.964 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.964 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:53.965 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:53.965 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.965 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.965 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.965 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.965 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.965 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.965 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.965 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.965 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.965 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.965 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.966 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:53.966 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:53.966 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.966 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.966 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.966 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:53.966 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.966 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.966 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.966 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.966 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.966 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.966 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.966 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.967 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.967 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.967 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.967 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.967 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.967 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.967 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:53.967 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:53.967 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.967 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.967 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.967 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.968 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.968 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.968 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.968 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.968 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.968 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.968 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.968 INFO:teuthology.orchestra.run.smithi096.stdout: "max_concurrent_snap_create": { 2024-04-25T14:15:53.968 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_concurrent_snap_create", 2024-04-25T14:15:53.968 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.968 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.968 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.968 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T14:15:53.968 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.969 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.969 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.969 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.969 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.969 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.969 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.969 INFO:teuthology.orchestra.run.smithi096.stdout: "mirror_snapshot_schedule": { 2024-04-25T14:15:53.969 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T14:15:53.969 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.969 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.969 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.969 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.970 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.970 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.970 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.970 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.970 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.970 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.970 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.970 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:53.970 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:53.970 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.970 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:53.970 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.970 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:53.971 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.971 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.971 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.971 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.971 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.971 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.971 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.971 INFO:teuthology.orchestra.run.smithi096.stdout: "trash_purge_schedule": { 2024-04-25T14:15:53.971 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "trash_purge_schedule", 2024-04-25T14:15:53.971 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.971 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.971 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.971 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.971 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.972 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.972 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.972 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.972 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.972 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.972 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.972 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.972 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:53.972 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "restful", 2024-04-25T14:15:53.972 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:53.972 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:53.972 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:53.973 INFO:teuthology.orchestra.run.smithi096.stdout: "enable_auth": { 2024-04-25T14:15:53.973 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enable_auth", 2024-04-25T14:15:53.973 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.973 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.973 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.973 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:53.973 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.973 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.973 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.973 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.973 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.973 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.973 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.974 INFO:teuthology.orchestra.run.smithi096.stdout: "key_file": { 2024-04-25T14:15:53.974 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "key_file", 2024-04-25T14:15:53.974 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.974 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.974 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.974 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.974 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.974 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.974 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.974 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.974 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.974 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.974 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.975 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:53.975 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:53.975 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.975 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.975 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.975 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.975 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.975 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.975 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.975 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.975 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.975 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.975 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.976 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.976 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.976 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.976 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.976 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.976 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.976 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.976 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:53.976 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:53.976 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.976 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.976 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.976 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.976 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.976 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.977 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.977 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.977 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.977 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.977 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.977 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:53.977 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:53.977 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.977 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.977 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.977 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:53.977 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.977 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.978 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.978 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.978 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.978 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.978 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.978 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.978 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.978 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.978 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.978 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.978 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.978 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.979 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:53.979 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:53.979 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.979 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.979 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.979 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.979 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.979 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.979 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.979 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.979 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.979 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.979 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.980 INFO:teuthology.orchestra.run.smithi096.stdout: "server_addr": { 2024-04-25T14:15:53.980 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_addr", 2024-04-25T14:15:53.980 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.980 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.980 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.980 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.980 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.980 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.980 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.980 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.980 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.980 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.980 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.981 INFO:teuthology.orchestra.run.smithi096.stdout: "server_port": { 2024-04-25T14:15:53.981 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_port", 2024-04-25T14:15:53.981 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.981 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.981 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.981 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.981 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.981 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.981 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.981 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.981 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.981 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.981 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.982 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:53.982 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:53.982 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.982 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:53.982 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.982 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:53.982 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.982 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.982 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.982 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.982 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.982 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.982 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.983 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.983 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:53.983 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rgw", 2024-04-25T14:15:53.983 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:53.983 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:53.983 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:53.983 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:53.983 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:53.983 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.983 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.983 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.984 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.984 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.984 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.984 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.984 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.984 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.984 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.984 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.984 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.984 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.984 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.984 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.984 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.984 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.984 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.985 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:53.985 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:53.985 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.985 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.985 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.985 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.985 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.985 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.985 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.985 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.985 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.985 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.985 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.986 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:53.986 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:53.986 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.986 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.986 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.986 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:53.986 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.986 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.986 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.986 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.986 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.986 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.986 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.987 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.987 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.987 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.987 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.987 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.987 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.987 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.987 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:53.987 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:53.987 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.987 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.988 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.988 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.988 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.988 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.988 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.988 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.988 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.988 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.988 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.988 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:53.988 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:53.988 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.988 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:53.989 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.989 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:53.989 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.989 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.989 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.989 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.989 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.989 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.989 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.989 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.989 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:53.989 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rook", 2024-04-25T14:15:53.990 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:53.990 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:53.990 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:53.990 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:53.990 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:53.990 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.990 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.990 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.990 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.990 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.990 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.990 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.990 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.990 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.991 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.991 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.991 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.991 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.991 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.991 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.991 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.991 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.991 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.991 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:53.991 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:53.992 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.992 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.992 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.992 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.992 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.992 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.992 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.992 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.992 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.992 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.992 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.992 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:53.992 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:53.993 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.993 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.993 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.993 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:53.993 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.993 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.993 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.993 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.993 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.993 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.993 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.993 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.993 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.993 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.993 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.994 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.994 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.994 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.994 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:53.994 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:53.994 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.994 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.994 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.994 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:53.994 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.994 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.994 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.995 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.995 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.995 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.995 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.995 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:53.995 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:53.995 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:53.995 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:53.995 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.995 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:53.995 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.995 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.996 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.996 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.996 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.996 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.996 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.996 INFO:teuthology.orchestra.run.smithi096.stdout: "storage_class": { 2024-04-25T14:15:53.996 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "storage_class", 2024-04-25T14:15:53.996 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.996 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.996 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:53.996 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "local", 2024-04-25T14:15:53.996 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.996 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.997 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:53.997 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T14:15:53.997 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.997 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.997 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.997 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.997 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:53.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:53.997 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "selftest", 2024-04-25T14:15:53.997 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:53.997 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:53.997 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:53.998 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:53.998 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:53.998 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:53.998 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:53.998 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:53.998 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:53.998 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:53.998 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:53.998 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:53.998 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:53.998 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:53.998 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:53.998 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:53.999 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:53.999 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:53.999 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:53.999 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:53.999 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:53.999 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:53.999 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:53.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:53.999 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:53.999 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:53.999 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:53.999 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.000 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.000 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.000 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.000 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.000 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.000 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.000 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.000 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.000 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.000 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.000 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.000 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.000 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.001 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.001 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.001 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.001 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.001 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.001 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.001 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.001 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.001 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.001 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.001 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.001 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.001 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.001 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.001 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.001 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.002 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.002 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.002 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.002 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.002 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.002 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.002 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.002 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.002 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.002 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.002 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.002 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.002 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.003 INFO:teuthology.orchestra.run.smithi096.stdout: "roption1": { 2024-04-25T14:15:54.003 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "roption1", 2024-04-25T14:15:54.003 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.003 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.003 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.003 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.005 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.005 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.005 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.005 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.005 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.006 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.006 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.006 INFO:teuthology.orchestra.run.smithi096.stdout: "roption2": { 2024-04-25T14:15:54.006 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "roption2", 2024-04-25T14:15:54.006 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.006 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.006 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.006 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "xyz", 2024-04-25T14:15:54.006 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.006 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.006 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.006 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.007 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.007 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.007 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.007 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption1": { 2024-04-25T14:15:54.007 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption1", 2024-04-25T14:15:54.007 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.007 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.007 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.007 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.007 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.007 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.007 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.008 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.008 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.008 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.008 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.008 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption2": { 2024-04-25T14:15:54.008 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption2", 2024-04-25T14:15:54.008 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.008 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.008 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.008 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.008 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.009 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.009 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.009 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.009 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.009 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.009 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.009 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption3": { 2024-04-25T14:15:54.009 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption3", 2024-04-25T14:15:54.009 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T14:15:54.009 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.009 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.009 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.009 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.010 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.010 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.010 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.010 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.010 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.010 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.010 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption4": { 2024-04-25T14:15:54.010 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption4", 2024-04-25T14:15:54.010 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.010 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.010 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.010 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.011 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.011 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.011 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.011 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.011 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.011 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.011 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.011 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption5": { 2024-04-25T14:15:54.011 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption5", 2024-04-25T14:15:54.011 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.011 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.011 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.011 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.012 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.012 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.012 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.012 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.012 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.012 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.012 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.012 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption6": { 2024-04-25T14:15:54.012 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption6", 2024-04-25T14:15:54.012 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.012 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.012 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.012 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.013 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.013 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.013 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.013 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.013 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.013 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.013 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.013 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption7": { 2024-04-25T14:15:54.013 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption7", 2024-04-25T14:15:54.013 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.013 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.013 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.013 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.013 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-04-25T14:15:54.014 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "42", 2024-04-25T14:15:54.014 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.014 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.014 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.014 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.014 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.014 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.014 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.014 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.014 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.014 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.014 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.015 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.015 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.015 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.015 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.015 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.015 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.015 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.015 INFO:teuthology.orchestra.run.smithi096.stdout: "testkey": { 2024-04-25T14:15:54.015 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "testkey", 2024-04-25T14:15:54.015 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.015 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.015 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.016 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.016 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.016 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.016 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.016 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.016 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.016 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.016 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.016 INFO:teuthology.orchestra.run.smithi096.stdout: "testlkey": { 2024-04-25T14:15:54.016 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "testlkey", 2024-04-25T14:15:54.016 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.016 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.016 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.017 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.017 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.017 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.017 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.017 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.017 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.017 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.017 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.017 INFO:teuthology.orchestra.run.smithi096.stdout: "testnewline": { 2024-04-25T14:15:54.017 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "testnewline", 2024-04-25T14:15:54.017 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.018 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.018 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.018 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.018 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.018 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.018 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.018 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.018 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.018 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.018 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.018 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.018 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.019 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snap_schedule", 2024-04-25T14:15:54.019 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.019 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.019 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.019 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_m_granularity": { 2024-04-25T14:15:54.019 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "allow_m_granularity", 2024-04-25T14:15:54.019 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.019 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.019 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.019 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.019 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.019 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.019 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.020 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T14:15:54.020 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.020 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.020 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.020 INFO:teuthology.orchestra.run.smithi096.stdout: "dump_on_update": { 2024-04-25T14:15:54.020 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "dump_on_update", 2024-04-25T14:15:54.020 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.020 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.020 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.020 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.020 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.020 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.021 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.021 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "dump database to debug log on update", 2024-04-25T14:15:54.021 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.021 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.021 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.021 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.021 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.021 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.021 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.021 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.021 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.021 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.021 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.022 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.022 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.022 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.022 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.022 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.022 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.022 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.022 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.022 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.022 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.022 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.022 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.023 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.023 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.023 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.023 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.023 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.023 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.023 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.023 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.023 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.023 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.023 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.023 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.024 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.024 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.024 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.024 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.024 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.024 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.024 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.024 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.024 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.024 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.024 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.024 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.025 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.025 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.025 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.025 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.025 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.025 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.025 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.025 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.025 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.025 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.025 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.025 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.026 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.026 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.026 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.026 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.026 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.026 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.026 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.026 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.026 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.026 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.026 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.026 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.027 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.027 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.027 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.027 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.027 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.027 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.027 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.027 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.027 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.027 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.027 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.027 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.027 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.028 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "stats", 2024-04-25T14:15:54.028 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.028 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.028 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.028 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.028 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.028 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.028 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.028 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.028 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.028 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.028 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.029 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.029 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.029 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.029 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.029 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.029 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.029 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.029 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.029 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.029 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.029 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.029 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.030 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.030 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.030 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.030 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.030 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.030 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.030 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.030 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.030 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.030 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.030 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.030 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.030 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.031 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.031 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.031 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.031 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.031 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.031 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.031 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.031 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.031 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.031 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.031 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.031 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.031 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.032 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.032 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.032 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.032 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.032 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.032 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.032 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.032 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.032 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.032 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.032 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.032 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.033 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.033 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.033 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.033 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.033 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.033 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.033 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.033 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.033 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.033 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.033 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.033 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.033 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.034 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.034 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.034 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.034 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.034 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.034 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.034 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.034 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.034 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.034 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.034 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "status", 2024-04-25T14:15:54.034 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.035 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.035 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.035 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.035 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.035 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.035 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.035 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.035 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.035 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.035 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.035 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.035 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.035 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.035 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.036 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.036 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.036 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.036 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.036 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.036 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.036 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.036 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.036 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.036 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.036 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.036 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.036 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.037 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.037 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.037 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.037 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.037 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.037 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.037 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.037 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.037 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.037 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.037 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.038 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.038 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.038 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.038 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.038 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.038 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.038 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.038 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.038 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.038 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.038 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.038 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.038 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.038 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.039 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.039 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.039 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.039 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.039 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.039 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.039 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.039 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.039 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.039 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.039 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.039 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.039 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.040 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.040 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.040 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.040 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.040 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.040 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.040 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.040 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.040 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.040 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.040 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.040 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.040 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.041 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.041 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.041 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.041 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.041 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.041 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "telegraf", 2024-04-25T14:15:54.041 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.041 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.041 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.041 INFO:teuthology.orchestra.run.smithi096.stdout: "address": { 2024-04-25T14:15:54.041 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "address", 2024-04-25T14:15:54.041 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.042 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.042 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.042 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T14:15:54.042 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.042 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.042 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.042 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.042 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.042 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.042 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.042 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-04-25T14:15:54.043 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-04-25T14:15:54.043 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:54.043 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.043 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.043 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "15", 2024-04-25T14:15:54.043 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.043 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.043 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.043 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.043 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.043 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.043 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.043 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.043 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.044 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.044 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.044 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.044 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.044 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.044 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.044 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.044 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.044 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.044 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.044 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.044 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.044 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.044 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.045 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.045 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.045 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.045 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.045 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.045 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.045 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.045 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.045 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.045 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.045 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.045 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.045 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.046 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.046 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.046 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.046 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.046 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.046 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.046 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.046 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.046 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.046 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.046 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.046 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.047 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.047 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.047 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.047 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.047 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.047 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.047 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.047 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.047 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.047 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.047 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.047 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.047 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.048 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.048 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.048 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.048 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.048 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.048 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.048 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.048 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.048 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.048 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.048 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.048 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.048 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.048 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.049 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.049 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.049 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.049 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.049 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.049 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.049 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.049 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.049 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.049 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.049 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.049 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.049 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.050 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "telemetry", 2024-04-25T14:15:54.050 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.050 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.050 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.050 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_basic": { 2024-04-25T14:15:54.050 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_basic", 2024-04-25T14:15:54.050 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.050 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.050 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.050 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.050 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.050 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.050 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.051 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T14:15:54.051 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.051 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.051 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.051 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_crash": { 2024-04-25T14:15:54.051 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_crash", 2024-04-25T14:15:54.051 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.051 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.051 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.051 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.051 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.052 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.052 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.052 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T14:15:54.052 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.052 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.052 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.052 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_device": { 2024-04-25T14:15:54.052 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_device", 2024-04-25T14:15:54.052 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.052 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.052 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.052 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.052 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.052 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.053 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.053 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T14:15:54.053 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.053 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.053 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.053 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_ident": { 2024-04-25T14:15:54.053 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_ident", 2024-04-25T14:15:54.053 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.053 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.053 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.053 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.053 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.053 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.053 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.054 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T14:15:54.054 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.054 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.054 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.054 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_perf": { 2024-04-25T14:15:54.054 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_perf", 2024-04-25T14:15:54.054 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.054 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.054 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.054 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.054 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.054 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.055 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.055 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T14:15:54.055 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.055 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.055 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.055 INFO:teuthology.orchestra.run.smithi096.stdout: "contact": { 2024-04-25T14:15:54.055 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "contact", 2024-04-25T14:15:54.055 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.055 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.055 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.055 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.055 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.055 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.056 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.056 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.056 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.056 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.056 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.056 INFO:teuthology.orchestra.run.smithi096.stdout: "description": { 2024-04-25T14:15:54.056 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "description", 2024-04-25T14:15:54.056 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.056 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.056 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.056 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.056 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.056 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.057 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.057 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.057 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.057 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.057 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.057 INFO:teuthology.orchestra.run.smithi096.stdout: "device_url": { 2024-04-25T14:15:54.057 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "device_url", 2024-04-25T14:15:54.057 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.057 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.057 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.057 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T14:15:54.057 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.058 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.058 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.058 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.058 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.058 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.058 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.058 INFO:teuthology.orchestra.run.smithi096.stdout: "enabled": { 2024-04-25T14:15:54.058 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enabled", 2024-04-25T14:15:54.058 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.058 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.058 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.058 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.058 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.059 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.059 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.059 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.059 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.059 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.059 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.059 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-04-25T14:15:54.059 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-04-25T14:15:54.059 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.059 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.059 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.059 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "24", 2024-04-25T14:15:54.059 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "8", 2024-04-25T14:15:54.059 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.060 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.060 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.060 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.060 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.060 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.060 INFO:teuthology.orchestra.run.smithi096.stdout: "last_opt_revision": { 2024-04-25T14:15:54.060 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "last_opt_revision", 2024-04-25T14:15:54.060 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.060 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.060 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.060 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "1", 2024-04-25T14:15:54.060 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.060 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.061 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.061 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.061 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.061 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.061 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.061 INFO:teuthology.orchestra.run.smithi096.stdout: "leaderboard": { 2024-04-25T14:15:54.061 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "leaderboard", 2024-04-25T14:15:54.061 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.061 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.061 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.061 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.061 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.061 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.061 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.062 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.062 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.062 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.062 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.062 INFO:teuthology.orchestra.run.smithi096.stdout: "leaderboard_description": { 2024-04-25T14:15:54.062 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "leaderboard_description", 2024-04-25T14:15:54.062 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.062 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.062 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.062 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.062 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.062 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.063 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.063 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.063 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.063 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.063 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.063 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.063 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.063 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.063 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.063 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.063 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.063 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.064 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.064 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.064 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.064 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.064 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.064 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.064 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.064 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.064 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.064 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.064 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.064 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.064 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.065 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.065 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.065 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.065 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.065 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.065 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.065 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.065 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.065 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.065 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.065 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.065 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.066 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.066 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.066 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.066 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.066 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.066 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.066 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.066 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.066 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.066 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.066 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.066 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.067 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.067 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.067 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.067 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.067 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.067 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.067 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.067 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.067 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.067 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.068 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.068 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.068 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.068 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.068 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.068 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.068 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.068 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.068 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.068 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.068 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.068 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.069 INFO:teuthology.orchestra.run.smithi096.stdout: "organization": { 2024-04-25T14:15:54.069 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "organization", 2024-04-25T14:15:54.069 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.069 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.069 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.069 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.069 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.069 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.069 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.069 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.069 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.069 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.069 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.070 INFO:teuthology.orchestra.run.smithi096.stdout: "proxy": { 2024-04-25T14:15:54.070 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "proxy", 2024-04-25T14:15:54.070 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.070 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.070 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.070 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.070 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.070 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.070 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.070 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.070 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.070 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.071 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.071 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.071 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.071 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.071 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.071 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.071 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.071 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.071 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.071 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.071 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.071 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.071 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.072 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.072 INFO:teuthology.orchestra.run.smithi096.stdout: "url": { 2024-04-25T14:15:54.072 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "url", 2024-04-25T14:15:54.072 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.072 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.072 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.072 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T14:15:54.072 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.072 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.072 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.072 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.072 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.072 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.073 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.073 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.073 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.073 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "test_orchestrator", 2024-04-25T14:15:54.073 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.073 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.073 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.073 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.073 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.073 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.073 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.073 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.074 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.074 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.074 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.074 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.074 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.074 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.074 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.074 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.074 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.074 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.074 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.074 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.074 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.075 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.075 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.075 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.075 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.075 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.075 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.075 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.075 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.075 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.075 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.075 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.075 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.076 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.076 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.076 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.076 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.076 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.076 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.076 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.076 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.076 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.076 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.076 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.076 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.076 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.076 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.077 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.077 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.077 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.077 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.077 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.077 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.077 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.077 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.077 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.077 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.077 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.077 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.077 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.078 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.078 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.078 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.078 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.078 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.078 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.078 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.078 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.078 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.078 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.078 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.078 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.079 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.079 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.079 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.079 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.079 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.079 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.079 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.079 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.079 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.079 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.079 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.079 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.080 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "volumes", 2024-04-25T14:15:54.080 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.080 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.080 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.080 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.080 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.080 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.080 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.080 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.081 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.081 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.081 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.081 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.081 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.081 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.081 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.081 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.081 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.081 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.081 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.081 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.082 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.082 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.082 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.082 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.082 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.082 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.082 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.082 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.082 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.082 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.083 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.083 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.083 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.083 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.083 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.083 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.083 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.083 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.083 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.083 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.083 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.083 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.083 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.084 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.084 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.084 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.084 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.084 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.084 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.084 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.084 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.084 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.084 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.084 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.084 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.084 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.085 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.085 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.085 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.085 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.085 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.085 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.085 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.085 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.085 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.085 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.085 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.085 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.086 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.086 INFO:teuthology.orchestra.run.smithi096.stdout: "max_concurrent_clones": { 2024-04-25T14:15:54.086 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_concurrent_clones", 2024-04-25T14:15:54.086 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.086 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.086 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.086 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "4", 2024-04-25T14:15:54.086 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.086 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.086 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.086 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T14:15:54.086 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.086 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.087 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.087 INFO:teuthology.orchestra.run.smithi096.stdout: "periodic_async_work": { 2024-04-25T14:15:54.087 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "periodic_async_work", 2024-04-25T14:15:54.087 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.087 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.087 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.087 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.087 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.087 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.087 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.087 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Periodically check for async work", 2024-04-25T14:15:54.088 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.088 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.088 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.088 INFO:teuthology.orchestra.run.smithi096.stdout: "snapshot_clone_delay": { 2024-04-25T14:15:54.088 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snapshot_clone_delay", 2024-04-25T14:15:54.088 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.088 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.088 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.088 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.088 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.088 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.088 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.089 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T14:15:54.089 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.089 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.089 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.089 INFO:teuthology.orchestra.run.smithi096.stdout: "snapshot_clone_no_wait": { 2024-04-25T14:15:54.089 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T14:15:54.089 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.089 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.089 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.089 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.089 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.089 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.089 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.090 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T14:15:54.090 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.090 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.090 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.090 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.090 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.090 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.090 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.090 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.090 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.090 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.090 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.090 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.091 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.091 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.091 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.091 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.091 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.091 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.091 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix", 2024-04-25T14:15:54.091 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.091 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.091 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.091 INFO:teuthology.orchestra.run.smithi096.stdout: "discovery_interval": { 2024-04-25T14:15:54.091 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "discovery_interval", 2024-04-25T14:15:54.092 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-04-25T14:15:54.092 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.092 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.092 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "100", 2024-04-25T14:15:54.092 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.092 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.092 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.092 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.092 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.092 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.092 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.092 INFO:teuthology.orchestra.run.smithi096.stdout: "identifier": { 2024-04-25T14:15:54.092 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "identifier", 2024-04-25T14:15:54.092 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.093 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.093 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.093 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.093 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.093 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.093 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.093 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.093 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.093 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.093 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.093 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-04-25T14:15:54.093 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-04-25T14:15:54.094 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:54.094 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.094 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.094 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-04-25T14:15:54.094 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.094 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.094 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.094 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.094 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.094 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.094 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.094 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.095 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.095 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.095 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.095 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.095 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.095 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.095 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.095 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.095 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.095 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.095 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.095 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.095 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.096 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.096 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.096 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.096 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.096 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.096 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.096 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.096 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.096 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.096 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.096 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.096 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.096 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.097 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.097 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.097 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.097 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.097 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.097 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.097 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.097 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.097 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.097 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.097 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.097 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.097 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.098 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.098 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.098 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.098 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.098 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.098 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.098 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.098 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.098 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.099 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.099 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.099 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.099 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.099 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.099 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.099 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.099 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.099 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.099 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.099 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.099 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.099 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.100 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.100 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.100 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.100 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.100 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.100 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.100 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.100 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.100 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.100 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.100 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.100 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.100 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.101 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.101 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.101 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.101 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.101 INFO:teuthology.orchestra.run.smithi096.stdout: "zabbix_host": { 2024-04-25T14:15:54.101 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix_host", 2024-04-25T14:15:54.101 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.101 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.101 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.101 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.101 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.101 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.101 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.102 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.102 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.102 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.102 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.102 INFO:teuthology.orchestra.run.smithi096.stdout: "zabbix_port": { 2024-04-25T14:15:54.102 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix_port", 2024-04-25T14:15:54.102 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.102 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.102 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.102 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10051", 2024-04-25T14:15:54.102 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.102 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.102 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.103 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.103 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.103 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.103 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.103 INFO:teuthology.orchestra.run.smithi096.stdout: "zabbix_sender": { 2024-04-25T14:15:54.103 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix_sender", 2024-04-25T14:15:54.103 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.103 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.103 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.103 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T14:15:54.103 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.103 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.103 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.103 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.104 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.104 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.104 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.104 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.104 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.104 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.104 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-25T14:15:54.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.104 INFO:teuthology.orchestra.run.smithi096.stdout: "gid": 24119, 2024-04-25T14:15:54.104 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "y", 2024-04-25T14:15:54.104 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_features": 4540701547738038271, 2024-04-25T14:15:54.104 INFO:teuthology.orchestra.run.smithi096.stdout: "available_modules": [ 2024-04-25T14:15:54.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.105 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "alerts", 2024-04-25T14:15:54.105 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.105 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.105 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.105 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-04-25T14:15:54.105 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-04-25T14:15:54.105 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:54.105 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.105 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.105 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-04-25T14:15:54.105 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.105 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.105 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.105 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T14:15:54.106 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.106 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.106 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.106 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.106 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.106 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.106 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.106 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.106 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.106 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.106 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.106 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.107 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.107 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.107 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.107 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.107 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.107 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.107 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.107 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.107 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.107 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.107 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.107 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.107 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.108 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.108 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.108 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.108 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.108 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.108 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.108 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.108 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.108 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.108 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.108 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.108 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.109 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.109 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.109 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.109 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.109 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.109 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.109 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.109 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.109 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.109 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.109 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.109 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.109 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.110 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.110 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.110 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.110 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.110 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.110 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.110 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.110 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.110 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.110 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.110 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.110 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.110 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.111 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.111 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.111 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.111 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.111 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.111 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.111 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_destination": { 2024-04-25T14:15:54.111 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_destination", 2024-04-25T14:15:54.111 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.111 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.111 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.111 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.111 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.112 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.112 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.112 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Email address to send alerts to", 2024-04-25T14:15:54.112 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.112 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.112 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.112 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_from_name": { 2024-04-25T14:15:54.112 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_from_name", 2024-04-25T14:15:54.112 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.112 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.112 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.112 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "Ceph", 2024-04-25T14:15:54.112 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.113 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.113 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.113 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Email From: name", 2024-04-25T14:15:54.113 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.113 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.113 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.113 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_host": { 2024-04-25T14:15:54.113 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_host", 2024-04-25T14:15:54.113 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.113 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.113 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.113 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.113 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.114 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.114 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.114 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SMTP server", 2024-04-25T14:15:54.114 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.114 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.114 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.114 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_password": { 2024-04-25T14:15:54.114 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_password", 2024-04-25T14:15:54.114 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.114 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.114 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.114 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.115 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.115 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.115 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.115 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Password to authenticate with", 2024-04-25T14:15:54.115 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.115 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.115 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.115 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_port": { 2024-04-25T14:15:54.115 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_port", 2024-04-25T14:15:54.115 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.115 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.115 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.115 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "465", 2024-04-25T14:15:54.116 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.116 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.116 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.116 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SMTP port", 2024-04-25T14:15:54.116 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.116 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.116 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.116 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_sender": { 2024-04-25T14:15:54.116 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_sender", 2024-04-25T14:15:54.116 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.116 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.116 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.117 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.117 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.117 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.117 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.117 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SMTP envelope sender", 2024-04-25T14:15:54.117 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.117 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.117 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.117 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_ssl": { 2024-04-25T14:15:54.117 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_ssl", 2024-04-25T14:15:54.117 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.117 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.117 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.117 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.118 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.118 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.118 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.118 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T14:15:54.118 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.118 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.118 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.118 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_user": { 2024-04-25T14:15:54.118 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_user", 2024-04-25T14:15:54.118 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.118 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.118 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.119 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.119 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.119 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.119 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.119 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "User to authenticate as", 2024-04-25T14:15:54.119 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.119 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.119 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.119 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.119 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.119 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.119 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.120 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.120 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.120 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.120 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.120 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.120 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.120 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.120 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.120 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.120 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.120 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.121 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "balancer", 2024-04-25T14:15:54.121 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.121 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.121 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.121 INFO:teuthology.orchestra.run.smithi096.stdout: "active": { 2024-04-25T14:15:54.121 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "active", 2024-04-25T14:15:54.121 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.121 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.121 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.121 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.121 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.121 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.121 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.121 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T14:15:54.121 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.122 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.122 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.122 INFO:teuthology.orchestra.run.smithi096.stdout: "begin_time": { 2024-04-25T14:15:54.122 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "begin_time", 2024-04-25T14:15:54.122 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.122 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.122 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.122 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0000", 2024-04-25T14:15:54.122 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.122 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.123 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.123 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T14:15:54.123 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T14:15:54.123 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.123 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.123 INFO:teuthology.orchestra.run.smithi096.stdout: "begin_weekday": { 2024-04-25T14:15:54.123 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "begin_weekday", 2024-04-25T14:15:54.123 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-04-25T14:15:54.123 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.123 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.123 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.123 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "0", 2024-04-25T14:15:54.123 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "6", 2024-04-25T14:15:54.124 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.124 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T14:15:54.124 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T14:15:54.124 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.124 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.124 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_compat_max_iterations": { 2024-04-25T14:15:54.124 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crush_compat_max_iterations", 2024-04-25T14:15:54.124 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-04-25T14:15:54.124 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.124 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.124 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "25", 2024-04-25T14:15:54.125 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-04-25T14:15:54.125 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "250", 2024-04-25T14:15:54.125 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.125 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T14:15:54.125 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.125 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.125 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.125 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_compat_metrics": { 2024-04-25T14:15:54.125 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crush_compat_metrics", 2024-04-25T14:15:54.125 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.125 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.126 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.126 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T14:15:54.126 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.126 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.126 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.126 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T14:15:54.126 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T14:15:54.126 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.126 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.126 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_compat_step": { 2024-04-25T14:15:54.126 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crush_compat_step", 2024-04-25T14:15:54.126 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T14:15:54.127 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.127 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.127 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0.5", 2024-04-25T14:15:54.127 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "0.001", 2024-04-25T14:15:54.127 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "0.999", 2024-04-25T14:15:54.127 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.127 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "aggressiveness of optimization", 2024-04-25T14:15:54.127 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T14:15:54.127 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.127 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.127 INFO:teuthology.orchestra.run.smithi096.stdout: "end_time": { 2024-04-25T14:15:54.127 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "end_time", 2024-04-25T14:15:54.128 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.128 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.128 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.128 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "2359", 2024-04-25T14:15:54.128 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.128 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.128 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.128 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T14:15:54.128 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T14:15:54.128 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.128 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.128 INFO:teuthology.orchestra.run.smithi096.stdout: "end_weekday": { 2024-04-25T14:15:54.128 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "end_weekday", 2024-04-25T14:15:54.129 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-04-25T14:15:54.129 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.129 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.129 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.129 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "0", 2024-04-25T14:15:54.129 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "6", 2024-04-25T14:15:54.129 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.129 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T14:15:54.129 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T14:15:54.129 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.129 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.129 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.130 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.130 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.130 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.130 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.130 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.130 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.130 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.130 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.130 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.130 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.130 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.130 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.130 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.131 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.131 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.131 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.131 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.131 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.131 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.131 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.131 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.131 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.131 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.131 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.131 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.132 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.132 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.132 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.132 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.132 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.132 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.132 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.132 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.132 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.132 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.132 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.132 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.133 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.133 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.133 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.133 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.133 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.133 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.133 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.133 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.133 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.133 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.133 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.133 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.133 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.133 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.134 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.134 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.134 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.134 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.134 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.134 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.134 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.134 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.134 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.134 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.134 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.134 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.135 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.135 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.135 INFO:teuthology.orchestra.run.smithi096.stdout: "min_score": { 2024-04-25T14:15:54.135 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "min_score", 2024-04-25T14:15:54.135 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T14:15:54.135 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.135 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.135 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.135 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.135 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.135 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.136 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T14:15:54.136 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.136 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.136 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.136 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": { 2024-04-25T14:15:54.136 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mode", 2024-04-25T14:15:54.136 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.136 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.136 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.136 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "upmap", 2024-04-25T14:15:54.136 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.136 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.136 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.137 INFO:teuthology.orchestra.run.smithi096.stdout: "crush-compat", 2024-04-25T14:15:54.137 INFO:teuthology.orchestra.run.smithi096.stdout: "none", 2024-04-25T14:15:54.137 INFO:teuthology.orchestra.run.smithi096.stdout: "read", 2024-04-25T14:15:54.137 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap", 2024-04-25T14:15:54.137 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap-read" 2024-04-25T14:15:54.137 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.137 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Balancer mode", 2024-04-25T14:15:54.137 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.137 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.137 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.137 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_ids": { 2024-04-25T14:15:54.137 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pool_ids", 2024-04-25T14:15:54.138 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.138 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.138 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.138 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.138 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.138 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.138 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.138 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T14:15:54.138 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.138 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.138 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.138 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-04-25T14:15:54.139 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-04-25T14:15:54.139 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:54.139 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.139 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.139 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-04-25T14:15:54.139 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.139 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.139 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.139 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T14:15:54.139 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.139 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.139 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.139 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.140 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.140 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.140 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.140 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.140 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.140 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.140 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.140 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.140 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.140 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.140 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.140 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.140 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap_max_deviation": { 2024-04-25T14:15:54.141 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "upmap_max_deviation", 2024-04-25T14:15:54.141 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.141 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.141 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.141 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-04-25T14:15:54.141 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-04-25T14:15:54.141 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.141 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.141 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T14:15:54.141 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T14:15:54.141 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.141 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.141 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap_max_optimizations": { 2024-04-25T14:15:54.142 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "upmap_max_optimizations", 2024-04-25T14:15:54.142 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-04-25T14:15:54.142 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.142 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.142 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T14:15:54.142 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.142 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.142 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.142 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T14:15:54.142 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.142 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.142 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.142 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.143 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.143 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cephadm", 2024-04-25T14:15:54.143 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.143 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.143 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.143 INFO:teuthology.orchestra.run.smithi096.stdout: "agent_down_multiplier": { 2024-04-25T14:15:54.143 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "agent_down_multiplier", 2024-04-25T14:15:54.143 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T14:15:54.143 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.143 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.143 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "3.0", 2024-04-25T14:15:54.143 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.144 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.144 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.144 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T14:15:54.144 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.144 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.144 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.144 INFO:teuthology.orchestra.run.smithi096.stdout: "agent_refresh_rate": { 2024-04-25T14:15:54.144 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "agent_refresh_rate", 2024-04-25T14:15:54.144 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:54.144 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.144 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.144 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "20", 2024-04-25T14:15:54.144 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.145 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.145 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.145 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T14:15:54.145 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.145 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.145 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.145 INFO:teuthology.orchestra.run.smithi096.stdout: "agent_starting_port": { 2024-04-25T14:15:54.145 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "agent_starting_port", 2024-04-25T14:15:54.145 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.145 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.145 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.145 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "4721", 2024-04-25T14:15:54.145 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.146 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.146 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.146 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T14:15:54.146 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.146 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.146 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.146 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_ptrace": { 2024-04-25T14:15:54.146 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "allow_ptrace", 2024-04-25T14:15:54.146 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.146 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.146 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.146 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.146 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.147 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.147 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.147 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T14:15:54.147 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T14:15:54.147 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.147 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.147 INFO:teuthology.orchestra.run.smithi096.stdout: "autotune_interval": { 2024-04-25T14:15:54.147 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "autotune_interval", 2024-04-25T14:15:54.147 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:54.147 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.147 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.148 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-04-25T14:15:54.148 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.148 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.148 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.148 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T14:15:54.148 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.148 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.148 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.148 INFO:teuthology.orchestra.run.smithi096.stdout: "autotune_memory_target_ratio": { 2024-04-25T14:15:54.148 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T14:15:54.148 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T14:15:54.148 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.149 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.149 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0.7", 2024-04-25T14:15:54.149 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.149 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.149 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.149 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T14:15:54.149 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.149 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.149 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.149 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm_log_destination": { 2024-04-25T14:15:54.149 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cephadm_log_destination", 2024-04-25T14:15:54.149 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.149 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.150 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.150 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.150 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.150 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.150 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.150 INFO:teuthology.orchestra.run.smithi096.stdout: "file", 2024-04-25T14:15:54.150 INFO:teuthology.orchestra.run.smithi096.stdout: "file,syslog", 2024-04-25T14:15:54.150 INFO:teuthology.orchestra.run.smithi096.stdout: "syslog" 2024-04-25T14:15:54.150 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.150 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T14:15:54.150 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.150 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.150 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.151 INFO:teuthology.orchestra.run.smithi096.stdout: "cgroups_split": { 2024-04-25T14:15:54.151 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cgroups_split", 2024-04-25T14:15:54.151 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.151 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.151 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.151 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.151 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.151 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.151 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.151 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T14:15:54.151 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.151 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.151 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.152 INFO:teuthology.orchestra.run.smithi096.stdout: "config_checks_enabled": { 2024-04-25T14:15:54.152 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "config_checks_enabled", 2024-04-25T14:15:54.152 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.152 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.152 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.152 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.152 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.152 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.152 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.152 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T14:15:54.152 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.152 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.152 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.153 INFO:teuthology.orchestra.run.smithi096.stdout: "config_dashboard": { 2024-04-25T14:15:54.153 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "config_dashboard", 2024-04-25T14:15:54.153 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.153 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.153 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.153 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.153 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.153 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.153 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.153 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T14:15:54.153 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.153 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.153 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.153 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_alertmanager": { 2024-04-25T14:15:54.154 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_alertmanager", 2024-04-25T14:15:54.154 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.154 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.154 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.154 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T14:15:54.154 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.154 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.154 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.154 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-04-25T14:15:54.154 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.154 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.154 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.154 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_base": { 2024-04-25T14:15:54.155 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_base", 2024-04-25T14:15:54.155 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.155 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.155 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.155 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T14:15:54.155 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.155 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.155 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.155 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Container image name, without the tag", 2024-04-25T14:15:54.155 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.155 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.155 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.155 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_elasticsearch": { 2024-04-25T14:15:54.156 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_elasticsearch", 2024-04-25T14:15:54.156 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.156 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.156 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.156 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T14:15:54.156 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.156 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.156 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.156 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "elasticsearch container image", 2024-04-25T14:15:54.156 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.156 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.156 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.156 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_grafana": { 2024-04-25T14:15:54.157 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_grafana", 2024-04-25T14:15:54.157 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.157 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.157 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.157 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T14:15:54.157 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.157 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.157 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.157 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-04-25T14:15:54.157 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.157 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.157 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.157 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_haproxy": { 2024-04-25T14:15:54.158 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_haproxy", 2024-04-25T14:15:54.158 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.158 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.158 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.158 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T14:15:54.158 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.158 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.158 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.158 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "HAproxy container image", 2024-04-25T14:15:54.158 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.158 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.158 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.158 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_jaeger_agent": { 2024-04-25T14:15:54.159 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_jaeger_agent", 2024-04-25T14:15:54.159 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.159 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.159 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.159 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T14:15:54.159 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.159 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.159 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.159 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Jaeger agent container image", 2024-04-25T14:15:54.159 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.159 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.159 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.159 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_jaeger_collector": { 2024-04-25T14:15:54.160 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_jaeger_collector", 2024-04-25T14:15:54.160 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.160 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.160 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.160 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T14:15:54.160 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.160 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.160 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.160 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Jaeger collector container image", 2024-04-25T14:15:54.160 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.160 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.160 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.161 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_jaeger_query": { 2024-04-25T14:15:54.161 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_jaeger_query", 2024-04-25T14:15:54.161 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.161 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.161 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.161 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T14:15:54.161 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.161 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.161 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.161 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Jaeger query container image", 2024-04-25T14:15:54.161 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.161 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.161 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.162 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_keepalived": { 2024-04-25T14:15:54.162 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_keepalived", 2024-04-25T14:15:54.162 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.162 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.162 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.162 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T14:15:54.162 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.162 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.162 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.162 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Keepalived container image", 2024-04-25T14:15:54.162 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.162 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.162 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.163 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_loki": { 2024-04-25T14:15:54.163 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_loki", 2024-04-25T14:15:54.163 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.163 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.163 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.163 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T14:15:54.163 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.163 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.163 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.163 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Loki container image", 2024-04-25T14:15:54.163 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.163 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.164 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.164 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_node_exporter": { 2024-04-25T14:15:54.164 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_node_exporter", 2024-04-25T14:15:54.164 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.164 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.164 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.164 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T14:15:54.164 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.164 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.164 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.164 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-04-25T14:15:54.164 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.164 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.164 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.165 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_nvmeof": { 2024-04-25T14:15:54.165 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_nvmeof", 2024-04-25T14:15:54.165 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.165 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.165 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.165 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T14:15:54.165 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.165 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.165 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.165 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Nvme-of container image", 2024-04-25T14:15:54.165 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.165 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.166 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.166 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_prometheus": { 2024-04-25T14:15:54.166 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_prometheus", 2024-04-25T14:15:54.166 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.166 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.166 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.166 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T14:15:54.166 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.166 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.166 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.166 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-04-25T14:15:54.166 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.166 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.167 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.167 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_promtail": { 2024-04-25T14:15:54.167 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_promtail", 2024-04-25T14:15:54.167 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.167 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.167 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.167 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T14:15:54.167 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.167 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.167 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.167 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Promtail container image", 2024-04-25T14:15:54.167 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.167 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.168 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.168 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_samba": { 2024-04-25T14:15:54.168 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_samba", 2024-04-25T14:15:54.168 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.168 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.168 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.168 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T14:15:54.168 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.168 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.168 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.168 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Samba/SMB container image", 2024-04-25T14:15:54.168 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.169 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.169 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.169 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_snmp_gateway": { 2024-04-25T14:15:54.169 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_snmp_gateway", 2024-04-25T14:15:54.169 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.169 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.169 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.169 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T14:15:54.169 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.169 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.169 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.169 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SNMP Gateway container image", 2024-04-25T14:15:54.169 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.170 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.170 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.170 INFO:teuthology.orchestra.run.smithi096.stdout: "container_init": { 2024-04-25T14:15:54.170 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_init", 2024-04-25T14:15:54.170 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.170 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.170 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.170 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.170 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.170 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.170 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.170 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T14:15:54.171 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.171 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.171 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.171 INFO:teuthology.orchestra.run.smithi096.stdout: "daemon_cache_timeout": { 2024-04-25T14:15:54.171 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "daemon_cache_timeout", 2024-04-25T14:15:54.171 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:54.171 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.171 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.171 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-04-25T14:15:54.171 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.171 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.171 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.172 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T14:15:54.172 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.172 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.172 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.172 INFO:teuthology.orchestra.run.smithi096.stdout: "default_cephadm_command_timeout": { 2024-04-25T14:15:54.172 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T14:15:54.172 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.172 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.172 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.172 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "900", 2024-04-25T14:15:54.172 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.172 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.172 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.173 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T14:15:54.173 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.173 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.173 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.173 INFO:teuthology.orchestra.run.smithi096.stdout: "default_registry": { 2024-04-25T14:15:54.173 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "default_registry", 2024-04-25T14:15:54.173 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.173 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.173 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.173 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io", 2024-04-25T14:15:54.173 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.173 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.174 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.174 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T14:15:54.174 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.174 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.174 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.174 INFO:teuthology.orchestra.run.smithi096.stdout: "device_cache_timeout": { 2024-04-25T14:15:54.174 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "device_cache_timeout", 2024-04-25T14:15:54.174 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:54.174 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.174 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.174 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "1800", 2024-04-25T14:15:54.174 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.174 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.175 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.175 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "seconds to cache device inventory", 2024-04-25T14:15:54.175 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.175 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.175 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.175 INFO:teuthology.orchestra.run.smithi096.stdout: "device_enhanced_scan": { 2024-04-25T14:15:54.175 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "device_enhanced_scan", 2024-04-25T14:15:54.175 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.175 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.175 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.175 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.175 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.175 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.176 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.176 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T14:15:54.176 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.176 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.176 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.176 INFO:teuthology.orchestra.run.smithi096.stdout: "facts_cache_timeout": { 2024-04-25T14:15:54.176 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "facts_cache_timeout", 2024-04-25T14:15:54.176 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:54.176 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.176 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.176 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-04-25T14:15:54.176 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.176 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.176 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.176 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "seconds to cache host facts data", 2024-04-25T14:15:54.177 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.177 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.177 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.177 INFO:teuthology.orchestra.run.smithi096.stdout: "grafana_dashboards_path": { 2024-04-25T14:15:54.177 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "grafana_dashboards_path", 2024-04-25T14:15:54.177 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.177 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.177 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.177 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T14:15:54.177 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.177 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.177 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.177 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T14:15:54.178 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.178 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.178 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.178 INFO:teuthology.orchestra.run.smithi096.stdout: "host_check_interval": { 2024-04-25T14:15:54.178 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "host_check_interval", 2024-04-25T14:15:54.178 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:54.178 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.178 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.178 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-04-25T14:15:54.178 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.178 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.179 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.179 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to perform a host check", 2024-04-25T14:15:54.179 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.179 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.179 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.179 INFO:teuthology.orchestra.run.smithi096.stdout: "hw_monitoring": { 2024-04-25T14:15:54.179 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "hw_monitoring", 2024-04-25T14:15:54.179 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.179 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.179 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.179 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.179 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.179 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.180 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.180 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T14:15:54.180 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.180 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.180 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.180 INFO:teuthology.orchestra.run.smithi096.stdout: "inventory_list_all": { 2024-04-25T14:15:54.180 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "inventory_list_all", 2024-04-25T14:15:54.180 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.180 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.180 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.180 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.180 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.180 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.181 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.181 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T14:15:54.181 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.181 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.181 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.181 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.181 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.181 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.181 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.181 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.181 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.181 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.181 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.182 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.182 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.182 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.182 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.182 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.182 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.182 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.182 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.182 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.182 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.182 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.182 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.182 INFO:teuthology.orchestra.run.smithi096.stdout: "log_refresh_metadata": { 2024-04-25T14:15:54.183 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_refresh_metadata", 2024-04-25T14:15:54.183 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.183 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.183 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.183 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.183 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.183 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.183 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.183 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T14:15:54.183 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.183 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.183 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.183 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.183 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.184 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.184 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.184 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.184 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.184 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.184 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.184 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.184 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T14:15:54.184 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.184 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.184 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.184 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.184 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.184 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.185 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.185 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.185 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.185 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.185 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.185 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.185 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.185 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.185 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.185 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.185 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.185 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.185 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.186 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.186 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.186 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.186 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.186 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.186 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.186 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.186 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.186 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.186 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.186 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.186 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.187 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.187 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.187 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.187 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.187 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.187 INFO:teuthology.orchestra.run.smithi096.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T14:15:54.187 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T14:15:54.187 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.187 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.187 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.187 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.187 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.187 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.187 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.188 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T14:15:54.188 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.188 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.188 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.188 INFO:teuthology.orchestra.run.smithi096.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T14:15:54.188 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T14:15:54.188 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.188 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.188 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.188 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "*", 2024-04-25T14:15:54.188 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.188 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.188 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.189 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T14:15:54.189 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.189 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.189 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.189 INFO:teuthology.orchestra.run.smithi096.stdout: "max_count_per_host": { 2024-04-25T14:15:54.189 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_count_per_host", 2024-04-25T14:15:54.189 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.189 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.189 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.189 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T14:15:54.189 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.189 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.189 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.189 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "max number of daemons per service per host", 2024-04-25T14:15:54.190 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.190 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.190 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.190 INFO:teuthology.orchestra.run.smithi096.stdout: "max_osd_draining_count": { 2024-04-25T14:15:54.190 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_osd_draining_count", 2024-04-25T14:15:54.190 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.190 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.190 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.190 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T14:15:54.190 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.190 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.190 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.190 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T14:15:54.191 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.191 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.191 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.191 INFO:teuthology.orchestra.run.smithi096.stdout: "migration_current": { 2024-04-25T14:15:54.191 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "migration_current", 2024-04-25T14:15:54.191 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.191 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.191 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.191 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.191 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.191 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.191 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.191 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "internal - do not modify", 2024-04-25T14:15:54.192 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.192 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.192 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.192 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": { 2024-04-25T14:15:54.192 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mode", 2024-04-25T14:15:54.192 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.192 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.192 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.192 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "root", 2024-04-25T14:15:54.192 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.192 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.192 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.192 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm-package", 2024-04-25T14:15:54.192 INFO:teuthology.orchestra.run.smithi096.stdout: "root" 2024-04-25T14:15:54.193 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.193 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T14:15:54.193 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.193 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.193 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.193 INFO:teuthology.orchestra.run.smithi096.stdout: "oob_default_addr": { 2024-04-25T14:15:54.193 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "oob_default_addr", 2024-04-25T14:15:54.193 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.193 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.193 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.193 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "169.254.1.1", 2024-04-25T14:15:54.193 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.194 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.194 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.194 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T14:15:54.194 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.194 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.194 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.194 INFO:teuthology.orchestra.run.smithi096.stdout: "prometheus_alerts_path": { 2024-04-25T14:15:54.194 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "prometheus_alerts_path", 2024-04-25T14:15:54.194 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.194 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.194 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.194 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T14:15:54.194 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.194 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.195 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.195 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T14:15:54.195 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.195 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.195 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.195 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_insecure": { 2024-04-25T14:15:54.195 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_insecure", 2024-04-25T14:15:54.195 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.195 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.195 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.195 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.195 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.195 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.196 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.196 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T14:15:54.196 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.196 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.196 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.196 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_password": { 2024-04-25T14:15:54.196 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_password", 2024-04-25T14:15:54.196 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.196 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.196 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.196 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.196 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.196 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.197 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.197 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T14:15:54.197 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.197 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.197 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.197 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_url": { 2024-04-25T14:15:54.197 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_url", 2024-04-25T14:15:54.197 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.197 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.197 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.197 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.197 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.197 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.198 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.198 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T14:15:54.198 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.198 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.198 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.198 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_username": { 2024-04-25T14:15:54.198 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_username", 2024-04-25T14:15:54.198 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.198 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.198 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.198 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.198 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.198 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.199 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.199 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T14:15:54.199 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.199 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.199 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.199 INFO:teuthology.orchestra.run.smithi096.stdout: "secure_monitoring_stack": { 2024-04-25T14:15:54.199 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "secure_monitoring_stack", 2024-04-25T14:15:54.199 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.199 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.199 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.199 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.199 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.199 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.200 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.200 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T14:15:54.200 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.200 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.200 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.200 INFO:teuthology.orchestra.run.smithi096.stdout: "service_discovery_port": { 2024-04-25T14:15:54.200 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "service_discovery_port", 2024-04-25T14:15:54.200 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.200 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.200 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.200 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8765", 2024-04-25T14:15:54.200 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.200 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.201 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.201 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "cephadm service discovery port", 2024-04-25T14:15:54.201 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.201 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.201 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.201 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.201 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.201 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.201 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.201 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.201 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.201 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.201 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.202 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.202 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.202 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.202 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.202 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.202 INFO:teuthology.orchestra.run.smithi096.stdout: "ssh_config_file": { 2024-04-25T14:15:54.202 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssh_config_file", 2024-04-25T14:15:54.202 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.202 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.202 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.202 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.202 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.203 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.203 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.203 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T14:15:54.203 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.203 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.203 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.203 INFO:teuthology.orchestra.run.smithi096.stdout: "use_agent": { 2024-04-25T14:15:54.203 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "use_agent", 2024-04-25T14:15:54.203 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.203 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.203 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.203 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.203 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.204 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.204 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.204 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T14:15:54.204 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.204 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.204 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.204 INFO:teuthology.orchestra.run.smithi096.stdout: "use_repo_digest": { 2024-04-25T14:15:54.204 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "use_repo_digest", 2024-04-25T14:15:54.204 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.204 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.204 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.204 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.204 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.204 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.205 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.205 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T14:15:54.205 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.205 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.205 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.205 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_on_failed_host_check": { 2024-04-25T14:15:54.205 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_on_failed_host_check", 2024-04-25T14:15:54.205 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.205 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.205 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.205 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.205 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.206 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.206 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.206 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T14:15:54.206 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.206 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.206 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.206 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_on_stray_daemons": { 2024-04-25T14:15:54.206 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_on_stray_daemons", 2024-04-25T14:15:54.206 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.206 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.206 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.206 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.206 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.207 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.207 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.207 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T14:15:54.207 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.207 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.207 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.207 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_on_stray_hosts": { 2024-04-25T14:15:54.207 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_on_stray_hosts", 2024-04-25T14:15:54.207 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.207 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.207 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.207 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.207 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.208 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.209 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.209 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T14:15:54.209 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.209 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.209 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.210 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.210 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.210 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crash", 2024-04-25T14:15:54.210 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.210 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.210 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.210 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.210 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.210 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.210 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.210 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.210 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.211 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.211 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.211 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.211 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.211 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.211 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.211 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.211 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.211 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.211 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.211 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.211 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.211 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.211 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.212 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.212 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.212 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.212 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.212 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.212 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.212 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.212 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.212 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.212 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.212 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.212 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.213 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.213 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.213 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.213 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.213 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.213 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.213 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.213 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.213 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.213 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.213 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.213 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.213 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.214 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.214 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.214 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.214 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.214 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.214 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.214 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.214 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.214 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.214 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.214 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.215 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.215 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.215 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.215 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.215 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.215 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.215 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.215 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.215 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.215 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.215 INFO:teuthology.orchestra.run.smithi096.stdout: "retain_interval": { 2024-04-25T14:15:54.216 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "retain_interval", 2024-04-25T14:15:54.216 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:54.216 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.216 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.216 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "31536000", 2024-04-25T14:15:54.216 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.216 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.216 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.216 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T14:15:54.216 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.216 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.217 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.217 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.217 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.217 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.217 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.217 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.217 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.217 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.217 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.217 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.217 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.217 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.218 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.218 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.218 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_recent_interval": { 2024-04-25T14:15:54.218 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_recent_interval", 2024-04-25T14:15:54.218 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:54.218 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.218 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.218 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "1209600", 2024-04-25T14:15:54.218 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.218 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.218 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.218 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T14:15:54.218 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.219 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.219 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.219 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.219 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.219 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "dashboard", 2024-04-25T14:15:54.219 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.219 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.219 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.219 INFO:teuthology.orchestra.run.smithi096.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T14:15:54.219 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T14:15:54.219 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.219 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.220 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.220 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T14:15:54.220 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.220 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.220 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.220 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.220 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.220 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.220 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.220 INFO:teuthology.orchestra.run.smithi096.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T14:15:54.220 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T14:15:54.220 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.221 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.221 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.221 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.221 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.221 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.221 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.221 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.221 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.221 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.221 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.221 INFO:teuthology.orchestra.run.smithi096.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T14:15:54.221 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T14:15:54.221 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.221 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.222 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.222 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.222 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.222 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.222 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.222 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.222 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.222 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.222 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.222 INFO:teuthology.orchestra.run.smithi096.stdout: "AUDIT_API_ENABLED": { 2024-04-25T14:15:54.222 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T14:15:54.222 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.222 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.223 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.223 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.223 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.223 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.223 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.223 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.223 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.223 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.223 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.223 INFO:teuthology.orchestra.run.smithi096.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T14:15:54.223 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T14:15:54.223 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.223 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.223 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.224 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.224 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.224 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.224 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.224 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.224 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.224 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.224 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.224 INFO:teuthology.orchestra.run.smithi096.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T14:15:54.224 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T14:15:54.224 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.224 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.224 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.225 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.225 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.225 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.225 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.225 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.225 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.225 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.225 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.225 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T14:15:54.225 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T14:15:54.225 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.225 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.225 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.226 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.226 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.226 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.226 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.226 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.226 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.226 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.226 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.226 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T14:15:54.226 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T14:15:54.226 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.226 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.227 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.227 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.227 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.227 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.227 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.227 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.227 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.227 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.227 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.227 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T14:15:54.227 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T14:15:54.227 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.227 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.228 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.228 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.228 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.228 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.228 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.228 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.228 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.228 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.228 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.228 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T14:15:54.228 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T14:15:54.228 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.229 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.229 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.229 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.229 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.229 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.229 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.229 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.229 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.229 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.229 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.229 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T14:15:54.229 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T14:15:54.229 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.229 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.230 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.230 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.230 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.230 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.230 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.230 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.230 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.230 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.230 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.230 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T14:15:54.230 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T14:15:54.231 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.231 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.231 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.231 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.231 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.231 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.231 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.231 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.231 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.231 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.231 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.231 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T14:15:54.232 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T14:15:54.232 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.232 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.232 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.232 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.232 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.232 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.232 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.232 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.232 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.232 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.232 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.233 INFO:teuthology.orchestra.run.smithi096.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T14:15:54.233 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T14:15:54.233 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.233 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.233 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.233 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.233 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.233 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.233 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.233 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.233 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.234 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.234 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.234 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T14:15:54.234 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T14:15:54.234 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.234 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.234 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.234 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "admin", 2024-04-25T14:15:54.234 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.234 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.234 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.234 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.234 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.235 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.235 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.235 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T14:15:54.235 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T14:15:54.235 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.235 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.235 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.235 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.235 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.235 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.235 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.235 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.236 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.236 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.236 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.236 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_URL": { 2024-04-25T14:15:54.236 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_URL", 2024-04-25T14:15:54.236 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.236 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.236 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.236 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.236 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.236 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.237 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.237 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.237 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.237 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.237 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.237 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T14:15:54.237 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T14:15:54.237 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.237 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.237 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.237 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "admin", 2024-04-25T14:15:54.238 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.238 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.238 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.238 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.238 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.238 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.238 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.238 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T14:15:54.238 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T14:15:54.238 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.238 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.239 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.239 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.239 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.239 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.239 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.239 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.239 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.239 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.239 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.239 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T14:15:54.239 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T14:15:54.239 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.240 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.240 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.240 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.240 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.240 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.240 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.240 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.240 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.240 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.240 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.240 INFO:teuthology.orchestra.run.smithi096.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T14:15:54.241 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T14:15:54.241 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.241 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.241 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.241 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.241 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.241 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.241 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.241 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.241 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.241 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.241 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.241 INFO:teuthology.orchestra.run.smithi096.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T14:15:54.242 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T14:15:54.242 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.242 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.242 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.242 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.242 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.242 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.242 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.242 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.242 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.242 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.242 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.243 INFO:teuthology.orchestra.run.smithi096.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T14:15:54.243 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T14:15:54.243 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.243 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.243 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.243 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "[]", 2024-04-25T14:15:54.243 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.243 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.243 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.243 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.243 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.243 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.243 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.244 INFO:teuthology.orchestra.run.smithi096.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T14:15:54.244 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T14:15:54.244 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.244 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.244 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.244 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "{}", 2024-04-25T14:15:54.244 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.244 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.244 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.244 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.244 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.244 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.244 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.245 INFO:teuthology.orchestra.run.smithi096.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T14:15:54.245 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T14:15:54.245 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.245 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.245 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.245 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.245 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.245 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.245 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.245 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.245 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.245 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.245 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.246 INFO:teuthology.orchestra.run.smithi096.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T14:15:54.246 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T14:15:54.246 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.246 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.246 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.246 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.246 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.246 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.246 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.246 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.246 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.246 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.247 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.247 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T14:15:54.247 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T14:15:54.247 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.247 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.247 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.247 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.247 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.247 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.247 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.247 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.247 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.247 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.247 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.248 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T14:15:54.248 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T14:15:54.248 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.248 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.248 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.248 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.248 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.248 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.248 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.248 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.248 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.248 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.249 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.249 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T14:15:54.249 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T14:15:54.249 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.249 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.249 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.249 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.249 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.249 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.249 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.249 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.249 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.249 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.249 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.250 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T14:15:54.250 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T14:15:54.250 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.250 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.250 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.250 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.250 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.250 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.250 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.250 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.250 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.251 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.251 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.251 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T14:15:54.251 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T14:15:54.251 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.251 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.251 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.251 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.251 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.251 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.251 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.251 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.251 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.251 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.252 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.252 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T14:15:54.252 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T14:15:54.252 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.252 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.252 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.252 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.252 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.252 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.252 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.252 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.252 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.252 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.253 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.253 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T14:15:54.253 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T14:15:54.253 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.253 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.253 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.253 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.253 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.253 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.253 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.253 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.253 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.253 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.254 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.254 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T14:15:54.254 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T14:15:54.254 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.254 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.254 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.254 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.254 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.254 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.254 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.254 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.254 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.254 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.255 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.255 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T14:15:54.255 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T14:15:54.255 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.255 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.255 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.255 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T14:15:54.255 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.255 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.255 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.255 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.255 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.256 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.256 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.256 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T14:15:54.256 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T14:15:54.256 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.256 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.256 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.256 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T14:15:54.256 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.256 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.256 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.256 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.256 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.257 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.257 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.257 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T14:15:54.257 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T14:15:54.257 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.257 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.257 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.257 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8", 2024-04-25T14:15:54.257 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.257 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.257 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.257 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.257 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.258 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.258 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.258 INFO:teuthology.orchestra.run.smithi096.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T14:15:54.258 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T14:15:54.258 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.258 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.258 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.258 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "45", 2024-04-25T14:15:54.258 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.258 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.258 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.258 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.258 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.259 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.259 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.259 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T14:15:54.259 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T14:15:54.259 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.259 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.259 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.259 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.259 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.259 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.259 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.259 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.259 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.260 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.260 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.260 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T14:15:54.260 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T14:15:54.260 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.260 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.260 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.260 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "admin", 2024-04-25T14:15:54.260 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.260 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.260 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.260 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.261 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.261 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.261 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.261 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T14:15:54.261 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T14:15:54.261 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.261 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.261 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.261 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.261 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.261 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.261 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.261 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.262 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.262 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.262 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.262 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T14:15:54.262 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T14:15:54.262 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.262 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.262 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.262 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.262 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.262 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.262 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.263 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.263 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.263 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.263 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.263 INFO:teuthology.orchestra.run.smithi096.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T14:15:54.263 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T14:15:54.263 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.263 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.263 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.263 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.263 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.263 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.263 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.263 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.264 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.264 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.264 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.264 INFO:teuthology.orchestra.run.smithi096.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T14:15:54.264 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T14:15:54.264 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.264 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.264 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.264 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.264 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.264 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.264 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.264 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.265 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.265 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.265 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.265 INFO:teuthology.orchestra.run.smithi096.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T14:15:54.265 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T14:15:54.265 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.265 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.265 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.265 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T14:15:54.265 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.265 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.265 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.266 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.266 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.266 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.266 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.266 INFO:teuthology.orchestra.run.smithi096.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T14:15:54.266 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T14:15:54.266 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.266 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.266 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.266 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-04-25T14:15:54.266 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.266 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.267 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.267 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.267 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.267 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.267 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.267 INFO:teuthology.orchestra.run.smithi096.stdout: "cross_origin_url": { 2024-04-25T14:15:54.267 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cross_origin_url", 2024-04-25T14:15:54.267 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.267 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.267 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.267 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.268 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.268 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.268 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.268 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.268 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.268 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.268 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.268 INFO:teuthology.orchestra.run.smithi096.stdout: "crt_file": { 2024-04-25T14:15:54.268 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crt_file", 2024-04-25T14:15:54.268 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.268 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.268 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.268 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.269 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.269 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.269 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.269 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.269 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.269 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.269 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.269 INFO:teuthology.orchestra.run.smithi096.stdout: "debug": { 2024-04-25T14:15:54.269 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "debug", 2024-04-25T14:15:54.269 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.269 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.269 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.269 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.270 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.270 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.270 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.270 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Enable/disable debug options", 2024-04-25T14:15:54.270 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.270 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.270 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.270 INFO:teuthology.orchestra.run.smithi096.stdout: "jwt_token_ttl": { 2024-04-25T14:15:54.270 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "jwt_token_ttl", 2024-04-25T14:15:54.270 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.270 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.270 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.271 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "28800", 2024-04-25T14:15:54.271 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.271 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.271 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.271 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.271 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.271 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.271 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.271 INFO:teuthology.orchestra.run.smithi096.stdout: "key_file": { 2024-04-25T14:15:54.271 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "key_file", 2024-04-25T14:15:54.271 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.271 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.272 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.272 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.272 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.272 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.272 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.272 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.272 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.272 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.272 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.272 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.272 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.273 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.273 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.273 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.273 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.273 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.273 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.273 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.273 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.273 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.273 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.273 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.274 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.274 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.274 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.274 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.274 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.274 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.274 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.274 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.274 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.274 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.274 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.274 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.274 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.275 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.275 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.275 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.275 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.275 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.275 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.275 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.275 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.275 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.275 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.275 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.275 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.276 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.276 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.276 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.276 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.276 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.276 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.276 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.276 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.276 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.276 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.276 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.276 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.276 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.276 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.277 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.277 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.277 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.277 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.277 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.277 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.277 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.277 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.277 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.277 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.277 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.277 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.277 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.278 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.278 INFO:teuthology.orchestra.run.smithi096.stdout: "motd": { 2024-04-25T14:15:54.278 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "motd", 2024-04-25T14:15:54.278 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.278 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.278 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.278 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.278 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.278 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.278 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.278 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "The message of the day", 2024-04-25T14:15:54.278 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.278 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.279 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.279 INFO:teuthology.orchestra.run.smithi096.stdout: "redirect_resolve_ip_addr": { 2024-04-25T14:15:54.279 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T14:15:54.279 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.279 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.279 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.279 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.279 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.279 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.279 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.279 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.279 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.279 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.280 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.280 INFO:teuthology.orchestra.run.smithi096.stdout: "server_addr": { 2024-04-25T14:15:54.280 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_addr", 2024-04-25T14:15:54.280 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.280 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.280 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.280 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "::", 2024-04-25T14:15:54.280 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.280 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.280 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.280 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.280 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.280 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.280 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.281 INFO:teuthology.orchestra.run.smithi096.stdout: "server_port": { 2024-04-25T14:15:54.281 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_port", 2024-04-25T14:15:54.281 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.281 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.281 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.281 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8080", 2024-04-25T14:15:54.281 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.281 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.281 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.281 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.281 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.281 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.282 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.282 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.282 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.282 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.282 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.282 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.282 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.282 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.282 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.282 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.282 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.282 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.282 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.283 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.283 INFO:teuthology.orchestra.run.smithi096.stdout: "ssl": { 2024-04-25T14:15:54.283 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssl", 2024-04-25T14:15:54.283 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.283 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.283 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.283 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.283 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.283 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.283 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.283 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.283 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.283 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.284 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.284 INFO:teuthology.orchestra.run.smithi096.stdout: "ssl_server_port": { 2024-04-25T14:15:54.284 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssl_server_port", 2024-04-25T14:15:54.284 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.284 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.284 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.284 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8443", 2024-04-25T14:15:54.284 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.284 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.284 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.284 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.284 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.284 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.284 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.285 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_behaviour": { 2024-04-25T14:15:54.285 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_behaviour", 2024-04-25T14:15:54.285 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.285 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.285 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.285 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "redirect", 2024-04-25T14:15:54.285 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.285 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.285 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.285 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.285 INFO:teuthology.orchestra.run.smithi096.stdout: "redirect" 2024-04-25T14:15:54.285 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.285 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.286 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.286 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.286 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.286 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_error_status_code": { 2024-04-25T14:15:54.286 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_error_status_code", 2024-04-25T14:15:54.286 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.286 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.286 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.286 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "500", 2024-04-25T14:15:54.286 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "400", 2024-04-25T14:15:54.286 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "599", 2024-04-25T14:15:54.286 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.287 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.287 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.287 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.287 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.287 INFO:teuthology.orchestra.run.smithi096.stdout: "url_prefix": { 2024-04-25T14:15:54.287 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "url_prefix", 2024-04-25T14:15:54.287 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.287 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.287 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.287 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.287 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.287 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.288 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.288 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.288 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.288 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.288 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.288 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.288 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.288 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "devicehealth", 2024-04-25T14:15:54.288 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.288 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.288 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.288 INFO:teuthology.orchestra.run.smithi096.stdout: "enable_monitoring": { 2024-04-25T14:15:54.289 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enable_monitoring", 2024-04-25T14:15:54.289 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.289 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.289 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.289 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.289 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.289 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.289 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.289 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "monitor device health metrics", 2024-04-25T14:15:54.289 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.289 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.289 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.290 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.290 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.290 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.290 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.290 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.290 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.290 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.290 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.290 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.290 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.290 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.290 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.290 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.291 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.291 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.291 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.291 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.291 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.291 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.291 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.291 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.291 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.291 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.291 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.291 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.292 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.292 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.292 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.292 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.292 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.292 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.292 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.292 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.292 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.292 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.292 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.292 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.292 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.293 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.293 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.293 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.293 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.293 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.293 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.293 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.293 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.293 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.293 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.293 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.293 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.293 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.293 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.294 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.294 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.294 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.294 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.294 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.294 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.294 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.294 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.294 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.294 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.294 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.294 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.295 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.295 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.295 INFO:teuthology.orchestra.run.smithi096.stdout: "mark_out_threshold": { 2024-04-25T14:15:54.295 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mark_out_threshold", 2024-04-25T14:15:54.295 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:54.295 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.295 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.295 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "2419200", 2024-04-25T14:15:54.295 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.295 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.295 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.295 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T14:15:54.295 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.296 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.296 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.296 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_name": { 2024-04-25T14:15:54.296 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pool_name", 2024-04-25T14:15:54.296 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.296 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.296 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.296 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "device_health_metrics", 2024-04-25T14:15:54.296 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.296 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.296 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.296 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T14:15:54.296 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.297 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.297 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.297 INFO:teuthology.orchestra.run.smithi096.stdout: "retention_period": { 2024-04-25T14:15:54.297 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "retention_period", 2024-04-25T14:15:54.297 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:54.297 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.297 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.297 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "15552000", 2024-04-25T14:15:54.297 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.297 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.297 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.297 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how long to retain device health metrics", 2024-04-25T14:15:54.297 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.298 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.298 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.298 INFO:teuthology.orchestra.run.smithi096.stdout: "scrape_frequency": { 2024-04-25T14:15:54.298 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "scrape_frequency", 2024-04-25T14:15:54.298 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:54.298 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.298 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.298 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "86400", 2024-04-25T14:15:54.298 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.298 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.298 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.298 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T14:15:54.298 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.299 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.299 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.299 INFO:teuthology.orchestra.run.smithi096.stdout: "self_heal": { 2024-04-25T14:15:54.299 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "self_heal", 2024-04-25T14:15:54.299 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.299 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.299 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.299 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.299 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.299 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.299 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.300 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T14:15:54.300 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.300 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.300 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.300 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-04-25T14:15:54.300 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-04-25T14:15:54.300 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:54.300 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.300 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.300 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-04-25T14:15:54.300 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.300 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.300 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.301 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T14:15:54.301 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.301 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.301 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.301 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.301 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.301 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.301 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.301 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.301 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.301 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.301 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.301 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.302 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.302 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.302 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.302 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.302 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_threshold": { 2024-04-25T14:15:54.302 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_threshold", 2024-04-25T14:15:54.302 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:54.302 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.302 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.302 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "7257600", 2024-04-25T14:15:54.302 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.302 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.302 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.303 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T14:15:54.303 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.303 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.303 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.303 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.303 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.303 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "diskprediction_local", 2024-04-25T14:15:54.303 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.303 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.303 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.303 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.303 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.303 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.304 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.304 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.304 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.304 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.304 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.304 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.304 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.304 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.304 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.304 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.304 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.304 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.304 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.304 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.305 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.305 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.305 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.305 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.305 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.305 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.305 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.305 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.305 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.305 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.305 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.305 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.305 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.305 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.306 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.306 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.306 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.306 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.306 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.306 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.306 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.306 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.306 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.306 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.306 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.306 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.306 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.306 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.307 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.307 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.307 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.307 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.307 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.307 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.307 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.307 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.307 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.307 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.307 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.307 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.307 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.308 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.308 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.308 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.308 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.308 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.308 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.308 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.308 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.308 INFO:teuthology.orchestra.run.smithi096.stdout: "predict_interval": { 2024-04-25T14:15:54.308 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "predict_interval", 2024-04-25T14:15:54.309 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.309 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.309 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.309 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "86400", 2024-04-25T14:15:54.309 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.309 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.309 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.309 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.309 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.309 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.309 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.310 INFO:teuthology.orchestra.run.smithi096.stdout: "predictor_model": { 2024-04-25T14:15:54.310 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "predictor_model", 2024-04-25T14:15:54.310 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.310 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.310 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.310 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "prophetstor", 2024-04-25T14:15:54.310 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.310 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.310 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.310 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.310 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.310 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.310 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.311 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-04-25T14:15:54.311 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-04-25T14:15:54.311 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.311 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.311 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.311 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-04-25T14:15:54.311 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.311 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.311 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.311 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.311 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.311 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.311 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.311 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.312 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.312 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.312 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.312 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.312 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.312 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.312 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.312 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.312 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.312 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.312 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.312 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.312 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.312 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.313 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "influx", 2024-04-25T14:15:54.313 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": false, 2024-04-25T14:15:54.313 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "influxdb python module not found", 2024-04-25T14:15:54.313 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.313 INFO:teuthology.orchestra.run.smithi096.stdout: "batch_size": { 2024-04-25T14:15:54.313 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "batch_size", 2024-04-25T14:15:54.313 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.313 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.313 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.313 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5000", 2024-04-25T14:15:54.313 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.314 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.314 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.314 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T14:15:54.314 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.314 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.314 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.314 INFO:teuthology.orchestra.run.smithi096.stdout: "database": { 2024-04-25T14:15:54.314 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "database", 2024-04-25T14:15:54.314 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.314 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.314 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.314 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "ceph", 2024-04-25T14:15:54.314 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.315 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.315 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.315 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T14:15:54.315 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.315 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.315 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.315 INFO:teuthology.orchestra.run.smithi096.stdout: "hostname": { 2024-04-25T14:15:54.315 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "hostname", 2024-04-25T14:15:54.315 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.315 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.315 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.315 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.315 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.316 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.316 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.316 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "InfluxDB server hostname", 2024-04-25T14:15:54.316 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.316 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.316 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.316 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-04-25T14:15:54.316 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-04-25T14:15:54.316 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:54.316 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.316 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.316 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "30", 2024-04-25T14:15:54.316 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "5", 2024-04-25T14:15:54.317 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.317 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.317 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T14:15:54.317 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.317 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.317 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.317 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.317 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.317 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.317 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.317 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.317 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.317 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.317 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.318 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.318 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.318 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.318 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.318 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.318 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.318 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.318 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.318 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.318 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.318 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.318 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.319 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.319 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.319 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.319 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.319 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.319 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.319 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.319 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.319 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.319 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.319 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.319 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.319 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.320 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.320 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.320 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.320 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.320 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.320 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.320 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.320 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.320 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.320 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.320 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.320 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.320 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.320 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.321 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.321 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.321 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.321 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.321 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.321 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.321 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.321 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.321 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.321 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.321 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.321 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.321 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.321 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.322 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.322 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.322 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.322 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.322 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.322 INFO:teuthology.orchestra.run.smithi096.stdout: "password": { 2024-04-25T14:15:54.322 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "password", 2024-04-25T14:15:54.322 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.322 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.322 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.322 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.323 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.323 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.323 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.323 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "password of InfluxDB server user", 2024-04-25T14:15:54.323 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.323 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.323 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.323 INFO:teuthology.orchestra.run.smithi096.stdout: "port": { 2024-04-25T14:15:54.323 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "port", 2024-04-25T14:15:54.323 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.323 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.323 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.323 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8086", 2024-04-25T14:15:54.324 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.324 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.324 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.324 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "InfluxDB server port", 2024-04-25T14:15:54.324 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.324 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.324 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.324 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.324 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.324 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.324 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.325 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.325 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.325 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.325 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.325 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.325 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.325 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.325 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.325 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.325 INFO:teuthology.orchestra.run.smithi096.stdout: "ssl": { 2024-04-25T14:15:54.325 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssl", 2024-04-25T14:15:54.326 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.326 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.326 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.326 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "false", 2024-04-25T14:15:54.326 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.326 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.326 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.326 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T14:15:54.326 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.326 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.326 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.326 INFO:teuthology.orchestra.run.smithi096.stdout: "threads": { 2024-04-25T14:15:54.327 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "threads", 2024-04-25T14:15:54.327 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.327 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.327 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.327 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-04-25T14:15:54.327 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-04-25T14:15:54.327 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "32", 2024-04-25T14:15:54.327 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.327 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T14:15:54.327 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.327 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.327 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.327 INFO:teuthology.orchestra.run.smithi096.stdout: "username": { 2024-04-25T14:15:54.328 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "username", 2024-04-25T14:15:54.328 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.328 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.328 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.328 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.328 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.328 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.328 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.328 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "username of InfluxDB server user", 2024-04-25T14:15:54.328 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.328 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.328 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.328 INFO:teuthology.orchestra.run.smithi096.stdout: "verify_ssl": { 2024-04-25T14:15:54.329 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "verify_ssl", 2024-04-25T14:15:54.329 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.329 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.329 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.329 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "true", 2024-04-25T14:15:54.329 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.329 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.329 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.329 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T14:15:54.329 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.329 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.329 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.329 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.329 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.330 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "insights", 2024-04-25T14:15:54.330 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.330 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.330 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.330 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.330 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.330 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.330 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.330 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.330 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.330 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.330 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.330 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.331 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.331 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.331 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.331 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.331 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.331 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.331 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.331 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.331 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.331 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.331 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.331 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.331 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.332 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.332 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.332 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.332 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.332 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.332 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.332 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.332 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.332 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.332 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.332 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.332 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.332 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.332 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.333 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.333 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.333 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.333 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.333 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.333 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.333 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.333 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.333 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.333 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.333 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.333 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.333 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.333 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.334 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.334 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.334 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.334 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.334 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.334 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.334 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.334 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.334 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.334 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.334 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.334 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.334 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.334 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.335 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.335 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.335 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.335 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.335 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.335 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.335 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.335 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.335 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.335 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.335 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.335 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.335 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.336 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.336 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.336 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.336 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.336 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "iostat", 2024-04-25T14:15:54.336 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.336 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.336 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.336 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.336 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.336 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.337 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.337 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.337 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.337 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.337 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.337 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.337 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.337 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.337 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.337 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.337 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.337 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.337 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.337 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.338 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.338 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.338 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.338 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.338 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.338 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.338 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.338 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.338 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.338 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.338 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.338 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.338 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.339 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.339 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.339 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.339 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.339 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.339 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.339 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.339 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.339 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.339 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.339 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.339 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.339 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.339 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.340 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.340 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.340 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.340 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.340 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.340 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.340 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.340 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.340 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.340 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.340 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.340 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.340 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.341 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.341 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.341 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.341 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.341 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.341 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.341 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.341 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.341 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.341 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.341 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.341 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.342 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.342 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.342 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.342 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.342 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.342 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.342 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.342 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.342 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.342 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.342 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.342 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.343 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "k8sevents", 2024-04-25T14:15:54.343 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.343 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.343 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.343 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_event_retention_days": { 2024-04-25T14:15:54.343 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ceph_event_retention_days", 2024-04-25T14:15:54.343 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.343 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.343 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.343 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "7", 2024-04-25T14:15:54.343 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.343 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.343 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.343 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T14:15:54.344 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.344 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.344 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.344 INFO:teuthology.orchestra.run.smithi096.stdout: "config_check_secs": { 2024-04-25T14:15:54.344 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "config_check_secs", 2024-04-25T14:15:54.344 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.344 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.344 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.344 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T14:15:54.344 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "10", 2024-04-25T14:15:54.344 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.344 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.345 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T14:15:54.345 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.345 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.345 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.345 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.345 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.345 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.345 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.345 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.345 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.345 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.345 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.345 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.345 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.346 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.346 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.346 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.346 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.346 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.346 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.346 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.346 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.346 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.346 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.346 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.346 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.347 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.347 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.347 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.347 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.347 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.347 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.347 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.347 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.347 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.347 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.347 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.347 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.348 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.348 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.348 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.348 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.348 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.348 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.348 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.348 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.348 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.348 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.348 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.348 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.348 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.348 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.349 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.349 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.349 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.349 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.349 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.349 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.349 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.349 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.349 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.349 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.349 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.349 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.349 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.349 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.350 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.350 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.350 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.350 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.350 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.350 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.350 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.350 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.350 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.350 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.350 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.351 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.351 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.351 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.351 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.351 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.351 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.351 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.351 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.351 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "localpool", 2024-04-25T14:15:54.351 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.351 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.351 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.352 INFO:teuthology.orchestra.run.smithi096.stdout: "failure_domain": { 2024-04-25T14:15:54.352 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "failure_domain", 2024-04-25T14:15:54.352 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.352 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.352 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.352 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "host", 2024-04-25T14:15:54.352 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.352 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.352 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.352 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "failure domain for any created local pool", 2024-04-25T14:15:54.352 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T14:15:54.352 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.352 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.352 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.353 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.353 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.353 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.353 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.353 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.353 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.353 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.353 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.353 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.353 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.353 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.353 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.353 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.353 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.354 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.354 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.354 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.354 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.354 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.354 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.354 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.354 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.354 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.354 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.354 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.354 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.355 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.355 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.355 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.355 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.355 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.355 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.355 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.355 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.355 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.355 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.355 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.355 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.356 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.356 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.356 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.356 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.356 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.356 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.356 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.356 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.356 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.356 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.356 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.356 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.356 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.356 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.357 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.357 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.357 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.357 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.357 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.357 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.357 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.357 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.357 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.357 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.357 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.357 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.357 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.358 INFO:teuthology.orchestra.run.smithi096.stdout: "min_size": { 2024-04-25T14:15:54.358 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "min_size", 2024-04-25T14:15:54.358 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.358 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.358 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.358 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.358 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.358 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.358 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.358 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "default min_size for any created local pool", 2024-04-25T14:15:54.358 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T14:15:54.358 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.358 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.359 INFO:teuthology.orchestra.run.smithi096.stdout: "num_rep": { 2024-04-25T14:15:54.359 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "num_rep", 2024-04-25T14:15:54.359 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.359 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.359 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.359 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "3", 2024-04-25T14:15:54.359 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.359 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.359 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.359 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "default replica count for any created local pool", 2024-04-25T14:15:54.359 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.360 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.360 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.360 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num": { 2024-04-25T14:15:54.360 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pg_num", 2024-04-25T14:15:54.360 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.360 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.360 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.360 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "128", 2024-04-25T14:15:54.360 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.360 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.360 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.360 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T14:15:54.360 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.360 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.361 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.361 INFO:teuthology.orchestra.run.smithi096.stdout: "prefix": { 2024-04-25T14:15:54.361 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "prefix", 2024-04-25T14:15:54.361 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.361 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.361 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.361 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.361 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.361 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.361 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.361 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "name prefix for any created local pool", 2024-04-25T14:15:54.361 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.361 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.362 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.362 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.362 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.362 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.362 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.362 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.362 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.362 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.362 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.362 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.362 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.362 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.362 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.362 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.363 INFO:teuthology.orchestra.run.smithi096.stdout: "subtree": { 2024-04-25T14:15:54.363 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "subtree", 2024-04-25T14:15:54.363 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.363 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.363 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.363 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "rack", 2024-04-25T14:15:54.363 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.363 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.363 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.363 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T14:15:54.363 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T14:15:54.364 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.364 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.364 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.364 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.364 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mds_autoscaler", 2024-04-25T14:15:54.364 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.364 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.364 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.364 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.364 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.364 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.365 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.365 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.365 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.365 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.365 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.365 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.365 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.365 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.365 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.365 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.365 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.365 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.365 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.365 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.366 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.366 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.366 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.366 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.366 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.366 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.366 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.366 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.366 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.366 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.366 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.366 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.367 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.367 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.367 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.367 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.367 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.367 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.367 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.367 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.367 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.367 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.367 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.367 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.367 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.368 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.368 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.368 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.368 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.368 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.368 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.368 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.368 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.368 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.368 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.368 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.368 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.369 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.369 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.369 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.369 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.369 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.369 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.369 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.369 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.369 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.369 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.369 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.369 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.369 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.370 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.370 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.370 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.370 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.370 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.370 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.370 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.370 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.370 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.370 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.370 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.370 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.370 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.370 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.371 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mirroring", 2024-04-25T14:15:54.371 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.371 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.371 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.371 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.371 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.371 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.371 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.371 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.371 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.371 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.371 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.372 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.372 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.372 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.372 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.372 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.372 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.372 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.372 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.372 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.372 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.372 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.372 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.373 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.373 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.373 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.373 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.373 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.373 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.373 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.373 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.373 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.373 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.373 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.373 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.373 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.374 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.374 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.374 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.374 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.374 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.374 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.374 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.374 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.374 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.374 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.374 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.374 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.375 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.375 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.375 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.375 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.375 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.375 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.375 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.375 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.375 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.375 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.375 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.375 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.376 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.376 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.376 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.376 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.376 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.376 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.376 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.376 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.376 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.376 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.376 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.376 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.377 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.377 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.377 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.377 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.377 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.377 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.377 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.377 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.377 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.377 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.377 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.377 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.377 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "nfs", 2024-04-25T14:15:54.378 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.378 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.378 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.378 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.378 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.378 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.378 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.378 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.378 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.378 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.378 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.378 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.378 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.379 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.379 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.379 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.379 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.379 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.379 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.379 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.379 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.379 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.379 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.379 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.379 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.379 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.380 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.380 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.380 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.380 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.380 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.380 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.380 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.380 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.380 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.380 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.380 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.380 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.380 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.381 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.381 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.381 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.381 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.381 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.381 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.381 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.381 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.381 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.381 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.381 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.381 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.381 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.381 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.381 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.382 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.382 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.382 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.382 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.382 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.382 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.382 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.382 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.382 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.382 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.382 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.382 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.382 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.383 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.383 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.383 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.383 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.383 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.383 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.383 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.383 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.383 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.383 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.383 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.383 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.384 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.384 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.384 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.384 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.384 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.384 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "orchestrator", 2024-04-25T14:15:54.384 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.384 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.384 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.384 INFO:teuthology.orchestra.run.smithi096.stdout: "fail_fs": { 2024-04-25T14:15:54.384 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "fail_fs", 2024-04-25T14:15:54.384 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.385 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.385 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.385 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.385 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.385 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.385 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.385 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T14:15:54.385 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.385 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.385 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.385 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.385 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.385 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.385 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.386 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.386 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.386 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.386 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.386 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.386 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.386 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.386 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.386 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.386 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.386 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.386 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.386 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.386 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.386 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.387 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.387 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.387 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.387 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.387 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.387 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.387 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.387 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.387 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.387 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.387 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.387 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.388 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.388 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.388 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.388 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.388 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.388 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.388 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.388 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.388 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.388 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.388 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.388 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.388 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.388 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.389 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.389 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.389 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.389 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.389 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.389 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.389 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.389 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.389 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.389 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.389 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.389 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.389 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.390 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.390 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.390 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.390 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.390 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.390 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.390 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.390 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.390 INFO:teuthology.orchestra.run.smithi096.stdout: "orchestrator": { 2024-04-25T14:15:54.390 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "orchestrator", 2024-04-25T14:15:54.390 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.391 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.391 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.391 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.391 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.391 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.391 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.391 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm", 2024-04-25T14:15:54.391 INFO:teuthology.orchestra.run.smithi096.stdout: "rook", 2024-04-25T14:15:54.391 INFO:teuthology.orchestra.run.smithi096.stdout: "test_orchestrator" 2024-04-25T14:15:54.391 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.391 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Orchestrator backend", 2024-04-25T14:15:54.391 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.392 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.392 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.392 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.392 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.392 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.392 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.392 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.392 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.392 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.392 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.392 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.393 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.393 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.393 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.393 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.393 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.393 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.393 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "osd_perf_query", 2024-04-25T14:15:54.393 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.393 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.393 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.394 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.394 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.394 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.394 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.394 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.394 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.394 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.394 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.394 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.394 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.395 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.395 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.395 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.395 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.395 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.395 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.395 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.395 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.395 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.395 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.396 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.396 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.396 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.396 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.396 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.396 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.396 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.396 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.396 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.396 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.396 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.396 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.396 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.397 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.397 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.397 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.397 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.397 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.397 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.397 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.397 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.397 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.397 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.397 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.397 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.397 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.398 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.398 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.398 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.398 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.398 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.398 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.398 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.398 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.398 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.398 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.398 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.398 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.399 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.399 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.399 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.399 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.399 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.399 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.399 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.399 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.399 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.399 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.399 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.399 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.399 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.399 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.400 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.400 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.400 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.400 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.400 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.400 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.400 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.400 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.400 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.400 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "osd_support", 2024-04-25T14:15:54.401 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.401 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.401 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.401 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.401 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.401 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.401 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.401 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.401 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.401 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.401 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.401 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.401 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.401 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.401 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.402 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.402 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.402 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.402 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.402 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.402 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.402 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.402 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.402 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.402 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.402 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.402 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.403 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.403 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.403 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.403 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.403 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.403 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.403 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.403 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.403 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.403 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.403 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.403 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.403 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.404 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.404 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.404 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.404 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.404 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.404 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.404 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.404 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.404 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.404 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.404 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.404 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.404 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.404 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.405 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.405 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.405 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.405 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.405 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.405 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.405 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.405 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.405 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.405 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.405 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.405 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.405 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.406 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.406 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.406 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.406 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.406 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.406 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.406 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.406 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.406 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.406 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.406 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.406 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.407 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.407 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.407 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.407 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.407 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.407 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pg_autoscaler", 2024-04-25T14:15:54.407 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.407 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.407 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.407 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.407 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.407 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.408 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.408 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.408 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.408 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.408 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.408 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.408 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.408 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.408 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.408 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.408 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.408 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.408 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.409 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.409 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.409 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.409 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.409 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.409 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.409 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.409 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.409 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.409 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.409 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.409 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.409 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.410 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.410 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.410 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.410 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.410 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.410 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.410 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.410 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.410 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.410 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.410 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.410 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.410 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.411 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.411 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.411 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.411 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.411 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.411 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.411 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.411 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.411 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.411 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.411 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.411 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.412 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.412 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.412 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.412 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.412 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.413 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.413 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.413 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.414 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.414 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.414 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.414 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.414 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-04-25T14:15:54.414 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-04-25T14:15:54.414 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:54.414 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.414 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.414 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-04-25T14:15:54.414 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.414 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.415 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.415 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.415 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.415 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.415 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.415 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.415 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.415 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.415 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.415 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.415 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.415 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.416 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.416 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.416 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.416 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.416 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.416 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.416 INFO:teuthology.orchestra.run.smithi096.stdout: "threshold": { 2024-04-25T14:15:54.416 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "threshold", 2024-04-25T14:15:54.416 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T14:15:54.416 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.416 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.417 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "3.0", 2024-04-25T14:15:54.417 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1.0", 2024-04-25T14:15:54.417 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.417 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.417 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "scaling threshold", 2024-04-25T14:15:54.417 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T14:15:54.417 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.417 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.417 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.417 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.417 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "progress", 2024-04-25T14:15:54.417 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.418 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.418 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.418 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_pg_recovery_event": { 2024-04-25T14:15:54.418 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "allow_pg_recovery_event", 2024-04-25T14:15:54.418 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.418 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.418 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.418 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.418 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.418 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.418 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.418 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T14:15:54.418 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.419 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.419 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.419 INFO:teuthology.orchestra.run.smithi096.stdout: "enabled": { 2024-04-25T14:15:54.419 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enabled", 2024-04-25T14:15:54.419 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.419 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.419 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.419 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.419 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.419 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.419 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.419 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.419 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.419 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.420 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.420 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.420 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.420 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.420 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.420 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.420 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.420 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.420 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.420 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.420 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.420 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.420 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.421 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.421 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.421 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.421 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.421 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.421 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.421 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.421 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.421 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.421 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.421 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.421 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.421 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.422 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.422 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.422 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.422 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.422 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.422 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.422 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.422 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.422 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.422 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.422 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.422 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.422 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.423 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.423 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.423 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.423 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.423 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.423 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.423 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.423 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.423 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.423 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.423 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.423 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.423 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.424 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.424 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.424 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.424 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.424 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.424 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.424 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.424 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.424 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.424 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.424 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.424 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.424 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.425 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.425 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.425 INFO:teuthology.orchestra.run.smithi096.stdout: "max_completed_events": { 2024-04-25T14:15:54.425 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_completed_events", 2024-04-25T14:15:54.425 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.425 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.425 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.425 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "50", 2024-04-25T14:15:54.425 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.425 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.425 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.425 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "number of past completed events to remember", 2024-04-25T14:15:54.425 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.426 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.426 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.426 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-04-25T14:15:54.426 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-04-25T14:15:54.426 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:54.426 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.426 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.426 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-04-25T14:15:54.426 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.426 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.426 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.426 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how long the module is going to sleep", 2024-04-25T14:15:54.426 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.427 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.427 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.427 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.427 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.427 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.427 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.427 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.427 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.427 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.427 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.427 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.427 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.427 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.428 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.428 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.428 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.428 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.428 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "prometheus", 2024-04-25T14:15:54.428 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.428 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.428 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.428 INFO:teuthology.orchestra.run.smithi096.stdout: "cache": { 2024-04-25T14:15:54.428 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cache", 2024-04-25T14:15:54.428 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.429 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.429 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.429 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.429 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.429 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.429 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.429 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.429 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.429 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.429 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.429 INFO:teuthology.orchestra.run.smithi096.stdout: "exclude_perf_counters": { 2024-04-25T14:15:54.429 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "exclude_perf_counters", 2024-04-25T14:15:54.429 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.430 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.430 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.430 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.430 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.430 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.430 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.430 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T14:15:54.430 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T14:15:54.430 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.430 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.430 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.430 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.431 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.431 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.431 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.431 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.431 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.431 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.431 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.431 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.431 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.431 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.431 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.431 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.431 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.431 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.432 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.432 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.432 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.432 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.432 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.432 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.432 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.432 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.432 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.432 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.432 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.432 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.432 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.433 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.433 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.433 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.433 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.433 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.433 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.433 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.433 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.433 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.433 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.433 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.433 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.434 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.434 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.434 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.434 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.434 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.434 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.434 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.434 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.434 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.434 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.434 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.434 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.434 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.435 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.435 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.435 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.435 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.435 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.435 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.435 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.435 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.435 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.435 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.435 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.435 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.435 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_stats_pools": { 2024-04-25T14:15:54.436 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_stats_pools", 2024-04-25T14:15:54.436 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.436 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.436 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.436 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.436 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.436 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.436 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.436 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.436 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.436 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.436 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.437 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T14:15:54.437 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T14:15:54.437 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.437 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.437 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.437 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "300", 2024-04-25T14:15:54.437 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.437 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.437 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.437 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.437 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.437 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.437 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.438 INFO:teuthology.orchestra.run.smithi096.stdout: "scrape_interval": { 2024-04-25T14:15:54.438 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "scrape_interval", 2024-04-25T14:15:54.438 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T14:15:54.438 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.438 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.438 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "15.0", 2024-04-25T14:15:54.438 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.438 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.438 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.438 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.438 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.438 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.438 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.439 INFO:teuthology.orchestra.run.smithi096.stdout: "server_addr": { 2024-04-25T14:15:54.439 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_addr", 2024-04-25T14:15:54.439 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.439 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.439 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.439 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "::", 2024-04-25T14:15:54.439 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.439 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.439 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.439 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T14:15:54.439 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.439 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.439 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.440 INFO:teuthology.orchestra.run.smithi096.stdout: "server_port": { 2024-04-25T14:15:54.440 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_port", 2024-04-25T14:15:54.440 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.440 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.440 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.440 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "9283", 2024-04-25T14:15:54.440 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.440 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.440 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.440 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T14:15:54.440 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.440 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.440 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.441 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.441 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.441 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.441 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.441 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.441 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.441 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.441 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.441 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.441 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.441 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.441 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.441 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.442 INFO:teuthology.orchestra.run.smithi096.stdout: "stale_cache_strategy": { 2024-04-25T14:15:54.442 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "stale_cache_strategy", 2024-04-25T14:15:54.442 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.442 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.442 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.442 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "log", 2024-04-25T14:15:54.442 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.442 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.442 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.442 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.442 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.442 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.442 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.443 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_behaviour": { 2024-04-25T14:15:54.443 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_behaviour", 2024-04-25T14:15:54.443 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.443 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.443 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.443 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "default", 2024-04-25T14:15:54.443 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.443 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.443 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.443 INFO:teuthology.orchestra.run.smithi096.stdout: "default", 2024-04-25T14:15:54.443 INFO:teuthology.orchestra.run.smithi096.stdout: "error" 2024-04-25T14:15:54.443 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.443 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.443 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.444 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.444 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.444 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_error_status_code": { 2024-04-25T14:15:54.444 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_error_status_code", 2024-04-25T14:15:54.444 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.444 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.444 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.444 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "500", 2024-04-25T14:15:54.444 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "400", 2024-04-25T14:15:54.444 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "599", 2024-04-25T14:15:54.444 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.444 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.444 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.444 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.445 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.445 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.445 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.445 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_support", 2024-04-25T14:15:54.445 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.445 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.445 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.445 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.445 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.445 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.445 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.445 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.446 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.446 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.446 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.446 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.446 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.446 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.446 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.446 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.446 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.446 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.446 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.446 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.446 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.447 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.447 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.447 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.447 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.447 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.447 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.447 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.447 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.447 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.447 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.447 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.447 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.447 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.448 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.448 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.448 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.448 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.448 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.448 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.448 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.448 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.448 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.448 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.448 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.448 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.448 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.448 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.449 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.449 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.449 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.449 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.449 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.449 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.449 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.449 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.449 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.449 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.449 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.449 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.449 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.449 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.450 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.450 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.450 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.450 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.450 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.450 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.450 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.450 INFO:teuthology.orchestra.run.smithi096.stdout: "max_concurrent_snap_create": { 2024-04-25T14:15:54.450 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_concurrent_snap_create", 2024-04-25T14:15:54.450 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.451 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.451 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.451 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T14:15:54.451 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.451 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.451 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.451 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.451 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.451 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.451 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.451 INFO:teuthology.orchestra.run.smithi096.stdout: "mirror_snapshot_schedule": { 2024-04-25T14:15:54.451 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T14:15:54.451 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.451 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.452 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.452 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.452 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.452 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.452 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.452 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.452 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.452 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.452 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.452 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.452 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.452 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.452 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.453 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.453 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.453 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.453 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.453 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.453 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.453 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.453 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.453 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.453 INFO:teuthology.orchestra.run.smithi096.stdout: "trash_purge_schedule": { 2024-04-25T14:15:54.453 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "trash_purge_schedule", 2024-04-25T14:15:54.453 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.453 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.454 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.454 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.454 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.454 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.454 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.454 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.454 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.454 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.454 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.454 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.454 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.454 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "restful", 2024-04-25T14:15:54.455 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.455 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.455 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.455 INFO:teuthology.orchestra.run.smithi096.stdout: "enable_auth": { 2024-04-25T14:15:54.455 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enable_auth", 2024-04-25T14:15:54.455 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.455 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.455 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.455 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.455 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.455 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.455 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.455 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.455 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.456 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.456 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.456 INFO:teuthology.orchestra.run.smithi096.stdout: "key_file": { 2024-04-25T14:15:54.456 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "key_file", 2024-04-25T14:15:54.456 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.456 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.456 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.456 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.456 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.456 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.456 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.456 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.456 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.457 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.457 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.457 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.457 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.457 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.457 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.457 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.457 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.457 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.457 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.457 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.457 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.457 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.458 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.458 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.458 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.458 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.458 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.458 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.458 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.458 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.458 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.458 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.458 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.458 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.458 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.459 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.459 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.459 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.459 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.459 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.459 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.459 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.459 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.459 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.459 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.459 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.459 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.460 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.460 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.460 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.460 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.460 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.460 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.460 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.460 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.460 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.460 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.460 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.460 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.460 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.460 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.461 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.461 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.461 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.461 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.461 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.461 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.461 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.461 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.461 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.461 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.461 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.461 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.461 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.462 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.462 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.462 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.462 INFO:teuthology.orchestra.run.smithi096.stdout: "server_addr": { 2024-04-25T14:15:54.462 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_addr", 2024-04-25T14:15:54.462 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.462 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.462 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.462 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.462 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.462 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.462 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.462 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.463 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.463 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.463 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.463 INFO:teuthology.orchestra.run.smithi096.stdout: "server_port": { 2024-04-25T14:15:54.463 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_port", 2024-04-25T14:15:54.463 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.463 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.463 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.463 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.463 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.463 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.463 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.463 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.464 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.464 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.464 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.464 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.464 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.464 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.464 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.464 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.464 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.464 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.464 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.464 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.464 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.465 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.465 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.465 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.465 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.465 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.465 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rgw", 2024-04-25T14:15:54.465 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.465 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.465 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.465 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.465 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.466 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.466 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.466 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.466 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.466 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.466 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.466 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.466 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.466 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.466 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.466 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.466 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.466 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.467 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.467 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.467 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.467 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.467 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.467 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.467 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.467 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.467 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.467 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.467 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.467 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.468 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.468 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.468 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.468 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.468 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.468 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.468 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.468 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.468 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.468 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.468 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.468 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.468 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.469 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.469 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.469 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.469 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.469 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.469 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.469 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.469 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.469 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.469 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.469 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.469 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.470 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.470 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.470 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.470 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.470 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.470 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.470 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.470 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.470 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.470 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.470 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.470 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.471 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.471 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.471 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.471 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.471 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.471 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.471 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.471 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.471 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.471 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.471 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.471 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.471 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.472 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.472 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.472 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.472 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.472 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rook", 2024-04-25T14:15:54.472 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.472 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.472 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.472 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.472 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.472 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.473 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.473 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.473 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.473 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.473 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.473 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.473 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.473 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.473 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.473 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.473 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.473 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.473 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.474 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.474 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.474 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.474 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.474 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.474 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.474 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.474 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.474 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.474 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.474 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.474 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.474 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.475 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.475 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.475 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.475 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.475 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.475 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.475 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.475 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.475 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.475 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.475 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.475 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.475 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.475 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.476 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.476 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.476 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.476 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.476 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.476 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.476 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.476 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.476 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.476 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.476 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.476 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.476 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.477 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.477 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.477 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.477 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.477 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.477 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.477 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.477 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.477 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.477 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.477 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.477 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.477 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.478 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.478 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.478 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.478 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.478 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.478 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.478 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.478 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.478 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.478 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.478 INFO:teuthology.orchestra.run.smithi096.stdout: "storage_class": { 2024-04-25T14:15:54.478 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "storage_class", 2024-04-25T14:15:54.478 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.479 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.479 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.479 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "local", 2024-04-25T14:15:54.479 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.479 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.479 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.479 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T14:15:54.479 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.479 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.479 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.479 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.479 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.480 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "selftest", 2024-04-25T14:15:54.480 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.480 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.480 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.480 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.480 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.480 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.480 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.480 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.480 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.480 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.480 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.480 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.481 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.481 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.481 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.481 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.481 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.481 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.481 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.481 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.481 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.481 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.481 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.481 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.481 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.482 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.482 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.482 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.482 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.482 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.482 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.482 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.482 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.482 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.482 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.482 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.482 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.482 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.483 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.483 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.483 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.483 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.483 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.483 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.483 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.483 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.483 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.483 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.483 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.483 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.483 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.483 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.484 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.484 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.484 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.484 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.484 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.484 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.484 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.484 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.484 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.484 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.484 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.484 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.484 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.485 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.485 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.485 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.485 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.485 INFO:teuthology.orchestra.run.smithi096.stdout: "roption1": { 2024-04-25T14:15:54.485 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "roption1", 2024-04-25T14:15:54.485 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.485 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.485 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.485 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.485 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.485 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.485 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.485 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.486 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.486 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.486 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.486 INFO:teuthology.orchestra.run.smithi096.stdout: "roption2": { 2024-04-25T14:15:54.486 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "roption2", 2024-04-25T14:15:54.486 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.486 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.486 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.486 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "xyz", 2024-04-25T14:15:54.486 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.486 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.486 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.486 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.487 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.487 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.487 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.487 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption1": { 2024-04-25T14:15:54.487 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption1", 2024-04-25T14:15:54.487 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.487 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.487 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.487 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.487 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.487 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.488 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.488 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.488 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.488 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.488 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.488 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption2": { 2024-04-25T14:15:54.488 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption2", 2024-04-25T14:15:54.488 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.488 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.488 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.488 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.488 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.488 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.488 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.489 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.489 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.489 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.489 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.489 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption3": { 2024-04-25T14:15:54.489 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption3", 2024-04-25T14:15:54.489 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T14:15:54.489 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.489 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.489 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.489 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.489 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.489 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.489 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.490 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.490 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.490 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.490 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption4": { 2024-04-25T14:15:54.490 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption4", 2024-04-25T14:15:54.490 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.490 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.490 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.490 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.490 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.490 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.490 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.490 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.491 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.491 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.491 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.491 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption5": { 2024-04-25T14:15:54.491 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption5", 2024-04-25T14:15:54.491 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.491 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.491 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.491 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.491 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.491 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.491 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.491 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.492 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.492 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.492 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.492 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption6": { 2024-04-25T14:15:54.492 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption6", 2024-04-25T14:15:54.492 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.492 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.492 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.492 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.492 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.492 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.492 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.492 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.493 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.493 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.493 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.493 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption7": { 2024-04-25T14:15:54.493 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption7", 2024-04-25T14:15:54.493 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.493 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.493 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.493 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.493 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-04-25T14:15:54.493 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "42", 2024-04-25T14:15:54.493 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.494 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.494 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.494 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.494 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.494 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.494 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.494 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.494 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.494 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.494 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.494 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.494 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.495 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.495 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.495 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.495 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.495 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.495 INFO:teuthology.orchestra.run.smithi096.stdout: "testkey": { 2024-04-25T14:15:54.495 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "testkey", 2024-04-25T14:15:54.495 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.495 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.495 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.495 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.496 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.496 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.496 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.496 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.496 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.496 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.496 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.496 INFO:teuthology.orchestra.run.smithi096.stdout: "testlkey": { 2024-04-25T14:15:54.496 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "testlkey", 2024-04-25T14:15:54.496 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.496 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.496 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.496 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.497 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.497 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.497 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.497 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.497 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.497 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.497 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.497 INFO:teuthology.orchestra.run.smithi096.stdout: "testnewline": { 2024-04-25T14:15:54.497 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "testnewline", 2024-04-25T14:15:54.497 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.497 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.497 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.497 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.497 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.498 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.498 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.498 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.498 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.498 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.498 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.498 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.498 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.498 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snap_schedule", 2024-04-25T14:15:54.498 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.499 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.499 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.499 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_m_granularity": { 2024-04-25T14:15:54.499 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "allow_m_granularity", 2024-04-25T14:15:54.499 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.499 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.499 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.499 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.499 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.499 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.499 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.499 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T14:15:54.499 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.499 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.500 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.500 INFO:teuthology.orchestra.run.smithi096.stdout: "dump_on_update": { 2024-04-25T14:15:54.500 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "dump_on_update", 2024-04-25T14:15:54.500 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.500 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.500 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.500 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.500 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.500 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.500 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.500 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "dump database to debug log on update", 2024-04-25T14:15:54.500 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.500 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.500 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.501 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.501 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.501 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.501 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.501 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.501 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.501 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.501 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.501 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.501 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.501 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.501 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.501 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.502 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.502 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.502 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.502 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.502 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.502 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.502 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.502 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.502 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.502 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.502 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.502 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.502 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.503 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.503 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.503 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.503 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.503 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.503 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.503 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.503 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.503 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.503 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.503 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.504 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.504 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.504 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.504 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.504 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.504 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.504 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.504 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.504 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.504 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.504 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.504 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.504 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.504 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.505 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.505 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.505 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.505 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.505 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.505 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.505 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.505 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.505 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.505 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.505 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.505 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.505 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.506 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.506 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.506 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.506 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.506 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.506 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.506 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.506 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.506 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.506 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.506 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.506 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.507 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.507 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.507 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.507 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.507 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.507 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "stats", 2024-04-25T14:15:54.507 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.507 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.507 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.507 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.507 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.507 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.508 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.508 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.508 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.508 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.508 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.508 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.508 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.508 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.508 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.508 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.508 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.508 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.508 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.509 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.509 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.509 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.509 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.509 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.509 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.509 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.509 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.509 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.509 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.509 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.509 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.509 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.509 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.510 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.510 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.510 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.510 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.510 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.510 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.510 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.510 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.510 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.510 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.510 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.510 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.511 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.511 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.511 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.511 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.511 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.511 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.511 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.511 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.511 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.511 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.511 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.511 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.511 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.512 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.512 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.512 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.512 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.512 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.512 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.512 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.512 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.512 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.512 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.512 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.512 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.512 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.513 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.513 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.513 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.513 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.513 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.513 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.513 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.513 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.513 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.513 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.513 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.513 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.513 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.514 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "status", 2024-04-25T14:15:54.514 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.514 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.514 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.514 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.514 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.514 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.514 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.514 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.514 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.514 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.514 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.514 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.514 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.515 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.515 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.515 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.515 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.515 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.515 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.515 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.515 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.515 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.515 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.515 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.515 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.515 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.516 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.516 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.516 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.516 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.516 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.516 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.516 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.516 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.516 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.516 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.516 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.516 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.517 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.517 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.517 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.517 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.517 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.517 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.517 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.517 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.517 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.517 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.517 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.517 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.517 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.517 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.518 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.518 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.518 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.518 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.518 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.518 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.518 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.518 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.518 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.518 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.518 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.518 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.519 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.519 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.519 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.519 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.519 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.519 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.519 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.519 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.519 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.519 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.519 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.519 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.519 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.520 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.520 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.520 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.520 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.520 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.520 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.520 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.520 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "telegraf", 2024-04-25T14:15:54.520 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.520 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.520 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.521 INFO:teuthology.orchestra.run.smithi096.stdout: "address": { 2024-04-25T14:15:54.521 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "address", 2024-04-25T14:15:54.521 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.521 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.521 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.521 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T14:15:54.521 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.521 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.521 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.521 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.521 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.521 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.521 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.522 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-04-25T14:15:54.522 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-04-25T14:15:54.522 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:54.522 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.522 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.522 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "15", 2024-04-25T14:15:54.522 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.522 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.522 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.522 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.522 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.522 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.522 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.523 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.523 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.523 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.523 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.523 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.523 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.523 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.523 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.523 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.523 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.523 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.523 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.523 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.523 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.524 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.524 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.524 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.524 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.524 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.524 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.524 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.524 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.524 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.524 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.524 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.524 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.524 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.524 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.525 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.525 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.525 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.525 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.525 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.525 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.525 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.525 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.525 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.525 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.525 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.525 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.525 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.526 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.526 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.526 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.526 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.526 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.526 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.526 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.526 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.526 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.526 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.526 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.526 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.526 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.527 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.527 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.527 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.527 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.527 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.527 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.527 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.527 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.527 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.527 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.527 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.527 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.527 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.528 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.528 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.528 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.528 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.528 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.528 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.528 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.528 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.528 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.528 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.528 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.528 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.528 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.528 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.529 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "telemetry", 2024-04-25T14:15:54.529 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.529 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.529 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.529 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_basic": { 2024-04-25T14:15:54.529 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_basic", 2024-04-25T14:15:54.529 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.529 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.529 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.529 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.529 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.529 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.529 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.529 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T14:15:54.530 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.530 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.530 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.530 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_crash": { 2024-04-25T14:15:54.530 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_crash", 2024-04-25T14:15:54.530 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.530 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.530 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.530 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.530 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.530 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.530 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.530 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T14:15:54.531 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.531 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.531 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.531 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_device": { 2024-04-25T14:15:54.531 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_device", 2024-04-25T14:15:54.531 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.531 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.531 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.531 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.531 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.531 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.531 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.531 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T14:15:54.531 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.532 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.532 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.532 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_ident": { 2024-04-25T14:15:54.532 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_ident", 2024-04-25T14:15:54.532 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.532 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.532 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.532 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.532 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.532 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.532 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.532 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T14:15:54.533 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.533 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.533 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.533 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_perf": { 2024-04-25T14:15:54.533 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_perf", 2024-04-25T14:15:54.533 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.533 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.533 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.533 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.533 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.533 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.533 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.533 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T14:15:54.534 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.534 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.534 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.534 INFO:teuthology.orchestra.run.smithi096.stdout: "contact": { 2024-04-25T14:15:54.534 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "contact", 2024-04-25T14:15:54.534 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.534 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.534 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.534 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.534 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.534 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.534 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.535 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.535 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.535 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.535 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.535 INFO:teuthology.orchestra.run.smithi096.stdout: "description": { 2024-04-25T14:15:54.535 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "description", 2024-04-25T14:15:54.535 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.535 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.535 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.535 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.535 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.536 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.536 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.536 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.536 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.536 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.536 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.536 INFO:teuthology.orchestra.run.smithi096.stdout: "device_url": { 2024-04-25T14:15:54.536 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "device_url", 2024-04-25T14:15:54.536 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.536 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.536 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.536 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T14:15:54.536 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.537 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.537 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.537 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.537 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.537 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.537 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.537 INFO:teuthology.orchestra.run.smithi096.stdout: "enabled": { 2024-04-25T14:15:54.537 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enabled", 2024-04-25T14:15:54.537 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.537 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.537 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.537 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.537 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.537 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.538 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.538 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.538 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.538 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.538 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.538 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-04-25T14:15:54.538 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-04-25T14:15:54.538 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.538 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.538 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.538 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "24", 2024-04-25T14:15:54.538 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "8", 2024-04-25T14:15:54.538 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.539 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.539 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.539 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.539 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.539 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.539 INFO:teuthology.orchestra.run.smithi096.stdout: "last_opt_revision": { 2024-04-25T14:15:54.539 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "last_opt_revision", 2024-04-25T14:15:54.539 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.539 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.539 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.539 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "1", 2024-04-25T14:15:54.539 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.540 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.540 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.540 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.540 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.540 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.540 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.540 INFO:teuthology.orchestra.run.smithi096.stdout: "leaderboard": { 2024-04-25T14:15:54.540 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "leaderboard", 2024-04-25T14:15:54.540 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.540 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.540 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.540 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.541 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.541 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.541 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.541 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.541 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.541 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.541 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.541 INFO:teuthology.orchestra.run.smithi096.stdout: "leaderboard_description": { 2024-04-25T14:15:54.541 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "leaderboard_description", 2024-04-25T14:15:54.541 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.541 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.541 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.542 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.542 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.542 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.542 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.542 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.542 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.542 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.542 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.542 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.542 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.542 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.542 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.542 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.543 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.543 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.543 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.543 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.543 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.543 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.543 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.543 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.543 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.543 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.543 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.543 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.543 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.543 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.543 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.544 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.544 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.544 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.544 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.544 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.544 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.544 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.544 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.544 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.544 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.544 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.544 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.544 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.545 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.545 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.545 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.545 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.545 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.545 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.545 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.545 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.545 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.545 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.545 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.545 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.545 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.546 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.546 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.546 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.546 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.546 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.546 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.546 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.546 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.546 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.546 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.546 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.546 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.546 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.547 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.547 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.547 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.547 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.547 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.547 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.547 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.547 INFO:teuthology.orchestra.run.smithi096.stdout: "organization": { 2024-04-25T14:15:54.547 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "organization", 2024-04-25T14:15:54.547 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.547 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.547 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.547 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.548 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.548 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.548 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.548 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.548 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.548 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.548 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.548 INFO:teuthology.orchestra.run.smithi096.stdout: "proxy": { 2024-04-25T14:15:54.548 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "proxy", 2024-04-25T14:15:54.548 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.548 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.548 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.548 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.549 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.549 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.549 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.549 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.549 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.549 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.549 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.549 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.549 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.549 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.549 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.549 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.550 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.550 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.550 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.550 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.550 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.550 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.550 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.550 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.550 INFO:teuthology.orchestra.run.smithi096.stdout: "url": { 2024-04-25T14:15:54.550 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "url", 2024-04-25T14:15:54.550 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.550 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.550 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.551 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T14:15:54.551 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.551 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.551 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.551 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.551 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.551 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.551 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.551 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.551 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.552 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "test_orchestrator", 2024-04-25T14:15:54.552 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.552 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.552 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.552 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.552 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.552 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.552 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.552 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.552 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.552 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.552 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.552 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.553 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.553 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.553 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.553 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.553 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.553 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.553 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.553 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.553 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.553 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.553 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.553 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.554 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.554 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.554 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.554 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.554 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.554 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.554 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.554 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.554 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.554 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.554 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.554 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.554 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.555 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.555 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.555 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.555 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.555 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.555 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.555 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.555 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.555 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.555 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.555 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.555 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.555 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.556 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.556 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.556 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.556 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.556 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.556 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.556 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.556 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.556 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.556 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.556 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.556 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.556 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.557 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.557 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.557 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.557 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.557 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.557 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.557 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.557 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.557 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.557 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.557 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.557 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.557 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.558 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.558 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.558 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.558 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.558 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.558 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.558 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.558 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.558 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "volumes", 2024-04-25T14:15:54.558 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.558 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.558 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.559 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.559 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.559 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.559 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.559 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.559 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.559 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.559 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.559 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.559 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.559 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.559 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.560 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.560 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.560 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.560 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.560 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.560 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.560 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.560 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.560 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.560 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.560 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.561 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.561 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.561 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.561 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.561 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.561 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.561 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.561 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.561 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.561 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.561 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.561 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.562 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.562 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.562 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.562 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.562 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.562 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.562 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.562 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.562 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.562 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.562 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.562 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.562 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.562 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.563 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.563 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.563 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.563 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.563 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.563 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.563 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.563 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.563 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.563 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.563 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.563 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.564 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.564 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.564 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.564 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.564 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.564 INFO:teuthology.orchestra.run.smithi096.stdout: "max_concurrent_clones": { 2024-04-25T14:15:54.564 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_concurrent_clones", 2024-04-25T14:15:54.564 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.564 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.564 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.564 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "4", 2024-04-25T14:15:54.564 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.564 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.565 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.565 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T14:15:54.565 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.565 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.565 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.565 INFO:teuthology.orchestra.run.smithi096.stdout: "periodic_async_work": { 2024-04-25T14:15:54.565 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "periodic_async_work", 2024-04-25T14:15:54.565 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.565 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.565 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.565 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.565 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.566 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.566 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.566 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Periodically check for async work", 2024-04-25T14:15:54.566 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.566 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.566 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.566 INFO:teuthology.orchestra.run.smithi096.stdout: "snapshot_clone_delay": { 2024-04-25T14:15:54.566 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snapshot_clone_delay", 2024-04-25T14:15:54.566 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.566 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.566 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.566 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.566 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.567 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.567 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.567 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T14:15:54.567 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.567 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.567 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.567 INFO:teuthology.orchestra.run.smithi096.stdout: "snapshot_clone_no_wait": { 2024-04-25T14:15:54.567 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T14:15:54.567 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.567 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.567 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.567 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.567 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.568 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.568 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.568 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T14:15:54.568 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.568 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.568 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.568 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.568 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.568 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.568 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.568 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.568 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.569 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.569 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.569 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.569 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.569 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.569 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.569 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.569 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.569 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.569 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix", 2024-04-25T14:15:54.569 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.570 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.570 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.570 INFO:teuthology.orchestra.run.smithi096.stdout: "discovery_interval": { 2024-04-25T14:15:54.570 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "discovery_interval", 2024-04-25T14:15:54.570 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-04-25T14:15:54.570 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.570 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.570 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "100", 2024-04-25T14:15:54.570 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.570 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.570 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.570 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.570 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.570 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.571 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.571 INFO:teuthology.orchestra.run.smithi096.stdout: "identifier": { 2024-04-25T14:15:54.571 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "identifier", 2024-04-25T14:15:54.571 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.571 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.571 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.571 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.571 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.571 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.571 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.571 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.572 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.572 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.572 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.572 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-04-25T14:15:54.572 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-04-25T14:15:54.572 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:54.572 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.572 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.572 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-04-25T14:15:54.572 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.572 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.572 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.572 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.573 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.573 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.573 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.573 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.573 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.573 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.573 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.573 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.573 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.573 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.573 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.573 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.573 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.574 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.574 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.574 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.574 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.574 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.574 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.574 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.574 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.574 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.574 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.574 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.574 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.575 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.575 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.575 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.575 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.575 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.575 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.575 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.575 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.575 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.575 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.575 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.575 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.576 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.576 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.576 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.576 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.576 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.576 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.576 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.576 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.576 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.576 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.576 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.576 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.576 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.577 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.577 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.577 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.577 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.577 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.577 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.577 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.577 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.577 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.577 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.577 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.578 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.578 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.578 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.578 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.578 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.578 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.578 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.578 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.578 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.578 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.578 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.579 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.579 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.579 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.579 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.579 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.579 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.579 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.579 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.579 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.579 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.579 INFO:teuthology.orchestra.run.smithi096.stdout: "zabbix_host": { 2024-04-25T14:15:54.579 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix_host", 2024-04-25T14:15:54.580 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.580 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.580 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.580 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.580 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.580 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.580 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.580 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.580 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.580 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.580 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.580 INFO:teuthology.orchestra.run.smithi096.stdout: "zabbix_port": { 2024-04-25T14:15:54.581 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix_port", 2024-04-25T14:15:54.581 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.581 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.581 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.581 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10051", 2024-04-25T14:15:54.581 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.581 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.581 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.581 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.581 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.581 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.581 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.581 INFO:teuthology.orchestra.run.smithi096.stdout: "zabbix_sender": { 2024-04-25T14:15:54.582 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix_sender", 2024-04-25T14:15:54.582 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.582 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.582 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.582 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T14:15:54.582 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.582 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.582 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.582 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.582 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.582 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.582 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.583 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.583 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.583 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.583 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-25T14:15:54.583 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.583 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.583 INFO:teuthology.orchestra.run.smithi096.stdout: "modules": [ 2024-04-25T14:15:54.583 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm", 2024-04-25T14:15:54.583 INFO:teuthology.orchestra.run.smithi096.stdout: "dashboard", 2024-04-25T14:15:54.583 INFO:teuthology.orchestra.run.smithi096.stdout: "iostat", 2024-04-25T14:15:54.583 INFO:teuthology.orchestra.run.smithi096.stdout: "nfs", 2024-04-25T14:15:54.583 INFO:teuthology.orchestra.run.smithi096.stdout: "restful" 2024-04-25T14:15:54.583 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.584 INFO:teuthology.orchestra.run.smithi096.stdout: "available_modules": [ 2024-04-25T14:15:54.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.584 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "alerts", 2024-04-25T14:15:54.584 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.584 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.584 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.584 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-04-25T14:15:54.584 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-04-25T14:15:54.584 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:54.584 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.584 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.584 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-04-25T14:15:54.584 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.584 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.585 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.585 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T14:15:54.585 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.585 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.585 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.585 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.585 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.585 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.585 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.585 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.585 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.585 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.585 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.586 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.586 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.586 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.586 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.586 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.586 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.586 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.586 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.586 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.586 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.586 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.586 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.586 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.587 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.587 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.587 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.587 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.587 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.587 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.587 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.587 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.587 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.587 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.587 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.587 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.587 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.588 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.588 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.588 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.588 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.588 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.588 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.588 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.588 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.588 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.588 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.588 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.588 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.588 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.589 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.589 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.589 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.589 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.589 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.589 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.589 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.589 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.589 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.589 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.589 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.589 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.589 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.589 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.590 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.590 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.590 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.590 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.590 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.590 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_destination": { 2024-04-25T14:15:54.590 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_destination", 2024-04-25T14:15:54.590 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.590 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.590 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.590 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.590 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.591 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.591 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.591 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Email address to send alerts to", 2024-04-25T14:15:54.591 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.591 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.591 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.591 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_from_name": { 2024-04-25T14:15:54.591 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_from_name", 2024-04-25T14:15:54.591 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.591 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.591 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.591 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "Ceph", 2024-04-25T14:15:54.591 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.591 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.592 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.592 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Email From: name", 2024-04-25T14:15:54.592 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.592 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.592 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.592 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_host": { 2024-04-25T14:15:54.592 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_host", 2024-04-25T14:15:54.592 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.592 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.592 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.592 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.592 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.592 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.593 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.593 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SMTP server", 2024-04-25T14:15:54.593 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.593 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.593 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.593 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_password": { 2024-04-25T14:15:54.593 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_password", 2024-04-25T14:15:54.593 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.593 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.593 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.593 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.593 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.593 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.594 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.594 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Password to authenticate with", 2024-04-25T14:15:54.594 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.594 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.594 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.594 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_port": { 2024-04-25T14:15:54.594 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_port", 2024-04-25T14:15:54.594 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.594 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.594 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.594 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "465", 2024-04-25T14:15:54.594 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.594 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.595 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.595 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SMTP port", 2024-04-25T14:15:54.595 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.595 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.595 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.595 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_sender": { 2024-04-25T14:15:54.595 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_sender", 2024-04-25T14:15:54.595 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.595 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.595 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.595 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.595 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.596 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.596 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.596 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SMTP envelope sender", 2024-04-25T14:15:54.596 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.596 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.596 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.596 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_ssl": { 2024-04-25T14:15:54.596 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_ssl", 2024-04-25T14:15:54.596 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.596 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.596 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.596 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.597 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.597 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.597 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.597 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T14:15:54.597 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.597 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.597 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.597 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_user": { 2024-04-25T14:15:54.597 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_user", 2024-04-25T14:15:54.597 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.597 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.597 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.597 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.598 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.598 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.598 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.598 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "User to authenticate as", 2024-04-25T14:15:54.598 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.598 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.598 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.598 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.598 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.598 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.598 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.598 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.599 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.599 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.599 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.599 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.599 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.599 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.599 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.599 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.599 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.599 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.599 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "balancer", 2024-04-25T14:15:54.599 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.600 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.600 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.600 INFO:teuthology.orchestra.run.smithi096.stdout: "active": { 2024-04-25T14:15:54.600 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "active", 2024-04-25T14:15:54.600 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.600 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.600 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.600 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.600 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.600 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.600 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.600 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T14:15:54.600 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.601 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.601 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.601 INFO:teuthology.orchestra.run.smithi096.stdout: "begin_time": { 2024-04-25T14:15:54.601 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "begin_time", 2024-04-25T14:15:54.601 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.602 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.602 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.602 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0000", 2024-04-25T14:15:54.602 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.602 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.602 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.602 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T14:15:54.603 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T14:15:54.603 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.603 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.603 INFO:teuthology.orchestra.run.smithi096.stdout: "begin_weekday": { 2024-04-25T14:15:54.603 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "begin_weekday", 2024-04-25T14:15:54.603 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-04-25T14:15:54.603 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.603 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.603 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.603 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "0", 2024-04-25T14:15:54.603 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "6", 2024-04-25T14:15:54.603 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.603 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T14:15:54.603 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T14:15:54.604 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.604 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.604 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_compat_max_iterations": { 2024-04-25T14:15:54.604 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crush_compat_max_iterations", 2024-04-25T14:15:54.604 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-04-25T14:15:54.604 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.604 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.604 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "25", 2024-04-25T14:15:54.604 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-04-25T14:15:54.604 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "250", 2024-04-25T14:15:54.604 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.604 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T14:15:54.604 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.605 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.605 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.605 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_compat_metrics": { 2024-04-25T14:15:54.605 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crush_compat_metrics", 2024-04-25T14:15:54.605 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.605 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.605 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.605 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T14:15:54.605 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.605 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.605 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.605 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T14:15:54.605 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T14:15:54.606 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.606 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.606 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_compat_step": { 2024-04-25T14:15:54.606 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crush_compat_step", 2024-04-25T14:15:54.606 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T14:15:54.606 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.606 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.606 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0.5", 2024-04-25T14:15:54.606 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "0.001", 2024-04-25T14:15:54.606 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "0.999", 2024-04-25T14:15:54.606 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.606 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "aggressiveness of optimization", 2024-04-25T14:15:54.606 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T14:15:54.607 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.607 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.607 INFO:teuthology.orchestra.run.smithi096.stdout: "end_time": { 2024-04-25T14:15:54.607 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "end_time", 2024-04-25T14:15:54.607 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.607 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.607 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.607 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "2359", 2024-04-25T14:15:54.607 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.607 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.607 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.607 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T14:15:54.607 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T14:15:54.608 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.608 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.608 INFO:teuthology.orchestra.run.smithi096.stdout: "end_weekday": { 2024-04-25T14:15:54.608 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "end_weekday", 2024-04-25T14:15:54.608 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-04-25T14:15:54.608 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.608 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.608 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.608 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "0", 2024-04-25T14:15:54.608 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "6", 2024-04-25T14:15:54.608 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.608 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T14:15:54.608 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T14:15:54.609 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.609 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.609 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.609 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.609 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.609 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.609 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.609 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.609 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.609 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.609 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.609 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.609 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.610 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.610 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.610 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.610 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.610 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.610 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.610 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.610 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.610 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.610 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.610 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.611 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.611 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.611 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.611 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.611 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.611 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.611 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.611 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.611 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.611 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.611 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.611 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.611 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.612 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.612 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.612 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.612 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.612 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.612 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.612 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.612 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.612 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.612 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.612 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.612 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.612 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.612 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.612 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.613 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.613 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.613 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.613 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.613 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.613 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.613 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.613 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.613 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.613 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.613 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.613 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.613 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.614 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.614 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.614 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.614 INFO:teuthology.orchestra.run.smithi096.stdout: "min_score": { 2024-04-25T14:15:54.614 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "min_score", 2024-04-25T14:15:54.614 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T14:15:54.614 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.614 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.614 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.614 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.614 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.615 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.615 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T14:15:54.615 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.615 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.618 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.618 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": { 2024-04-25T14:15:54.618 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mode", 2024-04-25T14:15:54.618 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.618 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.618 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.618 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "upmap", 2024-04-25T14:15:54.618 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.618 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.619 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.619 INFO:teuthology.orchestra.run.smithi096.stdout: "crush-compat", 2024-04-25T14:15:54.619 INFO:teuthology.orchestra.run.smithi096.stdout: "none", 2024-04-25T14:15:54.619 INFO:teuthology.orchestra.run.smithi096.stdout: "read", 2024-04-25T14:15:54.619 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap", 2024-04-25T14:15:54.619 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap-read" 2024-04-25T14:15:54.619 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.619 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Balancer mode", 2024-04-25T14:15:54.619 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.619 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.619 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.620 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_ids": { 2024-04-25T14:15:54.620 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pool_ids", 2024-04-25T14:15:54.620 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.620 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.620 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.620 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.620 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.620 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.620 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.620 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T14:15:54.620 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.620 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.620 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.621 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-04-25T14:15:54.621 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-04-25T14:15:54.621 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:54.621 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.621 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.621 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-04-25T14:15:54.621 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.621 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.621 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.621 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T14:15:54.621 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.621 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.621 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.622 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.622 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.622 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.622 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.622 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.622 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.622 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.622 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.622 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.622 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.622 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.622 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.622 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.623 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap_max_deviation": { 2024-04-25T14:15:54.623 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "upmap_max_deviation", 2024-04-25T14:15:54.623 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.623 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.623 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.623 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-04-25T14:15:54.623 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-04-25T14:15:54.623 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.623 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.623 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T14:15:54.623 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T14:15:54.623 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.623 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.624 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap_max_optimizations": { 2024-04-25T14:15:54.624 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "upmap_max_optimizations", 2024-04-25T14:15:54.624 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-04-25T14:15:54.624 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.624 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.624 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T14:15:54.624 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.624 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.624 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.624 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T14:15:54.624 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.624 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.624 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.624 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.625 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.625 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cephadm", 2024-04-25T14:15:54.625 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.625 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.625 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.625 INFO:teuthology.orchestra.run.smithi096.stdout: "agent_down_multiplier": { 2024-04-25T14:15:54.625 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "agent_down_multiplier", 2024-04-25T14:15:54.625 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T14:15:54.625 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.625 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.625 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "3.0", 2024-04-25T14:15:54.626 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.626 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.626 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.626 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T14:15:54.626 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.626 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.626 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.626 INFO:teuthology.orchestra.run.smithi096.stdout: "agent_refresh_rate": { 2024-04-25T14:15:54.626 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "agent_refresh_rate", 2024-04-25T14:15:54.626 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:54.626 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.626 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.626 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "20", 2024-04-25T14:15:54.627 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.627 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.627 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.627 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T14:15:54.627 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.627 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.627 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.627 INFO:teuthology.orchestra.run.smithi096.stdout: "agent_starting_port": { 2024-04-25T14:15:54.627 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "agent_starting_port", 2024-04-25T14:15:54.627 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.627 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.627 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.627 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "4721", 2024-04-25T14:15:54.628 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.628 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.628 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.628 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T14:15:54.628 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.628 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.628 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.628 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_ptrace": { 2024-04-25T14:15:54.628 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "allow_ptrace", 2024-04-25T14:15:54.628 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.628 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.628 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.628 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.629 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.629 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.629 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.629 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T14:15:54.629 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T14:15:54.629 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.629 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.629 INFO:teuthology.orchestra.run.smithi096.stdout: "autotune_interval": { 2024-04-25T14:15:54.629 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "autotune_interval", 2024-04-25T14:15:54.629 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:54.629 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.629 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.629 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-04-25T14:15:54.630 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.630 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.630 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.630 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T14:15:54.630 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.630 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.630 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.630 INFO:teuthology.orchestra.run.smithi096.stdout: "autotune_memory_target_ratio": { 2024-04-25T14:15:54.630 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T14:15:54.630 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T14:15:54.630 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.630 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.631 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0.7", 2024-04-25T14:15:54.631 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.631 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.631 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.631 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T14:15:54.631 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.631 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.631 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.631 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm_log_destination": { 2024-04-25T14:15:54.631 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cephadm_log_destination", 2024-04-25T14:15:54.631 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.631 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.631 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.632 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.632 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.632 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.632 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.632 INFO:teuthology.orchestra.run.smithi096.stdout: "file", 2024-04-25T14:15:54.632 INFO:teuthology.orchestra.run.smithi096.stdout: "file,syslog", 2024-04-25T14:15:54.632 INFO:teuthology.orchestra.run.smithi096.stdout: "syslog" 2024-04-25T14:15:54.632 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.632 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T14:15:54.632 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.632 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.632 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.632 INFO:teuthology.orchestra.run.smithi096.stdout: "cgroups_split": { 2024-04-25T14:15:54.633 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cgroups_split", 2024-04-25T14:15:54.633 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.633 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.633 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.633 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.633 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.633 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.633 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.633 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T14:15:54.633 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.633 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.633 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.633 INFO:teuthology.orchestra.run.smithi096.stdout: "config_checks_enabled": { 2024-04-25T14:15:54.634 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "config_checks_enabled", 2024-04-25T14:15:54.634 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.634 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.634 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.634 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.634 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.634 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.634 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.634 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T14:15:54.634 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.634 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.634 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.635 INFO:teuthology.orchestra.run.smithi096.stdout: "config_dashboard": { 2024-04-25T14:15:54.635 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "config_dashboard", 2024-04-25T14:15:54.635 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.635 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.635 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.635 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.635 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.635 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.635 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.635 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T14:15:54.635 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.635 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.635 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.636 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_alertmanager": { 2024-04-25T14:15:54.636 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_alertmanager", 2024-04-25T14:15:54.636 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.636 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.636 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.636 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T14:15:54.636 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.636 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.636 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.636 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-04-25T14:15:54.636 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.636 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.636 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.637 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_base": { 2024-04-25T14:15:54.637 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_base", 2024-04-25T14:15:54.637 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.637 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.637 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.637 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T14:15:54.637 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.637 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.637 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.637 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Container image name, without the tag", 2024-04-25T14:15:54.637 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.637 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.637 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.638 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_elasticsearch": { 2024-04-25T14:15:54.638 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_elasticsearch", 2024-04-25T14:15:54.638 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.638 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.638 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.638 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T14:15:54.638 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.638 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.638 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.638 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "elasticsearch container image", 2024-04-25T14:15:54.638 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.638 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.638 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.639 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_grafana": { 2024-04-25T14:15:54.639 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_grafana", 2024-04-25T14:15:54.639 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.639 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.639 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.639 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T14:15:54.639 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.639 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.639 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.639 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-04-25T14:15:54.639 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.639 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.639 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.640 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_haproxy": { 2024-04-25T14:15:54.640 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_haproxy", 2024-04-25T14:15:54.640 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.640 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.640 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.640 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T14:15:54.640 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.640 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.640 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.640 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "HAproxy container image", 2024-04-25T14:15:54.640 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.640 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.640 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.641 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_jaeger_agent": { 2024-04-25T14:15:54.641 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_jaeger_agent", 2024-04-25T14:15:54.641 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.641 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.641 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.641 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T14:15:54.641 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.641 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.641 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.641 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Jaeger agent container image", 2024-04-25T14:15:54.641 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.641 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.642 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.642 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_jaeger_collector": { 2024-04-25T14:15:54.642 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_jaeger_collector", 2024-04-25T14:15:54.642 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.642 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.642 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.642 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T14:15:54.642 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.642 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.642 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.642 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Jaeger collector container image", 2024-04-25T14:15:54.642 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.642 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.643 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.643 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_jaeger_query": { 2024-04-25T14:15:54.643 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_jaeger_query", 2024-04-25T14:15:54.643 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.643 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.643 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.643 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T14:15:54.643 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.643 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.643 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.643 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Jaeger query container image", 2024-04-25T14:15:54.643 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.643 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.643 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.644 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_keepalived": { 2024-04-25T14:15:54.644 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_keepalived", 2024-04-25T14:15:54.644 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.644 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.644 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.644 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T14:15:54.644 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.644 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.644 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.644 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Keepalived container image", 2024-04-25T14:15:54.644 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.644 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.645 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.645 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_loki": { 2024-04-25T14:15:54.645 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_loki", 2024-04-25T14:15:54.645 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.645 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.645 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.645 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T14:15:54.645 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.645 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.645 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.645 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Loki container image", 2024-04-25T14:15:54.645 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.645 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.645 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.646 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_node_exporter": { 2024-04-25T14:15:54.646 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_node_exporter", 2024-04-25T14:15:54.646 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.646 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.646 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.646 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T14:15:54.646 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.646 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.646 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.646 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-04-25T14:15:54.646 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.646 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.647 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.647 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_nvmeof": { 2024-04-25T14:15:54.647 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_nvmeof", 2024-04-25T14:15:54.647 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.647 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.647 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.647 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T14:15:54.647 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.647 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.647 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.647 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Nvme-of container image", 2024-04-25T14:15:54.648 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.648 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.648 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.648 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_prometheus": { 2024-04-25T14:15:54.648 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_prometheus", 2024-04-25T14:15:54.648 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.648 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.648 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.648 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T14:15:54.648 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.648 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.648 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.648 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-04-25T14:15:54.649 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.649 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.649 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.649 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_promtail": { 2024-04-25T14:15:54.649 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_promtail", 2024-04-25T14:15:54.649 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.649 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.649 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.649 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T14:15:54.649 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.649 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.649 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.649 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Promtail container image", 2024-04-25T14:15:54.650 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.650 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.650 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.650 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_samba": { 2024-04-25T14:15:54.650 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_samba", 2024-04-25T14:15:54.650 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.650 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.650 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.650 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T14:15:54.650 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.650 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.650 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.651 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Samba/SMB container image", 2024-04-25T14:15:54.651 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.651 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.651 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.651 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_snmp_gateway": { 2024-04-25T14:15:54.651 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_snmp_gateway", 2024-04-25T14:15:54.651 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.651 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.651 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.651 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T14:15:54.651 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.651 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.652 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.652 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SNMP Gateway container image", 2024-04-25T14:15:54.652 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.652 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.652 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.652 INFO:teuthology.orchestra.run.smithi096.stdout: "container_init": { 2024-04-25T14:15:54.652 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_init", 2024-04-25T14:15:54.652 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.652 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.652 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.652 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.652 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.652 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.653 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.653 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T14:15:54.653 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.653 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.653 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.653 INFO:teuthology.orchestra.run.smithi096.stdout: "daemon_cache_timeout": { 2024-04-25T14:15:54.653 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "daemon_cache_timeout", 2024-04-25T14:15:54.653 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:54.653 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.653 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.653 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-04-25T14:15:54.653 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.653 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.654 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.654 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T14:15:54.654 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.654 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.654 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.654 INFO:teuthology.orchestra.run.smithi096.stdout: "default_cephadm_command_timeout": { 2024-04-25T14:15:54.654 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T14:15:54.654 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.654 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.654 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.654 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "900", 2024-04-25T14:15:54.654 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.655 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.655 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.655 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T14:15:54.655 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.655 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.655 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.655 INFO:teuthology.orchestra.run.smithi096.stdout: "default_registry": { 2024-04-25T14:15:54.655 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "default_registry", 2024-04-25T14:15:54.655 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.655 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.655 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.655 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io", 2024-04-25T14:15:54.656 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.656 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.656 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.656 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T14:15:54.656 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.656 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.656 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.656 INFO:teuthology.orchestra.run.smithi096.stdout: "device_cache_timeout": { 2024-04-25T14:15:54.656 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "device_cache_timeout", 2024-04-25T14:15:54.656 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:54.656 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.656 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.656 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "1800", 2024-04-25T14:15:54.657 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.657 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.657 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.657 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "seconds to cache device inventory", 2024-04-25T14:15:54.657 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.657 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.657 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.657 INFO:teuthology.orchestra.run.smithi096.stdout: "device_enhanced_scan": { 2024-04-25T14:15:54.657 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "device_enhanced_scan", 2024-04-25T14:15:54.657 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.657 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.657 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.657 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.658 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.658 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.658 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.658 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T14:15:54.658 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.658 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.658 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.658 INFO:teuthology.orchestra.run.smithi096.stdout: "facts_cache_timeout": { 2024-04-25T14:15:54.658 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "facts_cache_timeout", 2024-04-25T14:15:54.658 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:54.658 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.658 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.659 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-04-25T14:15:54.659 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.659 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.659 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.659 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "seconds to cache host facts data", 2024-04-25T14:15:54.659 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.659 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.659 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.659 INFO:teuthology.orchestra.run.smithi096.stdout: "grafana_dashboards_path": { 2024-04-25T14:15:54.659 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "grafana_dashboards_path", 2024-04-25T14:15:54.659 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.660 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.660 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.660 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T14:15:54.660 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.660 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.660 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.660 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T14:15:54.660 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.660 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.660 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.660 INFO:teuthology.orchestra.run.smithi096.stdout: "host_check_interval": { 2024-04-25T14:15:54.660 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "host_check_interval", 2024-04-25T14:15:54.660 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:54.661 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.661 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.661 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-04-25T14:15:54.661 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.661 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.661 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.661 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to perform a host check", 2024-04-25T14:15:54.661 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.661 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.661 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.661 INFO:teuthology.orchestra.run.smithi096.stdout: "hw_monitoring": { 2024-04-25T14:15:54.661 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "hw_monitoring", 2024-04-25T14:15:54.661 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.662 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.662 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.662 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.662 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.662 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.662 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.662 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T14:15:54.662 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.662 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.662 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.662 INFO:teuthology.orchestra.run.smithi096.stdout: "inventory_list_all": { 2024-04-25T14:15:54.663 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "inventory_list_all", 2024-04-25T14:15:54.663 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.663 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.663 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.663 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.663 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.663 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.663 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.663 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T14:15:54.663 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.663 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.663 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.664 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.664 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.664 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.664 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.664 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.664 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.664 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.664 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.664 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.664 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.664 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.664 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.664 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.664 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.664 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.665 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.665 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.665 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.665 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.665 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.665 INFO:teuthology.orchestra.run.smithi096.stdout: "log_refresh_metadata": { 2024-04-25T14:15:54.665 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_refresh_metadata", 2024-04-25T14:15:54.665 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.665 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.665 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.665 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.665 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.665 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.666 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.666 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T14:15:54.666 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.666 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.666 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.666 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.666 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.666 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.666 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.666 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.666 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.666 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.666 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.667 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.667 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T14:15:54.667 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.667 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.667 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.667 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.667 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.667 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.667 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.667 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.667 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.667 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.668 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.668 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.668 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.668 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.668 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.668 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.668 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.668 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.668 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.668 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.668 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.668 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.668 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.669 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.669 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.669 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.669 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.669 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.669 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.669 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.669 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.669 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.669 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.669 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.669 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.669 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.670 INFO:teuthology.orchestra.run.smithi096.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T14:15:54.670 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T14:15:54.670 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.670 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.670 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.670 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.670 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.670 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.670 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.670 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T14:15:54.670 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.670 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.670 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.671 INFO:teuthology.orchestra.run.smithi096.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T14:15:54.671 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T14:15:54.671 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.671 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.671 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.671 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "*", 2024-04-25T14:15:54.671 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.671 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.671 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.671 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T14:15:54.671 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.671 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.671 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.672 INFO:teuthology.orchestra.run.smithi096.stdout: "max_count_per_host": { 2024-04-25T14:15:54.672 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_count_per_host", 2024-04-25T14:15:54.672 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.672 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.672 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.672 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T14:15:54.672 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.672 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.672 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.672 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "max number of daemons per service per host", 2024-04-25T14:15:54.672 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.672 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.672 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.673 INFO:teuthology.orchestra.run.smithi096.stdout: "max_osd_draining_count": { 2024-04-25T14:15:54.673 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_osd_draining_count", 2024-04-25T14:15:54.673 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.673 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.673 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.673 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T14:15:54.673 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.673 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.673 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.673 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T14:15:54.673 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.673 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.673 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.673 INFO:teuthology.orchestra.run.smithi096.stdout: "migration_current": { 2024-04-25T14:15:54.674 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "migration_current", 2024-04-25T14:15:54.674 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.674 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.674 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.674 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.674 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.674 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.674 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.674 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "internal - do not modify", 2024-04-25T14:15:54.674 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.674 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.674 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.675 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": { 2024-04-25T14:15:54.675 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mode", 2024-04-25T14:15:54.675 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.675 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.675 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.675 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "root", 2024-04-25T14:15:54.675 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.675 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.675 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.675 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm-package", 2024-04-25T14:15:54.675 INFO:teuthology.orchestra.run.smithi096.stdout: "root" 2024-04-25T14:15:54.675 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.675 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T14:15:54.676 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.676 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.676 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.676 INFO:teuthology.orchestra.run.smithi096.stdout: "oob_default_addr": { 2024-04-25T14:15:54.676 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "oob_default_addr", 2024-04-25T14:15:54.676 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.676 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.676 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.676 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "169.254.1.1", 2024-04-25T14:15:54.676 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.676 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.677 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.677 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T14:15:54.677 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.677 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.677 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.677 INFO:teuthology.orchestra.run.smithi096.stdout: "prometheus_alerts_path": { 2024-04-25T14:15:54.677 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "prometheus_alerts_path", 2024-04-25T14:15:54.677 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.677 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.677 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.677 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T14:15:54.677 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.677 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.678 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.678 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T14:15:54.678 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.678 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.678 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.678 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_insecure": { 2024-04-25T14:15:54.678 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_insecure", 2024-04-25T14:15:54.678 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.678 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.678 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.678 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.678 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.679 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.679 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.679 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T14:15:54.679 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.679 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.679 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.679 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_password": { 2024-04-25T14:15:54.679 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_password", 2024-04-25T14:15:54.679 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.679 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.679 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.679 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.679 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.680 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.680 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.680 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T14:15:54.680 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.680 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.680 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.680 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_url": { 2024-04-25T14:15:54.680 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_url", 2024-04-25T14:15:54.680 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.680 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.680 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.680 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.680 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.681 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.681 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.681 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T14:15:54.681 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.681 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.681 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.681 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_username": { 2024-04-25T14:15:54.681 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_username", 2024-04-25T14:15:54.681 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.681 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.681 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.681 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.681 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.681 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.682 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.682 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T14:15:54.682 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.682 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.682 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.682 INFO:teuthology.orchestra.run.smithi096.stdout: "secure_monitoring_stack": { 2024-04-25T14:15:54.682 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "secure_monitoring_stack", 2024-04-25T14:15:54.682 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.682 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.682 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.682 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.682 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.683 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.683 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.683 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T14:15:54.683 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.683 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.683 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.683 INFO:teuthology.orchestra.run.smithi096.stdout: "service_discovery_port": { 2024-04-25T14:15:54.683 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "service_discovery_port", 2024-04-25T14:15:54.683 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.683 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.683 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.683 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8765", 2024-04-25T14:15:54.683 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.684 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.684 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.684 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "cephadm service discovery port", 2024-04-25T14:15:54.684 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.684 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.684 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.684 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.684 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.684 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.684 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.684 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.684 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.685 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.685 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.685 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.685 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.685 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.685 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.685 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.685 INFO:teuthology.orchestra.run.smithi096.stdout: "ssh_config_file": { 2024-04-25T14:15:54.685 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssh_config_file", 2024-04-25T14:15:54.685 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.685 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.685 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.685 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.686 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.686 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.686 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.686 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T14:15:54.686 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.686 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.686 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.686 INFO:teuthology.orchestra.run.smithi096.stdout: "use_agent": { 2024-04-25T14:15:54.686 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "use_agent", 2024-04-25T14:15:54.686 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.686 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.686 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.687 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.687 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.687 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.687 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.687 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T14:15:54.687 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.687 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.687 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.687 INFO:teuthology.orchestra.run.smithi096.stdout: "use_repo_digest": { 2024-04-25T14:15:54.687 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "use_repo_digest", 2024-04-25T14:15:54.687 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.687 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.687 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.688 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.688 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.688 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.688 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.688 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T14:15:54.688 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.688 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.688 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.688 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_on_failed_host_check": { 2024-04-25T14:15:54.688 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_on_failed_host_check", 2024-04-25T14:15:54.688 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.688 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.688 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.689 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.689 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.689 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.689 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.689 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T14:15:54.689 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.689 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.689 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.689 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_on_stray_daemons": { 2024-04-25T14:15:54.689 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_on_stray_daemons", 2024-04-25T14:15:54.690 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.690 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.690 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.690 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.690 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.690 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.690 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.690 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T14:15:54.690 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.690 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.690 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.690 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_on_stray_hosts": { 2024-04-25T14:15:54.690 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_on_stray_hosts", 2024-04-25T14:15:54.691 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.691 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.691 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.691 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.691 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.691 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.691 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.691 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T14:15:54.691 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.691 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.691 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.691 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.692 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.692 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crash", 2024-04-25T14:15:54.692 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.692 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.692 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.692 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.692 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.692 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.692 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.692 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.692 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.692 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.693 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.693 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.693 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.693 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.693 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.693 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.693 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.693 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.693 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.693 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.693 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.693 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.693 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.694 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.694 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.694 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.694 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.694 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.694 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.694 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.694 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.694 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.694 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.694 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.694 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.694 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.695 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.695 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.695 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.695 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.695 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.695 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.695 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.695 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.695 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.695 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.695 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.695 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.695 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.695 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.696 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.696 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.696 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.696 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.696 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.696 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.696 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.696 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.696 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.696 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.696 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.696 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.697 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.697 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.697 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.697 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.697 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.697 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.697 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.697 INFO:teuthology.orchestra.run.smithi096.stdout: "retain_interval": { 2024-04-25T14:15:54.697 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "retain_interval", 2024-04-25T14:15:54.697 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:54.697 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.697 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.697 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "31536000", 2024-04-25T14:15:54.697 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.698 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.698 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.698 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T14:15:54.698 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.698 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.698 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.698 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.698 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.698 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.698 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.698 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.698 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.698 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.699 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.699 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.699 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.699 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.699 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.699 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.699 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_recent_interval": { 2024-04-25T14:15:54.699 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_recent_interval", 2024-04-25T14:15:54.699 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:54.699 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.699 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.699 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "1209600", 2024-04-25T14:15:54.700 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.700 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.700 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.700 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T14:15:54.700 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.700 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.700 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.700 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.700 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.700 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "dashboard", 2024-04-25T14:15:54.700 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.700 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.700 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.701 INFO:teuthology.orchestra.run.smithi096.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T14:15:54.701 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T14:15:54.701 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.701 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.701 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.701 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T14:15:54.701 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.701 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.701 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.701 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.701 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.701 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.701 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.702 INFO:teuthology.orchestra.run.smithi096.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T14:15:54.702 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T14:15:54.702 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.702 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.702 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.702 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.702 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.702 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.702 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.702 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.702 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.702 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.702 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.702 INFO:teuthology.orchestra.run.smithi096.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T14:15:54.703 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T14:15:54.703 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.703 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.703 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.703 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.703 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.703 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.703 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.703 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.703 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.703 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.703 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.703 INFO:teuthology.orchestra.run.smithi096.stdout: "AUDIT_API_ENABLED": { 2024-04-25T14:15:54.704 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T14:15:54.704 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.704 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.704 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.704 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.704 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.704 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.704 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.704 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.704 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.704 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.704 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.704 INFO:teuthology.orchestra.run.smithi096.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T14:15:54.704 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T14:15:54.705 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.705 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.705 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.705 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.705 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.705 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.705 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.705 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.705 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.705 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.705 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.705 INFO:teuthology.orchestra.run.smithi096.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T14:15:54.705 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T14:15:54.706 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.706 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.706 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.706 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.706 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.706 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.706 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.706 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.706 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.706 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.706 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.707 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T14:15:54.707 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T14:15:54.707 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.707 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.707 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.707 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.707 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.707 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.707 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.707 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.707 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.707 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.707 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.708 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T14:15:54.708 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T14:15:54.708 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.708 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.708 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.708 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.708 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.708 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.708 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.708 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.708 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.708 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.708 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.709 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T14:15:54.709 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T14:15:54.709 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.709 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.709 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.709 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.709 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.709 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.709 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.709 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.709 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.709 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.709 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.710 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T14:15:54.710 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T14:15:54.710 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.710 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.710 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.710 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.710 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.710 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.710 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.710 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.710 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.710 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.710 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.711 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T14:15:54.711 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T14:15:54.711 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.711 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.711 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.711 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.711 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.711 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.711 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.711 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.711 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.711 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.711 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.711 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T14:15:54.712 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T14:15:54.712 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.712 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.712 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.712 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.712 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.712 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.712 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.712 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.712 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.712 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.712 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.712 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T14:15:54.713 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T14:15:54.713 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.713 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.713 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.713 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.713 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.713 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.713 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.713 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.713 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.713 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.713 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.713 INFO:teuthology.orchestra.run.smithi096.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T14:15:54.713 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T14:15:54.714 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.714 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.714 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.714 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.714 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.714 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.714 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.714 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.714 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.714 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.714 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.714 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T14:15:54.714 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T14:15:54.715 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.715 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.715 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.715 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "admin", 2024-04-25T14:15:54.715 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.715 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.715 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.715 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.715 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.715 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.715 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.715 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T14:15:54.715 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T14:15:54.716 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.716 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.716 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.716 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.716 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.716 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.716 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.716 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.716 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.716 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.716 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.716 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_URL": { 2024-04-25T14:15:54.716 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_URL", 2024-04-25T14:15:54.716 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.717 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.717 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.717 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.717 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.717 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.717 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.717 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.717 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.717 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.717 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.717 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T14:15:54.717 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T14:15:54.717 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.718 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.718 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.718 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "admin", 2024-04-25T14:15:54.718 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.718 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.718 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.718 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.718 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.718 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.718 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.718 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T14:15:54.718 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T14:15:54.718 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.719 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.719 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.719 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.719 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.719 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.719 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.719 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.719 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.719 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.719 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.719 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T14:15:54.719 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T14:15:54.719 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.719 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.720 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.720 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.720 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.720 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.720 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.720 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.720 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.720 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.720 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.720 INFO:teuthology.orchestra.run.smithi096.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T14:15:54.720 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T14:15:54.720 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.720 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.721 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.721 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.721 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.721 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.721 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.721 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.721 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.721 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.721 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.721 INFO:teuthology.orchestra.run.smithi096.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T14:15:54.721 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T14:15:54.721 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.721 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.722 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.722 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.722 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.722 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.722 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.722 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.722 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.722 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.722 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.722 INFO:teuthology.orchestra.run.smithi096.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T14:15:54.722 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T14:15:54.722 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.722 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.723 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.723 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "[]", 2024-04-25T14:15:54.723 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.723 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.723 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.723 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.723 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.723 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.723 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.723 INFO:teuthology.orchestra.run.smithi096.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T14:15:54.723 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T14:15:54.724 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.724 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.724 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.724 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "{}", 2024-04-25T14:15:54.724 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.724 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.724 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.724 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.724 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.724 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.724 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.725 INFO:teuthology.orchestra.run.smithi096.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T14:15:54.725 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T14:15:54.725 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.725 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.725 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.725 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.725 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.725 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.725 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.725 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.725 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.725 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.725 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.726 INFO:teuthology.orchestra.run.smithi096.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T14:15:54.726 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T14:15:54.726 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.726 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.726 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.726 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.726 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.726 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.726 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.726 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.726 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.726 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.727 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.727 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T14:15:54.727 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T14:15:54.727 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.727 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.727 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.727 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.727 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.727 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.727 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.727 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.727 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.727 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.728 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.728 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T14:15:54.728 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T14:15:54.728 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.728 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.728 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.728 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.728 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.728 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.728 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.728 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.728 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.728 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.728 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.729 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T14:15:54.729 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T14:15:54.729 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.729 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.729 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.729 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.729 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.729 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.729 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.729 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.729 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.729 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.729 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.730 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T14:15:54.730 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T14:15:54.730 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.730 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.730 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.730 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.730 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.730 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.730 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.730 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.730 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.730 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.730 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.731 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T14:15:54.731 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T14:15:54.731 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.731 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.731 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.731 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.731 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.731 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.731 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.731 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.731 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.731 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.731 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.732 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T14:15:54.732 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T14:15:54.732 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.732 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.732 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.732 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.732 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.732 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.732 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.732 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.732 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.732 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.732 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.733 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T14:15:54.733 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T14:15:54.733 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.733 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.733 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.733 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.733 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.733 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.733 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.733 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.733 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.733 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.733 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.734 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T14:15:54.734 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T14:15:54.734 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.734 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.734 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.734 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.734 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.734 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.734 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.734 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.734 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.734 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.735 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.735 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T14:15:54.735 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T14:15:54.735 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.735 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.735 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.735 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T14:15:54.735 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.735 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.735 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.735 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.735 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.736 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.736 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.736 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T14:15:54.736 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T14:15:54.736 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.736 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.736 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.736 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T14:15:54.736 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.736 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.736 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.736 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.737 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.737 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.737 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.737 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T14:15:54.737 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T14:15:54.737 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.737 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.737 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.737 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8", 2024-04-25T14:15:54.737 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.737 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.737 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.737 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.738 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.738 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.738 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.738 INFO:teuthology.orchestra.run.smithi096.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T14:15:54.738 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T14:15:54.738 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.738 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.738 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.738 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "45", 2024-04-25T14:15:54.738 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.738 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.739 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.739 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.739 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.739 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.739 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.739 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T14:15:54.739 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T14:15:54.739 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.739 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.739 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.739 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.739 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.739 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.740 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.740 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.740 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.740 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.740 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.740 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T14:15:54.740 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T14:15:54.740 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.740 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.740 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.740 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "admin", 2024-04-25T14:15:54.741 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.741 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.741 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.741 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.741 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.741 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.741 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.741 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T14:15:54.741 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T14:15:54.741 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.741 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.741 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.742 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.742 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.742 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.742 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.742 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.742 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.742 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.742 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.742 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T14:15:54.742 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T14:15:54.742 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.742 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.743 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.743 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.743 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.743 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.743 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.743 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.743 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.743 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.743 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.743 INFO:teuthology.orchestra.run.smithi096.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T14:15:54.743 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T14:15:54.743 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.744 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.744 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.744 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.744 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.744 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.744 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.744 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.744 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.744 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.744 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.744 INFO:teuthology.orchestra.run.smithi096.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T14:15:54.744 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T14:15:54.744 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.744 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.745 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.745 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.745 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.745 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.745 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.745 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.745 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.745 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.745 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.745 INFO:teuthology.orchestra.run.smithi096.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T14:15:54.745 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T14:15:54.745 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.745 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.746 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.746 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T14:15:54.746 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.746 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.746 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.746 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.746 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.746 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.746 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.746 INFO:teuthology.orchestra.run.smithi096.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T14:15:54.746 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T14:15:54.746 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.747 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.747 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.747 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-04-25T14:15:54.747 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.747 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.747 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.747 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.747 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.747 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.747 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.747 INFO:teuthology.orchestra.run.smithi096.stdout: "cross_origin_url": { 2024-04-25T14:15:54.747 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cross_origin_url", 2024-04-25T14:15:54.747 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.747 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.748 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.748 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.748 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.748 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.748 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.748 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.748 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.748 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.748 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.748 INFO:teuthology.orchestra.run.smithi096.stdout: "crt_file": { 2024-04-25T14:15:54.748 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crt_file", 2024-04-25T14:15:54.748 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.748 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.749 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.749 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.749 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.749 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.749 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.749 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.749 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.749 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.749 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.749 INFO:teuthology.orchestra.run.smithi096.stdout: "debug": { 2024-04-25T14:15:54.749 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "debug", 2024-04-25T14:15:54.749 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.749 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.749 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.750 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.750 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.750 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.750 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.750 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Enable/disable debug options", 2024-04-25T14:15:54.750 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.750 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.750 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.750 INFO:teuthology.orchestra.run.smithi096.stdout: "jwt_token_ttl": { 2024-04-25T14:15:54.750 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "jwt_token_ttl", 2024-04-25T14:15:54.750 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.750 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.750 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.751 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "28800", 2024-04-25T14:15:54.751 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.751 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.751 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.751 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.751 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.751 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.751 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.751 INFO:teuthology.orchestra.run.smithi096.stdout: "key_file": { 2024-04-25T14:15:54.751 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "key_file", 2024-04-25T14:15:54.751 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.751 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.751 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.752 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.752 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.752 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.752 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.752 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.752 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.752 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.752 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.752 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.752 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.752 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.752 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.752 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.752 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.753 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.753 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.753 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.753 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.753 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.753 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.753 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.753 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.753 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.753 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.753 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.753 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.753 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.753 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.754 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.754 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.754 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.754 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.754 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.754 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.754 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.754 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.754 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.754 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.754 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.754 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.755 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.755 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.755 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.755 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.755 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.755 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.755 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.755 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.755 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.755 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.755 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.755 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.756 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.756 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.756 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.756 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.756 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.756 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.756 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.756 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.756 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.756 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.756 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.756 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.756 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.757 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.757 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.757 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.757 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.757 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.757 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.757 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.757 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.757 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.757 INFO:teuthology.orchestra.run.smithi096.stdout: "motd": { 2024-04-25T14:15:54.757 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "motd", 2024-04-25T14:15:54.757 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.757 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.758 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.758 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.758 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.758 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.758 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.758 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "The message of the day", 2024-04-25T14:15:54.758 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.758 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.758 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.758 INFO:teuthology.orchestra.run.smithi096.stdout: "redirect_resolve_ip_addr": { 2024-04-25T14:15:54.758 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T14:15:54.758 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.758 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.759 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.759 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.759 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.759 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.759 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.759 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.759 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.759 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.759 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.759 INFO:teuthology.orchestra.run.smithi096.stdout: "server_addr": { 2024-04-25T14:15:54.759 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_addr", 2024-04-25T14:15:54.759 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.759 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.760 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.760 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "::", 2024-04-25T14:15:54.760 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.760 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.760 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.760 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.760 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.760 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.760 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.760 INFO:teuthology.orchestra.run.smithi096.stdout: "server_port": { 2024-04-25T14:15:54.760 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_port", 2024-04-25T14:15:54.760 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.761 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.761 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.761 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8080", 2024-04-25T14:15:54.761 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.761 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.761 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.761 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.761 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.761 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.761 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.761 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.761 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.761 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.762 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.762 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.762 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.762 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.762 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.762 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.762 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.762 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.762 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.762 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.762 INFO:teuthology.orchestra.run.smithi096.stdout: "ssl": { 2024-04-25T14:15:54.762 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssl", 2024-04-25T14:15:54.763 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.763 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.763 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.763 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.763 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.763 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.763 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.763 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.763 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.763 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.763 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.763 INFO:teuthology.orchestra.run.smithi096.stdout: "ssl_server_port": { 2024-04-25T14:15:54.764 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssl_server_port", 2024-04-25T14:15:54.764 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.764 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.764 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.764 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8443", 2024-04-25T14:15:54.764 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.764 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.764 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.764 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.764 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.764 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.764 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.765 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_behaviour": { 2024-04-25T14:15:54.765 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_behaviour", 2024-04-25T14:15:54.765 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.765 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.765 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.765 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "redirect", 2024-04-25T14:15:54.765 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.765 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.765 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.765 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.765 INFO:teuthology.orchestra.run.smithi096.stdout: "redirect" 2024-04-25T14:15:54.765 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.765 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.766 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.766 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.766 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.766 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_error_status_code": { 2024-04-25T14:15:54.766 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_error_status_code", 2024-04-25T14:15:54.766 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.766 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.766 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.766 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "500", 2024-04-25T14:15:54.766 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "400", 2024-04-25T14:15:54.766 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "599", 2024-04-25T14:15:54.767 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.767 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.767 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.767 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.767 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.767 INFO:teuthology.orchestra.run.smithi096.stdout: "url_prefix": { 2024-04-25T14:15:54.767 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "url_prefix", 2024-04-25T14:15:54.767 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.767 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.767 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.767 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.768 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.768 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.768 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.768 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.768 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.768 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.768 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.768 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.768 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.768 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "devicehealth", 2024-04-25T14:15:54.769 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.769 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.769 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.769 INFO:teuthology.orchestra.run.smithi096.stdout: "enable_monitoring": { 2024-04-25T14:15:54.769 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enable_monitoring", 2024-04-25T14:15:54.769 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.769 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.769 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.769 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.769 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.769 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.770 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.770 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "monitor device health metrics", 2024-04-25T14:15:54.770 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.770 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.770 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.770 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.770 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.770 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.770 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.770 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.770 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.770 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.770 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.771 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.771 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.771 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.771 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.771 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.771 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.771 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.771 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.771 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.771 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.771 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.771 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.772 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.772 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.772 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.772 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.772 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.772 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.772 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.772 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.772 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.772 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.772 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.772 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.773 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.773 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.773 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.773 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.773 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.773 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.773 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.773 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.773 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.773 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.773 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.774 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.774 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.774 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.774 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.774 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.774 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.774 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.774 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.774 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.774 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.774 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.775 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.775 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.775 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.775 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.775 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.775 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.775 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.775 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.775 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.775 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.775 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.775 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.775 INFO:teuthology.orchestra.run.smithi096.stdout: "mark_out_threshold": { 2024-04-25T14:15:54.776 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mark_out_threshold", 2024-04-25T14:15:54.776 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:54.776 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.776 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.776 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "2419200", 2024-04-25T14:15:54.776 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.776 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.776 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.776 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T14:15:54.776 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.776 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.776 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.776 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_name": { 2024-04-25T14:15:54.777 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pool_name", 2024-04-25T14:15:54.777 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.777 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.777 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.777 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "device_health_metrics", 2024-04-25T14:15:54.777 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.777 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.777 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.777 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T14:15:54.777 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.777 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.777 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.777 INFO:teuthology.orchestra.run.smithi096.stdout: "retention_period": { 2024-04-25T14:15:54.777 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "retention_period", 2024-04-25T14:15:54.778 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:54.778 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.778 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.778 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "15552000", 2024-04-25T14:15:54.778 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.778 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.778 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.778 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how long to retain device health metrics", 2024-04-25T14:15:54.778 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.778 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.778 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.778 INFO:teuthology.orchestra.run.smithi096.stdout: "scrape_frequency": { 2024-04-25T14:15:54.779 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "scrape_frequency", 2024-04-25T14:15:54.779 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:54.779 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.779 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.779 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "86400", 2024-04-25T14:15:54.779 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.779 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.779 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.779 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T14:15:54.779 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.779 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.779 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.780 INFO:teuthology.orchestra.run.smithi096.stdout: "self_heal": { 2024-04-25T14:15:54.780 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "self_heal", 2024-04-25T14:15:54.780 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.780 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.780 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.780 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.780 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.780 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.780 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.780 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T14:15:54.780 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.780 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.780 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.781 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-04-25T14:15:54.781 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-04-25T14:15:54.781 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:54.781 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.781 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.781 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-04-25T14:15:54.781 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.781 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.781 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.781 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T14:15:54.781 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.781 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.781 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.782 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.782 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.782 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.782 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.782 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.782 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.782 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.782 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.782 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.782 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.782 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.783 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.783 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.783 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_threshold": { 2024-04-25T14:15:54.783 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_threshold", 2024-04-25T14:15:54.783 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:54.783 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.783 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.783 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "7257600", 2024-04-25T14:15:54.783 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.783 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.783 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.783 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T14:15:54.783 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.784 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.784 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.784 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.784 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.784 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "diskprediction_local", 2024-04-25T14:15:54.784 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.784 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.784 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.784 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.784 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.784 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.784 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.785 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.785 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.785 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.785 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.785 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.785 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.785 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.785 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.785 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.785 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.785 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.785 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.785 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.785 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.785 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.786 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.786 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.786 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.786 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.786 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.786 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.786 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.786 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.786 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.786 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.786 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.786 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.787 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.787 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.787 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.787 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.787 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.787 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.787 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.787 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.787 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.787 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.787 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.787 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.787 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.787 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.788 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.788 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.788 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.788 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.788 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.788 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.788 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.788 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.788 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.788 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.788 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.788 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.788 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.789 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.789 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.789 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.789 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.789 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.789 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.789 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.789 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.789 INFO:teuthology.orchestra.run.smithi096.stdout: "predict_interval": { 2024-04-25T14:15:54.789 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "predict_interval", 2024-04-25T14:15:54.789 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.789 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.789 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.789 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "86400", 2024-04-25T14:15:54.790 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.790 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.790 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.790 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.790 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.790 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.790 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.790 INFO:teuthology.orchestra.run.smithi096.stdout: "predictor_model": { 2024-04-25T14:15:54.790 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "predictor_model", 2024-04-25T14:15:54.790 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.790 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.790 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.790 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "prophetstor", 2024-04-25T14:15:54.791 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.791 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.791 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.791 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.791 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.791 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.791 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.791 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-04-25T14:15:54.791 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-04-25T14:15:54.791 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.791 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.791 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.792 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-04-25T14:15:54.792 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.792 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.792 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.792 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.792 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.792 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.792 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.792 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.792 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.792 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.792 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.792 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.793 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.793 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.793 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.793 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.793 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.793 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.793 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.793 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.793 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.793 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.793 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "influx", 2024-04-25T14:15:54.793 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": false, 2024-04-25T14:15:54.794 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "influxdb python module not found", 2024-04-25T14:15:54.794 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.794 INFO:teuthology.orchestra.run.smithi096.stdout: "batch_size": { 2024-04-25T14:15:54.794 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "batch_size", 2024-04-25T14:15:54.794 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.794 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.794 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.794 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5000", 2024-04-25T14:15:54.794 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.794 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.794 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.794 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T14:15:54.794 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.794 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.795 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.795 INFO:teuthology.orchestra.run.smithi096.stdout: "database": { 2024-04-25T14:15:54.795 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "database", 2024-04-25T14:15:54.795 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.795 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.795 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.795 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "ceph", 2024-04-25T14:15:54.795 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.795 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.795 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.795 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T14:15:54.795 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.796 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.796 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.796 INFO:teuthology.orchestra.run.smithi096.stdout: "hostname": { 2024-04-25T14:15:54.796 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "hostname", 2024-04-25T14:15:54.796 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.796 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.796 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.796 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.796 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.796 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.796 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.796 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "InfluxDB server hostname", 2024-04-25T14:15:54.797 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.797 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.797 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.797 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-04-25T14:15:54.797 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-04-25T14:15:54.797 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:54.797 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.797 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.797 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "30", 2024-04-25T14:15:54.797 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "5", 2024-04-25T14:15:54.797 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.797 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.797 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T14:15:54.798 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.798 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.798 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.798 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.798 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.798 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.798 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.798 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.798 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.798 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.798 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.798 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.798 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.799 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.799 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.799 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.799 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.799 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.799 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.799 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.799 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.799 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.799 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.799 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.799 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.799 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.799 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.800 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.800 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.800 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.800 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.800 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.800 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.800 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.800 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.800 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.800 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.800 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.800 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.800 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.801 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.801 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.801 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.801 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.801 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.801 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.801 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.801 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.801 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.801 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.801 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.801 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.801 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.801 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.801 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.802 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.802 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.802 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.802 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.802 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.802 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.802 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.802 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.802 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.802 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.802 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.802 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.802 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.803 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.803 INFO:teuthology.orchestra.run.smithi096.stdout: "password": { 2024-04-25T14:15:54.803 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "password", 2024-04-25T14:15:54.803 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.803 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.803 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.803 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.803 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.803 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.803 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.803 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "password of InfluxDB server user", 2024-04-25T14:15:54.803 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.803 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.803 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.804 INFO:teuthology.orchestra.run.smithi096.stdout: "port": { 2024-04-25T14:15:54.804 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "port", 2024-04-25T14:15:54.804 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.804 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.804 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.804 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8086", 2024-04-25T14:15:54.804 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.804 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.804 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.804 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "InfluxDB server port", 2024-04-25T14:15:54.804 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.804 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.804 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.805 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.805 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.805 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.805 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.805 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.805 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.805 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.805 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.805 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.805 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.805 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.805 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.805 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.806 INFO:teuthology.orchestra.run.smithi096.stdout: "ssl": { 2024-04-25T14:15:54.806 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssl", 2024-04-25T14:15:54.806 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.806 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.806 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.806 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "false", 2024-04-25T14:15:54.806 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.806 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.806 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.806 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T14:15:54.806 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.806 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.806 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.807 INFO:teuthology.orchestra.run.smithi096.stdout: "threads": { 2024-04-25T14:15:54.807 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "threads", 2024-04-25T14:15:54.807 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.807 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.807 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.807 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-04-25T14:15:54.807 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-04-25T14:15:54.807 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "32", 2024-04-25T14:15:54.807 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.807 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T14:15:54.807 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.807 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.808 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.808 INFO:teuthology.orchestra.run.smithi096.stdout: "username": { 2024-04-25T14:15:54.808 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "username", 2024-04-25T14:15:54.808 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.808 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.808 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.808 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.808 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.808 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.809 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.809 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "username of InfluxDB server user", 2024-04-25T14:15:54.809 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.809 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.809 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.809 INFO:teuthology.orchestra.run.smithi096.stdout: "verify_ssl": { 2024-04-25T14:15:54.809 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "verify_ssl", 2024-04-25T14:15:54.809 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.809 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.810 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.810 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "true", 2024-04-25T14:15:54.810 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.810 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.810 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.810 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T14:15:54.810 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.810 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.810 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.810 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.810 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.810 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "insights", 2024-04-25T14:15:54.811 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.811 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.811 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.811 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.811 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.811 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.811 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.811 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.811 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.811 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.811 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.811 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.811 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.811 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.812 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.812 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.812 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.812 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.812 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.812 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.812 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.812 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.812 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.812 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.812 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.812 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.812 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.812 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.813 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.813 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.813 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.813 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.813 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.813 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.813 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.813 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.813 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.813 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.813 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.813 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.813 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.814 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.814 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.814 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.814 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.814 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.814 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.814 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.814 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.814 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.814 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.814 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.814 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.814 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.814 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.815 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.815 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.815 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.815 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.815 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.815 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.815 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.815 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.815 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.815 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.815 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.815 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.815 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.816 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.816 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.816 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.816 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.816 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.816 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.816 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.816 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.816 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.816 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.816 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.816 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.816 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.817 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.817 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.817 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.817 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "iostat", 2024-04-25T14:15:54.817 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.817 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.817 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.817 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.817 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.817 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.817 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.817 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.817 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.818 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.818 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.818 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.818 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.818 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.818 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.818 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.818 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.818 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.818 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.818 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.818 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.818 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.818 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.819 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.819 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.819 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.819 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.819 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.819 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.819 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.819 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.819 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.819 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.819 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.819 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.820 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.820 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.820 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.820 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.820 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.820 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.820 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.820 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.820 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.820 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.820 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.820 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.820 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.821 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.821 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.821 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.821 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.821 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.821 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.821 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.821 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.821 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.821 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.821 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.821 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.821 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.821 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.822 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.822 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.822 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.822 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.822 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.822 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.822 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.822 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.822 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.822 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.822 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.822 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.823 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.823 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.823 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.823 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.823 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.823 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.823 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.823 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.823 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.823 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.823 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "k8sevents", 2024-04-25T14:15:54.823 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.824 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.824 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.824 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_event_retention_days": { 2024-04-25T14:15:54.824 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ceph_event_retention_days", 2024-04-25T14:15:54.824 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.824 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.824 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.824 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "7", 2024-04-25T14:15:54.824 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.824 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.824 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.824 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T14:15:54.825 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.825 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.825 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.825 INFO:teuthology.orchestra.run.smithi096.stdout: "config_check_secs": { 2024-04-25T14:15:54.825 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "config_check_secs", 2024-04-25T14:15:54.825 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.825 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.825 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.825 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T14:15:54.825 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "10", 2024-04-25T14:15:54.825 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.825 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.825 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T14:15:54.826 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.826 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.826 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.826 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.826 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.826 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.826 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.826 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.826 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.826 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.826 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.826 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.826 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.827 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.827 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.827 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.827 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.827 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.827 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.827 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.827 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.827 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.827 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.827 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.827 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.827 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.828 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.828 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.828 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.828 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.828 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.828 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.828 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.828 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.828 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.828 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.828 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.828 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.828 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.829 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.829 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.829 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.829 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.829 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.829 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.829 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.829 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.829 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.829 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.829 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.829 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.829 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.829 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.830 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.830 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.830 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.830 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.830 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.830 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.830 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.830 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.830 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.830 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.830 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.830 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.830 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.831 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.831 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.831 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.831 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.831 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.831 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.831 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.831 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.831 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.831 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.831 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.831 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.832 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.832 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.832 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.832 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.832 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.832 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.832 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "localpool", 2024-04-25T14:15:54.832 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.832 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.832 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.832 INFO:teuthology.orchestra.run.smithi096.stdout: "failure_domain": { 2024-04-25T14:15:54.832 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "failure_domain", 2024-04-25T14:15:54.832 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.833 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.833 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.833 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "host", 2024-04-25T14:15:54.833 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.833 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.833 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.833 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "failure domain for any created local pool", 2024-04-25T14:15:54.833 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T14:15:54.833 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.833 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.833 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.833 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.834 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.834 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.834 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.834 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.834 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.834 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.834 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.834 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.834 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.834 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.834 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.834 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.834 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.834 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.835 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.835 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.835 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.835 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.835 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.835 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.835 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.835 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.835 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.835 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.835 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.835 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.835 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.836 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.836 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.836 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.836 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.836 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.836 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.836 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.836 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.836 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.836 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.836 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.836 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.836 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.837 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.837 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.837 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.837 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.837 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.837 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.837 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.837 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.837 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.837 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.837 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.837 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.837 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.838 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.838 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.838 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.838 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.838 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.838 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.838 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.838 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.838 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.838 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.838 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.838 INFO:teuthology.orchestra.run.smithi096.stdout: "min_size": { 2024-04-25T14:15:54.838 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "min_size", 2024-04-25T14:15:54.839 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.839 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.839 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.839 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.839 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.839 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.839 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.839 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "default min_size for any created local pool", 2024-04-25T14:15:54.839 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T14:15:54.839 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.839 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.839 INFO:teuthology.orchestra.run.smithi096.stdout: "num_rep": { 2024-04-25T14:15:54.840 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "num_rep", 2024-04-25T14:15:54.840 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.840 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.840 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.840 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "3", 2024-04-25T14:15:54.840 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.840 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.840 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.840 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "default replica count for any created local pool", 2024-04-25T14:15:54.840 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.840 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.840 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.841 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num": { 2024-04-25T14:15:54.841 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pg_num", 2024-04-25T14:15:54.841 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.841 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.841 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.841 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "128", 2024-04-25T14:15:54.841 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.841 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.841 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.841 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T14:15:54.841 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.841 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.841 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.842 INFO:teuthology.orchestra.run.smithi096.stdout: "prefix": { 2024-04-25T14:15:54.842 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "prefix", 2024-04-25T14:15:54.842 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.842 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.842 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.842 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.842 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.842 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.842 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.842 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "name prefix for any created local pool", 2024-04-25T14:15:54.842 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.842 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.843 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.843 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.843 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.843 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.843 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.843 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.843 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.843 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.843 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.843 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.844 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.844 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.844 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.844 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.844 INFO:teuthology.orchestra.run.smithi096.stdout: "subtree": { 2024-04-25T14:15:54.844 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "subtree", 2024-04-25T14:15:54.844 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.844 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.844 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.844 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "rack", 2024-04-25T14:15:54.845 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.845 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.845 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.845 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T14:15:54.845 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T14:15:54.845 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.845 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.845 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.845 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.845 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mds_autoscaler", 2024-04-25T14:15:54.845 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.846 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.846 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.846 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.846 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.846 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.846 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.846 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.846 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.846 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.846 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.846 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.846 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.846 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.846 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.847 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.847 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.847 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.847 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.847 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.847 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.847 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.847 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.847 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.847 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.847 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.847 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.847 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.847 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.848 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.848 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.848 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.848 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.848 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.848 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.848 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.848 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.848 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.848 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.848 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.848 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.848 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.848 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.849 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.849 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.849 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.849 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.849 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.849 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.849 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.849 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.849 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.849 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.849 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.849 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.849 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.850 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.850 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.850 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.850 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.850 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.850 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.850 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.850 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.850 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.850 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.850 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.850 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.850 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.851 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.851 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.851 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.851 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.851 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.851 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.851 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.851 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.851 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.851 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.851 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.851 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.851 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.851 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.851 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.852 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mirroring", 2024-04-25T14:15:54.852 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.852 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.852 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.852 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.852 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.852 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.852 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.852 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.852 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.852 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.852 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.853 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.853 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.853 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.853 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.853 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.853 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.853 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.853 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.853 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.853 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.853 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.853 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.853 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.854 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.854 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.854 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.854 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.854 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.854 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.854 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.854 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.854 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.854 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.854 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.854 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.854 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.855 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.855 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.855 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.855 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.855 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.855 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.855 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.855 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.855 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.855 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.855 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.855 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.855 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.855 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.855 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.856 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.856 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.856 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.856 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.856 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.856 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.856 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.856 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.856 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.856 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.856 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.856 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.856 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.856 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.857 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.857 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.857 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.857 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.857 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.857 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.857 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.857 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.857 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.857 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.857 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.857 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.857 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.857 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.858 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.858 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.858 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.858 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.858 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "nfs", 2024-04-25T14:15:54.858 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.858 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.858 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.858 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.858 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.858 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.859 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.859 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.859 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.859 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.859 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.859 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.859 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.859 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.859 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.859 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.859 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.859 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.859 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.859 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.860 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.860 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.860 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.860 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.860 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.860 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.860 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.860 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.860 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.860 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.860 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.860 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.860 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.860 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.861 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.861 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.861 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.861 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.861 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.861 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.861 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.861 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.861 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.861 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.861 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.861 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.861 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.862 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.862 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.862 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.862 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.862 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.862 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.862 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.862 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.862 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.862 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.862 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.862 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.862 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.863 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.863 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.863 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.863 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.863 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.863 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.863 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.863 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.863 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.863 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.863 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.863 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.863 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.863 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.864 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.864 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.864 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.864 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.864 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.864 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.864 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.864 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.864 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.864 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.864 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "orchestrator", 2024-04-25T14:15:54.864 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.865 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.865 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.865 INFO:teuthology.orchestra.run.smithi096.stdout: "fail_fs": { 2024-04-25T14:15:54.865 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "fail_fs", 2024-04-25T14:15:54.865 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.865 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.865 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.865 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.865 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.865 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.865 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.865 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T14:15:54.865 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.865 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.866 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.866 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.866 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.866 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.866 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.866 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.866 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.866 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.866 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.866 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.866 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.866 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.866 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.867 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.867 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.867 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.867 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.867 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.867 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.867 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.867 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.867 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.867 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.867 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.867 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.867 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.868 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.868 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.868 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.868 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.868 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.868 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.868 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.868 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.868 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.868 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.868 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.868 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.868 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.868 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.869 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.869 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.869 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.869 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.869 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.869 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.869 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.869 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.869 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.869 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.869 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.869 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.869 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.870 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.870 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.870 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.870 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.870 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.870 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.870 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.870 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.870 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.870 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.870 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.870 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.871 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.871 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.871 INFO:teuthology.orchestra.run.smithi096.stdout: "orchestrator": { 2024-04-25T14:15:54.871 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "orchestrator", 2024-04-25T14:15:54.871 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.871 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.871 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.871 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.871 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.871 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.871 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.871 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm", 2024-04-25T14:15:54.871 INFO:teuthology.orchestra.run.smithi096.stdout: "rook", 2024-04-25T14:15:54.872 INFO:teuthology.orchestra.run.smithi096.stdout: "test_orchestrator" 2024-04-25T14:15:54.872 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.872 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Orchestrator backend", 2024-04-25T14:15:54.872 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.872 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.872 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.872 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.872 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.872 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.872 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.872 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.872 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.872 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.873 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.873 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.873 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.873 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.873 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.873 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.873 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.873 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.873 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "osd_perf_query", 2024-04-25T14:15:54.873 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.873 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.873 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.874 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.874 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.874 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.874 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.874 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.874 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.874 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.874 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.874 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.874 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.874 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.875 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.875 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.875 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.875 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.875 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.875 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.875 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.875 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.875 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.875 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.875 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.875 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.875 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.876 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.876 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.876 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.876 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.876 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.876 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.876 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.876 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.876 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.876 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.876 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.876 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.877 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.877 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.877 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.877 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.877 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.877 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.877 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.877 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.877 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.877 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.877 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.877 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.877 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.877 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.877 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.878 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.878 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.878 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.878 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.878 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.878 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.878 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.878 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.878 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.878 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.878 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.879 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.879 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.879 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.879 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.879 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.879 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.879 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.879 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.879 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.879 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.879 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.880 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.880 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.880 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.880 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.880 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.880 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.880 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.880 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.880 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "osd_support", 2024-04-25T14:15:54.880 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.880 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.880 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.880 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.881 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.881 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.881 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.881 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.881 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.881 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.881 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.881 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.881 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.881 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.881 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.881 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.881 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.881 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.882 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.882 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.882 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.882 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.882 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.882 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.882 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.882 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.882 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.882 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.882 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.882 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.883 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.883 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.883 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.883 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.883 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.883 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.883 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.883 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.883 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.883 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.883 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.884 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.884 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.884 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.884 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.884 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.884 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.884 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.884 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.884 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.884 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.884 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.884 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.884 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.885 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.885 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.885 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.885 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.885 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.885 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.885 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.885 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.885 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.885 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.885 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.885 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.885 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.886 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.886 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.886 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.886 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.886 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.886 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.886 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.886 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.886 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.886 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.886 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.886 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.886 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.887 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.887 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.887 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.887 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.887 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pg_autoscaler", 2024-04-25T14:15:54.887 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.887 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.887 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.887 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.887 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.887 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.887 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.888 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.888 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.888 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.888 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.888 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.888 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.888 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.888 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.888 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.888 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.888 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.888 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.888 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.888 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.888 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.889 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.889 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.889 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.889 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.889 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.889 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.889 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.889 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.889 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.889 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.889 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.889 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.889 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.890 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.890 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.890 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.890 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.890 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.890 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.890 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.890 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.890 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.890 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.890 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.890 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.891 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.891 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.891 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.891 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.891 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.891 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.891 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.891 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.891 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.891 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.891 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.891 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.891 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.892 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.892 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.892 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.892 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.892 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.892 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.892 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.892 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.892 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.892 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-04-25T14:15:54.892 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-04-25T14:15:54.892 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:54.892 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.893 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.893 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-04-25T14:15:54.893 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.893 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.893 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.893 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.893 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.893 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.893 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.893 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.893 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.893 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.893 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.893 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.894 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.894 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.894 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.894 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.894 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.894 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.894 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.894 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.894 INFO:teuthology.orchestra.run.smithi096.stdout: "threshold": { 2024-04-25T14:15:54.894 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "threshold", 2024-04-25T14:15:54.894 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T14:15:54.894 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.895 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.895 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "3.0", 2024-04-25T14:15:54.895 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1.0", 2024-04-25T14:15:54.895 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.895 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.895 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "scaling threshold", 2024-04-25T14:15:54.895 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T14:15:54.895 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.895 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.895 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.895 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.895 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "progress", 2024-04-25T14:15:54.896 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.896 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.896 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.896 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_pg_recovery_event": { 2024-04-25T14:15:54.896 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "allow_pg_recovery_event", 2024-04-25T14:15:54.896 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.896 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.896 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.896 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.896 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.896 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.896 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.896 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T14:15:54.896 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.897 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.897 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.897 INFO:teuthology.orchestra.run.smithi096.stdout: "enabled": { 2024-04-25T14:15:54.897 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enabled", 2024-04-25T14:15:54.897 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.897 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.897 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.897 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.897 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.897 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.897 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.897 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.897 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.897 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.898 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.898 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.898 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.898 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.898 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.898 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.898 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.898 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.898 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.898 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.898 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.898 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.899 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.899 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.899 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.899 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.899 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.899 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.899 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.899 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.899 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.899 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.899 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.899 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.899 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.899 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.900 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.900 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.900 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.900 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.900 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.900 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.900 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.900 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.900 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.900 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.900 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.901 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.901 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.901 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.901 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.901 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.901 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.901 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.901 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.901 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.901 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.901 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.901 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.901 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.901 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.902 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.902 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.902 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.902 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.902 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.902 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.902 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.902 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.902 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.902 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.902 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.902 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.902 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.903 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.903 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.903 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.903 INFO:teuthology.orchestra.run.smithi096.stdout: "max_completed_events": { 2024-04-25T14:15:54.903 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_completed_events", 2024-04-25T14:15:54.903 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.903 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.903 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.903 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "50", 2024-04-25T14:15:54.903 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.903 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.904 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.904 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "number of past completed events to remember", 2024-04-25T14:15:54.904 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.904 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.904 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.904 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-04-25T14:15:54.904 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-04-25T14:15:54.904 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:54.904 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.904 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.904 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-04-25T14:15:54.904 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.905 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.905 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.905 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how long the module is going to sleep", 2024-04-25T14:15:54.905 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.905 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.905 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.905 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.905 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.905 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.905 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.905 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.905 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.905 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.906 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.906 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.906 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.906 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.906 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.906 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.906 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.906 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.906 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "prometheus", 2024-04-25T14:15:54.906 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.906 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.907 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.907 INFO:teuthology.orchestra.run.smithi096.stdout: "cache": { 2024-04-25T14:15:54.907 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cache", 2024-04-25T14:15:54.907 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.907 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.907 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.907 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.907 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.907 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.907 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.907 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.907 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.907 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.907 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.907 INFO:teuthology.orchestra.run.smithi096.stdout: "exclude_perf_counters": { 2024-04-25T14:15:54.908 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "exclude_perf_counters", 2024-04-25T14:15:54.908 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.908 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.908 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.908 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.908 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.908 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.908 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.908 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T14:15:54.908 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T14:15:54.908 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.908 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.908 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.908 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.909 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.909 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.909 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.909 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.909 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.909 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.909 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.909 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.909 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.909 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.909 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.909 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.909 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.910 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.910 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.910 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.910 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.910 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.910 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.910 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.910 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.910 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.910 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.910 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.910 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.911 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.911 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.911 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.911 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.911 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.911 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.911 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.911 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.911 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.911 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.911 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.911 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.912 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.912 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.912 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.912 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.912 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.912 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.912 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.912 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.912 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.912 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.912 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.912 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.912 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.913 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.913 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.913 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.913 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.913 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.913 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.913 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.913 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.913 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.913 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.913 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.913 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.914 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.914 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.914 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_stats_pools": { 2024-04-25T14:15:54.914 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_stats_pools", 2024-04-25T14:15:54.914 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.914 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.914 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.914 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.914 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.914 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.914 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.914 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.914 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.915 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.915 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.915 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T14:15:54.915 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T14:15:54.915 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.915 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.915 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.915 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "300", 2024-04-25T14:15:54.915 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.915 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.915 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.915 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.915 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.916 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.916 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.916 INFO:teuthology.orchestra.run.smithi096.stdout: "scrape_interval": { 2024-04-25T14:15:54.916 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "scrape_interval", 2024-04-25T14:15:54.916 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T14:15:54.916 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.916 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.916 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "15.0", 2024-04-25T14:15:54.916 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.916 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.917 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.917 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.917 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.917 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.917 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.917 INFO:teuthology.orchestra.run.smithi096.stdout: "server_addr": { 2024-04-25T14:15:54.917 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_addr", 2024-04-25T14:15:54.917 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.917 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.917 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.917 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "::", 2024-04-25T14:15:54.918 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.918 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.918 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.918 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T14:15:54.918 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.918 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.918 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.918 INFO:teuthology.orchestra.run.smithi096.stdout: "server_port": { 2024-04-25T14:15:54.918 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_port", 2024-04-25T14:15:54.918 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.918 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.918 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.919 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "9283", 2024-04-25T14:15:54.919 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.919 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.919 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.919 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T14:15:54.919 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.919 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.919 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.919 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.919 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.919 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.919 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.919 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.920 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.920 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.920 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.920 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.920 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.920 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.920 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.920 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.920 INFO:teuthology.orchestra.run.smithi096.stdout: "stale_cache_strategy": { 2024-04-25T14:15:54.920 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "stale_cache_strategy", 2024-04-25T14:15:54.920 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.920 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.920 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.921 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "log", 2024-04-25T14:15:54.921 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.921 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.921 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.921 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.921 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.921 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.921 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.921 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_behaviour": { 2024-04-25T14:15:54.921 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_behaviour", 2024-04-25T14:15:54.921 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.921 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.922 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.922 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "default", 2024-04-25T14:15:54.922 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.922 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.922 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.922 INFO:teuthology.orchestra.run.smithi096.stdout: "default", 2024-04-25T14:15:54.922 INFO:teuthology.orchestra.run.smithi096.stdout: "error" 2024-04-25T14:15:54.922 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.922 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.922 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.922 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.922 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.923 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_error_status_code": { 2024-04-25T14:15:54.923 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_error_status_code", 2024-04-25T14:15:54.923 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.923 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.923 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.923 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "500", 2024-04-25T14:15:54.923 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "400", 2024-04-25T14:15:54.923 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "599", 2024-04-25T14:15:54.923 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.923 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.923 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.923 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.923 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.924 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.924 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.924 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_support", 2024-04-25T14:15:54.924 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.924 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.924 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.924 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.924 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.924 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.924 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.924 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.924 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.925 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.925 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.925 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.925 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.925 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.925 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.925 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.925 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.925 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.925 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.925 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.925 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.925 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.925 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.926 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.926 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.926 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.926 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.926 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.926 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.926 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.926 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.926 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.926 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.926 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.926 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.926 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.927 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.927 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.927 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.927 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.927 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.927 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.927 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.927 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.927 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.927 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.927 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.927 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.927 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.928 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.928 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.928 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.928 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.928 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.928 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.928 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.928 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.928 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.928 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.928 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.928 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.928 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.928 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.929 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.929 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.929 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.929 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.929 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.929 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.929 INFO:teuthology.orchestra.run.smithi096.stdout: "max_concurrent_snap_create": { 2024-04-25T14:15:54.929 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_concurrent_snap_create", 2024-04-25T14:15:54.929 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.929 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.929 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.929 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T14:15:54.929 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.930 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.930 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.930 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.930 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.930 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.930 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.930 INFO:teuthology.orchestra.run.smithi096.stdout: "mirror_snapshot_schedule": { 2024-04-25T14:15:54.930 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T14:15:54.930 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.930 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.930 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.930 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.930 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.931 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.931 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.931 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.931 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.931 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.931 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.931 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.931 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.931 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.931 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.931 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.931 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.932 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.932 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.932 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.932 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.932 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.932 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.932 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.932 INFO:teuthology.orchestra.run.smithi096.stdout: "trash_purge_schedule": { 2024-04-25T14:15:54.932 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "trash_purge_schedule", 2024-04-25T14:15:54.932 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.932 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.932 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.933 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.933 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.933 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.933 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.933 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.933 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.933 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.933 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.933 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.933 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.933 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "restful", 2024-04-25T14:15:54.933 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.934 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.934 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.934 INFO:teuthology.orchestra.run.smithi096.stdout: "enable_auth": { 2024-04-25T14:15:54.934 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enable_auth", 2024-04-25T14:15:54.934 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.934 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.934 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.934 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.934 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.934 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.934 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.934 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.934 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.934 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.935 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.935 INFO:teuthology.orchestra.run.smithi096.stdout: "key_file": { 2024-04-25T14:15:54.935 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "key_file", 2024-04-25T14:15:54.935 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.935 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.935 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.935 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.935 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.935 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.935 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.935 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.935 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.936 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.936 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.936 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.936 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.936 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.936 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.936 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.936 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.936 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.936 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.937 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.937 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.937 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.937 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.937 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.937 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.937 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.937 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.937 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.937 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.937 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.937 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.938 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.938 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.938 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.938 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.938 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.938 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.938 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.938 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.938 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.938 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.938 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.938 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.939 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.939 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.939 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.939 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.939 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.939 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.939 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.939 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.939 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.939 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.939 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.939 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.940 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.940 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.940 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.940 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.940 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.940 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.940 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.940 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.940 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.940 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.940 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.940 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.941 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.941 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.941 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.941 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.941 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.941 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.941 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.941 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.941 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.941 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.941 INFO:teuthology.orchestra.run.smithi096.stdout: "server_addr": { 2024-04-25T14:15:54.941 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_addr", 2024-04-25T14:15:54.941 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.942 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.942 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.942 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.942 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.942 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.942 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.942 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.942 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.942 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.942 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.942 INFO:teuthology.orchestra.run.smithi096.stdout: "server_port": { 2024-04-25T14:15:54.942 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_port", 2024-04-25T14:15:54.942 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.942 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.943 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.943 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.943 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.943 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.943 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.943 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.943 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.943 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.943 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.943 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.943 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.943 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.943 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.944 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.944 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.944 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.944 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.944 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.944 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.944 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.944 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.944 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.944 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.944 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.944 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rgw", 2024-04-25T14:15:54.945 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.945 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.945 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.945 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.945 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.945 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.945 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.945 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.945 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.945 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.945 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.945 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.945 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.946 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.946 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.946 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.946 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.946 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.946 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.946 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.946 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.946 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.946 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.946 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.946 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.947 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.947 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.947 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.947 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.947 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.947 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.947 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.947 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.947 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.947 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.947 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.947 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.948 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.948 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.948 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.948 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.948 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.948 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.948 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.948 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.948 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.948 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.948 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.948 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.948 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.949 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.949 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.949 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.949 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.949 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.949 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.949 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.949 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.949 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.949 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.949 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.949 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.950 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.950 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.950 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.950 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.950 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.950 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.950 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.950 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.950 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.950 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.950 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.950 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.951 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.951 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.951 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.951 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.951 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.951 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.951 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.951 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.951 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.951 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.951 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rook", 2024-04-25T14:15:54.952 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.952 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.952 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.952 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.952 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.952 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.952 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.952 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.952 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.952 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.952 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.952 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.952 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.952 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.953 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.953 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.953 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.953 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.953 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.953 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.953 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.953 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.953 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.953 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.953 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.953 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.953 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.954 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.954 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.954 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.954 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.954 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.954 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.954 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.954 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.954 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.954 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.954 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.954 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.954 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.955 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.955 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.955 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.955 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.955 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.955 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.955 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.955 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.955 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.955 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.955 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.955 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.955 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.955 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.955 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.956 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.956 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.956 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.956 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.956 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.956 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.956 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.956 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.956 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.956 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.956 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.956 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.956 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.956 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.957 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.957 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.957 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.957 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.957 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.957 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.957 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.957 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.957 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.957 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.957 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.957 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.958 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.958 INFO:teuthology.orchestra.run.smithi096.stdout: "storage_class": { 2024-04-25T14:15:54.958 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "storage_class", 2024-04-25T14:15:54.958 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.958 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.958 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.958 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "local", 2024-04-25T14:15:54.958 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.958 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.958 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.958 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T14:15:54.958 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.958 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.959 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.959 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.959 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.959 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "selftest", 2024-04-25T14:15:54.959 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.959 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.959 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.959 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.959 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.959 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.959 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.959 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.960 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.960 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.960 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.960 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.960 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.960 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.960 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.960 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.960 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.960 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.960 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.960 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.960 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.960 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.961 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.961 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.961 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.961 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.961 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.961 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.961 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.961 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.961 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.961 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.961 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.961 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.961 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.962 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.962 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.962 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.962 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.962 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.962 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.962 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.962 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.962 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.962 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.962 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.962 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.962 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.963 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.963 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.963 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.963 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.963 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.963 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.963 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.963 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.963 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.963 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.963 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.963 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.963 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.964 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.964 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.964 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.964 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.964 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.964 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.964 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.964 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.964 INFO:teuthology.orchestra.run.smithi096.stdout: "roption1": { 2024-04-25T14:15:54.964 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "roption1", 2024-04-25T14:15:54.964 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.964 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.964 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.964 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.965 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.965 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.965 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.965 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.965 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.965 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.965 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.965 INFO:teuthology.orchestra.run.smithi096.stdout: "roption2": { 2024-04-25T14:15:54.965 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "roption2", 2024-04-25T14:15:54.965 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.965 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.965 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.966 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "xyz", 2024-04-25T14:15:54.966 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.966 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.966 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.966 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.966 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.966 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.966 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.966 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption1": { 2024-04-25T14:15:54.966 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption1", 2024-04-25T14:15:54.966 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.966 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.967 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.967 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.967 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.967 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.967 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.967 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.967 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.967 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.967 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.967 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption2": { 2024-04-25T14:15:54.967 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption2", 2024-04-25T14:15:54.967 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.967 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.968 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.968 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.968 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.968 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.968 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.968 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.968 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.968 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.968 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.968 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption3": { 2024-04-25T14:15:54.968 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption3", 2024-04-25T14:15:54.968 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T14:15:54.968 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.969 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.969 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.969 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.969 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.969 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.969 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.969 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.969 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.969 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.969 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption4": { 2024-04-25T14:15:54.969 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption4", 2024-04-25T14:15:54.969 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.970 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.970 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.970 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.970 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.970 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.970 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.970 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.970 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.970 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.970 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.970 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption5": { 2024-04-25T14:15:54.970 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption5", 2024-04-25T14:15:54.971 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.971 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.971 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.971 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.971 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.971 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.971 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.971 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.971 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.971 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.971 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.971 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption6": { 2024-04-25T14:15:54.971 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption6", 2024-04-25T14:15:54.971 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.972 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.972 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.972 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:54.972 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.972 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.972 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.972 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.972 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.972 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.972 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.972 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption7": { 2024-04-25T14:15:54.972 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption7", 2024-04-25T14:15:54.973 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.973 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.973 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.973 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.973 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-04-25T14:15:54.973 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "42", 2024-04-25T14:15:54.973 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.973 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.973 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.973 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.973 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.973 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.974 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.974 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.974 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.974 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.974 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.974 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.974 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.974 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.974 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.974 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.974 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.974 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.975 INFO:teuthology.orchestra.run.smithi096.stdout: "testkey": { 2024-04-25T14:15:54.975 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "testkey", 2024-04-25T14:15:54.975 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.975 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.975 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.975 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.975 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.975 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.975 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.975 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.975 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.975 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.975 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.975 INFO:teuthology.orchestra.run.smithi096.stdout: "testlkey": { 2024-04-25T14:15:54.976 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "testlkey", 2024-04-25T14:15:54.976 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.976 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.976 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.976 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.976 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.976 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.976 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.976 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.976 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.976 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.976 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.976 INFO:teuthology.orchestra.run.smithi096.stdout: "testnewline": { 2024-04-25T14:15:54.977 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "testnewline", 2024-04-25T14:15:54.977 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.977 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.977 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:54.977 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.977 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.977 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.977 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.977 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.977 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.977 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.977 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.978 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.978 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.978 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snap_schedule", 2024-04-25T14:15:54.978 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.978 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.978 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.978 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_m_granularity": { 2024-04-25T14:15:54.978 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "allow_m_granularity", 2024-04-25T14:15:54.978 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.978 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.978 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.979 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.979 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.979 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.979 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.979 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T14:15:54.979 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.979 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.979 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.979 INFO:teuthology.orchestra.run.smithi096.stdout: "dump_on_update": { 2024-04-25T14:15:54.979 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "dump_on_update", 2024-04-25T14:15:54.979 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.979 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.979 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.980 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.980 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.980 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.980 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.980 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "dump database to debug log on update", 2024-04-25T14:15:54.980 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.980 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.980 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.980 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.980 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.980 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.980 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.980 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.981 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.981 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.981 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.981 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.981 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.981 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.981 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.981 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.981 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.981 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.981 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.981 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.981 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.981 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.982 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.982 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.982 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.982 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.982 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.982 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.982 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.982 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.982 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.982 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.982 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.982 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.982 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.983 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.983 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.983 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.983 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.983 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.983 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.983 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.983 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.983 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.983 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.983 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.983 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.983 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.984 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.984 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.984 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.984 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.984 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.984 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.984 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.984 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.984 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.984 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.984 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.984 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.985 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.985 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.985 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.985 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.985 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.985 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.985 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.985 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.985 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.985 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.985 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.985 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.986 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.986 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.986 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.986 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.986 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.986 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.986 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.986 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.986 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.986 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.986 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.986 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.987 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "stats", 2024-04-25T14:15:54.987 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.987 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.987 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.987 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.987 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.987 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.987 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.987 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.987 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.987 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.987 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.987 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.988 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.988 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.988 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.988 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.988 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.988 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.988 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.988 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.988 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.988 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.988 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.988 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.988 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.989 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.989 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.989 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.989 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.989 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.989 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.989 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.989 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.989 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.989 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.989 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.990 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.990 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.990 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.990 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.990 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.990 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.990 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.990 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.990 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.990 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.990 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.990 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.990 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.990 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.991 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.991 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.991 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.991 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.991 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.991 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.991 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.991 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.991 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.991 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.991 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.991 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.991 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.992 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.992 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.992 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.992 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.992 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.992 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.992 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.992 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.992 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.992 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.992 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.992 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.992 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.993 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.993 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.993 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.993 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.993 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.993 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.993 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.993 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:54.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:54.993 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "status", 2024-04-25T14:15:54.993 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:54.993 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:54.993 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:54.994 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:54.994 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:54.994 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.994 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.994 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.994 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:54.994 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.994 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.994 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.994 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.994 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.994 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.994 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.994 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.995 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.995 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.995 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.995 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.995 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.995 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.995 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:54.995 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:54.995 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.995 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.995 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.995 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.995 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.995 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.996 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.996 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.996 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.996 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.996 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.996 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:54.996 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:54.996 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:54.996 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.996 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.996 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:54.996 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.996 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.997 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:54.997 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:54.997 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:54.997 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:54.997 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:54.997 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:54.997 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:54.997 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:54.997 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.997 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.997 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.997 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.997 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:54.998 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:54.998 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:54.998 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:54.998 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.998 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:54.998 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.998 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.998 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.998 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.998 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.998 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.998 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:54.998 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:54.999 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:54.999 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:54.999 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:54.999 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:54.999 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:54.999 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:54.999 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:54.999 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:54.999 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:54.999 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:54.999 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:54.999 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:54.999 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:55.000 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:55.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:55.000 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "telegraf", 2024-04-25T14:15:55.000 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:55.000 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:55.000 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:55.000 INFO:teuthology.orchestra.run.smithi096.stdout: "address": { 2024-04-25T14:15:55.000 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "address", 2024-04-25T14:15:55.000 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:55.000 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.000 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:55.000 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T14:15:55.001 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.001 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.001 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:55.001 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:55.001 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.001 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.001 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.001 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-04-25T14:15:55.001 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-04-25T14:15:55.001 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:55.001 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.001 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:55.001 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "15", 2024-04-25T14:15:55.002 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.002 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.002 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:55.002 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:55.002 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.002 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.002 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.002 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:55.002 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:55.002 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:55.002 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.002 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:55.003 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:55.003 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.003 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.003 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:55.003 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:55.003 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:55.003 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:55.003 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:55.003 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:55.003 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:55.003 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:55.003 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:55.003 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.003 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.004 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.004 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:55.004 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:55.004 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:55.004 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.004 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:55.004 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:55.004 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.004 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.004 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:55.004 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:55.004 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.004 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.004 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.005 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:55.005 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:55.005 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:55.005 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.005 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:55.005 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:55.005 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.005 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.005 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:55.005 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:55.005 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:55.005 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:55.005 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:55.006 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:55.006 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:55.006 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:55.006 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:55.006 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.006 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.006 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.006 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:55.006 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:55.006 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:55.006 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.006 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:55.006 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:55.007 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.007 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.007 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:55.007 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:55.007 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.007 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.007 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.007 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:55.007 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:55.007 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:55.007 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:55.007 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:55.007 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:55.008 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.008 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.008 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:55.008 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:55.008 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.008 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.008 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.008 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:55.008 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:55.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:55.008 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "telemetry", 2024-04-25T14:15:55.008 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:55.009 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:55.009 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:55.009 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_basic": { 2024-04-25T14:15:55.009 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_basic", 2024-04-25T14:15:55.009 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:55.009 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.009 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:55.009 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:55.009 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.009 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.009 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:55.009 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T14:15:55.009 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.009 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.010 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.010 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_crash": { 2024-04-25T14:15:55.010 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_crash", 2024-04-25T14:15:55.010 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:55.010 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.010 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:55.010 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:55.010 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.010 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.010 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:55.010 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T14:15:55.010 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.010 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.011 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.011 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_device": { 2024-04-25T14:15:55.011 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_device", 2024-04-25T14:15:55.011 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:55.011 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.011 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:55.011 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:55.011 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.011 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.011 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:55.011 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T14:15:55.011 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.011 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.012 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.012 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_ident": { 2024-04-25T14:15:55.012 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_ident", 2024-04-25T14:15:55.012 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:55.012 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.012 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:55.012 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:55.012 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.012 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.012 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:55.012 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T14:15:55.012 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.012 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.013 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.013 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_perf": { 2024-04-25T14:15:55.013 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_perf", 2024-04-25T14:15:55.013 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:55.013 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.013 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:55.013 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:55.013 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.013 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.013 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:55.013 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T14:15:55.013 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.014 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.014 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.014 INFO:teuthology.orchestra.run.smithi096.stdout: "contact": { 2024-04-25T14:15:55.014 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "contact", 2024-04-25T14:15:55.014 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:55.014 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.014 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:55.014 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:55.014 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.014 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.014 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:55.014 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:55.015 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.015 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.015 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.015 INFO:teuthology.orchestra.run.smithi096.stdout: "description": { 2024-04-25T14:15:55.015 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "description", 2024-04-25T14:15:55.015 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:55.015 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.015 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:55.015 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:55.015 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.015 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.015 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:55.015 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:55.015 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.016 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.016 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.016 INFO:teuthology.orchestra.run.smithi096.stdout: "device_url": { 2024-04-25T14:15:55.016 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "device_url", 2024-04-25T14:15:55.016 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:55.016 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.016 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:55.016 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T14:15:55.016 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.016 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.016 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:55.016 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:55.016 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.017 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.017 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.017 INFO:teuthology.orchestra.run.smithi096.stdout: "enabled": { 2024-04-25T14:15:55.017 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enabled", 2024-04-25T14:15:55.017 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:55.017 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.017 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:55.017 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:55.017 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.017 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.017 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:55.017 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:55.017 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.018 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.018 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.018 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-04-25T14:15:55.018 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-04-25T14:15:55.018 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:55.018 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.018 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:55.018 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "24", 2024-04-25T14:15:55.018 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "8", 2024-04-25T14:15:55.018 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.018 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:55.018 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:55.019 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.019 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.019 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.019 INFO:teuthology.orchestra.run.smithi096.stdout: "last_opt_revision": { 2024-04-25T14:15:55.019 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "last_opt_revision", 2024-04-25T14:15:55.019 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:55.019 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.019 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:55.019 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "1", 2024-04-25T14:15:55.019 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.019 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.020 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:55.020 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:55.020 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.020 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.020 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.020 INFO:teuthology.orchestra.run.smithi096.stdout: "leaderboard": { 2024-04-25T14:15:55.020 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "leaderboard", 2024-04-25T14:15:55.020 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:55.020 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.020 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:55.020 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:55.020 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.020 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.021 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:55.021 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:55.021 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.021 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.021 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.021 INFO:teuthology.orchestra.run.smithi096.stdout: "leaderboard_description": { 2024-04-25T14:15:55.021 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "leaderboard_description", 2024-04-25T14:15:55.021 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:55.021 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.021 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:55.021 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:55.021 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.021 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.022 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:55.022 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:55.022 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.022 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.022 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.022 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:55.022 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:55.022 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:55.022 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.022 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:55.022 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:55.022 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.023 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.023 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:55.023 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:55.023 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:55.023 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:55.023 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:55.023 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:55.023 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:55.023 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:55.023 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:55.023 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.023 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.023 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.025 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:55.025 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:55.026 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:55.026 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.026 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:55.026 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:55.026 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.026 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.026 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:55.026 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:55.026 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.026 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.026 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.026 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:55.026 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:55.026 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:55.027 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.027 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:55.027 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:55.027 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.027 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.027 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:55.027 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:55.027 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:55.027 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:55.027 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:55.027 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:55.027 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:55.027 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:55.027 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:55.028 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.028 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.028 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.028 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:55.028 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:55.028 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:55.028 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.028 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:55.028 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:55.028 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.028 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.028 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:55.028 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:55.028 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.029 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.029 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.029 INFO:teuthology.orchestra.run.smithi096.stdout: "organization": { 2024-04-25T14:15:55.029 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "organization", 2024-04-25T14:15:55.029 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:55.029 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.029 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:55.029 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:55.029 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.029 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.029 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:55.029 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:55.029 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.029 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.030 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.030 INFO:teuthology.orchestra.run.smithi096.stdout: "proxy": { 2024-04-25T14:15:55.030 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "proxy", 2024-04-25T14:15:55.030 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:55.030 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.030 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:55.030 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:55.030 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.030 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.030 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:55.030 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:55.030 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.030 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.031 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.031 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:55.031 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:55.031 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:55.031 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:55.031 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:55.031 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:55.031 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.031 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.031 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:55.031 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:55.031 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.031 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.031 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.032 INFO:teuthology.orchestra.run.smithi096.stdout: "url": { 2024-04-25T14:15:55.032 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "url", 2024-04-25T14:15:55.032 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:55.032 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.032 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:55.032 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T14:15:55.032 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.032 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.032 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:55.032 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:55.032 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.032 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.032 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.033 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:55.033 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:55.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:55.033 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "test_orchestrator", 2024-04-25T14:15:55.033 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:55.033 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:55.033 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:55.033 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:55.033 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:55.033 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:55.033 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.033 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:55.033 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:55.033 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.034 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.034 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:55.034 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:55.034 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:55.034 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:55.034 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:55.034 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:55.034 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:55.034 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:55.034 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:55.034 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.034 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.034 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.035 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:55.035 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:55.035 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:55.035 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.035 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:55.035 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:55.035 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.035 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.035 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:55.035 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:55.035 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.035 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.035 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.036 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:55.036 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:55.036 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:55.036 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.036 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:55.036 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:55.036 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.036 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.036 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:55.036 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:55.036 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:55.036 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:55.036 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:55.036 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:55.036 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:55.037 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:55.037 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:55.037 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.037 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.037 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.037 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:55.037 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:55.037 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:55.037 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.037 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:55.037 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:55.037 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.037 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.038 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:55.038 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:55.038 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.038 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.038 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.038 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:55.038 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:55.038 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:55.038 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:55.038 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:55.038 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:55.038 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.038 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.039 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:55.039 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:55.039 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.039 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.039 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.039 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:55.039 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:55.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:55.039 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "volumes", 2024-04-25T14:15:55.039 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:55.039 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:55.039 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:55.039 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:55.040 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:55.040 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:55.040 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.040 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:55.040 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:55.040 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.040 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.040 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:55.040 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:55.040 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:55.040 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:55.040 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:55.040 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:55.041 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:55.041 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:55.041 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:55.041 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.041 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.041 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.041 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:55.041 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:55.041 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:55.041 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.041 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:55.041 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:55.041 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.042 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.042 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:55.042 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:55.042 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.042 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.042 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.042 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:55.042 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:55.042 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:55.042 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.042 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:55.042 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:55.042 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.043 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.043 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:55.043 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:55.043 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:55.043 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:55.043 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:55.043 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:55.043 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:55.043 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:55.043 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:55.043 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.043 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.043 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.044 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:55.044 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:55.044 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:55.044 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.044 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:55.044 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:55.044 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.044 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.044 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:55.044 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:55.044 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.044 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.044 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.044 INFO:teuthology.orchestra.run.smithi096.stdout: "max_concurrent_clones": { 2024-04-25T14:15:55.045 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_concurrent_clones", 2024-04-25T14:15:55.045 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:55.045 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.045 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:55.045 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "4", 2024-04-25T14:15:55.045 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.045 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.045 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:55.045 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T14:15:55.045 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.045 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.045 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.046 INFO:teuthology.orchestra.run.smithi096.stdout: "periodic_async_work": { 2024-04-25T14:15:55.046 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "periodic_async_work", 2024-04-25T14:15:55.046 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:55.046 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.046 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:55.046 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:55.046 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.046 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.046 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:55.046 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Periodically check for async work", 2024-04-25T14:15:55.046 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.046 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.046 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.047 INFO:teuthology.orchestra.run.smithi096.stdout: "snapshot_clone_delay": { 2024-04-25T14:15:55.047 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snapshot_clone_delay", 2024-04-25T14:15:55.047 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:55.047 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.047 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:55.047 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:55.047 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.047 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.047 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:55.047 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T14:15:55.047 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.047 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.047 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.047 INFO:teuthology.orchestra.run.smithi096.stdout: "snapshot_clone_no_wait": { 2024-04-25T14:15:55.048 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T14:15:55.048 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:55.048 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.048 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:55.048 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T14:15:55.048 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.048 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.048 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:55.048 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T14:15:55.048 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.048 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.048 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.049 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:55.049 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:55.049 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:55.049 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:55.049 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:55.049 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:55.049 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.049 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.049 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:55.049 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:55.049 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.049 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.049 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.049 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:55.050 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:55.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:55.050 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix", 2024-04-25T14:15:55.050 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T14:15:55.050 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T14:15:55.050 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T14:15:55.050 INFO:teuthology.orchestra.run.smithi096.stdout: "discovery_interval": { 2024-04-25T14:15:55.050 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "discovery_interval", 2024-04-25T14:15:55.050 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-04-25T14:15:55.050 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.050 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:55.050 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "100", 2024-04-25T14:15:55.051 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.051 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.051 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:55.051 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:55.051 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.051 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.051 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.051 INFO:teuthology.orchestra.run.smithi096.stdout: "identifier": { 2024-04-25T14:15:55.051 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "identifier", 2024-04-25T14:15:55.051 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:55.051 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.051 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:55.052 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:55.052 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.052 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.052 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:55.052 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:55.052 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.052 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.052 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.052 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-04-25T14:15:55.052 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-04-25T14:15:55.052 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T14:15:55.052 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.053 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:55.053 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-04-25T14:15:55.053 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.053 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.053 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:55.053 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:55.053 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.053 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.053 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.053 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T14:15:55.053 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T14:15:55.053 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:55.054 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.054 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:55.054 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:55.054 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.054 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.054 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:55.054 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:55.054 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:55.054 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:55.054 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:55.054 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:55.054 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:55.054 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:55.055 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:55.055 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.055 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.055 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.055 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T14:15:55.055 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T14:15:55.055 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:55.055 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.055 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:55.055 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:55.055 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.055 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.055 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:55.056 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:55.056 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.056 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.056 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.056 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T14:15:55.056 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T14:15:55.056 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:55.056 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.056 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:55.056 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T14:15:55.056 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.056 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.056 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T14:15:55.056 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T14:15:55.057 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T14:15:55.057 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T14:15:55.057 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T14:15:55.057 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T14:15:55.057 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T14:15:55.057 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:55.057 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:55.057 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.057 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.057 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.057 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T14:15:55.057 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T14:15:55.057 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T14:15:55.058 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.058 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:55.058 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T14:15:55.058 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.058 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.058 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:55.058 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:55.058 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.058 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.058 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.058 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T14:15:55.058 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:15:55.058 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:55.059 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T14:15:55.059 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T14:15:55.059 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T14:15:55.059 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.059 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.059 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:55.059 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:55.059 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.059 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.059 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.059 INFO:teuthology.orchestra.run.smithi096.stdout: "zabbix_host": { 2024-04-25T14:15:55.059 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix_host", 2024-04-25T14:15:55.059 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:55.060 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.060 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:55.060 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T14:15:55.060 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.060 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.060 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:55.060 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:55.060 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.060 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.060 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.060 INFO:teuthology.orchestra.run.smithi096.stdout: "zabbix_port": { 2024-04-25T14:15:55.060 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix_port", 2024-04-25T14:15:55.061 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T14:15:55.061 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.061 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:55.061 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10051", 2024-04-25T14:15:55.061 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.061 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.061 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:55.061 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:55.061 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.061 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.061 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.062 INFO:teuthology.orchestra.run.smithi096.stdout: "zabbix_sender": { 2024-04-25T14:15:55.062 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix_sender", 2024-04-25T14:15:55.062 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T14:15:55.062 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T14:15:55.062 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T14:15:55.062 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T14:15:55.062 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T14:15:55.062 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T14:15:55.062 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T14:15:55.062 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T14:15:55.062 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T14:15:55.062 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T14:15:55.062 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T14:15:55.063 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:55.063 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:55.063 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:55.063 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:55.063 INFO:teuthology.orchestra.run.smithi096.stdout: "services": { 2024-04-25T14:15:55.063 INFO:teuthology.orchestra.run.smithi096.stdout: "dashboard": "https://172.21.15.96:8443/" 2024-04-25T14:15:55.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.063 INFO:teuthology.orchestra.run.smithi096.stdout: "always_on_modules": { 2024-04-25T14:15:55.063 INFO:teuthology.orchestra.run.smithi096.stdout: "octopus": [ 2024-04-25T14:15:55.063 INFO:teuthology.orchestra.run.smithi096.stdout: "balancer", 2024-04-25T14:15:55.063 INFO:teuthology.orchestra.run.smithi096.stdout: "crash", 2024-04-25T14:15:55.063 INFO:teuthology.orchestra.run.smithi096.stdout: "devicehealth", 2024-04-25T14:15:55.063 INFO:teuthology.orchestra.run.smithi096.stdout: "orchestrator", 2024-04-25T14:15:55.064 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_autoscaler", 2024-04-25T14:15:55.064 INFO:teuthology.orchestra.run.smithi096.stdout: "progress", 2024-04-25T14:15:55.064 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_support", 2024-04-25T14:15:55.064 INFO:teuthology.orchestra.run.smithi096.stdout: "status", 2024-04-25T14:15:55.064 INFO:teuthology.orchestra.run.smithi096.stdout: "telemetry", 2024-04-25T14:15:55.064 INFO:teuthology.orchestra.run.smithi096.stdout: "volumes" 2024-04-25T14:15:55.064 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:55.064 INFO:teuthology.orchestra.run.smithi096.stdout: "pacific": [ 2024-04-25T14:15:55.064 INFO:teuthology.orchestra.run.smithi096.stdout: "balancer", 2024-04-25T14:15:55.064 INFO:teuthology.orchestra.run.smithi096.stdout: "crash", 2024-04-25T14:15:55.064 INFO:teuthology.orchestra.run.smithi096.stdout: "devicehealth", 2024-04-25T14:15:55.064 INFO:teuthology.orchestra.run.smithi096.stdout: "orchestrator", 2024-04-25T14:15:55.064 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_autoscaler", 2024-04-25T14:15:55.064 INFO:teuthology.orchestra.run.smithi096.stdout: "progress", 2024-04-25T14:15:55.064 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_support", 2024-04-25T14:15:55.065 INFO:teuthology.orchestra.run.smithi096.stdout: "status", 2024-04-25T14:15:55.065 INFO:teuthology.orchestra.run.smithi096.stdout: "telemetry", 2024-04-25T14:15:55.065 INFO:teuthology.orchestra.run.smithi096.stdout: "volumes" 2024-04-25T14:15:55.065 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:55.065 INFO:teuthology.orchestra.run.smithi096.stdout: "quincy": [ 2024-04-25T14:15:55.065 INFO:teuthology.orchestra.run.smithi096.stdout: "balancer", 2024-04-25T14:15:55.065 INFO:teuthology.orchestra.run.smithi096.stdout: "crash", 2024-04-25T14:15:55.065 INFO:teuthology.orchestra.run.smithi096.stdout: "devicehealth", 2024-04-25T14:15:55.065 INFO:teuthology.orchestra.run.smithi096.stdout: "orchestrator", 2024-04-25T14:15:55.065 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_autoscaler", 2024-04-25T14:15:55.065 INFO:teuthology.orchestra.run.smithi096.stdout: "progress", 2024-04-25T14:15:55.065 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_support", 2024-04-25T14:15:55.065 INFO:teuthology.orchestra.run.smithi096.stdout: "status", 2024-04-25T14:15:55.065 INFO:teuthology.orchestra.run.smithi096.stdout: "telemetry", 2024-04-25T14:15:55.066 INFO:teuthology.orchestra.run.smithi096.stdout: "volumes" 2024-04-25T14:15:55.066 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:55.066 INFO:teuthology.orchestra.run.smithi096.stdout: "reef": [ 2024-04-25T14:15:55.066 INFO:teuthology.orchestra.run.smithi096.stdout: "balancer", 2024-04-25T14:15:55.066 INFO:teuthology.orchestra.run.smithi096.stdout: "crash", 2024-04-25T14:15:55.066 INFO:teuthology.orchestra.run.smithi096.stdout: "devicehealth", 2024-04-25T14:15:55.066 INFO:teuthology.orchestra.run.smithi096.stdout: "orchestrator", 2024-04-25T14:15:55.066 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_autoscaler", 2024-04-25T14:15:55.066 INFO:teuthology.orchestra.run.smithi096.stdout: "progress", 2024-04-25T14:15:55.066 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_support", 2024-04-25T14:15:55.066 INFO:teuthology.orchestra.run.smithi096.stdout: "status", 2024-04-25T14:15:55.066 INFO:teuthology.orchestra.run.smithi096.stdout: "telemetry", 2024-04-25T14:15:55.066 INFO:teuthology.orchestra.run.smithi096.stdout: "volumes" 2024-04-25T14:15:55.067 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T14:15:55.067 INFO:teuthology.orchestra.run.smithi096.stdout: "squid": [ 2024-04-25T14:15:55.067 INFO:teuthology.orchestra.run.smithi096.stdout: "balancer", 2024-04-25T14:15:55.067 INFO:teuthology.orchestra.run.smithi096.stdout: "crash", 2024-04-25T14:15:55.067 INFO:teuthology.orchestra.run.smithi096.stdout: "devicehealth", 2024-04-25T14:15:55.067 INFO:teuthology.orchestra.run.smithi096.stdout: "orchestrator", 2024-04-25T14:15:55.067 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_autoscaler", 2024-04-25T14:15:55.067 INFO:teuthology.orchestra.run.smithi096.stdout: "progress", 2024-04-25T14:15:55.067 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_support", 2024-04-25T14:15:55.067 INFO:teuthology.orchestra.run.smithi096.stdout: "status", 2024-04-25T14:15:55.067 INFO:teuthology.orchestra.run.smithi096.stdout: "telemetry", 2024-04-25T14:15:55.067 INFO:teuthology.orchestra.run.smithi096.stdout: "volumes" 2024-04-25T14:15:55.067 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-25T14:15:55.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.068 INFO:teuthology.orchestra.run.smithi096.stdout: "last_failure_osd_epoch": 3, 2024-04-25T14:15:55.068 INFO:teuthology.orchestra.run.smithi096.stdout: "active_clients": [ 2024-04-25T14:15:55.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:55.068 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "devicehealth", 2024-04-25T14:15:55.068 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-25T14:15:55.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:55.068 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-25T14:15:55.068 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:0", 2024-04-25T14:15:55.068 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2939571162 2024-04-25T14:15:55.068 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:55.068 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-25T14:15:55.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:55.068 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "libcephsqlite", 2024-04-25T14:15:55.068 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-25T14:15:55.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:55.069 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-25T14:15:55.069 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:0", 2024-04-25T14:15:55.069 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 344482924 2024-04-25T14:15:55.069 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:55.069 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-25T14:15:55.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:55.069 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_support", 2024-04-25T14:15:55.069 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-25T14:15:55.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:55.069 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-25T14:15:55.069 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:0", 2024-04-25T14:15:55.070 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3955641945 2024-04-25T14:15:55.070 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:55.070 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-25T14:15:55.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:55.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:55.070 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "volumes", 2024-04-25T14:15:55.070 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-25T14:15:55.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T14:15:55.070 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-25T14:15:55.070 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:0", 2024-04-25T14:15:55.070 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 588225129 2024-04-25T14:15:55.070 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:55.070 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-25T14:15:55.071 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:55.071 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-25T14:15:55.071 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T14:15:55.072 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:53 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:15:55.072 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:53 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/2085589832' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:15:55.072 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:53 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:15:55.072 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:53 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:15:55.072 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:53 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/2613403959' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:15:55.073 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:54 smithi151 ceph-mon[32990]: pgmap v414: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-25T14:15:55.073 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:54 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/392973496' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T14:15:55.073 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:54 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:15:55.073 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:54 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:15:55.073 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:54 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:15:55.073 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:53 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:15:55.073 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:53 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/2085589832' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:15:55.073 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:53 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:15:55.073 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:53 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:15:55.073 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:53 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/2613403959' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:15:55.073 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:54 smithi097 ceph-mon[32720]: pgmap v414: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-25T14:15:55.073 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:54 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/392973496' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T14:15:55.073 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:54 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:15:55.074 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:54 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:15:55.074 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:54 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:15:55.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.612+0000 7f66ab534640 1 -- 172.21.15.96:0/392973496 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f66900afaf0 msgr2=0x7f66900b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:55.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.612+0000 7f66ab534640 1 --2- 172.21.15.96:0/392973496 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f66900afaf0 0x7f66900b2000 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f66a406b520 tx=0x7f6698005f90 comp rx=0 tx=0).stop 2024-04-25T14:15:55.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.612+0000 7f66ab534640 1 -- 172.21.15.96:0/392973496 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66a4108e80 msgr2=0x7f66a419c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:55.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.612+0000 7f66ab534640 1 --2- 172.21.15.96:0/392973496 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66a4108e80 0x7f66a419c8e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f66a000e9f0 tx=0x7f66a000eeb0 comp rx=0 tx=0).stop 2024-04-25T14:15:55.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.612+0000 7f66ab534640 1 -- 172.21.15.96:0/392973496 shutdown_connections 2024-04-25T14:15:55.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.612+0000 7f66ab534640 1 --2- 172.21.15.96:0/392973496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f66a410b9b0 0x7f66a41a3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:55.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.612+0000 7f66ab534640 1 --2- 172.21.15.96:0/392973496 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f66900afaf0 0x7f66900b2000 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:55.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.612+0000 7f66ab534640 1 --2- 172.21.15.96:0/392973496 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66a4108e80 0x7f66a419c8e0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:55.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.612+0000 7f66ab534640 1 --2- 172.21.15.96:0/392973496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66a4103fd0 0x7f66a419c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:55.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.612+0000 7f66ab534640 1 -- 172.21.15.96:0/392973496 >> 172.21.15.96:0/392973496 conn(0x7f66a40fd3b0 msgr2=0x7f66a4109e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:55.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.612+0000 7f66ab534640 1 -- 172.21.15.96:0/392973496 shutdown_connections 2024-04-25T14:15:55.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:15:53.612+0000 7f66ab534640 1 -- 172.21.15.96:0/392973496 wait complete. 2024-04-25T14:15:55.082 DEBUG:tasks.check_counter:Getting stats from a 2024-04-25T14:15:55.082 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-25T14:15:55.083 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:53 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:15:55.083 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:53 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/2085589832' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:15:55.083 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:53 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:15:55.083 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:53 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:15:55.083 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:53 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/2613403959' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:15:55.083 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:54 smithi096 ceph-mon[29207]: pgmap v414: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-25T14:15:55.083 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:54 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/392973496' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T14:15:55.083 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:54 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:15:55.083 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:54 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:15:55.083 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:54 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' 2024-04-25T14:15:55.333 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:15:56.835 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:56 smithi096 ceph-mon[29207]: pgmap v415: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-25T14:15:56.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:56 smithi097 ceph-mon[32720]: pgmap v415: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-25T14:15:56.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:56 smithi151 ceph-mon[32990]: pgmap v415: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-25T14:15:57.255 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T14:15:57.255 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:15:57.255 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 526, 2024-04-25T14:15:57.256 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 528, 2024-04-25T14:15:57.256 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 157791, 2024-04-25T14:15:57.256 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 167269, 2024-04-25T14:15:57.256 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 6, 2024-04-25T14:15:57.256 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 2, 2024-04-25T14:15:57.256 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.041795316, 2024-04-25T14:15:57.256 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.013522745, 2024-04-25T14:15:57.256 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.011028389, 2024-04-25T14:15:57.256 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.012832640, 2024-04-25T14:15:57.256 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:15:57.256 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 526, 2024-04-25T14:15:57.256 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.014922980, 2024-04-25T14:15:57.256 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000028370 2024-04-25T14:15:57.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.257 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:15:57.257 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:15:57.257 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:15:57.257 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:15:57.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.257 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T14:15:57.257 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T14:15:57.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.257 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:15:57.257 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 97, 2024-04-25T14:15:57.257 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 91, 2024-04-25T14:15:57.257 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 495566, 2024-04-25T14:15:57.257 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 49726, 2024-04-25T14:15:57.257 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 4, 2024-04-25T14:15:57.257 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 3, 2024-04-25T14:15:57.258 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.015546502, 2024-04-25T14:15:57.258 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.004961653, 2024-04-25T14:15:57.258 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.012159919, 2024-04-25T14:15:57.258 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.001516173, 2024-04-25T14:15:57.258 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:15:57.258 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 91, 2024-04-25T14:15:57.258 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.006910203, 2024-04-25T14:15:57.258 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000075936 2024-04-25T14:15:57.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.258 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:15:57.258 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:15:57.258 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:15:57.258 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:15:57.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.259 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 494176, 2024-04-25T14:15:57.259 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 48704 2024-04-25T14:15:57.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.259 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:15:57.259 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 2, 2024-04-25T14:15:57.259 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 60, 2024-04-25T14:15:57.259 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 428, 2024-04-25T14:15:57.259 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 184094, 2024-04-25T14:15:57.259 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 4, 2024-04-25T14:15:57.259 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 3, 2024-04-25T14:15:57.259 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.005015020, 2024-04-25T14:15:57.259 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.003031958, 2024-04-25T14:15:57.259 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.001421074, 2024-04-25T14:15:57.260 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000027372, 2024-04-25T14:15:57.260 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:15:57.260 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 58, 2024-04-25T14:15:57.260 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.003114507, 2024-04-25T14:15:57.260 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000053698 2024-04-25T14:15:57.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.260 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:15:57.260 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:15:57.260 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:15:57.260 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:15:57.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.260 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T14:15:57.260 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 183776 2024-04-25T14:15:57.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.260 INFO:teuthology.orchestra.run.smithi096.stdout: "cct": { 2024-04-25T14:15:57.261 INFO:teuthology.orchestra.run.smithi096.stdout: "total_workers": 1, 2024-04-25T14:15:57.261 INFO:teuthology.orchestra.run.smithi096.stdout: "unhealthy_workers": 0 2024-04-25T14:15:57.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.261 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-MDSRank": { 2024-04-25T14:15:57.261 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-04-25T14:15:57.261 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-04-25T14:15:57.261 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1053, 2024-04-25T14:15:57.261 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.027461607, 2024-04-25T14:15:57.261 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000026079 2024-04-25T14:15:57.261 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:57.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.261 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-PurgeQueue": { 2024-04-25T14:15:57.261 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-04-25T14:15:57.261 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-04-25T14:15:57.261 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:15:57.262 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:15:57.262 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:15:57.262 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:57.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.262 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": { 2024-04-25T14:15:57.262 INFO:teuthology.orchestra.run.smithi096.stdout: "request": 0, 2024-04-25T14:15:57.262 INFO:teuthology.orchestra.run.smithi096.stdout: "reply": 0, 2024-04-25T14:15:57.262 INFO:teuthology.orchestra.run.smithi096.stdout: "reply_latency": { 2024-04-25T14:15:57.262 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:15:57.262 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:15:57.262 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:15:57.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.262 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_reply": 0, 2024-04-25T14:15:57.263 INFO:teuthology.orchestra.run.smithi096.stdout: "forward": 0, 2024-04-25T14:15:57.263 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_complete": 0, 2024-04-25T14:15:57.263 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_keys": 0, 2024-04-25T14:15:57.263 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_commit": 0, 2024-04-25T14:15:57.263 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_split": 0, 2024-04-25T14:15:57.263 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_merge": 0, 2024-04-25T14:15:57.263 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes": 0, 2024-04-25T14:15:57.263 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_top": 0, 2024-04-25T14:15:57.263 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_bottom": 0, 2024-04-25T14:15:57.263 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pin_tail": 0, 2024-04-25T14:15:57.263 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pinned": 0, 2024-04-25T14:15:57.263 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_expired": 0, 2024-04-25T14:15:57.263 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_with_caps": 0, 2024-04-25T14:15:57.263 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": 0, 2024-04-25T14:15:57.264 INFO:teuthology.orchestra.run.smithi096.stdout: "subtrees": 1, 2024-04-25T14:15:57.264 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse": 0, 2024-04-25T14:15:57.264 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_hit": 0, 2024-04-25T14:15:57.264 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_forward": 0, 2024-04-25T14:15:57.264 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_discover": 0, 2024-04-25T14:15:57.264 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_dir_fetch": 0, 2024-04-25T14:15:57.264 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_remote_ino": 0, 2024-04-25T14:15:57.264 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_lock": 0, 2024-04-25T14:15:57.264 INFO:teuthology.orchestra.run.smithi096.stdout: "load_cent": 0, 2024-04-25T14:15:57.264 INFO:teuthology.orchestra.run.smithi096.stdout: "q": 0, 2024-04-25T14:15:57.264 INFO:teuthology.orchestra.run.smithi096.stdout: "exported": 0, 2024-04-25T14:15:57.264 INFO:teuthology.orchestra.run.smithi096.stdout: "exported_inodes": 0, 2024-04-25T14:15:57.264 INFO:teuthology.orchestra.run.smithi096.stdout: "imported": 0, 2024-04-25T14:15:57.265 INFO:teuthology.orchestra.run.smithi096.stdout: "imported_inodes": 0, 2024-04-25T14:15:57.265 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_dir_fetch": 0, 2024-04-25T14:15:57.265 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_backtrace_fetch": 0, 2024-04-25T14:15:57.265 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_peer_discover": 0, 2024-04-25T14:15:57.265 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rfiles": 0, 2024-04-25T14:15:57.265 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rbytes": 0, 2024-04-25T14:15:57.265 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rsnaps": 0, 2024-04-25T14:15:57.265 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T14:15:57.265 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_set_tag": 0, 2024-04-25T14:15:57.265 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T14:15:57.265 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_inotable_repaired": 0, 2024-04-25T14:15:57.265 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_inodes": 0, 2024-04-25T14:15:57.266 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T14:15:57.266 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T14:15:57.266 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_file_inodes": 0, 2024-04-25T14:15:57.266 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_inode_file_caps": 0, 2024-04-25T14:15:57.267 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T14:15:57.267 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_grant": 0, 2024-04-25T14:15:57.267 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T14:15:57.267 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T14:15:57.267 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T14:15:57.267 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps": 0, 2024-04-25T14:15:57.267 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps_dirty": 0, 2024-04-25T14:15:57.267 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_cap_release": 0, 2024-04-25T14:15:57.267 INFO:teuthology.orchestra.run.smithi096.stdout: "process_request_cap_release": 0 2024-04-25T14:15:57.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.267 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cache": { 2024-04-25T14:15:57.267 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update": 0, 2024-04-25T14:15:57.267 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update_receipt": 0, 2024-04-25T14:15:57.268 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_try_discover": 0, 2024-04-25T14:15:57.268 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_send_discover": 0, 2024-04-25T14:15:57.268 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_handle_discover": 0, 2024-04-25T14:15:57.268 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays": 0, 2024-04-25T14:15:57.268 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_delayed": 0, 2024-04-25T14:15:57.268 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_enqueuing": 0, 2024-04-25T14:15:57.268 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_created": 0, 2024-04-25T14:15:57.268 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_enqueued": 0, 2024-04-25T14:15:57.268 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_reintegrated": 0, 2024-04-25T14:15:57.268 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_migrated": 0, 2024-04-25T14:15:57.268 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_processing": 0, 2024-04-25T14:15:57.268 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_enqueued": 0, 2024-04-25T14:15:57.268 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_prioritized": 0, 2024-04-25T14:15:57.268 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_started": 0, 2024-04-25T14:15:57.269 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_completed": 0, 2024-04-25T14:15:57.269 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_path": 0, 2024-04-25T14:15:57.269 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_inode": 0, 2024-04-25T14:15:57.269 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T14:15:57.269 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_exportdir": 0, 2024-04-25T14:15:57.269 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_flush": 0, 2024-04-25T14:15:57.269 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragmentdir": 0, 2024-04-25T14:15:57.269 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragstats": 0, 2024-04-25T14:15:57.269 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_inodestats": 0 2024-04-25T14:15:57.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.269 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log": { 2024-04-25T14:15:57.269 INFO:teuthology.orchestra.run.smithi096.stdout: "evlrg": 0, 2024-04-25T14:15:57.269 INFO:teuthology.orchestra.run.smithi096.stdout: "evadd": 0, 2024-04-25T14:15:57.269 INFO:teuthology.orchestra.run.smithi096.stdout: "evex": 0, 2024-04-25T14:15:57.269 INFO:teuthology.orchestra.run.smithi096.stdout: "evtrm": 0, 2024-04-25T14:15:57.270 INFO:teuthology.orchestra.run.smithi096.stdout: "ev": 2, 2024-04-25T14:15:57.270 INFO:teuthology.orchestra.run.smithi096.stdout: "evexg": 0, 2024-04-25T14:15:57.270 INFO:teuthology.orchestra.run.smithi096.stdout: "evexd": 0, 2024-04-25T14:15:57.270 INFO:teuthology.orchestra.run.smithi096.stdout: "segadd": 0, 2024-04-25T14:15:57.270 INFO:teuthology.orchestra.run.smithi096.stdout: "segex": 0, 2024-04-25T14:15:57.270 INFO:teuthology.orchestra.run.smithi096.stdout: "segtrm": 0, 2024-04-25T14:15:57.270 INFO:teuthology.orchestra.run.smithi096.stdout: "seg": 2, 2024-04-25T14:15:57.270 INFO:teuthology.orchestra.run.smithi096.stdout: "segmjr": 2, 2024-04-25T14:15:57.270 INFO:teuthology.orchestra.run.smithi096.stdout: "segexg": 0, 2024-04-25T14:15:57.270 INFO:teuthology.orchestra.run.smithi096.stdout: "segexd": 0, 2024-04-25T14:15:57.270 INFO:teuthology.orchestra.run.smithi096.stdout: "expos": 4194304, 2024-04-25T14:15:57.270 INFO:teuthology.orchestra.run.smithi096.stdout: "wrpos": 4194892, 2024-04-25T14:15:57.270 INFO:teuthology.orchestra.run.smithi096.stdout: "rdpos": 4194352, 2024-04-25T14:15:57.271 INFO:teuthology.orchestra.run.smithi096.stdout: "jlat": { 2024-04-25T14:15:57.271 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:15:57.271 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:15:57.271 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:15:57.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.271 INFO:teuthology.orchestra.run.smithi096.stdout: "replayed": 2 2024-04-25T14:15:57.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.271 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_mem": { 2024-04-25T14:15:57.271 INFO:teuthology.orchestra.run.smithi096.stdout: "ino": 3, 2024-04-25T14:15:57.271 INFO:teuthology.orchestra.run.smithi096.stdout: "ino+": 3, 2024-04-25T14:15:57.271 INFO:teuthology.orchestra.run.smithi096.stdout: "ino-": 0, 2024-04-25T14:15:57.271 INFO:teuthology.orchestra.run.smithi096.stdout: "dir": 1, 2024-04-25T14:15:57.271 INFO:teuthology.orchestra.run.smithi096.stdout: "dir+": 1, 2024-04-25T14:15:57.271 INFO:teuthology.orchestra.run.smithi096.stdout: "dir-": 0, 2024-04-25T14:15:57.271 INFO:teuthology.orchestra.run.smithi096.stdout: "dn": 0, 2024-04-25T14:15:57.272 INFO:teuthology.orchestra.run.smithi096.stdout: "dn+": 0, 2024-04-25T14:15:57.272 INFO:teuthology.orchestra.run.smithi096.stdout: "dn-": 0, 2024-04-25T14:15:57.272 INFO:teuthology.orchestra.run.smithi096.stdout: "cap": 0, 2024-04-25T14:15:57.272 INFO:teuthology.orchestra.run.smithi096.stdout: "cap+": 0, 2024-04-25T14:15:57.272 INFO:teuthology.orchestra.run.smithi096.stdout: "cap-": 0, 2024-04-25T14:15:57.273 INFO:teuthology.orchestra.run.smithi096.stdout: "rss": 120848, 2024-04-25T14:15:57.273 INFO:teuthology.orchestra.run.smithi096.stdout: "heap": 215440 2024-04-25T14:15:57.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.273 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_server": { 2024-04-25T14:15:57.274 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_client_request": 0, 2024-04-25T14:15:57.274 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_server_request": 0, 2024-04-25T14:15:57.274 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_request": 0, 2024-04-25T14:15:57.274 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_session": 0, 2024-04-25T14:15:57.274 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_peer_request": 0, 2024-04-25T14:15:57.274 INFO:teuthology.orchestra.run.smithi096.stdout: "req_create_latency": { 2024-04-25T14:15:57.274 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:15:57.274 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:15:57.274 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:15:57.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.274 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getattr_latency": { 2024-04-25T14:15:57.274 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:15:57.274 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:15:57.274 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:15:57.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.275 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getfilelock_latency": { 2024-04-25T14:15:57.275 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:15:57.275 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:15:57.275 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:15:57.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.275 INFO:teuthology.orchestra.run.smithi096.stdout: "req_link_latency": { 2024-04-25T14:15:57.275 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:15:57.275 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:15:57.275 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:15:57.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.275 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookup_latency": { 2024-04-25T14:15:57.275 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:15:57.275 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:15:57.276 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:15:57.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.276 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookuphash_latency": { 2024-04-25T14:15:57.276 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:15:57.276 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:15:57.276 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:15:57.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.276 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupino_latency": { 2024-04-25T14:15:57.276 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:15:57.276 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:15:57.277 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:15:57.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.277 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupname_latency": { 2024-04-25T14:15:57.277 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:15:57.277 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:15:57.277 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:15:57.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.277 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupparent_latency": { 2024-04-25T14:15:57.277 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:15:57.277 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:15:57.277 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:15:57.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.278 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupsnap_latency": { 2024-04-25T14:15:57.278 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:15:57.278 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:15:57.278 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:15:57.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.278 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lssnap_latency": { 2024-04-25T14:15:57.278 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:15:57.278 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:15:57.278 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:15:57.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.278 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mkdir_latency": { 2024-04-25T14:15:57.278 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:15:57.278 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:15:57.279 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:15:57.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.279 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mknod_latency": { 2024-04-25T14:15:57.279 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:15:57.279 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:15:57.279 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:15:57.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.280 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mksnap_latency": { 2024-04-25T14:15:57.280 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:15:57.280 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:15:57.280 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:15:57.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.280 INFO:teuthology.orchestra.run.smithi096.stdout: "req_open_latency": { 2024-04-25T14:15:57.280 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:15:57.280 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:15:57.280 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:15:57.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.280 INFO:teuthology.orchestra.run.smithi096.stdout: "req_readdir_latency": { 2024-04-25T14:15:57.280 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:15:57.281 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:15:57.281 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:15:57.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.281 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rename_latency": { 2024-04-25T14:15:57.281 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:15:57.281 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:15:57.281 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:15:57.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.281 INFO:teuthology.orchestra.run.smithi096.stdout: "req_renamesnap_latency": { 2024-04-25T14:15:57.281 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:15:57.281 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:15:57.281 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:15:57.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.281 INFO:teuthology.orchestra.run.smithi096.stdout: "req_snapdiff_latency": { 2024-04-25T14:15:57.282 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:15:57.282 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:15:57.282 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:15:57.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.282 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmdir_latency": { 2024-04-25T14:15:57.282 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:15:57.282 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:15:57.282 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:15:57.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.282 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmsnap_latency": { 2024-04-25T14:15:57.282 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:15:57.282 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:15:57.282 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:15:57.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.283 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmxattr_latency": { 2024-04-25T14:15:57.283 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:15:57.283 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:15:57.283 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:15:57.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.283 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setattr_latency": { 2024-04-25T14:15:57.283 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:15:57.283 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:15:57.283 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:15:57.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.283 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setdirlayout_latency": { 2024-04-25T14:15:57.283 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:15:57.283 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:15:57.283 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:15:57.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.284 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setfilelock_latency": { 2024-04-25T14:15:57.284 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:15:57.284 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:15:57.284 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:15:57.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.284 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setlayout_latency": { 2024-04-25T14:15:57.284 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:15:57.284 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:15:57.284 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:15:57.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.284 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setxattr_latency": { 2024-04-25T14:15:57.284 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:15:57.284 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:15:57.285 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:15:57.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.285 INFO:teuthology.orchestra.run.smithi096.stdout: "req_symlink_latency": { 2024-04-25T14:15:57.285 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:15:57.285 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:15:57.285 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:15:57.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.285 INFO:teuthology.orchestra.run.smithi096.stdout: "req_unlink_latency": { 2024-04-25T14:15:57.285 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:15:57.285 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:15:57.285 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:15:57.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.285 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_revoke_eviction": 0, 2024-04-25T14:15:57.285 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_acquisition_throttle": 0, 2024-04-25T14:15:57.286 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getvxattr_latency": { 2024-04-25T14:15:57.286 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:15:57.286 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:15:57.286 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:15:57.286 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:57.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.286 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_sessions": { 2024-04-25T14:15:57.286 INFO:teuthology.orchestra.run.smithi096.stdout: "session_count": 0, 2024-04-25T14:15:57.286 INFO:teuthology.orchestra.run.smithi096.stdout: "session_add": 0, 2024-04-25T14:15:57.286 INFO:teuthology.orchestra.run.smithi096.stdout: "session_remove": 0, 2024-04-25T14:15:57.286 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_open": 0, 2024-04-25T14:15:57.286 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_stale": 0, 2024-04-25T14:15:57.286 INFO:teuthology.orchestra.run.smithi096.stdout: "total_load": 0, 2024-04-25T14:15:57.286 INFO:teuthology.orchestra.run.smithi096.stdout: "average_load": 0, 2024-04-25T14:15:57.287 INFO:teuthology.orchestra.run.smithi096.stdout: "avg_session_uptime": 0, 2024-04-25T14:15:57.287 INFO:teuthology.orchestra.run.smithi096.stdout: "mdthresh_evicted": 0 2024-04-25T14:15:57.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.287 INFO:teuthology.orchestra.run.smithi096.stdout: "mempool": { 2024-04-25T14:15:57.287 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:15:57.287 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_items": 0, 2024-04-25T14:15:57.287 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:15:57.287 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:15:57.287 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:15:57.287 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:15:57.287 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:15:57.287 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:15:57.287 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:15:57.288 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:15:57.288 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:15:57.288 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:15:57.288 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:15:57.288 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:15:57.288 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:15:57.288 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_items": 0, 2024-04-25T14:15:57.288 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:15:57.288 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_items": 0, 2024-04-25T14:15:57.288 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:15:57.288 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:15:57.288 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:15:57.288 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:15:57.288 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:15:57.288 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:15:57.289 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:15:57.289 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_items": 0, 2024-04-25T14:15:57.289 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:15:57.289 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:15:57.289 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:15:57.289 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_items": 0, 2024-04-25T14:15:57.289 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_bytes": 0, 2024-04-25T14:15:57.289 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_items": 0, 2024-04-25T14:15:57.289 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:15:57.289 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:15:57.289 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:15:57.289 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:15:57.289 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_bytes": 189239, 2024-04-25T14:15:57.289 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_items": 48, 2024-04-25T14:15:57.290 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_bytes": 0, 2024-04-25T14:15:57.290 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_items": 0, 2024-04-25T14:15:57.290 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bytes": 0, 2024-04-25T14:15:57.290 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_items": 0, 2024-04-25T14:15:57.290 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:15:57.290 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_items": 0, 2024-04-25T14:15:57.290 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:15:57.290 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_items": 0, 2024-04-25T14:15:57.290 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_bytes": 7720, 2024-04-25T14:15:57.290 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_items": 162, 2024-04-25T14:15:57.290 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:15:57.290 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:15:57.290 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_bytes": 0, 2024-04-25T14:15:57.290 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_items": 0, 2024-04-25T14:15:57.291 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_bytes": 8512, 2024-04-25T14:15:57.291 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_items": 13, 2024-04-25T14:15:57.291 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_bytes": 0, 2024-04-25T14:15:57.291 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_items": 0, 2024-04-25T14:15:57.291 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_bytes": 0, 2024-04-25T14:15:57.291 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_items": 0 2024-04-25T14:15:57.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.291 INFO:teuthology.orchestra.run.smithi096.stdout: "objecter": { 2024-04-25T14:15:57.291 INFO:teuthology.orchestra.run.smithi096.stdout: "op_active": 0, 2024-04-25T14:15:57.291 INFO:teuthology.orchestra.run.smithi096.stdout: "op_laggy": 0, 2024-04-25T14:15:57.291 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send": 530, 2024-04-25T14:15:57.291 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send_bytes": 21, 2024-04-25T14:15:57.291 INFO:teuthology.orchestra.run.smithi096.stdout: "op_resend": 3, 2024-04-25T14:15:57.291 INFO:teuthology.orchestra.run.smithi096.stdout: "op_reply": 530, 2024-04-25T14:15:57.291 INFO:teuthology.orchestra.run.smithi096.stdout: "op_latency": { 2024-04-25T14:15:57.292 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 530, 2024-04-25T14:15:57.292 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.298993642, 2024-04-25T14:15:57.292 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000564138 2024-04-25T14:15:57.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.292 INFO:teuthology.orchestra.run.smithi096.stdout: "op_inflight": 0, 2024-04-25T14:15:57.292 INFO:teuthology.orchestra.run.smithi096.stdout: "oplen_avg": { 2024-04-25T14:15:57.292 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 530, 2024-04-25T14:15:57.292 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 531 2024-04-25T14:15:57.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.292 INFO:teuthology.orchestra.run.smithi096.stdout: "op": 530, 2024-04-25T14:15:57.292 INFO:teuthology.orchestra.run.smithi096.stdout: "op_r": 530, 2024-04-25T14:15:57.292 INFO:teuthology.orchestra.run.smithi096.stdout: "op_w": 0, 2024-04-25T14:15:57.292 INFO:teuthology.orchestra.run.smithi096.stdout: "op_rmw": 0, 2024-04-25T14:15:57.292 INFO:teuthology.orchestra.run.smithi096.stdout: "op_pg": 0, 2024-04-25T14:15:57.293 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_stat": 262, 2024-04-25T14:15:57.293 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_create": 0, 2024-04-25T14:15:57.293 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_read": 266, 2024-04-25T14:15:57.293 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_write": 0, 2024-04-25T14:15:57.293 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writefull": 0, 2024-04-25T14:15:57.293 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writesame": 0, 2024-04-25T14:15:57.293 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_append": 0, 2024-04-25T14:15:57.293 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_zero": 0, 2024-04-25T14:15:57.293 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_truncate": 0, 2024-04-25T14:15:57.293 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_delete": 0, 2024-04-25T14:15:57.293 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_mapext": 0, 2024-04-25T14:15:57.293 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_sparse_read": 0, 2024-04-25T14:15:57.293 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_clonerange": 0, 2024-04-25T14:15:57.293 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_getxattr": 1, 2024-04-25T14:15:57.293 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_setxattr": 0, 2024-04-25T14:15:57.294 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_cmpxattr": 0, 2024-04-25T14:15:57.294 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_rmxattr": 0, 2024-04-25T14:15:57.294 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_resetxattrs": 0, 2024-04-25T14:15:57.294 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_call": 0, 2024-04-25T14:15:57.294 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_watch": 0, 2024-04-25T14:15:57.294 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_notify": 0, 2024-04-25T14:15:57.294 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T14:15:57.294 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls": 0, 2024-04-25T14:15:57.294 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls_filter": 0, 2024-04-25T14:15:57.294 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_other": 0, 2024-04-25T14:15:57.294 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_active": 0, 2024-04-25T14:15:57.294 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_send": 0, 2024-04-25T14:15:57.294 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_resend": 0, 2024-04-25T14:15:57.294 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_ping": 0, 2024-04-25T14:15:57.295 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_active": 0, 2024-04-25T14:15:57.295 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_send": 0, 2024-04-25T14:15:57.295 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_resend": 0, 2024-04-25T14:15:57.295 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_active": 0, 2024-04-25T14:15:57.295 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_send": 0, 2024-04-25T14:15:57.295 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_resend": 0, 2024-04-25T14:15:57.295 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_active": 0, 2024-04-25T14:15:57.295 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_send": 0, 2024-04-25T14:15:57.295 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_resend": 0, 2024-04-25T14:15:57.295 INFO:teuthology.orchestra.run.smithi096.stdout: "command_active": 0, 2024-04-25T14:15:57.295 INFO:teuthology.orchestra.run.smithi096.stdout: "command_send": 0, 2024-04-25T14:15:57.295 INFO:teuthology.orchestra.run.smithi096.stdout: "command_resend": 0, 2024-04-25T14:15:57.295 INFO:teuthology.orchestra.run.smithi096.stdout: "map_epoch": 78, 2024-04-25T14:15:57.295 INFO:teuthology.orchestra.run.smithi096.stdout: "map_full": 0, 2024-04-25T14:15:57.296 INFO:teuthology.orchestra.run.smithi096.stdout: "map_inc": 2, 2024-04-25T14:15:57.296 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_sessions": 5, 2024-04-25T14:15:57.296 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_open": 5, 2024-04-25T14:15:57.296 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_close": 0, 2024-04-25T14:15:57.296 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_laggy": 0, 2024-04-25T14:15:57.296 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_wr": 0, 2024-04-25T14:15:57.296 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_rd": 2, 2024-04-25T14:15:57.296 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_del": 0 2024-04-25T14:15:57.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.296 INFO:teuthology.orchestra.run.smithi096.stdout: "oft": { 2024-04-25T14:15:57.296 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_objs": 0, 2024-04-25T14:15:57.296 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_kv_pairs": 0, 2024-04-25T14:15:57.296 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_updates": 0, 2024-04-25T14:15:57.296 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_removes": 0 2024-04-25T14:15:57.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.297 INFO:teuthology.orchestra.run.smithi096.stdout: "osd-slow-ops": { 2024-04-25T14:15:57.297 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_ops_count": 0 2024-04-25T14:15:57.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.297 INFO:teuthology.orchestra.run.smithi096.stdout: "purge_queue": { 2024-04-25T14:15:57.297 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops": 0, 2024-04-25T14:15:57.297 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T14:15:57.297 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing": 0, 2024-04-25T14:15:57.297 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_high_water": 0, 2024-04-25T14:15:57.297 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed_ops": 0, 2024-04-25T14:15:57.298 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed": 0, 2024-04-25T14:15:57.298 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_item_in_journal": 0 2024-04-25T14:15:57.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.298 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:15:57.298 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-04-25T14:15:57.298 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-04-25T14:15:57.298 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-04-25T14:15:57.298 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 621, 2024-04-25T14:15:57.298 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 590301, 2024-04-25T14:15:57.298 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-04-25T14:15:57.298 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 621, 2024-04-25T14:15:57.298 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-04-25T14:15:57.298 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-04-25T14:15:57.299 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 621, 2024-04-25T14:15:57.299 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 590301, 2024-04-25T14:15:57.299 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-04-25T14:15:57.299 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:15:57.299 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:15:57.299 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:15:57.299 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:57.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.299 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_bytes": { 2024-04-25T14:15:57.299 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-04-25T14:15:57.299 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-04-25T14:15:57.299 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-04-25T14:15:57.299 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-04-25T14:15:57.300 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-04-25T14:15:57.300 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-04-25T14:15:57.300 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-04-25T14:15:57.300 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 530, 2024-04-25T14:15:57.300 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 593, 2024-04-25T14:15:57.300 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 530, 2024-04-25T14:15:57.300 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 593, 2024-04-25T14:15:57.300 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-04-25T14:15:57.300 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:15:57.300 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:15:57.300 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:15:57.300 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:57.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.300 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_ops": { 2024-04-25T14:15:57.301 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-04-25T14:15:57.301 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 1024, 2024-04-25T14:15:57.301 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-04-25T14:15:57.301 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-04-25T14:15:57.301 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-04-25T14:15:57.301 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-04-25T14:15:57.301 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-04-25T14:15:57.301 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 530, 2024-04-25T14:15:57.301 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 530, 2024-04-25T14:15:57.301 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 530, 2024-04-25T14:15:57.301 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 530, 2024-04-25T14:15:57.301 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-04-25T14:15:57.301 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:15:57.301 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:15:57.302 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:15:57.302 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:57.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.302 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle": { 2024-04-25T14:15:57.302 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-04-25T14:15:57.302 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-04-25T14:15:57.302 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-04-25T14:15:57.302 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-04-25T14:15:57.302 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-04-25T14:15:57.302 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-04-25T14:15:57.302 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-04-25T14:15:57.302 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-04-25T14:15:57.302 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-04-25T14:15:57.302 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 0, 2024-04-25T14:15:57.303 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 0, 2024-04-25T14:15:57.303 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-04-25T14:15:57.303 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:15:57.303 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:15:57.303 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:15:57.303 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:57.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:15:57.303 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle-0x55837b96e8f0": { 2024-04-25T14:15:57.303 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-04-25T14:15:57.303 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-04-25T14:15:57.303 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-04-25T14:15:57.303 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-04-25T14:15:57.303 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-04-25T14:15:57.303 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-04-25T14:15:57.304 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-04-25T14:15:57.304 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-04-25T14:15:57.304 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-04-25T14:15:57.304 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 0, 2024-04-25T14:15:57.304 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 0, 2024-04-25T14:15:57.304 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-04-25T14:15:57.304 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:15:57.304 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:15:57.304 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:15:57.304 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:57.304 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:15:57.304 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T14:15:57.743 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:15:57 smithi096 ceph-mon[29207]: pgmap v416: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 511 B/s wr, 12 op/s 2024-04-25T14:15:57.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:15:57 smithi097 ceph-mon[32720]: pgmap v416: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 511 B/s wr, 12 op/s 2024-04-25T14:15:57.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:15:57 smithi151 ceph-mon[32990]: pgmap v416: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 511 B/s wr, 12 op/s 2024-04-25T14:15:58.096 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-04-25T14:15:58.096 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-04-25T14:15:58.096 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-04-25T14:15:58.097 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-04-25T14:15:58.097 DEBUG:tasks.check_counter:Getting stats from d 2024-04-25T14:15:58.097 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-25T14:15:58.342 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:16:00.244 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T14:16:00.244 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:16:00.244 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 4, 2024-04-25T14:16:00.244 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 4, 2024-04-25T14:16:00.244 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 2992, 2024-04-25T14:16:00.244 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 448, 2024-04-25T14:16:00.244 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 4, 2024-04-25T14:16:00.244 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 0, 2024-04-25T14:16:00.244 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.001401410, 2024-04-25T14:16:00.244 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.000038027, 2024-04-25T14:16:00.245 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.000558540, 2024-04-25T14:16:00.245 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T14:16:00.245 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:16:00.245 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 3, 2024-04-25T14:16:00.245 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000676859, 2024-04-25T14:16:00.245 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000225619 2024-04-25T14:16:00.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:00.245 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:16:00.245 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:00.245 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:00.245 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:00.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:00.245 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-25T14:16:00.245 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T14:16:00.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:00.246 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:16:00.246 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 1, 2024-04-25T14:16:00.246 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 63, 2024-04-25T14:16:00.246 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 160, 2024-04-25T14:16:00.246 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 89552, 2024-04-25T14:16:00.246 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 3, 2024-04-25T14:16:00.246 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 2, 2024-04-25T14:16:00.246 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.005328661, 2024-04-25T14:16:00.246 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.003195616, 2024-04-25T14:16:00.246 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.001512243, 2024-04-25T14:16:00.246 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T14:16:00.246 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:16:00.247 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 61, 2024-04-25T14:16:00.247 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.002924077, 2024-04-25T14:16:00.247 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000047935 2024-04-25T14:16:00.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:00.247 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:16:00.247 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:00.247 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:00.247 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:00.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:00.247 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T14:16:00.247 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 89552 2024-04-25T14:16:00.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:00.247 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:16:00.247 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 87, 2024-04-25T14:16:00.247 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 84, 2024-04-25T14:16:00.248 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 469296, 2024-04-25T14:16:00.248 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 50272, 2024-04-25T14:16:00.248 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 2, 2024-04-25T14:16:00.248 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 1, 2024-04-25T14:16:00.248 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.015522190, 2024-04-25T14:16:00.248 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.004775199, 2024-04-25T14:16:00.248 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.012295741, 2024-04-25T14:16:00.248 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.001530377, 2024-04-25T14:16:00.248 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:16:00.248 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 83, 2024-04-25T14:16:00.248 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.003739741, 2024-04-25T14:16:00.248 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000045057 2024-04-25T14:16:00.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:00.248 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:16:00.248 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:00.248 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:00.249 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:00.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:00.249 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 469296, 2024-04-25T14:16:00.249 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 50272 2024-04-25T14:16:00.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:00.249 INFO:teuthology.orchestra.run.smithi096.stdout: "cct": { 2024-04-25T14:16:00.249 INFO:teuthology.orchestra.run.smithi096.stdout: "total_workers": 0, 2024-04-25T14:16:00.249 INFO:teuthology.orchestra.run.smithi096.stdout: "unhealthy_workers": 0 2024-04-25T14:16:00.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:00.249 INFO:teuthology.orchestra.run.smithi096.stdout: "mempool": { 2024-04-25T14:16:00.249 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:16:00.249 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_items": 0, 2024-04-25T14:16:00.249 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:16:00.249 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:16:00.249 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:16:00.250 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:16:00.250 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:16:00.250 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:16:00.250 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:16:00.250 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:16:00.250 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:16:00.250 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:16:00.250 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:16:00.250 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:16:00.250 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:16:00.250 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_items": 0, 2024-04-25T14:16:00.250 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:16:00.250 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_items": 0, 2024-04-25T14:16:00.250 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:16:00.250 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:16:00.251 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:16:00.251 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:16:00.251 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:16:00.251 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:16:00.251 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:16:00.251 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_items": 0, 2024-04-25T14:16:00.251 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:16:00.251 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:16:00.251 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:16:00.251 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_items": 0, 2024-04-25T14:16:00.252 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_bytes": 0, 2024-04-25T14:16:00.252 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_items": 0, 2024-04-25T14:16:00.252 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:16:00.252 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:16:00.252 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:16:00.252 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:16:00.252 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_bytes": 148823, 2024-04-25T14:16:00.252 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_items": 27, 2024-04-25T14:16:00.252 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_bytes": 0, 2024-04-25T14:16:00.252 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_items": 0, 2024-04-25T14:16:00.252 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bytes": 0, 2024-04-25T14:16:00.252 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_items": 0, 2024-04-25T14:16:00.252 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:16:00.252 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_items": 0, 2024-04-25T14:16:00.252 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:16:00.253 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_items": 0, 2024-04-25T14:16:00.253 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_bytes": 0, 2024-04-25T14:16:00.253 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_items": 0, 2024-04-25T14:16:00.253 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:16:00.253 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:16:00.253 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_bytes": 0, 2024-04-25T14:16:00.253 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_items": 0, 2024-04-25T14:16:00.253 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_bytes": 928, 2024-04-25T14:16:00.253 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_items": 2, 2024-04-25T14:16:00.253 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_bytes": 0, 2024-04-25T14:16:00.253 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_items": 0, 2024-04-25T14:16:00.253 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_bytes": 0, 2024-04-25T14:16:00.253 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_items": 0 2024-04-25T14:16:00.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:00.254 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:16:00.254 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-04-25T14:16:00.254 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-04-25T14:16:00.254 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-04-25T14:16:00.254 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 88, 2024-04-25T14:16:00.254 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 457051, 2024-04-25T14:16:00.254 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-04-25T14:16:00.255 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 88, 2024-04-25T14:16:00.255 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-04-25T14:16:00.255 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-04-25T14:16:00.255 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 88, 2024-04-25T14:16:00.255 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 457051, 2024-04-25T14:16:00.255 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-04-25T14:16:00.255 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:00.255 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:00.255 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:00.255 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:16:00.255 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:16:00.255 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T14:16:00.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:00 smithi096 ceph-mon[29207]: pgmap v417: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-25T14:16:00.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:00 smithi097 ceph-mon[32720]: pgmap v417: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-25T14:16:00.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:00 smithi151 ceph-mon[32990]: pgmap v417: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-25T14:16:00.981 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-25T14:16:00.981 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-25T14:16:00.981 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-04-25T14:16:00.981 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-04-25T14:16:00.981 DEBUG:tasks.check_counter:Getting stats from g 2024-04-25T14:16:00.981 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-25T14:16:01.221 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:16:02.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:02 smithi096 ceph-mon[29207]: pgmap v418: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T14:16:02.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:02 smithi097 ceph-mon[32720]: pgmap v418: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T14:16:02.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:02 smithi151 ceph-mon[32990]: pgmap v418: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T14:16:03.221 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T14:16:03.221 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:16:03.221 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 269, 2024-04-25T14:16:03.221 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 270, 2024-04-25T14:16:03.222 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 90411, 2024-04-25T14:16:03.222 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 85225, 2024-04-25T14:16:03.222 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 6, 2024-04-25T14:16:03.222 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 2, 2024-04-25T14:16:03.222 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.029923772, 2024-04-25T14:16:03.222 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.011621477, 2024-04-25T14:16:03.222 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.007293889, 2024-04-25T14:16:03.222 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.007832477, 2024-04-25T14:16:03.222 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:16:03.222 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 268, 2024-04-25T14:16:03.222 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.011803968, 2024-04-25T14:16:03.222 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000044044 2024-04-25T14:16:03.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.223 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:16:03.223 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:03.223 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:03.223 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:03.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.223 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T14:16:03.223 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T14:16:03.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.223 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:16:03.223 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 365, 2024-04-25T14:16:03.223 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 359, 2024-04-25T14:16:03.223 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 565818, 2024-04-25T14:16:03.223 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 135583, 2024-04-25T14:16:03.224 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 4, 2024-04-25T14:16:03.224 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 3, 2024-04-25T14:16:03.224 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.034370324, 2024-04-25T14:16:03.224 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.011172831, 2024-04-25T14:16:03.224 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.017475036, 2024-04-25T14:16:03.224 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.007570997, 2024-04-25T14:16:03.224 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:16:03.224 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 359, 2024-04-25T14:16:03.224 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.011622982, 2024-04-25T14:16:03.224 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000032375 2024-04-25T14:16:03.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.224 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:16:03.224 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:03.225 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:03.225 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:03.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.225 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 494720, 2024-04-25T14:16:03.225 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 49984 2024-04-25T14:16:03.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.225 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:16:03.225 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 4, 2024-04-25T14:16:03.225 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 64, 2024-04-25T14:16:03.225 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 1508, 2024-04-25T14:16:03.225 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 186709, 2024-04-25T14:16:03.225 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 5, 2024-04-25T14:16:03.225 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 4, 2024-04-25T14:16:03.225 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.005957008, 2024-04-25T14:16:03.226 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.003485827, 2024-04-25T14:16:03.226 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.001818654, 2024-04-25T14:16:03.226 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000061221, 2024-04-25T14:16:03.226 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:16:03.226 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 62, 2024-04-25T14:16:03.226 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.004020398, 2024-04-25T14:16:03.226 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000064845 2024-04-25T14:16:03.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.226 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:16:03.226 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:03.226 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:03.227 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:03.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.227 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T14:16:03.227 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 185744 2024-04-25T14:16:03.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.227 INFO:teuthology.orchestra.run.smithi096.stdout: "cct": { 2024-04-25T14:16:03.227 INFO:teuthology.orchestra.run.smithi096.stdout: "total_workers": 1, 2024-04-25T14:16:03.227 INFO:teuthology.orchestra.run.smithi096.stdout: "unhealthy_workers": 0 2024-04-25T14:16:03.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.227 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-MDSRank": { 2024-04-25T14:16:03.227 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-04-25T14:16:03.227 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-04-25T14:16:03.227 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1073, 2024-04-25T14:16:03.227 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.026943872, 2024-04-25T14:16:03.228 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000025110 2024-04-25T14:16:03.228 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:16:03.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.228 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-PurgeQueue": { 2024-04-25T14:16:03.228 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-04-25T14:16:03.228 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-04-25T14:16:03.228 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:03.228 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:03.228 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:03.228 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:16:03.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.228 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": { 2024-04-25T14:16:03.228 INFO:teuthology.orchestra.run.smithi096.stdout: "request": 0, 2024-04-25T14:16:03.229 INFO:teuthology.orchestra.run.smithi096.stdout: "reply": 0, 2024-04-25T14:16:03.229 INFO:teuthology.orchestra.run.smithi096.stdout: "reply_latency": { 2024-04-25T14:16:03.229 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:03.229 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:03.229 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:03.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.229 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_reply": 0, 2024-04-25T14:16:03.229 INFO:teuthology.orchestra.run.smithi096.stdout: "forward": 0, 2024-04-25T14:16:03.229 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_complete": 0, 2024-04-25T14:16:03.229 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_keys": 0, 2024-04-25T14:16:03.229 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_commit": 0, 2024-04-25T14:16:03.229 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_split": 0, 2024-04-25T14:16:03.230 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_merge": 0, 2024-04-25T14:16:03.230 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes": 0, 2024-04-25T14:16:03.230 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_top": 0, 2024-04-25T14:16:03.230 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_bottom": 0, 2024-04-25T14:16:03.230 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pin_tail": 0, 2024-04-25T14:16:03.230 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pinned": 0, 2024-04-25T14:16:03.230 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_expired": 0, 2024-04-25T14:16:03.230 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_with_caps": 0, 2024-04-25T14:16:03.230 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": 0, 2024-04-25T14:16:03.230 INFO:teuthology.orchestra.run.smithi096.stdout: "subtrees": 1, 2024-04-25T14:16:03.230 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse": 0, 2024-04-25T14:16:03.230 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_hit": 0, 2024-04-25T14:16:03.231 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_forward": 0, 2024-04-25T14:16:03.231 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_discover": 0, 2024-04-25T14:16:03.231 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_dir_fetch": 0, 2024-04-25T14:16:03.231 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_remote_ino": 0, 2024-04-25T14:16:03.231 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_lock": 0, 2024-04-25T14:16:03.231 INFO:teuthology.orchestra.run.smithi096.stdout: "load_cent": 0, 2024-04-25T14:16:03.231 INFO:teuthology.orchestra.run.smithi096.stdout: "q": 0, 2024-04-25T14:16:03.231 INFO:teuthology.orchestra.run.smithi096.stdout: "exported": 0, 2024-04-25T14:16:03.231 INFO:teuthology.orchestra.run.smithi096.stdout: "exported_inodes": 0, 2024-04-25T14:16:03.231 INFO:teuthology.orchestra.run.smithi096.stdout: "imported": 0, 2024-04-25T14:16:03.231 INFO:teuthology.orchestra.run.smithi096.stdout: "imported_inodes": 0, 2024-04-25T14:16:03.231 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_dir_fetch": 0, 2024-04-25T14:16:03.231 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_backtrace_fetch": 0, 2024-04-25T14:16:03.231 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_peer_discover": 0, 2024-04-25T14:16:03.232 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rfiles": 0, 2024-04-25T14:16:03.232 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rbytes": 0, 2024-04-25T14:16:03.232 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rsnaps": 0, 2024-04-25T14:16:03.232 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T14:16:03.232 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_set_tag": 0, 2024-04-25T14:16:03.232 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T14:16:03.232 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_inotable_repaired": 0, 2024-04-25T14:16:03.232 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_inodes": 0, 2024-04-25T14:16:03.232 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T14:16:03.232 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T14:16:03.232 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_file_inodes": 0, 2024-04-25T14:16:03.232 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_inode_file_caps": 0, 2024-04-25T14:16:03.232 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T14:16:03.233 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_grant": 0, 2024-04-25T14:16:03.233 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T14:16:03.233 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T14:16:03.233 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T14:16:03.233 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps": 0, 2024-04-25T14:16:03.233 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps_dirty": 0, 2024-04-25T14:16:03.233 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_cap_release": 0, 2024-04-25T14:16:03.233 INFO:teuthology.orchestra.run.smithi096.stdout: "process_request_cap_release": 0 2024-04-25T14:16:03.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.233 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cache": { 2024-04-25T14:16:03.233 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update": 0, 2024-04-25T14:16:03.233 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update_receipt": 0, 2024-04-25T14:16:03.233 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_try_discover": 0, 2024-04-25T14:16:03.233 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_send_discover": 0, 2024-04-25T14:16:03.234 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_handle_discover": 0, 2024-04-25T14:16:03.234 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays": 0, 2024-04-25T14:16:03.234 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_delayed": 0, 2024-04-25T14:16:03.234 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_enqueuing": 0, 2024-04-25T14:16:03.234 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_created": 0, 2024-04-25T14:16:03.234 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_enqueued": 0, 2024-04-25T14:16:03.234 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_reintegrated": 0, 2024-04-25T14:16:03.234 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_migrated": 0, 2024-04-25T14:16:03.234 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_processing": 0, 2024-04-25T14:16:03.234 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_enqueued": 0, 2024-04-25T14:16:03.234 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_prioritized": 0, 2024-04-25T14:16:03.234 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_started": 0, 2024-04-25T14:16:03.234 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_completed": 0, 2024-04-25T14:16:03.234 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_path": 0, 2024-04-25T14:16:03.235 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_inode": 0, 2024-04-25T14:16:03.235 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T14:16:03.235 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_exportdir": 0, 2024-04-25T14:16:03.235 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_flush": 0, 2024-04-25T14:16:03.235 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragmentdir": 0, 2024-04-25T14:16:03.235 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragstats": 0, 2024-04-25T14:16:03.235 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_inodestats": 0 2024-04-25T14:16:03.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.235 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log": { 2024-04-25T14:16:03.235 INFO:teuthology.orchestra.run.smithi096.stdout: "evlrg": 0, 2024-04-25T14:16:03.235 INFO:teuthology.orchestra.run.smithi096.stdout: "evadd": 0, 2024-04-25T14:16:03.235 INFO:teuthology.orchestra.run.smithi096.stdout: "evex": 0, 2024-04-25T14:16:03.235 INFO:teuthology.orchestra.run.smithi096.stdout: "evtrm": 0, 2024-04-25T14:16:03.235 INFO:teuthology.orchestra.run.smithi096.stdout: "ev": 2, 2024-04-25T14:16:03.235 INFO:teuthology.orchestra.run.smithi096.stdout: "evexg": 0, 2024-04-25T14:16:03.236 INFO:teuthology.orchestra.run.smithi096.stdout: "evexd": 0, 2024-04-25T14:16:03.236 INFO:teuthology.orchestra.run.smithi096.stdout: "segadd": 0, 2024-04-25T14:16:03.236 INFO:teuthology.orchestra.run.smithi096.stdout: "segex": 0, 2024-04-25T14:16:03.236 INFO:teuthology.orchestra.run.smithi096.stdout: "segtrm": 0, 2024-04-25T14:16:03.236 INFO:teuthology.orchestra.run.smithi096.stdout: "seg": 2, 2024-04-25T14:16:03.236 INFO:teuthology.orchestra.run.smithi096.stdout: "segmjr": 2, 2024-04-25T14:16:03.236 INFO:teuthology.orchestra.run.smithi096.stdout: "segexg": 0, 2024-04-25T14:16:03.236 INFO:teuthology.orchestra.run.smithi096.stdout: "segexd": 0, 2024-04-25T14:16:03.236 INFO:teuthology.orchestra.run.smithi096.stdout: "expos": 4194304, 2024-04-25T14:16:03.236 INFO:teuthology.orchestra.run.smithi096.stdout: "wrpos": 4194892, 2024-04-25T14:16:03.236 INFO:teuthology.orchestra.run.smithi096.stdout: "rdpos": 4194352, 2024-04-25T14:16:03.236 INFO:teuthology.orchestra.run.smithi096.stdout: "jlat": { 2024-04-25T14:16:03.236 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:03.236 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:03.237 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:03.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.237 INFO:teuthology.orchestra.run.smithi096.stdout: "replayed": 2 2024-04-25T14:16:03.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.237 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_mem": { 2024-04-25T14:16:03.237 INFO:teuthology.orchestra.run.smithi096.stdout: "ino": 3, 2024-04-25T14:16:03.237 INFO:teuthology.orchestra.run.smithi096.stdout: "ino+": 3, 2024-04-25T14:16:03.237 INFO:teuthology.orchestra.run.smithi096.stdout: "ino-": 0, 2024-04-25T14:16:03.237 INFO:teuthology.orchestra.run.smithi096.stdout: "dir": 1, 2024-04-25T14:16:03.237 INFO:teuthology.orchestra.run.smithi096.stdout: "dir+": 1, 2024-04-25T14:16:03.237 INFO:teuthology.orchestra.run.smithi096.stdout: "dir-": 0, 2024-04-25T14:16:03.237 INFO:teuthology.orchestra.run.smithi096.stdout: "dn": 0, 2024-04-25T14:16:03.237 INFO:teuthology.orchestra.run.smithi096.stdout: "dn+": 0, 2024-04-25T14:16:03.237 INFO:teuthology.orchestra.run.smithi096.stdout: "dn-": 0, 2024-04-25T14:16:03.237 INFO:teuthology.orchestra.run.smithi096.stdout: "cap": 0, 2024-04-25T14:16:03.237 INFO:teuthology.orchestra.run.smithi096.stdout: "cap+": 0, 2024-04-25T14:16:03.238 INFO:teuthology.orchestra.run.smithi096.stdout: "cap-": 0, 2024-04-25T14:16:03.238 INFO:teuthology.orchestra.run.smithi096.stdout: "rss": 118012, 2024-04-25T14:16:03.238 INFO:teuthology.orchestra.run.smithi096.stdout: "heap": 215440 2024-04-25T14:16:03.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.238 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_server": { 2024-04-25T14:16:03.238 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_client_request": 0, 2024-04-25T14:16:03.238 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_server_request": 0, 2024-04-25T14:16:03.238 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_request": 0, 2024-04-25T14:16:03.238 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_session": 0, 2024-04-25T14:16:03.238 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_peer_request": 0, 2024-04-25T14:16:03.238 INFO:teuthology.orchestra.run.smithi096.stdout: "req_create_latency": { 2024-04-25T14:16:03.238 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:03.238 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:03.239 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:03.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.239 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getattr_latency": { 2024-04-25T14:16:03.239 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:03.239 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:03.239 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:03.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.239 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getfilelock_latency": { 2024-04-25T14:16:03.239 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:03.239 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:03.239 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:03.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.239 INFO:teuthology.orchestra.run.smithi096.stdout: "req_link_latency": { 2024-04-25T14:16:03.240 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:03.240 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:03.240 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:03.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.240 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookup_latency": { 2024-04-25T14:16:03.240 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:03.240 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:03.240 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:03.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.240 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookuphash_latency": { 2024-04-25T14:16:03.240 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:03.240 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:03.240 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:03.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.241 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupino_latency": { 2024-04-25T14:16:03.241 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:03.241 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:03.241 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:03.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.241 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupname_latency": { 2024-04-25T14:16:03.241 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:03.241 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:03.241 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:03.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.241 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupparent_latency": { 2024-04-25T14:16:03.241 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:03.241 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:03.241 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:03.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.242 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupsnap_latency": { 2024-04-25T14:16:03.242 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:03.242 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:03.242 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:03.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.242 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lssnap_latency": { 2024-04-25T14:16:03.242 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:03.242 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:03.242 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:03.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.242 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mkdir_latency": { 2024-04-25T14:16:03.242 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:03.242 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:03.243 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:03.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.243 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mknod_latency": { 2024-04-25T14:16:03.243 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:03.243 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:03.244 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:03.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.244 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mksnap_latency": { 2024-04-25T14:16:03.244 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:03.244 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:03.244 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:03.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.244 INFO:teuthology.orchestra.run.smithi096.stdout: "req_open_latency": { 2024-04-25T14:16:03.244 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:03.245 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:03.245 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:03.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.245 INFO:teuthology.orchestra.run.smithi096.stdout: "req_readdir_latency": { 2024-04-25T14:16:03.245 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:03.245 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:03.245 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:03.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.245 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rename_latency": { 2024-04-25T14:16:03.245 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:03.246 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:03.246 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:03.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.246 INFO:teuthology.orchestra.run.smithi096.stdout: "req_renamesnap_latency": { 2024-04-25T14:16:03.246 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:03.246 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:03.246 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:03.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.246 INFO:teuthology.orchestra.run.smithi096.stdout: "req_snapdiff_latency": { 2024-04-25T14:16:03.246 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:03.247 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:03.247 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:03.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.247 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmdir_latency": { 2024-04-25T14:16:03.247 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:03.247 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:03.247 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:03.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.247 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmsnap_latency": { 2024-04-25T14:16:03.247 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:03.247 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:03.248 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:03.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.248 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmxattr_latency": { 2024-04-25T14:16:03.248 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:03.248 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:03.248 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:03.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.248 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setattr_latency": { 2024-04-25T14:16:03.248 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:03.248 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:03.248 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:03.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.248 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setdirlayout_latency": { 2024-04-25T14:16:03.249 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:03.249 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:03.249 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:03.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.249 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setfilelock_latency": { 2024-04-25T14:16:03.249 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:03.249 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:03.249 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:03.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.249 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setlayout_latency": { 2024-04-25T14:16:03.249 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:03.250 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:03.250 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:03.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.250 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setxattr_latency": { 2024-04-25T14:16:03.250 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:03.250 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:03.250 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:03.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.250 INFO:teuthology.orchestra.run.smithi096.stdout: "req_symlink_latency": { 2024-04-25T14:16:03.250 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:03.250 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:03.251 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:03.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.251 INFO:teuthology.orchestra.run.smithi096.stdout: "req_unlink_latency": { 2024-04-25T14:16:03.251 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:03.251 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:03.251 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:03.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.251 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_revoke_eviction": 0, 2024-04-25T14:16:03.251 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_acquisition_throttle": 0, 2024-04-25T14:16:03.251 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getvxattr_latency": { 2024-04-25T14:16:03.251 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:03.252 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:03.252 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:03.252 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:16:03.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.252 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_sessions": { 2024-04-25T14:16:03.252 INFO:teuthology.orchestra.run.smithi096.stdout: "session_count": 0, 2024-04-25T14:16:03.252 INFO:teuthology.orchestra.run.smithi096.stdout: "session_add": 0, 2024-04-25T14:16:03.252 INFO:teuthology.orchestra.run.smithi096.stdout: "session_remove": 0, 2024-04-25T14:16:03.252 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_open": 0, 2024-04-25T14:16:03.252 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_stale": 0, 2024-04-25T14:16:03.252 INFO:teuthology.orchestra.run.smithi096.stdout: "total_load": 0, 2024-04-25T14:16:03.253 INFO:teuthology.orchestra.run.smithi096.stdout: "average_load": 0, 2024-04-25T14:16:03.253 INFO:teuthology.orchestra.run.smithi096.stdout: "avg_session_uptime": 0, 2024-04-25T14:16:03.253 INFO:teuthology.orchestra.run.smithi096.stdout: "mdthresh_evicted": 0 2024-04-25T14:16:03.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.253 INFO:teuthology.orchestra.run.smithi096.stdout: "mempool": { 2024-04-25T14:16:03.253 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:16:03.253 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_items": 0, 2024-04-25T14:16:03.253 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:16:03.253 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:16:03.253 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:16:03.253 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:16:03.254 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:16:03.254 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:16:03.254 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:16:03.254 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:16:03.254 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:16:03.254 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:16:03.254 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:16:03.254 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:16:03.254 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:16:03.254 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_items": 0, 2024-04-25T14:16:03.254 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:16:03.254 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_items": 0, 2024-04-25T14:16:03.255 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:16:03.255 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:16:03.255 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:16:03.255 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:16:03.255 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:16:03.255 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:16:03.255 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:16:03.255 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_items": 0, 2024-04-25T14:16:03.255 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:16:03.255 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:16:03.255 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:16:03.255 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_items": 0, 2024-04-25T14:16:03.255 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_bytes": 0, 2024-04-25T14:16:03.256 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_items": 0, 2024-04-25T14:16:03.256 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:16:03.256 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:16:03.256 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:16:03.256 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:16:03.256 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_bytes": 197399, 2024-04-25T14:16:03.256 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_items": 54, 2024-04-25T14:16:03.256 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_bytes": 0, 2024-04-25T14:16:03.256 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_items": 0, 2024-04-25T14:16:03.256 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bytes": 0, 2024-04-25T14:16:03.256 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_items": 0, 2024-04-25T14:16:03.256 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:16:03.257 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_items": 0, 2024-04-25T14:16:03.257 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:16:03.257 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_items": 0, 2024-04-25T14:16:03.257 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_bytes": 7720, 2024-04-25T14:16:03.257 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_items": 162, 2024-04-25T14:16:03.257 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:16:03.257 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:16:03.257 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_bytes": 0, 2024-04-25T14:16:03.257 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_items": 0, 2024-04-25T14:16:03.257 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_bytes": 8512, 2024-04-25T14:16:03.257 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_items": 13, 2024-04-25T14:16:03.257 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_bytes": 0, 2024-04-25T14:16:03.258 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_items": 0, 2024-04-25T14:16:03.258 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_bytes": 0, 2024-04-25T14:16:03.258 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_items": 0 2024-04-25T14:16:03.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.258 INFO:teuthology.orchestra.run.smithi096.stdout: "objecter": { 2024-04-25T14:16:03.258 INFO:teuthology.orchestra.run.smithi096.stdout: "op_active": 0, 2024-04-25T14:16:03.258 INFO:teuthology.orchestra.run.smithi096.stdout: "op_laggy": 0, 2024-04-25T14:16:03.258 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send": 540, 2024-04-25T14:16:03.258 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send_bytes": 21, 2024-04-25T14:16:03.258 INFO:teuthology.orchestra.run.smithi096.stdout: "op_resend": 3, 2024-04-25T14:16:03.258 INFO:teuthology.orchestra.run.smithi096.stdout: "op_reply": 540, 2024-04-25T14:16:03.258 INFO:teuthology.orchestra.run.smithi096.stdout: "op_latency": { 2024-04-25T14:16:03.258 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 540, 2024-04-25T14:16:03.259 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 1.215974148, 2024-04-25T14:16:03.259 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.002251803 2024-04-25T14:16:03.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.259 INFO:teuthology.orchestra.run.smithi096.stdout: "op_inflight": 0, 2024-04-25T14:16:03.259 INFO:teuthology.orchestra.run.smithi096.stdout: "oplen_avg": { 2024-04-25T14:16:03.259 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 540, 2024-04-25T14:16:03.259 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 541 2024-04-25T14:16:03.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.259 INFO:teuthology.orchestra.run.smithi096.stdout: "op": 540, 2024-04-25T14:16:03.259 INFO:teuthology.orchestra.run.smithi096.stdout: "op_r": 540, 2024-04-25T14:16:03.259 INFO:teuthology.orchestra.run.smithi096.stdout: "op_w": 0, 2024-04-25T14:16:03.259 INFO:teuthology.orchestra.run.smithi096.stdout: "op_rmw": 0, 2024-04-25T14:16:03.260 INFO:teuthology.orchestra.run.smithi096.stdout: "op_pg": 0, 2024-04-25T14:16:03.260 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_stat": 267, 2024-04-25T14:16:03.260 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_create": 0, 2024-04-25T14:16:03.260 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_read": 271, 2024-04-25T14:16:03.260 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_write": 0, 2024-04-25T14:16:03.260 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writefull": 0, 2024-04-25T14:16:03.260 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writesame": 0, 2024-04-25T14:16:03.260 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_append": 0, 2024-04-25T14:16:03.260 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_zero": 0, 2024-04-25T14:16:03.260 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_truncate": 0, 2024-04-25T14:16:03.260 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_delete": 0, 2024-04-25T14:16:03.260 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_mapext": 0, 2024-04-25T14:16:03.260 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_sparse_read": 0, 2024-04-25T14:16:03.261 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_clonerange": 0, 2024-04-25T14:16:03.261 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_getxattr": 1, 2024-04-25T14:16:03.261 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_setxattr": 0, 2024-04-25T14:16:03.261 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_cmpxattr": 0, 2024-04-25T14:16:03.261 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_rmxattr": 0, 2024-04-25T14:16:03.261 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_resetxattrs": 0, 2024-04-25T14:16:03.261 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_call": 0, 2024-04-25T14:16:03.261 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_watch": 0, 2024-04-25T14:16:03.261 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_notify": 0, 2024-04-25T14:16:03.261 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T14:16:03.261 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls": 0, 2024-04-25T14:16:03.261 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls_filter": 0, 2024-04-25T14:16:03.262 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_other": 0, 2024-04-25T14:16:03.262 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_active": 0, 2024-04-25T14:16:03.262 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_send": 0, 2024-04-25T14:16:03.262 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_resend": 0, 2024-04-25T14:16:03.262 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_ping": 0, 2024-04-25T14:16:03.262 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_active": 0, 2024-04-25T14:16:03.262 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_send": 0, 2024-04-25T14:16:03.262 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_resend": 0, 2024-04-25T14:16:03.262 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_active": 0, 2024-04-25T14:16:03.262 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_send": 0, 2024-04-25T14:16:03.262 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_resend": 0, 2024-04-25T14:16:03.262 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_active": 0, 2024-04-25T14:16:03.262 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_send": 0, 2024-04-25T14:16:03.263 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_resend": 0, 2024-04-25T14:16:03.263 INFO:teuthology.orchestra.run.smithi096.stdout: "command_active": 0, 2024-04-25T14:16:03.263 INFO:teuthology.orchestra.run.smithi096.stdout: "command_send": 0, 2024-04-25T14:16:03.263 INFO:teuthology.orchestra.run.smithi096.stdout: "command_resend": 0, 2024-04-25T14:16:03.263 INFO:teuthology.orchestra.run.smithi096.stdout: "map_epoch": 78, 2024-04-25T14:16:03.263 INFO:teuthology.orchestra.run.smithi096.stdout: "map_full": 0, 2024-04-25T14:16:03.263 INFO:teuthology.orchestra.run.smithi096.stdout: "map_inc": 2, 2024-04-25T14:16:03.263 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_sessions": 6, 2024-04-25T14:16:03.263 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_open": 6, 2024-04-25T14:16:03.263 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_close": 0, 2024-04-25T14:16:03.263 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_laggy": 0, 2024-04-25T14:16:03.264 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_wr": 0, 2024-04-25T14:16:03.264 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_rd": 2, 2024-04-25T14:16:03.264 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_del": 0 2024-04-25T14:16:03.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.264 INFO:teuthology.orchestra.run.smithi096.stdout: "oft": { 2024-04-25T14:16:03.264 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_objs": 0, 2024-04-25T14:16:03.264 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_kv_pairs": 0, 2024-04-25T14:16:03.264 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_updates": 0, 2024-04-25T14:16:03.264 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_removes": 0 2024-04-25T14:16:03.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.264 INFO:teuthology.orchestra.run.smithi096.stdout: "osd-slow-ops": { 2024-04-25T14:16:03.264 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_ops_count": 0 2024-04-25T14:16:03.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.265 INFO:teuthology.orchestra.run.smithi096.stdout: "purge_queue": { 2024-04-25T14:16:03.265 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops": 0, 2024-04-25T14:16:03.265 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T14:16:03.265 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing": 0, 2024-04-25T14:16:03.265 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_high_water": 0, 2024-04-25T14:16:03.265 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed_ops": 0, 2024-04-25T14:16:03.265 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed": 0, 2024-04-25T14:16:03.265 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_item_in_journal": 0 2024-04-25T14:16:03.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.265 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:16:03.265 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-04-25T14:16:03.266 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-04-25T14:16:03.266 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-04-25T14:16:03.266 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 634, 2024-04-25T14:16:03.266 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 592979, 2024-04-25T14:16:03.266 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-04-25T14:16:03.266 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 634, 2024-04-25T14:16:03.266 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-04-25T14:16:03.266 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-04-25T14:16:03.266 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 634, 2024-04-25T14:16:03.266 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 592979, 2024-04-25T14:16:03.266 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-04-25T14:16:03.267 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:03.267 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:03.267 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:03.267 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:16:03.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.267 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_bytes": { 2024-04-25T14:16:03.267 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-04-25T14:16:03.267 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-04-25T14:16:03.267 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-04-25T14:16:03.267 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-04-25T14:16:03.267 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-04-25T14:16:03.267 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-04-25T14:16:03.268 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-04-25T14:16:03.268 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 540, 2024-04-25T14:16:03.268 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 593, 2024-04-25T14:16:03.268 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 540, 2024-04-25T14:16:03.268 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 593, 2024-04-25T14:16:03.268 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-04-25T14:16:03.268 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:03.268 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:03.268 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:03.268 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:16:03.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.269 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_ops": { 2024-04-25T14:16:03.269 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-04-25T14:16:03.269 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 1024, 2024-04-25T14:16:03.269 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-04-25T14:16:03.269 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-04-25T14:16:03.269 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-04-25T14:16:03.269 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-04-25T14:16:03.269 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-04-25T14:16:03.269 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 540, 2024-04-25T14:16:03.269 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 540, 2024-04-25T14:16:03.269 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 540, 2024-04-25T14:16:03.270 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 540, 2024-04-25T14:16:03.270 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-04-25T14:16:03.270 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:03.270 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:03.270 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:03.270 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:16:03.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.270 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle": { 2024-04-25T14:16:03.270 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-04-25T14:16:03.270 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-04-25T14:16:03.271 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-04-25T14:16:03.271 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-04-25T14:16:03.271 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-04-25T14:16:03.271 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-04-25T14:16:03.271 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-04-25T14:16:03.271 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-04-25T14:16:03.271 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-04-25T14:16:03.271 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 0, 2024-04-25T14:16:03.271 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 0, 2024-04-25T14:16:03.271 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-04-25T14:16:03.271 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:03.271 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:03.272 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:03.272 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:16:03.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:03.272 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle-0x55c48fe6a8f0": { 2024-04-25T14:16:03.272 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-04-25T14:16:03.272 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-04-25T14:16:03.272 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-04-25T14:16:03.272 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-04-25T14:16:03.272 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-04-25T14:16:03.272 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-04-25T14:16:03.272 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-04-25T14:16:03.272 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-04-25T14:16:03.272 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-04-25T14:16:03.273 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 0, 2024-04-25T14:16:03.273 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 0, 2024-04-25T14:16:03.273 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-04-25T14:16:03.273 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:03.273 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:03.273 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:03.273 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:16:03.273 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:16:03.273 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T14:16:04.051 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-04-25T14:16:04.051 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-04-25T14:16:04.051 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-04-25T14:16:04.051 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-04-25T14:16:04.051 DEBUG:tasks.check_counter:Getting stats from j 2024-04-25T14:16:04.051 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-25T14:16:04.291 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:16:04.728 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:04 smithi096 ceph-mon[29207]: pgmap v419: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T14:16:04.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:04 smithi097 ceph-mon[32720]: pgmap v419: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T14:16:04.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:04 smithi151 ceph-mon[32990]: pgmap v419: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T14:16:06.132 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T14:16:06.132 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:16:06.132 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 4, 2024-04-25T14:16:06.133 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 4, 2024-04-25T14:16:06.133 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 2992, 2024-04-25T14:16:06.133 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 448, 2024-04-25T14:16:06.133 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 4, 2024-04-25T14:16:06.133 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 0, 2024-04-25T14:16:06.133 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.001435584, 2024-04-25T14:16:06.133 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.000032746, 2024-04-25T14:16:06.133 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.000636707, 2024-04-25T14:16:06.133 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T14:16:06.133 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:16:06.133 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 3, 2024-04-25T14:16:06.133 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000746219, 2024-04-25T14:16:06.133 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000248739 2024-04-25T14:16:06.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:06.133 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:16:06.134 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:06.134 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:06.134 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:06.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:06.134 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-25T14:16:06.134 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T14:16:06.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:06.134 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:16:06.134 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 90, 2024-04-25T14:16:06.134 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 87, 2024-04-25T14:16:06.134 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 470112, 2024-04-25T14:16:06.134 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 52192, 2024-04-25T14:16:06.134 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 2, 2024-04-25T14:16:06.134 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 1, 2024-04-25T14:16:06.135 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.014848983, 2024-04-25T14:16:06.135 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.004692418, 2024-04-25T14:16:06.135 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.011448617, 2024-04-25T14:16:06.135 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.001615953, 2024-04-25T14:16:06.135 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:16:06.135 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 86, 2024-04-25T14:16:06.135 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.003774000, 2024-04-25T14:16:06.135 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000043883 2024-04-25T14:16:06.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:06.135 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:16:06.135 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:06.135 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:06.135 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:06.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:06.136 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 470112, 2024-04-25T14:16:06.136 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 52192 2024-04-25T14:16:06.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:06.136 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:16:06.136 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 1, 2024-04-25T14:16:06.136 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 65, 2024-04-25T14:16:06.136 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 160, 2024-04-25T14:16:06.136 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 89904, 2024-04-25T14:16:06.136 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 3, 2024-04-25T14:16:06.136 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 2, 2024-04-25T14:16:06.136 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.005703312, 2024-04-25T14:16:06.136 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.003683528, 2024-04-25T14:16:06.136 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.001292366, 2024-04-25T14:16:06.136 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T14:16:06.136 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:16:06.137 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 63, 2024-04-25T14:16:06.137 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.003237371, 2024-04-25T14:16:06.137 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000051386 2024-04-25T14:16:06.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:06.137 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:16:06.137 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:06.137 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:06.137 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:06.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:06.137 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T14:16:06.137 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 89904 2024-04-25T14:16:06.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:06.137 INFO:teuthology.orchestra.run.smithi096.stdout: "cct": { 2024-04-25T14:16:06.137 INFO:teuthology.orchestra.run.smithi096.stdout: "total_workers": 0, 2024-04-25T14:16:06.138 INFO:teuthology.orchestra.run.smithi096.stdout: "unhealthy_workers": 0 2024-04-25T14:16:06.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:06.138 INFO:teuthology.orchestra.run.smithi096.stdout: "mempool": { 2024-04-25T14:16:06.138 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:16:06.138 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_items": 0, 2024-04-25T14:16:06.138 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:16:06.138 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:16:06.138 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:16:06.138 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:16:06.138 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:16:06.138 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:16:06.138 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:16:06.138 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:16:06.138 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:16:06.139 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:16:06.139 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:16:06.139 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:16:06.139 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:16:06.139 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_items": 0, 2024-04-25T14:16:06.139 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:16:06.139 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_items": 0, 2024-04-25T14:16:06.139 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:16:06.139 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:16:06.139 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:16:06.139 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:16:06.139 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:16:06.139 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:16:06.139 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:16:06.140 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_items": 0, 2024-04-25T14:16:06.140 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:16:06.140 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:16:06.140 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:16:06.140 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_items": 0, 2024-04-25T14:16:06.140 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_bytes": 0, 2024-04-25T14:16:06.140 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_items": 0, 2024-04-25T14:16:06.140 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:16:06.140 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:16:06.140 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:16:06.140 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:16:06.140 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_bytes": 148863, 2024-04-25T14:16:06.140 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_items": 28, 2024-04-25T14:16:06.140 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_bytes": 0, 2024-04-25T14:16:06.140 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_items": 0, 2024-04-25T14:16:06.141 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bytes": 0, 2024-04-25T14:16:06.141 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_items": 0, 2024-04-25T14:16:06.141 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:16:06.141 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_items": 0, 2024-04-25T14:16:06.141 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:16:06.141 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_items": 0, 2024-04-25T14:16:06.141 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_bytes": 0, 2024-04-25T14:16:06.141 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_items": 0, 2024-04-25T14:16:06.141 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:16:06.141 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:16:06.141 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_bytes": 0, 2024-04-25T14:16:06.141 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_items": 0, 2024-04-25T14:16:06.141 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_bytes": 928, 2024-04-25T14:16:06.141 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_items": 2, 2024-04-25T14:16:06.142 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_bytes": 0, 2024-04-25T14:16:06.142 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_items": 0, 2024-04-25T14:16:06.142 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_bytes": 0, 2024-04-25T14:16:06.142 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_items": 0 2024-04-25T14:16:06.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T14:16:06.142 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:16:06.142 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-04-25T14:16:06.142 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-04-25T14:16:06.142 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-04-25T14:16:06.142 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 91, 2024-04-25T14:16:06.142 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 457441, 2024-04-25T14:16:06.142 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-04-25T14:16:06.142 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 91, 2024-04-25T14:16:06.142 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-04-25T14:16:06.143 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-04-25T14:16:06.143 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 91, 2024-04-25T14:16:06.143 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 457441, 2024-04-25T14:16:06.143 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-04-25T14:16:06.143 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T14:16:06.143 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T14:16:06.143 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T14:16:06.143 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:16:06.143 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T14:16:06.143 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T14:16:06.808 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:06 smithi096 ceph-mon[29207]: pgmap v420: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T14:16:06.837 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-25T14:16:06.838 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-25T14:16:06.838 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-04-25T14:16:06.838 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-04-25T14:16:06.838 DEBUG:tasks.check_counter:Getting stats from b 2024-04-25T14:16:06.838 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-25T14:16:06.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:06 smithi097 ceph-mon[32720]: pgmap v420: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T14:16:06.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:06 smithi151 ceph-mon[32990]: pgmap v420: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T14:16:07.085 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T14:16:08.640 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:08 smithi097 ceph-mon[32720]: pgmap v421: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T14:16:08.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:08 smithi096 ceph-mon[29207]: pgmap v421: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T14:16:08.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:08 smithi151 ceph-mon[32990]: pgmap v421: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T14:16:09.084 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T14:16:09.084 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:16:09.084 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 3116, 2024-04-25T14:16:09.085 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 3661, 2024-04-25T14:16:09.085 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 1266563, 2024-04-25T14:16:09.085 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 5373833, 2024-04-25T14:16:09.085 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 11, 2024-04-25T14:16:09.085 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 3, 2024-04-25T14:16:09.085 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.228604511, 2024-04-25T14:16:09.085 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.075975696, 2024-04-25T14:16:09.085 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.089980048, 2024-04-25T14:16:09.085 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.057477939, 2024-04-25T14:16:09.085 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:16:09.085 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 3660, 2024-04-25T14:16:09.085 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.709240013, 2024-04-25T14:16:09.085 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000193781 2024-04-25T14:16:09.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.086 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:16:09.086 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2527, 2024-04-25T14:16:09.086 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000478896, 2024-04-25T14:16:09.086 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000189 2024-04-25T14:16:09.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.086 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 534912, 2024-04-25T14:16:09.086 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 60736 2024-04-25T14:16:09.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.086 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:16:09.086 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 7985, 2024-04-25T14:16:09.086 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 12665, 2024-04-25T14:16:09.087 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 2789838, 2024-04-25T14:16:09.087 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 10020687, 2024-04-25T14:16:09.087 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 16, 2024-04-25T14:16:09.087 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 5, 2024-04-25T14:16:09.087 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.599418876, 2024-04-25T14:16:09.087 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.239318711, 2024-04-25T14:16:09.087 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.359518289, 2024-04-25T14:16:09.087 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.078299780, 2024-04-25T14:16:09.087 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:16:09.087 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 12664, 2024-04-25T14:16:09.087 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 1.368224380, 2024-04-25T14:16:09.087 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000108040 2024-04-25T14:16:09.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.088 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:16:09.088 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 13056, 2024-04-25T14:16:09.088 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.001992232, 2024-04-25T14:16:09.088 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000152 2024-04-25T14:16:09.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.088 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 1949136, 2024-04-25T14:16:09.088 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 4598208 2024-04-25T14:16:09.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.088 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:16:09.088 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 3221, 2024-04-25T14:16:09.088 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 3778, 2024-04-25T14:16:09.088 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 805390, 2024-04-25T14:16:09.088 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 6161247, 2024-04-25T14:16:09.089 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 62, 2024-04-25T14:16:09.089 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 6, 2024-04-25T14:16:09.089 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.277168896, 2024-04-25T14:16:09.089 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.104741413, 2024-04-25T14:16:09.089 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.102812057, 2024-04-25T14:16:09.089 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.061870964, 2024-04-25T14:16:09.089 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:16:09.089 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 3776, 2024-04-25T14:16:09.089 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.468115660, 2024-04-25T14:16:09.089 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000123971 2024-04-25T14:16:09.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.089 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:16:09.089 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2618, 2024-04-25T14:16:09.089 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000506920, 2024-04-25T14:16:09.090 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000193 2024-04-25T14:16:09.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.090 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T14:16:09.090 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T14:16:09.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.090 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2024-04-25T14:16:09.090 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 1, 2024-04-25T14:16:09.090 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2024-04-25T14:16:09.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.090 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-MDSRank": { 2024-04-25T14:16:09.090 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-04-25T14:16:09.090 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-04-25T14:16:09.091 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 4549, 2024-04-25T14:16:09.091 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 2.641919681, 2024-04-25T14:16:09.091 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000580769 2024-04-25T14:16:09.091 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T14:16:09.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.091 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-PurgeQueue": { 2024-04-25T14:16:09.091 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-04-25T14:16:09.091 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-04-25T14:16:09.091 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 5552, 2024-04-25T14:16:09.091 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.209365769, 2024-04-25T14:16:09.091 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000037709 2024-04-25T14:16:09.091 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T14:16:09.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.091 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": { 2024-04-25T14:16:09.092 INFO:teuthology.orchestra.run.smithi097.stdout: "request": 2468, 2024-04-25T14:16:09.092 INFO:teuthology.orchestra.run.smithi097.stdout: "reply": 2468, 2024-04-25T14:16:09.092 INFO:teuthology.orchestra.run.smithi097.stdout: "reply_latency": { 2024-04-25T14:16:09.092 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2468, 2024-04-25T14:16:09.092 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 3.217322282, 2024-04-25T14:16:09.092 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.001303615 2024-04-25T14:16:09.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.092 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_reply": 0, 2024-04-25T14:16:09.092 INFO:teuthology.orchestra.run.smithi097.stdout: "forward": 0, 2024-04-25T14:16:09.092 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_complete": 0, 2024-04-25T14:16:09.092 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_keys": 0, 2024-04-25T14:16:09.092 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_commit": 46, 2024-04-25T14:16:09.092 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_split": 0, 2024-04-25T14:16:09.093 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_merge": 0, 2024-04-25T14:16:09.093 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes": 183, 2024-04-25T14:16:09.093 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_top": 5, 2024-04-25T14:16:09.093 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_bottom": 178, 2024-04-25T14:16:09.093 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pin_tail": 0, 2024-04-25T14:16:09.093 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pinned": 175, 2024-04-25T14:16:09.093 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_expired": 0, 2024-04-25T14:16:09.093 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_with_caps": 1, 2024-04-25T14:16:09.093 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": 1, 2024-04-25T14:16:09.093 INFO:teuthology.orchestra.run.smithi097.stdout: "subtrees": 2, 2024-04-25T14:16:09.093 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse": 2496, 2024-04-25T14:16:09.093 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_hit": 2483, 2024-04-25T14:16:09.093 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_forward": 0, 2024-04-25T14:16:09.094 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_discover": 0, 2024-04-25T14:16:09.094 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_dir_fetch": 0, 2024-04-25T14:16:09.094 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_remote_ino": 0, 2024-04-25T14:16:09.094 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_lock": 0, 2024-04-25T14:16:09.094 INFO:teuthology.orchestra.run.smithi097.stdout: "load_cent": 12, 2024-04-25T14:16:09.094 INFO:teuthology.orchestra.run.smithi097.stdout: "q": 0, 2024-04-25T14:16:09.094 INFO:teuthology.orchestra.run.smithi097.stdout: "exported": 0, 2024-04-25T14:16:09.094 INFO:teuthology.orchestra.run.smithi097.stdout: "exported_inodes": 0, 2024-04-25T14:16:09.094 INFO:teuthology.orchestra.run.smithi097.stdout: "imported": 0, 2024-04-25T14:16:09.094 INFO:teuthology.orchestra.run.smithi097.stdout: "imported_inodes": 0, 2024-04-25T14:16:09.094 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_dir_fetch": 0, 2024-04-25T14:16:09.094 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_backtrace_fetch": 62, 2024-04-25T14:16:09.094 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_peer_discover": 0, 2024-04-25T14:16:09.095 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rfiles": 4, 2024-04-25T14:16:09.095 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rbytes": 234, 2024-04-25T14:16:09.095 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rsnaps": 0, 2024-04-25T14:16:09.095 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_fetch": 62, 2024-04-25T14:16:09.095 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_set_tag": 0, 2024-04-25T14:16:09.095 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T14:16:09.095 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_inotable_repaired": 0, 2024-04-25T14:16:09.095 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_inodes": 41, 2024-04-25T14:16:09.095 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_base_inodes": 5, 2024-04-25T14:16:09.095 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dirfrag_rstats": 36, 2024-04-25T14:16:09.095 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_file_inodes": 21, 2024-04-25T14:16:09.095 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_inode_file_caps": 0, 2024-04-25T14:16:09.095 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_revoke": 21, 2024-04-25T14:16:09.095 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_grant": 3640, 2024-04-25T14:16:09.096 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T14:16:09.096 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T14:16:09.096 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T14:16:09.096 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps": 2439, 2024-04-25T14:16:09.096 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps_dirty": 1212, 2024-04-25T14:16:09.096 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_cap_release": 11, 2024-04-25T14:16:09.096 INFO:teuthology.orchestra.run.smithi097.stdout: "process_request_cap_release": 1241 2024-04-25T14:16:09.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.096 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_cache": { 2024-04-25T14:16:09.096 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update": 19, 2024-04-25T14:16:09.096 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update_receipt": 0, 2024-04-25T14:16:09.096 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_try_discover": 0, 2024-04-25T14:16:09.096 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_send_discover": 0, 2024-04-25T14:16:09.096 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_handle_discover": 34, 2024-04-25T14:16:09.097 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays": 0, 2024-04-25T14:16:09.097 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_delayed": 0, 2024-04-25T14:16:09.097 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_enqueuing": 0, 2024-04-25T14:16:09.097 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_created": 1212, 2024-04-25T14:16:09.097 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_enqueued": 1212, 2024-04-25T14:16:09.097 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_reintegrated": 0, 2024-04-25T14:16:09.097 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_migrated": 0, 2024-04-25T14:16:09.097 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_processing": 0, 2024-04-25T14:16:09.097 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_enqueued": 0, 2024-04-25T14:16:09.097 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_prioritized": 0, 2024-04-25T14:16:09.097 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_started": 0, 2024-04-25T14:16:09.097 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_completed": 0, 2024-04-25T14:16:09.097 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_path": 0, 2024-04-25T14:16:09.098 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_inode": 0, 2024-04-25T14:16:09.098 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_enqueue_scrub": 5, 2024-04-25T14:16:09.098 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_exportdir": 0, 2024-04-25T14:16:09.098 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_flush": 0, 2024-04-25T14:16:09.098 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragmentdir": 0, 2024-04-25T14:16:09.098 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragstats": 0, 2024-04-25T14:16:09.098 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_inodestats": 0 2024-04-25T14:16:09.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.098 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_log": { 2024-04-25T14:16:09.098 INFO:teuthology.orchestra.run.smithi097.stdout: "evlrg": 0, 2024-04-25T14:16:09.098 INFO:teuthology.orchestra.run.smithi097.stdout: "evadd": 6117, 2024-04-25T14:16:09.098 INFO:teuthology.orchestra.run.smithi097.stdout: "evex": 3169, 2024-04-25T14:16:09.098 INFO:teuthology.orchestra.run.smithi097.stdout: "evtrm": 3169, 2024-04-25T14:16:09.099 INFO:teuthology.orchestra.run.smithi097.stdout: "ev": 2948, 2024-04-25T14:16:09.099 INFO:teuthology.orchestra.run.smithi097.stdout: "evexg": 0, 2024-04-25T14:16:09.099 INFO:teuthology.orchestra.run.smithi097.stdout: "evexd": 3112, 2024-04-25T14:16:09.099 INFO:teuthology.orchestra.run.smithi097.stdout: "segadd": 16, 2024-04-25T14:16:09.099 INFO:teuthology.orchestra.run.smithi097.stdout: "segex": 12, 2024-04-25T14:16:09.099 INFO:teuthology.orchestra.run.smithi097.stdout: "segtrm": 12, 2024-04-25T14:16:09.099 INFO:teuthology.orchestra.run.smithi097.stdout: "seg": 4, 2024-04-25T14:16:09.099 INFO:teuthology.orchestra.run.smithi097.stdout: "segmjr": 1, 2024-04-25T14:16:09.099 INFO:teuthology.orchestra.run.smithi097.stdout: "segexg": 0, 2024-04-25T14:16:09.099 INFO:teuthology.orchestra.run.smithi097.stdout: "segexd": 4, 2024-04-25T14:16:09.099 INFO:teuthology.orchestra.run.smithi097.stdout: "expos": 10944674, 2024-04-25T14:16:09.099 INFO:teuthology.orchestra.run.smithi097.stdout: "wrpos": 16776090, 2024-04-25T14:16:09.099 INFO:teuthology.orchestra.run.smithi097.stdout: "rdpos": 0, 2024-04-25T14:16:09.099 INFO:teuthology.orchestra.run.smithi097.stdout: "jlat": { 2024-04-25T14:16:09.100 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2442, 2024-04-25T14:16:09.100 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 4.068705479, 2024-04-25T14:16:09.100 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.001666136 2024-04-25T14:16:09.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.100 INFO:teuthology.orchestra.run.smithi097.stdout: "replayed": 0 2024-04-25T14:16:09.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.100 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_mem": { 2024-04-25T14:16:09.100 INFO:teuthology.orchestra.run.smithi097.stdout: "ino": 23, 2024-04-25T14:16:09.100 INFO:teuthology.orchestra.run.smithi097.stdout: "ino+": 1240, 2024-04-25T14:16:09.100 INFO:teuthology.orchestra.run.smithi097.stdout: "ino-": 1217, 2024-04-25T14:16:09.100 INFO:teuthology.orchestra.run.smithi097.stdout: "dir": 18, 2024-04-25T14:16:09.100 INFO:teuthology.orchestra.run.smithi097.stdout: "dir+": 24, 2024-04-25T14:16:09.100 INFO:teuthology.orchestra.run.smithi097.stdout: "dir-": 6, 2024-04-25T14:16:09.101 INFO:teuthology.orchestra.run.smithi097.stdout: "dn": 183, 2024-04-25T14:16:09.101 INFO:teuthology.orchestra.run.smithi097.stdout: "dn+": 1243, 2024-04-25T14:16:09.101 INFO:teuthology.orchestra.run.smithi097.stdout: "dn-": 1060, 2024-04-25T14:16:09.101 INFO:teuthology.orchestra.run.smithi097.stdout: "cap": 1, 2024-04-25T14:16:09.101 INFO:teuthology.orchestra.run.smithi097.stdout: "cap+": 1224, 2024-04-25T14:16:09.101 INFO:teuthology.orchestra.run.smithi097.stdout: "cap-": 1223, 2024-04-25T14:16:09.101 INFO:teuthology.orchestra.run.smithi097.stdout: "rss": 132804, 2024-04-25T14:16:09.101 INFO:teuthology.orchestra.run.smithi097.stdout: "heap": 231824 2024-04-25T14:16:09.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.101 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_server": { 2024-04-25T14:16:09.101 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_client_request": 2754, 2024-04-25T14:16:09.101 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_server_request": 0, 2024-04-25T14:16:09.101 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_request": 2468, 2024-04-25T14:16:09.101 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_session": 10, 2024-04-25T14:16:09.101 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_peer_request": 0, 2024-04-25T14:16:09.102 INFO:teuthology.orchestra.run.smithi097.stdout: "req_create_latency": { 2024-04-25T14:16:09.102 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1210, 2024-04-25T14:16:09.102 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 1.257563433, 2024-04-25T14:16:09.102 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.001039308 2024-04-25T14:16:09.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.102 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getattr_latency": { 2024-04-25T14:16:09.102 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 16, 2024-04-25T14:16:09.102 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.004288500, 2024-04-25T14:16:09.102 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000268031 2024-04-25T14:16:09.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.102 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getfilelock_latency": { 2024-04-25T14:16:09.102 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:09.102 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:09.103 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:09.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.103 INFO:teuthology.orchestra.run.smithi097.stdout: "req_link_latency": { 2024-04-25T14:16:09.103 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:09.103 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:09.103 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:09.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.103 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookup_latency": { 2024-04-25T14:16:09.103 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1, 2024-04-25T14:16:09.103 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000254194, 2024-04-25T14:16:09.103 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000254194 2024-04-25T14:16:09.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.103 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookuphash_latency": { 2024-04-25T14:16:09.103 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:09.104 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:09.104 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:09.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.104 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupino_latency": { 2024-04-25T14:16:09.104 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:09.104 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:09.104 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:09.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.104 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupname_latency": { 2024-04-25T14:16:09.104 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:09.104 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:09.104 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:09.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.104 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupparent_latency": { 2024-04-25T14:16:09.105 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:09.105 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:09.105 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:09.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.105 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupsnap_latency": { 2024-04-25T14:16:09.105 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:09.105 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:09.105 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:09.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.105 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lssnap_latency": { 2024-04-25T14:16:09.105 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:09.106 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:09.106 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:09.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.106 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mkdir_latency": { 2024-04-25T14:16:09.106 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 12, 2024-04-25T14:16:09.106 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.017930664, 2024-04-25T14:16:09.106 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.001494222 2024-04-25T14:16:09.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.107 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mknod_latency": { 2024-04-25T14:16:09.107 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:09.107 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:09.107 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:09.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.107 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mksnap_latency": { 2024-04-25T14:16:09.107 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:09.107 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:09.107 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:09.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.107 INFO:teuthology.orchestra.run.smithi097.stdout: "req_open_latency": { 2024-04-25T14:16:09.107 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1, 2024-04-25T14:16:09.107 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000202159, 2024-04-25T14:16:09.107 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000202159 2024-04-25T14:16:09.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.108 INFO:teuthology.orchestra.run.smithi097.stdout: "req_readdir_latency": { 2024-04-25T14:16:09.108 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 6, 2024-04-25T14:16:09.108 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.006172992, 2024-04-25T14:16:09.108 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.001028832 2024-04-25T14:16:09.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.108 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rename_latency": { 2024-04-25T14:16:09.108 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2, 2024-04-25T14:16:09.108 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.002929611, 2024-04-25T14:16:09.108 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.001464805 2024-04-25T14:16:09.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.108 INFO:teuthology.orchestra.run.smithi097.stdout: "req_renamesnap_latency": { 2024-04-25T14:16:09.108 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:09.108 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:09.109 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:09.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.109 INFO:teuthology.orchestra.run.smithi097.stdout: "req_snapdiff_latency": { 2024-04-25T14:16:09.109 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:09.109 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:09.109 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:09.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.109 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmdir_latency": { 2024-04-25T14:16:09.109 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 6, 2024-04-25T14:16:09.109 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.036576364, 2024-04-25T14:16:09.109 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.006096060 2024-04-25T14:16:09.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.109 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmsnap_latency": { 2024-04-25T14:16:09.109 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:09.110 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:09.110 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:09.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.110 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmxattr_latency": { 2024-04-25T14:16:09.110 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:09.110 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:09.110 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:09.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.110 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setattr_latency": { 2024-04-25T14:16:09.110 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1, 2024-04-25T14:16:09.110 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.001050674, 2024-04-25T14:16:09.110 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.001050674 2024-04-25T14:16:09.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.111 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setdirlayout_latency": { 2024-04-25T14:16:09.111 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:09.111 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:09.111 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:09.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.111 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setfilelock_latency": { 2024-04-25T14:16:09.111 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:09.111 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:09.111 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:09.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.111 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setlayout_latency": { 2024-04-25T14:16:09.111 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:09.111 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:09.111 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:09.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.112 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setxattr_latency": { 2024-04-25T14:16:09.112 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 7, 2024-04-25T14:16:09.112 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.022214343, 2024-04-25T14:16:09.112 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.003173477 2024-04-25T14:16:09.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.112 INFO:teuthology.orchestra.run.smithi097.stdout: "req_symlink_latency": { 2024-04-25T14:16:09.112 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:09.112 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:09.112 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:09.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.112 INFO:teuthology.orchestra.run.smithi097.stdout: "req_unlink_latency": { 2024-04-25T14:16:09.112 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1206, 2024-04-25T14:16:09.112 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 1.868139348, 2024-04-25T14:16:09.113 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.001549037 2024-04-25T14:16:09.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.113 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_revoke_eviction": 0, 2024-04-25T14:16:09.113 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_acquisition_throttle": 0, 2024-04-25T14:16:09.113 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getvxattr_latency": { 2024-04-25T14:16:09.113 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:09.113 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:09.113 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:09.113 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T14:16:09.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.113 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_sessions": { 2024-04-25T14:16:09.113 INFO:teuthology.orchestra.run.smithi097.stdout: "session_count": 1, 2024-04-25T14:16:09.113 INFO:teuthology.orchestra.run.smithi097.stdout: "session_add": 2, 2024-04-25T14:16:09.113 INFO:teuthology.orchestra.run.smithi097.stdout: "session_remove": 1, 2024-04-25T14:16:09.114 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_open": 1, 2024-04-25T14:16:09.114 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_stale": 0, 2024-04-25T14:16:09.114 INFO:teuthology.orchestra.run.smithi097.stdout: "total_load": 2217, 2024-04-25T14:16:09.114 INFO:teuthology.orchestra.run.smithi097.stdout: "average_load": 2217, 2024-04-25T14:16:09.114 INFO:teuthology.orchestra.run.smithi097.stdout: "avg_session_uptime": 157, 2024-04-25T14:16:09.114 INFO:teuthology.orchestra.run.smithi097.stdout: "mdthresh_evicted": 0 2024-04-25T14:16:09.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.114 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2024-04-25T14:16:09.114 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:16:09.114 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 0, 2024-04-25T14:16:09.114 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:16:09.114 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:16:09.114 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:16:09.114 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:16:09.115 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:16:09.115 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:16:09.115 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:16:09.115 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:16:09.115 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:16:09.115 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:16:09.115 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:16:09.115 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:16:09.115 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:16:09.115 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2024-04-25T14:16:09.115 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:16:09.115 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2024-04-25T14:16:09.115 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:16:09.115 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:16:09.116 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:16:09.116 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:16:09.116 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:16:09.116 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:16:09.116 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:16:09.116 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2024-04-25T14:16:09.116 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:16:09.116 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:16:09.116 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:16:09.116 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2024-04-25T14:16:09.116 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2024-04-25T14:16:09.116 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2024-04-25T14:16:09.116 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:16:09.117 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:16:09.117 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:16:09.117 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:16:09.117 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 323415, 2024-04-25T14:16:09.117 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 133, 2024-04-25T14:16:09.117 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 616, 2024-04-25T14:16:09.117 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 7, 2024-04-25T14:16:09.117 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2024-04-25T14:16:09.117 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2024-04-25T14:16:09.117 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:16:09.117 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2024-04-25T14:16:09.117 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:16:09.117 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2024-04-25T14:16:09.117 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 7720, 2024-04-25T14:16:09.118 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 162, 2024-04-25T14:16:09.118 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:16:09.118 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:16:09.118 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2024-04-25T14:16:09.118 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2024-04-25T14:16:09.118 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 215090, 2024-04-25T14:16:09.118 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 2024, 2024-04-25T14:16:09.118 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2024-04-25T14:16:09.118 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2024-04-25T14:16:09.118 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2024-04-25T14:16:09.118 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2024-04-25T14:16:09.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.118 INFO:teuthology.orchestra.run.smithi097.stdout: "objecter": { 2024-04-25T14:16:09.118 INFO:teuthology.orchestra.run.smithi097.stdout: "op_active": 0, 2024-04-25T14:16:09.119 INFO:teuthology.orchestra.run.smithi097.stdout: "op_laggy": 0, 2024-04-25T14:16:09.119 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send": 6920, 2024-04-25T14:16:09.119 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send_bytes": 12993893, 2024-04-25T14:16:09.119 INFO:teuthology.orchestra.run.smithi097.stdout: "op_resend": 37, 2024-04-25T14:16:09.119 INFO:teuthology.orchestra.run.smithi097.stdout: "op_reply": 6920, 2024-04-25T14:16:09.119 INFO:teuthology.orchestra.run.smithi097.stdout: "op_latency": { 2024-04-25T14:16:09.119 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 6920, 2024-04-25T14:16:09.119 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 35.543378113, 2024-04-25T14:16:09.119 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.005136326 2024-04-25T14:16:09.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.119 INFO:teuthology.orchestra.run.smithi097.stdout: "op_inflight": 0, 2024-04-25T14:16:09.119 INFO:teuthology.orchestra.run.smithi097.stdout: "oplen_avg": { 2024-04-25T14:16:09.119 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 6920, 2024-04-25T14:16:09.120 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 7414 2024-04-25T14:16:09.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.120 INFO:teuthology.orchestra.run.smithi097.stdout: "op": 6920, 2024-04-25T14:16:09.120 INFO:teuthology.orchestra.run.smithi097.stdout: "op_r": 100, 2024-04-25T14:16:09.120 INFO:teuthology.orchestra.run.smithi097.stdout: "op_w": 6820, 2024-04-25T14:16:09.120 INFO:teuthology.orchestra.run.smithi097.stdout: "op_rmw": 0, 2024-04-25T14:16:09.120 INFO:teuthology.orchestra.run.smithi097.stdout: "op_pg": 0, 2024-04-25T14:16:09.120 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_stat": 25, 2024-04-25T14:16:09.120 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_create": 192, 2024-04-25T14:16:09.120 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_read": 33, 2024-04-25T14:16:09.120 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_write": 2470, 2024-04-25T14:16:09.120 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writefull": 50, 2024-04-25T14:16:09.120 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writesame": 0, 2024-04-25T14:16:09.121 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_append": 0, 2024-04-25T14:16:09.121 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_zero": 0, 2024-04-25T14:16:09.121 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_truncate": 0, 2024-04-25T14:16:09.121 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_delete": 4013, 2024-04-25T14:16:09.121 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_mapext": 0, 2024-04-25T14:16:09.121 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_sparse_read": 0, 2024-04-25T14:16:09.121 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_clonerange": 0, 2024-04-25T14:16:09.121 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_getxattr": 67, 2024-04-25T14:16:09.121 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_setxattr": 384, 2024-04-25T14:16:09.121 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_cmpxattr": 0, 2024-04-25T14:16:09.121 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_rmxattr": 0, 2024-04-25T14:16:09.121 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_resetxattrs": 0, 2024-04-25T14:16:09.121 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_call": 0, 2024-04-25T14:16:09.121 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_watch": 0, 2024-04-25T14:16:09.122 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_notify": 0, 2024-04-25T14:16:09.122 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T14:16:09.122 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls": 0, 2024-04-25T14:16:09.122 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls_filter": 0, 2024-04-25T14:16:09.122 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_other": 0, 2024-04-25T14:16:09.122 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_active": 0, 2024-04-25T14:16:09.122 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_send": 0, 2024-04-25T14:16:09.122 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_resend": 0, 2024-04-25T14:16:09.122 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_ping": 0, 2024-04-25T14:16:09.122 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_active": 0, 2024-04-25T14:16:09.122 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_send": 0, 2024-04-25T14:16:09.122 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_resend": 0, 2024-04-25T14:16:09.122 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_active": 0, 2024-04-25T14:16:09.122 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_send": 0, 2024-04-25T14:16:09.123 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_resend": 0, 2024-04-25T14:16:09.123 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_active": 0, 2024-04-25T14:16:09.123 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_send": 0, 2024-04-25T14:16:09.123 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_resend": 0, 2024-04-25T14:16:09.123 INFO:teuthology.orchestra.run.smithi097.stdout: "command_active": 0, 2024-04-25T14:16:09.123 INFO:teuthology.orchestra.run.smithi097.stdout: "command_send": 0, 2024-04-25T14:16:09.123 INFO:teuthology.orchestra.run.smithi097.stdout: "command_resend": 0, 2024-04-25T14:16:09.123 INFO:teuthology.orchestra.run.smithi097.stdout: "map_epoch": 78, 2024-04-25T14:16:09.123 INFO:teuthology.orchestra.run.smithi097.stdout: "map_full": 0, 2024-04-25T14:16:09.123 INFO:teuthology.orchestra.run.smithi097.stdout: "map_inc": 3, 2024-04-25T14:16:09.123 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_sessions": 12, 2024-04-25T14:16:09.123 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_open": 12, 2024-04-25T14:16:09.123 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_close": 0, 2024-04-25T14:16:09.123 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_laggy": 0, 2024-04-25T14:16:09.123 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_wr": 159, 2024-04-25T14:16:09.124 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_rd": 0, 2024-04-25T14:16:09.124 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_del": 21 2024-04-25T14:16:09.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.124 INFO:teuthology.orchestra.run.smithi097.stdout: "oft": { 2024-04-25T14:16:09.124 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_objs": 1, 2024-04-25T14:16:09.124 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_kv_pairs": 0, 2024-04-25T14:16:09.124 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_updates": 0, 2024-04-25T14:16:09.124 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_removes": 0 2024-04-25T14:16:09.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.124 INFO:teuthology.orchestra.run.smithi097.stdout: "osd-slow-ops": { 2024-04-25T14:16:09.124 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_ops_count": 0 2024-04-25T14:16:09.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.124 INFO:teuthology.orchestra.run.smithi097.stdout: "purge_queue": { 2024-04-25T14:16:09.124 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops": 0, 2024-04-25T14:16:09.125 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops_high_water": 9, 2024-04-25T14:16:09.125 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing": 0, 2024-04-25T14:16:09.125 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_high_water": 4, 2024-04-25T14:16:09.125 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed_ops": 3498, 2024-04-25T14:16:09.125 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed": 1212, 2024-04-25T14:16:09.125 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_item_in_journal": 0 2024-04-25T14:16:09.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.125 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:16:09.125 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T14:16:09.125 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-04-25T14:16:09.125 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T14:16:09.125 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 14318, 2024-04-25T14:16:09.125 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 3319467, 2024-04-25T14:16:09.125 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T14:16:09.125 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 14318, 2024-04-25T14:16:09.126 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-25T14:16:09.126 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-25T14:16:09.126 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 14318, 2024-04-25T14:16:09.126 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 3319467, 2024-04-25T14:16:09.126 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T14:16:09.126 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:09.126 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:09.126 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:09.126 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T14:16:09.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.126 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_bytes": { 2024-04-25T14:16:09.126 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T14:16:09.126 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-04-25T14:16:09.126 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T14:16:09.127 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-25T14:16:09.127 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-25T14:16:09.127 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T14:16:09.127 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-25T14:16:09.127 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 6920, 2024-04-25T14:16:09.127 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 13116305, 2024-04-25T14:16:09.127 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 6920, 2024-04-25T14:16:09.127 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 13116305, 2024-04-25T14:16:09.127 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T14:16:09.127 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:09.127 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:09.127 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:09.127 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T14:16:09.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.128 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_ops": { 2024-04-25T14:16:09.128 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T14:16:09.128 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 1024, 2024-04-25T14:16:09.128 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T14:16:09.128 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-25T14:16:09.128 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-25T14:16:09.128 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T14:16:09.128 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-25T14:16:09.128 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 6920, 2024-04-25T14:16:09.128 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 6920, 2024-04-25T14:16:09.128 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 6920, 2024-04-25T14:16:09.128 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 6920, 2024-04-25T14:16:09.128 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T14:16:09.129 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:09.129 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:09.129 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:09.129 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T14:16:09.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.129 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle": { 2024-04-25T14:16:09.129 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T14:16:09.129 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-04-25T14:16:09.129 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T14:16:09.129 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 1212, 2024-04-25T14:16:09.129 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 122412, 2024-04-25T14:16:09.129 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T14:16:09.129 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 1212, 2024-04-25T14:16:09.129 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-25T14:16:09.130 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-25T14:16:09.130 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 28, 2024-04-25T14:16:09.130 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 122412, 2024-04-25T14:16:09.130 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T14:16:09.130 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:09.130 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:09.130 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:09.130 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T14:16:09.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:09.130 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle-0x561712756750": { 2024-04-25T14:16:09.130 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T14:16:09.130 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-04-25T14:16:09.130 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T14:16:09.131 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 6117, 2024-04-25T14:16:09.131 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 12581786, 2024-04-25T14:16:09.131 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T14:16:09.131 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 6117, 2024-04-25T14:16:09.131 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-25T14:16:09.131 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-25T14:16:09.131 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 2442, 2024-04-25T14:16:09.131 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 12581786, 2024-04-25T14:16:09.131 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T14:16:09.131 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:09.131 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:09.131 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:09.131 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T14:16:09.131 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T14:16:09.132 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T14:16:09.941 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-04-25T14:16:09.941 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-04-25T14:16:09.941 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=19 2024-04-25T14:16:09.941 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-04-25T14:16:09.941 DEBUG:tasks.check_counter:Getting stats from e 2024-04-25T14:16:09.942 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-25T14:16:10.228 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T14:16:10.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:10 smithi097 ceph-mon[32720]: pgmap v422: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T14:16:10.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:10 smithi096 ceph-mon[29207]: pgmap v422: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T14:16:10.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:10 smithi151 ceph-mon[32990]: pgmap v422: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T14:16:11.938 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T14:16:11.939 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:16:11.939 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 816, 2024-04-25T14:16:11.939 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 531, 2024-04-25T14:16:11.939 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 257842, 2024-04-25T14:16:11.939 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 118189, 2024-04-25T14:16:11.939 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 9, 2024-04-25T14:16:11.939 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 2, 2024-04-25T14:16:11.939 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.060695835, 2024-04-25T14:16:11.939 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.024616498, 2024-04-25T14:16:11.939 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.040670254, 2024-04-25T14:16:11.940 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.001501214, 2024-04-25T14:16:11.940 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:16:11.940 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 536, 2024-04-25T14:16:11.940 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.041120632, 2024-04-25T14:16:11.940 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000076717 2024-04-25T14:16:11.940 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.940 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:16:11.940 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1291, 2024-04-25T14:16:11.940 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000255026, 2024-04-25T14:16:11.940 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000197 2024-04-25T14:16:11.940 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.941 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-25T14:16:11.941 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T14:16:11.941 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.941 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:16:11.941 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 157, 2024-04-25T14:16:11.941 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 149, 2024-04-25T14:16:11.941 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 530314, 2024-04-25T14:16:11.941 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 73990, 2024-04-25T14:16:11.941 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 7, 2024-04-25T14:16:11.941 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 5, 2024-04-25T14:16:11.941 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.020695348, 2024-04-25T14:16:11.941 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.005848987, 2024-04-25T14:16:11.942 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.016461004, 2024-04-25T14:16:11.942 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.002258432, 2024-04-25T14:16:11.942 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:16:11.942 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 148, 2024-04-25T14:16:11.942 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.027698077, 2024-04-25T14:16:11.942 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000187149 2024-04-25T14:16:11.942 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.942 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:16:11.942 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 19, 2024-04-25T14:16:11.942 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000004604, 2024-04-25T14:16:11.942 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000242 2024-04-25T14:16:11.942 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.943 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 516880, 2024-04-25T14:16:11.943 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 58176 2024-04-25T14:16:11.943 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.943 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:16:11.943 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 34, 2024-04-25T14:16:11.943 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 104, 2024-04-25T14:16:11.943 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 13928, 2024-04-25T14:16:11.943 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 375950, 2024-04-25T14:16:11.943 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 23, 2024-04-25T14:16:11.943 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 6, 2024-04-25T14:16:11.943 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.013119578, 2024-04-25T14:16:11.943 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.003725447, 2024-04-25T14:16:11.944 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.008220704, 2024-04-25T14:16:11.944 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.000105334, 2024-04-25T14:16:11.944 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:16:11.944 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 96, 2024-04-25T14:16:11.944 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.019005391, 2024-04-25T14:16:11.944 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000197972 2024-04-25T14:16:11.944 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.944 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:16:11.944 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 35, 2024-04-25T14:16:11.944 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000006618, 2024-04-25T14:16:11.945 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000189 2024-04-25T14:16:11.945 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.945 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T14:16:11.945 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 189680 2024-04-25T14:16:11.945 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.945 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2024-04-25T14:16:11.945 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 1, 2024-04-25T14:16:11.945 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2024-04-25T14:16:11.945 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.946 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-MDSRank": { 2024-04-25T14:16:11.946 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-04-25T14:16:11.946 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-04-25T14:16:11.946 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 46, 2024-04-25T14:16:11.946 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.001132986, 2024-04-25T14:16:11.946 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000024630 2024-04-25T14:16:11.946 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T14:16:11.946 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.946 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-PurgeQueue": { 2024-04-25T14:16:11.946 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-04-25T14:16:11.946 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-04-25T14:16:11.946 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2, 2024-04-25T14:16:11.947 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000007354, 2024-04-25T14:16:11.947 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000003677 2024-04-25T14:16:11.947 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T14:16:11.947 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.947 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": { 2024-04-25T14:16:11.947 INFO:teuthology.orchestra.run.smithi097.stdout: "request": 0, 2024-04-25T14:16:11.947 INFO:teuthology.orchestra.run.smithi097.stdout: "reply": 0, 2024-04-25T14:16:11.947 INFO:teuthology.orchestra.run.smithi097.stdout: "reply_latency": { 2024-04-25T14:16:11.947 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:11.947 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:11.947 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:11.947 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.948 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_reply": 0, 2024-04-25T14:16:11.948 INFO:teuthology.orchestra.run.smithi097.stdout: "forward": 0, 2024-04-25T14:16:11.948 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_complete": 0, 2024-04-25T14:16:11.948 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_keys": 0, 2024-04-25T14:16:11.948 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_commit": 11, 2024-04-25T14:16:11.948 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_split": 0, 2024-04-25T14:16:11.948 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_merge": 0, 2024-04-25T14:16:11.948 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes": 17, 2024-04-25T14:16:11.948 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_top": 1, 2024-04-25T14:16:11.948 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_bottom": 16, 2024-04-25T14:16:11.948 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pin_tail": 0, 2024-04-25T14:16:11.948 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pinned": 13, 2024-04-25T14:16:11.949 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_expired": 8, 2024-04-25T14:16:11.949 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_with_caps": 0, 2024-04-25T14:16:11.949 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": 0, 2024-04-25T14:16:11.949 INFO:teuthology.orchestra.run.smithi097.stdout: "subtrees": 3, 2024-04-25T14:16:11.949 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse": 6, 2024-04-25T14:16:11.949 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_hit": 4, 2024-04-25T14:16:11.949 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_forward": 0, 2024-04-25T14:16:11.949 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_discover": 2, 2024-04-25T14:16:11.949 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_dir_fetch": 0, 2024-04-25T14:16:11.949 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_remote_ino": 0, 2024-04-25T14:16:11.950 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_lock": 0, 2024-04-25T14:16:11.950 INFO:teuthology.orchestra.run.smithi097.stdout: "load_cent": 0, 2024-04-25T14:16:11.950 INFO:teuthology.orchestra.run.smithi097.stdout: "q": 0, 2024-04-25T14:16:11.950 INFO:teuthology.orchestra.run.smithi097.stdout: "exported": 0, 2024-04-25T14:16:11.950 INFO:teuthology.orchestra.run.smithi097.stdout: "exported_inodes": 0, 2024-04-25T14:16:11.950 INFO:teuthology.orchestra.run.smithi097.stdout: "imported": 0, 2024-04-25T14:16:11.950 INFO:teuthology.orchestra.run.smithi097.stdout: "imported_inodes": 0, 2024-04-25T14:16:11.950 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_dir_fetch": 0, 2024-04-25T14:16:11.950 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_backtrace_fetch": 0, 2024-04-25T14:16:11.950 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_peer_discover": 0, 2024-04-25T14:16:11.950 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rfiles": 4, 2024-04-25T14:16:11.951 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rbytes": 234, 2024-04-25T14:16:11.951 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rsnaps": 0, 2024-04-25T14:16:11.951 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T14:16:11.951 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_set_tag": 0, 2024-04-25T14:16:11.951 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T14:16:11.951 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_inotable_repaired": 0, 2024-04-25T14:16:11.951 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_inodes": 0, 2024-04-25T14:16:11.951 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T14:16:11.951 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T14:16:11.951 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_file_inodes": 0, 2024-04-25T14:16:11.952 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_inode_file_caps": 0, 2024-04-25T14:16:11.952 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T14:16:11.952 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_grant": 0, 2024-04-25T14:16:11.952 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T14:16:11.952 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T14:16:11.952 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T14:16:11.952 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps": 0, 2024-04-25T14:16:11.952 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps_dirty": 0, 2024-04-25T14:16:11.953 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_cap_release": 0, 2024-04-25T14:16:11.953 INFO:teuthology.orchestra.run.smithi097.stdout: "process_request_cap_release": 0 2024-04-25T14:16:11.953 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.953 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_cache": { 2024-04-25T14:16:11.953 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update": 0, 2024-04-25T14:16:11.953 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update_receipt": 10, 2024-04-25T14:16:11.953 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_try_discover": 6, 2024-04-25T14:16:11.953 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_send_discover": 7, 2024-04-25T14:16:11.953 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_handle_discover": 0, 2024-04-25T14:16:11.953 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays": 0, 2024-04-25T14:16:11.953 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_delayed": 0, 2024-04-25T14:16:11.953 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_enqueuing": 0, 2024-04-25T14:16:11.953 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_created": 0, 2024-04-25T14:16:11.954 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_enqueued": 0, 2024-04-25T14:16:11.954 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_reintegrated": 0, 2024-04-25T14:16:11.954 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_migrated": 0, 2024-04-25T14:16:11.954 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_processing": 0, 2024-04-25T14:16:11.954 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_enqueued": 0, 2024-04-25T14:16:11.954 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_prioritized": 0, 2024-04-25T14:16:11.954 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_started": 0, 2024-04-25T14:16:11.954 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_completed": 0, 2024-04-25T14:16:11.954 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_path": 0, 2024-04-25T14:16:11.954 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_inode": 0, 2024-04-25T14:16:11.954 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T14:16:11.954 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_exportdir": 0, 2024-04-25T14:16:11.954 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_flush": 0, 2024-04-25T14:16:11.954 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragmentdir": 0, 2024-04-25T14:16:11.955 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragstats": 0, 2024-04-25T14:16:11.955 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_inodestats": 0 2024-04-25T14:16:11.955 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.955 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_log": { 2024-04-25T14:16:11.955 INFO:teuthology.orchestra.run.smithi097.stdout: "evlrg": 0, 2024-04-25T14:16:11.955 INFO:teuthology.orchestra.run.smithi097.stdout: "evadd": 2, 2024-04-25T14:16:11.955 INFO:teuthology.orchestra.run.smithi097.stdout: "evex": 0, 2024-04-25T14:16:11.955 INFO:teuthology.orchestra.run.smithi097.stdout: "evtrm": 0, 2024-04-25T14:16:11.955 INFO:teuthology.orchestra.run.smithi097.stdout: "ev": 2, 2024-04-25T14:16:11.955 INFO:teuthology.orchestra.run.smithi097.stdout: "evexg": 0, 2024-04-25T14:16:11.955 INFO:teuthology.orchestra.run.smithi097.stdout: "evexd": 0, 2024-04-25T14:16:11.955 INFO:teuthology.orchestra.run.smithi097.stdout: "segadd": 2, 2024-04-25T14:16:11.955 INFO:teuthology.orchestra.run.smithi097.stdout: "segex": 0, 2024-04-25T14:16:11.955 INFO:teuthology.orchestra.run.smithi097.stdout: "segtrm": 0, 2024-04-25T14:16:11.956 INFO:teuthology.orchestra.run.smithi097.stdout: "seg": 2, 2024-04-25T14:16:11.956 INFO:teuthology.orchestra.run.smithi097.stdout: "segmjr": 2, 2024-04-25T14:16:11.956 INFO:teuthology.orchestra.run.smithi097.stdout: "segexg": 0, 2024-04-25T14:16:11.956 INFO:teuthology.orchestra.run.smithi097.stdout: "segexd": 0, 2024-04-25T14:16:11.956 INFO:teuthology.orchestra.run.smithi097.stdout: "expos": 4194304, 2024-04-25T14:16:11.956 INFO:teuthology.orchestra.run.smithi097.stdout: "wrpos": 4194892, 2024-04-25T14:16:11.956 INFO:teuthology.orchestra.run.smithi097.stdout: "rdpos": 0, 2024-04-25T14:16:11.956 INFO:teuthology.orchestra.run.smithi097.stdout: "jlat": { 2024-04-25T14:16:11.956 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1, 2024-04-25T14:16:11.956 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.001284091, 2024-04-25T14:16:11.956 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.001284091 2024-04-25T14:16:11.956 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.956 INFO:teuthology.orchestra.run.smithi097.stdout: "replayed": 0 2024-04-25T14:16:11.956 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.956 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_mem": { 2024-04-25T14:16:11.957 INFO:teuthology.orchestra.run.smithi097.stdout: "ino": 21, 2024-04-25T14:16:11.957 INFO:teuthology.orchestra.run.smithi097.stdout: "ino+": 27, 2024-04-25T14:16:11.957 INFO:teuthology.orchestra.run.smithi097.stdout: "ino-": 6, 2024-04-25T14:16:11.957 INFO:teuthology.orchestra.run.smithi097.stdout: "dir": 20, 2024-04-25T14:16:11.957 INFO:teuthology.orchestra.run.smithi097.stdout: "dir+": 24, 2024-04-25T14:16:11.957 INFO:teuthology.orchestra.run.smithi097.stdout: "dir-": 4, 2024-04-25T14:16:11.957 INFO:teuthology.orchestra.run.smithi097.stdout: "dn": 17, 2024-04-25T14:16:11.957 INFO:teuthology.orchestra.run.smithi097.stdout: "dn+": 25, 2024-04-25T14:16:11.957 INFO:teuthology.orchestra.run.smithi097.stdout: "dn-": 8, 2024-04-25T14:16:11.957 INFO:teuthology.orchestra.run.smithi097.stdout: "cap": 0, 2024-04-25T14:16:11.957 INFO:teuthology.orchestra.run.smithi097.stdout: "cap+": 0, 2024-04-25T14:16:11.957 INFO:teuthology.orchestra.run.smithi097.stdout: "cap-": 0, 2024-04-25T14:16:11.957 INFO:teuthology.orchestra.run.smithi097.stdout: "rss": 123212, 2024-04-25T14:16:11.957 INFO:teuthology.orchestra.run.smithi097.stdout: "heap": 223632 2024-04-25T14:16:11.958 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.958 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_server": { 2024-04-25T14:16:11.958 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_client_request": 0, 2024-04-25T14:16:11.958 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_server_request": 0, 2024-04-25T14:16:11.958 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_request": 0, 2024-04-25T14:16:11.958 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_session": 0, 2024-04-25T14:16:11.958 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_peer_request": 0, 2024-04-25T14:16:11.958 INFO:teuthology.orchestra.run.smithi097.stdout: "req_create_latency": { 2024-04-25T14:16:11.958 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:11.958 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:11.958 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:11.958 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.958 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getattr_latency": { 2024-04-25T14:16:11.958 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:11.959 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:11.959 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:11.959 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.959 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getfilelock_latency": { 2024-04-25T14:16:11.959 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:11.959 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:11.959 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:11.959 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.959 INFO:teuthology.orchestra.run.smithi097.stdout: "req_link_latency": { 2024-04-25T14:16:11.959 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:11.959 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:11.959 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:11.959 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.959 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookup_latency": { 2024-04-25T14:16:11.960 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:11.960 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:11.960 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:11.960 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.960 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookuphash_latency": { 2024-04-25T14:16:11.960 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:11.960 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:11.960 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:11.960 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.960 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupino_latency": { 2024-04-25T14:16:11.960 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:11.960 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:11.960 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:11.960 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.961 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupname_latency": { 2024-04-25T14:16:11.961 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:11.961 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:11.961 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:11.961 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.961 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupparent_latency": { 2024-04-25T14:16:11.961 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:11.961 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:11.961 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:11.961 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.961 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupsnap_latency": { 2024-04-25T14:16:11.961 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:11.961 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:11.961 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:11.961 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.962 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lssnap_latency": { 2024-04-25T14:16:11.962 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:11.962 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:11.962 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:11.962 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.962 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mkdir_latency": { 2024-04-25T14:16:11.962 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:11.963 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:11.963 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:11.963 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.963 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mknod_latency": { 2024-04-25T14:16:11.963 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:11.963 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:11.963 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:11.963 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.963 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mksnap_latency": { 2024-04-25T14:16:11.964 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:11.964 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:11.964 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:11.964 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.964 INFO:teuthology.orchestra.run.smithi097.stdout: "req_open_latency": { 2024-04-25T14:16:11.964 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:11.964 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:11.964 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:11.964 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.964 INFO:teuthology.orchestra.run.smithi097.stdout: "req_readdir_latency": { 2024-04-25T14:16:11.964 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:11.964 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:11.964 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:11.965 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.965 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rename_latency": { 2024-04-25T14:16:11.965 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:11.965 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:11.965 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:11.965 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.965 INFO:teuthology.orchestra.run.smithi097.stdout: "req_renamesnap_latency": { 2024-04-25T14:16:11.965 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:11.965 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:11.965 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:11.965 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.965 INFO:teuthology.orchestra.run.smithi097.stdout: "req_snapdiff_latency": { 2024-04-25T14:16:11.965 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:11.965 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:11.965 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:11.966 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.966 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmdir_latency": { 2024-04-25T14:16:11.966 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:11.966 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:11.966 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:11.966 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.966 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmsnap_latency": { 2024-04-25T14:16:11.966 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:11.966 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:11.966 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:11.966 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.966 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmxattr_latency": { 2024-04-25T14:16:11.966 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:11.967 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:11.967 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:11.967 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.967 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setattr_latency": { 2024-04-25T14:16:11.967 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:11.967 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:11.967 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:11.967 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.967 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setdirlayout_latency": { 2024-04-25T14:16:11.967 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:11.967 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:11.967 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:11.967 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.967 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setfilelock_latency": { 2024-04-25T14:16:11.968 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:11.968 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:11.968 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:11.968 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.968 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setlayout_latency": { 2024-04-25T14:16:11.968 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:11.968 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:11.968 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:11.968 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.968 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setxattr_latency": { 2024-04-25T14:16:11.968 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:11.968 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:11.968 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:11.968 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.969 INFO:teuthology.orchestra.run.smithi097.stdout: "req_symlink_latency": { 2024-04-25T14:16:11.969 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:11.969 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:11.969 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:11.969 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.969 INFO:teuthology.orchestra.run.smithi097.stdout: "req_unlink_latency": { 2024-04-25T14:16:11.969 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:11.969 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:11.969 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:11.969 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.969 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_revoke_eviction": 0, 2024-04-25T14:16:11.969 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_acquisition_throttle": 0, 2024-04-25T14:16:11.969 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getvxattr_latency": { 2024-04-25T14:16:11.970 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:11.970 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:11.970 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:11.970 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T14:16:11.970 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.970 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_sessions": { 2024-04-25T14:16:11.970 INFO:teuthology.orchestra.run.smithi097.stdout: "session_count": 0, 2024-04-25T14:16:11.970 INFO:teuthology.orchestra.run.smithi097.stdout: "session_add": 0, 2024-04-25T14:16:11.970 INFO:teuthology.orchestra.run.smithi097.stdout: "session_remove": 0, 2024-04-25T14:16:11.970 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_open": 0, 2024-04-25T14:16:11.970 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_stale": 0, 2024-04-25T14:16:11.970 INFO:teuthology.orchestra.run.smithi097.stdout: "total_load": 0, 2024-04-25T14:16:11.970 INFO:teuthology.orchestra.run.smithi097.stdout: "average_load": 0, 2024-04-25T14:16:11.971 INFO:teuthology.orchestra.run.smithi097.stdout: "avg_session_uptime": 0, 2024-04-25T14:16:11.971 INFO:teuthology.orchestra.run.smithi097.stdout: "mdthresh_evicted": 0 2024-04-25T14:16:11.971 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.971 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2024-04-25T14:16:11.971 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:16:11.971 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 0, 2024-04-25T14:16:11.971 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:16:11.971 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:16:11.971 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:16:11.971 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:16:11.971 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:16:11.971 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:16:11.971 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:16:11.972 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:16:11.972 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:16:11.972 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:16:11.972 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:16:11.972 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:16:11.972 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:16:11.972 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2024-04-25T14:16:11.972 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:16:11.972 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2024-04-25T14:16:11.972 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:16:11.972 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:16:11.972 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:16:11.972 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:16:11.972 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:16:11.972 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:16:11.973 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:16:11.973 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2024-04-25T14:16:11.973 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:16:11.973 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:16:11.973 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:16:11.973 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2024-04-25T14:16:11.973 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2024-04-25T14:16:11.973 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2024-04-25T14:16:11.974 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:16:11.974 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:16:11.974 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:16:11.974 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:16:11.974 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 221575, 2024-04-25T14:16:11.974 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 65, 2024-04-25T14:16:11.974 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 0, 2024-04-25T14:16:11.974 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 0, 2024-04-25T14:16:11.974 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2024-04-25T14:16:11.974 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2024-04-25T14:16:11.974 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:16:11.974 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2024-04-25T14:16:11.974 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:16:11.975 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2024-04-25T14:16:11.975 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 7720, 2024-04-25T14:16:11.975 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 162, 2024-04-25T14:16:11.975 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:16:11.975 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:16:11.975 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2024-04-25T14:16:11.975 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2024-04-25T14:16:11.975 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 85069, 2024-04-25T14:16:11.975 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 329, 2024-04-25T14:16:11.975 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2024-04-25T14:16:11.975 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2024-04-25T14:16:11.975 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2024-04-25T14:16:11.975 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2024-04-25T14:16:11.975 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.975 INFO:teuthology.orchestra.run.smithi097.stdout: "objecter": { 2024-04-25T14:16:11.976 INFO:teuthology.orchestra.run.smithi097.stdout: "op_active": 0, 2024-04-25T14:16:11.976 INFO:teuthology.orchestra.run.smithi097.stdout: "op_laggy": 0, 2024-04-25T14:16:11.976 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send": 35, 2024-04-25T14:16:11.976 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send_bytes": 10710, 2024-04-25T14:16:11.976 INFO:teuthology.orchestra.run.smithi097.stdout: "op_resend": 33, 2024-04-25T14:16:11.976 INFO:teuthology.orchestra.run.smithi097.stdout: "op_reply": 35, 2024-04-25T14:16:11.976 INFO:teuthology.orchestra.run.smithi097.stdout: "op_latency": { 2024-04-25T14:16:11.976 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 35, 2024-04-25T14:16:11.976 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.216996239, 2024-04-25T14:16:11.976 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.006199892 2024-04-25T14:16:11.976 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.976 INFO:teuthology.orchestra.run.smithi097.stdout: "op_inflight": 0, 2024-04-25T14:16:11.976 INFO:teuthology.orchestra.run.smithi097.stdout: "oplen_avg": { 2024-04-25T14:16:11.976 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 35, 2024-04-25T14:16:11.976 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 57 2024-04-25T14:16:11.977 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.977 INFO:teuthology.orchestra.run.smithi097.stdout: "op": 35, 2024-04-25T14:16:11.977 INFO:teuthology.orchestra.run.smithi097.stdout: "op_r": 0, 2024-04-25T14:16:11.977 INFO:teuthology.orchestra.run.smithi097.stdout: "op_w": 35, 2024-04-25T14:16:11.977 INFO:teuthology.orchestra.run.smithi097.stdout: "op_rmw": 0, 2024-04-25T14:16:11.977 INFO:teuthology.orchestra.run.smithi097.stdout: "op_pg": 0, 2024-04-25T14:16:11.977 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_stat": 0, 2024-04-25T14:16:11.977 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_create": 10, 2024-04-25T14:16:11.977 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_read": 0, 2024-04-25T14:16:11.977 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_write": 1, 2024-04-25T14:16:11.977 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writefull": 5, 2024-04-25T14:16:11.977 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writesame": 0, 2024-04-25T14:16:11.977 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_append": 0, 2024-04-25T14:16:11.977 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_zero": 0, 2024-04-25T14:16:11.978 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_truncate": 0, 2024-04-25T14:16:11.978 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_delete": 6, 2024-04-25T14:16:11.978 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_mapext": 0, 2024-04-25T14:16:11.978 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_sparse_read": 0, 2024-04-25T14:16:11.978 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_clonerange": 0, 2024-04-25T14:16:11.978 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_getxattr": 0, 2024-04-25T14:16:11.978 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_setxattr": 20, 2024-04-25T14:16:11.978 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_cmpxattr": 0, 2024-04-25T14:16:11.978 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_rmxattr": 0, 2024-04-25T14:16:11.978 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_resetxattrs": 0, 2024-04-25T14:16:11.978 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_call": 0, 2024-04-25T14:16:11.978 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_watch": 0, 2024-04-25T14:16:11.978 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_notify": 0, 2024-04-25T14:16:11.978 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T14:16:11.979 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls": 0, 2024-04-25T14:16:11.979 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls_filter": 0, 2024-04-25T14:16:11.979 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_other": 0, 2024-04-25T14:16:11.979 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_active": 0, 2024-04-25T14:16:11.979 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_send": 0, 2024-04-25T14:16:11.979 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_resend": 0, 2024-04-25T14:16:11.979 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_ping": 0, 2024-04-25T14:16:11.979 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_active": 0, 2024-04-25T14:16:11.979 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_send": 0, 2024-04-25T14:16:11.979 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_resend": 0, 2024-04-25T14:16:11.979 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_active": 0, 2024-04-25T14:16:11.979 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_send": 0, 2024-04-25T14:16:11.979 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_resend": 0, 2024-04-25T14:16:11.979 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_active": 0, 2024-04-25T14:16:11.980 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_send": 0, 2024-04-25T14:16:11.980 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_resend": 0, 2024-04-25T14:16:11.980 INFO:teuthology.orchestra.run.smithi097.stdout: "command_active": 0, 2024-04-25T14:16:11.980 INFO:teuthology.orchestra.run.smithi097.stdout: "command_send": 0, 2024-04-25T14:16:11.980 INFO:teuthology.orchestra.run.smithi097.stdout: "command_resend": 0, 2024-04-25T14:16:11.980 INFO:teuthology.orchestra.run.smithi097.stdout: "map_epoch": 78, 2024-04-25T14:16:11.980 INFO:teuthology.orchestra.run.smithi097.stdout: "map_full": 0, 2024-04-25T14:16:11.980 INFO:teuthology.orchestra.run.smithi097.stdout: "map_inc": 2, 2024-04-25T14:16:11.980 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_sessions": 8, 2024-04-25T14:16:11.980 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_open": 8, 2024-04-25T14:16:11.980 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_close": 0, 2024-04-25T14:16:11.980 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_laggy": 0, 2024-04-25T14:16:11.980 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_wr": 14, 2024-04-25T14:16:11.980 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_rd": 0, 2024-04-25T14:16:11.980 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_del": 1 2024-04-25T14:16:11.981 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.981 INFO:teuthology.orchestra.run.smithi097.stdout: "oft": { 2024-04-25T14:16:11.981 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_objs": 1, 2024-04-25T14:16:11.981 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_kv_pairs": 0, 2024-04-25T14:16:11.981 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_updates": 0, 2024-04-25T14:16:11.981 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_removes": 0 2024-04-25T14:16:11.981 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.981 INFO:teuthology.orchestra.run.smithi097.stdout: "osd-slow-ops": { 2024-04-25T14:16:11.981 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_ops_count": 0 2024-04-25T14:16:11.981 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.981 INFO:teuthology.orchestra.run.smithi097.stdout: "purge_queue": { 2024-04-25T14:16:11.981 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops": 0, 2024-04-25T14:16:11.981 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T14:16:11.982 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing": 0, 2024-04-25T14:16:11.982 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_high_water": 0, 2024-04-25T14:16:11.982 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed_ops": 0, 2024-04-25T14:16:11.982 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed": 0, 2024-04-25T14:16:11.982 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_item_in_journal": 0 2024-04-25T14:16:11.982 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.982 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:16:11.982 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T14:16:11.982 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-04-25T14:16:11.982 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T14:16:11.982 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 1003, 2024-04-25T14:16:11.982 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 702238, 2024-04-25T14:16:11.982 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T14:16:11.982 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 1003, 2024-04-25T14:16:11.983 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-25T14:16:11.983 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-25T14:16:11.983 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 1003, 2024-04-25T14:16:11.983 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 702238, 2024-04-25T14:16:11.983 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T14:16:11.983 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:11.983 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:11.983 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:11.983 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T14:16:11.983 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.983 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_bytes": { 2024-04-25T14:16:11.983 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T14:16:11.983 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-04-25T14:16:11.983 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T14:16:11.983 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-25T14:16:11.984 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-25T14:16:11.984 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T14:16:11.984 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-25T14:16:11.984 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 35, 2024-04-25T14:16:11.984 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 10710, 2024-04-25T14:16:11.984 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 35, 2024-04-25T14:16:11.984 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 10710, 2024-04-25T14:16:11.984 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T14:16:11.984 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:11.984 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:11.984 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:11.984 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T14:16:11.984 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.984 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_ops": { 2024-04-25T14:16:11.985 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T14:16:11.985 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 1024, 2024-04-25T14:16:11.985 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T14:16:11.985 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-25T14:16:11.985 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-25T14:16:11.985 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T14:16:11.985 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-25T14:16:11.985 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 35, 2024-04-25T14:16:11.985 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 35, 2024-04-25T14:16:11.985 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 35, 2024-04-25T14:16:11.985 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 35, 2024-04-25T14:16:11.985 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T14:16:11.985 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:11.985 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:11.986 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:11.986 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T14:16:11.986 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.986 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle": { 2024-04-25T14:16:11.986 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T14:16:11.986 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-04-25T14:16:11.986 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T14:16:11.986 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-25T14:16:11.986 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-25T14:16:11.986 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T14:16:11.986 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-25T14:16:11.986 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-25T14:16:11.986 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-25T14:16:11.986 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 0, 2024-04-25T14:16:11.987 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 0, 2024-04-25T14:16:11.987 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T14:16:11.987 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:11.987 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:11.987 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:11.987 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T14:16:11.987 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:11.987 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle-0x55a788986750": { 2024-04-25T14:16:11.987 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T14:16:11.987 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-04-25T14:16:11.987 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T14:16:11.987 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 2, 2024-04-25T14:16:11.987 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 588, 2024-04-25T14:16:11.987 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T14:16:11.987 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 2, 2024-04-25T14:16:11.988 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-25T14:16:11.988 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-25T14:16:11.988 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 1, 2024-04-25T14:16:11.988 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 588, 2024-04-25T14:16:11.988 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T14:16:11.988 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:11.988 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:11.988 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:11.988 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T14:16:11.988 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T14:16:11.988 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T14:16:12.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:12 smithi097 ceph-mon[32720]: pgmap v423: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T14:16:12.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:12 smithi096 ceph-mon[29207]: pgmap v423: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T14:16:12.868 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-25T14:16:12.868 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-25T14:16:12.868 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-04-25T14:16:12.868 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=10 2024-04-25T14:16:12.868 DEBUG:tasks.check_counter:Getting stats from h 2024-04-25T14:16:12.869 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-25T14:16:12.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:12 smithi151 ceph-mon[32990]: pgmap v423: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T14:16:13.108 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T14:16:14.833 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:14 smithi097 ceph-mon[32720]: pgmap v424: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T14:16:14.834 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T14:16:14.834 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:16:14.834 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 544, 2024-04-25T14:16:14.834 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 546, 2024-04-25T14:16:14.834 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 4357825, 2024-04-25T14:16:14.835 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 173002, 2024-04-25T14:16:14.835 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 6, 2024-04-25T14:16:14.835 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 2, 2024-04-25T14:16:14.835 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.053885287, 2024-04-25T14:16:14.835 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.020225132, 2024-04-25T14:16:14.835 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.013740316, 2024-04-25T14:16:14.835 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.014475020, 2024-04-25T14:16:14.835 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:16:14.835 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 544, 2024-04-25T14:16:14.835 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.022454847, 2024-04-25T14:16:14.835 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000041277 2024-04-25T14:16:14.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.835 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:16:14.836 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:14.836 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:14.836 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:14.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.836 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T14:16:14.836 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T14:16:14.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.836 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:16:14.836 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 203, 2024-04-25T14:16:14.836 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 190, 2024-04-25T14:16:14.836 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 4743364, 2024-04-25T14:16:14.836 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 82980, 2024-04-25T14:16:14.836 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 4, 2024-04-25T14:16:14.836 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 3, 2024-04-25T14:16:14.837 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.027608653, 2024-04-25T14:16:14.837 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.007243198, 2024-04-25T14:16:14.837 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.018219788, 2024-04-25T14:16:14.837 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.004446465, 2024-04-25T14:16:14.837 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:16:14.837 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 190, 2024-04-25T14:16:14.837 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.012322297, 2024-04-25T14:16:14.837 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000064854 2024-04-25T14:16:14.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.837 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:16:14.837 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:14.837 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:14.837 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:14.838 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.838 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 525360, 2024-04-25T14:16:14.838 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 52704 2024-04-25T14:16:14.838 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.838 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:16:14.838 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 237, 2024-04-25T14:16:14.838 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 300, 2024-04-25T14:16:14.838 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 4252959, 2024-04-25T14:16:14.838 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 264553, 2024-04-25T14:16:14.838 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 4, 2024-04-25T14:16:14.838 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 3, 2024-04-25T14:16:14.838 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.027650874, 2024-04-25T14:16:14.838 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.008550994, 2024-04-25T14:16:14.838 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.008710570, 2024-04-25T14:16:14.839 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.008086556, 2024-04-25T14:16:14.839 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:16:14.839 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 298, 2024-04-25T14:16:14.839 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.010311082, 2024-04-25T14:16:14.839 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000034600 2024-04-25T14:16:14.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.839 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:16:14.839 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:14.839 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:14.839 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:14.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.839 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T14:16:14.839 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 189504 2024-04-25T14:16:14.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.840 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2024-04-25T14:16:14.840 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 1, 2024-04-25T14:16:14.840 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2024-04-25T14:16:14.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.840 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-MDSRank": { 2024-04-25T14:16:14.840 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-04-25T14:16:14.840 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-04-25T14:16:14.840 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1472, 2024-04-25T14:16:14.840 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.040960029, 2024-04-25T14:16:14.840 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000027826 2024-04-25T14:16:14.840 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T14:16:14.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.840 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-PurgeQueue": { 2024-04-25T14:16:14.840 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-04-25T14:16:14.841 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-04-25T14:16:14.841 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:14.841 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:14.841 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:14.841 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T14:16:14.841 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.841 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": { 2024-04-25T14:16:14.841 INFO:teuthology.orchestra.run.smithi097.stdout: "request": 0, 2024-04-25T14:16:14.841 INFO:teuthology.orchestra.run.smithi097.stdout: "reply": 0, 2024-04-25T14:16:14.841 INFO:teuthology.orchestra.run.smithi097.stdout: "reply_latency": { 2024-04-25T14:16:14.841 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:14.841 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:14.841 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:14.841 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.841 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_reply": 0, 2024-04-25T14:16:14.842 INFO:teuthology.orchestra.run.smithi097.stdout: "forward": 0, 2024-04-25T14:16:14.842 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_complete": 0, 2024-04-25T14:16:14.842 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_keys": 0, 2024-04-25T14:16:14.842 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_commit": 0, 2024-04-25T14:16:14.842 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_split": 0, 2024-04-25T14:16:14.842 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_merge": 0, 2024-04-25T14:16:14.842 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes": 1233, 2024-04-25T14:16:14.842 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_top": 352, 2024-04-25T14:16:14.842 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_bottom": 881, 2024-04-25T14:16:14.842 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pin_tail": 0, 2024-04-25T14:16:14.842 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pinned": 729, 2024-04-25T14:16:14.842 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_expired": 0, 2024-04-25T14:16:14.842 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_with_caps": 0, 2024-04-25T14:16:14.843 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": 0, 2024-04-25T14:16:14.843 INFO:teuthology.orchestra.run.smithi097.stdout: "subtrees": 2, 2024-04-25T14:16:14.843 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse": 0, 2024-04-25T14:16:14.843 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_hit": 0, 2024-04-25T14:16:14.843 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_forward": 0, 2024-04-25T14:16:14.843 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_discover": 0, 2024-04-25T14:16:14.843 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_dir_fetch": 0, 2024-04-25T14:16:14.843 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_remote_ino": 0, 2024-04-25T14:16:14.843 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_lock": 0, 2024-04-25T14:16:14.843 INFO:teuthology.orchestra.run.smithi097.stdout: "load_cent": 0, 2024-04-25T14:16:14.843 INFO:teuthology.orchestra.run.smithi097.stdout: "q": 0, 2024-04-25T14:16:14.843 INFO:teuthology.orchestra.run.smithi097.stdout: "exported": 0, 2024-04-25T14:16:14.843 INFO:teuthology.orchestra.run.smithi097.stdout: "exported_inodes": 0, 2024-04-25T14:16:14.843 INFO:teuthology.orchestra.run.smithi097.stdout: "imported": 0, 2024-04-25T14:16:14.843 INFO:teuthology.orchestra.run.smithi097.stdout: "imported_inodes": 0, 2024-04-25T14:16:14.844 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_dir_fetch": 0, 2024-04-25T14:16:14.844 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_backtrace_fetch": 0, 2024-04-25T14:16:14.844 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_peer_discover": 0, 2024-04-25T14:16:14.844 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rfiles": 4, 2024-04-25T14:16:14.844 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rbytes": 234, 2024-04-25T14:16:14.844 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rsnaps": 0, 2024-04-25T14:16:14.844 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T14:16:14.844 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_set_tag": 0, 2024-04-25T14:16:14.844 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T14:16:14.844 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_inotable_repaired": 0, 2024-04-25T14:16:14.844 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_inodes": 0, 2024-04-25T14:16:14.844 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T14:16:14.844 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T14:16:14.844 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_file_inodes": 0, 2024-04-25T14:16:14.845 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_inode_file_caps": 0, 2024-04-25T14:16:14.845 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T14:16:14.845 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_grant": 0, 2024-04-25T14:16:14.845 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T14:16:14.845 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T14:16:14.845 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T14:16:14.845 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps": 0, 2024-04-25T14:16:14.845 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps_dirty": 0, 2024-04-25T14:16:14.845 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_cap_release": 0, 2024-04-25T14:16:14.845 INFO:teuthology.orchestra.run.smithi097.stdout: "process_request_cap_release": 0 2024-04-25T14:16:14.845 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.845 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_cache": { 2024-04-25T14:16:14.845 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update": 0, 2024-04-25T14:16:14.845 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update_receipt": 0, 2024-04-25T14:16:14.845 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_try_discover": 0, 2024-04-25T14:16:14.846 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_send_discover": 0, 2024-04-25T14:16:14.846 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_handle_discover": 0, 2024-04-25T14:16:14.846 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays": 0, 2024-04-25T14:16:14.846 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_delayed": 0, 2024-04-25T14:16:14.846 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_enqueuing": 0, 2024-04-25T14:16:14.846 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_created": 0, 2024-04-25T14:16:14.846 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_enqueued": 0, 2024-04-25T14:16:14.846 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_reintegrated": 0, 2024-04-25T14:16:14.846 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_migrated": 0, 2024-04-25T14:16:14.846 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_processing": 0, 2024-04-25T14:16:14.847 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_enqueued": 0, 2024-04-25T14:16:14.847 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_prioritized": 0, 2024-04-25T14:16:14.847 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_started": 0, 2024-04-25T14:16:14.847 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_completed": 0, 2024-04-25T14:16:14.847 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_path": 0, 2024-04-25T14:16:14.847 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_inode": 0, 2024-04-25T14:16:14.847 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T14:16:14.847 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_exportdir": 0, 2024-04-25T14:16:14.847 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_flush": 0, 2024-04-25T14:16:14.847 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragmentdir": 0, 2024-04-25T14:16:14.847 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragstats": 0, 2024-04-25T14:16:14.847 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_inodestats": 0 2024-04-25T14:16:14.847 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.847 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_log": { 2024-04-25T14:16:14.848 INFO:teuthology.orchestra.run.smithi097.stdout: "evlrg": 0, 2024-04-25T14:16:14.848 INFO:teuthology.orchestra.run.smithi097.stdout: "evadd": 0, 2024-04-25T14:16:14.848 INFO:teuthology.orchestra.run.smithi097.stdout: "evex": 0, 2024-04-25T14:16:14.848 INFO:teuthology.orchestra.run.smithi097.stdout: "evtrm": 0, 2024-04-25T14:16:14.848 INFO:teuthology.orchestra.run.smithi097.stdout: "ev": 6117, 2024-04-25T14:16:14.848 INFO:teuthology.orchestra.run.smithi097.stdout: "evexg": 0, 2024-04-25T14:16:14.848 INFO:teuthology.orchestra.run.smithi097.stdout: "evexd": 0, 2024-04-25T14:16:14.848 INFO:teuthology.orchestra.run.smithi097.stdout: "segadd": 0, 2024-04-25T14:16:14.848 INFO:teuthology.orchestra.run.smithi097.stdout: "segex": 0, 2024-04-25T14:16:14.848 INFO:teuthology.orchestra.run.smithi097.stdout: "segtrm": 0, 2024-04-25T14:16:14.849 INFO:teuthology.orchestra.run.smithi097.stdout: "seg": 4, 2024-04-25T14:16:14.849 INFO:teuthology.orchestra.run.smithi097.stdout: "segmjr": 10, 2024-04-25T14:16:14.849 INFO:teuthology.orchestra.run.smithi097.stdout: "segexg": 0, 2024-04-25T14:16:14.849 INFO:teuthology.orchestra.run.smithi097.stdout: "segexd": 0, 2024-04-25T14:16:14.849 INFO:teuthology.orchestra.run.smithi097.stdout: "expos": 10944674, 2024-04-25T14:16:14.849 INFO:teuthology.orchestra.run.smithi097.stdout: "wrpos": 16776090, 2024-04-25T14:16:14.849 INFO:teuthology.orchestra.run.smithi097.stdout: "rdpos": 16774438, 2024-04-25T14:16:14.849 INFO:teuthology.orchestra.run.smithi097.stdout: "jlat": { 2024-04-25T14:16:14.849 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:14.849 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:14.849 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:14.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.849 INFO:teuthology.orchestra.run.smithi097.stdout: "replayed": 6117 2024-04-25T14:16:14.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.850 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_mem": { 2024-04-25T14:16:14.850 INFO:teuthology.orchestra.run.smithi097.stdout: "ino": 21, 2024-04-25T14:16:14.850 INFO:teuthology.orchestra.run.smithi097.stdout: "ino+": 1233, 2024-04-25T14:16:14.850 INFO:teuthology.orchestra.run.smithi097.stdout: "ino-": 1212, 2024-04-25T14:16:14.850 INFO:teuthology.orchestra.run.smithi097.stdout: "dir": 16, 2024-04-25T14:16:14.850 INFO:teuthology.orchestra.run.smithi097.stdout: "dir+": 22, 2024-04-25T14:16:14.850 INFO:teuthology.orchestra.run.smithi097.stdout: "dir-": 6, 2024-04-25T14:16:14.850 INFO:teuthology.orchestra.run.smithi097.stdout: "dn": 1233, 2024-04-25T14:16:14.850 INFO:teuthology.orchestra.run.smithi097.stdout: "dn+": 1240, 2024-04-25T14:16:14.850 INFO:teuthology.orchestra.run.smithi097.stdout: "dn-": 7, 2024-04-25T14:16:14.850 INFO:teuthology.orchestra.run.smithi097.stdout: "cap": 0, 2024-04-25T14:16:14.850 INFO:teuthology.orchestra.run.smithi097.stdout: "cap+": 0, 2024-04-25T14:16:14.850 INFO:teuthology.orchestra.run.smithi097.stdout: "cap-": 0, 2024-04-25T14:16:14.850 INFO:teuthology.orchestra.run.smithi097.stdout: "rss": 120572, 2024-04-25T14:16:14.851 INFO:teuthology.orchestra.run.smithi097.stdout: "heap": 215440 2024-04-25T14:16:14.851 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.851 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_server": { 2024-04-25T14:16:14.851 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_client_request": 0, 2024-04-25T14:16:14.851 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_server_request": 0, 2024-04-25T14:16:14.851 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_request": 0, 2024-04-25T14:16:14.851 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_session": 0, 2024-04-25T14:16:14.851 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_peer_request": 0, 2024-04-25T14:16:14.851 INFO:teuthology.orchestra.run.smithi097.stdout: "req_create_latency": { 2024-04-25T14:16:14.851 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:14.851 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:14.851 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:14.851 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.851 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getattr_latency": { 2024-04-25T14:16:14.852 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:14.852 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:14.852 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:14.852 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.852 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getfilelock_latency": { 2024-04-25T14:16:14.852 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:14.852 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:14.852 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:14.852 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.852 INFO:teuthology.orchestra.run.smithi097.stdout: "req_link_latency": { 2024-04-25T14:16:14.852 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:14.852 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:14.852 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:14.852 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.852 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookup_latency": { 2024-04-25T14:16:14.853 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:14.853 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:14.853 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:14.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.853 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookuphash_latency": { 2024-04-25T14:16:14.853 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:14.853 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:14.853 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:14.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.853 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupino_latency": { 2024-04-25T14:16:14.853 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:14.853 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:14.853 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:14.854 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.854 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupname_latency": { 2024-04-25T14:16:14.854 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:14.854 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:14.854 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:14.854 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.854 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupparent_latency": { 2024-04-25T14:16:14.854 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:14.854 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:14.854 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:14.854 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.854 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupsnap_latency": { 2024-04-25T14:16:14.854 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:14.855 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:14.855 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:14.855 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.855 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lssnap_latency": { 2024-04-25T14:16:14.855 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:14.855 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:14.855 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:14.855 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.855 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mkdir_latency": { 2024-04-25T14:16:14.856 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:14.856 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:14.856 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:14.856 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.857 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mknod_latency": { 2024-04-25T14:16:14.857 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:14.857 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:14.857 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:14.857 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.857 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mksnap_latency": { 2024-04-25T14:16:14.857 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:14.857 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:14.857 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:14.857 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.857 INFO:teuthology.orchestra.run.smithi097.stdout: "req_open_latency": { 2024-04-25T14:16:14.858 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:14.858 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:14.858 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:14.858 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.858 INFO:teuthology.orchestra.run.smithi097.stdout: "req_readdir_latency": { 2024-04-25T14:16:14.858 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:14.858 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:14.858 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:14.858 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.858 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rename_latency": { 2024-04-25T14:16:14.858 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:14.858 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:14.859 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:14.859 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.859 INFO:teuthology.orchestra.run.smithi097.stdout: "req_renamesnap_latency": { 2024-04-25T14:16:14.859 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:14.859 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:14.859 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:14.859 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.859 INFO:teuthology.orchestra.run.smithi097.stdout: "req_snapdiff_latency": { 2024-04-25T14:16:14.859 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:14.859 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:14.859 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:14.859 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.859 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmdir_latency": { 2024-04-25T14:16:14.860 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:14.860 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:14.860 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:14.860 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.860 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmsnap_latency": { 2024-04-25T14:16:14.860 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:14.860 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:14.860 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:14.860 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.860 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmxattr_latency": { 2024-04-25T14:16:14.860 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:14.860 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:14.860 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:14.860 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.860 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setattr_latency": { 2024-04-25T14:16:14.861 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:14.861 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:14.861 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:14.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.861 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setdirlayout_latency": { 2024-04-25T14:16:14.861 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:14.861 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:14.861 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:14.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.861 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setfilelock_latency": { 2024-04-25T14:16:14.861 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:14.861 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:14.861 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:14.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.861 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setlayout_latency": { 2024-04-25T14:16:14.862 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:14.862 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:14.862 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:14.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.862 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setxattr_latency": { 2024-04-25T14:16:14.862 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:14.862 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:14.862 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:14.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.862 INFO:teuthology.orchestra.run.smithi097.stdout: "req_symlink_latency": { 2024-04-25T14:16:14.862 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:14.862 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:14.862 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:14.863 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.863 INFO:teuthology.orchestra.run.smithi097.stdout: "req_unlink_latency": { 2024-04-25T14:16:14.863 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:14.863 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:14.863 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:14.863 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.863 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_revoke_eviction": 0, 2024-04-25T14:16:14.863 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_acquisition_throttle": 0, 2024-04-25T14:16:14.863 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getvxattr_latency": { 2024-04-25T14:16:14.863 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:14.863 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:14.863 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:14.863 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T14:16:14.863 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.864 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_sessions": { 2024-04-25T14:16:14.864 INFO:teuthology.orchestra.run.smithi097.stdout: "session_count": 1, 2024-04-25T14:16:14.864 INFO:teuthology.orchestra.run.smithi097.stdout: "session_add": 2, 2024-04-25T14:16:14.864 INFO:teuthology.orchestra.run.smithi097.stdout: "session_remove": 1, 2024-04-25T14:16:14.864 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_open": 1, 2024-04-25T14:16:14.864 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_stale": 0, 2024-04-25T14:16:14.864 INFO:teuthology.orchestra.run.smithi097.stdout: "total_load": 0, 2024-04-25T14:16:14.864 INFO:teuthology.orchestra.run.smithi097.stdout: "average_load": 0, 2024-04-25T14:16:14.864 INFO:teuthology.orchestra.run.smithi097.stdout: "avg_session_uptime": 1292, 2024-04-25T14:16:14.864 INFO:teuthology.orchestra.run.smithi097.stdout: "mdthresh_evicted": 0 2024-04-25T14:16:14.864 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.864 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2024-04-25T14:16:14.864 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:16:14.864 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 0, 2024-04-25T14:16:14.865 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:16:14.865 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:16:14.865 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:16:14.865 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:16:14.865 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:16:14.865 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:16:14.865 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:16:14.865 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:16:14.865 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:16:14.865 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:16:14.865 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:16:14.865 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:16:14.865 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:16:14.865 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2024-04-25T14:16:14.866 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:16:14.866 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2024-04-25T14:16:14.866 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:16:14.866 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:16:14.866 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:16:14.866 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:16:14.866 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:16:14.866 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:16:14.866 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:16:14.866 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2024-04-25T14:16:14.866 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:16:14.866 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:16:14.866 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:16:14.867 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2024-04-25T14:16:14.867 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2024-04-25T14:16:14.867 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2024-04-25T14:16:14.867 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:16:14.867 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:16:14.867 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:16:14.867 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:16:14.867 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 189256, 2024-04-25T14:16:14.867 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 49, 2024-04-25T14:16:14.867 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 0, 2024-04-25T14:16:14.867 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 0, 2024-04-25T14:16:14.867 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2024-04-25T14:16:14.867 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2024-04-25T14:16:14.867 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:16:14.868 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2024-04-25T14:16:14.868 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:16:14.868 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2024-04-25T14:16:14.868 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 7720, 2024-04-25T14:16:14.868 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 162, 2024-04-25T14:16:14.868 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:16:14.868 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:16:14.868 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2024-04-25T14:16:14.868 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2024-04-25T14:16:14.868 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 935418, 2024-04-25T14:16:14.868 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 5199, 2024-04-25T14:16:14.868 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2024-04-25T14:16:14.868 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2024-04-25T14:16:14.869 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2024-04-25T14:16:14.869 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2024-04-25T14:16:14.869 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.869 INFO:teuthology.orchestra.run.smithi097.stdout: "objecter": { 2024-04-25T14:16:14.869 INFO:teuthology.orchestra.run.smithi097.stdout: "op_active": 0, 2024-04-25T14:16:14.869 INFO:teuthology.orchestra.run.smithi097.stdout: "op_laggy": 0, 2024-04-25T14:16:14.869 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send": 875, 2024-04-25T14:16:14.869 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send_bytes": 26, 2024-04-25T14:16:14.869 INFO:teuthology.orchestra.run.smithi097.stdout: "op_resend": 4, 2024-04-25T14:16:14.869 INFO:teuthology.orchestra.run.smithi097.stdout: "op_reply": 875, 2024-04-25T14:16:14.869 INFO:teuthology.orchestra.run.smithi097.stdout: "op_latency": { 2024-04-25T14:16:14.869 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 875, 2024-04-25T14:16:14.869 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.411992888, 2024-04-25T14:16:14.870 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000470849 2024-04-25T14:16:14.870 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.870 INFO:teuthology.orchestra.run.smithi097.stdout: "op_inflight": 0, 2024-04-25T14:16:14.870 INFO:teuthology.orchestra.run.smithi097.stdout: "oplen_avg": { 2024-04-25T14:16:14.870 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 875, 2024-04-25T14:16:14.870 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 876 2024-04-25T14:16:14.870 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.870 INFO:teuthology.orchestra.run.smithi097.stdout: "op": 875, 2024-04-25T14:16:14.870 INFO:teuthology.orchestra.run.smithi097.stdout: "op_r": 875, 2024-04-25T14:16:14.870 INFO:teuthology.orchestra.run.smithi097.stdout: "op_w": 0, 2024-04-25T14:16:14.870 INFO:teuthology.orchestra.run.smithi097.stdout: "op_rmw": 0, 2024-04-25T14:16:14.870 INFO:teuthology.orchestra.run.smithi097.stdout: "op_pg": 0, 2024-04-25T14:16:14.870 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_stat": 542, 2024-04-25T14:16:14.871 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_create": 0, 2024-04-25T14:16:14.871 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_read": 330, 2024-04-25T14:16:14.871 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_write": 0, 2024-04-25T14:16:14.871 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writefull": 0, 2024-04-25T14:16:14.871 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writesame": 0, 2024-04-25T14:16:14.871 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_append": 0, 2024-04-25T14:16:14.871 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_zero": 0, 2024-04-25T14:16:14.871 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_truncate": 0, 2024-04-25T14:16:14.871 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_delete": 0, 2024-04-25T14:16:14.871 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_mapext": 0, 2024-04-25T14:16:14.871 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_sparse_read": 0, 2024-04-25T14:16:14.871 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_clonerange": 0, 2024-04-25T14:16:14.871 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_getxattr": 2, 2024-04-25T14:16:14.871 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_setxattr": 0, 2024-04-25T14:16:14.872 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_cmpxattr": 0, 2024-04-25T14:16:14.872 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_rmxattr": 0, 2024-04-25T14:16:14.872 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_resetxattrs": 0, 2024-04-25T14:16:14.872 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_call": 0, 2024-04-25T14:16:14.872 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_watch": 0, 2024-04-25T14:16:14.872 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_notify": 0, 2024-04-25T14:16:14.872 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T14:16:14.872 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls": 0, 2024-04-25T14:16:14.872 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls_filter": 0, 2024-04-25T14:16:14.872 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_other": 0, 2024-04-25T14:16:14.872 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_active": 0, 2024-04-25T14:16:14.872 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_send": 0, 2024-04-25T14:16:14.872 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_resend": 0, 2024-04-25T14:16:14.873 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_ping": 0, 2024-04-25T14:16:14.873 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_active": 0, 2024-04-25T14:16:14.873 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_send": 0, 2024-04-25T14:16:14.873 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_resend": 0, 2024-04-25T14:16:14.873 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_active": 0, 2024-04-25T14:16:14.873 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_send": 0, 2024-04-25T14:16:14.873 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_resend": 0, 2024-04-25T14:16:14.873 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_active": 0, 2024-04-25T14:16:14.873 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_send": 0, 2024-04-25T14:16:14.873 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_resend": 0, 2024-04-25T14:16:14.873 INFO:teuthology.orchestra.run.smithi097.stdout: "command_active": 0, 2024-04-25T14:16:14.873 INFO:teuthology.orchestra.run.smithi097.stdout: "command_send": 0, 2024-04-25T14:16:14.873 INFO:teuthology.orchestra.run.smithi097.stdout: "command_resend": 0, 2024-04-25T14:16:14.873 INFO:teuthology.orchestra.run.smithi097.stdout: "map_epoch": 78, 2024-04-25T14:16:14.874 INFO:teuthology.orchestra.run.smithi097.stdout: "map_full": 0, 2024-04-25T14:16:14.874 INFO:teuthology.orchestra.run.smithi097.stdout: "map_inc": 2, 2024-04-25T14:16:14.874 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_sessions": 5, 2024-04-25T14:16:14.874 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_open": 5, 2024-04-25T14:16:14.874 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_close": 0, 2024-04-25T14:16:14.874 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_laggy": 0, 2024-04-25T14:16:14.874 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_wr": 0, 2024-04-25T14:16:14.874 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_rd": 2, 2024-04-25T14:16:14.874 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_del": 0 2024-04-25T14:16:14.874 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.874 INFO:teuthology.orchestra.run.smithi097.stdout: "oft": { 2024-04-25T14:16:14.874 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_objs": 0, 2024-04-25T14:16:14.874 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_kv_pairs": 0, 2024-04-25T14:16:14.875 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_updates": 0, 2024-04-25T14:16:14.875 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_removes": 0 2024-04-25T14:16:14.875 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.875 INFO:teuthology.orchestra.run.smithi097.stdout: "osd-slow-ops": { 2024-04-25T14:16:14.875 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_ops_count": 0 2024-04-25T14:16:14.875 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.875 INFO:teuthology.orchestra.run.smithi097.stdout: "purge_queue": { 2024-04-25T14:16:14.875 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops": 0, 2024-04-25T14:16:14.875 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T14:16:14.875 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing": 0, 2024-04-25T14:16:14.875 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_high_water": 0, 2024-04-25T14:16:14.875 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed_ops": 0, 2024-04-25T14:16:14.875 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed": 0, 2024-04-25T14:16:14.876 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_item_in_journal": 0 2024-04-25T14:16:14.876 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.876 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:16:14.876 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T14:16:14.876 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-04-25T14:16:14.876 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T14:16:14.876 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 980, 2024-04-25T14:16:14.876 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 13257671, 2024-04-25T14:16:14.876 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T14:16:14.876 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 980, 2024-04-25T14:16:14.876 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-25T14:16:14.876 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-25T14:16:14.876 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 980, 2024-04-25T14:16:14.877 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 13257671, 2024-04-25T14:16:14.877 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T14:16:14.877 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:14.877 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:14.877 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:14.877 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T14:16:14.877 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.877 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_bytes": { 2024-04-25T14:16:14.877 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T14:16:14.877 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-04-25T14:16:14.877 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T14:16:14.877 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-25T14:16:14.877 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-25T14:16:14.877 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T14:16:14.878 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-25T14:16:14.878 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 875, 2024-04-25T14:16:14.878 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 12581796, 2024-04-25T14:16:14.878 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 875, 2024-04-25T14:16:14.878 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 12581796, 2024-04-25T14:16:14.878 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T14:16:14.878 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:14.878 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:14.878 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:14.878 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T14:16:14.878 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.878 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_ops": { 2024-04-25T14:16:14.879 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T14:16:14.879 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 1024, 2024-04-25T14:16:14.879 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T14:16:14.879 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-25T14:16:14.879 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-25T14:16:14.879 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T14:16:14.879 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-25T14:16:14.879 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 875, 2024-04-25T14:16:14.879 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 875, 2024-04-25T14:16:14.879 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 875, 2024-04-25T14:16:14.879 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 875, 2024-04-25T14:16:14.879 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T14:16:14.879 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:14.879 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:14.880 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:14.880 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T14:16:14.880 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.880 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle": { 2024-04-25T14:16:14.880 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T14:16:14.880 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-04-25T14:16:14.880 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T14:16:14.880 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-25T14:16:14.880 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-25T14:16:14.880 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T14:16:14.880 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-25T14:16:14.880 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-25T14:16:14.880 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-25T14:16:14.880 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 0, 2024-04-25T14:16:14.881 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 0, 2024-04-25T14:16:14.881 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T14:16:14.881 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:14.881 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:14.881 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:14.881 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T14:16:14.881 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:14.881 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle-0x55be98de6750": { 2024-04-25T14:16:14.881 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T14:16:14.881 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-04-25T14:16:14.881 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T14:16:14.881 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-25T14:16:14.881 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-25T14:16:14.881 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T14:16:14.881 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-25T14:16:14.882 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-25T14:16:14.882 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-25T14:16:14.882 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 0, 2024-04-25T14:16:14.882 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 0, 2024-04-25T14:16:14.882 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T14:16:14.882 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:14.882 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:14.882 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:14.882 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T14:16:14.882 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T14:16:14.882 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T14:16:14.883 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:14 smithi096 ceph-mon[29207]: pgmap v424: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T14:16:14.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:14 smithi151 ceph-mon[32990]: pgmap v424: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T14:16:15.651 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-25T14:16:15.652 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-25T14:16:15.652 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-04-25T14:16:15.652 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2024-04-25T14:16:15.652 DEBUG:tasks.check_counter:Getting stats from k 2024-04-25T14:16:15.652 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-25T14:16:15.891 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T14:16:16.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:16 smithi097 ceph-mon[32720]: pgmap v425: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T14:16:16.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:16 smithi096 ceph-mon[29207]: pgmap v425: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T14:16:16.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:16 smithi151 ceph-mon[32990]: pgmap v425: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T14:16:17.562 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T14:16:17.563 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:16:17.563 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 288, 2024-04-25T14:16:17.563 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 290, 2024-04-25T14:16:17.563 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 77475, 2024-04-25T14:16:17.563 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 91648, 2024-04-25T14:16:17.563 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 6, 2024-04-25T14:16:17.563 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 2, 2024-04-25T14:16:17.563 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.026823999, 2024-04-25T14:16:17.563 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.010114038, 2024-04-25T14:16:17.563 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.006592271, 2024-04-25T14:16:17.564 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.006899306, 2024-04-25T14:16:17.564 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:16:17.564 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 288, 2024-04-25T14:16:17.564 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.010310395, 2024-04-25T14:16:17.564 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000035799 2024-04-25T14:16:17.564 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.564 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:16:17.564 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:17.565 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:17.565 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:17.565 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.565 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T14:16:17.565 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T14:16:17.565 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.565 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:16:17.565 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 209, 2024-04-25T14:16:17.565 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 276, 2024-04-25T14:16:17.565 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 51350, 2024-04-25T14:16:17.566 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 256357, 2024-04-25T14:16:17.566 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 5, 2024-04-25T14:16:17.566 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 4, 2024-04-25T14:16:17.566 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.022759564, 2024-04-25T14:16:17.566 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.008181389, 2024-04-25T14:16:17.566 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.005552908, 2024-04-25T14:16:17.566 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.006785086, 2024-04-25T14:16:17.566 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:16:17.567 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 274, 2024-04-25T14:16:17.567 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.010621547, 2024-04-25T14:16:17.567 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000038764 2024-04-25T14:16:17.567 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.567 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:16:17.567 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:17.567 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:17.567 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:17.567 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.567 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T14:16:17.567 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 190208 2024-04-25T14:16:17.567 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.567 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:16:17.568 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 387, 2024-04-25T14:16:17.568 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 381, 2024-04-25T14:16:17.568 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 591920, 2024-04-25T14:16:17.568 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 145420, 2024-04-25T14:16:17.568 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 3, 2024-04-25T14:16:17.568 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 2, 2024-04-25T14:16:17.568 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.050552056, 2024-04-25T14:16:17.568 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.016055034, 2024-04-25T14:16:17.568 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.021567758, 2024-04-25T14:16:17.568 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.012747700, 2024-04-25T14:16:17.568 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:16:17.569 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 381, 2024-04-25T14:16:17.569 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.022621581, 2024-04-25T14:16:17.569 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000059374 2024-04-25T14:16:17.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.569 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:16:17.569 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:17.569 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:17.569 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:17.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.569 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 497168, 2024-04-25T14:16:17.569 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 55744 2024-04-25T14:16:17.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.570 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2024-04-25T14:16:17.570 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 1, 2024-04-25T14:16:17.570 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2024-04-25T14:16:17.570 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.570 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-MDSRank": { 2024-04-25T14:16:17.570 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-04-25T14:16:17.570 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-04-25T14:16:17.570 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1328, 2024-04-25T14:16:17.570 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.037975683, 2024-04-25T14:16:17.570 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000028596 2024-04-25T14:16:17.570 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T14:16:17.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.571 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-PurgeQueue": { 2024-04-25T14:16:17.571 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-04-25T14:16:17.571 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-04-25T14:16:17.571 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:17.571 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:17.571 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:17.571 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T14:16:17.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.571 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": { 2024-04-25T14:16:17.571 INFO:teuthology.orchestra.run.smithi097.stdout: "request": 0, 2024-04-25T14:16:17.571 INFO:teuthology.orchestra.run.smithi097.stdout: "reply": 0, 2024-04-25T14:16:17.571 INFO:teuthology.orchestra.run.smithi097.stdout: "reply_latency": { 2024-04-25T14:16:17.571 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:17.572 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:17.572 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:17.572 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.572 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_reply": 0, 2024-04-25T14:16:17.572 INFO:teuthology.orchestra.run.smithi097.stdout: "forward": 0, 2024-04-25T14:16:17.572 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_complete": 0, 2024-04-25T14:16:17.572 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_keys": 0, 2024-04-25T14:16:17.572 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_commit": 0, 2024-04-25T14:16:17.572 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_split": 0, 2024-04-25T14:16:17.572 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_merge": 0, 2024-04-25T14:16:17.572 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes": 0, 2024-04-25T14:16:17.572 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_top": 0, 2024-04-25T14:16:17.572 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_bottom": 0, 2024-04-25T14:16:17.572 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pin_tail": 0, 2024-04-25T14:16:17.573 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pinned": 0, 2024-04-25T14:16:17.573 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_expired": 0, 2024-04-25T14:16:17.573 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_with_caps": 0, 2024-04-25T14:16:17.573 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": 0, 2024-04-25T14:16:17.573 INFO:teuthology.orchestra.run.smithi097.stdout: "subtrees": 1, 2024-04-25T14:16:17.573 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse": 0, 2024-04-25T14:16:17.573 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_hit": 0, 2024-04-25T14:16:17.573 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_forward": 0, 2024-04-25T14:16:17.573 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_discover": 0, 2024-04-25T14:16:17.573 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_dir_fetch": 0, 2024-04-25T14:16:17.573 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_remote_ino": 0, 2024-04-25T14:16:17.573 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_lock": 0, 2024-04-25T14:16:17.573 INFO:teuthology.orchestra.run.smithi097.stdout: "load_cent": 0, 2024-04-25T14:16:17.573 INFO:teuthology.orchestra.run.smithi097.stdout: "q": 0, 2024-04-25T14:16:17.574 INFO:teuthology.orchestra.run.smithi097.stdout: "exported": 0, 2024-04-25T14:16:17.574 INFO:teuthology.orchestra.run.smithi097.stdout: "exported_inodes": 0, 2024-04-25T14:16:17.574 INFO:teuthology.orchestra.run.smithi097.stdout: "imported": 0, 2024-04-25T14:16:17.574 INFO:teuthology.orchestra.run.smithi097.stdout: "imported_inodes": 0, 2024-04-25T14:16:17.574 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_dir_fetch": 0, 2024-04-25T14:16:17.574 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_backtrace_fetch": 0, 2024-04-25T14:16:17.574 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_peer_discover": 0, 2024-04-25T14:16:17.574 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rfiles": 0, 2024-04-25T14:16:17.574 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rbytes": 0, 2024-04-25T14:16:17.574 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rsnaps": 0, 2024-04-25T14:16:17.574 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T14:16:17.575 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_set_tag": 0, 2024-04-25T14:16:17.575 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T14:16:17.575 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_inotable_repaired": 0, 2024-04-25T14:16:17.575 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_inodes": 0, 2024-04-25T14:16:17.575 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T14:16:17.575 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T14:16:17.575 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_file_inodes": 0, 2024-04-25T14:16:17.575 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_inode_file_caps": 0, 2024-04-25T14:16:17.575 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T14:16:17.575 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_grant": 0, 2024-04-25T14:16:17.575 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T14:16:17.575 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T14:16:17.575 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T14:16:17.575 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps": 0, 2024-04-25T14:16:17.576 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps_dirty": 0, 2024-04-25T14:16:17.576 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_cap_release": 0, 2024-04-25T14:16:17.576 INFO:teuthology.orchestra.run.smithi097.stdout: "process_request_cap_release": 0 2024-04-25T14:16:17.576 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.576 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_cache": { 2024-04-25T14:16:17.576 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update": 0, 2024-04-25T14:16:17.576 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update_receipt": 0, 2024-04-25T14:16:17.576 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_try_discover": 0, 2024-04-25T14:16:17.576 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_send_discover": 0, 2024-04-25T14:16:17.576 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_handle_discover": 0, 2024-04-25T14:16:17.576 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays": 0, 2024-04-25T14:16:17.576 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_delayed": 0, 2024-04-25T14:16:17.576 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_enqueuing": 0, 2024-04-25T14:16:17.577 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_created": 0, 2024-04-25T14:16:17.577 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_enqueued": 0, 2024-04-25T14:16:17.577 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_reintegrated": 0, 2024-04-25T14:16:17.577 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_migrated": 0, 2024-04-25T14:16:17.577 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_processing": 0, 2024-04-25T14:16:17.577 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_enqueued": 0, 2024-04-25T14:16:17.577 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_prioritized": 0, 2024-04-25T14:16:17.577 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_started": 0, 2024-04-25T14:16:17.577 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_completed": 0, 2024-04-25T14:16:17.577 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_path": 0, 2024-04-25T14:16:17.578 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_inode": 0, 2024-04-25T14:16:17.578 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T14:16:17.578 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_exportdir": 0, 2024-04-25T14:16:17.578 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_flush": 0, 2024-04-25T14:16:17.578 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragmentdir": 0, 2024-04-25T14:16:17.578 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragstats": 0, 2024-04-25T14:16:17.578 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_inodestats": 0 2024-04-25T14:16:17.578 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.578 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_log": { 2024-04-25T14:16:17.578 INFO:teuthology.orchestra.run.smithi097.stdout: "evlrg": 0, 2024-04-25T14:16:17.578 INFO:teuthology.orchestra.run.smithi097.stdout: "evadd": 0, 2024-04-25T14:16:17.578 INFO:teuthology.orchestra.run.smithi097.stdout: "evex": 0, 2024-04-25T14:16:17.578 INFO:teuthology.orchestra.run.smithi097.stdout: "evtrm": 0, 2024-04-25T14:16:17.579 INFO:teuthology.orchestra.run.smithi097.stdout: "ev": 5, 2024-04-25T14:16:17.579 INFO:teuthology.orchestra.run.smithi097.stdout: "evexg": 0, 2024-04-25T14:16:17.579 INFO:teuthology.orchestra.run.smithi097.stdout: "evexd": 0, 2024-04-25T14:16:17.579 INFO:teuthology.orchestra.run.smithi097.stdout: "segadd": 0, 2024-04-25T14:16:17.579 INFO:teuthology.orchestra.run.smithi097.stdout: "segex": 0, 2024-04-25T14:16:17.579 INFO:teuthology.orchestra.run.smithi097.stdout: "segtrm": 0, 2024-04-25T14:16:17.579 INFO:teuthology.orchestra.run.smithi097.stdout: "seg": 2, 2024-04-25T14:16:17.579 INFO:teuthology.orchestra.run.smithi097.stdout: "segmjr": 2, 2024-04-25T14:16:17.579 INFO:teuthology.orchestra.run.smithi097.stdout: "segexg": 0, 2024-04-25T14:16:17.580 INFO:teuthology.orchestra.run.smithi097.stdout: "segexd": 0, 2024-04-25T14:16:17.580 INFO:teuthology.orchestra.run.smithi097.stdout: "expos": 4194304, 2024-04-25T14:16:17.580 INFO:teuthology.orchestra.run.smithi097.stdout: "wrpos": 4195732, 2024-04-25T14:16:17.580 INFO:teuthology.orchestra.run.smithi097.stdout: "rdpos": 4195424, 2024-04-25T14:16:17.580 INFO:teuthology.orchestra.run.smithi097.stdout: "jlat": { 2024-04-25T14:16:17.580 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:17.580 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:17.580 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:17.580 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.580 INFO:teuthology.orchestra.run.smithi097.stdout: "replayed": 5 2024-04-25T14:16:17.580 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.580 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_mem": { 2024-04-25T14:16:17.581 INFO:teuthology.orchestra.run.smithi097.stdout: "ino": 3, 2024-04-25T14:16:17.581 INFO:teuthology.orchestra.run.smithi097.stdout: "ino+": 3, 2024-04-25T14:16:17.581 INFO:teuthology.orchestra.run.smithi097.stdout: "ino-": 0, 2024-04-25T14:16:17.581 INFO:teuthology.orchestra.run.smithi097.stdout: "dir": 1, 2024-04-25T14:16:17.581 INFO:teuthology.orchestra.run.smithi097.stdout: "dir+": 1, 2024-04-25T14:16:17.581 INFO:teuthology.orchestra.run.smithi097.stdout: "dir-": 0, 2024-04-25T14:16:17.581 INFO:teuthology.orchestra.run.smithi097.stdout: "dn": 0, 2024-04-25T14:16:17.581 INFO:teuthology.orchestra.run.smithi097.stdout: "dn+": 0, 2024-04-25T14:16:17.581 INFO:teuthology.orchestra.run.smithi097.stdout: "dn-": 0, 2024-04-25T14:16:17.581 INFO:teuthology.orchestra.run.smithi097.stdout: "cap": 0, 2024-04-25T14:16:17.581 INFO:teuthology.orchestra.run.smithi097.stdout: "cap+": 0, 2024-04-25T14:16:17.581 INFO:teuthology.orchestra.run.smithi097.stdout: "cap-": 0, 2024-04-25T14:16:17.582 INFO:teuthology.orchestra.run.smithi097.stdout: "rss": 118176, 2024-04-25T14:16:17.582 INFO:teuthology.orchestra.run.smithi097.stdout: "heap": 215440 2024-04-25T14:16:17.582 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.582 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_server": { 2024-04-25T14:16:17.582 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_client_request": 0, 2024-04-25T14:16:17.582 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_server_request": 0, 2024-04-25T14:16:17.582 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_request": 0, 2024-04-25T14:16:17.582 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_session": 0, 2024-04-25T14:16:17.582 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_peer_request": 0, 2024-04-25T14:16:17.583 INFO:teuthology.orchestra.run.smithi097.stdout: "req_create_latency": { 2024-04-25T14:16:17.583 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:17.583 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:17.583 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:17.583 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.583 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getattr_latency": { 2024-04-25T14:16:17.583 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:17.583 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:17.583 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:17.583 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.583 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getfilelock_latency": { 2024-04-25T14:16:17.583 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:17.583 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:17.584 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:17.584 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.584 INFO:teuthology.orchestra.run.smithi097.stdout: "req_link_latency": { 2024-04-25T14:16:17.584 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:17.584 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:17.584 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:17.584 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.584 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookup_latency": { 2024-04-25T14:16:17.584 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:17.585 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:17.585 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:17.585 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.585 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookuphash_latency": { 2024-04-25T14:16:17.585 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:17.585 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:17.585 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:17.585 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.585 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupino_latency": { 2024-04-25T14:16:17.585 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:17.585 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:17.585 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:17.586 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.586 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupname_latency": { 2024-04-25T14:16:17.586 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:17.586 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:17.586 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:17.586 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.586 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupparent_latency": { 2024-04-25T14:16:17.586 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:17.586 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:17.587 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:17.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.587 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupsnap_latency": { 2024-04-25T14:16:17.587 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:17.587 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:17.587 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:17.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.587 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lssnap_latency": { 2024-04-25T14:16:17.587 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:17.587 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:17.588 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:17.588 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.588 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mkdir_latency": { 2024-04-25T14:16:17.588 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:17.589 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:17.589 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:17.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.589 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mknod_latency": { 2024-04-25T14:16:17.589 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:17.589 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:17.589 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:17.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.589 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mksnap_latency": { 2024-04-25T14:16:17.589 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:17.589 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:17.589 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:17.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.590 INFO:teuthology.orchestra.run.smithi097.stdout: "req_open_latency": { 2024-04-25T14:16:17.590 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:17.590 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:17.590 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:17.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.590 INFO:teuthology.orchestra.run.smithi097.stdout: "req_readdir_latency": { 2024-04-25T14:16:17.590 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:17.590 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:17.590 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:17.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.590 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rename_latency": { 2024-04-25T14:16:17.590 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:17.591 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:17.591 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:17.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.591 INFO:teuthology.orchestra.run.smithi097.stdout: "req_renamesnap_latency": { 2024-04-25T14:16:17.591 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:17.591 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:17.591 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:17.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.591 INFO:teuthology.orchestra.run.smithi097.stdout: "req_snapdiff_latency": { 2024-04-25T14:16:17.591 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:17.591 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:17.591 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:17.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.591 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmdir_latency": { 2024-04-25T14:16:17.592 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:17.592 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:17.592 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:17.592 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.592 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmsnap_latency": { 2024-04-25T14:16:17.592 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:17.592 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:17.592 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:17.592 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.592 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmxattr_latency": { 2024-04-25T14:16:17.592 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:17.592 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:17.592 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:17.593 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.593 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setattr_latency": { 2024-04-25T14:16:17.593 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:17.593 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:17.593 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:17.593 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.593 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setdirlayout_latency": { 2024-04-25T14:16:17.593 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:17.593 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:17.593 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:17.593 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.593 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setfilelock_latency": { 2024-04-25T14:16:17.594 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:17.594 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:17.594 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:17.594 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.594 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setlayout_latency": { 2024-04-25T14:16:17.594 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:17.595 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:17.595 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:17.595 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.595 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setxattr_latency": { 2024-04-25T14:16:17.595 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:17.595 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:17.596 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:17.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.596 INFO:teuthology.orchestra.run.smithi097.stdout: "req_symlink_latency": { 2024-04-25T14:16:17.596 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:17.596 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:17.596 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:17.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.597 INFO:teuthology.orchestra.run.smithi097.stdout: "req_unlink_latency": { 2024-04-25T14:16:17.597 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:17.597 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:17.597 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:17.597 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.597 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_revoke_eviction": 0, 2024-04-25T14:16:17.598 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_acquisition_throttle": 0, 2024-04-25T14:16:17.598 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getvxattr_latency": { 2024-04-25T14:16:17.598 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:17.598 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:17.598 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:17.598 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T14:16:17.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.599 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_sessions": { 2024-04-25T14:16:17.599 INFO:teuthology.orchestra.run.smithi097.stdout: "session_count": 1, 2024-04-25T14:16:17.599 INFO:teuthology.orchestra.run.smithi097.stdout: "session_add": 2, 2024-04-25T14:16:17.599 INFO:teuthology.orchestra.run.smithi097.stdout: "session_remove": 1, 2024-04-25T14:16:17.599 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_open": 1, 2024-04-25T14:16:17.600 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_stale": 0, 2024-04-25T14:16:17.600 INFO:teuthology.orchestra.run.smithi097.stdout: "total_load": 0, 2024-04-25T14:16:17.600 INFO:teuthology.orchestra.run.smithi097.stdout: "average_load": 0, 2024-04-25T14:16:17.600 INFO:teuthology.orchestra.run.smithi097.stdout: "avg_session_uptime": 1294, 2024-04-25T14:16:17.600 INFO:teuthology.orchestra.run.smithi097.stdout: "mdthresh_evicted": 0 2024-04-25T14:16:17.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.601 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2024-04-25T14:16:17.601 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:16:17.601 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 0, 2024-04-25T14:16:17.601 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:16:17.601 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:16:17.601 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:16:17.601 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:16:17.602 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:16:17.602 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:16:17.602 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:16:17.602 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:16:17.602 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:16:17.603 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:16:17.603 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:16:17.603 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:16:17.603 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:16:17.603 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2024-04-25T14:16:17.603 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:16:17.603 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2024-04-25T14:16:17.604 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:16:17.604 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:16:17.604 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:16:17.604 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:16:17.604 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:16:17.604 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:16:17.604 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:16:17.605 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2024-04-25T14:16:17.605 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:16:17.605 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:16:17.605 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:16:17.605 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2024-04-25T14:16:17.605 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2024-04-25T14:16:17.606 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2024-04-25T14:16:17.606 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:16:17.606 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:16:17.606 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:16:17.606 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:16:17.606 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 189239, 2024-04-25T14:16:17.607 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 48, 2024-04-25T14:16:17.607 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 0, 2024-04-25T14:16:17.607 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 0, 2024-04-25T14:16:17.607 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2024-04-25T14:16:17.607 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2024-04-25T14:16:17.607 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:16:17.608 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2024-04-25T14:16:17.608 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:16:17.608 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2024-04-25T14:16:17.608 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 7720, 2024-04-25T14:16:17.608 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 162, 2024-04-25T14:16:17.608 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:16:17.609 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:16:17.609 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2024-04-25T14:16:17.609 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2024-04-25T14:16:17.609 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 8512, 2024-04-25T14:16:17.609 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 13, 2024-04-25T14:16:17.609 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2024-04-25T14:16:17.610 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2024-04-25T14:16:17.610 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2024-04-25T14:16:17.610 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2024-04-25T14:16:17.610 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.610 INFO:teuthology.orchestra.run.smithi097.stdout: "objecter": { 2024-04-25T14:16:17.610 INFO:teuthology.orchestra.run.smithi097.stdout: "op_active": 0, 2024-04-25T14:16:17.611 INFO:teuthology.orchestra.run.smithi097.stdout: "op_laggy": 0, 2024-04-25T14:16:17.611 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send": 778, 2024-04-25T14:16:17.611 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send_bytes": 21, 2024-04-25T14:16:17.611 INFO:teuthology.orchestra.run.smithi097.stdout: "op_resend": 3, 2024-04-25T14:16:17.611 INFO:teuthology.orchestra.run.smithi097.stdout: "op_reply": 778, 2024-04-25T14:16:17.611 INFO:teuthology.orchestra.run.smithi097.stdout: "op_latency": { 2024-04-25T14:16:17.611 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 778, 2024-04-25T14:16:17.612 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.340994122, 2024-04-25T14:16:17.612 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000438295 2024-04-25T14:16:17.612 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.612 INFO:teuthology.orchestra.run.smithi097.stdout: "op_inflight": 0, 2024-04-25T14:16:17.612 INFO:teuthology.orchestra.run.smithi097.stdout: "oplen_avg": { 2024-04-25T14:16:17.612 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 778, 2024-04-25T14:16:17.613 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 779 2024-04-25T14:16:17.613 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.613 INFO:teuthology.orchestra.run.smithi097.stdout: "op": 778, 2024-04-25T14:16:17.613 INFO:teuthology.orchestra.run.smithi097.stdout: "op_r": 778, 2024-04-25T14:16:17.613 INFO:teuthology.orchestra.run.smithi097.stdout: "op_w": 0, 2024-04-25T14:16:17.613 INFO:teuthology.orchestra.run.smithi097.stdout: "op_rmw": 0, 2024-04-25T14:16:17.614 INFO:teuthology.orchestra.run.smithi097.stdout: "op_pg": 0, 2024-04-25T14:16:17.614 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_stat": 488, 2024-04-25T14:16:17.614 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_create": 0, 2024-04-25T14:16:17.614 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_read": 288, 2024-04-25T14:16:17.614 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_write": 0, 2024-04-25T14:16:17.615 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writefull": 0, 2024-04-25T14:16:17.615 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writesame": 0, 2024-04-25T14:16:17.615 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_append": 0, 2024-04-25T14:16:17.615 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_zero": 0, 2024-04-25T14:16:17.615 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_truncate": 0, 2024-04-25T14:16:17.615 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_delete": 0, 2024-04-25T14:16:17.616 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_mapext": 0, 2024-04-25T14:16:17.616 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_sparse_read": 0, 2024-04-25T14:16:17.616 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_clonerange": 0, 2024-04-25T14:16:17.616 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_getxattr": 1, 2024-04-25T14:16:17.616 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_setxattr": 0, 2024-04-25T14:16:17.616 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_cmpxattr": 0, 2024-04-25T14:16:17.616 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_rmxattr": 0, 2024-04-25T14:16:17.617 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_resetxattrs": 0, 2024-04-25T14:16:17.617 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_call": 0, 2024-04-25T14:16:17.617 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_watch": 0, 2024-04-25T14:16:17.617 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_notify": 0, 2024-04-25T14:16:17.617 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T14:16:17.617 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls": 0, 2024-04-25T14:16:17.618 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls_filter": 0, 2024-04-25T14:16:17.618 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_other": 0, 2024-04-25T14:16:17.618 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_active": 0, 2024-04-25T14:16:17.618 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_send": 0, 2024-04-25T14:16:17.618 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_resend": 0, 2024-04-25T14:16:17.618 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_ping": 0, 2024-04-25T14:16:17.619 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_active": 0, 2024-04-25T14:16:17.619 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_send": 0, 2024-04-25T14:16:17.619 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_resend": 0, 2024-04-25T14:16:17.619 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_active": 0, 2024-04-25T14:16:17.619 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_send": 0, 2024-04-25T14:16:17.619 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_resend": 0, 2024-04-25T14:16:17.620 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_active": 0, 2024-04-25T14:16:17.620 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_send": 0, 2024-04-25T14:16:17.620 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_resend": 0, 2024-04-25T14:16:17.620 INFO:teuthology.orchestra.run.smithi097.stdout: "command_active": 0, 2024-04-25T14:16:17.620 INFO:teuthology.orchestra.run.smithi097.stdout: "command_send": 0, 2024-04-25T14:16:17.620 INFO:teuthology.orchestra.run.smithi097.stdout: "command_resend": 0, 2024-04-25T14:16:17.621 INFO:teuthology.orchestra.run.smithi097.stdout: "map_epoch": 78, 2024-04-25T14:16:17.621 INFO:teuthology.orchestra.run.smithi097.stdout: "map_full": 0, 2024-04-25T14:16:17.621 INFO:teuthology.orchestra.run.smithi097.stdout: "map_inc": 2, 2024-04-25T14:16:17.621 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_sessions": 5, 2024-04-25T14:16:17.621 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_open": 5, 2024-04-25T14:16:17.621 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_close": 0, 2024-04-25T14:16:17.622 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_laggy": 0, 2024-04-25T14:16:17.622 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_wr": 0, 2024-04-25T14:16:17.622 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_rd": 2, 2024-04-25T14:16:17.622 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_del": 0 2024-04-25T14:16:17.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.623 INFO:teuthology.orchestra.run.smithi097.stdout: "oft": { 2024-04-25T14:16:17.623 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_objs": 0, 2024-04-25T14:16:17.623 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_kv_pairs": 0, 2024-04-25T14:16:17.623 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_updates": 0, 2024-04-25T14:16:17.623 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_removes": 0 2024-04-25T14:16:17.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.624 INFO:teuthology.orchestra.run.smithi097.stdout: "osd-slow-ops": { 2024-04-25T14:16:17.624 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_ops_count": 0 2024-04-25T14:16:17.624 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.624 INFO:teuthology.orchestra.run.smithi097.stdout: "purge_queue": { 2024-04-25T14:16:17.624 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops": 0, 2024-04-25T14:16:17.624 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T14:16:17.625 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing": 0, 2024-04-25T14:16:17.625 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_high_water": 0, 2024-04-25T14:16:17.625 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed_ops": 0, 2024-04-25T14:16:17.626 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed": 0, 2024-04-25T14:16:17.626 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_item_in_journal": 0 2024-04-25T14:16:17.626 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.626 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:16:17.626 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T14:16:17.626 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-04-25T14:16:17.627 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T14:16:17.627 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 880, 2024-04-25T14:16:17.627 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 633379, 2024-04-25T14:16:17.627 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T14:16:17.627 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 880, 2024-04-25T14:16:17.628 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-25T14:16:17.628 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-25T14:16:17.628 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 880, 2024-04-25T14:16:17.628 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 633379, 2024-04-25T14:16:17.628 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T14:16:17.628 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:17.629 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:17.629 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:17.629 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T14:16:17.629 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.629 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_bytes": { 2024-04-25T14:16:17.629 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T14:16:17.630 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-04-25T14:16:17.630 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T14:16:17.630 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-25T14:16:17.630 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-25T14:16:17.630 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T14:16:17.631 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-25T14:16:17.631 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 778, 2024-04-25T14:16:17.631 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 1433, 2024-04-25T14:16:17.631 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 778, 2024-04-25T14:16:17.631 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 1433, 2024-04-25T14:16:17.631 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T14:16:17.632 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:17.632 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:17.632 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:17.632 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T14:16:17.632 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.632 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_ops": { 2024-04-25T14:16:17.633 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T14:16:17.633 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 1024, 2024-04-25T14:16:17.633 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T14:16:17.633 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-25T14:16:17.634 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-25T14:16:17.634 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T14:16:17.634 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-25T14:16:17.634 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 778, 2024-04-25T14:16:17.634 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 778, 2024-04-25T14:16:17.635 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 778, 2024-04-25T14:16:17.635 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 778, 2024-04-25T14:16:17.635 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T14:16:17.635 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:17.635 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:17.635 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:17.636 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T14:16:17.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.636 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle": { 2024-04-25T14:16:17.636 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T14:16:17.636 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-04-25T14:16:17.636 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T14:16:17.637 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-25T14:16:17.637 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-25T14:16:17.637 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T14:16:17.637 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-25T14:16:17.637 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-25T14:16:17.637 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-25T14:16:17.638 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 0, 2024-04-25T14:16:17.638 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 0, 2024-04-25T14:16:17.638 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T14:16:17.638 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:17.638 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:17.638 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:17.639 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T14:16:17.639 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T14:16:17.639 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle-0x55756ca4e750": { 2024-04-25T14:16:17.639 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T14:16:17.639 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-04-25T14:16:17.639 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T14:16:17.640 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-25T14:16:17.640 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-25T14:16:17.640 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T14:16:17.640 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-25T14:16:17.640 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-25T14:16:17.640 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-25T14:16:17.641 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 0, 2024-04-25T14:16:17.641 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 0, 2024-04-25T14:16:17.641 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T14:16:17.641 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T14:16:17.641 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T14:16:17.641 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T14:16:17.642 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T14:16:17.642 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T14:16:17.642 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T14:16:18.425 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-04-25T14:16:18.425 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-04-25T14:16:18.425 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-04-25T14:16:18.425 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-04-25T14:16:18.425 DEBUG:tasks.check_counter:Getting stats from c 2024-04-25T14:16:18.426 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-25T14:16:18.672 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:18 smithi097 ceph-mon[32720]: pgmap v426: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T14:16:18.678 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:16:18.800 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:18 smithi151 ceph-mon[32990]: pgmap v426: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T14:16:18.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:18 smithi096 ceph-mon[29207]: pgmap v426: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T14:16:20.565 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-25T14:16:20.565 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:16:20.565 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 791, 2024-04-25T14:16:20.565 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 793, 2024-04-25T14:16:20.565 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 227165, 2024-04-25T14:16:20.565 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 251545, 2024-04-25T14:16:20.565 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 6, 2024-04-25T14:16:20.565 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 2, 2024-04-25T14:16:20.566 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.074927302, 2024-04-25T14:16:20.566 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.024519907, 2024-04-25T14:16:20.566 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.019148753, 2024-04-25T14:16:20.566 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.024616402, 2024-04-25T14:16:20.566 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:16:20.566 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 791, 2024-04-25T14:16:20.566 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.032318077, 2024-04-25T14:16:20.566 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000040857 2024-04-25T14:16:20.566 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.566 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:16:20.566 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:20.566 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:20.566 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:20.567 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.567 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T14:16:20.567 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T14:16:20.567 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.567 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:16:20.567 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 107, 2024-04-25T14:16:20.567 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 96, 2024-04-25T14:16:20.567 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 523308, 2024-04-25T14:16:20.567 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 54074, 2024-04-25T14:16:20.567 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 3, 2024-04-25T14:16:20.567 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 2, 2024-04-25T14:16:20.567 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.018726784, 2024-04-25T14:16:20.567 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.005696409, 2024-04-25T14:16:20.568 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.013771626, 2024-04-25T14:16:20.568 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.002260113, 2024-04-25T14:16:20.568 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:16:20.568 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 95, 2024-04-25T14:16:20.568 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.006760184, 2024-04-25T14:16:20.568 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000071159 2024-04-25T14:16:20.568 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.568 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:16:20.568 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:20.568 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:20.568 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:20.568 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.568 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 522752, 2024-04-25T14:16:20.569 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 53376 2024-04-25T14:16:20.569 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.569 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:16:20.569 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 6, 2024-04-25T14:16:20.569 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 69, 2024-04-25T14:16:20.569 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 3420, 2024-04-25T14:16:20.569 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 193854, 2024-04-25T14:16:20.569 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 4, 2024-04-25T14:16:20.569 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 3, 2024-04-25T14:16:20.569 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.005008894, 2024-04-25T14:16:20.569 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.002646610, 2024-04-25T14:16:20.569 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.001788342, 2024-04-25T14:16:20.569 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.000053411, 2024-04-25T14:16:20.569 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:16:20.570 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 68, 2024-04-25T14:16:20.570 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.004644012, 2024-04-25T14:16:20.570 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000068294 2024-04-25T14:16:20.570 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.570 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:16:20.570 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:20.571 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:20.571 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:20.571 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.571 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-04-25T14:16:20.571 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 193536 2024-04-25T14:16:20.571 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.571 INFO:teuthology.orchestra.run.smithi151.stdout: "cct": { 2024-04-25T14:16:20.571 INFO:teuthology.orchestra.run.smithi151.stdout: "total_workers": 1, 2024-04-25T14:16:20.571 INFO:teuthology.orchestra.run.smithi151.stdout: "unhealthy_workers": 0 2024-04-25T14:16:20.571 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.571 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-MDSRank": { 2024-04-25T14:16:20.571 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-04-25T14:16:20.571 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-04-25T14:16:20.571 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 1372, 2024-04-25T14:16:20.572 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.041860785, 2024-04-25T14:16:20.572 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000030510 2024-04-25T14:16:20.572 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-25T14:16:20.572 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.572 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-PurgeQueue": { 2024-04-25T14:16:20.572 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-04-25T14:16:20.572 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-04-25T14:16:20.572 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:20.572 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:20.572 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:20.572 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-25T14:16:20.572 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.572 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": { 2024-04-25T14:16:20.572 INFO:teuthology.orchestra.run.smithi151.stdout: "request": 0, 2024-04-25T14:16:20.572 INFO:teuthology.orchestra.run.smithi151.stdout: "reply": 0, 2024-04-25T14:16:20.573 INFO:teuthology.orchestra.run.smithi151.stdout: "reply_latency": { 2024-04-25T14:16:20.573 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:20.573 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:20.573 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:20.573 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.573 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_reply": 0, 2024-04-25T14:16:20.573 INFO:teuthology.orchestra.run.smithi151.stdout: "forward": 0, 2024-04-25T14:16:20.573 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_complete": 0, 2024-04-25T14:16:20.573 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_keys": 0, 2024-04-25T14:16:20.573 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_commit": 0, 2024-04-25T14:16:20.573 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_split": 0, 2024-04-25T14:16:20.573 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_merge": 0, 2024-04-25T14:16:20.573 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes": 0, 2024-04-25T14:16:20.573 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_top": 0, 2024-04-25T14:16:20.574 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_bottom": 0, 2024-04-25T14:16:20.574 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pin_tail": 0, 2024-04-25T14:16:20.574 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pinned": 0, 2024-04-25T14:16:20.574 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_expired": 0, 2024-04-25T14:16:20.574 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_with_caps": 0, 2024-04-25T14:16:20.574 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": 0, 2024-04-25T14:16:20.574 INFO:teuthology.orchestra.run.smithi151.stdout: "subtrees": 1, 2024-04-25T14:16:20.574 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse": 0, 2024-04-25T14:16:20.574 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_hit": 0, 2024-04-25T14:16:20.574 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_forward": 0, 2024-04-25T14:16:20.574 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_discover": 0, 2024-04-25T14:16:20.574 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_dir_fetch": 0, 2024-04-25T14:16:20.574 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_remote_ino": 0, 2024-04-25T14:16:20.574 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_lock": 0, 2024-04-25T14:16:20.575 INFO:teuthology.orchestra.run.smithi151.stdout: "load_cent": 0, 2024-04-25T14:16:20.575 INFO:teuthology.orchestra.run.smithi151.stdout: "q": 0, 2024-04-25T14:16:20.575 INFO:teuthology.orchestra.run.smithi151.stdout: "exported": 0, 2024-04-25T14:16:20.575 INFO:teuthology.orchestra.run.smithi151.stdout: "exported_inodes": 0, 2024-04-25T14:16:20.575 INFO:teuthology.orchestra.run.smithi151.stdout: "imported": 0, 2024-04-25T14:16:20.575 INFO:teuthology.orchestra.run.smithi151.stdout: "imported_inodes": 0, 2024-04-25T14:16:20.575 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_dir_fetch": 0, 2024-04-25T14:16:20.575 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_backtrace_fetch": 0, 2024-04-25T14:16:20.575 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_peer_discover": 0, 2024-04-25T14:16:20.575 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rfiles": 0, 2024-04-25T14:16:20.575 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rbytes": 0, 2024-04-25T14:16:20.575 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rsnaps": 0, 2024-04-25T14:16:20.575 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T14:16:20.575 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_set_tag": 0, 2024-04-25T14:16:20.576 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T14:16:20.576 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_inotable_repaired": 0, 2024-04-25T14:16:20.576 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_inodes": 0, 2024-04-25T14:16:20.576 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T14:16:20.576 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T14:16:20.576 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_file_inodes": 0, 2024-04-25T14:16:20.576 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_inode_file_caps": 0, 2024-04-25T14:16:20.576 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T14:16:20.576 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_grant": 0, 2024-04-25T14:16:20.576 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T14:16:20.576 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T14:16:20.576 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T14:16:20.577 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps": 0, 2024-04-25T14:16:20.577 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps_dirty": 0, 2024-04-25T14:16:20.577 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_cap_release": 0, 2024-04-25T14:16:20.577 INFO:teuthology.orchestra.run.smithi151.stdout: "process_request_cap_release": 0 2024-04-25T14:16:20.577 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.577 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_cache": { 2024-04-25T14:16:20.577 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update": 0, 2024-04-25T14:16:20.577 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update_receipt": 0, 2024-04-25T14:16:20.577 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_try_discover": 0, 2024-04-25T14:16:20.577 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_send_discover": 0, 2024-04-25T14:16:20.577 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_handle_discover": 0, 2024-04-25T14:16:20.577 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays": 0, 2024-04-25T14:16:20.577 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_delayed": 0, 2024-04-25T14:16:20.578 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_enqueuing": 0, 2024-04-25T14:16:20.578 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_created": 0, 2024-04-25T14:16:20.578 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_enqueued": 0, 2024-04-25T14:16:20.578 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_reintegrated": 0, 2024-04-25T14:16:20.578 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_migrated": 0, 2024-04-25T14:16:20.578 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_processing": 0, 2024-04-25T14:16:20.578 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_enqueued": 0, 2024-04-25T14:16:20.578 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_prioritized": 0, 2024-04-25T14:16:20.578 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_started": 0, 2024-04-25T14:16:20.578 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_completed": 0, 2024-04-25T14:16:20.578 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_path": 0, 2024-04-25T14:16:20.578 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_inode": 0, 2024-04-25T14:16:20.578 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T14:16:20.578 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_exportdir": 0, 2024-04-25T14:16:20.579 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_flush": 0, 2024-04-25T14:16:20.579 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragmentdir": 0, 2024-04-25T14:16:20.579 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragstats": 0, 2024-04-25T14:16:20.579 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_inodestats": 0 2024-04-25T14:16:20.579 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.579 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_log": { 2024-04-25T14:16:20.579 INFO:teuthology.orchestra.run.smithi151.stdout: "evlrg": 0, 2024-04-25T14:16:20.579 INFO:teuthology.orchestra.run.smithi151.stdout: "evadd": 0, 2024-04-25T14:16:20.579 INFO:teuthology.orchestra.run.smithi151.stdout: "evex": 0, 2024-04-25T14:16:20.579 INFO:teuthology.orchestra.run.smithi151.stdout: "evtrm": 0, 2024-04-25T14:16:20.579 INFO:teuthology.orchestra.run.smithi151.stdout: "ev": 5, 2024-04-25T14:16:20.579 INFO:teuthology.orchestra.run.smithi151.stdout: "evexg": 0, 2024-04-25T14:16:20.579 INFO:teuthology.orchestra.run.smithi151.stdout: "evexd": 0, 2024-04-25T14:16:20.579 INFO:teuthology.orchestra.run.smithi151.stdout: "segadd": 0, 2024-04-25T14:16:20.580 INFO:teuthology.orchestra.run.smithi151.stdout: "segex": 0, 2024-04-25T14:16:20.580 INFO:teuthology.orchestra.run.smithi151.stdout: "segtrm": 0, 2024-04-25T14:16:20.580 INFO:teuthology.orchestra.run.smithi151.stdout: "seg": 2, 2024-04-25T14:16:20.580 INFO:teuthology.orchestra.run.smithi151.stdout: "segmjr": 2, 2024-04-25T14:16:20.580 INFO:teuthology.orchestra.run.smithi151.stdout: "segexg": 0, 2024-04-25T14:16:20.580 INFO:teuthology.orchestra.run.smithi151.stdout: "segexd": 0, 2024-04-25T14:16:20.580 INFO:teuthology.orchestra.run.smithi151.stdout: "expos": 4194304, 2024-04-25T14:16:20.580 INFO:teuthology.orchestra.run.smithi151.stdout: "wrpos": 4195732, 2024-04-25T14:16:20.580 INFO:teuthology.orchestra.run.smithi151.stdout: "rdpos": 4195424, 2024-04-25T14:16:20.580 INFO:teuthology.orchestra.run.smithi151.stdout: "jlat": { 2024-04-25T14:16:20.580 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:20.580 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:20.580 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:20.580 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.581 INFO:teuthology.orchestra.run.smithi151.stdout: "replayed": 5 2024-04-25T14:16:20.581 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.581 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_mem": { 2024-04-25T14:16:20.581 INFO:teuthology.orchestra.run.smithi151.stdout: "ino": 3, 2024-04-25T14:16:20.581 INFO:teuthology.orchestra.run.smithi151.stdout: "ino+": 3, 2024-04-25T14:16:20.581 INFO:teuthology.orchestra.run.smithi151.stdout: "ino-": 0, 2024-04-25T14:16:20.581 INFO:teuthology.orchestra.run.smithi151.stdout: "dir": 1, 2024-04-25T14:16:20.581 INFO:teuthology.orchestra.run.smithi151.stdout: "dir+": 1, 2024-04-25T14:16:20.581 INFO:teuthology.orchestra.run.smithi151.stdout: "dir-": 0, 2024-04-25T14:16:20.581 INFO:teuthology.orchestra.run.smithi151.stdout: "dn": 0, 2024-04-25T14:16:20.581 INFO:teuthology.orchestra.run.smithi151.stdout: "dn+": 0, 2024-04-25T14:16:20.581 INFO:teuthology.orchestra.run.smithi151.stdout: "dn-": 0, 2024-04-25T14:16:20.581 INFO:teuthology.orchestra.run.smithi151.stdout: "cap": 0, 2024-04-25T14:16:20.581 INFO:teuthology.orchestra.run.smithi151.stdout: "cap+": 0, 2024-04-25T14:16:20.582 INFO:teuthology.orchestra.run.smithi151.stdout: "cap-": 0, 2024-04-25T14:16:20.582 INFO:teuthology.orchestra.run.smithi151.stdout: "rss": 121440, 2024-04-25T14:16:20.582 INFO:teuthology.orchestra.run.smithi151.stdout: "heap": 215440 2024-04-25T14:16:20.582 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.582 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_server": { 2024-04-25T14:16:20.582 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_client_request": 0, 2024-04-25T14:16:20.582 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_server_request": 0, 2024-04-25T14:16:20.582 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_request": 0, 2024-04-25T14:16:20.582 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_session": 0, 2024-04-25T14:16:20.582 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_peer_request": 0, 2024-04-25T14:16:20.582 INFO:teuthology.orchestra.run.smithi151.stdout: "req_create_latency": { 2024-04-25T14:16:20.582 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:20.583 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:20.583 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:20.583 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.583 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getattr_latency": { 2024-04-25T14:16:20.583 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:20.583 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:20.583 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:20.583 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.583 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getfilelock_latency": { 2024-04-25T14:16:20.583 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:20.583 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:20.583 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:20.583 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.584 INFO:teuthology.orchestra.run.smithi151.stdout: "req_link_latency": { 2024-04-25T14:16:20.584 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:20.584 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:20.584 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:20.584 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.584 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookup_latency": { 2024-04-25T14:16:20.584 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:20.584 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:20.584 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:20.584 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.584 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookuphash_latency": { 2024-04-25T14:16:20.584 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:20.584 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:20.584 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:20.585 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.585 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupino_latency": { 2024-04-25T14:16:20.585 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:20.585 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:20.585 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:20.585 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.585 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupname_latency": { 2024-04-25T14:16:20.585 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:20.585 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:20.585 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:20.585 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.585 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupparent_latency": { 2024-04-25T14:16:20.585 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:20.586 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:20.586 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:20.586 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.586 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupsnap_latency": { 2024-04-25T14:16:20.586 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:20.586 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:20.586 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:20.586 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.586 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lssnap_latency": { 2024-04-25T14:16:20.586 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:20.586 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:20.586 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:20.586 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.586 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mkdir_latency": { 2024-04-25T14:16:20.586 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:20.587 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:20.587 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:20.587 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.588 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mknod_latency": { 2024-04-25T14:16:20.588 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:20.588 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:20.588 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:20.588 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.588 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mksnap_latency": { 2024-04-25T14:16:20.588 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:20.588 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:20.588 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:20.588 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.588 INFO:teuthology.orchestra.run.smithi151.stdout: "req_open_latency": { 2024-04-25T14:16:20.588 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:20.588 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:20.588 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:20.589 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.589 INFO:teuthology.orchestra.run.smithi151.stdout: "req_readdir_latency": { 2024-04-25T14:16:20.589 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:20.589 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:20.589 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:20.589 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.589 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rename_latency": { 2024-04-25T14:16:20.589 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:20.589 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:20.589 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:20.589 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.589 INFO:teuthology.orchestra.run.smithi151.stdout: "req_renamesnap_latency": { 2024-04-25T14:16:20.589 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:20.589 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:20.589 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:20.590 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.590 INFO:teuthology.orchestra.run.smithi151.stdout: "req_snapdiff_latency": { 2024-04-25T14:16:20.590 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:20.590 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:20.590 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:20.590 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.590 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmdir_latency": { 2024-04-25T14:16:20.590 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:20.590 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:20.590 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:20.590 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.590 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmsnap_latency": { 2024-04-25T14:16:20.590 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:20.590 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:20.591 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:20.591 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.591 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmxattr_latency": { 2024-04-25T14:16:20.591 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:20.591 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:20.591 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:20.591 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.591 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setattr_latency": { 2024-04-25T14:16:20.591 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:20.591 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:20.591 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:20.591 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.591 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setdirlayout_latency": { 2024-04-25T14:16:20.591 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:20.591 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:20.592 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:20.592 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.592 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setfilelock_latency": { 2024-04-25T14:16:20.592 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:20.592 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:20.592 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:20.592 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.592 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setlayout_latency": { 2024-04-25T14:16:20.592 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:20.592 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:20.592 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:20.592 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.592 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setxattr_latency": { 2024-04-25T14:16:20.592 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:20.593 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:20.593 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:20.593 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.593 INFO:teuthology.orchestra.run.smithi151.stdout: "req_symlink_latency": { 2024-04-25T14:16:20.593 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:20.593 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:20.593 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:20.593 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.593 INFO:teuthology.orchestra.run.smithi151.stdout: "req_unlink_latency": { 2024-04-25T14:16:20.593 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:20.593 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:20.593 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:20.593 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.593 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_revoke_eviction": 0, 2024-04-25T14:16:20.593 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_acquisition_throttle": 0, 2024-04-25T14:16:20.594 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getvxattr_latency": { 2024-04-25T14:16:20.594 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:20.594 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:20.594 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:20.594 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-25T14:16:20.594 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.594 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_sessions": { 2024-04-25T14:16:20.594 INFO:teuthology.orchestra.run.smithi151.stdout: "session_count": 1, 2024-04-25T14:16:20.594 INFO:teuthology.orchestra.run.smithi151.stdout: "session_add": 2, 2024-04-25T14:16:20.594 INFO:teuthology.orchestra.run.smithi151.stdout: "session_remove": 1, 2024-04-25T14:16:20.594 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_open": 1, 2024-04-25T14:16:20.594 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_stale": 0, 2024-04-25T14:16:20.594 INFO:teuthology.orchestra.run.smithi151.stdout: "total_load": 0, 2024-04-25T14:16:20.594 INFO:teuthology.orchestra.run.smithi151.stdout: "average_load": 0, 2024-04-25T14:16:20.595 INFO:teuthology.orchestra.run.smithi151.stdout: "avg_session_uptime": 1286, 2024-04-25T14:16:20.595 INFO:teuthology.orchestra.run.smithi151.stdout: "mdthresh_evicted": 0 2024-04-25T14:16:20.595 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.595 INFO:teuthology.orchestra.run.smithi151.stdout: "mempool": { 2024-04-25T14:16:20.595 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:16:20.595 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_items": 0, 2024-04-25T14:16:20.595 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:16:20.595 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:16:20.595 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:16:20.595 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:16:20.595 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:16:20.595 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:16:20.595 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:16:20.595 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:16:20.596 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:16:20.596 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:16:20.596 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:16:20.596 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:16:20.596 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:16:20.596 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_items": 0, 2024-04-25T14:16:20.596 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:16:20.596 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_items": 0, 2024-04-25T14:16:20.596 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:16:20.596 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:16:20.596 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:16:20.596 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:16:20.597 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:16:20.597 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:16:20.597 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:16:20.597 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_items": 0, 2024-04-25T14:16:20.597 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:16:20.597 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:16:20.597 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:16:20.597 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_items": 0, 2024-04-25T14:16:20.597 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_bytes": 0, 2024-04-25T14:16:20.598 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_items": 0, 2024-04-25T14:16:20.598 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:16:20.598 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:16:20.598 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:16:20.598 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:16:20.598 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_bytes": 181159, 2024-04-25T14:16:20.598 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_items": 44, 2024-04-25T14:16:20.598 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_bytes": 0, 2024-04-25T14:16:20.598 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_items": 0, 2024-04-25T14:16:20.598 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_bytes": 0, 2024-04-25T14:16:20.598 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_items": 0, 2024-04-25T14:16:20.598 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:16:20.598 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_items": 0, 2024-04-25T14:16:20.598 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:16:20.599 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_items": 0, 2024-04-25T14:16:20.599 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_bytes": 7720, 2024-04-25T14:16:20.599 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_items": 162, 2024-04-25T14:16:20.599 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:16:20.599 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:16:20.599 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_bytes": 0, 2024-04-25T14:16:20.599 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_items": 0, 2024-04-25T14:16:20.599 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_bytes": 8512, 2024-04-25T14:16:20.599 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_items": 13, 2024-04-25T14:16:20.599 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_bytes": 0, 2024-04-25T14:16:20.599 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_items": 0, 2024-04-25T14:16:20.599 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_bytes": 0, 2024-04-25T14:16:20.599 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_items": 0 2024-04-25T14:16:20.599 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.600 INFO:teuthology.orchestra.run.smithi151.stdout: "objecter": { 2024-04-25T14:16:20.600 INFO:teuthology.orchestra.run.smithi151.stdout: "op_active": 0, 2024-04-25T14:16:20.600 INFO:teuthology.orchestra.run.smithi151.stdout: "op_laggy": 0, 2024-04-25T14:16:20.600 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send": 794, 2024-04-25T14:16:20.600 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send_bytes": 21, 2024-04-25T14:16:20.600 INFO:teuthology.orchestra.run.smithi151.stdout: "op_resend": 3, 2024-04-25T14:16:20.600 INFO:teuthology.orchestra.run.smithi151.stdout: "op_reply": 794, 2024-04-25T14:16:20.600 INFO:teuthology.orchestra.run.smithi151.stdout: "op_latency": { 2024-04-25T14:16:20.600 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 794, 2024-04-25T14:16:20.600 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.382994726, 2024-04-25T14:16:20.600 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000482361 2024-04-25T14:16:20.600 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.600 INFO:teuthology.orchestra.run.smithi151.stdout: "op_inflight": 0, 2024-04-25T14:16:20.601 INFO:teuthology.orchestra.run.smithi151.stdout: "oplen_avg": { 2024-04-25T14:16:20.601 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 794, 2024-04-25T14:16:20.601 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 795 2024-04-25T14:16:20.601 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.601 INFO:teuthology.orchestra.run.smithi151.stdout: "op": 794, 2024-04-25T14:16:20.601 INFO:teuthology.orchestra.run.smithi151.stdout: "op_r": 794, 2024-04-25T14:16:20.601 INFO:teuthology.orchestra.run.smithi151.stdout: "op_w": 0, 2024-04-25T14:16:20.601 INFO:teuthology.orchestra.run.smithi151.stdout: "op_rmw": 0, 2024-04-25T14:16:20.601 INFO:teuthology.orchestra.run.smithi151.stdout: "op_pg": 0, 2024-04-25T14:16:20.601 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_stat": 497, 2024-04-25T14:16:20.601 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_create": 0, 2024-04-25T14:16:20.601 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_read": 295, 2024-04-25T14:16:20.601 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_write": 0, 2024-04-25T14:16:20.602 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writefull": 0, 2024-04-25T14:16:20.602 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writesame": 0, 2024-04-25T14:16:20.602 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_append": 0, 2024-04-25T14:16:20.602 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_zero": 0, 2024-04-25T14:16:20.602 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_truncate": 0, 2024-04-25T14:16:20.602 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_delete": 0, 2024-04-25T14:16:20.602 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_mapext": 0, 2024-04-25T14:16:20.602 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_sparse_read": 0, 2024-04-25T14:16:20.602 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_clonerange": 0, 2024-04-25T14:16:20.602 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_getxattr": 1, 2024-04-25T14:16:20.602 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_setxattr": 0, 2024-04-25T14:16:20.602 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_cmpxattr": 0, 2024-04-25T14:16:20.602 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_rmxattr": 0, 2024-04-25T14:16:20.603 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_resetxattrs": 0, 2024-04-25T14:16:20.603 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_call": 0, 2024-04-25T14:16:20.603 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_watch": 0, 2024-04-25T14:16:20.603 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_notify": 0, 2024-04-25T14:16:20.603 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T14:16:20.603 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls": 0, 2024-04-25T14:16:20.603 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls_filter": 0, 2024-04-25T14:16:20.603 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_other": 0, 2024-04-25T14:16:20.603 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_active": 0, 2024-04-25T14:16:20.603 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_send": 0, 2024-04-25T14:16:20.603 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_resend": 0, 2024-04-25T14:16:20.603 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_ping": 0, 2024-04-25T14:16:20.603 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_active": 0, 2024-04-25T14:16:20.603 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_send": 0, 2024-04-25T14:16:20.603 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_resend": 0, 2024-04-25T14:16:20.604 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_active": 0, 2024-04-25T14:16:20.604 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_send": 0, 2024-04-25T14:16:20.604 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_resend": 0, 2024-04-25T14:16:20.604 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_active": 0, 2024-04-25T14:16:20.604 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_send": 0, 2024-04-25T14:16:20.604 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_resend": 0, 2024-04-25T14:16:20.604 INFO:teuthology.orchestra.run.smithi151.stdout: "command_active": 0, 2024-04-25T14:16:20.604 INFO:teuthology.orchestra.run.smithi151.stdout: "command_send": 0, 2024-04-25T14:16:20.604 INFO:teuthology.orchestra.run.smithi151.stdout: "command_resend": 0, 2024-04-25T14:16:20.604 INFO:teuthology.orchestra.run.smithi151.stdout: "map_epoch": 78, 2024-04-25T14:16:20.604 INFO:teuthology.orchestra.run.smithi151.stdout: "map_full": 0, 2024-04-25T14:16:20.604 INFO:teuthology.orchestra.run.smithi151.stdout: "map_inc": 2, 2024-04-25T14:16:20.604 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_sessions": 4, 2024-04-25T14:16:20.604 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_open": 4, 2024-04-25T14:16:20.604 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_close": 0, 2024-04-25T14:16:20.605 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_laggy": 0, 2024-04-25T14:16:20.605 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_wr": 0, 2024-04-25T14:16:20.605 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_rd": 2, 2024-04-25T14:16:20.605 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_del": 0 2024-04-25T14:16:20.605 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.605 INFO:teuthology.orchestra.run.smithi151.stdout: "oft": { 2024-04-25T14:16:20.605 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_objs": 0, 2024-04-25T14:16:20.605 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_kv_pairs": 0, 2024-04-25T14:16:20.605 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_updates": 0, 2024-04-25T14:16:20.605 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_removes": 0 2024-04-25T14:16:20.605 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.605 INFO:teuthology.orchestra.run.smithi151.stdout: "osd-slow-ops": { 2024-04-25T14:16:20.605 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_ops_count": 0 2024-04-25T14:16:20.606 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.606 INFO:teuthology.orchestra.run.smithi151.stdout: "purge_queue": { 2024-04-25T14:16:20.606 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops": 0, 2024-04-25T14:16:20.606 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T14:16:20.606 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing": 0, 2024-04-25T14:16:20.606 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_high_water": 0, 2024-04-25T14:16:20.606 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed_ops": 0, 2024-04-25T14:16:20.606 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed": 0, 2024-04-25T14:16:20.606 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_item_in_journal": 0 2024-04-25T14:16:20.606 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.606 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:16:20.606 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-04-25T14:16:20.606 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-04-25T14:16:20.606 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-04-25T14:16:20.607 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 900, 2024-04-25T14:16:20.607 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 664558, 2024-04-25T14:16:20.607 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-04-25T14:16:20.607 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 900, 2024-04-25T14:16:20.607 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-04-25T14:16:20.607 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-04-25T14:16:20.607 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 900, 2024-04-25T14:16:20.607 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 664558, 2024-04-25T14:16:20.607 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-04-25T14:16:20.607 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:20.607 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:20.607 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:20.607 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-25T14:16:20.607 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.607 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_bytes": { 2024-04-25T14:16:20.608 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-04-25T14:16:20.608 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-04-25T14:16:20.608 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-04-25T14:16:20.608 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-04-25T14:16:20.608 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-04-25T14:16:20.608 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-04-25T14:16:20.608 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-04-25T14:16:20.608 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 794, 2024-04-25T14:16:20.608 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 1433, 2024-04-25T14:16:20.608 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 794, 2024-04-25T14:16:20.608 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 1433, 2024-04-25T14:16:20.608 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-04-25T14:16:20.608 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:20.608 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:20.609 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:20.609 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-25T14:16:20.609 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.609 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_ops": { 2024-04-25T14:16:20.609 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-04-25T14:16:20.609 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 1024, 2024-04-25T14:16:20.609 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-04-25T14:16:20.609 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-04-25T14:16:20.609 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-04-25T14:16:20.609 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-04-25T14:16:20.609 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-04-25T14:16:20.609 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 794, 2024-04-25T14:16:20.609 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 794, 2024-04-25T14:16:20.609 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 794, 2024-04-25T14:16:20.609 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 794, 2024-04-25T14:16:20.610 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-04-25T14:16:20.610 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:20.610 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:20.610 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:20.610 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-25T14:16:20.610 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.610 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle": { 2024-04-25T14:16:20.610 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-04-25T14:16:20.610 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-04-25T14:16:20.610 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-04-25T14:16:20.610 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-04-25T14:16:20.610 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-04-25T14:16:20.610 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-04-25T14:16:20.610 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-04-25T14:16:20.611 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-04-25T14:16:20.611 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-04-25T14:16:20.611 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 0, 2024-04-25T14:16:20.611 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 0, 2024-04-25T14:16:20.611 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-04-25T14:16:20.611 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:20.611 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:20.611 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:20.611 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-25T14:16:20.611 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:20.611 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle-0x55fa3c57ec30": { 2024-04-25T14:16:20.611 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-04-25T14:16:20.611 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-04-25T14:16:20.611 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-04-25T14:16:20.611 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-04-25T14:16:20.612 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-04-25T14:16:20.612 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-04-25T14:16:20.612 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-04-25T14:16:20.612 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-04-25T14:16:20.612 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-04-25T14:16:20.612 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 0, 2024-04-25T14:16:20.612 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 0, 2024-04-25T14:16:20.612 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-04-25T14:16:20.612 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:20.612 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:20.612 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:20.612 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-25T14:16:20.612 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-25T14:16:20.612 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-25T14:16:20.673 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:20 smithi097 ceph-mon[32720]: pgmap v427: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T14:16:20.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:20 smithi096 ceph-mon[29207]: pgmap v427: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T14:16:20.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:20 smithi151 ceph-mon[32990]: pgmap v427: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T14:16:21.468 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-25T14:16:21.468 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-25T14:16:21.468 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-04-25T14:16:21.468 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-04-25T14:16:21.468 DEBUG:tasks.check_counter:Getting stats from f 2024-04-25T14:16:21.469 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-25T14:16:21.717 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:16:22.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:22 smithi096 ceph-mon[29207]: pgmap v428: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T14:16:22.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:22 smithi097 ceph-mon[32720]: pgmap v428: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T14:16:22.931 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:22 smithi151 ceph-mon[32990]: pgmap v428: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T14:16:23.539 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-25T14:16:23.539 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:16:23.539 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 831, 2024-04-25T14:16:23.539 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 549, 2024-04-25T14:16:23.540 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 256767, 2024-04-25T14:16:23.540 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 157924, 2024-04-25T14:16:23.540 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 10, 2024-04-25T14:16:23.540 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 4, 2024-04-25T14:16:23.540 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.065361713, 2024-04-25T14:16:23.540 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.023032856, 2024-04-25T14:16:23.540 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.046516473, 2024-04-25T14:16:23.540 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.002230996, 2024-04-25T14:16:23.540 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:16:23.540 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 549, 2024-04-25T14:16:23.540 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.038431277, 2024-04-25T14:16:23.540 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000070002 2024-04-25T14:16:23.540 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.541 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:16:23.541 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 1324, 2024-04-25T14:16:23.541 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000324717, 2024-04-25T14:16:23.541 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000245 2024-04-25T14:16:23.541 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.541 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 1216, 2024-04-25T14:16:23.541 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 1712 2024-04-25T14:16:23.541 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.541 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:16:23.541 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 44, 2024-04-25T14:16:23.541 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 108, 2024-04-25T14:16:23.541 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 19110, 2024-04-25T14:16:23.541 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 336053, 2024-04-25T14:16:23.541 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 18, 2024-04-25T14:16:23.541 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 6, 2024-04-25T14:16:23.542 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.012576994, 2024-04-25T14:16:23.542 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.003795072, 2024-04-25T14:16:23.542 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.007409089, 2024-04-25T14:16:23.542 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.000529789, 2024-04-25T14:16:23.542 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:16:23.542 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 107, 2024-04-25T14:16:23.542 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.027521898, 2024-04-25T14:16:23.542 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000257214 2024-04-25T14:16:23.542 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.542 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:16:23.542 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 34, 2024-04-25T14:16:23.542 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000008255, 2024-04-25T14:16:23.542 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000242 2024-04-25T14:16:23.542 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.543 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-04-25T14:16:23.543 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 193712 2024-04-25T14:16:23.543 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.543 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:16:23.543 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 179, 2024-04-25T14:16:23.543 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 171, 2024-04-25T14:16:23.543 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 549360, 2024-04-25T14:16:23.543 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 132598, 2024-04-25T14:16:23.543 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 13, 2024-04-25T14:16:23.543 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 4, 2024-04-25T14:16:23.543 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.026365130, 2024-04-25T14:16:23.543 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.007487586, 2024-04-25T14:16:23.543 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.020958484, 2024-04-25T14:16:23.543 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.002401297, 2024-04-25T14:16:23.544 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:16:23.544 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 168, 2024-04-25T14:16:23.544 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.028737146, 2024-04-25T14:16:23.544 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000171054 2024-04-25T14:16:23.544 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.544 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:16:23.544 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 65, 2024-04-25T14:16:23.544 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000013846, 2024-04-25T14:16:23.544 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000213 2024-04-25T14:16:23.544 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.544 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 526528, 2024-04-25T14:16:23.544 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 60896 2024-04-25T14:16:23.544 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.544 INFO:teuthology.orchestra.run.smithi151.stdout: "cct": { 2024-04-25T14:16:23.545 INFO:teuthology.orchestra.run.smithi151.stdout: "total_workers": 1, 2024-04-25T14:16:23.545 INFO:teuthology.orchestra.run.smithi151.stdout: "unhealthy_workers": 0 2024-04-25T14:16:23.545 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.545 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-MDSRank": { 2024-04-25T14:16:23.545 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-04-25T14:16:23.545 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-04-25T14:16:23.545 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 50, 2024-04-25T14:16:23.545 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.002199197, 2024-04-25T14:16:23.545 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000043983 2024-04-25T14:16:23.545 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-25T14:16:23.545 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.545 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-PurgeQueue": { 2024-04-25T14:16:23.545 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-04-25T14:16:23.546 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-04-25T14:16:23.546 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 2, 2024-04-25T14:16:23.546 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000010275, 2024-04-25T14:16:23.546 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000005137 2024-04-25T14:16:23.546 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-25T14:16:23.546 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.546 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": { 2024-04-25T14:16:23.546 INFO:teuthology.orchestra.run.smithi151.stdout: "request": 4, 2024-04-25T14:16:23.546 INFO:teuthology.orchestra.run.smithi151.stdout: "reply": 3, 2024-04-25T14:16:23.546 INFO:teuthology.orchestra.run.smithi151.stdout: "reply_latency": { 2024-04-25T14:16:23.546 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 3, 2024-04-25T14:16:23.546 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000470016, 2024-04-25T14:16:23.546 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000156672 2024-04-25T14:16:23.546 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.547 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_reply": 0, 2024-04-25T14:16:23.547 INFO:teuthology.orchestra.run.smithi151.stdout: "forward": 1, 2024-04-25T14:16:23.547 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_complete": 0, 2024-04-25T14:16:23.547 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_keys": 0, 2024-04-25T14:16:23.547 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_commit": 11, 2024-04-25T14:16:23.547 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_split": 0, 2024-04-25T14:16:23.547 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_merge": 0, 2024-04-25T14:16:23.547 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes": 18, 2024-04-25T14:16:23.547 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_top": 2, 2024-04-25T14:16:23.547 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_bottom": 16, 2024-04-25T14:16:23.547 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pin_tail": 0, 2024-04-25T14:16:23.547 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pinned": 13, 2024-04-25T14:16:23.547 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_expired": 8, 2024-04-25T14:16:23.547 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_with_caps": 2, 2024-04-25T14:16:23.547 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": 2, 2024-04-25T14:16:23.547 INFO:teuthology.orchestra.run.smithi151.stdout: "subtrees": 3, 2024-04-25T14:16:23.548 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse": 12, 2024-04-25T14:16:23.548 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_hit": 7, 2024-04-25T14:16:23.548 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_forward": 0, 2024-04-25T14:16:23.548 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_discover": 4, 2024-04-25T14:16:23.548 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_dir_fetch": 0, 2024-04-25T14:16:23.548 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_remote_ino": 0, 2024-04-25T14:16:23.548 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_lock": 0, 2024-04-25T14:16:23.548 INFO:teuthology.orchestra.run.smithi151.stdout: "load_cent": 1000, 2024-04-25T14:16:23.548 INFO:teuthology.orchestra.run.smithi151.stdout: "q": 1, 2024-04-25T14:16:23.548 INFO:teuthology.orchestra.run.smithi151.stdout: "exported": 0, 2024-04-25T14:16:23.548 INFO:teuthology.orchestra.run.smithi151.stdout: "exported_inodes": 0, 2024-04-25T14:16:23.548 INFO:teuthology.orchestra.run.smithi151.stdout: "imported": 0, 2024-04-25T14:16:23.548 INFO:teuthology.orchestra.run.smithi151.stdout: "imported_inodes": 0, 2024-04-25T14:16:23.548 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_dir_fetch": 0, 2024-04-25T14:16:23.548 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_backtrace_fetch": 0, 2024-04-25T14:16:23.549 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_peer_discover": 0, 2024-04-25T14:16:23.549 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rfiles": 4, 2024-04-25T14:16:23.549 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rbytes": 234, 2024-04-25T14:16:23.549 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rsnaps": 0, 2024-04-25T14:16:23.549 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T14:16:23.549 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_set_tag": 0, 2024-04-25T14:16:23.549 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T14:16:23.549 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_inotable_repaired": 0, 2024-04-25T14:16:23.549 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_inodes": 0, 2024-04-25T14:16:23.549 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T14:16:23.549 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T14:16:23.549 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_file_inodes": 0, 2024-04-25T14:16:23.549 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_inode_file_caps": 0, 2024-04-25T14:16:23.549 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T14:16:23.549 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_grant": 0, 2024-04-25T14:16:23.549 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T14:16:23.550 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T14:16:23.550 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T14:16:23.550 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps": 0, 2024-04-25T14:16:23.550 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps_dirty": 0, 2024-04-25T14:16:23.550 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_cap_release": 0, 2024-04-25T14:16:23.550 INFO:teuthology.orchestra.run.smithi151.stdout: "process_request_cap_release": 1 2024-04-25T14:16:23.550 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.550 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_cache": { 2024-04-25T14:16:23.550 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update": 0, 2024-04-25T14:16:23.550 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update_receipt": 13, 2024-04-25T14:16:23.550 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_try_discover": 8, 2024-04-25T14:16:23.550 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_send_discover": 10, 2024-04-25T14:16:23.550 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_handle_discover": 0, 2024-04-25T14:16:23.550 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays": 0, 2024-04-25T14:16:23.551 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_delayed": 0, 2024-04-25T14:16:23.551 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_enqueuing": 0, 2024-04-25T14:16:23.551 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_created": 0, 2024-04-25T14:16:23.551 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_enqueued": 0, 2024-04-25T14:16:23.551 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_reintegrated": 0, 2024-04-25T14:16:23.551 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_migrated": 0, 2024-04-25T14:16:23.551 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_processing": 0, 2024-04-25T14:16:23.551 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_enqueued": 0, 2024-04-25T14:16:23.551 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_prioritized": 0, 2024-04-25T14:16:23.551 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_started": 0, 2024-04-25T14:16:23.551 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_completed": 0, 2024-04-25T14:16:23.551 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_path": 0, 2024-04-25T14:16:23.551 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_inode": 0, 2024-04-25T14:16:23.551 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T14:16:23.551 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_exportdir": 0, 2024-04-25T14:16:23.552 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_flush": 0, 2024-04-25T14:16:23.552 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragmentdir": 0, 2024-04-25T14:16:23.552 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragstats": 0, 2024-04-25T14:16:23.552 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_inodestats": 0 2024-04-25T14:16:23.552 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.552 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_log": { 2024-04-25T14:16:23.552 INFO:teuthology.orchestra.run.smithi151.stdout: "evlrg": 0, 2024-04-25T14:16:23.552 INFO:teuthology.orchestra.run.smithi151.stdout: "evadd": 5, 2024-04-25T14:16:23.552 INFO:teuthology.orchestra.run.smithi151.stdout: "evex": 0, 2024-04-25T14:16:23.552 INFO:teuthology.orchestra.run.smithi151.stdout: "evtrm": 0, 2024-04-25T14:16:23.552 INFO:teuthology.orchestra.run.smithi151.stdout: "ev": 5, 2024-04-25T14:16:23.552 INFO:teuthology.orchestra.run.smithi151.stdout: "evexg": 0, 2024-04-25T14:16:23.552 INFO:teuthology.orchestra.run.smithi151.stdout: "evexd": 0, 2024-04-25T14:16:23.552 INFO:teuthology.orchestra.run.smithi151.stdout: "segadd": 2, 2024-04-25T14:16:23.553 INFO:teuthology.orchestra.run.smithi151.stdout: "segex": 0, 2024-04-25T14:16:23.553 INFO:teuthology.orchestra.run.smithi151.stdout: "segtrm": 0, 2024-04-25T14:16:23.553 INFO:teuthology.orchestra.run.smithi151.stdout: "seg": 2, 2024-04-25T14:16:23.553 INFO:teuthology.orchestra.run.smithi151.stdout: "segmjr": 2, 2024-04-25T14:16:23.553 INFO:teuthology.orchestra.run.smithi151.stdout: "segexg": 0, 2024-04-25T14:16:23.553 INFO:teuthology.orchestra.run.smithi151.stdout: "segexd": 0, 2024-04-25T14:16:23.553 INFO:teuthology.orchestra.run.smithi151.stdout: "expos": 4194304, 2024-04-25T14:16:23.553 INFO:teuthology.orchestra.run.smithi151.stdout: "wrpos": 4195732, 2024-04-25T14:16:23.553 INFO:teuthology.orchestra.run.smithi151.stdout: "rdpos": 0, 2024-04-25T14:16:23.553 INFO:teuthology.orchestra.run.smithi151.stdout: "jlat": { 2024-04-25T14:16:23.553 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 4, 2024-04-25T14:16:23.553 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.008167255, 2024-04-25T14:16:23.553 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.002041813 2024-04-25T14:16:23.553 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.554 INFO:teuthology.orchestra.run.smithi151.stdout: "replayed": 0 2024-04-25T14:16:23.554 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.554 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_mem": { 2024-04-25T14:16:23.554 INFO:teuthology.orchestra.run.smithi151.stdout: "ino": 22, 2024-04-25T14:16:23.554 INFO:teuthology.orchestra.run.smithi151.stdout: "ino+": 28, 2024-04-25T14:16:23.554 INFO:teuthology.orchestra.run.smithi151.stdout: "ino-": 6, 2024-04-25T14:16:23.554 INFO:teuthology.orchestra.run.smithi151.stdout: "dir": 21, 2024-04-25T14:16:23.554 INFO:teuthology.orchestra.run.smithi151.stdout: "dir+": 26, 2024-04-25T14:16:23.554 INFO:teuthology.orchestra.run.smithi151.stdout: "dir-": 5, 2024-04-25T14:16:23.554 INFO:teuthology.orchestra.run.smithi151.stdout: "dn": 18, 2024-04-25T14:16:23.554 INFO:teuthology.orchestra.run.smithi151.stdout: "dn+": 26, 2024-04-25T14:16:23.554 INFO:teuthology.orchestra.run.smithi151.stdout: "dn-": 8, 2024-04-25T14:16:23.554 INFO:teuthology.orchestra.run.smithi151.stdout: "cap": 2, 2024-04-25T14:16:23.555 INFO:teuthology.orchestra.run.smithi151.stdout: "cap+": 3, 2024-04-25T14:16:23.555 INFO:teuthology.orchestra.run.smithi151.stdout: "cap-": 1, 2024-04-25T14:16:23.555 INFO:teuthology.orchestra.run.smithi151.stdout: "rss": 121048, 2024-04-25T14:16:23.555 INFO:teuthology.orchestra.run.smithi151.stdout: "heap": 223632 2024-04-25T14:16:23.555 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.555 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_server": { 2024-04-25T14:16:23.555 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_client_request": 5, 2024-04-25T14:16:23.555 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_server_request": 0, 2024-04-25T14:16:23.555 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_request": 4, 2024-04-25T14:16:23.555 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_session": 7, 2024-04-25T14:16:23.555 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_peer_request": 0, 2024-04-25T14:16:23.555 INFO:teuthology.orchestra.run.smithi151.stdout: "req_create_latency": { 2024-04-25T14:16:23.555 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:23.556 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:23.556 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:23.556 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.556 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getattr_latency": { 2024-04-25T14:16:23.556 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 1, 2024-04-25T14:16:23.556 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000251647, 2024-04-25T14:16:23.556 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000251647 2024-04-25T14:16:23.556 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.556 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getfilelock_latency": { 2024-04-25T14:16:23.556 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:23.556 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:23.556 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:23.556 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.556 INFO:teuthology.orchestra.run.smithi151.stdout: "req_link_latency": { 2024-04-25T14:16:23.556 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:23.557 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:23.557 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:23.557 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.557 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookup_latency": { 2024-04-25T14:16:23.557 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:23.557 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:23.557 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:23.557 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.557 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookuphash_latency": { 2024-04-25T14:16:23.557 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:23.557 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:23.557 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:23.557 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.557 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupino_latency": { 2024-04-25T14:16:23.558 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 2, 2024-04-25T14:16:23.558 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000218369, 2024-04-25T14:16:23.558 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000109184 2024-04-25T14:16:23.558 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.558 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupname_latency": { 2024-04-25T14:16:23.558 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:23.558 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:23.558 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:23.558 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.558 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupparent_latency": { 2024-04-25T14:16:23.558 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:23.558 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:23.558 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:23.559 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.559 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupsnap_latency": { 2024-04-25T14:16:23.559 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:23.559 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:23.559 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:23.559 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.559 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lssnap_latency": { 2024-04-25T14:16:23.559 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:23.559 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:23.559 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:23.559 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.559 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mkdir_latency": { 2024-04-25T14:16:23.559 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:23.560 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:23.560 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:23.560 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.560 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mknod_latency": { 2024-04-25T14:16:23.560 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:23.560 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:23.560 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:23.560 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.560 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mksnap_latency": { 2024-04-25T14:16:23.560 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:23.561 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:23.561 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:23.561 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.561 INFO:teuthology.orchestra.run.smithi151.stdout: "req_open_latency": { 2024-04-25T14:16:23.561 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:23.561 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:23.561 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:23.561 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.561 INFO:teuthology.orchestra.run.smithi151.stdout: "req_readdir_latency": { 2024-04-25T14:16:23.561 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:23.562 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:23.562 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:23.562 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.562 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rename_latency": { 2024-04-25T14:16:23.562 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:23.562 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:23.562 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:23.562 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.562 INFO:teuthology.orchestra.run.smithi151.stdout: "req_renamesnap_latency": { 2024-04-25T14:16:23.562 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:23.562 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:23.562 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:23.562 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.562 INFO:teuthology.orchestra.run.smithi151.stdout: "req_snapdiff_latency": { 2024-04-25T14:16:23.563 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:23.563 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:23.563 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:23.563 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.563 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmdir_latency": { 2024-04-25T14:16:23.563 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:23.563 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:23.563 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:23.563 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.563 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmsnap_latency": { 2024-04-25T14:16:23.563 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:23.563 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:23.563 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:23.563 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.564 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmxattr_latency": { 2024-04-25T14:16:23.564 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:23.564 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:23.564 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:23.564 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.564 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setattr_latency": { 2024-04-25T14:16:23.564 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:23.564 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:23.564 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:23.564 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.564 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setdirlayout_latency": { 2024-04-25T14:16:23.564 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:23.564 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:23.564 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:23.565 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.565 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setfilelock_latency": { 2024-04-25T14:16:23.565 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:23.565 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:23.565 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:23.565 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.565 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setlayout_latency": { 2024-04-25T14:16:23.565 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:23.565 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:23.565 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:23.565 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.565 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setxattr_latency": { 2024-04-25T14:16:23.565 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:23.566 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:23.566 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:23.566 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.566 INFO:teuthology.orchestra.run.smithi151.stdout: "req_symlink_latency": { 2024-04-25T14:16:23.566 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:23.566 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:23.566 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:23.566 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.566 INFO:teuthology.orchestra.run.smithi151.stdout: "req_unlink_latency": { 2024-04-25T14:16:23.566 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:23.566 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:23.566 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:23.566 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.567 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_revoke_eviction": 0, 2024-04-25T14:16:23.567 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_acquisition_throttle": 0, 2024-04-25T14:16:23.567 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getvxattr_latency": { 2024-04-25T14:16:23.567 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:23.567 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:23.567 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:23.567 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-25T14:16:23.567 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.567 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_sessions": { 2024-04-25T14:16:23.567 INFO:teuthology.orchestra.run.smithi151.stdout: "session_count": 1, 2024-04-25T14:16:23.567 INFO:teuthology.orchestra.run.smithi151.stdout: "session_add": 2, 2024-04-25T14:16:23.567 INFO:teuthology.orchestra.run.smithi151.stdout: "session_remove": 1, 2024-04-25T14:16:23.567 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_open": 1, 2024-04-25T14:16:23.567 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_stale": 0, 2024-04-25T14:16:23.568 INFO:teuthology.orchestra.run.smithi151.stdout: "total_load": 2, 2024-04-25T14:16:23.568 INFO:teuthology.orchestra.run.smithi151.stdout: "average_load": 2, 2024-04-25T14:16:23.568 INFO:teuthology.orchestra.run.smithi151.stdout: "avg_session_uptime": 162, 2024-04-25T14:16:23.568 INFO:teuthology.orchestra.run.smithi151.stdout: "mdthresh_evicted": 0 2024-04-25T14:16:23.568 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.568 INFO:teuthology.orchestra.run.smithi151.stdout: "mempool": { 2024-04-25T14:16:23.568 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:16:23.568 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_items": 0, 2024-04-25T14:16:23.568 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:16:23.568 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:16:23.568 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:16:23.568 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:16:23.568 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:16:23.569 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:16:23.569 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:16:23.569 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:16:23.569 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:16:23.569 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:16:23.569 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:16:23.569 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:16:23.569 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:16:23.569 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_items": 0, 2024-04-25T14:16:23.569 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:16:23.569 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_items": 0, 2024-04-25T14:16:23.569 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:16:23.569 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:16:23.569 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:16:23.570 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:16:23.570 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:16:23.570 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:16:23.570 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:16:23.570 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_items": 0, 2024-04-25T14:16:23.570 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:16:23.570 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:16:23.570 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:16:23.570 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_items": 0, 2024-04-25T14:16:23.570 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_bytes": 0, 2024-04-25T14:16:23.570 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_items": 0, 2024-04-25T14:16:23.570 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:16:23.570 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:16:23.571 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:16:23.571 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:16:23.571 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_bytes": 246935, 2024-04-25T14:16:23.571 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_items": 84, 2024-04-25T14:16:23.571 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_bytes": 0, 2024-04-25T14:16:23.571 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_items": 0, 2024-04-25T14:16:23.571 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_bytes": 0, 2024-04-25T14:16:23.571 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_items": 0, 2024-04-25T14:16:23.571 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:16:23.571 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_items": 0, 2024-04-25T14:16:23.571 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:16:23.571 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_items": 0, 2024-04-25T14:16:23.571 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_bytes": 7720, 2024-04-25T14:16:23.571 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_items": 162, 2024-04-25T14:16:23.572 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:16:23.572 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:16:23.572 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_bytes": 0, 2024-04-25T14:16:23.572 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_items": 0, 2024-04-25T14:16:23.572 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_bytes": 90029, 2024-04-25T14:16:23.572 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_items": 350, 2024-04-25T14:16:23.572 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_bytes": 0, 2024-04-25T14:16:23.572 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_items": 0, 2024-04-25T14:16:23.572 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_bytes": 0, 2024-04-25T14:16:23.572 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_items": 0 2024-04-25T14:16:23.572 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.572 INFO:teuthology.orchestra.run.smithi151.stdout: "objecter": { 2024-04-25T14:16:23.572 INFO:teuthology.orchestra.run.smithi151.stdout: "op_active": 0, 2024-04-25T14:16:23.573 INFO:teuthology.orchestra.run.smithi151.stdout: "op_laggy": 0, 2024-04-25T14:16:23.573 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send": 41, 2024-04-25T14:16:23.573 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send_bytes": 11941, 2024-04-25T14:16:23.573 INFO:teuthology.orchestra.run.smithi151.stdout: "op_resend": 33, 2024-04-25T14:16:23.573 INFO:teuthology.orchestra.run.smithi151.stdout: "op_reply": 41, 2024-04-25T14:16:23.573 INFO:teuthology.orchestra.run.smithi151.stdout: "op_latency": { 2024-04-25T14:16:23.573 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 41, 2024-04-25T14:16:23.573 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.271996290, 2024-04-25T14:16:23.573 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.006634055 2024-04-25T14:16:23.573 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.573 INFO:teuthology.orchestra.run.smithi151.stdout: "op_inflight": 0, 2024-04-25T14:16:23.573 INFO:teuthology.orchestra.run.smithi151.stdout: "oplen_avg": { 2024-04-25T14:16:23.573 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 41, 2024-04-25T14:16:23.573 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 64 2024-04-25T14:16:23.573 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.573 INFO:teuthology.orchestra.run.smithi151.stdout: "op": 41, 2024-04-25T14:16:23.574 INFO:teuthology.orchestra.run.smithi151.stdout: "op_r": 0, 2024-04-25T14:16:23.574 INFO:teuthology.orchestra.run.smithi151.stdout: "op_w": 41, 2024-04-25T14:16:23.574 INFO:teuthology.orchestra.run.smithi151.stdout: "op_rmw": 0, 2024-04-25T14:16:23.574 INFO:teuthology.orchestra.run.smithi151.stdout: "op_pg": 0, 2024-04-25T14:16:23.574 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_stat": 0, 2024-04-25T14:16:23.574 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_create": 10, 2024-04-25T14:16:23.574 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_read": 0, 2024-04-25T14:16:23.574 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_write": 4, 2024-04-25T14:16:23.574 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writefull": 7, 2024-04-25T14:16:23.574 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writesame": 0, 2024-04-25T14:16:23.574 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_append": 0, 2024-04-25T14:16:23.574 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_zero": 0, 2024-04-25T14:16:23.574 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_truncate": 0, 2024-04-25T14:16:23.574 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_delete": 6, 2024-04-25T14:16:23.575 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_mapext": 0, 2024-04-25T14:16:23.575 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_sparse_read": 0, 2024-04-25T14:16:23.575 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_clonerange": 0, 2024-04-25T14:16:23.575 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_getxattr": 0, 2024-04-25T14:16:23.575 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_setxattr": 20, 2024-04-25T14:16:23.575 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_cmpxattr": 0, 2024-04-25T14:16:23.575 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_rmxattr": 0, 2024-04-25T14:16:23.575 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_resetxattrs": 0, 2024-04-25T14:16:23.575 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_call": 0, 2024-04-25T14:16:23.575 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_watch": 0, 2024-04-25T14:16:23.575 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_notify": 0, 2024-04-25T14:16:23.575 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T14:16:23.575 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls": 0, 2024-04-25T14:16:23.575 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls_filter": 0, 2024-04-25T14:16:23.575 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_other": 0, 2024-04-25T14:16:23.576 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_active": 0, 2024-04-25T14:16:23.576 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_send": 0, 2024-04-25T14:16:23.576 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_resend": 0, 2024-04-25T14:16:23.576 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_ping": 0, 2024-04-25T14:16:23.576 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_active": 0, 2024-04-25T14:16:23.576 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_send": 0, 2024-04-25T14:16:23.576 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_resend": 0, 2024-04-25T14:16:23.576 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_active": 0, 2024-04-25T14:16:23.576 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_send": 0, 2024-04-25T14:16:23.576 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_resend": 0, 2024-04-25T14:16:23.576 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_active": 0, 2024-04-25T14:16:23.576 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_send": 0, 2024-04-25T14:16:23.576 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_resend": 0, 2024-04-25T14:16:23.576 INFO:teuthology.orchestra.run.smithi151.stdout: "command_active": 0, 2024-04-25T14:16:23.576 INFO:teuthology.orchestra.run.smithi151.stdout: "command_send": 0, 2024-04-25T14:16:23.577 INFO:teuthology.orchestra.run.smithi151.stdout: "command_resend": 0, 2024-04-25T14:16:23.577 INFO:teuthology.orchestra.run.smithi151.stdout: "map_epoch": 78, 2024-04-25T14:16:23.577 INFO:teuthology.orchestra.run.smithi151.stdout: "map_full": 0, 2024-04-25T14:16:23.577 INFO:teuthology.orchestra.run.smithi151.stdout: "map_inc": 2, 2024-04-25T14:16:23.577 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_sessions": 8, 2024-04-25T14:16:23.577 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_open": 8, 2024-04-25T14:16:23.577 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_close": 0, 2024-04-25T14:16:23.577 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_laggy": 0, 2024-04-25T14:16:23.577 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_wr": 16, 2024-04-25T14:16:23.577 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_rd": 0, 2024-04-25T14:16:23.577 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_del": 1 2024-04-25T14:16:23.577 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.577 INFO:teuthology.orchestra.run.smithi151.stdout: "oft": { 2024-04-25T14:16:23.577 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_objs": 1, 2024-04-25T14:16:23.578 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_kv_pairs": 0, 2024-04-25T14:16:23.578 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_updates": 0, 2024-04-25T14:16:23.578 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_removes": 0 2024-04-25T14:16:23.578 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.578 INFO:teuthology.orchestra.run.smithi151.stdout: "osd-slow-ops": { 2024-04-25T14:16:23.578 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_ops_count": 0 2024-04-25T14:16:23.578 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.578 INFO:teuthology.orchestra.run.smithi151.stdout: "purge_queue": { 2024-04-25T14:16:23.578 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops": 0, 2024-04-25T14:16:23.578 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T14:16:23.578 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing": 0, 2024-04-25T14:16:23.578 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_high_water": 0, 2024-04-25T14:16:23.578 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed_ops": 0, 2024-04-25T14:16:23.579 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed": 0, 2024-04-25T14:16:23.579 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_item_in_journal": 0 2024-04-25T14:16:23.579 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.579 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:16:23.579 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-04-25T14:16:23.579 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-04-25T14:16:23.579 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-04-25T14:16:23.579 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 1050, 2024-04-25T14:16:23.579 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 720495, 2024-04-25T14:16:23.579 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-04-25T14:16:23.579 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 1050, 2024-04-25T14:16:23.580 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-04-25T14:16:23.580 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-04-25T14:16:23.580 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 1050, 2024-04-25T14:16:23.580 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 720495, 2024-04-25T14:16:23.580 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-04-25T14:16:23.580 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:23.580 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:23.580 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:23.580 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-25T14:16:23.580 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.580 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_bytes": { 2024-04-25T14:16:23.580 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-04-25T14:16:23.581 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-04-25T14:16:23.581 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-04-25T14:16:23.581 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-04-25T14:16:23.581 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-04-25T14:16:23.581 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-04-25T14:16:23.581 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-04-25T14:16:23.581 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 41, 2024-04-25T14:16:23.581 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 11941, 2024-04-25T14:16:23.581 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 41, 2024-04-25T14:16:23.581 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 11941, 2024-04-25T14:16:23.581 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-04-25T14:16:23.581 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:23.581 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:23.581 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:23.582 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-25T14:16:23.582 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.582 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_ops": { 2024-04-25T14:16:23.582 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-04-25T14:16:23.582 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 1024, 2024-04-25T14:16:23.582 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-04-25T14:16:23.582 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-04-25T14:16:23.582 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-04-25T14:16:23.582 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-04-25T14:16:23.582 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-04-25T14:16:23.582 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 41, 2024-04-25T14:16:23.582 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 41, 2024-04-25T14:16:23.582 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 41, 2024-04-25T14:16:23.583 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 41, 2024-04-25T14:16:23.583 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-04-25T14:16:23.583 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:23.583 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:23.583 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:23.583 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-25T14:16:23.583 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.583 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle": { 2024-04-25T14:16:23.583 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-04-25T14:16:23.583 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-04-25T14:16:23.583 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-04-25T14:16:23.583 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-04-25T14:16:23.583 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-04-25T14:16:23.583 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-04-25T14:16:23.584 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-04-25T14:16:23.584 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-04-25T14:16:23.584 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-04-25T14:16:23.584 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 0, 2024-04-25T14:16:23.584 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 0, 2024-04-25T14:16:23.584 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-04-25T14:16:23.584 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:23.584 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:23.584 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:23.584 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-25T14:16:23.584 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:23.584 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle-0x5597fc21c750": { 2024-04-25T14:16:23.584 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-04-25T14:16:23.584 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-04-25T14:16:23.584 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-04-25T14:16:23.584 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 5, 2024-04-25T14:16:23.585 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 1428, 2024-04-25T14:16:23.585 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-04-25T14:16:23.585 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 5, 2024-04-25T14:16:23.585 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-04-25T14:16:23.585 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-04-25T14:16:23.585 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 4, 2024-04-25T14:16:23.585 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 1428, 2024-04-25T14:16:23.585 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-04-25T14:16:23.585 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:23.585 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:23.585 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:23.585 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-25T14:16:23.585 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-25T14:16:23.585 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-25T14:16:24.369 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-25T14:16:24.369 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-25T14:16:24.369 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-04-25T14:16:24.369 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=13 2024-04-25T14:16:24.369 DEBUG:tasks.check_counter:Getting stats from i 2024-04-25T14:16:24.369 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-25T14:16:24.609 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:16:24.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:24 smithi151 ceph-mon[32990]: pgmap v429: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T14:16:24.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:24 smithi096 ceph-mon[29207]: pgmap v429: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T14:16:24.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:24 smithi097 ceph-mon[32720]: pgmap v429: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T14:16:26.443 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-25T14:16:26.444 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:16:26.444 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 852, 2024-04-25T14:16:26.444 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 576, 2024-04-25T14:16:26.444 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 275020, 2024-04-25T14:16:26.444 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 183409, 2024-04-25T14:16:26.444 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 10, 2024-04-25T14:16:26.444 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 3, 2024-04-25T14:16:26.444 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.066373527, 2024-04-25T14:16:26.444 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.023522415, 2024-04-25T14:16:26.444 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.047172547, 2024-04-25T14:16:26.444 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.002417809, 2024-04-25T14:16:26.444 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:16:26.444 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 580, 2024-04-25T14:16:26.445 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.049521810, 2024-04-25T14:16:26.445 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000085382 2024-04-25T14:16:26.445 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.445 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:16:26.445 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 1367, 2024-04-25T14:16:26.445 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000338149, 2024-04-25T14:16:26.445 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000247 2024-04-25T14:16:26.445 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.445 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T14:16:26.445 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T14:16:26.445 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.445 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:16:26.446 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 37, 2024-04-25T14:16:26.446 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 106, 2024-04-25T14:16:26.446 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 15895, 2024-04-25T14:16:26.446 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 259951, 2024-04-25T14:16:26.446 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 15, 2024-04-25T14:16:26.446 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 6, 2024-04-25T14:16:26.446 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.014792237, 2024-04-25T14:16:26.446 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.004176280, 2024-04-25T14:16:26.446 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.008912475, 2024-04-25T14:16:26.446 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.000917332, 2024-04-25T14:16:26.446 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:16:26.446 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 100, 2024-04-25T14:16:26.446 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.018652261, 2024-04-25T14:16:26.447 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000186522 2024-04-25T14:16:26.447 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.447 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:16:26.447 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 46, 2024-04-25T14:16:26.447 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000011931, 2024-04-25T14:16:26.447 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000259 2024-04-25T14:16:26.447 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.447 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 2096, 2024-04-25T14:16:26.447 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 196160 2024-04-25T14:16:26.447 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.447 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:16:26.447 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 159, 2024-04-25T14:16:26.447 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 147, 2024-04-25T14:16:26.447 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 536472, 2024-04-25T14:16:26.448 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 187558, 2024-04-25T14:16:26.448 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 17, 2024-04-25T14:16:26.448 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 5, 2024-04-25T14:16:26.448 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.025723744, 2024-04-25T14:16:26.448 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.006731835, 2024-04-25T14:16:26.448 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.019558106, 2024-04-25T14:16:26.448 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.002382068, 2024-04-25T14:16:26.448 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:16:26.448 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 145, 2024-04-25T14:16:26.448 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.017028559, 2024-04-25T14:16:26.448 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000117438 2024-04-25T14:16:26.448 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.449 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:16:26.449 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 10, 2024-04-25T14:16:26.449 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000001174, 2024-04-25T14:16:26.449 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000117 2024-04-25T14:16:26.449 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.449 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 532800, 2024-04-25T14:16:26.449 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 59264 2024-04-25T14:16:26.449 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.449 INFO:teuthology.orchestra.run.smithi151.stdout: "cct": { 2024-04-25T14:16:26.449 INFO:teuthology.orchestra.run.smithi151.stdout: "total_workers": 1, 2024-04-25T14:16:26.449 INFO:teuthology.orchestra.run.smithi151.stdout: "unhealthy_workers": 0 2024-04-25T14:16:26.449 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.449 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-MDSRank": { 2024-04-25T14:16:26.449 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-04-25T14:16:26.450 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-04-25T14:16:26.450 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 51, 2024-04-25T14:16:26.450 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.001709859, 2024-04-25T14:16:26.450 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000033526 2024-04-25T14:16:26.450 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-25T14:16:26.450 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.450 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-PurgeQueue": { 2024-04-25T14:16:26.450 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-04-25T14:16:26.450 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-04-25T14:16:26.450 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 2, 2024-04-25T14:16:26.450 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000008028, 2024-04-25T14:16:26.450 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000004014 2024-04-25T14:16:26.450 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-25T14:16:26.451 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.451 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": { 2024-04-25T14:16:26.451 INFO:teuthology.orchestra.run.smithi151.stdout: "request": 1, 2024-04-25T14:16:26.451 INFO:teuthology.orchestra.run.smithi151.stdout: "reply": 0, 2024-04-25T14:16:26.451 INFO:teuthology.orchestra.run.smithi151.stdout: "reply_latency": { 2024-04-25T14:16:26.451 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:26.451 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:26.451 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:26.451 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.451 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_reply": 0, 2024-04-25T14:16:26.451 INFO:teuthology.orchestra.run.smithi151.stdout: "forward": 1, 2024-04-25T14:16:26.451 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_complete": 0, 2024-04-25T14:16:26.451 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_keys": 0, 2024-04-25T14:16:26.451 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_commit": 11, 2024-04-25T14:16:26.452 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_split": 0, 2024-04-25T14:16:26.452 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_merge": 0, 2024-04-25T14:16:26.452 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes": 18, 2024-04-25T14:16:26.452 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_top": 2, 2024-04-25T14:16:26.452 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_bottom": 16, 2024-04-25T14:16:26.452 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pin_tail": 0, 2024-04-25T14:16:26.452 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pinned": 13, 2024-04-25T14:16:26.452 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_expired": 8, 2024-04-25T14:16:26.452 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_with_caps": 0, 2024-04-25T14:16:26.452 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": 0, 2024-04-25T14:16:26.452 INFO:teuthology.orchestra.run.smithi151.stdout: "subtrees": 3, 2024-04-25T14:16:26.452 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse": 10, 2024-04-25T14:16:26.452 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_hit": 6, 2024-04-25T14:16:26.453 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_forward": 1, 2024-04-25T14:16:26.453 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_discover": 3, 2024-04-25T14:16:26.453 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_dir_fetch": 0, 2024-04-25T14:16:26.453 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_remote_ino": 0, 2024-04-25T14:16:26.453 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_lock": 0, 2024-04-25T14:16:26.453 INFO:teuthology.orchestra.run.smithi151.stdout: "load_cent": 1000, 2024-04-25T14:16:26.453 INFO:teuthology.orchestra.run.smithi151.stdout: "q": 1, 2024-04-25T14:16:26.453 INFO:teuthology.orchestra.run.smithi151.stdout: "exported": 0, 2024-04-25T14:16:26.453 INFO:teuthology.orchestra.run.smithi151.stdout: "exported_inodes": 0, 2024-04-25T14:16:26.453 INFO:teuthology.orchestra.run.smithi151.stdout: "imported": 0, 2024-04-25T14:16:26.453 INFO:teuthology.orchestra.run.smithi151.stdout: "imported_inodes": 0, 2024-04-25T14:16:26.453 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_dir_fetch": 0, 2024-04-25T14:16:26.453 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_backtrace_fetch": 0, 2024-04-25T14:16:26.453 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_peer_discover": 0, 2024-04-25T14:16:26.454 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rfiles": 4, 2024-04-25T14:16:26.454 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rbytes": 234, 2024-04-25T14:16:26.454 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rsnaps": 0, 2024-04-25T14:16:26.454 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T14:16:26.454 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_set_tag": 0, 2024-04-25T14:16:26.454 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T14:16:26.454 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_inotable_repaired": 0, 2024-04-25T14:16:26.454 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_inodes": 0, 2024-04-25T14:16:26.454 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T14:16:26.454 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T14:16:26.454 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_file_inodes": 0, 2024-04-25T14:16:26.454 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_inode_file_caps": 0, 2024-04-25T14:16:26.454 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T14:16:26.454 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_grant": 0, 2024-04-25T14:16:26.455 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T14:16:26.455 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T14:16:26.455 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T14:16:26.455 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps": 0, 2024-04-25T14:16:26.455 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps_dirty": 0, 2024-04-25T14:16:26.455 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_cap_release": 0, 2024-04-25T14:16:26.455 INFO:teuthology.orchestra.run.smithi151.stdout: "process_request_cap_release": 0 2024-04-25T14:16:26.455 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.455 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_cache": { 2024-04-25T14:16:26.455 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update": 0, 2024-04-25T14:16:26.455 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update_receipt": 15, 2024-04-25T14:16:26.455 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_try_discover": 9, 2024-04-25T14:16:26.455 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_send_discover": 10, 2024-04-25T14:16:26.455 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_handle_discover": 0, 2024-04-25T14:16:26.456 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays": 0, 2024-04-25T14:16:26.456 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_delayed": 0, 2024-04-25T14:16:26.456 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_enqueuing": 0, 2024-04-25T14:16:26.456 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_created": 0, 2024-04-25T14:16:26.456 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_enqueued": 0, 2024-04-25T14:16:26.456 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_reintegrated": 0, 2024-04-25T14:16:26.456 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_migrated": 0, 2024-04-25T14:16:26.456 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_processing": 0, 2024-04-25T14:16:26.456 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_enqueued": 0, 2024-04-25T14:16:26.456 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_prioritized": 0, 2024-04-25T14:16:26.456 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_started": 0, 2024-04-25T14:16:26.456 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_completed": 0, 2024-04-25T14:16:26.456 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_path": 0, 2024-04-25T14:16:26.456 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_inode": 0, 2024-04-25T14:16:26.457 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T14:16:26.457 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_exportdir": 0, 2024-04-25T14:16:26.457 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_flush": 0, 2024-04-25T14:16:26.457 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragmentdir": 0, 2024-04-25T14:16:26.457 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragstats": 0, 2024-04-25T14:16:26.457 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_inodestats": 0 2024-04-25T14:16:26.457 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.457 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_log": { 2024-04-25T14:16:26.457 INFO:teuthology.orchestra.run.smithi151.stdout: "evlrg": 0, 2024-04-25T14:16:26.457 INFO:teuthology.orchestra.run.smithi151.stdout: "evadd": 5, 2024-04-25T14:16:26.457 INFO:teuthology.orchestra.run.smithi151.stdout: "evex": 0, 2024-04-25T14:16:26.457 INFO:teuthology.orchestra.run.smithi151.stdout: "evtrm": 0, 2024-04-25T14:16:26.457 INFO:teuthology.orchestra.run.smithi151.stdout: "ev": 5, 2024-04-25T14:16:26.457 INFO:teuthology.orchestra.run.smithi151.stdout: "evexg": 0, 2024-04-25T14:16:26.458 INFO:teuthology.orchestra.run.smithi151.stdout: "evexd": 0, 2024-04-25T14:16:26.458 INFO:teuthology.orchestra.run.smithi151.stdout: "segadd": 2, 2024-04-25T14:16:26.458 INFO:teuthology.orchestra.run.smithi151.stdout: "segex": 0, 2024-04-25T14:16:26.458 INFO:teuthology.orchestra.run.smithi151.stdout: "segtrm": 0, 2024-04-25T14:16:26.458 INFO:teuthology.orchestra.run.smithi151.stdout: "seg": 2, 2024-04-25T14:16:26.458 INFO:teuthology.orchestra.run.smithi151.stdout: "segmjr": 2, 2024-04-25T14:16:26.458 INFO:teuthology.orchestra.run.smithi151.stdout: "segexg": 0, 2024-04-25T14:16:26.458 INFO:teuthology.orchestra.run.smithi151.stdout: "segexd": 0, 2024-04-25T14:16:26.458 INFO:teuthology.orchestra.run.smithi151.stdout: "expos": 4194304, 2024-04-25T14:16:26.458 INFO:teuthology.orchestra.run.smithi151.stdout: "wrpos": 4195732, 2024-04-25T14:16:26.458 INFO:teuthology.orchestra.run.smithi151.stdout: "rdpos": 0, 2024-04-25T14:16:26.459 INFO:teuthology.orchestra.run.smithi151.stdout: "jlat": { 2024-04-25T14:16:26.459 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 4, 2024-04-25T14:16:26.459 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.007026824, 2024-04-25T14:16:26.459 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.001756706 2024-04-25T14:16:26.459 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.459 INFO:teuthology.orchestra.run.smithi151.stdout: "replayed": 0 2024-04-25T14:16:26.459 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.459 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_mem": { 2024-04-25T14:16:26.459 INFO:teuthology.orchestra.run.smithi151.stdout: "ino": 22, 2024-04-25T14:16:26.459 INFO:teuthology.orchestra.run.smithi151.stdout: "ino+": 28, 2024-04-25T14:16:26.459 INFO:teuthology.orchestra.run.smithi151.stdout: "ino-": 6, 2024-04-25T14:16:26.459 INFO:teuthology.orchestra.run.smithi151.stdout: "dir": 21, 2024-04-25T14:16:26.459 INFO:teuthology.orchestra.run.smithi151.stdout: "dir+": 26, 2024-04-25T14:16:26.460 INFO:teuthology.orchestra.run.smithi151.stdout: "dir-": 5, 2024-04-25T14:16:26.460 INFO:teuthology.orchestra.run.smithi151.stdout: "dn": 18, 2024-04-25T14:16:26.460 INFO:teuthology.orchestra.run.smithi151.stdout: "dn+": 26, 2024-04-25T14:16:26.460 INFO:teuthology.orchestra.run.smithi151.stdout: "dn-": 8, 2024-04-25T14:16:26.460 INFO:teuthology.orchestra.run.smithi151.stdout: "cap": 0, 2024-04-25T14:16:26.460 INFO:teuthology.orchestra.run.smithi151.stdout: "cap+": 0, 2024-04-25T14:16:26.460 INFO:teuthology.orchestra.run.smithi151.stdout: "cap-": 0, 2024-04-25T14:16:26.460 INFO:teuthology.orchestra.run.smithi151.stdout: "rss": 120468, 2024-04-25T14:16:26.460 INFO:teuthology.orchestra.run.smithi151.stdout: "heap": 223632 2024-04-25T14:16:26.460 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.460 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_server": { 2024-04-25T14:16:26.460 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_client_request": 1, 2024-04-25T14:16:26.461 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_server_request": 0, 2024-04-25T14:16:26.461 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_request": 1, 2024-04-25T14:16:26.461 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_session": 7, 2024-04-25T14:16:26.461 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_peer_request": 0, 2024-04-25T14:16:26.461 INFO:teuthology.orchestra.run.smithi151.stdout: "req_create_latency": { 2024-04-25T14:16:26.461 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:26.461 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:26.461 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:26.461 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.461 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getattr_latency": { 2024-04-25T14:16:26.461 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:26.461 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:26.461 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:26.462 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.462 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getfilelock_latency": { 2024-04-25T14:16:26.462 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:26.462 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:26.462 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:26.462 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.462 INFO:teuthology.orchestra.run.smithi151.stdout: "req_link_latency": { 2024-04-25T14:16:26.462 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:26.462 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:26.462 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:26.462 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.462 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookup_latency": { 2024-04-25T14:16:26.462 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:26.463 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:26.463 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:26.463 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.463 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookuphash_latency": { 2024-04-25T14:16:26.463 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:26.463 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:26.463 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:26.463 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.463 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupino_latency": { 2024-04-25T14:16:26.463 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:26.463 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:26.463 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:26.464 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.464 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupname_latency": { 2024-04-25T14:16:26.464 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:26.464 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:26.464 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:26.464 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.464 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupparent_latency": { 2024-04-25T14:16:26.464 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:26.464 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:26.464 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:26.464 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.464 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupsnap_latency": { 2024-04-25T14:16:26.465 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:26.465 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:26.465 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:26.465 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.465 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lssnap_latency": { 2024-04-25T14:16:26.465 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:26.465 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:26.465 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:26.465 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.465 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mkdir_latency": { 2024-04-25T14:16:26.465 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:26.466 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:26.466 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:26.466 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.466 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mknod_latency": { 2024-04-25T14:16:26.466 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:26.467 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:26.467 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:26.467 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.467 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mksnap_latency": { 2024-04-25T14:16:26.467 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:26.467 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:26.467 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:26.467 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.467 INFO:teuthology.orchestra.run.smithi151.stdout: "req_open_latency": { 2024-04-25T14:16:26.467 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:26.467 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:26.467 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:26.467 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.468 INFO:teuthology.orchestra.run.smithi151.stdout: "req_readdir_latency": { 2024-04-25T14:16:26.468 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:26.468 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:26.468 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:26.468 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.468 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rename_latency": { 2024-04-25T14:16:26.468 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:26.468 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:26.468 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:26.468 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.468 INFO:teuthology.orchestra.run.smithi151.stdout: "req_renamesnap_latency": { 2024-04-25T14:16:26.468 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:26.468 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:26.469 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:26.469 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.469 INFO:teuthology.orchestra.run.smithi151.stdout: "req_snapdiff_latency": { 2024-04-25T14:16:26.469 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:26.469 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:26.469 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:26.469 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.469 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmdir_latency": { 2024-04-25T14:16:26.469 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:26.469 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:26.469 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:26.469 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.470 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmsnap_latency": { 2024-04-25T14:16:26.470 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:26.470 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:26.470 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:26.470 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.470 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmxattr_latency": { 2024-04-25T14:16:26.470 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:26.470 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:26.471 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:26.471 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.471 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setattr_latency": { 2024-04-25T14:16:26.471 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:26.471 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:26.471 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:26.471 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.472 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setdirlayout_latency": { 2024-04-25T14:16:26.472 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:26.472 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:26.472 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:26.472 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.472 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setfilelock_latency": { 2024-04-25T14:16:26.472 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:26.472 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:26.473 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:26.473 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.473 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setlayout_latency": { 2024-04-25T14:16:26.473 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:26.473 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:26.473 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:26.473 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.473 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setxattr_latency": { 2024-04-25T14:16:26.474 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:26.474 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:26.474 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:26.474 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.474 INFO:teuthology.orchestra.run.smithi151.stdout: "req_symlink_latency": { 2024-04-25T14:16:26.474 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:26.474 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:26.474 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:26.475 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.475 INFO:teuthology.orchestra.run.smithi151.stdout: "req_unlink_latency": { 2024-04-25T14:16:26.475 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:26.475 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:26.475 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:26.475 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.475 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_revoke_eviction": 0, 2024-04-25T14:16:26.475 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_acquisition_throttle": 0, 2024-04-25T14:16:26.476 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getvxattr_latency": { 2024-04-25T14:16:26.476 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:26.476 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:26.476 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:26.476 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-25T14:16:26.476 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.476 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_sessions": { 2024-04-25T14:16:26.476 INFO:teuthology.orchestra.run.smithi151.stdout: "session_count": 1, 2024-04-25T14:16:26.477 INFO:teuthology.orchestra.run.smithi151.stdout: "session_add": 2, 2024-04-25T14:16:26.477 INFO:teuthology.orchestra.run.smithi151.stdout: "session_remove": 1, 2024-04-25T14:16:26.477 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_open": 1, 2024-04-25T14:16:26.477 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_stale": 0, 2024-04-25T14:16:26.477 INFO:teuthology.orchestra.run.smithi151.stdout: "total_load": 0, 2024-04-25T14:16:26.477 INFO:teuthology.orchestra.run.smithi151.stdout: "average_load": 0, 2024-04-25T14:16:26.477 INFO:teuthology.orchestra.run.smithi151.stdout: "avg_session_uptime": 173, 2024-04-25T14:16:26.477 INFO:teuthology.orchestra.run.smithi151.stdout: "mdthresh_evicted": 0 2024-04-25T14:16:26.478 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.478 INFO:teuthology.orchestra.run.smithi151.stdout: "mempool": { 2024-04-25T14:16:26.478 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:16:26.478 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_items": 0, 2024-04-25T14:16:26.478 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:16:26.478 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:16:26.478 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:16:26.478 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:16:26.479 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:16:26.479 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:16:26.479 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:16:26.479 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:16:26.479 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:16:26.479 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:16:26.479 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:16:26.479 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:16:26.479 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:16:26.479 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_items": 0, 2024-04-25T14:16:26.479 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:16:26.479 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_items": 0, 2024-04-25T14:16:26.480 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:16:26.480 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:16:26.480 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:16:26.480 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:16:26.480 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:16:26.480 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:16:26.480 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:16:26.480 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_items": 0, 2024-04-25T14:16:26.480 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:16:26.480 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:16:26.480 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:16:26.480 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_items": 0, 2024-04-25T14:16:26.480 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_bytes": 0, 2024-04-25T14:16:26.481 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_items": 0, 2024-04-25T14:16:26.481 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:16:26.481 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:16:26.481 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:16:26.481 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:16:26.481 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_bytes": 246327, 2024-04-25T14:16:26.481 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_items": 81, 2024-04-25T14:16:26.481 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_bytes": 0, 2024-04-25T14:16:26.481 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_items": 0, 2024-04-25T14:16:26.481 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_bytes": 0, 2024-04-25T14:16:26.481 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_items": 0, 2024-04-25T14:16:26.481 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:16:26.481 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_items": 0, 2024-04-25T14:16:26.481 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:16:26.482 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_items": 0, 2024-04-25T14:16:26.482 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_bytes": 7720, 2024-04-25T14:16:26.482 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_items": 162, 2024-04-25T14:16:26.482 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:16:26.482 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:16:26.482 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_bytes": 0, 2024-04-25T14:16:26.482 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_items": 0, 2024-04-25T14:16:26.482 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_bytes": 89293, 2024-04-25T14:16:26.482 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_items": 346, 2024-04-25T14:16:26.482 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_bytes": 0, 2024-04-25T14:16:26.482 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_items": 0, 2024-04-25T14:16:26.482 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_bytes": 0, 2024-04-25T14:16:26.482 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_items": 0 2024-04-25T14:16:26.483 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.483 INFO:teuthology.orchestra.run.smithi151.stdout: "objecter": { 2024-04-25T14:16:26.483 INFO:teuthology.orchestra.run.smithi151.stdout: "op_active": 0, 2024-04-25T14:16:26.483 INFO:teuthology.orchestra.run.smithi151.stdout: "op_laggy": 0, 2024-04-25T14:16:26.483 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send": 40, 2024-04-25T14:16:26.483 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send_bytes": 11730, 2024-04-25T14:16:26.483 INFO:teuthology.orchestra.run.smithi151.stdout: "op_resend": 33, 2024-04-25T14:16:26.483 INFO:teuthology.orchestra.run.smithi151.stdout: "op_reply": 40, 2024-04-25T14:16:26.483 INFO:teuthology.orchestra.run.smithi151.stdout: "op_latency": { 2024-04-25T14:16:26.483 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 40, 2024-04-25T14:16:26.483 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.256996501, 2024-04-25T14:16:26.483 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.006424912 2024-04-25T14:16:26.483 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.483 INFO:teuthology.orchestra.run.smithi151.stdout: "op_inflight": 0, 2024-04-25T14:16:26.483 INFO:teuthology.orchestra.run.smithi151.stdout: "oplen_avg": { 2024-04-25T14:16:26.484 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 40, 2024-04-25T14:16:26.484 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 62 2024-04-25T14:16:26.484 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.484 INFO:teuthology.orchestra.run.smithi151.stdout: "op": 40, 2024-04-25T14:16:26.484 INFO:teuthology.orchestra.run.smithi151.stdout: "op_r": 0, 2024-04-25T14:16:26.484 INFO:teuthology.orchestra.run.smithi151.stdout: "op_w": 40, 2024-04-25T14:16:26.484 INFO:teuthology.orchestra.run.smithi151.stdout: "op_rmw": 0, 2024-04-25T14:16:26.484 INFO:teuthology.orchestra.run.smithi151.stdout: "op_pg": 0, 2024-04-25T14:16:26.484 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_stat": 0, 2024-04-25T14:16:26.484 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_create": 10, 2024-04-25T14:16:26.484 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_read": 0, 2024-04-25T14:16:26.484 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_write": 4, 2024-04-25T14:16:26.484 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writefull": 7, 2024-04-25T14:16:26.485 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writesame": 0, 2024-04-25T14:16:26.485 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_append": 0, 2024-04-25T14:16:26.485 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_zero": 0, 2024-04-25T14:16:26.485 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_truncate": 0, 2024-04-25T14:16:26.485 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_delete": 6, 2024-04-25T14:16:26.485 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_mapext": 0, 2024-04-25T14:16:26.485 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_sparse_read": 0, 2024-04-25T14:16:26.485 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_clonerange": 0, 2024-04-25T14:16:26.485 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_getxattr": 0, 2024-04-25T14:16:26.485 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_setxattr": 20, 2024-04-25T14:16:26.485 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_cmpxattr": 0, 2024-04-25T14:16:26.485 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_rmxattr": 0, 2024-04-25T14:16:26.485 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_resetxattrs": 0, 2024-04-25T14:16:26.485 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_call": 0, 2024-04-25T14:16:26.486 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_watch": 0, 2024-04-25T14:16:26.486 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_notify": 0, 2024-04-25T14:16:26.486 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T14:16:26.486 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls": 0, 2024-04-25T14:16:26.486 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls_filter": 0, 2024-04-25T14:16:26.486 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_other": 0, 2024-04-25T14:16:26.486 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_active": 0, 2024-04-25T14:16:26.486 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_send": 0, 2024-04-25T14:16:26.486 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_resend": 0, 2024-04-25T14:16:26.486 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_ping": 0, 2024-04-25T14:16:26.486 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_active": 0, 2024-04-25T14:16:26.486 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_send": 0, 2024-04-25T14:16:26.486 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_resend": 0, 2024-04-25T14:16:26.486 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_active": 0, 2024-04-25T14:16:26.487 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_send": 0, 2024-04-25T14:16:26.487 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_resend": 0, 2024-04-25T14:16:26.487 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_active": 0, 2024-04-25T14:16:26.487 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_send": 0, 2024-04-25T14:16:26.487 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_resend": 0, 2024-04-25T14:16:26.487 INFO:teuthology.orchestra.run.smithi151.stdout: "command_active": 0, 2024-04-25T14:16:26.487 INFO:teuthology.orchestra.run.smithi151.stdout: "command_send": 0, 2024-04-25T14:16:26.487 INFO:teuthology.orchestra.run.smithi151.stdout: "command_resend": 0, 2024-04-25T14:16:26.487 INFO:teuthology.orchestra.run.smithi151.stdout: "map_epoch": 78, 2024-04-25T14:16:26.487 INFO:teuthology.orchestra.run.smithi151.stdout: "map_full": 0, 2024-04-25T14:16:26.487 INFO:teuthology.orchestra.run.smithi151.stdout: "map_inc": 2, 2024-04-25T14:16:26.487 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_sessions": 9, 2024-04-25T14:16:26.487 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_open": 9, 2024-04-25T14:16:26.487 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_close": 0, 2024-04-25T14:16:26.488 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_laggy": 0, 2024-04-25T14:16:26.488 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_wr": 14, 2024-04-25T14:16:26.488 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_rd": 0, 2024-04-25T14:16:26.488 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_del": 1 2024-04-25T14:16:26.488 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.488 INFO:teuthology.orchestra.run.smithi151.stdout: "oft": { 2024-04-25T14:16:26.488 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_objs": 1, 2024-04-25T14:16:26.488 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_kv_pairs": 0, 2024-04-25T14:16:26.488 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_updates": 0, 2024-04-25T14:16:26.488 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_removes": 0 2024-04-25T14:16:26.488 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.488 INFO:teuthology.orchestra.run.smithi151.stdout: "osd-slow-ops": { 2024-04-25T14:16:26.488 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_ops_count": 0 2024-04-25T14:16:26.488 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.489 INFO:teuthology.orchestra.run.smithi151.stdout: "purge_queue": { 2024-04-25T14:16:26.489 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops": 0, 2024-04-25T14:16:26.489 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T14:16:26.489 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing": 0, 2024-04-25T14:16:26.489 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_high_water": 0, 2024-04-25T14:16:26.489 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed_ops": 0, 2024-04-25T14:16:26.489 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed": 0, 2024-04-25T14:16:26.490 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_item_in_journal": 0 2024-04-25T14:16:26.490 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.490 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:16:26.490 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-04-25T14:16:26.490 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-04-25T14:16:26.490 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-04-25T14:16:26.490 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 1044, 2024-04-25T14:16:26.490 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 723256, 2024-04-25T14:16:26.490 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-04-25T14:16:26.490 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 1044, 2024-04-25T14:16:26.490 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-04-25T14:16:26.490 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-04-25T14:16:26.490 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 1044, 2024-04-25T14:16:26.491 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 723256, 2024-04-25T14:16:26.491 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-04-25T14:16:26.491 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:26.491 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:26.491 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:26.491 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-25T14:16:26.491 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.491 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_bytes": { 2024-04-25T14:16:26.491 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-04-25T14:16:26.491 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-04-25T14:16:26.492 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-04-25T14:16:26.492 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-04-25T14:16:26.492 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-04-25T14:16:26.492 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-04-25T14:16:26.492 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-04-25T14:16:26.492 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 40, 2024-04-25T14:16:26.492 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 11730, 2024-04-25T14:16:26.492 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 40, 2024-04-25T14:16:26.492 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 11730, 2024-04-25T14:16:26.492 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-04-25T14:16:26.492 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:26.492 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:26.493 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:26.493 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-25T14:16:26.493 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.493 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_ops": { 2024-04-25T14:16:26.493 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-04-25T14:16:26.493 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 1024, 2024-04-25T14:16:26.493 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-04-25T14:16:26.493 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-04-25T14:16:26.493 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-04-25T14:16:26.493 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-04-25T14:16:26.493 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-04-25T14:16:26.493 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 40, 2024-04-25T14:16:26.493 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 40, 2024-04-25T14:16:26.493 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 40, 2024-04-25T14:16:26.494 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 40, 2024-04-25T14:16:26.494 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-04-25T14:16:26.494 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:26.494 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:26.494 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:26.494 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-25T14:16:26.494 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.494 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle": { 2024-04-25T14:16:26.494 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-04-25T14:16:26.494 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-04-25T14:16:26.494 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-04-25T14:16:26.494 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-04-25T14:16:26.494 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-04-25T14:16:26.494 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-04-25T14:16:26.495 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-04-25T14:16:26.495 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-04-25T14:16:26.495 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-04-25T14:16:26.495 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 0, 2024-04-25T14:16:26.495 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 0, 2024-04-25T14:16:26.495 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-04-25T14:16:26.495 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:26.495 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:26.495 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:26.495 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-25T14:16:26.495 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:26.496 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle-0x55eea4482680": { 2024-04-25T14:16:26.496 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-04-25T14:16:26.496 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-04-25T14:16:26.496 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-04-25T14:16:26.496 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 5, 2024-04-25T14:16:26.496 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 1428, 2024-04-25T14:16:26.496 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-04-25T14:16:26.496 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 5, 2024-04-25T14:16:26.496 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-04-25T14:16:26.496 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-04-25T14:16:26.496 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 4, 2024-04-25T14:16:26.496 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 1428, 2024-04-25T14:16:26.496 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-04-25T14:16:26.497 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:26.497 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:26.497 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:26.497 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-25T14:16:26.497 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-25T14:16:26.497 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-25T14:16:26.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:26 smithi151 ceph-mon[32990]: pgmap v430: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T14:16:26.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:26 smithi096 ceph-mon[29207]: pgmap v430: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T14:16:26.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:26 smithi097 ceph-mon[32720]: pgmap v430: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T14:16:27.203 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-04-25T14:16:27.203 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-04-25T14:16:27.203 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-04-25T14:16:27.203 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=15 2024-04-25T14:16:27.203 DEBUG:tasks.check_counter:Getting stats from l 2024-04-25T14:16:27.203 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 59334182-030c-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-25T14:16:27.441 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:16:28.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:28 smithi096 ceph-mon[29207]: pgmap v431: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T14:16:28.921 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:28 smithi151 ceph-mon[32990]: pgmap v431: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T14:16:28.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:28 smithi097 ceph-mon[32720]: pgmap v431: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T14:16:29.237 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-25T14:16:29.237 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:16:29.237 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 40, 2024-04-25T14:16:29.237 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 46, 2024-04-25T14:16:29.237 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 15694, 2024-04-25T14:16:29.237 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 183834, 2024-04-25T14:16:29.237 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 25, 2024-04-25T14:16:29.237 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 2, 2024-04-25T14:16:29.237 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.011337695, 2024-04-25T14:16:29.238 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.001381114, 2024-04-25T14:16:29.238 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.007608743, 2024-04-25T14:16:29.238 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.000241840, 2024-04-25T14:16:29.238 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:16:29.238 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 40, 2024-04-25T14:16:29.238 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.017493852, 2024-04-25T14:16:29.238 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000437346 2024-04-25T14:16:29.238 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.238 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:16:29.238 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 40, 2024-04-25T14:16:29.239 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000008905, 2024-04-25T14:16:29.239 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000222 2024-04-25T14:16:29.239 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.239 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T14:16:29.239 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T14:16:29.239 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.239 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:16:29.239 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 937, 2024-04-25T14:16:29.239 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 657, 2024-04-25T14:16:29.239 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 759374, 2024-04-25T14:16:29.239 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 178492, 2024-04-25T14:16:29.240 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 7, 2024-04-25T14:16:29.240 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 5, 2024-04-25T14:16:29.240 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.078201560, 2024-04-25T14:16:29.240 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.027825070, 2024-04-25T14:16:29.240 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.058474054, 2024-04-25T14:16:29.240 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.004347128, 2024-04-25T14:16:29.240 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:16:29.240 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 661, 2024-04-25T14:16:29.240 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.047158933, 2024-04-25T14:16:29.240 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000071344 2024-04-25T14:16:29.240 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.240 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:16:29.241 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 1293, 2024-04-25T14:16:29.241 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000292037, 2024-04-25T14:16:29.241 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000225 2024-04-25T14:16:29.241 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.241 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 513024, 2024-04-25T14:16:29.241 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 63104 2024-04-25T14:16:29.241 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.241 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:16:29.241 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 31, 2024-04-25T14:16:29.241 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 101, 2024-04-25T14:16:29.241 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 14960, 2024-04-25T14:16:29.242 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 213668, 2024-04-25T14:16:29.242 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 8, 2024-04-25T14:16:29.242 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 4, 2024-04-25T14:16:29.242 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.008308040, 2024-04-25T14:16:29.242 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.003719439, 2024-04-25T14:16:29.242 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.003128060, 2024-04-25T14:16:29.242 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.000670817, 2024-04-25T14:16:29.242 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:16:29.242 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 99, 2024-04-25T14:16:29.242 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.019298232, 2024-04-25T14:16:29.243 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000194931 2024-04-25T14:16:29.243 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.243 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:16:29.243 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 22, 2024-04-25T14:16:29.243 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000005888, 2024-04-25T14:16:29.243 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000267 2024-04-25T14:16:29.243 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.243 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T14:16:29.243 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 195584 2024-04-25T14:16:29.243 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.243 INFO:teuthology.orchestra.run.smithi151.stdout: "cct": { 2024-04-25T14:16:29.244 INFO:teuthology.orchestra.run.smithi151.stdout: "total_workers": 1, 2024-04-25T14:16:29.244 INFO:teuthology.orchestra.run.smithi151.stdout: "unhealthy_workers": 0 2024-04-25T14:16:29.244 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.244 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-MDSRank": { 2024-04-25T14:16:29.244 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-04-25T14:16:29.244 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-04-25T14:16:29.244 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 45, 2024-04-25T14:16:29.244 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.001430262, 2024-04-25T14:16:29.244 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000031783 2024-04-25T14:16:29.244 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-25T14:16:29.244 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.245 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-PurgeQueue": { 2024-04-25T14:16:29.245 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-04-25T14:16:29.245 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-04-25T14:16:29.245 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 2, 2024-04-25T14:16:29.245 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000007022, 2024-04-25T14:16:29.245 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000003511 2024-04-25T14:16:29.245 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-25T14:16:29.245 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.245 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": { 2024-04-25T14:16:29.245 INFO:teuthology.orchestra.run.smithi151.stdout: "request": 0, 2024-04-25T14:16:29.245 INFO:teuthology.orchestra.run.smithi151.stdout: "reply": 0, 2024-04-25T14:16:29.246 INFO:teuthology.orchestra.run.smithi151.stdout: "reply_latency": { 2024-04-25T14:16:29.246 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:29.246 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:29.246 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:29.246 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.246 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_reply": 0, 2024-04-25T14:16:29.246 INFO:teuthology.orchestra.run.smithi151.stdout: "forward": 0, 2024-04-25T14:16:29.246 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_complete": 0, 2024-04-25T14:16:29.246 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_keys": 0, 2024-04-25T14:16:29.246 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_commit": 11, 2024-04-25T14:16:29.247 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_split": 0, 2024-04-25T14:16:29.247 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_merge": 0, 2024-04-25T14:16:29.247 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes": 17, 2024-04-25T14:16:29.247 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_top": 1, 2024-04-25T14:16:29.247 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_bottom": 16, 2024-04-25T14:16:29.247 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pin_tail": 0, 2024-04-25T14:16:29.247 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pinned": 13, 2024-04-25T14:16:29.247 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_expired": 8, 2024-04-25T14:16:29.247 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_with_caps": 0, 2024-04-25T14:16:29.247 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": 0, 2024-04-25T14:16:29.247 INFO:teuthology.orchestra.run.smithi151.stdout: "subtrees": 3, 2024-04-25T14:16:29.248 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse": 6, 2024-04-25T14:16:29.248 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_hit": 4, 2024-04-25T14:16:29.248 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_forward": 0, 2024-04-25T14:16:29.248 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_discover": 2, 2024-04-25T14:16:29.248 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_dir_fetch": 0, 2024-04-25T14:16:29.248 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_remote_ino": 0, 2024-04-25T14:16:29.248 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_lock": 0, 2024-04-25T14:16:29.248 INFO:teuthology.orchestra.run.smithi151.stdout: "load_cent": 0, 2024-04-25T14:16:29.248 INFO:teuthology.orchestra.run.smithi151.stdout: "q": 0, 2024-04-25T14:16:29.248 INFO:teuthology.orchestra.run.smithi151.stdout: "exported": 0, 2024-04-25T14:16:29.248 INFO:teuthology.orchestra.run.smithi151.stdout: "exported_inodes": 0, 2024-04-25T14:16:29.248 INFO:teuthology.orchestra.run.smithi151.stdout: "imported": 0, 2024-04-25T14:16:29.249 INFO:teuthology.orchestra.run.smithi151.stdout: "imported_inodes": 0, 2024-04-25T14:16:29.249 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_dir_fetch": 0, 2024-04-25T14:16:29.249 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_backtrace_fetch": 0, 2024-04-25T14:16:29.249 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_peer_discover": 0, 2024-04-25T14:16:29.249 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rfiles": 4, 2024-04-25T14:16:29.249 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rbytes": 234, 2024-04-25T14:16:29.249 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rsnaps": 0, 2024-04-25T14:16:29.249 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T14:16:29.249 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_set_tag": 0, 2024-04-25T14:16:29.249 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T14:16:29.249 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_inotable_repaired": 0, 2024-04-25T14:16:29.249 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_inodes": 0, 2024-04-25T14:16:29.250 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T14:16:29.250 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T14:16:29.250 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_file_inodes": 0, 2024-04-25T14:16:29.250 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_inode_file_caps": 0, 2024-04-25T14:16:29.250 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T14:16:29.250 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_grant": 0, 2024-04-25T14:16:29.250 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T14:16:29.250 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T14:16:29.250 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T14:16:29.250 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps": 0, 2024-04-25T14:16:29.250 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps_dirty": 0, 2024-04-25T14:16:29.251 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_cap_release": 0, 2024-04-25T14:16:29.251 INFO:teuthology.orchestra.run.smithi151.stdout: "process_request_cap_release": 0 2024-04-25T14:16:29.251 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.251 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_cache": { 2024-04-25T14:16:29.251 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update": 0, 2024-04-25T14:16:29.251 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update_receipt": 10, 2024-04-25T14:16:29.251 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_try_discover": 6, 2024-04-25T14:16:29.251 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_send_discover": 7, 2024-04-25T14:16:29.251 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_handle_discover": 0, 2024-04-25T14:16:29.251 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays": 0, 2024-04-25T14:16:29.251 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_delayed": 0, 2024-04-25T14:16:29.251 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_enqueuing": 0, 2024-04-25T14:16:29.252 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_created": 0, 2024-04-25T14:16:29.252 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_enqueued": 0, 2024-04-25T14:16:29.252 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_reintegrated": 0, 2024-04-25T14:16:29.252 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_migrated": 0, 2024-04-25T14:16:29.252 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_processing": 0, 2024-04-25T14:16:29.252 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_enqueued": 0, 2024-04-25T14:16:29.252 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_prioritized": 0, 2024-04-25T14:16:29.252 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_started": 0, 2024-04-25T14:16:29.252 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_completed": 0, 2024-04-25T14:16:29.252 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_path": 0, 2024-04-25T14:16:29.252 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_inode": 0, 2024-04-25T14:16:29.252 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T14:16:29.252 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_exportdir": 0, 2024-04-25T14:16:29.253 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_flush": 0, 2024-04-25T14:16:29.253 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragmentdir": 0, 2024-04-25T14:16:29.253 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragstats": 0, 2024-04-25T14:16:29.253 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_inodestats": 0 2024-04-25T14:16:29.253 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.253 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_log": { 2024-04-25T14:16:29.253 INFO:teuthology.orchestra.run.smithi151.stdout: "evlrg": 0, 2024-04-25T14:16:29.253 INFO:teuthology.orchestra.run.smithi151.stdout: "evadd": 2, 2024-04-25T14:16:29.253 INFO:teuthology.orchestra.run.smithi151.stdout: "evex": 0, 2024-04-25T14:16:29.253 INFO:teuthology.orchestra.run.smithi151.stdout: "evtrm": 0, 2024-04-25T14:16:29.253 INFO:teuthology.orchestra.run.smithi151.stdout: "ev": 2, 2024-04-25T14:16:29.253 INFO:teuthology.orchestra.run.smithi151.stdout: "evexg": 0, 2024-04-25T14:16:29.254 INFO:teuthology.orchestra.run.smithi151.stdout: "evexd": 0, 2024-04-25T14:16:29.254 INFO:teuthology.orchestra.run.smithi151.stdout: "segadd": 2, 2024-04-25T14:16:29.254 INFO:teuthology.orchestra.run.smithi151.stdout: "segex": 0, 2024-04-25T14:16:29.254 INFO:teuthology.orchestra.run.smithi151.stdout: "segtrm": 0, 2024-04-25T14:16:29.254 INFO:teuthology.orchestra.run.smithi151.stdout: "seg": 2, 2024-04-25T14:16:29.254 INFO:teuthology.orchestra.run.smithi151.stdout: "segmjr": 2, 2024-04-25T14:16:29.254 INFO:teuthology.orchestra.run.smithi151.stdout: "segexg": 0, 2024-04-25T14:16:29.254 INFO:teuthology.orchestra.run.smithi151.stdout: "segexd": 0, 2024-04-25T14:16:29.254 INFO:teuthology.orchestra.run.smithi151.stdout: "expos": 4194304, 2024-04-25T14:16:29.254 INFO:teuthology.orchestra.run.smithi151.stdout: "wrpos": 4194892, 2024-04-25T14:16:29.254 INFO:teuthology.orchestra.run.smithi151.stdout: "rdpos": 0, 2024-04-25T14:16:29.254 INFO:teuthology.orchestra.run.smithi151.stdout: "jlat": { 2024-04-25T14:16:29.254 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 1, 2024-04-25T14:16:29.255 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.001287020, 2024-04-25T14:16:29.255 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.001287020 2024-04-25T14:16:29.255 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.255 INFO:teuthology.orchestra.run.smithi151.stdout: "replayed": 0 2024-04-25T14:16:29.255 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.255 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_mem": { 2024-04-25T14:16:29.255 INFO:teuthology.orchestra.run.smithi151.stdout: "ino": 21, 2024-04-25T14:16:29.255 INFO:teuthology.orchestra.run.smithi151.stdout: "ino+": 27, 2024-04-25T14:16:29.255 INFO:teuthology.orchestra.run.smithi151.stdout: "ino-": 6, 2024-04-25T14:16:29.255 INFO:teuthology.orchestra.run.smithi151.stdout: "dir": 20, 2024-04-25T14:16:29.255 INFO:teuthology.orchestra.run.smithi151.stdout: "dir+": 25, 2024-04-25T14:16:29.255 INFO:teuthology.orchestra.run.smithi151.stdout: "dir-": 5, 2024-04-25T14:16:29.255 INFO:teuthology.orchestra.run.smithi151.stdout: "dn": 17, 2024-04-25T14:16:29.255 INFO:teuthology.orchestra.run.smithi151.stdout: "dn+": 25, 2024-04-25T14:16:29.256 INFO:teuthology.orchestra.run.smithi151.stdout: "dn-": 8, 2024-04-25T14:16:29.256 INFO:teuthology.orchestra.run.smithi151.stdout: "cap": 0, 2024-04-25T14:16:29.256 INFO:teuthology.orchestra.run.smithi151.stdout: "cap+": 0, 2024-04-25T14:16:29.256 INFO:teuthology.orchestra.run.smithi151.stdout: "cap-": 0, 2024-04-25T14:16:29.256 INFO:teuthology.orchestra.run.smithi151.stdout: "rss": 121432, 2024-04-25T14:16:29.256 INFO:teuthology.orchestra.run.smithi151.stdout: "heap": 223632 2024-04-25T14:16:29.256 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.256 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_server": { 2024-04-25T14:16:29.256 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_client_request": 0, 2024-04-25T14:16:29.256 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_server_request": 0, 2024-04-25T14:16:29.256 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_request": 0, 2024-04-25T14:16:29.256 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_session": 0, 2024-04-25T14:16:29.256 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_peer_request": 0, 2024-04-25T14:16:29.256 INFO:teuthology.orchestra.run.smithi151.stdout: "req_create_latency": { 2024-04-25T14:16:29.256 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:29.257 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:29.257 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:29.257 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.257 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getattr_latency": { 2024-04-25T14:16:29.257 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:29.257 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:29.257 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:29.257 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.257 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getfilelock_latency": { 2024-04-25T14:16:29.257 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:29.257 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:29.257 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:29.257 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.257 INFO:teuthology.orchestra.run.smithi151.stdout: "req_link_latency": { 2024-04-25T14:16:29.258 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:29.258 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:29.258 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:29.258 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.258 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookup_latency": { 2024-04-25T14:16:29.258 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:29.258 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:29.258 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:29.258 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.258 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookuphash_latency": { 2024-04-25T14:16:29.258 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:29.258 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:29.258 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:29.259 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.259 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupino_latency": { 2024-04-25T14:16:29.259 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:29.259 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:29.259 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:29.259 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.259 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupname_latency": { 2024-04-25T14:16:29.259 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:29.259 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:29.259 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:29.259 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.259 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupparent_latency": { 2024-04-25T14:16:29.259 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:29.259 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:29.259 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:29.260 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.260 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupsnap_latency": { 2024-04-25T14:16:29.260 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:29.260 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:29.260 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:29.260 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.260 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lssnap_latency": { 2024-04-25T14:16:29.260 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:29.260 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:29.260 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:29.260 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.260 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mkdir_latency": { 2024-04-25T14:16:29.260 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:29.261 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:29.261 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:29.261 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.261 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mknod_latency": { 2024-04-25T14:16:29.262 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:29.262 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:29.262 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:29.262 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.262 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mksnap_latency": { 2024-04-25T14:16:29.262 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:29.262 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:29.262 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:29.262 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.262 INFO:teuthology.orchestra.run.smithi151.stdout: "req_open_latency": { 2024-04-25T14:16:29.262 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:29.262 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:29.262 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:29.263 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.263 INFO:teuthology.orchestra.run.smithi151.stdout: "req_readdir_latency": { 2024-04-25T14:16:29.263 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:29.263 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:29.263 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:29.263 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.263 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rename_latency": { 2024-04-25T14:16:29.263 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:29.263 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:29.263 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:29.263 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.263 INFO:teuthology.orchestra.run.smithi151.stdout: "req_renamesnap_latency": { 2024-04-25T14:16:29.263 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:29.264 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:29.264 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:29.264 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.264 INFO:teuthology.orchestra.run.smithi151.stdout: "req_snapdiff_latency": { 2024-04-25T14:16:29.264 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:29.264 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:29.265 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:29.265 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.265 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmdir_latency": { 2024-04-25T14:16:29.265 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:29.265 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:29.265 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:29.265 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.266 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmsnap_latency": { 2024-04-25T14:16:29.266 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:29.266 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:29.266 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:29.266 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.266 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmxattr_latency": { 2024-04-25T14:16:29.266 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:29.266 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:29.266 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:29.266 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.266 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setattr_latency": { 2024-04-25T14:16:29.266 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:29.267 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:29.267 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:29.267 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.267 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setdirlayout_latency": { 2024-04-25T14:16:29.267 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:29.267 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:29.267 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:29.267 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.267 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setfilelock_latency": { 2024-04-25T14:16:29.267 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:29.267 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:29.268 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:29.268 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.268 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setlayout_latency": { 2024-04-25T14:16:29.268 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:29.268 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:29.268 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:29.268 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.268 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setxattr_latency": { 2024-04-25T14:16:29.268 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:29.268 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:29.268 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:29.269 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.269 INFO:teuthology.orchestra.run.smithi151.stdout: "req_symlink_latency": { 2024-04-25T14:16:29.269 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:29.269 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:29.269 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:29.269 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.269 INFO:teuthology.orchestra.run.smithi151.stdout: "req_unlink_latency": { 2024-04-25T14:16:29.269 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:29.269 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:29.269 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:29.269 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.269 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_revoke_eviction": 0, 2024-04-25T14:16:29.270 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_acquisition_throttle": 0, 2024-04-25T14:16:29.270 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getvxattr_latency": { 2024-04-25T14:16:29.270 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:29.270 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:29.270 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:29.270 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-25T14:16:29.270 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.270 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_sessions": { 2024-04-25T14:16:29.270 INFO:teuthology.orchestra.run.smithi151.stdout: "session_count": 0, 2024-04-25T14:16:29.270 INFO:teuthology.orchestra.run.smithi151.stdout: "session_add": 0, 2024-04-25T14:16:29.270 INFO:teuthology.orchestra.run.smithi151.stdout: "session_remove": 0, 2024-04-25T14:16:29.271 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_open": 0, 2024-04-25T14:16:29.271 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_stale": 0, 2024-04-25T14:16:29.271 INFO:teuthology.orchestra.run.smithi151.stdout: "total_load": 0, 2024-04-25T14:16:29.271 INFO:teuthology.orchestra.run.smithi151.stdout: "average_load": 0, 2024-04-25T14:16:29.271 INFO:teuthology.orchestra.run.smithi151.stdout: "avg_session_uptime": 0, 2024-04-25T14:16:29.271 INFO:teuthology.orchestra.run.smithi151.stdout: "mdthresh_evicted": 0 2024-04-25T14:16:29.271 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.271 INFO:teuthology.orchestra.run.smithi151.stdout: "mempool": { 2024-04-25T14:16:29.271 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:16:29.271 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_items": 0, 2024-04-25T14:16:29.271 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:16:29.272 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:16:29.272 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:16:29.272 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:16:29.272 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:16:29.272 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:16:29.272 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:16:29.272 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:16:29.272 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:16:29.272 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:16:29.272 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:16:29.272 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:16:29.272 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:16:29.273 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_items": 0, 2024-04-25T14:16:29.273 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:16:29.273 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_items": 0, 2024-04-25T14:16:29.273 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:16:29.273 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:16:29.273 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:16:29.273 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:16:29.273 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:16:29.273 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:16:29.273 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:16:29.274 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_items": 0, 2024-04-25T14:16:29.274 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:16:29.274 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:16:29.274 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:16:29.274 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_items": 0, 2024-04-25T14:16:29.274 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_bytes": 0, 2024-04-25T14:16:29.274 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_items": 0, 2024-04-25T14:16:29.274 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:16:29.274 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:16:29.274 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:16:29.274 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:16:29.275 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_bytes": 253799, 2024-04-25T14:16:29.275 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_items": 78, 2024-04-25T14:16:29.275 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_bytes": 0, 2024-04-25T14:16:29.275 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_items": 0, 2024-04-25T14:16:29.275 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_bytes": 0, 2024-04-25T14:16:29.275 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_items": 0, 2024-04-25T14:16:29.275 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:16:29.275 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_items": 0, 2024-04-25T14:16:29.275 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:16:29.275 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_items": 0, 2024-04-25T14:16:29.275 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_bytes": 7720, 2024-04-25T14:16:29.276 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_items": 162, 2024-04-25T14:16:29.276 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:16:29.276 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:16:29.276 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_bytes": 0, 2024-04-25T14:16:29.276 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_items": 0, 2024-04-25T14:16:29.276 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_bytes": 85045, 2024-04-25T14:16:29.276 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_items": 326, 2024-04-25T14:16:29.276 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_bytes": 0, 2024-04-25T14:16:29.276 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_items": 0, 2024-04-25T14:16:29.276 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_bytes": 0, 2024-04-25T14:16:29.276 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_items": 0 2024-04-25T14:16:29.276 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.277 INFO:teuthology.orchestra.run.smithi151.stdout: "objecter": { 2024-04-25T14:16:29.277 INFO:teuthology.orchestra.run.smithi151.stdout: "op_active": 0, 2024-04-25T14:16:29.277 INFO:teuthology.orchestra.run.smithi151.stdout: "op_laggy": 0, 2024-04-25T14:16:29.277 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send": 35, 2024-04-25T14:16:29.277 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send_bytes": 10710, 2024-04-25T14:16:29.277 INFO:teuthology.orchestra.run.smithi151.stdout: "op_resend": 33, 2024-04-25T14:16:29.277 INFO:teuthology.orchestra.run.smithi151.stdout: "op_reply": 35, 2024-04-25T14:16:29.277 INFO:teuthology.orchestra.run.smithi151.stdout: "op_latency": { 2024-04-25T14:16:29.277 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 35, 2024-04-25T14:16:29.277 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.208997155, 2024-04-25T14:16:29.277 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.005971347 2024-04-25T14:16:29.278 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.278 INFO:teuthology.orchestra.run.smithi151.stdout: "op_inflight": 0, 2024-04-25T14:16:29.278 INFO:teuthology.orchestra.run.smithi151.stdout: "oplen_avg": { 2024-04-25T14:16:29.278 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 35, 2024-04-25T14:16:29.278 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 57 2024-04-25T14:16:29.278 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.278 INFO:teuthology.orchestra.run.smithi151.stdout: "op": 35, 2024-04-25T14:16:29.278 INFO:teuthology.orchestra.run.smithi151.stdout: "op_r": 0, 2024-04-25T14:16:29.278 INFO:teuthology.orchestra.run.smithi151.stdout: "op_w": 35, 2024-04-25T14:16:29.278 INFO:teuthology.orchestra.run.smithi151.stdout: "op_rmw": 0, 2024-04-25T14:16:29.278 INFO:teuthology.orchestra.run.smithi151.stdout: "op_pg": 0, 2024-04-25T14:16:29.279 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_stat": 0, 2024-04-25T14:16:29.279 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_create": 10, 2024-04-25T14:16:29.279 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_read": 0, 2024-04-25T14:16:29.279 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_write": 1, 2024-04-25T14:16:29.279 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writefull": 5, 2024-04-25T14:16:29.279 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writesame": 0, 2024-04-25T14:16:29.279 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_append": 0, 2024-04-25T14:16:29.279 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_zero": 0, 2024-04-25T14:16:29.279 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_truncate": 0, 2024-04-25T14:16:29.279 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_delete": 6, 2024-04-25T14:16:29.279 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_mapext": 0, 2024-04-25T14:16:29.279 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_sparse_read": 0, 2024-04-25T14:16:29.279 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_clonerange": 0, 2024-04-25T14:16:29.280 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_getxattr": 0, 2024-04-25T14:16:29.280 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_setxattr": 20, 2024-04-25T14:16:29.280 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_cmpxattr": 0, 2024-04-25T14:16:29.280 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_rmxattr": 0, 2024-04-25T14:16:29.280 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_resetxattrs": 0, 2024-04-25T14:16:29.280 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_call": 0, 2024-04-25T14:16:29.280 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_watch": 0, 2024-04-25T14:16:29.280 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_notify": 0, 2024-04-25T14:16:29.280 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T14:16:29.280 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls": 0, 2024-04-25T14:16:29.280 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls_filter": 0, 2024-04-25T14:16:29.280 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_other": 0, 2024-04-25T14:16:29.281 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_active": 0, 2024-04-25T14:16:29.281 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_send": 0, 2024-04-25T14:16:29.281 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_resend": 0, 2024-04-25T14:16:29.281 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_ping": 0, 2024-04-25T14:16:29.281 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_active": 0, 2024-04-25T14:16:29.281 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_send": 0, 2024-04-25T14:16:29.281 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_resend": 0, 2024-04-25T14:16:29.281 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_active": 0, 2024-04-25T14:16:29.281 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_send": 0, 2024-04-25T14:16:29.281 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_resend": 0, 2024-04-25T14:16:29.281 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_active": 0, 2024-04-25T14:16:29.281 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_send": 0, 2024-04-25T14:16:29.282 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_resend": 0, 2024-04-25T14:16:29.282 INFO:teuthology.orchestra.run.smithi151.stdout: "command_active": 0, 2024-04-25T14:16:29.282 INFO:teuthology.orchestra.run.smithi151.stdout: "command_send": 0, 2024-04-25T14:16:29.282 INFO:teuthology.orchestra.run.smithi151.stdout: "command_resend": 0, 2024-04-25T14:16:29.282 INFO:teuthology.orchestra.run.smithi151.stdout: "map_epoch": 78, 2024-04-25T14:16:29.282 INFO:teuthology.orchestra.run.smithi151.stdout: "map_full": 0, 2024-04-25T14:16:29.282 INFO:teuthology.orchestra.run.smithi151.stdout: "map_inc": 2, 2024-04-25T14:16:29.282 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_sessions": 9, 2024-04-25T14:16:29.282 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_open": 9, 2024-04-25T14:16:29.282 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_close": 0, 2024-04-25T14:16:29.282 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_laggy": 0, 2024-04-25T14:16:29.283 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_wr": 14, 2024-04-25T14:16:29.283 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_rd": 0, 2024-04-25T14:16:29.283 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_del": 1 2024-04-25T14:16:29.283 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.283 INFO:teuthology.orchestra.run.smithi151.stdout: "oft": { 2024-04-25T14:16:29.283 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_objs": 1, 2024-04-25T14:16:29.283 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_kv_pairs": 0, 2024-04-25T14:16:29.283 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_updates": 0, 2024-04-25T14:16:29.283 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_removes": 0 2024-04-25T14:16:29.283 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.283 INFO:teuthology.orchestra.run.smithi151.stdout: "osd-slow-ops": { 2024-04-25T14:16:29.283 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_ops_count": 0 2024-04-25T14:16:29.283 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.284 INFO:teuthology.orchestra.run.smithi151.stdout: "purge_queue": { 2024-04-25T14:16:29.284 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops": 0, 2024-04-25T14:16:29.284 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T14:16:29.284 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing": 0, 2024-04-25T14:16:29.284 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_high_water": 0, 2024-04-25T14:16:29.284 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed_ops": 0, 2024-04-25T14:16:29.284 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed": 0, 2024-04-25T14:16:29.284 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_item_in_journal": 0 2024-04-25T14:16:29.284 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.284 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:16:29.284 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-04-25T14:16:29.284 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-04-25T14:16:29.284 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-04-25T14:16:29.285 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 1004, 2024-04-25T14:16:29.285 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 689826, 2024-04-25T14:16:29.285 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-04-25T14:16:29.285 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 1004, 2024-04-25T14:16:29.285 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-04-25T14:16:29.285 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-04-25T14:16:29.285 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 1004, 2024-04-25T14:16:29.285 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 689826, 2024-04-25T14:16:29.285 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-04-25T14:16:29.285 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:29.285 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:29.285 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:29.285 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-25T14:16:29.286 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.286 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_bytes": { 2024-04-25T14:16:29.286 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-04-25T14:16:29.286 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-04-25T14:16:29.286 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-04-25T14:16:29.286 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-04-25T14:16:29.286 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-04-25T14:16:29.286 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-04-25T14:16:29.286 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-04-25T14:16:29.286 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 35, 2024-04-25T14:16:29.286 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 10710, 2024-04-25T14:16:29.286 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 35, 2024-04-25T14:16:29.286 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 10710, 2024-04-25T14:16:29.287 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-04-25T14:16:29.287 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:29.287 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:29.287 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:29.287 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-25T14:16:29.287 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.287 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_ops": { 2024-04-25T14:16:29.287 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-04-25T14:16:29.287 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 1024, 2024-04-25T14:16:29.287 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-04-25T14:16:29.287 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-04-25T14:16:29.287 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-04-25T14:16:29.287 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-04-25T14:16:29.288 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-04-25T14:16:29.288 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 35, 2024-04-25T14:16:29.288 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 35, 2024-04-25T14:16:29.288 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 35, 2024-04-25T14:16:29.288 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 35, 2024-04-25T14:16:29.288 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-04-25T14:16:29.288 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:29.288 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:29.288 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:29.288 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-25T14:16:29.288 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.288 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle": { 2024-04-25T14:16:29.288 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-04-25T14:16:29.288 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-04-25T14:16:29.289 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-04-25T14:16:29.289 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-04-25T14:16:29.289 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-04-25T14:16:29.289 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-04-25T14:16:29.289 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-04-25T14:16:29.289 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-04-25T14:16:29.289 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-04-25T14:16:29.289 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 0, 2024-04-25T14:16:29.289 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 0, 2024-04-25T14:16:29.289 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-04-25T14:16:29.289 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:29.289 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:29.289 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:29.290 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-25T14:16:29.290 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-25T14:16:29.290 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle-0x5635d1f3a5b0": { 2024-04-25T14:16:29.290 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-04-25T14:16:29.290 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-04-25T14:16:29.290 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-04-25T14:16:29.290 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 2, 2024-04-25T14:16:29.290 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 588, 2024-04-25T14:16:29.290 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-04-25T14:16:29.290 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 2, 2024-04-25T14:16:29.290 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-04-25T14:16:29.290 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-04-25T14:16:29.290 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 1, 2024-04-25T14:16:29.290 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 588, 2024-04-25T14:16:29.291 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-04-25T14:16:29.291 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-25T14:16:29.291 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-25T14:16:29.291 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-25T14:16:29.291 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-25T14:16:29.291 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-25T14:16:29.291 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-25T14:16:30.036 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-04-25T14:16:30.036 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-04-25T14:16:30.037 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-04-25T14:16:30.037 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=10 2024-04-25T14:16:30.037 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T14:16:30.048 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T14:16:30.048 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T14:16:30.048 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:16:30.069 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T14:16:30.069 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:16:30.128 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:16:30.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.288+0000 7faf80042640 1 -- 172.21.15.96:0/1255539895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf78103b00 msgr2=0x7faf78104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:30.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.288+0000 7faf80042640 1 --2- 172.21.15.96:0/1255539895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf78103b00 0x7faf78104800 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7faf74009900 tx=0x7faf7402a410 comp rx=0 tx=0).stop 2024-04-25T14:16:30.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.288+0000 7faf80042640 1 -- 172.21.15.96:0/1255539895 shutdown_connections 2024-04-25T14:16:30.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.288+0000 7faf80042640 1 --2- 172.21.15.96:0/1255539895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf78109a40 0x7faf7810be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:30.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.288+0000 7faf80042640 1 --2- 172.21.15.96:0/1255539895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf78104d40 0x7faf78105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:30.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.288+0000 7faf80042640 1 --2- 172.21.15.96:0/1255539895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf78103b00 0x7faf78104800 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:30.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.288+0000 7faf80042640 1 -- 172.21.15.96:0/1255539895 >> 172.21.15.96:0/1255539895 conn(0x7faf780fed10 msgr2=0x7faf781011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:30.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.289+0000 7faf80042640 1 -- 172.21.15.96:0/1255539895 shutdown_connections 2024-04-25T14:16:30.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.289+0000 7faf80042640 1 -- 172.21.15.96:0/1255539895 wait complete. 2024-04-25T14:16:30.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.289+0000 7faf80042640 1 Processor -- start 2024-04-25T14:16:30.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.289+0000 7faf80042640 1 -- start start 2024-04-25T14:16:30.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.289+0000 7faf80042640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf78103b00 0x7faf7819c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:30.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.289+0000 7faf80042640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf78104d40 0x7faf7819c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:30.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.289+0000 7faf80042640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf78109a40 0x7faf781a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:30.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.289+0000 7faf80042640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faf780769d0 con 0x7faf78103b00 2024-04-25T14:16:30.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.289+0000 7faf80042640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7faf78076850 con 0x7faf78104d40 2024-04-25T14:16:30.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.289+0000 7faf80042640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faf78076b50 con 0x7faf78109a40 2024-04-25T14:16:30.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.289+0000 7faf7ddde640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf78103b00 0x7faf7819c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:30.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.289+0000 7faf7ddde640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf78103b00 0x7faf7819c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41578/0 (socket says 172.21.15.96:41578) 2024-04-25T14:16:30.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.289+0000 7faf7ddde640 1 -- 172.21.15.96:0/3389877343 learned_addr learned my addr 172.21.15.96:0/3389877343 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:16:30.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.290+0000 7faf7e5df640 1 --2- 172.21.15.96:0/3389877343 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf78109a40 0x7faf781a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:30.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.290+0000 7faf7d5dd640 1 --2- 172.21.15.96:0/3389877343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf78104d40 0x7faf7819c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:30.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.290+0000 7faf7ddde640 1 -- 172.21.15.96:0/3389877343 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf78109a40 msgr2=0x7faf781a38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:30.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.290+0000 7faf7ddde640 1 --2- 172.21.15.96:0/3389877343 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf78109a40 0x7faf781a38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:30.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.290+0000 7faf7ddde640 1 -- 172.21.15.96:0/3389877343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf78104d40 msgr2=0x7faf7819c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:30.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.290+0000 7faf7ddde640 1 --2- 172.21.15.96:0/3389877343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf78104d40 0x7faf7819c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:30.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.290+0000 7faf7ddde640 1 -- 172.21.15.96:0/3389877343 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf781a3f40 con 0x7faf78103b00 2024-04-25T14:16:30.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.290+0000 7faf7ddde640 1 --2- 172.21.15.96:0/3389877343 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf78103b00 0x7faf7819c400 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7faf74006010 tx=0x7faf74031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:30.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.290+0000 7faf6effd640 1 -- 172.21.15.96:0/3389877343 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7403f030 con 0x7faf78103b00 2024-04-25T14:16:30.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.290+0000 7faf80042640 1 -- 172.21.15.96:0/3389877343 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf781a41d0 con 0x7faf78103b00 2024-04-25T14:16:30.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.290+0000 7faf6effd640 1 -- 172.21.15.96:0/3389877343 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf74003aa0 con 0x7faf78103b00 2024-04-25T14:16:30.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.290+0000 7faf80042640 1 -- 172.21.15.96:0/3389877343 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf781a4710 con 0x7faf78103b00 2024-04-25T14:16:30.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.290+0000 7faf6effd640 1 -- 172.21.15.96:0/3389877343 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf74035070 con 0x7faf78103b00 2024-04-25T14:16:30.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.291+0000 7faf80042640 1 -- 172.21.15.96:0/3389877343 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf7806a190 con 0x7faf78103b00 2024-04-25T14:16:30.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.293+0000 7faf6effd640 1 -- 172.21.15.96:0/3389877343 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7faf74042030 con 0x7faf78103b00 2024-04-25T14:16:30.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.293+0000 7faf6effd640 1 --2- 172.21.15.96:0/3389877343 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7faf640afaf0 0x7faf640b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:30.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.293+0000 7faf7d5dd640 1 --2- 172.21.15.96:0/3389877343 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7faf640afaf0 0x7faf640b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:30.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.293+0000 7faf6effd640 1 -- 172.21.15.96:0/3389877343 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7faf740ff610 con 0x7faf78103b00 2024-04-25T14:16:30.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.294+0000 7faf7d5dd640 1 --2- 172.21.15.96:0/3389877343 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7faf640afaf0 0x7faf640b2000 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7faf68005d00 tx=0x7faf68005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:30.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.297+0000 7faf6effd640 1 -- 172.21.15.96:0/3389877343 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faf740c81d0 con 0x7faf78103b00 2024-04-25T14:16:30.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.476+0000 7faf80042640 1 -- 172.21.15.96:0/3389877343 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faf781a4b50 con 0x7faf78103b00 2024-04-25T14:16:30.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.477+0000 7faf6effd640 1 -- 172.21.15.96:0/3389877343 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22616 (secure 0 0 0) 0x7faf740c7ad0 con 0x7faf78103b00 2024-04-25T14:16:30.477 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:16:30.477 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":78,"fsid":"59334182-030c-11ef-bc93-c7b262605968","created":"2024-04-25T14:03:25.680745+0000","modified":"2024-04-25T14:12:22.496143+0000","last_up_change":"2024-04-25T14:10:35.894312+0000","last_in_change":"2024-04-25T14:10:15.583440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:06:47.402927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:11:27.434331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:11:27.854004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cf4ba3ac-142e-4e8e-96d4-410d77157eb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6803","nonce":3832506168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6805","nonce":3832506168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832506168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6807","nonce":3832506168}]},"public_addr":"172.21.15.96:6803/3832506168","cluster_addr":"172.21.15.96:6805/3832506168","heartbeat_back_addr":"172.21.15.96:6809/3832506168","heartbeat_front_addr":"172.21.15.96:6807/3832506168","state":["exists","up"]},{"osd":1,"uuid":"866e73b9-4a26-48ba-badf-704fe22f71cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6801","nonce":1083311494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6803","nonce":1083311494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6807","nonce":1083311494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6805","nonce":1083311494}]},"public_addr":"172.21.15.97:6801/1083311494","cluster_addr":"172.21.15.97:6803/1083311494","heartbeat_back_addr":"172.21.15.97:6807/1083311494","heartbeat_front_addr":"172.21.15.97:6805/1083311494","state":["exists","up"]},{"osd":2,"uuid":"45c79ecb-abc4-4dba-bae8-74804e4bc1bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6801","nonce":981902966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6803","nonce":981902966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6807","nonce":981902966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6805","nonce":981902966}]},"public_addr":"172.21.15.151:6801/981902966","cluster_addr":"172.21.15.151:6803/981902966","heartbeat_back_addr":"172.21.15.151:6807/981902966","heartbeat_front_addr":"172.21.15.151:6805/981902966","state":["exists","up"]},{"osd":3,"uuid":"9fc39468-a7ea-4ccf-b365-335ad2ea4c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6811","nonce":2274158748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6813","nonce":2274158748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6817","nonce":2274158748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6815","nonce":2274158748}]},"public_addr":"172.21.15.96:6811/2274158748","cluster_addr":"172.21.15.96:6813/2274158748","heartbeat_back_addr":"172.21.15.96:6817/2274158748","heartbeat_front_addr":"172.21.15.96:6815/2274158748","state":["exists","up"]},{"osd":4,"uuid":"616582f1-12d2-456b-b7b3-e5517800e8a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6809","nonce":3727031962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6811","nonce":3727031962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6815","nonce":3727031962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6813","nonce":3727031962}]},"public_addr":"172.21.15.97:6809/3727031962","cluster_addr":"172.21.15.97:6811/3727031962","heartbeat_back_addr":"172.21.15.97:6815/3727031962","heartbeat_front_addr":"172.21.15.97:6813/3727031962","state":["exists","up"]},{"osd":5,"uuid":"3d277784-ab0f-4c90-81a1-2a4034c52a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6809","nonce":223882546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6811","nonce":223882546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6815","nonce":223882546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6813","nonce":223882546}]},"public_addr":"172.21.15.151:6809/223882546","cluster_addr":"172.21.15.151:6811/223882546","heartbeat_back_addr":"172.21.15.151:6815/223882546","heartbeat_front_addr":"172.21.15.151:6813/223882546","state":["exists","up"]},{"osd":6,"uuid":"9a9ad585-cb1a-4239-9d18-055f59164049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6819","nonce":96197407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6821","nonce":96197407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6825","nonce":96197407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6823","nonce":96197407}]},"public_addr":"172.21.15.96:6819/96197407","cluster_addr":"172.21.15.96:6821/96197407","heartbeat_back_addr":"172.21.15.96:6825/96197407","heartbeat_front_addr":"172.21.15.96:6823/96197407","state":["exists","up"]},{"osd":7,"uuid":"303551c8-6af0-42f2-844f-45bf8db5baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6817","nonce":2247216400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6819","nonce":2247216400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6823","nonce":2247216400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6821","nonce":2247216400}]},"public_addr":"172.21.15.97:6817/2247216400","cluster_addr":"172.21.15.97:6819/2247216400","heartbeat_back_addr":"172.21.15.97:6823/2247216400","heartbeat_front_addr":"172.21.15.97:6821/2247216400","state":["exists","up"]},{"osd":8,"uuid":"81f6cc66-e4ca-430c-b336-b6678d968818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6817","nonce":2549822185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6819","nonce":2549822185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6823","nonce":2549822185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6821","nonce":2549822185}]},"public_addr":"172.21.15.151:6817/2549822185","cluster_addr":"172.21.15.151:6819/2549822185","heartbeat_back_addr":"172.21.15.151:6823/2549822185","heartbeat_front_addr":"172.21.15.151:6821/2549822185","state":["exists","up"]},{"osd":9,"uuid":"29d4da16-feab-4d1c-9d1d-de4f799bd724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6827","nonce":1099399643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6829","nonce":1099399643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6833","nonce":1099399643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6831","nonce":1099399643}]},"public_addr":"172.21.15.96:6827/1099399643","cluster_addr":"172.21.15.96:6829/1099399643","heartbeat_back_addr":"172.21.15.96:6833/1099399643","heartbeat_front_addr":"172.21.15.96:6831/1099399643","state":["exists","up"]},{"osd":10,"uuid":"197fee9b-eee4-4893-be1c-3d57f42250e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6825","nonce":155754520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6827","nonce":155754520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6831","nonce":155754520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6829","nonce":155754520}]},"public_addr":"172.21.15.97:6825/155754520","cluster_addr":"172.21.15.97:6827/155754520","heartbeat_back_addr":"172.21.15.97:6831/155754520","heartbeat_front_addr":"172.21.15.97:6829/155754520","state":["exists","up"]},{"osd":11,"uuid":"9c1d0a83-fc12-41a2-8eb0-80c4abdd2770","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6825","nonce":2027958746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6827","nonce":2027958746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6831","nonce":2027958746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6829","nonce":2027958746}]},"public_addr":"172.21.15.151:6825/2027958746","cluster_addr":"172.21.15.151:6827/2027958746","heartbeat_back_addr":"172.21.15.151:6831/2027958746","heartbeat_front_addr":"172.21.15.151:6829/2027958746","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:05:56.177390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:06:19.917663+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:06:43.608052+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:08.540382+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:33.474072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:58.242306+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:08:24.120442+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:08:49.661064+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:09:15.097271+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:09:41.904505+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:10:08.233252+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:10:33.712296+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/3839807484":"2024-04-26T14:04:21.307574+0000","172.21.15.96:0/1495237551":"2024-04-26T14:04:21.307574+0000","172.21.15.96:0/2793751450":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/129575834":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/2233075139":"2024-04-26T14:04:21.307574+0000","172.21.15.96:6801/2348397481":"2024-04-26T14:03:54.249537+0000","172.21.15.96:6801/433591062":"2024-04-26T14:04:21.307574+0000","172.21.15.96:6800/2348397481":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/3517674924":"2024-04-26T14:03:54.249537+0000","172.21.15.96:6800/433591062":"2024-04-26T14:04:21.307574+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:16:30.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.479+0000 7faf80042640 1 -- 172.21.15.96:0/3389877343 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7faf640afaf0 msgr2=0x7faf640b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:30.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.479+0000 7faf80042640 1 --2- 172.21.15.96:0/3389877343 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7faf640afaf0 0x7faf640b2000 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7faf68005d00 tx=0x7faf68005c70 comp rx=0 tx=0).stop 2024-04-25T14:16:30.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.479+0000 7faf80042640 1 -- 172.21.15.96:0/3389877343 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf78103b00 msgr2=0x7faf7819c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:30.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.479+0000 7faf80042640 1 --2- 172.21.15.96:0/3389877343 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf78103b00 0x7faf7819c400 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7faf74006010 tx=0x7faf74031a50 comp rx=0 tx=0).stop 2024-04-25T14:16:30.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.479+0000 7faf80042640 1 -- 172.21.15.96:0/3389877343 shutdown_connections 2024-04-25T14:16:30.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.479+0000 7faf80042640 1 --2- 172.21.15.96:0/3389877343 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf78109a40 0x7faf781a38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:30.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.479+0000 7faf80042640 1 --2- 172.21.15.96:0/3389877343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf78104d40 0x7faf7819c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:30.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.479+0000 7faf80042640 1 --2- 172.21.15.96:0/3389877343 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7faf640afaf0 0x7faf640b2000 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:30.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.479+0000 7faf80042640 1 --2- 172.21.15.96:0/3389877343 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf78103b00 0x7faf7819c400 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:30.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.479+0000 7faf80042640 1 -- 172.21.15.96:0/3389877343 >> 172.21.15.96:0/3389877343 conn(0x7faf780fed10 msgr2=0x7faf78101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:30.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.479+0000 7faf80042640 1 -- 172.21.15.96:0/3389877343 shutdown_connections 2024-04-25T14:16:30.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.479+0000 7faf80042640 1 -- 172.21.15.96:0/3389877343 wait complete. 2024-04-25T14:16:30.491 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:06:47.402927+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:11:27.434331+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:11:27.854004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:16:30.492 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:16:30.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.604+0000 7fc79fef2640 1 -- 172.21.15.96:0/1725914052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc798103af0 msgr2=0x7fc7981047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:30.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.604+0000 7fc79fef2640 1 --2- 172.21.15.96:0/1725914052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc798103af0 0x7fc7981047f0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fc78c009900 tx=0x7fc78c02a410 comp rx=0 tx=0).stop 2024-04-25T14:16:30.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.604+0000 7fc79fef2640 1 -- 172.21.15.96:0/1725914052 shutdown_connections 2024-04-25T14:16:30.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.604+0000 7fc79fef2640 1 --2- 172.21.15.96:0/1725914052 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc798109a30 0x7fc79810be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:30.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.604+0000 7fc79fef2640 1 --2- 172.21.15.96:0/1725914052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc798104d30 0x7fc798105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:30.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.604+0000 7fc79fef2640 1 --2- 172.21.15.96:0/1725914052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc798103af0 0x7fc7981047f0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:30.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.604+0000 7fc79fef2640 1 -- 172.21.15.96:0/1725914052 >> 172.21.15.96:0/1725914052 conn(0x7fc7980fece0 msgr2=0x7fc798101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:30.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.604+0000 7fc79fef2640 1 -- 172.21.15.96:0/1725914052 shutdown_connections 2024-04-25T14:16:30.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.604+0000 7fc79fef2640 1 -- 172.21.15.96:0/1725914052 wait complete. 2024-04-25T14:16:30.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.605+0000 7fc79fef2640 1 Processor -- start 2024-04-25T14:16:30.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.605+0000 7fc79fef2640 1 -- start start 2024-04-25T14:16:30.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.605+0000 7fc79fef2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc798103af0 0x7fc79819c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:30.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.605+0000 7fc79fef2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc798104d30 0x7fc79819c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:30.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.605+0000 7fc79fef2640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc798109a30 0x7fc7981a3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:30.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.605+0000 7fc79fef2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc79810f5f0 con 0x7fc798104d30 2024-04-25T14:16:30.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.605+0000 7fc79fef2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc79810f470 con 0x7fc798103af0 2024-04-25T14:16:30.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.605+0000 7fc79fef2640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc79810f770 con 0x7fc798109a30 2024-04-25T14:16:30.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.605+0000 7fc79d48d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc798104d30 0x7fc79819c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:30.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.605+0000 7fc79d48d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc798104d30 0x7fc79819c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41590/0 (socket says 172.21.15.96:41590) 2024-04-25T14:16:30.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.605+0000 7fc79d48d640 1 -- 172.21.15.96:0/1640507484 learned_addr learned my addr 172.21.15.96:0/1640507484 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:16:30.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.605+0000 7fc79e48f640 1 --2- 172.21.15.96:0/1640507484 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc798109a30 0x7fc7981a3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:30.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.605+0000 7fc79dc8e640 1 --2- 172.21.15.96:0/1640507484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc798103af0 0x7fc79819c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:30.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.605+0000 7fc79d48d640 1 -- 172.21.15.96:0/1640507484 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc798109a30 msgr2=0x7fc7981a3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:30.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.605+0000 7fc79d48d640 1 --2- 172.21.15.96:0/1640507484 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc798109a30 0x7fc7981a3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:30.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.605+0000 7fc79d48d640 1 -- 172.21.15.96:0/1640507484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc798103af0 msgr2=0x7fc79819c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:30.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.605+0000 7fc79d48d640 1 --2- 172.21.15.96:0/1640507484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc798103af0 0x7fc79819c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:30.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.605+0000 7fc79d48d640 1 -- 172.21.15.96:0/1640507484 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7981a3fc0 con 0x7fc798104d30 2024-04-25T14:16:30.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.605+0000 7fc79dc8e640 1 --2- 172.21.15.96:0/1640507484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc798103af0 0x7fc79819c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:16:30.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.605+0000 7fc79d48d640 1 --2- 172.21.15.96:0/1640507484 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc798104d30 0x7fc79819c9c0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fc79400d950 tx=0x7fc79400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:30.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.606+0000 7fc78affd640 1 -- 172.21.15.96:0/1640507484 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc794017070 con 0x7fc798104d30 2024-04-25T14:16:30.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.606+0000 7fc78affd640 1 -- 172.21.15.96:0/1640507484 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc794009c30 con 0x7fc798104d30 2024-04-25T14:16:30.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.606+0000 7fc79fef2640 1 -- 172.21.15.96:0/1640507484 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7981a42b0 con 0x7fc798104d30 2024-04-25T14:16:30.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.606+0000 7fc78affd640 1 -- 172.21.15.96:0/1640507484 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79400f620 con 0x7fc798104d30 2024-04-25T14:16:30.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.606+0000 7fc79fef2640 1 -- 172.21.15.96:0/1640507484 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7981a4800 con 0x7fc798104d30 2024-04-25T14:16:30.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.607+0000 7fc79fef2640 1 -- 172.21.15.96:0/1640507484 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7981a4b20 con 0x7fc798104d30 2024-04-25T14:16:30.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.608+0000 7fc78affd640 1 -- 172.21.15.96:0/1640507484 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fc794010460 con 0x7fc798104d30 2024-04-25T14:16:30.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.609+0000 7fc78affd640 1 --2- 172.21.15.96:0/1640507484 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fc7840afaf0 0x7fc7840b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:30.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.609+0000 7fc79dc8e640 1 --2- 172.21.15.96:0/1640507484 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fc7840afaf0 0x7fc7840b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:30.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.609+0000 7fc78affd640 1 -- 172.21.15.96:0/1640507484 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fc7940de820 con 0x7fc798104d30 2024-04-25T14:16:30.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.609+0000 7fc79dc8e640 1 --2- 172.21.15.96:0/1640507484 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fc7840afaf0 0x7fc7840b2000 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc78c009230 tx=0x7fc78c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:30.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.613+0000 7fc78affd640 1 -- 172.21.15.96:0/1640507484 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc7940a73e0 con 0x7fc798104d30 2024-04-25T14:16:30.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.791+0000 7fc79fef2640 1 -- 172.21.15.96:0/1640507484 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc7981a4cf0 con 0x7fc798104d30 2024-04-25T14:16:30.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.792+0000 7fc78affd640 1 -- 172.21.15.96:0/1640507484 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) ==== 93+0+10 (secure 0 0 0) 0x7fc794009de0 con 0x7fc798104d30 2024-04-25T14:16:30.792 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-04-25T14:16:30.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.794+0000 7fc79fef2640 1 -- 172.21.15.96:0/1640507484 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fc7840afaf0 msgr2=0x7fc7840b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:30.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.794+0000 7fc79fef2640 1 --2- 172.21.15.96:0/1640507484 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fc7840afaf0 0x7fc7840b2000 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc78c009230 tx=0x7fc78c000bc0 comp rx=0 tx=0).stop 2024-04-25T14:16:30.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.794+0000 7fc79fef2640 1 -- 172.21.15.96:0/1640507484 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc798104d30 msgr2=0x7fc79819c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:30.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.794+0000 7fc79fef2640 1 --2- 172.21.15.96:0/1640507484 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc798104d30 0x7fc79819c9c0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fc79400d950 tx=0x7fc79400de10 comp rx=0 tx=0).stop 2024-04-25T14:16:30.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.794+0000 7fc79fef2640 1 -- 172.21.15.96:0/1640507484 shutdown_connections 2024-04-25T14:16:30.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.794+0000 7fc79fef2640 1 --2- 172.21.15.96:0/1640507484 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc798109a30 0x7fc7981a3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:30.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.794+0000 7fc79fef2640 1 --2- 172.21.15.96:0/1640507484 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fc7840afaf0 0x7fc7840b2000 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:30.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.794+0000 7fc79fef2640 1 --2- 172.21.15.96:0/1640507484 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc798104d30 0x7fc79819c9c0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:30.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.794+0000 7fc79fef2640 1 --2- 172.21.15.96:0/1640507484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc798103af0 0x7fc79819c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:30.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.794+0000 7fc79fef2640 1 -- 172.21.15.96:0/1640507484 >> 172.21.15.96:0/1640507484 conn(0x7fc7980fece0 msgr2=0x7fc798101100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:30.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.794+0000 7fc79fef2640 1 -- 172.21.15.96:0/1640507484 shutdown_connections 2024-04-25T14:16:30.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.794+0000 7fc79fef2640 1 -- 172.21.15.96:0/1640507484 wait complete. 2024-04-25T14:16:30.804 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:30 smithi096 ceph-mon[29207]: pgmap v432: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T14:16:30.804 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:30 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/3389877343' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:16:30.805 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:16:30.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.915+0000 7f21799ba640 1 -- 172.21.15.96:0/2674738159 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2174101760 msgr2=0x7f2174101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:30.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.915+0000 7f21799ba640 1 --2- 172.21.15.96:0/2674738159 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2174101760 0x7f2174101bd0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f2168009900 tx=0x7f216802a410 comp rx=0 tx=0).stop 2024-04-25T14:16:30.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.915+0000 7f21799ba640 1 -- 172.21.15.96:0/2674738159 shutdown_connections 2024-04-25T14:16:30.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.915+0000 7f21799ba640 1 --2- 172.21.15.96:0/2674738159 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2174102260 0x7f217410bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:30.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.915+0000 7f21799ba640 1 --2- 172.21.15.96:0/2674738159 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2174101760 0x7f2174101bd0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:30.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.915+0000 7f21799ba640 1 --2- 172.21.15.96:0/2674738159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21741050b0 0x7f21741054c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:30.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.915+0000 7f21799ba640 1 -- 172.21.15.96:0/2674738159 >> 172.21.15.96:0/2674738159 conn(0x7f21740fd380 msgr2=0x7f21740ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:30.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.915+0000 7f21799ba640 1 -- 172.21.15.96:0/2674738159 shutdown_connections 2024-04-25T14:16:30.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.915+0000 7f21799ba640 1 -- 172.21.15.96:0/2674738159 wait complete. 2024-04-25T14:16:30.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.916+0000 7f21799ba640 1 Processor -- start 2024-04-25T14:16:30.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.916+0000 7f21799ba640 1 -- start start 2024-04-25T14:16:30.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.916+0000 7f21799ba640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2174101760 0x7f217419a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:30.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.916+0000 7f21799ba640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2174102260 0x7f217419a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:30.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.916+0000 7f21799ba640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21741050b0 0x7f21741a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:30.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.916+0000 7f21799ba640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f217410f620 con 0x7f21741050b0 2024-04-25T14:16:30.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.916+0000 7f21799ba640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f217410f4a0 con 0x7f2174101760 2024-04-25T14:16:30.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.916+0000 7f21799ba640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f217410f7a0 con 0x7f2174102260 2024-04-25T14:16:30.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.916+0000 7f21737fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21741050b0 0x7f21741a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:30.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.916+0000 7f21737fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21741050b0 0x7f21741a1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41602/0 (socket says 172.21.15.96:41602) 2024-04-25T14:16:30.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.916+0000 7f21737fe640 1 -- 172.21.15.96:0/1900336374 learned_addr learned my addr 172.21.15.96:0/1900336374 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:16:30.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.917+0000 7f2172ffd640 1 --2- 172.21.15.96:0/1900336374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2174101760 0x7f217419a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:30.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.917+0000 7f21727fc640 1 --2- 172.21.15.96:0/1900336374 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2174102260 0x7f217419a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:30.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.917+0000 7f21737fe640 1 -- 172.21.15.96:0/1900336374 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2174102260 msgr2=0x7f217419a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:30.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.917+0000 7f21737fe640 1 --2- 172.21.15.96:0/1900336374 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2174102260 0x7f217419a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:30.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.917+0000 7f21737fe640 1 -- 172.21.15.96:0/1900336374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2174101760 msgr2=0x7f217419a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:30.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.917+0000 7f21737fe640 1 --2- 172.21.15.96:0/1900336374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2174101760 0x7f217419a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:30.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.917+0000 7f21737fe640 1 -- 172.21.15.96:0/1900336374 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21741a1dc0 con 0x7f21741050b0 2024-04-25T14:16:30.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.917+0000 7f21737fe640 1 --2- 172.21.15.96:0/1900336374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21741050b0 0x7f21741a1750 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f216c00d950 tx=0x7f216c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:30.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.917+0000 7f21789b8640 1 -- 172.21.15.96:0/1900336374 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f216c017030 con 0x7f21741050b0 2024-04-25T14:16:30.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.917+0000 7f21789b8640 1 -- 172.21.15.96:0/1900336374 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f216c004020 con 0x7f21741050b0 2024-04-25T14:16:30.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.917+0000 7f21799ba640 1 -- 172.21.15.96:0/1900336374 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21741a20b0 con 0x7f21741050b0 2024-04-25T14:16:30.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.917+0000 7f21789b8640 1 -- 172.21.15.96:0/1900336374 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f216c003b50 con 0x7f21741050b0 2024-04-25T14:16:30.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.917+0000 7f21799ba640 1 -- 172.21.15.96:0/1900336374 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21741a25b0 con 0x7f21741050b0 2024-04-25T14:16:30.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.918+0000 7f21799ba640 1 -- 172.21.15.96:0/1900336374 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2174108a40 con 0x7f21741050b0 2024-04-25T14:16:30.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.920+0000 7f21789b8640 1 -- 172.21.15.96:0/1900336374 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f216c01a070 con 0x7f21741050b0 2024-04-25T14:16:30.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.920+0000 7f21789b8640 1 --2- 172.21.15.96:0/1900336374 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f215c0afaf0 0x7f215c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:30.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.920+0000 7f2172ffd640 1 --2- 172.21.15.96:0/1900336374 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f215c0afaf0 0x7f215c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:30.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.920+0000 7f21789b8640 1 -- 172.21.15.96:0/1900336374 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f216c0de860 con 0x7f21741050b0 2024-04-25T14:16:30.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.921+0000 7f2172ffd640 1 --2- 172.21.15.96:0/1900336374 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f215c0afaf0 0x7f215c0b2000 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f217406b2d0 tx=0x7f21640097f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:30.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:30 smithi097 ceph-mon[32720]: pgmap v432: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T14:16:30.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:30 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/3389877343' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:16:30.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:30.924+0000 7f21789b8640 1 -- 172.21.15.96:0/1900336374 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f216c021070 con 0x7f21741050b0 2024-04-25T14:16:30.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:30 smithi151 ceph-mon[32990]: pgmap v432: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T14:16:30.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:30 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/3389877343' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:16:31.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.101+0000 7f21799ba640 1 -- 172.21.15.96:0/1900336374 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f21740637a0 con 0x7f21741050b0 2024-04-25T14:16:31.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.102+0000 7f21789b8640 1 -- 172.21.15.96:0/1900336374 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v78) ==== 104+0+11 (secure 0 0 0) 0x7f216c0a7420 con 0x7f21741050b0 2024-04-25T14:16:31.102 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-04-25T14:16:31.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.103+0000 7f21799ba640 1 -- 172.21.15.96:0/1900336374 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f215c0afaf0 msgr2=0x7f215c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:31.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.103+0000 7f21799ba640 1 --2- 172.21.15.96:0/1900336374 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f215c0afaf0 0x7f215c0b2000 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f217406b2d0 tx=0x7f21640097f0 comp rx=0 tx=0).stop 2024-04-25T14:16:31.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.103+0000 7f21799ba640 1 -- 172.21.15.96:0/1900336374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21741050b0 msgr2=0x7f21741a1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:31.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.103+0000 7f21799ba640 1 --2- 172.21.15.96:0/1900336374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21741050b0 0x7f21741a1750 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f216c00d950 tx=0x7f216c00de10 comp rx=0 tx=0).stop 2024-04-25T14:16:31.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.103+0000 7f21799ba640 1 -- 172.21.15.96:0/1900336374 shutdown_connections 2024-04-25T14:16:31.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.103+0000 7f21799ba640 1 --2- 172.21.15.96:0/1900336374 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f215c0afaf0 0x7f215c0b2000 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:31.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.103+0000 7f21799ba640 1 --2- 172.21.15.96:0/1900336374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21741050b0 0x7f21741a1750 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:31.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.103+0000 7f21799ba640 1 --2- 172.21.15.96:0/1900336374 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2174102260 0x7f217419a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:31.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.103+0000 7f21799ba640 1 --2- 172.21.15.96:0/1900336374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2174101760 0x7f217419a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:31.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.103+0000 7f21799ba640 1 -- 172.21.15.96:0/1900336374 >> 172.21.15.96:0/1900336374 conn(0x7f21740fd380 msgr2=0x7f2174105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:31.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.104+0000 7f21799ba640 1 -- 172.21.15.96:0/1900336374 shutdown_connections 2024-04-25T14:16:31.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.104+0000 7f21799ba640 1 -- 172.21.15.96:0/1900336374 wait complete. 2024-04-25T14:16:31.115 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:16:31.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.226+0000 7f0751346640 1 -- 172.21.15.96:0/4179815764 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f074c1029e0 msgr2=0x7f074c1036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:31.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.226+0000 7f0751346640 1 --2- 172.21.15.96:0/4179815764 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f074c1029e0 0x7f074c1036e0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f073c009900 tx=0x7f073c02a410 comp rx=0 tx=0).stop 2024-04-25T14:16:31.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.226+0000 7f0751346640 1 -- 172.21.15.96:0/4179815764 shutdown_connections 2024-04-25T14:16:31.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.226+0000 7f0751346640 1 --2- 172.21.15.96:0/4179815764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f074c108920 0x7f074c10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:31.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.226+0000 7f0751346640 1 --2- 172.21.15.96:0/4179815764 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f074c103c20 0x7f074c104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:31.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.226+0000 7f0751346640 1 --2- 172.21.15.96:0/4179815764 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f074c1029e0 0x7f074c1036e0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:31.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.226+0000 7f0751346640 1 -- 172.21.15.96:0/4179815764 >> 172.21.15.96:0/4179815764 conn(0x7f074c0fdbd0 msgr2=0x7f074c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:31.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.226+0000 7f0751346640 1 -- 172.21.15.96:0/4179815764 shutdown_connections 2024-04-25T14:16:31.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.226+0000 7f0751346640 1 -- 172.21.15.96:0/4179815764 wait complete. 2024-04-25T14:16:31.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.227+0000 7f0751346640 1 Processor -- start 2024-04-25T14:16:31.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.227+0000 7f0751346640 1 -- start start 2024-04-25T14:16:31.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.227+0000 7f0751346640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f074c1029e0 0x7f074c19a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:31.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.227+0000 7f0751346640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f074c103c20 0x7f074c19a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:31.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.227+0000 7f0751346640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f074c108920 0x7f074c1a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:31.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.227+0000 7f0751346640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f074c10f610 con 0x7f074c103c20 2024-04-25T14:16:31.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.227+0000 7f0751346640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f074c10f490 con 0x7f074c1029e0 2024-04-25T14:16:31.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.227+0000 7f0751346640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f074c10f790 con 0x7f074c108920 2024-04-25T14:16:31.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.227+0000 7f0743fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f074c103c20 0x7f074c19a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:31.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.227+0000 7f0743fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f074c103c20 0x7f074c19a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41626/0 (socket says 172.21.15.96:41626) 2024-04-25T14:16:31.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.227+0000 7f0743fff640 1 -- 172.21.15.96:0/1683330206 learned_addr learned my addr 172.21.15.96:0/1683330206 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:16:31.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.227+0000 7f074affd640 1 --2- 172.21.15.96:0/1683330206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f074c1029e0 0x7f074c19a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:31.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.227+0000 7f074b7fe640 1 --2- 172.21.15.96:0/1683330206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f074c108920 0x7f074c1a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:31.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.228+0000 7f0743fff640 1 -- 172.21.15.96:0/1683330206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f074c108920 msgr2=0x7f074c1a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:31.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.228+0000 7f0743fff640 1 --2- 172.21.15.96:0/1683330206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f074c108920 0x7f074c1a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:31.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.228+0000 7f0743fff640 1 -- 172.21.15.96:0/1683330206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f074c1029e0 msgr2=0x7f074c19a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:31.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.228+0000 7f0743fff640 1 --2- 172.21.15.96:0/1683330206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f074c1029e0 0x7f074c19a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:31.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.228+0000 7f0743fff640 1 -- 172.21.15.96:0/1683330206 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f074c1a1dc0 con 0x7f074c103c20 2024-04-25T14:16:31.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.228+0000 7f0743fff640 1 --2- 172.21.15.96:0/1683330206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f074c103c20 0x7f074c19a7c0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f073400e9f0 tx=0x7f073400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:31.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.228+0000 7f0748ff9640 1 -- 172.21.15.96:0/1683330206 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073400cd80 con 0x7f074c103c20 2024-04-25T14:16:31.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.228+0000 7f0751346640 1 -- 172.21.15.96:0/1683330206 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f074c1a20b0 con 0x7f074c103c20 2024-04-25T14:16:31.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.228+0000 7f0748ff9640 1 -- 172.21.15.96:0/1683330206 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07340054d0 con 0x7f074c103c20 2024-04-25T14:16:31.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.228+0000 7f0748ff9640 1 -- 172.21.15.96:0/1683330206 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0734010690 con 0x7f074c103c20 2024-04-25T14:16:31.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.228+0000 7f0751346640 1 -- 172.21.15.96:0/1683330206 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f074c1a25b0 con 0x7f074c103c20 2024-04-25T14:16:31.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.229+0000 7f0751346640 1 -- 172.21.15.96:0/1683330206 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f074c107730 con 0x7f074c103c20 2024-04-25T14:16:31.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.230+0000 7f0748ff9640 1 -- 172.21.15.96:0/1683330206 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f0734004e90 con 0x7f074c103c20 2024-04-25T14:16:31.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.231+0000 7f0748ff9640 1 --2- 172.21.15.96:0/1683330206 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f07380afaf0 0x7f07380b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:31.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.231+0000 7f074affd640 1 --2- 172.21.15.96:0/1683330206 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f07380afaf0 0x7f07380b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:31.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.231+0000 7f0748ff9640 1 -- 172.21.15.96:0/1683330206 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f07340df100 con 0x7f074c103c20 2024-04-25T14:16:31.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.231+0000 7f074affd640 1 --2- 172.21.15.96:0/1683330206 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f07380afaf0 0x7f07380b2000 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f073c005ab0 tx=0x7f073c031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:31.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.235+0000 7f0748ff9640 1 -- 172.21.15.96:0/1683330206 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f07340a7cc0 con 0x7f074c103c20 2024-04-25T14:16:31.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.413+0000 7f0751346640 1 -- 172.21.15.96:0/1683330206 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f074c0637a0 con 0x7f074c103c20 2024-04-25T14:16:31.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.414+0000 7f0748ff9640 1 -- 172.21.15.96:0/1683330206 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v78) ==== 100+0+11 (secure 0 0 0) 0x7f07340a75c0 con 0x7f074c103c20 2024-04-25T14:16:31.414 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-04-25T14:16:31.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.416+0000 7f0751346640 1 -- 172.21.15.96:0/1683330206 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f07380afaf0 msgr2=0x7f07380b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:31.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.416+0000 7f0751346640 1 --2- 172.21.15.96:0/1683330206 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f07380afaf0 0x7f07380b2000 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f073c005ab0 tx=0x7f073c031400 comp rx=0 tx=0).stop 2024-04-25T14:16:31.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.416+0000 7f0751346640 1 -- 172.21.15.96:0/1683330206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f074c103c20 msgr2=0x7f074c19a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:31.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.416+0000 7f0751346640 1 --2- 172.21.15.96:0/1683330206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f074c103c20 0x7f074c19a7c0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f073400e9f0 tx=0x7f073400edf0 comp rx=0 tx=0).stop 2024-04-25T14:16:31.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.416+0000 7f0751346640 1 -- 172.21.15.96:0/1683330206 shutdown_connections 2024-04-25T14:16:31.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.416+0000 7f0751346640 1 --2- 172.21.15.96:0/1683330206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f074c108920 0x7f074c1a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:31.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.416+0000 7f0751346640 1 --2- 172.21.15.96:0/1683330206 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f07380afaf0 0x7f07380b2000 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:31.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.416+0000 7f0751346640 1 --2- 172.21.15.96:0/1683330206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f074c103c20 0x7f074c19a7c0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:31.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.416+0000 7f0751346640 1 --2- 172.21.15.96:0/1683330206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f074c1029e0 0x7f074c19a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:31.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.416+0000 7f0751346640 1 -- 172.21.15.96:0/1683330206 >> 172.21.15.96:0/1683330206 conn(0x7f074c0fdbd0 msgr2=0x7f074c104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:31.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.416+0000 7f0751346640 1 -- 172.21.15.96:0/1683330206 shutdown_connections 2024-04-25T14:16:31.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.416+0000 7f0751346640 1 -- 172.21.15.96:0/1683330206 wait complete. 2024-04-25T14:16:31.429 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T14:16:31.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.539+0000 7fdb046a1640 1 -- 172.21.15.96:0/2886699344 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdafc102180 msgr2=0x7fdafc102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:31.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.539+0000 7fdb046a1640 1 --2- 172.21.15.96:0/2886699344 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdafc102180 0x7fdafc102e80 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8009900 tx=0x7fdaf802a410 comp rx=0 tx=0).stop 2024-04-25T14:16:31.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.539+0000 7fdb046a1640 1 -- 172.21.15.96:0/2886699344 shutdown_connections 2024-04-25T14:16:31.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.539+0000 7fdb046a1640 1 --2- 172.21.15.96:0/2886699344 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdafc1080c0 0x7fdafc10a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:31.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.539+0000 7fdb046a1640 1 --2- 172.21.15.96:0/2886699344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdafc1033c0 0x7fdafc1037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:31.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.539+0000 7fdb046a1640 1 --2- 172.21.15.96:0/2886699344 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdafc102180 0x7fdafc102e80 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:31.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.539+0000 7fdb046a1640 1 -- 172.21.15.96:0/2886699344 >> 172.21.15.96:0/2886699344 conn(0x7fdafc0fd350 msgr2=0x7fdafc0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:31.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.539+0000 7fdb046a1640 1 -- 172.21.15.96:0/2886699344 shutdown_connections 2024-04-25T14:16:31.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.539+0000 7fdb046a1640 1 -- 172.21.15.96:0/2886699344 wait complete. 2024-04-25T14:16:31.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.539+0000 7fdb046a1640 1 Processor -- start 2024-04-25T14:16:31.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.539+0000 7fdb046a1640 1 -- start start 2024-04-25T14:16:31.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.540+0000 7fdb046a1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdafc102180 0x7fdafc19a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:31.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.540+0000 7fdb046a1640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdafc1033c0 0x7fdafc19a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:31.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.540+0000 7fdb046a1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdafc1080c0 0x7fdafc1a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:31.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.540+0000 7fdb046a1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc10f620 con 0x7fdafc1080c0 2024-04-25T14:16:31.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.540+0000 7fdb046a1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc10f4a0 con 0x7fdafc102180 2024-04-25T14:16:31.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.540+0000 7fdb046a1640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc10f7a0 con 0x7fdafc1033c0 2024-04-25T14:16:31.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.540+0000 7fdb02c3e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdafc1080c0 0x7fdafc1a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:31.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.540+0000 7fdb02c3e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdafc1080c0 0x7fdafc1a16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38792/0 (socket says 172.21.15.96:38792) 2024-04-25T14:16:31.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.540+0000 7fdb02c3e640 1 -- 172.21.15.96:0/2941363522 learned_addr learned my addr 172.21.15.96:0/2941363522 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:16:31.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.540+0000 7fdb01c3c640 1 --2- 172.21.15.96:0/2941363522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdafc1033c0 0x7fdafc19a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:31.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.540+0000 7fdb0243d640 1 --2- 172.21.15.96:0/2941363522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdafc102180 0x7fdafc19a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:31.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.540+0000 7fdb02c3e640 1 -- 172.21.15.96:0/2941363522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdafc1033c0 msgr2=0x7fdafc19a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:31.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.540+0000 7fdb02c3e640 1 --2- 172.21.15.96:0/2941363522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdafc1033c0 0x7fdafc19a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:31.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.540+0000 7fdb02c3e640 1 -- 172.21.15.96:0/2941363522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdafc102180 msgr2=0x7fdafc19a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:31.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.540+0000 7fdb02c3e640 1 --2- 172.21.15.96:0/2941363522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdafc102180 0x7fdafc19a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:31.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.540+0000 7fdb02c3e640 1 -- 172.21.15.96:0/2941363522 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdafc1a1d60 con 0x7fdafc1080c0 2024-04-25T14:16:31.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.540+0000 7fdb02c3e640 1 --2- 172.21.15.96:0/2941363522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdafc1080c0 0x7fdafc1a16f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fdaf400eaa0 tx=0x7fdaf400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:31.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.540+0000 7fdaf37fe640 1 -- 172.21.15.96:0/2941363522 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf400cdb0 con 0x7fdafc1080c0 2024-04-25T14:16:31.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.540+0000 7fdaf37fe640 1 -- 172.21.15.96:0/2941363522 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaf4005020 con 0x7fdafc1080c0 2024-04-25T14:16:31.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.540+0000 7fdb046a1640 1 -- 172.21.15.96:0/2941363522 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdafc1a1ff0 con 0x7fdafc1080c0 2024-04-25T14:16:31.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.540+0000 7fdaf37fe640 1 -- 172.21.15.96:0/2941363522 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf40116d0 con 0x7fdafc1080c0 2024-04-25T14:16:31.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.540+0000 7fdb046a1640 1 -- 172.21.15.96:0/2941363522 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdafc1a24b0 con 0x7fdafc1080c0 2024-04-25T14:16:31.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.541+0000 7fdb046a1640 1 -- 172.21.15.96:0/2941363522 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdafc106ed0 con 0x7fdafc1080c0 2024-04-25T14:16:31.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.544+0000 7fdaf37fe640 1 -- 172.21.15.96:0/2941363522 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fdaf40051d0 con 0x7fdafc1080c0 2024-04-25T14:16:31.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.544+0000 7fdaf37fe640 1 --2- 172.21.15.96:0/2941363522 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fdae80afaf0 0x7fdae80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:31.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.544+0000 7fdb0243d640 1 --2- 172.21.15.96:0/2941363522 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fdae80afaf0 0x7fdae80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:31.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.544+0000 7fdaf37fe640 1 -- 172.21.15.96:0/2941363522 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7fdaf40df120 con 0x7fdafc1080c0 2024-04-25T14:16:31.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.545+0000 7fdb0243d640 1 --2- 172.21.15.96:0/2941363522 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fdae80afaf0 0x7fdae80b2000 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8009230 tx=0x7fdaf8031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:31.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.548+0000 7fdaf37fe640 1 -- 172.21.15.96:0/2941363522 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdaf4016030 con 0x7fdafc1080c0 2024-04-25T14:16:31.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.739+0000 7fdb046a1640 1 -- 172.21.15.96:0/2941363522 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdafc02db50 con 0x7fdafc1080c0 2024-04-25T14:16:31.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.739+0000 7fdaf37fe640 1 -- 172.21.15.96:0/2941363522 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v78) ==== 81+0+589 (secure 0 0 0) 0x7fdaf40a7ce0 con 0x7fdafc1080c0 2024-04-25T14:16:31.739 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/3839807484 2024-04-26T14:04:21.307574+0000 2024-04-25T14:16:31.740 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/1495237551 2024-04-26T14:04:21.307574+0000 2024-04-25T14:16:31.740 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/2793751450 2024-04-26T14:03:54.249537+0000 2024-04-25T14:16:31.740 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/129575834 2024-04-26T14:03:54.249537+0000 2024-04-25T14:16:31.740 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/2233075139 2024-04-26T14:04:21.307574+0000 2024-04-25T14:16:31.740 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:6801/2348397481 2024-04-26T14:03:54.249537+0000 2024-04-25T14:16:31.740 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:6801/433591062 2024-04-26T14:04:21.307574+0000 2024-04-25T14:16:31.740 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:6800/2348397481 2024-04-26T14:03:54.249537+0000 2024-04-25T14:16:31.740 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/3517674924 2024-04-26T14:03:54.249537+0000 2024-04-25T14:16:31.740 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:6800/433591062 2024-04-26T14:04:21.307574+0000 2024-04-25T14:16:31.740 INFO:teuthology.orchestra.run.smithi096.stderr:listed 10 entries 2024-04-25T14:16:31.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.741+0000 7fdb046a1640 1 -- 172.21.15.96:0/2941363522 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fdae80afaf0 msgr2=0x7fdae80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:31.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.741+0000 7fdb046a1640 1 --2- 172.21.15.96:0/2941363522 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fdae80afaf0 0x7fdae80b2000 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8009230 tx=0x7fdaf8031400 comp rx=0 tx=0).stop 2024-04-25T14:16:31.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.741+0000 7fdb046a1640 1 -- 172.21.15.96:0/2941363522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdafc1080c0 msgr2=0x7fdafc1a16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:31.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.741+0000 7fdb046a1640 1 --2- 172.21.15.96:0/2941363522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdafc1080c0 0x7fdafc1a16f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fdaf400eaa0 tx=0x7fdaf400ef60 comp rx=0 tx=0).stop 2024-04-25T14:16:31.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.742+0000 7fdb046a1640 1 -- 172.21.15.96:0/2941363522 shutdown_connections 2024-04-25T14:16:31.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.742+0000 7fdb046a1640 1 --2- 172.21.15.96:0/2941363522 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fdae80afaf0 0x7fdae80b2000 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:31.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.742+0000 7fdb046a1640 1 --2- 172.21.15.96:0/2941363522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdafc1080c0 0x7fdafc1a16f0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:31.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.742+0000 7fdb046a1640 1 --2- 172.21.15.96:0/2941363522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdafc1033c0 0x7fdafc19a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:31.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.742+0000 7fdb046a1640 1 --2- 172.21.15.96:0/2941363522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdafc102180 0x7fdafc19a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:31.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.742+0000 7fdb046a1640 1 -- 172.21.15.96:0/2941363522 >> 172.21.15.96:0/2941363522 conn(0x7fdafc0fd350 msgr2=0x7fdafc1040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:31.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.742+0000 7fdb046a1640 1 -- 172.21.15.96:0/2941363522 shutdown_connections 2024-04-25T14:16:31.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:31.742+0000 7fdb046a1640 1 -- 172.21.15.96:0/2941363522 wait complete. 2024-04-25T14:16:31.752 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:31 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/1640507484' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:16:31.752 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:31 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/1900336374' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:16:31.752 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:31 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/1683330206' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:16:31.753 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T14:16:31.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:16:31.753 DEBUG:teuthology.orchestra.run.smithi096:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T14:16:31.791 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi096.front.sepia.ceph.com 2024-04-25T14:16:31.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:16:31.792 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T14:16:31.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:16:31.862 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list 2024-04-25T14:16:31.917 INFO:teuthology.orchestra.run.smithi096.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T14:16:31.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:16:31.917 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T14:16:31.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:31 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/1640507484' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:16:31.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:31 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/1900336374' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:16:31.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:31 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/1683330206' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:16:31.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:31 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/1640507484' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:16:31.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:31 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/1900336374' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:16:31.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:31 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/1683330206' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:16:31.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:16:31.980 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link delete ceph-brx 2024-04-25T14:16:32.076 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-25T14:16:32.092 INFO:tasks.cephadm:Teardown begin 2024-04-25T14:16:32.092 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T14:16:32.092 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:16:32.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.201+0000 7f9983d38640 1 -- 172.21.15.96:0/3998796325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f997c101750 msgr2=0x7f997c101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:32.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.201+0000 7f9983d38640 1 --2- 172.21.15.96:0/3998796325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f997c101750 0x7f997c101bc0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f9970009900 tx=0x7f997002a410 comp rx=0 tx=0).stop 2024-04-25T14:16:32.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.201+0000 7f9983d38640 1 -- 172.21.15.96:0/3998796325 shutdown_connections 2024-04-25T14:16:32.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.201+0000 7f9983d38640 1 --2- 172.21.15.96:0/3998796325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f997c102250 0x7f997c10bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:32.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.201+0000 7f9983d38640 1 --2- 172.21.15.96:0/3998796325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f997c101750 0x7f997c101bc0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:32.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.201+0000 7f9983d38640 1 --2- 172.21.15.96:0/3998796325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f997c1050a0 0x7f997c1054b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:32.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.201+0000 7f9983d38640 1 -- 172.21.15.96:0/3998796325 >> 172.21.15.96:0/3998796325 conn(0x7f997c0fd370 msgr2=0x7f997c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:32.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.202+0000 7f9983d38640 1 -- 172.21.15.96:0/3998796325 shutdown_connections 2024-04-25T14:16:32.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.202+0000 7f9983d38640 1 -- 172.21.15.96:0/3998796325 wait complete. 2024-04-25T14:16:32.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.202+0000 7f9983d38640 1 Processor -- start 2024-04-25T14:16:32.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.202+0000 7f9983d38640 1 -- start start 2024-04-25T14:16:32.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.202+0000 7f9983d38640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f997c101750 0x7f997c19a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:32.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.202+0000 7f9983d38640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f997c102250 0x7f997c19a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:32.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.202+0000 7f9983d38640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f997c1050a0 0x7f997c1a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:32.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.202+0000 7f9983d38640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f997c10f610 con 0x7f997c101750 2024-04-25T14:16:32.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.202+0000 7f9983d38640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f997c10f490 con 0x7f997c1050a0 2024-04-25T14:16:32.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.202+0000 7f9983d38640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f997c10f790 con 0x7f997c102250 2024-04-25T14:16:32.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.202+0000 7f9981ad4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f997c101750 0x7f997c19a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:32.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.202+0000 7f9981ad4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f997c101750 0x7f997c19a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38812/0 (socket says 172.21.15.96:38812) 2024-04-25T14:16:32.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.203+0000 7f9981ad4640 1 -- 172.21.15.96:0/1119952484 learned_addr learned my addr 172.21.15.96:0/1119952484 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:16:32.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.203+0000 7f99812d3640 1 --2- 172.21.15.96:0/1119952484 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f997c102250 0x7f997c19a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:32.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.203+0000 7f9981ad4640 1 -- 172.21.15.96:0/1119952484 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f997c102250 msgr2=0x7f997c19a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:32.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.203+0000 7f9981ad4640 1 --2- 172.21.15.96:0/1119952484 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f997c102250 0x7f997c19a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:32.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.203+0000 7f9981ad4640 1 -- 172.21.15.96:0/1119952484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f997c1050a0 msgr2=0x7f997c1a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:32.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.203+0000 7f99822d5640 1 --2- 172.21.15.96:0/1119952484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f997c1050a0 0x7f997c1a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:32.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.203+0000 7f9981ad4640 1 --2- 172.21.15.96:0/1119952484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f997c1050a0 0x7f997c1a16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:32.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.203+0000 7f9981ad4640 1 -- 172.21.15.96:0/1119952484 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f997c1a1c80 con 0x7f997c101750 2024-04-25T14:16:32.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.203+0000 7f99812d3640 1 --2- 172.21.15.96:0/1119952484 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f997c102250 0x7f997c19a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:16:32.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.203+0000 7f9981ad4640 1 --2- 172.21.15.96:0/1119952484 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f997c101750 0x7f997c19a1f0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f997400c970 tx=0x7f997400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:32.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.203+0000 7f997affd640 1 -- 172.21.15.96:0/1119952484 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9974007c70 con 0x7f997c101750 2024-04-25T14:16:32.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.203+0000 7f997affd640 1 -- 172.21.15.96:0/1119952484 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9974007e10 con 0x7f997c101750 2024-04-25T14:16:32.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.203+0000 7f9983d38640 1 -- 172.21.15.96:0/1119952484 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f997c1a1f70 con 0x7f997c101750 2024-04-25T14:16:32.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.203+0000 7f997affd640 1 -- 172.21.15.96:0/1119952484 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9974018760 con 0x7f997c101750 2024-04-25T14:16:32.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.203+0000 7f9983d38640 1 -- 172.21.15.96:0/1119952484 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f997c1a23d0 con 0x7f997c101750 2024-04-25T14:16:32.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.204+0000 7f9983d38640 1 -- 172.21.15.96:0/1119952484 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f997c06a190 con 0x7f997c101750 2024-04-25T14:16:32.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.205+0000 7f997affd640 1 -- 172.21.15.96:0/1119952484 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f997400fab0 con 0x7f997c101750 2024-04-25T14:16:32.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.206+0000 7f997affd640 1 --2- 172.21.15.96:0/1119952484 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f99640afaf0 0x7f99640b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:32.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.206+0000 7f99812d3640 1 --2- 172.21.15.96:0/1119952484 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f99640afaf0 0x7f99640b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:32.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.206+0000 7f997affd640 1 -- 172.21.15.96:0/1119952484 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f99740ded60 con 0x7f997c101750 2024-04-25T14:16:32.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.207+0000 7f99812d3640 1 --2- 172.21.15.96:0/1119952484 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f99640afaf0 0x7f99640b2000 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9970006010 tx=0x7f9970000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:32.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.211+0000 7f997affd640 1 -- 172.21.15.96:0/1119952484 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9974014030 con 0x7f997c101750 2024-04-25T14:16:32.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.428+0000 7f9983d38640 1 -- 172.21.15.96:0/1119952484 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f997c0637a0 con 0x7f997c101750 2024-04-25T14:16:32.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.429+0000 7f997affd640 1 -- 172.21.15.96:0/1119952484 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11927 (secure 0 0 0) 0x7f99740a7920 con 0x7f997c101750 2024-04-25T14:16:32.429 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:16:32.429 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/606227917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":606227917},{"type":"v1","addr":"172.21.15.96:6835","nonce":606227917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/3265300710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3265300710},{"type":"v1","addr":"172.21.15.96:6837","nonce":3265300710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T14:11:28.886961+0000","modified":"2024-04-25T14:13:32.120297+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24517,"mds_2":24478,"mds_3":24455,"mds_4":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/4080308509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":4080308509},{"type":"v1","addr":"172.21.15.96:6839","nonce":4080308509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/1606748971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":1606748971},{"type":"v1","addr":"172.21.15.96:6841","nonce":1606748971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6833/1995324831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1995324831},{"type":"v1","addr":"172.21.15.97:6833","nonce":1995324831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":4,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.151:6833/1382721608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1382721608},{"type":"v1","addr":"172.21.15.151:6833","nonce":1382721608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6835/3872301726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3872301726},{"type":"v1","addr":"172.21.15.97:6835","nonce":3872301726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.151:6835/1313823638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":1313823638},{"type":"v1","addr":"172.21.15.151:6835","nonce":1313823638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/2691348366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2691348366},{"type":"v1","addr":"172.21.15.151:6837","nonce":2691348366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3306780081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3306780081},{"type":"v1","addr":"172.21.15.97:6837","nonce":3306780081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.151:6839/245699981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":245699981},{"type":"v1","addr":"172.21.15.151:6839","nonce":245699981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6839/2850442269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":2850442269},{"type":"v1","addr":"172.21.15.97:6839","nonce":2850442269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24427,24455,24478,24517,24536]},"id":1}]} 2024-04-25T14:16:32.430 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 26 2024-04-25T14:16:32.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.431+0000 7f9983d38640 1 -- 172.21.15.96:0/1119952484 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f99640afaf0 msgr2=0x7f99640b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:32.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.431+0000 7f9983d38640 1 --2- 172.21.15.96:0/1119952484 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f99640afaf0 0x7f99640b2000 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9970006010 tx=0x7f9970000bc0 comp rx=0 tx=0).stop 2024-04-25T14:16:32.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.431+0000 7f9983d38640 1 -- 172.21.15.96:0/1119952484 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f997c101750 msgr2=0x7f997c19a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:32.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.431+0000 7f9983d38640 1 --2- 172.21.15.96:0/1119952484 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f997c101750 0x7f997c19a1f0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f997400c970 tx=0x7f997400ce30 comp rx=0 tx=0).stop 2024-04-25T14:16:32.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.431+0000 7f9983d38640 1 -- 172.21.15.96:0/1119952484 shutdown_connections 2024-04-25T14:16:32.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.431+0000 7f9983d38640 1 --2- 172.21.15.96:0/1119952484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f997c1050a0 0x7f997c1a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:32.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.431+0000 7f9983d38640 1 --2- 172.21.15.96:0/1119952484 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f997c102250 0x7f997c19a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:32.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.431+0000 7f9983d38640 1 --2- 172.21.15.96:0/1119952484 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f99640afaf0 0x7f99640b2000 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:32.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.431+0000 7f9983d38640 1 --2- 172.21.15.96:0/1119952484 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f997c101750 0x7f997c19a1f0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:32.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.431+0000 7f9983d38640 1 -- 172.21.15.96:0/1119952484 >> 172.21.15.96:0/1119952484 conn(0x7f997c0fd370 msgr2=0x7f997c105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:32.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.431+0000 7f9983d38640 1 -- 172.21.15.96:0/1119952484 shutdown_connections 2024-04-25T14:16:32.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.431+0000 7f9983d38640 1 -- 172.21.15.96:0/1119952484 wait complete. 2024-04-25T14:16:32.443 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:16:32.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.553+0000 7f2481596640 1 -- 172.21.15.96:0/2439819113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f247c108900 msgr2=0x7f247c10ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:32.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.553+0000 7f2481596640 1 --2- 172.21.15.96:0/2439819113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f247c108900 0x7f247c10ad20 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f2474009ca0 tx=0x7f247402a8c0 comp rx=0 tx=0).stop 2024-04-25T14:16:32.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.553+0000 7f2481596640 1 -- 172.21.15.96:0/2439819113 shutdown_connections 2024-04-25T14:16:32.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.553+0000 7f2481596640 1 --2- 172.21.15.96:0/2439819113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f247c108900 0x7f247c10ad20 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:32.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.553+0000 7f2481596640 1 --2- 172.21.15.96:0/2439819113 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f247c103c00 0x7f247c104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:32.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.553+0000 7f2481596640 1 --2- 172.21.15.96:0/2439819113 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f247c1029c0 0x7f247c1036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:32.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.553+0000 7f2481596640 1 -- 172.21.15.96:0/2439819113 >> 172.21.15.96:0/2439819113 conn(0x7f247c0fdbd0 msgr2=0x7f247c100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:32.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.553+0000 7f2481596640 1 -- 172.21.15.96:0/2439819113 shutdown_connections 2024-04-25T14:16:32.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.553+0000 7f2481596640 1 -- 172.21.15.96:0/2439819113 wait complete. 2024-04-25T14:16:32.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.553+0000 7f2481596640 1 Processor -- start 2024-04-25T14:16:32.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.553+0000 7f2481596640 1 -- start start 2024-04-25T14:16:32.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.554+0000 7f2481596640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f247c1029c0 0x7f247c1990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:32.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.554+0000 7f2481596640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f247c103c00 0x7f247c199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:32.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.554+0000 7f2481596640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f247c108900 0x7f247c1a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:32.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.554+0000 7f2481596640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f247c10e4f0 con 0x7f247c103c00 2024-04-25T14:16:32.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.554+0000 7f2481596640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f247c10e370 con 0x7f247c108900 2024-04-25T14:16:32.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.554+0000 7f2481596640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f247c10e670 con 0x7f247c1029c0 2024-04-25T14:16:32.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.554+0000 7f2473fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f247c103c00 0x7f247c199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:32.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.554+0000 7f2473fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f247c103c00 0x7f247c199610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38832/0 (socket says 172.21.15.96:38832) 2024-04-25T14:16:32.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.554+0000 7f2473fff640 1 -- 172.21.15.96:0/853604729 learned_addr learned my addr 172.21.15.96:0/853604729 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:16:32.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.554+0000 7f247affd640 1 --2- 172.21.15.96:0/853604729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f247c1029c0 0x7f247c1990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:32.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.554+0000 7f247b7fe640 1 --2- 172.21.15.96:0/853604729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f247c108900 0x7f247c1a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:32.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.554+0000 7f2473fff640 1 -- 172.21.15.96:0/853604729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f247c1029c0 msgr2=0x7f247c1990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:32.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.554+0000 7f2473fff640 1 --2- 172.21.15.96:0/853604729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f247c1029c0 0x7f247c1990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:32.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.554+0000 7f2473fff640 1 -- 172.21.15.96:0/853604729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f247c108900 msgr2=0x7f247c1a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:32.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.554+0000 7f2473fff640 1 --2- 172.21.15.96:0/853604729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f247c108900 0x7f247c1a05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:32.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.554+0000 7f2473fff640 1 -- 172.21.15.96:0/853604729 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f247c1a0c10 con 0x7f247c103c00 2024-04-25T14:16:32.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.554+0000 7f247b7fe640 1 --2- 172.21.15.96:0/853604729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f247c108900 0x7f247c1a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:16:32.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.554+0000 7f2473fff640 1 --2- 172.21.15.96:0/853604729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f247c103c00 0x7f247c199610 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f246400e9f0 tx=0x7f246400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:32.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.554+0000 7f2478ff9640 1 -- 172.21.15.96:0/853604729 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246400cd80 con 0x7f247c103c00 2024-04-25T14:16:32.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.554+0000 7f2478ff9640 1 -- 172.21.15.96:0/853604729 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24640054d0 con 0x7f247c103c00 2024-04-25T14:16:32.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.554+0000 7f2481596640 1 -- 172.21.15.96:0/853604729 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f247c1a0f00 con 0x7f247c103c00 2024-04-25T14:16:32.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.555+0000 7f2478ff9640 1 -- 172.21.15.96:0/853604729 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2464010690 con 0x7f247c103c00 2024-04-25T14:16:32.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.555+0000 7f2481596640 1 -- 172.21.15.96:0/853604729 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f247c1a1360 con 0x7f247c103c00 2024-04-25T14:16:32.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.555+0000 7f2481596640 1 -- 172.21.15.96:0/853604729 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f247c107710 con 0x7f247c103c00 2024-04-25T14:16:32.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.557+0000 7f2478ff9640 1 -- 172.21.15.96:0/853604729 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f2464004e30 con 0x7f247c103c00 2024-04-25T14:16:32.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.558+0000 7f2478ff9640 1 --2- 172.21.15.96:0/853604729 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f24680afaf0 0x7f24680b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:32.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.558+0000 7f247affd640 1 --2- 172.21.15.96:0/853604729 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f24680afaf0 0x7f24680b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:32.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.558+0000 7f2478ff9640 1 -- 172.21.15.96:0/853604729 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f24640df2e0 con 0x7f247c103c00 2024-04-25T14:16:32.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.558+0000 7f247affd640 1 --2- 172.21.15.96:0/853604729 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f24680afaf0 0x7f24680b2000 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f247c1035c0 tx=0x7f246c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:32.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.562+0000 7f2478ff9640 1 -- 172.21.15.96:0/853604729 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2464053ff0 con 0x7f247c103c00 2024-04-25T14:16:32.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.781+0000 7f2481596640 1 -- 172.21.15.96:0/853604729 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f247c02db50 con 0x7f247c103c00 2024-04-25T14:16:32.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.782+0000 7f2478ff9640 1 -- 172.21.15.96:0/853604729 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11927 (secure 0 0 0) 0x7f24640a7ea0 con 0x7f247c103c00 2024-04-25T14:16:32.782 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:16:32.782 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/606227917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":606227917},{"type":"v1","addr":"172.21.15.96:6835","nonce":606227917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/3265300710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3265300710},{"type":"v1","addr":"172.21.15.96:6837","nonce":3265300710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T14:11:28.886961+0000","modified":"2024-04-25T14:13:32.120297+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24517,"mds_2":24478,"mds_3":24455,"mds_4":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/4080308509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":4080308509},{"type":"v1","addr":"172.21.15.96:6839","nonce":4080308509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/1606748971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":1606748971},{"type":"v1","addr":"172.21.15.96:6841","nonce":1606748971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6833/1995324831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1995324831},{"type":"v1","addr":"172.21.15.97:6833","nonce":1995324831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":4,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.151:6833/1382721608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1382721608},{"type":"v1","addr":"172.21.15.151:6833","nonce":1382721608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6835/3872301726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3872301726},{"type":"v1","addr":"172.21.15.97:6835","nonce":3872301726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.151:6835/1313823638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":1313823638},{"type":"v1","addr":"172.21.15.151:6835","nonce":1313823638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/2691348366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2691348366},{"type":"v1","addr":"172.21.15.151:6837","nonce":2691348366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3306780081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3306780081},{"type":"v1","addr":"172.21.15.97:6837","nonce":3306780081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.151:6839/245699981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":245699981},{"type":"v1","addr":"172.21.15.151:6839","nonce":245699981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6839/2850442269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":2850442269},{"type":"v1","addr":"172.21.15.97:6839","nonce":2850442269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24427,24455,24478,24517,24536]},"id":1}]} 2024-04-25T14:16:32.782 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 26 2024-04-25T14:16:32.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.784+0000 7f2481596640 1 -- 172.21.15.96:0/853604729 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f24680afaf0 msgr2=0x7f24680b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:32.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.784+0000 7f2481596640 1 --2- 172.21.15.96:0/853604729 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f24680afaf0 0x7f24680b2000 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f247c1035c0 tx=0x7f246c005f90 comp rx=0 tx=0).stop 2024-04-25T14:16:32.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.784+0000 7f2481596640 1 -- 172.21.15.96:0/853604729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f247c103c00 msgr2=0x7f247c199610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:32.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.784+0000 7f2481596640 1 --2- 172.21.15.96:0/853604729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f247c103c00 0x7f247c199610 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f246400e9f0 tx=0x7f246400edf0 comp rx=0 tx=0).stop 2024-04-25T14:16:32.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.784+0000 7f2481596640 1 -- 172.21.15.96:0/853604729 shutdown_connections 2024-04-25T14:16:32.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.784+0000 7f2481596640 1 --2- 172.21.15.96:0/853604729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f247c108900 0x7f247c1a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:32.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.784+0000 7f2481596640 1 --2- 172.21.15.96:0/853604729 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f24680afaf0 0x7f24680b2000 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:32.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.784+0000 7f2481596640 1 --2- 172.21.15.96:0/853604729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f247c103c00 0x7f247c199610 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:32.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.784+0000 7f2481596640 1 --2- 172.21.15.96:0/853604729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f247c1029c0 0x7f247c1990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:32.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.784+0000 7f2481596640 1 -- 172.21.15.96:0/853604729 >> 172.21.15.96:0/853604729 conn(0x7f247c0fdbd0 msgr2=0x7f247c1048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:32.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.784+0000 7f2481596640 1 -- 172.21.15.96:0/853604729 shutdown_connections 2024-04-25T14:16:32.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.784+0000 7f2481596640 1 -- 172.21.15.96:0/853604729 wait complete. 2024-04-25T14:16:32.794 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:32 smithi096 ceph-mon[29207]: pgmap v433: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T14:16:32.794 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:32 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/2941363522' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T14:16:32.794 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:32 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/1119952484' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:16:32.796 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:16:32.842 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:32 smithi097 ceph-mon[32720]: pgmap v433: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T14:16:32.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:32 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/2941363522' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T14:16:32.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:32 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/1119952484' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:16:32.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.903+0000 7f4d94fce640 1 -- 172.21.15.96:0/1271598336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d901080c0 msgr2=0x7f4d9010a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:32.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.903+0000 7f4d94fce640 1 --2- 172.21.15.96:0/1271598336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d901080c0 0x7f4d9010a4e0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f4d88009ca0 tx=0x7f4d8802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:16:32.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.903+0000 7f4d94fce640 1 -- 172.21.15.96:0/1271598336 shutdown_connections 2024-04-25T14:16:32.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.903+0000 7f4d94fce640 1 --2- 172.21.15.96:0/1271598336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d901080c0 0x7f4d9010a4e0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:32.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.903+0000 7f4d94fce640 1 --2- 172.21.15.96:0/1271598336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d901033c0 0x7f4d901037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:32.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.903+0000 7f4d94fce640 1 --2- 172.21.15.96:0/1271598336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d90102180 0x7f4d90102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:32.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.903+0000 7f4d94fce640 1 -- 172.21.15.96:0/1271598336 >> 172.21.15.96:0/1271598336 conn(0x7f4d900fd370 msgr2=0x7f4d900ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:32.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.903+0000 7f4d94fce640 1 -- 172.21.15.96:0/1271598336 shutdown_connections 2024-04-25T14:16:32.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.903+0000 7f4d94fce640 1 -- 172.21.15.96:0/1271598336 wait complete. 2024-04-25T14:16:32.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.904+0000 7f4d94fce640 1 Processor -- start 2024-04-25T14:16:32.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.904+0000 7f4d94fce640 1 -- start start 2024-04-25T14:16:32.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.904+0000 7f4d94fce640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d90102180 0x7f4d9010b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:32.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.904+0000 7f4d94fce640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d901033c0 0x7f4d9019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:32.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.904+0000 7f4d94fce640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d901080c0 0x7f4d901a1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:32.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.904+0000 7f4d94fce640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4d9010e530 con 0x7f4d901080c0 2024-04-25T14:16:32.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.904+0000 7f4d94fce640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4d9010e3b0 con 0x7f4d901033c0 2024-04-25T14:16:32.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.904+0000 7f4d94fce640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4d9010e6b0 con 0x7f4d90102180 2024-04-25T14:16:32.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.904+0000 7f4d8ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d901080c0 0x7f4d901a1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:32.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.904+0000 7f4d8ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d901080c0 0x7f4d901a1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38850/0 (socket says 172.21.15.96:38850) 2024-04-25T14:16:32.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.904+0000 7f4d8ed9d640 1 -- 172.21.15.96:0/4004493244 learned_addr learned my addr 172.21.15.96:0/4004493244 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:16:32.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.904+0000 7f4d8e59c640 1 --2- 172.21.15.96:0/4004493244 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d90102180 0x7f4d9010b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:32.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.905+0000 7f4d8dd9b640 1 --2- 172.21.15.96:0/4004493244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d901033c0 0x7f4d9019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:32.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.905+0000 7f4d8ed9d640 1 -- 172.21.15.96:0/4004493244 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d90102180 msgr2=0x7f4d9010b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:32.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.905+0000 7f4d8ed9d640 1 --2- 172.21.15.96:0/4004493244 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d90102180 0x7f4d9010b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:32.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.905+0000 7f4d8ed9d640 1 -- 172.21.15.96:0/4004493244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d901033c0 msgr2=0x7f4d9019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:32.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.905+0000 7f4d8ed9d640 1 --2- 172.21.15.96:0/4004493244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d901033c0 0x7f4d9019ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:32.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.905+0000 7f4d8ed9d640 1 -- 172.21.15.96:0/4004493244 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d901a1c90 con 0x7f4d901080c0 2024-04-25T14:16:32.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.905+0000 7f4d8ed9d640 1 --2- 172.21.15.96:0/4004493244 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d901080c0 0x7f4d901a1470 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f4d880097b0 tx=0x7f4d88031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:32.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.905+0000 7f4d7f7fe640 1 -- 172.21.15.96:0/4004493244 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d88003bf0 con 0x7f4d901080c0 2024-04-25T14:16:32.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.905+0000 7f4d7f7fe640 1 -- 172.21.15.96:0/4004493244 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d88003d90 con 0x7f4d901080c0 2024-04-25T14:16:32.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.905+0000 7f4d94fce640 1 -- 172.21.15.96:0/4004493244 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d901a1f20 con 0x7f4d901080c0 2024-04-25T14:16:32.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.905+0000 7f4d7f7fe640 1 -- 172.21.15.96:0/4004493244 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8802fc30 con 0x7f4d901080c0 2024-04-25T14:16:32.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.905+0000 7f4d94fce640 1 -- 172.21.15.96:0/4004493244 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d901a2380 con 0x7f4d901080c0 2024-04-25T14:16:32.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.906+0000 7f4d94fce640 1 -- 172.21.15.96:0/4004493244 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d9006a190 con 0x7f4d901080c0 2024-04-25T14:16:32.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.908+0000 7f4d7f7fe640 1 -- 172.21.15.96:0/4004493244 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f4d88004040 con 0x7f4d901080c0 2024-04-25T14:16:32.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.908+0000 7f4d7f7fe640 1 --2- 172.21.15.96:0/4004493244 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f4d780afaf0 0x7f4d780b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:32.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.908+0000 7f4d7f7fe640 1 -- 172.21.15.96:0/4004493244 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f4d881003b0 con 0x7f4d901080c0 2024-04-25T14:16:32.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.908+0000 7f4d8e59c640 1 --2- 172.21.15.96:0/4004493244 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f4d780afaf0 0x7f4d780b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:32.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.909+0000 7f4d8e59c640 1 --2- 172.21.15.96:0/4004493244 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f4d780afaf0 0x7f4d780b2000 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4d80005fd0 tx=0x7f4d8000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:32.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:32.912+0000 7f4d7f7fe640 1 -- 172.21.15.96:0/4004493244 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4d880c8f70 con 0x7f4d901080c0 2024-04-25T14:16:32.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:32 smithi151 ceph-mon[32990]: pgmap v433: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T14:16:32.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:32 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/2941363522' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T14:16:32.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:32 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/1119952484' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:16:33.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.091+0000 7f4d94fce640 1 -- 172.21.15.96:0/4004493244 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4d9002dc10 con 0x7f4d901080c0 2024-04-25T14:16:33.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.092+0000 7f4d7f7fe640 1 -- 172.21.15.96:0/4004493244 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22616 (secure 0 0 0) 0x7f4d880c8870 con 0x7f4d901080c0 2024-04-25T14:16:33.093 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T14:16:33.093 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":78,"fsid":"59334182-030c-11ef-bc93-c7b262605968","created":"2024-04-25T14:03:25.680745+0000","modified":"2024-04-25T14:12:22.496143+0000","last_up_change":"2024-04-25T14:10:35.894312+0000","last_in_change":"2024-04-25T14:10:15.583440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:06:47.402927+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:11:27.434331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:11:27.854004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cf4ba3ac-142e-4e8e-96d4-410d77157eb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6803","nonce":3832506168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6805","nonce":3832506168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832506168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3832506168},{"type":"v1","addr":"172.21.15.96:6807","nonce":3832506168}]},"public_addr":"172.21.15.96:6803/3832506168","cluster_addr":"172.21.15.96:6805/3832506168","heartbeat_back_addr":"172.21.15.96:6809/3832506168","heartbeat_front_addr":"172.21.15.96:6807/3832506168","state":["exists","up"]},{"osd":1,"uuid":"866e73b9-4a26-48ba-badf-704fe22f71cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6801","nonce":1083311494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6803","nonce":1083311494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6807","nonce":1083311494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1083311494},{"type":"v1","addr":"172.21.15.97:6805","nonce":1083311494}]},"public_addr":"172.21.15.97:6801/1083311494","cluster_addr":"172.21.15.97:6803/1083311494","heartbeat_back_addr":"172.21.15.97:6807/1083311494","heartbeat_front_addr":"172.21.15.97:6805/1083311494","state":["exists","up"]},{"osd":2,"uuid":"45c79ecb-abc4-4dba-bae8-74804e4bc1bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6801","nonce":981902966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6803","nonce":981902966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6807","nonce":981902966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":981902966},{"type":"v1","addr":"172.21.15.151:6805","nonce":981902966}]},"public_addr":"172.21.15.151:6801/981902966","cluster_addr":"172.21.15.151:6803/981902966","heartbeat_back_addr":"172.21.15.151:6807/981902966","heartbeat_front_addr":"172.21.15.151:6805/981902966","state":["exists","up"]},{"osd":3,"uuid":"9fc39468-a7ea-4ccf-b365-335ad2ea4c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6811","nonce":2274158748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6813","nonce":2274158748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6817","nonce":2274158748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2274158748},{"type":"v1","addr":"172.21.15.96:6815","nonce":2274158748}]},"public_addr":"172.21.15.96:6811/2274158748","cluster_addr":"172.21.15.96:6813/2274158748","heartbeat_back_addr":"172.21.15.96:6817/2274158748","heartbeat_front_addr":"172.21.15.96:6815/2274158748","state":["exists","up"]},{"osd":4,"uuid":"616582f1-12d2-456b-b7b3-e5517800e8a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6809","nonce":3727031962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6811","nonce":3727031962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6815","nonce":3727031962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3727031962},{"type":"v1","addr":"172.21.15.97:6813","nonce":3727031962}]},"public_addr":"172.21.15.97:6809/3727031962","cluster_addr":"172.21.15.97:6811/3727031962","heartbeat_back_addr":"172.21.15.97:6815/3727031962","heartbeat_front_addr":"172.21.15.97:6813/3727031962","state":["exists","up"]},{"osd":5,"uuid":"3d277784-ab0f-4c90-81a1-2a4034c52a27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6809","nonce":223882546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6811","nonce":223882546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6815","nonce":223882546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":223882546},{"type":"v1","addr":"172.21.15.151:6813","nonce":223882546}]},"public_addr":"172.21.15.151:6809/223882546","cluster_addr":"172.21.15.151:6811/223882546","heartbeat_back_addr":"172.21.15.151:6815/223882546","heartbeat_front_addr":"172.21.15.151:6813/223882546","state":["exists","up"]},{"osd":6,"uuid":"9a9ad585-cb1a-4239-9d18-055f59164049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6819","nonce":96197407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6821","nonce":96197407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6825","nonce":96197407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":96197407},{"type":"v1","addr":"172.21.15.96:6823","nonce":96197407}]},"public_addr":"172.21.15.96:6819/96197407","cluster_addr":"172.21.15.96:6821/96197407","heartbeat_back_addr":"172.21.15.96:6825/96197407","heartbeat_front_addr":"172.21.15.96:6823/96197407","state":["exists","up"]},{"osd":7,"uuid":"303551c8-6af0-42f2-844f-45bf8db5baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6817","nonce":2247216400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6819","nonce":2247216400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6823","nonce":2247216400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2247216400},{"type":"v1","addr":"172.21.15.97:6821","nonce":2247216400}]},"public_addr":"172.21.15.97:6817/2247216400","cluster_addr":"172.21.15.97:6819/2247216400","heartbeat_back_addr":"172.21.15.97:6823/2247216400","heartbeat_front_addr":"172.21.15.97:6821/2247216400","state":["exists","up"]},{"osd":8,"uuid":"81f6cc66-e4ca-430c-b336-b6678d968818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6817","nonce":2549822185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6819","nonce":2549822185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6823","nonce":2549822185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2549822185},{"type":"v1","addr":"172.21.15.151:6821","nonce":2549822185}]},"public_addr":"172.21.15.151:6817/2549822185","cluster_addr":"172.21.15.151:6819/2549822185","heartbeat_back_addr":"172.21.15.151:6823/2549822185","heartbeat_front_addr":"172.21.15.151:6821/2549822185","state":["exists","up"]},{"osd":9,"uuid":"29d4da16-feab-4d1c-9d1d-de4f799bd724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6827","nonce":1099399643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6829","nonce":1099399643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6833","nonce":1099399643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1099399643},{"type":"v1","addr":"172.21.15.96:6831","nonce":1099399643}]},"public_addr":"172.21.15.96:6827/1099399643","cluster_addr":"172.21.15.96:6829/1099399643","heartbeat_back_addr":"172.21.15.96:6833/1099399643","heartbeat_front_addr":"172.21.15.96:6831/1099399643","state":["exists","up"]},{"osd":10,"uuid":"197fee9b-eee4-4893-be1c-3d57f42250e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6825","nonce":155754520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6827","nonce":155754520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6831","nonce":155754520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":155754520},{"type":"v1","addr":"172.21.15.97:6829","nonce":155754520}]},"public_addr":"172.21.15.97:6825/155754520","cluster_addr":"172.21.15.97:6827/155754520","heartbeat_back_addr":"172.21.15.97:6831/155754520","heartbeat_front_addr":"172.21.15.97:6829/155754520","state":["exists","up"]},{"osd":11,"uuid":"9c1d0a83-fc12-41a2-8eb0-80c4abdd2770","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6825","nonce":2027958746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6827","nonce":2027958746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6831","nonce":2027958746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":2027958746},{"type":"v1","addr":"172.21.15.151:6829","nonce":2027958746}]},"public_addr":"172.21.15.151:6825/2027958746","cluster_addr":"172.21.15.151:6827/2027958746","heartbeat_back_addr":"172.21.15.151:6831/2027958746","heartbeat_front_addr":"172.21.15.151:6829/2027958746","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:05:56.177390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:06:19.917663+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:06:43.608052+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:08.540382+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:33.474072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:07:58.242306+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:08:24.120442+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:08:49.661064+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:09:15.097271+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:09:41.904505+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:10:08.233252+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:10:33.712296+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/3839807484":"2024-04-26T14:04:21.307574+0000","172.21.15.96:0/1495237551":"2024-04-26T14:04:21.307574+0000","172.21.15.96:0/2793751450":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/129575834":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/2233075139":"2024-04-26T14:04:21.307574+0000","172.21.15.96:6801/2348397481":"2024-04-26T14:03:54.249537+0000","172.21.15.96:6801/433591062":"2024-04-26T14:04:21.307574+0000","172.21.15.96:6800/2348397481":"2024-04-26T14:03:54.249537+0000","172.21.15.96:0/3517674924":"2024-04-26T14:03:54.249537+0000","172.21.15.96:6800/433591062":"2024-04-26T14:04:21.307574+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:16:33.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.094+0000 7f4d94fce640 1 -- 172.21.15.96:0/4004493244 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f4d780afaf0 msgr2=0x7f4d780b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:33.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.094+0000 7f4d94fce640 1 --2- 172.21.15.96:0/4004493244 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f4d780afaf0 0x7f4d780b2000 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4d80005fd0 tx=0x7f4d8000b410 comp rx=0 tx=0).stop 2024-04-25T14:16:33.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.094+0000 7f4d94fce640 1 -- 172.21.15.96:0/4004493244 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d901080c0 msgr2=0x7f4d901a1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:33.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.094+0000 7f4d94fce640 1 --2- 172.21.15.96:0/4004493244 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d901080c0 0x7f4d901a1470 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f4d880097b0 tx=0x7f4d88031840 comp rx=0 tx=0).stop 2024-04-25T14:16:33.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.094+0000 7f4d94fce640 1 -- 172.21.15.96:0/4004493244 shutdown_connections 2024-04-25T14:16:33.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.094+0000 7f4d94fce640 1 --2- 172.21.15.96:0/4004493244 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f4d780afaf0 0x7f4d780b2000 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:33.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.094+0000 7f4d94fce640 1 --2- 172.21.15.96:0/4004493244 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d901080c0 0x7f4d901a1470 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:33.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.094+0000 7f4d94fce640 1 --2- 172.21.15.96:0/4004493244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d901033c0 0x7f4d9019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:33.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.094+0000 7f4d94fce640 1 --2- 172.21.15.96:0/4004493244 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d90102180 0x7f4d9010b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:33.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.094+0000 7f4d94fce640 1 -- 172.21.15.96:0/4004493244 >> 172.21.15.96:0/4004493244 conn(0x7f4d900fd370 msgr2=0x7f4d901040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:33.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.095+0000 7f4d94fce640 1 -- 172.21.15.96:0/4004493244 shutdown_connections 2024-04-25T14:16:33.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.095+0000 7f4d94fce640 1 -- 172.21.15.96:0/4004493244 wait complete. 2024-04-25T14:16:33.107 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T14:16:33.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.217+0000 7f1e1da13640 1 -- 172.21.15.96:0/2047184954 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e18109890 msgr2=0x7f1e1810bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:33.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.217+0000 7f1e1da13640 1 --2- 172.21.15.96:0/2047184954 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e18109890 0x7f1e1810bcb0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f1e10009ca0 tx=0x7f1e1002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:16:33.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.218+0000 7f1e1da13640 1 -- 172.21.15.96:0/2047184954 shutdown_connections 2024-04-25T14:16:33.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.218+0000 7f1e1da13640 1 --2- 172.21.15.96:0/2047184954 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e18109890 0x7f1e1810bcb0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:33.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.218+0000 7f1e1da13640 1 --2- 172.21.15.96:0/2047184954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e18106f30 0x7f1e18109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:33.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.218+0000 7f1e1da13640 1 --2- 172.21.15.96:0/2047184954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e181045d0 0x7f1e181069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:33.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.218+0000 7f1e1da13640 1 -- 172.21.15.96:0/2047184954 >> 172.21.15.96:0/2047184954 conn(0x7f1e180fdbd0 msgr2=0x7f1e18100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:33.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.218+0000 7f1e1da13640 1 -- 172.21.15.96:0/2047184954 shutdown_connections 2024-04-25T14:16:33.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.218+0000 7f1e1da13640 1 -- 172.21.15.96:0/2047184954 wait complete. 2024-04-25T14:16:33.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.218+0000 7f1e1da13640 1 Processor -- start 2024-04-25T14:16:33.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.218+0000 7f1e1da13640 1 -- start start 2024-04-25T14:16:33.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.219+0000 7f1e1da13640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e181045d0 0x7f1e1819a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:33.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.219+0000 7f1e1da13640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e18106f30 0x7f1e1819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:33.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.219+0000 7f1e1da13640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e18109890 0x7f1e181a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:33.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.219+0000 7f1e1da13640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1810f5f0 con 0x7f1e18106f30 2024-04-25T14:16:33.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.219+0000 7f1e1da13640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1810f470 con 0x7f1e181045d0 2024-04-25T14:16:33.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.219+0000 7f1e1da13640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1810f770 con 0x7f1e18109890 2024-04-25T14:16:33.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.219+0000 7f1e167fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e18106f30 0x7f1e1819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:33.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.219+0000 7f1e167fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e18106f30 0x7f1e1819aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38868/0 (socket says 172.21.15.96:38868) 2024-04-25T14:16:33.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.219+0000 7f1e167fc640 1 -- 172.21.15.96:0/186114586 learned_addr learned my addr 172.21.15.96:0/186114586 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:16:33.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.219+0000 7f1e16ffd640 1 --2- 172.21.15.96:0/186114586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e181045d0 0x7f1e1819a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:33.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.219+0000 7f1e177fe640 1 --2- 172.21.15.96:0/186114586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e18109890 0x7f1e181a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:33.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.219+0000 7f1e167fc640 1 -- 172.21.15.96:0/186114586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e18109890 msgr2=0x7f1e181a19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:33.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.219+0000 7f1e167fc640 1 --2- 172.21.15.96:0/186114586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e18109890 0x7f1e181a19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:33.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.219+0000 7f1e167fc640 1 -- 172.21.15.96:0/186114586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e181045d0 msgr2=0x7f1e1819a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:33.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.219+0000 7f1e167fc640 1 --2- 172.21.15.96:0/186114586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e181045d0 0x7f1e1819a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:33.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.219+0000 7f1e167fc640 1 -- 172.21.15.96:0/186114586 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e181a2060 con 0x7f1e18106f30 2024-04-25T14:16:33.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.219+0000 7f1e167fc640 1 --2- 172.21.15.96:0/186114586 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e18106f30 0x7f1e1819aa60 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f1e0800e9f0 tx=0x7f1e0800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:33.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.219+0000 7f1e1ca11640 1 -- 172.21.15.96:0/186114586 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0800cd80 con 0x7f1e18106f30 2024-04-25T14:16:33.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.219+0000 7f1e1da13640 1 -- 172.21.15.96:0/186114586 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e181a2350 con 0x7f1e18106f30 2024-04-25T14:16:33.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.219+0000 7f1e1ca11640 1 -- 172.21.15.96:0/186114586 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e080108e0 con 0x7f1e18106f30 2024-04-25T14:16:33.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.219+0000 7f1e1ca11640 1 -- 172.21.15.96:0/186114586 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e08018980 con 0x7f1e18106f30 2024-04-25T14:16:33.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.220+0000 7f1e1da13640 1 -- 172.21.15.96:0/186114586 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e181a28a0 con 0x7f1e18106f30 2024-04-25T14:16:33.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.221+0000 7f1e1da13640 1 -- 172.21.15.96:0/186114586 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e1806c2a0 con 0x7f1e18106f30 2024-04-25T14:16:33.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.222+0000 7f1e1ca11640 1 -- 172.21.15.96:0/186114586 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f1e08010430 con 0x7f1e18106f30 2024-04-25T14:16:33.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.223+0000 7f1e1ca11640 1 --2- 172.21.15.96:0/186114586 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1e000afaf0 0x7f1e000b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:33.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.223+0000 7f1e16ffd640 1 --2- 172.21.15.96:0/186114586 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1e000afaf0 0x7f1e000b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:33.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.223+0000 7f1e1ca11640 1 -- 172.21.15.96:0/186114586 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f1e080df3f0 con 0x7f1e18106f30 2024-04-25T14:16:33.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.223+0000 7f1e16ffd640 1 --2- 172.21.15.96:0/186114586 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1e000afaf0 0x7f1e000b2000 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1e1806a8b0 tx=0x7f1e0c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:33.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.226+0000 7f1e1ca11640 1 -- 172.21.15.96:0/186114586 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1e080a7960 con 0x7f1e18106f30 2024-04-25T14:16:33.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.444+0000 7f1e1da13640 1 -- 172.21.15.96:0/186114586 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f1e181a2d60 con 0x7f1e18106f30 2024-04-25T14:16:33.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.446+0000 7f1e1ca11640 1 -- 172.21.15.96:0/186114586 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) ==== 161+0+0 (secure 0 0 0) 0x7f1e080abc60 con 0x7f1e18106f30 2024-04-25T14:16:33.447 INFO:teuthology.orchestra.run.smithi096.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T14:16:33.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.450+0000 7f1e1da13640 1 -- 172.21.15.96:0/186114586 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1e000afaf0 msgr2=0x7f1e000b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:33.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.450+0000 7f1e1da13640 1 --2- 172.21.15.96:0/186114586 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1e000afaf0 0x7f1e000b2000 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1e1806a8b0 tx=0x7f1e0c005f90 comp rx=0 tx=0).stop 2024-04-25T14:16:33.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.450+0000 7f1e1da13640 1 -- 172.21.15.96:0/186114586 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e18106f30 msgr2=0x7f1e1819aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:33.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.450+0000 7f1e1da13640 1 --2- 172.21.15.96:0/186114586 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e18106f30 0x7f1e1819aa60 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f1e0800e9f0 tx=0x7f1e0800eeb0 comp rx=0 tx=0).stop 2024-04-25T14:16:33.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.450+0000 7f1e1da13640 1 -- 172.21.15.96:0/186114586 shutdown_connections 2024-04-25T14:16:33.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.450+0000 7f1e1da13640 1 --2- 172.21.15.96:0/186114586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e18109890 0x7f1e181a19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:33.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.450+0000 7f1e1da13640 1 --2- 172.21.15.96:0/186114586 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f1e000afaf0 0x7f1e000b2000 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:33.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.450+0000 7f1e1da13640 1 --2- 172.21.15.96:0/186114586 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e18106f30 0x7f1e1819aa60 secure :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f1e0800e9f0 tx=0x7f1e0800eeb0 comp rx=0 tx=0).stop 2024-04-25T14:16:33.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.450+0000 7f1e1da13640 1 --2- 172.21.15.96:0/186114586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e181045d0 0x7f1e1819a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:33.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.450+0000 7f1e1da13640 1 -- 172.21.15.96:0/186114586 >> 172.21.15.96:0/186114586 conn(0x7f1e180fdbd0 msgr2=0x7f1e18105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:33.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.450+0000 7f1e1da13640 1 -- 172.21.15.96:0/186114586 shutdown_connections 2024-04-25T14:16:33.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.450+0000 7f1e1da13640 1 -- 172.21.15.96:0/186114586 wait complete. 2024-04-25T14:16:33.466 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T14:16:33.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.599+0000 7f27b6700640 1 -- 172.21.15.96:0/417529258 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27b0103c30 msgr2=0x7f27b0104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:33.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.599+0000 7f27b6700640 1 --2- 172.21.15.96:0/417529258 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27b0103c30 0x7f27b0104040 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f27a4009900 tx=0x7f27a402a410 comp rx=0 tx=0).stop 2024-04-25T14:16:33.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.599+0000 7f27b6700640 1 -- 172.21.15.96:0/417529258 shutdown_connections 2024-04-25T14:16:33.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.599+0000 7f27b6700640 1 --2- 172.21.15.96:0/417529258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27b0108930 0x7f27b010ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:33.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.599+0000 7f27b6700640 1 --2- 172.21.15.96:0/417529258 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27b0103c30 0x7f27b0104040 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:33.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.599+0000 7f27b6700640 1 --2- 172.21.15.96:0/417529258 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27b01029f0 0x7f27b01036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:33.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.599+0000 7f27b6700640 1 -- 172.21.15.96:0/417529258 >> 172.21.15.96:0/417529258 conn(0x7f27b00fdbe0 msgr2=0x7f27b0100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:33.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.599+0000 7f27b6700640 1 -- 172.21.15.96:0/417529258 shutdown_connections 2024-04-25T14:16:33.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.599+0000 7f27b6700640 1 -- 172.21.15.96:0/417529258 wait complete. 2024-04-25T14:16:33.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.599+0000 7f27b6700640 1 Processor -- start 2024-04-25T14:16:33.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.600+0000 7f27b6700640 1 -- start start 2024-04-25T14:16:33.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.600+0000 7f27b6700640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27b01029f0 0x7f27b006bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:33.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.600+0000 7f27b6700640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27b0103c30 0x7f27b019f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:33.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.600+0000 7f27b6700640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27b0108930 0x7f27b01a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:33.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.600+0000 7f27b6700640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f27b010e510 con 0x7f27b01029f0 2024-04-25T14:16:33.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.600+0000 7f27b6700640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f27b010e390 con 0x7f27b0108930 2024-04-25T14:16:33.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.600+0000 7f27b6700640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f27b010e690 con 0x7f27b0103c30 2024-04-25T14:16:33.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.600+0000 7f27affff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27b01029f0 0x7f27b006bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:33.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.600+0000 7f27affff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27b01029f0 0x7f27b006bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38948/0 (socket says 172.21.15.96:38948) 2024-04-25T14:16:33.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.600+0000 7f27affff640 1 -- 172.21.15.96:0/4210979076 learned_addr learned my addr 172.21.15.96:0/4210979076 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:16:33.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.600+0000 7f27b4c9d640 1 --2- 172.21.15.96:0/4210979076 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27b0108930 0x7f27b01a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:33.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.600+0000 7f27af7fe640 1 --2- 172.21.15.96:0/4210979076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27b0103c30 0x7f27b019f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:33.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.600+0000 7f27b4c9d640 1 -- 172.21.15.96:0/4210979076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27b0103c30 msgr2=0x7f27b019f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:33.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.600+0000 7f27b4c9d640 1 --2- 172.21.15.96:0/4210979076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27b0103c30 0x7f27b019f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:33.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.600+0000 7f27b4c9d640 1 -- 172.21.15.96:0/4210979076 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27b01029f0 msgr2=0x7f27b006bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:33.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.600+0000 7f27b4c9d640 1 --2- 172.21.15.96:0/4210979076 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27b01029f0 0x7f27b006bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:33.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.600+0000 7f27b4c9d640 1 -- 172.21.15.96:0/4210979076 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27b01a1df0 con 0x7f27b0108930 2024-04-25T14:16:33.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.600+0000 7f27affff640 1 --2- 172.21.15.96:0/4210979076 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27b01029f0 0x7f27b006bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:16:33.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.601+0000 7f27b4c9d640 1 --2- 172.21.15.96:0/4210979076 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27b0108930 0x7f27b01a15d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f27a800ec50 tx=0x7f27a800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:33.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.601+0000 7f27ad7fa640 1 -- 172.21.15.96:0/4210979076 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a8003b70 con 0x7f27b0108930 2024-04-25T14:16:33.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.601+0000 7f27b6700640 1 -- 172.21.15.96:0/4210979076 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27b01a20e0 con 0x7f27b0108930 2024-04-25T14:16:33.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.601+0000 7f27ad7fa640 1 -- 172.21.15.96:0/4210979076 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27a8003d10 con 0x7f27b0108930 2024-04-25T14:16:33.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.601+0000 7f27ad7fa640 1 -- 172.21.15.96:0/4210979076 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a8010830 con 0x7f27b0108930 2024-04-25T14:16:33.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.601+0000 7f27b6700640 1 -- 172.21.15.96:0/4210979076 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27b01a25e0 con 0x7f27b0108930 2024-04-25T14:16:33.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.602+0000 7f27b6700640 1 -- 172.21.15.96:0/4210979076 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27b0107740 con 0x7f27b0108930 2024-04-25T14:16:33.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.603+0000 7f27ad7fa640 1 -- 172.21.15.96:0/4210979076 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f27a8004b40 con 0x7f27b0108930 2024-04-25T14:16:33.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.604+0000 7f27ad7fa640 1 --2- 172.21.15.96:0/4210979076 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f27980afaf0 0x7f27980b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:33.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.604+0000 7f27ad7fa640 1 -- 172.21.15.96:0/4210979076 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8699+0+0 (secure 0 0 0) 0x7f27a80df500 con 0x7f27b0108930 2024-04-25T14:16:33.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.604+0000 7f27affff640 1 --2- 172.21.15.96:0/4210979076 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f27980afaf0 0x7f27980b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:33.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.604+0000 7f27affff640 1 --2- 172.21.15.96:0/4210979076 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f27980afaf0 0x7f27980b2000 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f27a0009fd0 tx=0x7f27a0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:33.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.608+0000 7f27ad7fa640 1 -- 172.21.15.96:0/4210979076 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f27a8076ff0 con 0x7f27b0108930 2024-04-25T14:16:33.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:33.831+0000 7f27b6700640 1 -- 172.21.15.96:0/4210979076 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f27b00637a0 con 0x7f27b0108930 2024-04-25T14:16:33.859 INFO:journalctl@ceph.mds.a.smithi096.stdout:Apr 25 14:16:33 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-a[70807]: ignoring --setuser ceph since I am not root 2024-04-25T14:16:33.859 INFO:journalctl@ceph.mds.a.smithi096.stdout:Apr 25 14:16:33 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-a[70807]: ignoring --setgroup ceph since I am not root 2024-04-25T14:16:33.860 INFO:journalctl@ceph.mds.a.smithi096.stdout:Apr 25 14:16:33 smithi096 ceph-mds[70813]: -- 172.21.15.96:0/1663361703 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561058aeba40 con 0x561058b33600 2024-04-25T14:16:33.860 INFO:journalctl@ceph.mds.a.smithi096.stdout:Apr 25 14:16:33 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-a[70807]: starting mds.a at 2024-04-25T14:16:33.860 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 14:16:33 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-g[70446]: ignoring --setuser ceph since I am not root 2024-04-25T14:16:33.860 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 14:16:33 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-g[70446]: ignoring --setgroup ceph since I am not root 2024-04-25T14:16:33.861 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 14:16:33 smithi096 ceph-mds[70450]: -- 172.21.15.96:0/3073653774 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55685b291a40 con 0x55685b2d9a80 2024-04-25T14:16:33.861 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 14:16:33 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-g[70446]: starting mds.g at 2024-04-25T14:16:33.861 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:33 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mon-a[29203]: 2024-04-25T14:16:33.445+0000 7f0df06a1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T14:16:33.861 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:33 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/853604729' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:16:33.861 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:33 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/4004493244' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:16:33.861 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:33 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/186114586' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T14:16:33.862 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:33 smithi096 ceph-mon[29207]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T14:16:33.862 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:33 smithi096 ceph-mon[29207]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T14:16:33.862 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:33 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/186114586' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T14:16:33.863 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:33 smithi096 ceph-mon[29207]: osdmap e79: 12 total, 12 up, 12 in 2024-04-25T14:16:33.863 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:33 smithi096 ceph-mon[29207]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-25T14:16:33.922 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 14:16:33 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-e[62606]: ignoring --setuser ceph since I am not root 2024-04-25T14:16:33.923 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 14:16:33 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-e[62606]: ignoring --setgroup ceph since I am not root 2024-04-25T14:16:33.923 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 14:16:33 smithi097 ceph-mds[62610]: -- 172.21.15.97:0/3869805828 <== mon.2 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56111c111a40 con 0x56111c159180 2024-04-25T14:16:33.923 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 14:16:33 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-e[62606]: starting mds.e at 2024-04-25T14:16:33.923 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 14:16:33 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-h[62915]: ignoring --setuser ceph since I am not root 2024-04-25T14:16:33.923 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 14:16:33 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-h[62915]: ignoring --setgroup ceph since I am not root 2024-04-25T14:16:33.923 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 14:16:33 smithi097 ceph-mds[62919]: -- 172.21.15.97:0/428758735 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5640dc27da40 con 0x5640dc2c5180 2024-04-25T14:16:33.923 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 14:16:33 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-h[62915]: starting mds.h at 2024-04-25T14:16:33.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:33 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/853604729' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:16:33.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:33 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/4004493244' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:16:33.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:33 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/186114586' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T14:16:33.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:33 smithi097 ceph-mon[32720]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T14:16:33.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:33 smithi097 ceph-mon[32720]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T14:16:33.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:33 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/186114586' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T14:16:33.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:33 smithi097 ceph-mon[32720]: osdmap e79: 12 total, 12 up, 12 in 2024-04-25T14:16:33.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:33 smithi097 ceph-mon[32720]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-25T14:16:33.924 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 14:16:33 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-b[63225]: ignoring --setuser ceph since I am not root 2024-04-25T14:16:33.925 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 14:16:33 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-b[63225]: ignoring --setgroup ceph since I am not root 2024-04-25T14:16:33.925 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 14:16:33 smithi097 ceph-mds[63229]: -- 172.21.15.97:0/3594982080 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x557185b224e0 con 0x557185b6d600 2024-04-25T14:16:33.925 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 14:16:33 smithi097 ceph-mds[63229]: -- 172.21.15.97:0/3594982080 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557185b25a40 con 0x557185b6d600 2024-04-25T14:16:33.925 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 14:16:33 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-b[63225]: starting mds.b at 2024-04-25T14:16:33.925 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 14:16:33 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-k[62290]: ignoring --setuser ceph since I am not root 2024-04-25T14:16:33.925 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 14:16:33 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-k[62290]: ignoring --setgroup ceph since I am not root 2024-04-25T14:16:33.925 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 14:16:33 smithi097 ceph-mds[62294]: -- 172.21.15.97:0/3572511313 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x557a80fd24e0 con 0x557a8101d600 2024-04-25T14:16:33.925 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 14:16:33 smithi097 ceph-mds[62294]: -- 172.21.15.97:0/3572511313 <== mon.2 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557a80fd5a40 con 0x557a8101d600 2024-04-25T14:16:33.925 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 14:16:33 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-k[62290]: starting mds.k at 2024-04-25T14:16:33.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:33 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/853604729' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:16:33.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:33 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/4004493244' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:16:33.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:33 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/186114586' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T14:16:33.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:33 smithi151 ceph-mon[32990]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T14:16:33.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:33 smithi151 ceph-mon[32990]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T14:16:33.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:33 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/186114586' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T14:16:33.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:33 smithi151 ceph-mon[32990]: osdmap e79: 12 total, 12 up, 12 in 2024-04-25T14:16:33.940 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:33 smithi151 ceph-mon[32990]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-25T14:16:33.940 INFO:journalctl@ceph.mds.f.smithi151.stdout:Apr 25 14:16:33 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-f[63245]: ignoring --setuser ceph since I am not root 2024-04-25T14:16:33.940 INFO:journalctl@ceph.mds.f.smithi151.stdout:Apr 25 14:16:33 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-f[63245]: ignoring --setgroup ceph since I am not root 2024-04-25T14:16:33.941 INFO:journalctl@ceph.mds.f.smithi151.stdout:Apr 25 14:16:33 smithi151 ceph-mds[63249]: -- 172.21.15.151:0/1364588543 <== mon.2 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558ef7b67a40 con 0x558ef7bafa80 2024-04-25T14:16:33.941 INFO:journalctl@ceph.mds.f.smithi151.stdout:Apr 25 14:16:33 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-f[63245]: starting mds.f at 2024-04-25T14:16:33.941 INFO:journalctl@ceph.mds.i.smithi151.stdout:Apr 25 14:16:33 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-i[63862]: ignoring --setuser ceph since I am not root 2024-04-25T14:16:33.941 INFO:journalctl@ceph.mds.i.smithi151.stdout:Apr 25 14:16:33 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-i[63862]: ignoring --setgroup ceph since I am not root 2024-04-25T14:16:33.941 INFO:journalctl@ceph.mds.i.smithi151.stdout:Apr 25 14:16:33 smithi151 ceph-mds[63867]: -- 172.21.15.151:0/3931325924 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563e274fba40 con 0x563e27543180 2024-04-25T14:16:33.941 INFO:journalctl@ceph.mds.i.smithi151.stdout:Apr 25 14:16:33 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-i[63862]: starting mds.i at 2024-04-25T14:16:33.941 INFO:journalctl@ceph.mds.l.smithi151.stdout:Apr 25 14:16:33 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-l[62926]: ignoring --setuser ceph since I am not root 2024-04-25T14:16:33.942 INFO:journalctl@ceph.mds.l.smithi151.stdout:Apr 25 14:16:33 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-l[62926]: ignoring --setgroup ceph since I am not root 2024-04-25T14:16:33.942 INFO:journalctl@ceph.mds.l.smithi151.stdout:Apr 25 14:16:33 smithi151 ceph-mds[62930]: -- 172.21.15.151:0/217146410 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562f92c484e0 con 0x562f92c93a80 2024-04-25T14:16:33.942 INFO:journalctl@ceph.mds.l.smithi151.stdout:Apr 25 14:16:33 smithi151 ceph-mds[62930]: -- 172.21.15.151:0/217146410 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562f92c4ba40 con 0x562f92c93a80 2024-04-25T14:16:33.942 INFO:journalctl@ceph.mds.l.smithi151.stdout:Apr 25 14:16:33 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-l[62926]: starting mds.l at 2024-04-25T14:16:33.942 INFO:journalctl@ceph.mds.c.smithi151.stdout:Apr 25 14:16:33 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-c[63552]: ignoring --setuser ceph since I am not root 2024-04-25T14:16:33.942 INFO:journalctl@ceph.mds.c.smithi151.stdout:Apr 25 14:16:33 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-c[63552]: ignoring --setgroup ceph since I am not root 2024-04-25T14:16:33.942 INFO:journalctl@ceph.mds.c.smithi151.stdout:Apr 25 14:16:33 smithi151 ceph-mds[63556]: -- 172.21.15.151:0/324937201 <== mon.0 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5587e842ba40 con 0x5587e8473a80 2024-04-25T14:16:33.942 INFO:journalctl@ceph.mds.c.smithi151.stdout:Apr 25 14:16:33 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-c[63552]: starting mds.c at 2024-04-25T14:16:34.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.455+0000 7f27ad7fa640 1 -- 172.21.15.96:0/4210979076 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) ==== 104+0+0 (secure 0 0 0) 0x7f27a80a7f10 con 0x7f27b0108930 2024-04-25T14:16:34.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.458+0000 7f27b6700640 1 -- 172.21.15.96:0/4210979076 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f27980afaf0 msgr2=0x7f27980b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:34.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.458+0000 7f27b6700640 1 --2- 172.21.15.96:0/4210979076 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f27980afaf0 0x7f27980b2000 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f27a0009fd0 tx=0x7f27a0005f90 comp rx=0 tx=0).stop 2024-04-25T14:16:34.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.458+0000 7f27b6700640 1 -- 172.21.15.96:0/4210979076 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27b0108930 msgr2=0x7f27b01a15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:34.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.458+0000 7f27b6700640 1 --2- 172.21.15.96:0/4210979076 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27b0108930 0x7f27b01a15d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f27a800ec50 tx=0x7f27a800c6a0 comp rx=0 tx=0).stop 2024-04-25T14:16:34.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.458+0000 7f27b6700640 1 -- 172.21.15.96:0/4210979076 shutdown_connections 2024-04-25T14:16:34.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.458+0000 7f27b6700640 1 --2- 172.21.15.96:0/4210979076 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27b0108930 0x7f27b01a15d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:34.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.458+0000 7f27b6700640 1 --2- 172.21.15.96:0/4210979076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27b0103c30 0x7f27b019f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:34.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.458+0000 7f27b6700640 1 --2- 172.21.15.96:0/4210979076 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f27980afaf0 0x7f27980b2000 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:34.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.458+0000 7f27b6700640 1 --2- 172.21.15.96:0/4210979076 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27b01029f0 0x7f27b006bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:34.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.458+0000 7f27b6700640 1 -- 172.21.15.96:0/4210979076 >> 172.21.15.96:0/4210979076 conn(0x7f27b00fdbe0 msgr2=0x7f27b0104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:34.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.458+0000 7f27b6700640 1 -- 172.21.15.96:0/4210979076 shutdown_connections 2024-04-25T14:16:34.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.458+0000 7f27b6700640 1 -- 172.21.15.96:0/4210979076 wait complete. 2024-04-25T14:16:34.471 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T14:16:34.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.583+0000 7f7cbe74d640 1 -- 172.21.15.96:0/309894065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7cb8109a40 msgr2=0x7f7cb810be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:34.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.583+0000 7f7cbe74d640 1 --2- 172.21.15.96:0/309894065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7cb8109a40 0x7f7cb810be60 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0009ca0 tx=0x7f7cb002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:16:34.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.583+0000 7f7cbe74d640 1 -- 172.21.15.96:0/309894065 shutdown_connections 2024-04-25T14:16:34.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.583+0000 7f7cbe74d640 1 --2- 172.21.15.96:0/309894065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7cb8109a40 0x7f7cb810be60 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:34.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.583+0000 7f7cbe74d640 1 --2- 172.21.15.96:0/309894065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7cb8104d40 0x7f7cb8105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:34.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.583+0000 7f7cbe74d640 1 --2- 172.21.15.96:0/309894065 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cb8103b00 0x7f7cb8104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:34.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.583+0000 7f7cbe74d640 1 -- 172.21.15.96:0/309894065 >> 172.21.15.96:0/309894065 conn(0x7f7cb80fed10 msgr2=0x7f7cb81011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:34.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.583+0000 7f7cbe74d640 1 -- 172.21.15.96:0/309894065 shutdown_connections 2024-04-25T14:16:34.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.583+0000 7f7cbe74d640 1 -- 172.21.15.96:0/309894065 wait complete. 2024-04-25T14:16:34.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.584+0000 7f7cbe74d640 1 Processor -- start 2024-04-25T14:16:34.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.584+0000 7f7cbe74d640 1 -- start start 2024-04-25T14:16:34.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.584+0000 7f7cbe74d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7cb8103b00 0x7f7cb819a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:34.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.584+0000 7f7cbe74d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7cb8104d40 0x7f7cb819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:34.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.584+0000 7f7cbe74d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cb8109a40 0x7f7cb81a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:34.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.585+0000 7f7cbe74d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb810f630 con 0x7f7cb8103b00 2024-04-25T14:16:34.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.585+0000 7f7cbe74d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb810f4b0 con 0x7f7cb8104d40 2024-04-25T14:16:34.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.585+0000 7f7cbe74d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb810f7b0 con 0x7f7cb8109a40 2024-04-25T14:16:34.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.585+0000 7f7cb7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7cb8103b00 0x7f7cb819a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:34.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.585+0000 7f7cb7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7cb8103b00 0x7f7cb819a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38974/0 (socket says 172.21.15.96:38974) 2024-04-25T14:16:34.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.585+0000 7f7cb7fff640 1 -- 172.21.15.96:0/2308648534 learned_addr learned my addr 172.21.15.96:0/2308648534 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:16:34.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.585+0000 7f7cb77fe640 1 --2- 172.21.15.96:0/2308648534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7cb8104d40 0x7f7cb819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:34.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.585+0000 7f7cbccea640 1 --2- 172.21.15.96:0/2308648534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cb8109a40 0x7f7cb81a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:34.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.585+0000 7f7cb7fff640 1 -- 172.21.15.96:0/2308648534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cb8109a40 msgr2=0x7f7cb81a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:34.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.585+0000 7f7cb7fff640 1 --2- 172.21.15.96:0/2308648534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cb8109a40 0x7f7cb81a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:34.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.585+0000 7f7cb7fff640 1 -- 172.21.15.96:0/2308648534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7cb8104d40 msgr2=0x7f7cb819a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:34.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.585+0000 7f7cb7fff640 1 --2- 172.21.15.96:0/2308648534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7cb8104d40 0x7f7cb819a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:34.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.585+0000 7f7cb7fff640 1 -- 172.21.15.96:0/2308648534 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cb81a1d50 con 0x7f7cb8103b00 2024-04-25T14:16:34.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.585+0000 7f7cb7fff640 1 --2- 172.21.15.96:0/2308648534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7cb8103b00 0x7f7cb819a210 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8005d00 tx=0x7f7ca800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:34.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.589+0000 7f7cb57fa640 1 -- 172.21.15.96:0/2308648534 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca8017070 con 0x7f7cb8103b00 2024-04-25T14:16:34.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.589+0000 7f7cb57fa640 1 -- 172.21.15.96:0/2308648534 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ca8009c30 con 0x7f7cb8103b00 2024-04-25T14:16:34.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.589+0000 7f7cbe74d640 1 -- 172.21.15.96:0/2308648534 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cb81a1fe0 con 0x7f7cb8103b00 2024-04-25T14:16:34.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.589+0000 7f7cb57fa640 1 -- 172.21.15.96:0/2308648534 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca800f620 con 0x7f7cb8103b00 2024-04-25T14:16:34.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.589+0000 7f7cbe74d640 1 -- 172.21.15.96:0/2308648534 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cb81a24a0 con 0x7f7cb8103b00 2024-04-25T14:16:34.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.590+0000 7f7cbe74d640 1 -- 172.21.15.96:0/2308648534 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cb8108850 con 0x7f7cb8103b00 2024-04-25T14:16:34.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.592+0000 7f7cb57fa640 1 -- 172.21.15.96:0/2308648534 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f7ca801a070 con 0x7f7cb8103b00 2024-04-25T14:16:34.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.592+0000 7f7cb57fa640 1 --2- 172.21.15.96:0/2308648534 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f7ca00afb70 0x7f7ca00b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:34.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.592+0000 7f7cb77fe640 1 --2- 172.21.15.96:0/2308648534 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f7ca00afb70 0x7f7ca00b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:34.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.592+0000 7f7cb57fa640 1 -- 172.21.15.96:0/2308648534 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8699+0+0 (secure 0 0 0) 0x7f7ca80de710 con 0x7f7cb8103b00 2024-04-25T14:16:34.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.593+0000 7f7cb77fe640 1 --2- 172.21.15.96:0/2308648534 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f7ca00afb70 0x7f7ca00b2080 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7cb819b2b0 tx=0x7f7cac005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:34.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.597+0000 7f7cb57fa640 1 -- 172.21.15.96:0/2308648534 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7ca8009de0 con 0x7f7cb8103b00 2024-04-25T14:16:34.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:34 smithi096 ceph-mon[29207]: pgmap v435: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 307 B/s wr, 10 op/s 2024-04-25T14:16:34.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:34 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/4210979076' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T14:16:34.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:34 smithi096 ceph-mon[29207]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T14:16:34.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:34 smithi096 ceph-mon[29207]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T14:16:34.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:34 smithi096 ceph-mon[29207]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T14:16:34.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:34 smithi096 ceph-mon[29207]: Cluster is now healthy 2024-04-25T14:16:34.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:34 smithi096 ceph-mon[29207]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T14:16:34.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:34 smithi096 ceph-mon[29207]: mds.? [v2:172.21.15.151:6832/576976460,v1:172.21.15.151:6836/576976460] up:boot 2024-04-25T14:16:34.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:34 smithi096 ceph-mon[29207]: mds.? [v2:172.21.15.151:6833/3082251983,v1:172.21.15.151:6837/3082251983] up:boot 2024-04-25T14:16:34.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:34 smithi096 ceph-mon[29207]: mds.? [v2:172.21.15.151:6835/1973836709,v1:172.21.15.151:6838/1973836709] up:boot 2024-04-25T14:16:34.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:34 smithi096 ceph-mon[29207]: mds.? [v2:172.21.15.151:6834/2026250965,v1:172.21.15.151:6839/2026250965] up:boot 2024-04-25T14:16:34.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:34 smithi096 ceph-mon[29207]: mds.? [v2:172.21.15.97:6832/796042906,v1:172.21.15.97:6833/796042906] up:boot 2024-04-25T14:16:34.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:34 smithi096 ceph-mon[29207]: mds.? [v2:172.21.15.96:6839/2026622544,v1:172.21.15.96:6841/2026622544] up:boot 2024-04-25T14:16:34.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:34 smithi096 ceph-mon[29207]: mds.? [v2:172.21.15.96:6838/2027137285,v1:172.21.15.96:6840/2027137285] up:boot 2024-04-25T14:16:34.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:34 smithi096 ceph-mon[29207]: mds.? [v2:172.21.15.97:6835/1795966922,v1:172.21.15.97:6837/1795966922] up:boot 2024-04-25T14:16:34.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:34 smithi096 ceph-mon[29207]: mds.? [v2:172.21.15.97:6834/714167328,v1:172.21.15.97:6836/714167328] up:boot 2024-04-25T14:16:34.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:34 smithi096 ceph-mon[29207]: mds.? [v2:172.21.15.97:6838/1676747278,v1:172.21.15.97:6839/1676747278] up:boot 2024-04-25T14:16:34.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:34 smithi096 ceph-mon[29207]: fsmap 12 up:standby 2024-04-25T14:16:34.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:34 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T14:16:34.611 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:34 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T14:16:34.611 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:34 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T14:16:34.611 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:34 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T14:16:34.611 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:34 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T14:16:34.611 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:34 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T14:16:34.611 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:34 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T14:16:34.611 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:34 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T14:16:34.611 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:34 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T14:16:34.611 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:34 smithi096 ceph-mon[29207]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T14:16:34.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:34.787+0000 7f7cbe74d640 1 -- 172.21.15.96:0/2308648534 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7cb802db50 con 0x7f7cb8103b00 2024-04-25T14:16:34.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:34 smithi097 ceph-mon[32720]: pgmap v435: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 307 B/s wr, 10 op/s 2024-04-25T14:16:34.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:34 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/4210979076' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T14:16:34.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:34 smithi097 ceph-mon[32720]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T14:16:34.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:34 smithi097 ceph-mon[32720]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T14:16:34.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:34 smithi097 ceph-mon[32720]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T14:16:34.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:34 smithi097 ceph-mon[32720]: Cluster is now healthy 2024-04-25T14:16:34.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:34 smithi097 ceph-mon[32720]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T14:16:34.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:34 smithi097 ceph-mon[32720]: mds.? [v2:172.21.15.151:6832/576976460,v1:172.21.15.151:6836/576976460] up:boot 2024-04-25T14:16:34.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:34 smithi097 ceph-mon[32720]: mds.? [v2:172.21.15.151:6833/3082251983,v1:172.21.15.151:6837/3082251983] up:boot 2024-04-25T14:16:34.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:34 smithi097 ceph-mon[32720]: mds.? [v2:172.21.15.151:6835/1973836709,v1:172.21.15.151:6838/1973836709] up:boot 2024-04-25T14:16:34.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:34 smithi097 ceph-mon[32720]: mds.? [v2:172.21.15.151:6834/2026250965,v1:172.21.15.151:6839/2026250965] up:boot 2024-04-25T14:16:34.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:34 smithi097 ceph-mon[32720]: mds.? [v2:172.21.15.97:6832/796042906,v1:172.21.15.97:6833/796042906] up:boot 2024-04-25T14:16:34.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:34 smithi097 ceph-mon[32720]: mds.? [v2:172.21.15.96:6839/2026622544,v1:172.21.15.96:6841/2026622544] up:boot 2024-04-25T14:16:34.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:34 smithi097 ceph-mon[32720]: mds.? [v2:172.21.15.96:6838/2027137285,v1:172.21.15.96:6840/2027137285] up:boot 2024-04-25T14:16:34.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:34 smithi097 ceph-mon[32720]: mds.? [v2:172.21.15.97:6835/1795966922,v1:172.21.15.97:6837/1795966922] up:boot 2024-04-25T14:16:34.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:34 smithi097 ceph-mon[32720]: mds.? [v2:172.21.15.97:6834/714167328,v1:172.21.15.97:6836/714167328] up:boot 2024-04-25T14:16:34.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:34 smithi097 ceph-mon[32720]: mds.? [v2:172.21.15.97:6838/1676747278,v1:172.21.15.97:6839/1676747278] up:boot 2024-04-25T14:16:34.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:34 smithi097 ceph-mon[32720]: fsmap 12 up:standby 2024-04-25T14:16:34.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:34 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T14:16:34.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:34 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T14:16:34.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:34 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T14:16:34.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:34 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T14:16:34.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:34 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T14:16:34.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:34 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T14:16:34.924 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:34 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T14:16:34.925 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:34 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T14:16:34.925 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:34 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T14:16:34.925 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:34 smithi097 ceph-mon[32720]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T14:16:34.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:34 smithi151 ceph-mon[32990]: pgmap v435: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 307 B/s wr, 10 op/s 2024-04-25T14:16:34.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:34 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/4210979076' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T14:16:34.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:34 smithi151 ceph-mon[32990]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T14:16:34.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:34 smithi151 ceph-mon[32990]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T14:16:34.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:34 smithi151 ceph-mon[32990]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T14:16:34.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:34 smithi151 ceph-mon[32990]: Cluster is now healthy 2024-04-25T14:16:34.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:34 smithi151 ceph-mon[32990]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T14:16:34.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:34 smithi151 ceph-mon[32990]: mds.? [v2:172.21.15.151:6832/576976460,v1:172.21.15.151:6836/576976460] up:boot 2024-04-25T14:16:34.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:34 smithi151 ceph-mon[32990]: mds.? [v2:172.21.15.151:6833/3082251983,v1:172.21.15.151:6837/3082251983] up:boot 2024-04-25T14:16:34.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:34 smithi151 ceph-mon[32990]: mds.? [v2:172.21.15.151:6835/1973836709,v1:172.21.15.151:6838/1973836709] up:boot 2024-04-25T14:16:34.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:34 smithi151 ceph-mon[32990]: mds.? [v2:172.21.15.151:6834/2026250965,v1:172.21.15.151:6839/2026250965] up:boot 2024-04-25T14:16:34.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:34 smithi151 ceph-mon[32990]: mds.? [v2:172.21.15.97:6832/796042906,v1:172.21.15.97:6833/796042906] up:boot 2024-04-25T14:16:34.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:34 smithi151 ceph-mon[32990]: mds.? [v2:172.21.15.96:6839/2026622544,v1:172.21.15.96:6841/2026622544] up:boot 2024-04-25T14:16:34.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:34 smithi151 ceph-mon[32990]: mds.? [v2:172.21.15.96:6838/2027137285,v1:172.21.15.96:6840/2027137285] up:boot 2024-04-25T14:16:34.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:34 smithi151 ceph-mon[32990]: mds.? [v2:172.21.15.97:6835/1795966922,v1:172.21.15.97:6837/1795966922] up:boot 2024-04-25T14:16:34.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:34 smithi151 ceph-mon[32990]: mds.? [v2:172.21.15.97:6834/714167328,v1:172.21.15.97:6836/714167328] up:boot 2024-04-25T14:16:34.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:34 smithi151 ceph-mon[32990]: mds.? [v2:172.21.15.97:6838/1676747278,v1:172.21.15.97:6839/1676747278] up:boot 2024-04-25T14:16:34.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:34 smithi151 ceph-mon[32990]: fsmap 12 up:standby 2024-04-25T14:16:34.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:34 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T14:16:34.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:34 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T14:16:34.940 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:34 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T14:16:34.940 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:34 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T14:16:34.940 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:34 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T14:16:34.940 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:34 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T14:16:34.940 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:34 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T14:16:34.940 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:34 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T14:16:34.940 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:34 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T14:16:34.940 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:34 smithi151 ceph-mon[32990]: from='mgr.14152 172.21.15.96:0/451258616' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T14:16:35.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.594+0000 7f7cb57fa640 1 -- 172.21.15.96:0/2308648534 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v80) ==== 181+0+0 (secure 0 0 0) 0x7f7ca80a6ad0 con 0x7f7cb8103b00 2024-04-25T14:16:35.594 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_metadata' removed 2024-04-25T14:16:35.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.597+0000 7f7cbe74d640 1 -- 172.21.15.96:0/2308648534 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f7ca00afb70 msgr2=0x7f7ca00b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:35.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.597+0000 7f7cbe74d640 1 --2- 172.21.15.96:0/2308648534 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f7ca00afb70 0x7f7ca00b2080 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7cb819b2b0 tx=0x7f7cac005f90 comp rx=0 tx=0).stop 2024-04-25T14:16:35.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.597+0000 7f7cbe74d640 1 -- 172.21.15.96:0/2308648534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7cb8103b00 msgr2=0x7f7cb819a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:35.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.597+0000 7f7cbe74d640 1 --2- 172.21.15.96:0/2308648534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7cb8103b00 0x7f7cb819a210 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8005d00 tx=0x7f7ca800dde0 comp rx=0 tx=0).stop 2024-04-25T14:16:35.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.597+0000 7f7cbe74d640 1 -- 172.21.15.96:0/2308648534 shutdown_connections 2024-04-25T14:16:35.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.597+0000 7f7cbe74d640 1 --2- 172.21.15.96:0/2308648534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cb8109a40 0x7f7cb81a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:35.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.597+0000 7f7cbe74d640 1 --2- 172.21.15.96:0/2308648534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7cb8104d40 0x7f7cb819a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:35.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.597+0000 7f7cbe74d640 1 --2- 172.21.15.96:0/2308648534 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7f7ca00afb70 0x7f7ca00b2080 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:35.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.597+0000 7f7cbe74d640 1 --2- 172.21.15.96:0/2308648534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7cb8103b00 0x7f7cb819a210 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:35.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.597+0000 7f7cbe74d640 1 -- 172.21.15.96:0/2308648534 >> 172.21.15.96:0/2308648534 conn(0x7f7cb80fed10 msgr2=0x7f7cb8105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:35.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.597+0000 7f7cbe74d640 1 -- 172.21.15.96:0/2308648534 shutdown_connections 2024-04-25T14:16:35.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.597+0000 7f7cbe74d640 1 -- 172.21.15.96:0/2308648534 wait complete. 2024-04-25T14:16:35.613 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T14:16:35.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.731+0000 7fcba14e2640 1 -- 172.21.15.96:0/340635446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb9c103c20 msgr2=0x7fcb9c104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:35.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.731+0000 7fcba14e2640 1 --2- 172.21.15.96:0/340635446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb9c103c20 0x7fcb9c104030 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fcb84009900 tx=0x7fcb8402a410 comp rx=0 tx=0).stop 2024-04-25T14:16:35.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.731+0000 7fcba14e2640 1 -- 172.21.15.96:0/340635446 shutdown_connections 2024-04-25T14:16:35.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.731+0000 7fcba14e2640 1 --2- 172.21.15.96:0/340635446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb9c108920 0x7fcb9c10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:35.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.731+0000 7fcba14e2640 1 --2- 172.21.15.96:0/340635446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb9c103c20 0x7fcb9c104030 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:35.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.731+0000 7fcba14e2640 1 --2- 172.21.15.96:0/340635446 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb9c1029e0 0x7fcb9c1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:35.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.731+0000 7fcba14e2640 1 -- 172.21.15.96:0/340635446 >> 172.21.15.96:0/340635446 conn(0x7fcb9c0fdbd0 msgr2=0x7fcb9c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:35.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.731+0000 7fcba14e2640 1 -- 172.21.15.96:0/340635446 shutdown_connections 2024-04-25T14:16:35.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.731+0000 7fcba14e2640 1 -- 172.21.15.96:0/340635446 wait complete. 2024-04-25T14:16:35.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.732+0000 7fcba14e2640 1 Processor -- start 2024-04-25T14:16:35.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.732+0000 7fcba14e2640 1 -- start start 2024-04-25T14:16:35.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.732+0000 7fcba14e2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb9c1029e0 0x7fcb9c06bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:35.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.732+0000 7fcba14e2640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb9c103c20 0x7fcb9c19f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:35.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.732+0000 7fcba14e2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb9c108920 0x7fcb9c1a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:35.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.732+0000 7fcba14e2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9c10e500 con 0x7fcb9c1029e0 2024-04-25T14:16:35.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.732+0000 7fcba14e2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9c10e380 con 0x7fcb9c108920 2024-04-25T14:16:35.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.732+0000 7fcba14e2640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9c10e680 con 0x7fcb9c103c20 2024-04-25T14:16:35.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.732+0000 7fcb9affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb9c1029e0 0x7fcb9c06bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:35.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.732+0000 7fcb9affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb9c1029e0 0x7fcb9c06bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38996/0 (socket says 172.21.15.96:38996) 2024-04-25T14:16:35.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.732+0000 7fcb9affd640 1 -- 172.21.15.96:0/2989827481 learned_addr learned my addr 172.21.15.96:0/2989827481 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T14:16:35.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.733+0000 7fcb9b7fe640 1 --2- 172.21.15.96:0/2989827481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb9c108920 0x7fcb9c1a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:35.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.733+0000 7fcb93fff640 1 --2- 172.21.15.96:0/2989827481 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb9c103c20 0x7fcb9c19f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:35.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.733+0000 7fcb9affd640 1 -- 172.21.15.96:0/2989827481 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb9c103c20 msgr2=0x7fcb9c19f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:35.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.733+0000 7fcb9affd640 1 --2- 172.21.15.96:0/2989827481 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb9c103c20 0x7fcb9c19f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:35.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.733+0000 7fcb9affd640 1 -- 172.21.15.96:0/2989827481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb9c108920 msgr2=0x7fcb9c1a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:35.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.733+0000 7fcb9affd640 1 --2- 172.21.15.96:0/2989827481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb9c108920 0x7fcb9c1a17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:35.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.733+0000 7fcb9affd640 1 -- 172.21.15.96:0/2989827481 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb9c1a2010 con 0x7fcb9c1029e0 2024-04-25T14:16:35.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.733+0000 7fcb9b7fe640 1 --2- 172.21.15.96:0/2989827481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb9c108920 0x7fcb9c1a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:16:35.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.733+0000 7fcb9affd640 1 --2- 172.21.15.96:0/2989827481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb9c1029e0 0x7fcb9c06bc60 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c00c970 tx=0x7fcb8c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:35.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.733+0000 7fcb98ff9640 1 -- 172.21.15.96:0/2989827481 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c004220 con 0x7fcb9c1029e0 2024-04-25T14:16:35.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.733+0000 7fcb98ff9640 1 -- 172.21.15.96:0/2989827481 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb8c0043c0 con 0x7fcb9c1029e0 2024-04-25T14:16:35.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.733+0000 7fcba14e2640 1 -- 172.21.15.96:0/2989827481 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb9c1a2300 con 0x7fcb9c1029e0 2024-04-25T14:16:35.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.733+0000 7fcb98ff9640 1 -- 172.21.15.96:0/2989827481 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c004a00 con 0x7fcb9c1029e0 2024-04-25T14:16:35.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.733+0000 7fcba14e2640 1 -- 172.21.15.96:0/2989827481 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb9c1a2800 con 0x7fcb9c1029e0 2024-04-25T14:16:35.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.734+0000 7fcba14e2640 1 -- 172.21.15.96:0/2989827481 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb9c107730 con 0x7fcb9c1029e0 2024-04-25T14:16:35.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.736+0000 7fcb98ff9640 1 -- 172.21.15.96:0/2989827481 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fcb8c0079b0 con 0x7fcb9c1029e0 2024-04-25T14:16:35.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.736+0000 7fcb98ff9640 1 --2- 172.21.15.96:0/2989827481 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fcb880afaf0 0x7fcb880b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:35.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.736+0000 7fcb93fff640 1 --2- 172.21.15.96:0/2989827481 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fcb880afaf0 0x7fcb880b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:35.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.736+0000 7fcb98ff9640 1 -- 172.21.15.96:0/2989827481 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8145+0+0 (secure 0 0 0) 0x7fcb8c0dee10 con 0x7fcb9c1029e0 2024-04-25T14:16:35.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.737+0000 7fcb93fff640 1 --2- 172.21.15.96:0/2989827481 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fcb880afaf0 0x7fcb880b2000 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcb8402a930 tx=0x7fcb84005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:35.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.740+0000 7fcb98ff9640 1 -- 172.21.15.96:0/2989827481 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcb8c014030 con 0x7fcb9c1029e0 2024-04-25T14:16:35.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:35 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/2308648534' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T14:16:35.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:35.919+0000 7fcba14e2640 1 -- 172.21.15.96:0/2989827481 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fcb9c069b70 con 0x7fcb9c1029e0 2024-04-25T14:16:35.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:35 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/2308648534' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T14:16:35.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:35 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/2308648534' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T14:16:36.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:36.596+0000 7fcb98ff9640 1 -- 172.21.15.96:0/2989827481 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v81) ==== 169+0+0 (secure 0 0 0) 0x7fcb8c0a7a90 con 0x7fcb9c1029e0 2024-04-25T14:16:36.596 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_data' removed 2024-04-25T14:16:36.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:36.598+0000 7fcba14e2640 1 -- 172.21.15.96:0/2989827481 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fcb880afaf0 msgr2=0x7fcb880b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:36.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:36.598+0000 7fcba14e2640 1 --2- 172.21.15.96:0/2989827481 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fcb880afaf0 0x7fcb880b2000 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcb8402a930 tx=0x7fcb84005fd0 comp rx=0 tx=0).stop 2024-04-25T14:16:36.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:36.599+0000 7fcba14e2640 1 -- 172.21.15.96:0/2989827481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb9c1029e0 msgr2=0x7fcb9c06bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:36.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:36.599+0000 7fcba14e2640 1 --2- 172.21.15.96:0/2989827481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb9c1029e0 0x7fcb9c06bc60 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c00c970 tx=0x7fcb8c00ce30 comp rx=0 tx=0).stop 2024-04-25T14:16:36.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:36.599+0000 7fcba14e2640 1 -- 172.21.15.96:0/2989827481 shutdown_connections 2024-04-25T14:16:36.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:36.599+0000 7fcba14e2640 1 --2- 172.21.15.96:0/2989827481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb9c108920 0x7fcb9c1a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:36.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:36.599+0000 7fcba14e2640 1 --2- 172.21.15.96:0/2989827481 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb9c103c20 0x7fcb9c19f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:36.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:36.599+0000 7fcba14e2640 1 --2- 172.21.15.96:0/2989827481 >> [v2:172.21.15.96:6800/536332425,v1:172.21.15.96:6801/536332425] conn(0x7fcb880afaf0 0x7fcb880b2000 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:36.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:36.599+0000 7fcba14e2640 1 --2- 172.21.15.96:0/2989827481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb9c1029e0 0x7fcb9c06bc60 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:36.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:36.599+0000 7fcba14e2640 1 -- 172.21.15.96:0/2989827481 >> 172.21.15.96:0/2989827481 conn(0x7fcb9c0fdbd0 msgr2=0x7fcb9c104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:36.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:36.599+0000 7fcba14e2640 1 -- 172.21.15.96:0/2989827481 shutdown_connections 2024-04-25T14:16:36.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T14:16:36.599+0000 7fcba14e2640 1 -- 172.21.15.96:0/2989827481 wait complete. 2024-04-25T14:16:36.617 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T14:16:36.642 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T14:16:36.681 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T14:16:36.711 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-25T14:16:36.711 DEBUG:teuthology.orchestra.run.smithi096:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T14:16:36.727 DEBUG:teuthology.orchestra.run.smithi097:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T14:16:36.743 DEBUG:teuthology.orchestra.run.smithi151:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T14:16:36.768 INFO:tasks.cephadm:Stopping all daemons... 2024-04-25T14:16:36.768 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-25T14:16:36.768 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-59334182-030c-11ef-bc93-c7b262605968@mon.a 2024-04-25T14:16:36.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:36 smithi096 ceph-mon[29207]: pgmap v436: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 10 op/s 2024-04-25T14:16:36.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:36 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/2308648534' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T14:16:36.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:36 smithi096 ceph-mon[29207]: osdmap e80: 12 total, 12 up, 12 in 2024-04-25T14:16:36.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:36 smithi096 ceph-mon[29207]: from='client.? 172.21.15.96:0/2989827481' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T14:16:36.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:36 smithi096 systemd[1]: Stopping Ceph mon.a for 59334182-030c-11ef-bc93-c7b262605968... 2024-04-25T14:16:36.922 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:36 smithi097 ceph-mon[32720]: pgmap v436: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 10 op/s 2024-04-25T14:16:36.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:36 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/2308648534' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T14:16:36.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:36 smithi097 ceph-mon[32720]: osdmap e80: 12 total, 12 up, 12 in 2024-04-25T14:16:36.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:36 smithi097 ceph-mon[32720]: from='client.? 172.21.15.96:0/2989827481' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T14:16:36.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:36 smithi151 ceph-mon[32990]: pgmap v436: 97 pgs: 97 active+clean; 8.7 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 10 op/s 2024-04-25T14:16:36.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:36 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/2308648534' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T14:16:36.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:36 smithi151 ceph-mon[32990]: osdmap e80: 12 total, 12 up, 12 in 2024-04-25T14:16:36.938 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:36 smithi151 ceph-mon[32990]: from='client.? 172.21.15.96:0/2989827481' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T14:16:37.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:37 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mon-a[29203]: 2024-04-25T14:16:37.070+0000 7f0df5eac700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:16:37.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:37 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mon-a[29203]: 2024-04-25T14:16:37.070+0000 7f0df5eac700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-25T14:16:37.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:37 smithi096 podman[88070]: 2024-04-25 14:16:37.245323681 +0000 UTC m=+0.291230467 container died 18d82172990ba4676e789e5d85a7d94bce2856d86216b79876a2f7939b23c04b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T14:16:37.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:37 smithi096 podman[88070]: 2024-04-25 14:16:37.764900878 +0000 UTC m=+0.810807662 container cleanup 18d82172990ba4676e789e5d85a7d94bce2856d86216b79876a2f7939b23c04b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-25T14:16:37.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 14:16:37 smithi096 bash[88070]: ceph-59334182-030c-11ef-bc93-c7b262605968-mon-a 2024-04-25T14:16:38.196 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@mon.a.service' 2024-04-25T14:16:38.243 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:16:38.243 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-25T14:16:38.243 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-25T14:16:38.244 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-59334182-030c-11ef-bc93-c7b262605968@mon.b 2024-04-25T14:16:38.545 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:38 smithi097 systemd[1]: Stopping Ceph mon.b for 59334182-030c-11ef-bc93-c7b262605968... 2024-04-25T14:16:38.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:38 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mon-b[32715]: 2024-04-25T14:16:38.544+0000 7f1a18c53700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:16:38.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:38 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mon-b[32715]: 2024-04-25T14:16:38.544+0000 7f1a18c53700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-25T14:16:38.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:38 smithi097 podman[66177]: 2024-04-25 14:16:38.689571439 +0000 UTC m=+0.253683810 container died affb5a16a9fcac0a7229a99273711c799f934e2c4e9ee9523c6a3cc25df5d153 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mon-b, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T14:16:39.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:39 smithi097 podman[66177]: 2024-04-25 14:16:39.297953755 +0000 UTC m=+0.862066126 container cleanup affb5a16a9fcac0a7229a99273711c799f934e2c4e9ee9523c6a3cc25df5d153 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mon-b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T14:16:39.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 14:16:39 smithi097 bash[66177]: ceph-59334182-030c-11ef-bc93-c7b262605968-mon-b 2024-04-25T14:16:39.678 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@mon.b.service' 2024-04-25T14:16:39.730 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:16:39.730 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-25T14:16:39.730 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-25T14:16:39.730 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-59334182-030c-11ef-bc93-c7b262605968@mon.c 2024-04-25T14:16:40.064 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:39 smithi151 systemd[1]: Stopping Ceph mon.c for 59334182-030c-11ef-bc93-c7b262605968... 2024-04-25T14:16:40.432 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:40 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mon-c[32975]: 2024-04-25T14:16:40.064+0000 7f46baafd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:16:40.432 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:40 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mon-c[32975]: 2024-04-25T14:16:40.064+0000 7f46baafd700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-25T14:16:40.432 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:40 smithi151 podman[66860]: 2024-04-25 14:16:40.137005599 +0000 UTC m=+0.216642518 container died 412fdde6b2cb4f22fdaf2512bea08a63805959d5ce8da1a7faf300847598c535 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mon-c, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True) 2024-04-25T14:16:40.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:40 smithi151 podman[66860]: 2024-04-25 14:16:40.524231299 +0000 UTC m=+0.603868207 container cleanup 412fdde6b2cb4f22fdaf2512bea08a63805959d5ce8da1a7faf300847598c535 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mon-c, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T14:16:40.688 INFO:journalctl@ceph.mon.c.smithi151.stdout:Apr 25 14:16:40 smithi151 bash[66860]: ceph-59334182-030c-11ef-bc93-c7b262605968-mon-c 2024-04-25T14:16:40.948 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@mon.c.service' 2024-04-25T14:16:41.000 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:16:41.000 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-25T14:16:41.001 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-25T14:16:41.001 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-59334182-030c-11ef-bc93-c7b262605968@mgr.x 2024-04-25T14:16:41.318 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:16:41 smithi096 systemd[1]: Stopping Ceph mgr.x for 59334182-030c-11ef-bc93-c7b262605968... 2024-04-25T14:16:41.609 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:16:41 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:16:41.318+0000 7fedf02c4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:16:41.609 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:16:41 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x[29442]: 2024-04-25T14:16:41.318+0000 7fedf02c4700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-25T14:16:41.609 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:16:41 smithi096 podman[88190]: 2024-04-25 14:16:41.350472364 +0000 UTC m=+0.153104723 container died de6d45d4d2364df8115ad27d496348110559a43126c24a05ac72a74f93e2793a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T14:16:42.105 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:16:41 smithi096 podman[88190]: 2024-04-25 14:16:41.879418674 +0000 UTC m=+0.682051079 container cleanup de6d45d4d2364df8115ad27d496348110559a43126c24a05ac72a74f93e2793a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-25T14:16:42.105 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:16:41 smithi096 bash[88190]: ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x 2024-04-25T14:16:42.339 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-25T14:16:42.359 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:16:42 smithi096 podman[88204]: 2024-04-25 14:16:42.106191282 +0000 UTC m=+0.753441550 container remove de6d45d4d2364df8115ad27d496348110559a43126c24a05ac72a74f93e2793a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-x, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T14:16:42.359 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:16:42 smithi096 systemd[1]: ceph-59334182-030c-11ef-bc93-c7b262605968@mgr.x.service: Deactivated successfully. 2024-04-25T14:16:42.359 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:16:42 smithi096 systemd[1]: Stopped Ceph mgr.x for 59334182-030c-11ef-bc93-c7b262605968. 2024-04-25T14:16:42.359 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 14:16:42 smithi096 systemd[1]: ceph-59334182-030c-11ef-bc93-c7b262605968@mgr.x.service: Consumed 58.467s CPU time. 2024-04-25T14:16:42.372 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:16:42.373 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-25T14:16:42.373 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-25T14:16:42.373 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-59334182-030c-11ef-bc93-c7b262605968@mgr.y 2024-04-25T14:16:42.672 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 14:16:42 smithi097 systemd[1]: Stopping Ceph mgr.y for 59334182-030c-11ef-bc93-c7b262605968... 2024-04-25T14:16:42.971 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 14:16:42 smithi097 podman[66301]: 2024-04-25 14:16:42.700024833 +0000 UTC m=+0.137246447 container died 38308e63f649ec1908770fae5052036b5a046f75404b34cafd5a11fb87b23253 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-y, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-25T14:16:43.341 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 14:16:43 smithi097 podman[66301]: 2024-04-25 14:16:43.071626669 +0000 UTC m=+0.508848284 container cleanup 38308e63f649ec1908770fae5052036b5a046f75404b34cafd5a11fb87b23253 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-25T14:16:43.341 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 14:16:43 smithi097 bash[66301]: ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-y 2024-04-25T14:16:43.444 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-25T14:16:43.479 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:16:43.479 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-25T14:16:43.480 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-25T14:16:43.480 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-59334182-030c-11ef-bc93-c7b262605968@mgr.z 2024-04-25T14:16:43.688 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Apr 25 14:16:43 smithi151 systemd[1]: Stopping Ceph mgr.z for 59334182-030c-11ef-bc93-c7b262605968... 2024-04-25T14:16:44.070 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Apr 25 14:16:43 smithi151 podman[66982]: 2024-04-25 14:16:43.808210968 +0000 UTC m=+0.137522965 container died e616504caf4aa5e4095e97a508a2adc42b3608fab4c219c9b49318395656ac60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T14:16:44.438 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Apr 25 14:16:44 smithi151 podman[66982]: 2024-04-25 14:16:44.179396873 +0000 UTC m=+0.508708860 container cleanup e616504caf4aa5e4095e97a508a2adc42b3608fab4c219c9b49318395656ac60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-25T14:16:44.438 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Apr 25 14:16:44 smithi151 bash[66982]: ceph-59334182-030c-11ef-bc93-c7b262605968-mgr-z 2024-04-25T14:16:44.594 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-25T14:16:44.627 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:16:44.627 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-25T14:16:44.628 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-25T14:16:44.628 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-59334182-030c-11ef-bc93-c7b262605968@osd.0 2024-04-25T14:16:44.974 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 14:16:44 smithi096 systemd[1]: Stopping Ceph osd.0 for 59334182-030c-11ef-bc93-c7b262605968... 2024-04-25T14:16:45.359 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 14:16:44 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-0[44132]: 2024-04-25T14:16:44.974+0000 7f9ea03ec700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:16:45.359 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 14:16:44 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-0[44132]: 2024-04-25T14:16:44.974+0000 7f9ea03ec700 -1 osd.0 81 *** Got signal Terminated *** 2024-04-25T14:16:45.359 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 14:16:44 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-0[44132]: 2024-04-25T14:16:44.974+0000 7f9ea03ec700 -1 osd.0 81 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:16:50.734 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 14:16:50 smithi096 podman[88312]: 2024-04-25 14:16:50.397859708 +0000 UTC m=+5.574330995 container died 14666124012b3f1ea518d8a86a1207b26471ac9c857ce0c57ce780677111636e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424) 2024-04-25T14:16:51.109 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 14:16:50 smithi096 podman[88312]: 2024-04-25 14:16:50.942577364 +0000 UTC m=+6.119048651 container cleanup 14666124012b3f1ea518d8a86a1207b26471ac9c857ce0c57ce780677111636e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-0, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=) 2024-04-25T14:16:51.109 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 14:16:50 smithi096 bash[88312]: ceph-59334182-030c-11ef-bc93-c7b262605968-osd-0 2024-04-25T14:16:51.609 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 14:16:51 smithi096 podman[88329]: 2024-04-25 14:16:51.185822495 +0000 UTC m=+0.781816718 container remove 14666124012b3f1ea518d8a86a1207b26471ac9c857ce0c57ce780677111636e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-0, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD) 2024-04-25T14:16:51.859 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 14:16:51 smithi096 podman[88423]: 2024-04-25 14:16:51.512513893 +0000 UTC m=+0.018057381 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:16:51.859 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 14:16:51 smithi096 podman[88423]: 2024-04-25 14:16:51.639281551 +0000 UTC m=+0.144825085 container create 5f9d65d59e8a74880c219f285417f523d7d51c4d3f5890f1be58ce74cd2ddf19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-0-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424) 2024-04-25T14:16:52.205 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 14:16:51 smithi096 podman[88423]: 2024-04-25 14:16:51.906009626 +0000 UTC m=+0.411553119 container init 5f9d65d59e8a74880c219f285417f523d7d51c4d3f5890f1be58ce74cd2ddf19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T14:16:52.205 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 14:16:51 smithi096 podman[88423]: 2024-04-25 14:16:51.909591136 +0000 UTC m=+0.415134627 container start 5f9d65d59e8a74880c219f285417f523d7d51c4d3f5890f1be58ce74cd2ddf19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2) 2024-04-25T14:16:52.205 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 14:16:52 smithi096 podman[88423]: 2024-04-25 14:16:52.006082525 +0000 UTC m=+0.511626040 container attach 5f9d65d59e8a74880c219f285417f523d7d51c4d3f5890f1be58ce74cd2ddf19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T14:16:52.550 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 14:16:52 smithi096 podman[88423]: 2024-04-25 14:16:52.206311145 +0000 UTC m=+0.711854628 container died 5f9d65d59e8a74880c219f285417f523d7d51c4d3f5890f1be58ce74cd2ddf19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS) 2024-04-25T14:16:52.988 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@osd.0.service' 2024-04-25T14:16:53.027 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:16:53.027 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-25T14:16:53.027 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-25T14:16:53.027 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-59334182-030c-11ef-bc93-c7b262605968@osd.1 2024-04-25T14:16:53.344 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 14:16:53 smithi097 systemd[1]: Stopping Ceph osd.1 for 59334182-030c-11ef-bc93-c7b262605968... 2024-04-25T14:16:53.673 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 14:16:53 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-1[37313]: 2024-04-25T14:16:53.344+0000 7f1f7f1fc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:16:53.673 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 14:16:53 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-1[37313]: 2024-04-25T14:16:53.344+0000 7f1f7f1fc700 -1 osd.1 81 *** Got signal Terminated *** 2024-04-25T14:16:53.673 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 14:16:53 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-1[37313]: 2024-04-25T14:16:53.344+0000 7f1f7f1fc700 -1 osd.1 81 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:16:58.923 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 14:16:58 smithi097 podman[66430]: 2024-04-25 14:16:58.666997364 +0000 UTC m=+5.437895644 container died 25f071ec389f30ef8d7a95897f5ecec41c576a2aff177162be451090532314fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-1, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-25T14:16:59.423 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 14:16:58 smithi097 podman[66430]: 2024-04-25 14:16:58.95344499 +0000 UTC m=+5.724343272 container cleanup 25f071ec389f30ef8d7a95897f5ecec41c576a2aff177162be451090532314fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-1, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True) 2024-04-25T14:16:59.423 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 14:16:58 smithi097 bash[66430]: ceph-59334182-030c-11ef-bc93-c7b262605968-osd-1 2024-04-25T14:16:59.423 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 14:16:59 smithi097 podman[66447]: 2024-04-25 14:16:59.19630757 +0000 UTC m=+0.528897723 container remove 25f071ec389f30ef8d7a95897f5ecec41c576a2aff177162be451090532314fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-1, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424) 2024-04-25T14:16:59.832 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 14:16:59 smithi097 podman[66541]: 2024-04-25 14:16:59.402065556 +0000 UTC m=+0.017922843 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:16:59.832 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 14:16:59 smithi097 podman[66541]: 2024-04-25 14:16:59.547511774 +0000 UTC m=+0.163369045 container create 1620dd13eddf8d5a1a7fc28f2639e1938b176cf775af6bbcde509b112a9f1fdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T14:16:59.833 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 14:16:59 smithi097 podman[66541]: 2024-04-25 14:16:59.833606349 +0000 UTC m=+0.449463620 container init 1620dd13eddf8d5a1a7fc28f2639e1938b176cf775af6bbcde509b112a9f1fdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-1-deactivate, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T14:17:00.123 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 14:16:59 smithi097 podman[66541]: 2024-04-25 14:16:59.837351397 +0000 UTC m=+0.453208668 container start 1620dd13eddf8d5a1a7fc28f2639e1938b176cf775af6bbcde509b112a9f1fdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-25T14:17:00.123 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 14:16:59 smithi097 podman[66541]: 2024-04-25 14:16:59.933643505 +0000 UTC m=+0.549500777 container attach 1620dd13eddf8d5a1a7fc28f2639e1938b176cf775af6bbcde509b112a9f1fdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-25T14:17:00.393 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 14:17:00 smithi097 podman[66541]: 2024-04-25 14:17:00.124251846 +0000 UTC m=+0.740109119 container died 1620dd13eddf8d5a1a7fc28f2639e1938b176cf775af6bbcde509b112a9f1fdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-25T14:17:00.838 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@osd.1.service' 2024-04-25T14:17:00.875 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:17:00.876 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-25T14:17:00.876 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-25T14:17:00.876 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-59334182-030c-11ef-bc93-c7b262605968@osd.2 2024-04-25T14:17:01.183 INFO:journalctl@ceph.osd.2.smithi151.stdout:Apr 25 14:17:00 smithi151 systemd[1]: Stopping Ceph osd.2 for 59334182-030c-11ef-bc93-c7b262605968... 2024-04-25T14:17:01.438 INFO:journalctl@ceph.osd.2.smithi151.stdout:Apr 25 14:17:01 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-2[38142]: 2024-04-25T14:17:01.183+0000 7f4cacdd9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:17:01.438 INFO:journalctl@ceph.osd.2.smithi151.stdout:Apr 25 14:17:01 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-2[38142]: 2024-04-25T14:17:01.183+0000 7f4cacdd9700 -1 osd.2 81 *** Got signal Terminated *** 2024-04-25T14:17:01.438 INFO:journalctl@ceph.osd.2.smithi151.stdout:Apr 25 14:17:01 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-2[38142]: 2024-04-25T14:17:01.183+0000 7f4cacdd9700 -1 osd.2 81 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:17:06.844 INFO:journalctl@ceph.osd.2.smithi151.stdout:Apr 25 14:17:06 smithi151 podman[67105]: 2024-04-25 14:17:06.548266156 +0000 UTC m=+5.482542290 container died 442f7d1ce4dbcb331ca929a49d4687f5416e1fe56d7e348c45c1f066b5bb901e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5) 2024-04-25T14:17:07.188 INFO:journalctl@ceph.osd.2.smithi151.stdout:Apr 25 14:17:06 smithi151 podman[67105]: 2024-04-25 14:17:06.975151622 +0000 UTC m=+5.909427759 container cleanup 442f7d1ce4dbcb331ca929a49d4687f5416e1fe56d7e348c45c1f066b5bb901e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T14:17:07.188 INFO:journalctl@ceph.osd.2.smithi151.stdout:Apr 25 14:17:06 smithi151 bash[67105]: ceph-59334182-030c-11ef-bc93-c7b262605968-osd-2 2024-04-25T14:17:07.596 INFO:journalctl@ceph.osd.2.smithi151.stdout:Apr 25 14:17:07 smithi151 podman[67124]: 2024-04-25 14:17:07.237323269 +0000 UTC m=+0.691912626 container remove 442f7d1ce4dbcb331ca929a49d4687f5416e1fe56d7e348c45c1f066b5bb901e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:17:07.907 INFO:journalctl@ceph.osd.2.smithi151.stdout:Apr 25 14:17:07 smithi151 podman[67216]: 2024-04-25 14:17:07.500557761 +0000 UTC m=+0.018366363 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:17:07.908 INFO:journalctl@ceph.osd.2.smithi151.stdout:Apr 25 14:17:07 smithi151 podman[67216]: 2024-04-25 14:17:07.63031486 +0000 UTC m=+0.148123461 container create 41c92fcb375d4c86a78298df8225fa2ebfeec8e6ca0301e2033a1bbd1b8c27a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-25T14:17:07.908 INFO:journalctl@ceph.osd.2.smithi151.stdout:Apr 25 14:17:07 smithi151 podman[67216]: 2024-04-25 14:17:07.908888453 +0000 UTC m=+0.426697055 container init 41c92fcb375d4c86a78298df8225fa2ebfeec8e6ca0301e2033a1bbd1b8c27a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424) 2024-04-25T14:17:08.188 INFO:journalctl@ceph.osd.2.smithi151.stdout:Apr 25 14:17:07 smithi151 podman[67216]: 2024-04-25 14:17:07.912158988 +0000 UTC m=+0.429967591 container start 41c92fcb375d4c86a78298df8225fa2ebfeec8e6ca0301e2033a1bbd1b8c27a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T14:17:08.188 INFO:journalctl@ceph.osd.2.smithi151.stdout:Apr 25 14:17:08 smithi151 podman[67216]: 2024-04-25 14:17:08.009009778 +0000 UTC m=+0.526818381 container attach 41c92fcb375d4c86a78298df8225fa2ebfeec8e6ca0301e2033a1bbd1b8c27a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0) 2024-04-25T14:17:08.488 INFO:journalctl@ceph.osd.2.smithi151.stdout:Apr 25 14:17:08 smithi151 podman[67216]: 2024-04-25 14:17:08.21189884 +0000 UTC m=+0.729707444 container died 41c92fcb375d4c86a78298df8225fa2ebfeec8e6ca0301e2033a1bbd1b8c27a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-25T14:17:08.837 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@osd.2.service' 2024-04-25T14:17:08.875 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:17:08.876 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-25T14:17:08.876 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-25T14:17:08.876 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-59334182-030c-11ef-bc93-c7b262605968@osd.3 2024-04-25T14:17:09.220 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 14:17:08 smithi096 systemd[1]: Stopping Ceph osd.3 for 59334182-030c-11ef-bc93-c7b262605968... 2024-04-25T14:17:09.481 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 14:17:09 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-3[50623]: 2024-04-25T14:17:09.219+0000 7fcd5b122700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:17:09.481 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 14:17:09 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-3[50623]: 2024-04-25T14:17:09.219+0000 7fcd5b122700 -1 osd.3 81 *** Got signal Terminated *** 2024-04-25T14:17:09.481 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 14:17:09 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-3[50623]: 2024-04-25T14:17:09.219+0000 7fcd5b122700 -1 osd.3 81 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:17:11.188 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:10 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:10.937+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:11.859 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:11 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:11.454+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:12.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:11 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:11.969+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:12.859 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:12 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:12.477+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:12.974 INFO:journalctl@ceph.osd.5.smithi151.stdout:Apr 25 14:17:12 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-5[44759]: 2024-04-25T14:17:12.696+0000 7fb233d33700 -1 osd.5 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.722828+0000 front 2024-04-25T14:16:49.722628+0000 (oldest deadline 2024-04-25T14:17:12.022695+0000) 2024-04-25T14:17:13.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:12 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:12.974+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:13.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:13 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:13.406+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:17:13.859 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:13 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:13.515+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:13.859 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:13 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:13.717+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.140614+0000 front 2024-04-25T14:16:47.140643+0000 (oldest deadline 2024-04-25T14:17:13.040930+0000) 2024-04-25T14:17:13.938 INFO:journalctl@ceph.osd.5.smithi151.stdout:Apr 25 14:17:13 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-5[44759]: 2024-04-25T14:17:13.682+0000 7fb233d33700 -1 osd.5 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.722828+0000 front 2024-04-25T14:16:49.722628+0000 (oldest deadline 2024-04-25T14:17:12.022695+0000) 2024-04-25T14:17:14.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:13 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:13.942+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:14.672 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:14 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:14.401+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:17:14.859 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:14 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:14.737+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.140614+0000 front 2024-04-25T14:16:47.140643+0000 (oldest deadline 2024-04-25T14:17:13.040930+0000) 2024-04-25T14:17:14.859 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:14 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:14.559+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:14.859 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 14:17:14 smithi096 podman[88532]: 2024-04-25 14:17:14.672640799 +0000 UTC m=+5.603758591 container died 8f7622abc3a1624c7c83f0257a8052ec04de8bfae3557d2b5dda76ced8668dd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-3, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-25T14:17:14.910 INFO:journalctl@ceph.osd.5.smithi151.stdout:Apr 25 14:17:14 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-5[44759]: 2024-04-25T14:17:14.647+0000 7fb233d33700 -1 osd.5 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.722828+0000 front 2024-04-25T14:16:49.722628+0000 (oldest deadline 2024-04-25T14:17:12.022695+0000) 2024-04-25T14:17:15.188 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:14 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:14.911+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:15.359 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 14:17:15 smithi096 podman[88532]: 2024-04-25 14:17:15.186764342 +0000 UTC m=+6.117882138 container cleanup 8f7622abc3a1624c7c83f0257a8052ec04de8bfae3557d2b5dda76ced8668dd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:17:15.359 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 14:17:15 smithi096 bash[88532]: ceph-59334182-030c-11ef-bc93-c7b262605968-osd-3 2024-04-25T14:17:15.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:15 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:15.370+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:17:15.768 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:15 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:15.542+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:15.769 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 14:17:15 smithi096 podman[88562]: 2024-04-25 14:17:15.430038231 +0000 UTC m=+0.759363791 container remove 8f7622abc3a1624c7c83f0257a8052ec04de8bfae3557d2b5dda76ced8668dd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-3, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T14:17:15.909 INFO:journalctl@ceph.osd.5.smithi151.stdout:Apr 25 14:17:15 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-5[44759]: 2024-04-25T14:17:15.639+0000 7fb233d33700 -1 osd.5 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.722828+0000 front 2024-04-25T14:16:49.722628+0000 (oldest deadline 2024-04-25T14:17:12.022695+0000) 2024-04-25T14:17:16.032 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:15 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:15.768+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.140614+0000 front 2024-04-25T14:16:47.140643+0000 (oldest deadline 2024-04-25T14:17:13.040930+0000) 2024-04-25T14:17:16.032 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 14:17:15 smithi096 podman[88653]: 2024-04-25 14:17:15.758140484 +0000 UTC m=+0.018336006 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:17:16.032 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 14:17:15 smithi096 podman[88653]: 2024-04-25 14:17:15.897720661 +0000 UTC m=+0.157916180 container create 17432251fc01e24b57b42271a7f1d3553de567869c2607f127826b63d09e4254 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-3-deactivate, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-25T14:17:16.188 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:15 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:15.909+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:16.359 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 14:17:16 smithi096 podman[88653]: 2024-04-25 14:17:16.142349597 +0000 UTC m=+0.402545120 container init 17432251fc01e24b57b42271a7f1d3553de567869c2607f127826b63d09e4254 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240424, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T14:17:16.359 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 14:17:16 smithi096 podman[88653]: 2024-04-25 14:17:16.146003779 +0000 UTC m=+0.406199301 container start 17432251fc01e24b57b42271a7f1d3553de567869c2607f127826b63d09e4254 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T14:17:16.359 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 14:17:16 smithi096 podman[88653]: 2024-04-25 14:17:16.200213196 +0000 UTC m=+0.460408718 container attach 17432251fc01e24b57b42271a7f1d3553de567869c2607f127826b63d09e4254 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T14:17:16.672 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:16 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:16.360+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:17:16.720 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 14:17:16 smithi096 podman[88653]: 2024-04-25 14:17:16.437312403 +0000 UTC m=+0.697507923 container died 17432251fc01e24b57b42271a7f1d3553de567869c2607f127826b63d09e4254 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-3-deactivate, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T14:17:16.720 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:16 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:16.545+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:16.938 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:16 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:16.872+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:16.938 INFO:journalctl@ceph.osd.5.smithi151.stdout:Apr 25 14:17:16 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-5[44759]: 2024-04-25T14:17:16.654+0000 7fb233d33700 -1 osd.5 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.722828+0000 front 2024-04-25T14:16:49.722628+0000 (oldest deadline 2024-04-25T14:17:12.022695+0000) 2024-04-25T14:17:17.078 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:16 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:16.720+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.140614+0000 front 2024-04-25T14:16:47.140643+0000 (oldest deadline 2024-04-25T14:17:13.040930+0000) 2024-04-25T14:17:17.106 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@osd.3.service' 2024-04-25T14:17:17.144 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:17:17.144 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-25T14:17:17.144 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-25T14:17:17.145 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-59334182-030c-11ef-bc93-c7b262605968@osd.4 2024-04-25T14:17:17.422 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 14:17:17 smithi097 systemd[1]: Stopping Ceph osd.4 for 59334182-030c-11ef-bc93-c7b262605968... 2024-04-25T14:17:17.423 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:17 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:17.391+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:17:17.859 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:17 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:17.770+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.140614+0000 front 2024-04-25T14:16:47.140643+0000 (oldest deadline 2024-04-25T14:17:13.040930+0000) 2024-04-25T14:17:17.859 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:17 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:17.589+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:17.922 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 14:17:17 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-4[44233]: 2024-04-25T14:17:17.449+0000 7fb9a135b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:17:17.923 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 14:17:17 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-4[44233]: 2024-04-25T14:17:17.449+0000 7fb9a135b700 -1 osd.4 81 *** Got signal Terminated *** 2024-04-25T14:17:17.923 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 14:17:17 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-4[44233]: 2024-04-25T14:17:17.449+0000 7fb9a135b700 -1 osd.4 81 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:17:17.937 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:17 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:17.856+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:17.938 INFO:journalctl@ceph.osd.5.smithi151.stdout:Apr 25 14:17:17 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-5[44759]: 2024-04-25T14:17:17.673+0000 7fb233d33700 -1 osd.5 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.722828+0000 front 2024-04-25T14:16:49.722628+0000 (oldest deadline 2024-04-25T14:17:12.022695+0000) 2024-04-25T14:17:18.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:18 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:18.344+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:17:18.859 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:18 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:18.751+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.140614+0000 front 2024-04-25T14:16:47.140643+0000 (oldest deadline 2024-04-25T14:17:13.040930+0000) 2024-04-25T14:17:18.859 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:18 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:18.544+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:18.936 INFO:journalctl@ceph.osd.5.smithi151.stdout:Apr 25 14:17:18 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-5[44759]: 2024-04-25T14:17:18.680+0000 7fb233d33700 -1 osd.5 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.722828+0000 front 2024-04-25T14:16:49.722628+0000 (oldest deadline 2024-04-25T14:17:12.022695+0000) 2024-04-25T14:17:18.937 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:18 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:18.829+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:19.381 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:18 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:18.998+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.237196+0000 front 2024-04-25T14:16:55.237235+0000 (oldest deadline 2024-04-25T14:17:18.737337+0000) 2024-04-25T14:17:19.672 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:19 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:19.382+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:17:19.827 INFO:journalctl@ceph.osd.5.smithi151.stdout:Apr 25 14:17:19 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-5[44759]: 2024-04-25T14:17:19.695+0000 7fb233d33700 -1 osd.5 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.722828+0000 front 2024-04-25T14:16:49.722628+0000 (oldest deadline 2024-04-25T14:17:12.022695+0000) 2024-04-25T14:17:19.859 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:19 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:19.765+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.140614+0000 front 2024-04-25T14:16:47.140643+0000 (oldest deadline 2024-04-25T14:17:13.040930+0000) 2024-04-25T14:17:19.859 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:19 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:19.765+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.242964+0000 front 2024-04-25T14:16:58.242994+0000 (oldest deadline 2024-04-25T14:17:19.343170+0000) 2024-04-25T14:17:19.859 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:19 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:19.529+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:20.188 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:19 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:19.827+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:20.188 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:19 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:19.827+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:17:20.367 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:20 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:20.037+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.237196+0000 front 2024-04-25T14:16:55.237235+0000 (oldest deadline 2024-04-25T14:17:18.737337+0000) 2024-04-25T14:17:20.672 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:20 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:20.368+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:17:20.807 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:20 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:20.500+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:21.109 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:20 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:20.807+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.140614+0000 front 2024-04-25T14:16:47.140643+0000 (oldest deadline 2024-04-25T14:17:13.040930+0000) 2024-04-25T14:17:21.109 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:20 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:20.807+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.242964+0000 front 2024-04-25T14:16:58.242994+0000 (oldest deadline 2024-04-25T14:17:19.343170+0000) 2024-04-25T14:17:21.188 INFO:journalctl@ceph.osd.5.smithi151.stdout:Apr 25 14:17:20 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-5[44759]: 2024-04-25T14:17:20.711+0000 7fb233d33700 -1 osd.5 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.722828+0000 front 2024-04-25T14:16:49.722628+0000 (oldest deadline 2024-04-25T14:17:12.022695+0000) 2024-04-25T14:17:21.188 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:20 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:20.859+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:21.188 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:20 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:20.859+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:17:21.414 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:21 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:21.033+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.237196+0000 front 2024-04-25T14:16:55.237235+0000 (oldest deadline 2024-04-25T14:17:18.737337+0000) 2024-04-25T14:17:21.672 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:21 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:21.415+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:17:21.848 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:21 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:21.527+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:22.099 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:21 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:21.848+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:22.100 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:21 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:21.848+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:17:22.100 INFO:journalctl@ceph.osd.5.smithi151.stdout:Apr 25 14:17:21 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-5[44759]: 2024-04-25T14:17:21.697+0000 7fb233d33700 -1 osd.5 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.722828+0000 front 2024-04-25T14:16:49.722628+0000 (oldest deadline 2024-04-25T14:17:12.022695+0000) 2024-04-25T14:17:22.100 INFO:journalctl@ceph.osd.5.smithi151.stdout:Apr 25 14:17:21 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-5[44759]: 2024-04-25T14:17:21.697+0000 7fb233d33700 -1 osd.5 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.923838+0000 front 2024-04-25T14:16:57.923855+0000 (oldest deadline 2024-04-25T14:17:20.823710+0000) 2024-04-25T14:17:22.109 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:21 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:21.849+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.140614+0000 front 2024-04-25T14:16:47.140643+0000 (oldest deadline 2024-04-25T14:17:13.040930+0000) 2024-04-25T14:17:22.109 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:21 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:21.849+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.242964+0000 front 2024-04-25T14:16:58.242994+0000 (oldest deadline 2024-04-25T14:17:19.343170+0000) 2024-04-25T14:17:22.422 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:22 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:22.075+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.237196+0000 front 2024-04-25T14:16:55.237235+0000 (oldest deadline 2024-04-25T14:17:18.737337+0000) 2024-04-25T14:17:22.438 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:22 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:22.100+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:53.756445+0000 front 2024-04-25T14:16:53.756264+0000 (oldest deadline 2024-04-25T14:17:21.956954+0000) 2024-04-25T14:17:22.787 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:22 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:22.435+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:17:22.859 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:22 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:22.572+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:23.049 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 14:17:22 smithi097 podman[66661]: 2024-04-25 14:17:22.788423673 +0000 UTC m=+5.451090699 container died 7ee9cc2432a608bc891e4502f14667fcfc5b032794dfbd4e48210408c9b91584 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-4, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-25T14:17:23.122 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:22 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:22.800+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:23.122 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:22 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:22.800+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:17:23.122 INFO:journalctl@ceph.osd.5.smithi151.stdout:Apr 25 14:17:22 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-5[44759]: 2024-04-25T14:17:22.694+0000 7fb233d33700 -1 osd.5 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.722828+0000 front 2024-04-25T14:16:49.722628+0000 (oldest deadline 2024-04-25T14:17:12.022695+0000) 2024-04-25T14:17:23.123 INFO:journalctl@ceph.osd.5.smithi151.stdout:Apr 25 14:17:22 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-5[44759]: 2024-04-25T14:17:22.694+0000 7fb233d33700 -1 osd.5 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.923838+0000 front 2024-04-25T14:16:57.923855+0000 (oldest deadline 2024-04-25T14:17:20.823710+0000) 2024-04-25T14:17:23.357 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:23 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:23.122+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.237196+0000 front 2024-04-25T14:16:55.237235+0000 (oldest deadline 2024-04-25T14:17:18.737337+0000) 2024-04-25T14:17:23.358 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 14:17:23 smithi097 podman[66661]: 2024-04-25 14:17:23.132520083 +0000 UTC m=+5.795187110 container cleanup 7ee9cc2432a608bc891e4502f14667fcfc5b032794dfbd4e48210408c9b91584 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-25T14:17:23.358 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 14:17:23 smithi097 bash[66661]: ceph-59334182-030c-11ef-bc93-c7b262605968-osd-4 2024-04-25T14:17:23.359 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:22 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:22.868+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.140614+0000 front 2024-04-25T14:16:47.140643+0000 (oldest deadline 2024-04-25T14:17:13.040930+0000) 2024-04-25T14:17:23.359 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:22 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:22.868+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.242964+0000 front 2024-04-25T14:16:58.242994+0000 (oldest deadline 2024-04-25T14:17:19.343170+0000) 2024-04-25T14:17:23.437 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:23 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:23.123+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:53.756445+0000 front 2024-04-25T14:16:53.756264+0000 (oldest deadline 2024-04-25T14:17:21.956954+0000) 2024-04-25T14:17:23.672 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 14:17:23 smithi097 podman[66678]: 2024-04-25 14:17:23.35883476 +0000 UTC m=+0.569647030 container remove 7ee9cc2432a608bc891e4502f14667fcfc5b032794dfbd4e48210408c9b91584 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-4, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:17:23.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:23 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:23.397+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:17:23.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:23 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:23.397+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:17:23.847 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:23 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:23.589+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:23.847 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:23 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:23.589+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.795847+0000 front 2024-04-25T14:16:57.795573+0000 (oldest deadline 2024-04-25T14:17:23.095608+0000) 2024-04-25T14:17:24.078 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 14:17:23 smithi097 podman[66773]: 2024-04-25 14:17:23.708510634 +0000 UTC m=+0.018220103 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:17:24.078 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 14:17:23 smithi097 podman[66773]: 2024-04-25 14:17:23.817934714 +0000 UTC m=+0.127644161 container create 26f545a569b9d8a6697e31f6c21ab0fdbb89688c6d1201d827f66f20b55892fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-25T14:17:24.089 INFO:journalctl@ceph.osd.5.smithi151.stdout:Apr 25 14:17:23 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-5[44759]: 2024-04-25T14:17:23.723+0000 7fb233d33700 -1 osd.5 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.722828+0000 front 2024-04-25T14:16:49.722628+0000 (oldest deadline 2024-04-25T14:17:12.022695+0000) 2024-04-25T14:17:24.089 INFO:journalctl@ceph.osd.5.smithi151.stdout:Apr 25 14:17:23 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-5[44759]: 2024-04-25T14:17:23.723+0000 7fb233d33700 -1 osd.5 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.923838+0000 front 2024-04-25T14:16:57.923855+0000 (oldest deadline 2024-04-25T14:17:20.823710+0000) 2024-04-25T14:17:24.090 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:23 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:23.773+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:24.090 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:23 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:23.773+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:17:24.109 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:23 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:23.847+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.140614+0000 front 2024-04-25T14:16:47.140643+0000 (oldest deadline 2024-04-25T14:17:13.040930+0000) 2024-04-25T14:17:24.109 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:23 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:23.847+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.242964+0000 front 2024-04-25T14:16:58.242994+0000 (oldest deadline 2024-04-25T14:17:19.343170+0000) 2024-04-25T14:17:24.384 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 14:17:24 smithi097 podman[66773]: 2024-04-25 14:17:24.079051075 +0000 UTC m=+0.388760530 container init 26f545a569b9d8a6697e31f6c21ab0fdbb89688c6d1201d827f66f20b55892fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-25T14:17:24.384 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 14:17:24 smithi097 podman[66773]: 2024-04-25 14:17:24.08255379 +0000 UTC m=+0.392263237 container start 26f545a569b9d8a6697e31f6c21ab0fdbb89688c6d1201d827f66f20b55892fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-25T14:17:24.384 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 14:17:24 smithi097 podman[66773]: 2024-04-25 14:17:24.179077782 +0000 UTC m=+0.488787264 container attach 26f545a569b9d8a6697e31f6c21ab0fdbb89688c6d1201d827f66f20b55892fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-4-deactivate, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-25T14:17:24.384 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:24 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:24.088+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.237196+0000 front 2024-04-25T14:16:55.237235+0000 (oldest deadline 2024-04-25T14:17:18.737337+0000) 2024-04-25T14:17:24.438 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:24 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:24.089+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:53.756445+0000 front 2024-04-25T14:16:53.756264+0000 (oldest deadline 2024-04-25T14:17:21.956954+0000) 2024-04-25T14:17:24.664 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 14:17:24 smithi097 podman[66773]: 2024-04-25 14:17:24.385002026 +0000 UTC m=+0.694711477 container died 26f545a569b9d8a6697e31f6c21ab0fdbb89688c6d1201d827f66f20b55892fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T14:17:24.665 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:24 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:24.415+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:17:24.665 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:24 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:24.415+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:17:24.987 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@osd.4.service' 2024-04-25T14:17:25.027 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:17:25.027 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-25T14:17:25.027 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-25T14:17:25.027 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-59334182-030c-11ef-bc93-c7b262605968@osd.5 2024-04-25T14:17:25.054 INFO:journalctl@ceph.osd.5.smithi151.stdout:Apr 25 14:17:24 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-5[44759]: 2024-04-25T14:17:24.735+0000 7fb233d33700 -1 osd.5 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.722828+0000 front 2024-04-25T14:16:49.722628+0000 (oldest deadline 2024-04-25T14:17:12.022695+0000) 2024-04-25T14:17:25.054 INFO:journalctl@ceph.osd.5.smithi151.stdout:Apr 25 14:17:24 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-5[44759]: 2024-04-25T14:17:24.735+0000 7fb233d33700 -1 osd.5 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.923838+0000 front 2024-04-25T14:16:57.923855+0000 (oldest deadline 2024-04-25T14:17:20.823710+0000) 2024-04-25T14:17:25.054 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:24 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:24.745+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:25.055 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:24 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:24.745+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:17:25.109 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:24 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:24.864+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.140614+0000 front 2024-04-25T14:16:47.140643+0000 (oldest deadline 2024-04-25T14:17:13.040930+0000) 2024-04-25T14:17:25.109 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:24 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:24.864+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.242964+0000 front 2024-04-25T14:16:58.242994+0000 (oldest deadline 2024-04-25T14:17:19.343170+0000) 2024-04-25T14:17:25.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:24 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:24.631+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:25.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:24 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:24.631+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.795847+0000 front 2024-04-25T14:16:57.795573+0000 (oldest deadline 2024-04-25T14:17:23.095608+0000) 2024-04-25T14:17:25.342 INFO:journalctl@ceph.osd.5.smithi151.stdout:Apr 25 14:17:25 smithi151 systemd[1]: Stopping Ceph osd.5 for 59334182-030c-11ef-bc93-c7b262605968... 2024-04-25T14:17:25.343 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:25 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:25.100+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:53.756445+0000 front 2024-04-25T14:16:53.756264+0000 (oldest deadline 2024-04-25T14:17:21.956954+0000) 2024-04-25T14:17:25.373 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:25 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:25.073+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.237196+0000 front 2024-04-25T14:16:55.237235+0000 (oldest deadline 2024-04-25T14:17:18.737337+0000) 2024-04-25T14:17:25.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:25 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:25.373+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:17:25.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:25 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:25.373+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:17:25.688 INFO:journalctl@ceph.osd.5.smithi151.stdout:Apr 25 14:17:25 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-5[44759]: 2024-04-25T14:17:25.342+0000 7fb23777d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:17:25.688 INFO:journalctl@ceph.osd.5.smithi151.stdout:Apr 25 14:17:25 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-5[44759]: 2024-04-25T14:17:25.342+0000 7fb23777d700 -1 osd.5 81 *** Got signal Terminated *** 2024-04-25T14:17:25.688 INFO:journalctl@ceph.osd.5.smithi151.stdout:Apr 25 14:17:25 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-5[44759]: 2024-04-25T14:17:25.342+0000 7fb23777d700 -1 osd.5 81 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:17:25.861 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:25 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:25.587+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:25.861 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:25 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:25.587+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.795847+0000 front 2024-04-25T14:16:57.795573+0000 (oldest deadline 2024-04-25T14:17:23.095608+0000) 2024-04-25T14:17:26.060 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:25 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:25.793+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:26.060 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:25 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:25.793+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:17:26.060 INFO:journalctl@ceph.osd.5.smithi151.stdout:Apr 25 14:17:25 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-5[44759]: 2024-04-25T14:17:25.733+0000 7fb233d33700 -1 osd.5 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.722828+0000 front 2024-04-25T14:16:49.722628+0000 (oldest deadline 2024-04-25T14:17:12.022695+0000) 2024-04-25T14:17:26.060 INFO:journalctl@ceph.osd.5.smithi151.stdout:Apr 25 14:17:25 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-5[44759]: 2024-04-25T14:17:25.733+0000 7fb233d33700 -1 osd.5 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.923838+0000 front 2024-04-25T14:16:57.923855+0000 (oldest deadline 2024-04-25T14:17:20.823710+0000) 2024-04-25T14:17:26.125 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:25 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:25.874+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.140614+0000 front 2024-04-25T14:16:47.140643+0000 (oldest deadline 2024-04-25T14:17:13.040930+0000) 2024-04-25T14:17:26.126 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:25 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:25.874+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.242964+0000 front 2024-04-25T14:16:58.242994+0000 (oldest deadline 2024-04-25T14:17:19.343170+0000) 2024-04-25T14:17:26.372 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:26 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:26.081+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.237196+0000 front 2024-04-25T14:16:55.237235+0000 (oldest deadline 2024-04-25T14:17:18.737337+0000) 2024-04-25T14:17:26.437 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:26 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:26.060+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:53.756445+0000 front 2024-04-25T14:16:53.756264+0000 (oldest deadline 2024-04-25T14:17:21.956954+0000) 2024-04-25T14:17:26.672 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:26 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:26.373+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:17:26.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:26 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:26.373+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:17:26.859 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:26 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:26.539+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:26.859 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:26 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:26.539+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.795847+0000 front 2024-04-25T14:16:57.795573+0000 (oldest deadline 2024-04-25T14:17:23.095608+0000) 2024-04-25T14:17:26.938 INFO:journalctl@ceph.osd.5.smithi151.stdout:Apr 25 14:17:26 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-5[44759]: 2024-04-25T14:17:26.686+0000 7fb233d33700 -1 osd.5 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.722828+0000 front 2024-04-25T14:16:49.722628+0000 (oldest deadline 2024-04-25T14:17:12.022695+0000) 2024-04-25T14:17:26.938 INFO:journalctl@ceph.osd.5.smithi151.stdout:Apr 25 14:17:26 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-5[44759]: 2024-04-25T14:17:26.686+0000 7fb233d33700 -1 osd.5 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.923838+0000 front 2024-04-25T14:16:57.923855+0000 (oldest deadline 2024-04-25T14:17:20.823710+0000) 2024-04-25T14:17:26.938 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:26 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:26.821+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:26.938 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:26 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:26.821+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:17:27.337 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:27 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:27.070+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.237196+0000 front 2024-04-25T14:16:55.237235+0000 (oldest deadline 2024-04-25T14:17:18.737337+0000) 2024-04-25T14:17:27.359 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:26 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:26.920+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.140614+0000 front 2024-04-25T14:16:47.140643+0000 (oldest deadline 2024-04-25T14:17:13.040930+0000) 2024-04-25T14:17:27.359 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:26 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:26.920+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.242964+0000 front 2024-04-25T14:16:58.242994+0000 (oldest deadline 2024-04-25T14:17:19.343170+0000) 2024-04-25T14:17:27.438 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:27 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:27.062+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:53.756445+0000 front 2024-04-25T14:16:53.756264+0000 (oldest deadline 2024-04-25T14:17:21.956954+0000) 2024-04-25T14:17:27.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:27 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:27.339+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:17:27.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:27 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:27.339+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:17:27.859 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:27 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:27.564+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:27.859 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:27 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:27.564+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.795847+0000 front 2024-04-25T14:16:57.795573+0000 (oldest deadline 2024-04-25T14:17:23.095608+0000) 2024-04-25T14:17:27.938 INFO:journalctl@ceph.osd.5.smithi151.stdout:Apr 25 14:17:27 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-5[44759]: 2024-04-25T14:17:27.646+0000 7fb233d33700 -1 osd.5 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.722828+0000 front 2024-04-25T14:16:49.722628+0000 (oldest deadline 2024-04-25T14:17:12.022695+0000) 2024-04-25T14:17:27.938 INFO:journalctl@ceph.osd.5.smithi151.stdout:Apr 25 14:17:27 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-5[44759]: 2024-04-25T14:17:27.646+0000 7fb233d33700 -1 osd.5 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.923838+0000 front 2024-04-25T14:16:57.923855+0000 (oldest deadline 2024-04-25T14:17:20.823710+0000) 2024-04-25T14:17:27.938 INFO:journalctl@ceph.osd.5.smithi151.stdout:Apr 25 14:17:27 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-5[44759]: 2024-04-25T14:17:27.646+0000 7fb233d33700 -1 osd.5 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:00.824220+0000 front 2024-04-25T14:17:00.824106+0000 (oldest deadline 2024-04-25T14:17:26.724317+0000) 2024-04-25T14:17:27.938 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:27 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:27.859+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:27.938 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:27 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:27.859+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:17:28.359 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:27 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:27.906+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.140614+0000 front 2024-04-25T14:16:47.140643+0000 (oldest deadline 2024-04-25T14:17:13.040930+0000) 2024-04-25T14:17:28.359 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:27 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:27.906+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.242964+0000 front 2024-04-25T14:16:58.242994+0000 (oldest deadline 2024-04-25T14:17:19.343170+0000) 2024-04-25T14:17:28.423 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:28 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:28.112+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.237196+0000 front 2024-04-25T14:16:55.237235+0000 (oldest deadline 2024-04-25T14:17:18.737337+0000) 2024-04-25T14:17:28.423 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:28 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:28.305+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:17:28.423 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:28 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:28.305+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:17:28.423 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:28 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:28.305+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.789654+0000 front 2024-04-25T14:17:02.789638+0000 (oldest deadline 2024-04-25T14:17:28.089414+0000) 2024-04-25T14:17:28.438 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:28 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:28.016+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:53.756445+0000 front 2024-04-25T14:16:53.756264+0000 (oldest deadline 2024-04-25T14:17:21.956954+0000) 2024-04-25T14:17:28.859 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:28 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:28.586+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:28.859 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:28 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:28.586+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.795847+0000 front 2024-04-25T14:16:57.795573+0000 (oldest deadline 2024-04-25T14:17:23.095608+0000) 2024-04-25T14:17:28.859 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:28 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:28.586+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:03.096238+0000 front 2024-04-25T14:17:03.096401+0000 (oldest deadline 2024-04-25T14:17:27.796160+0000) 2024-04-25T14:17:28.938 INFO:journalctl@ceph.osd.5.smithi151.stdout:Apr 25 14:17:28 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-5[44759]: 2024-04-25T14:17:28.646+0000 7fb233d33700 -1 osd.5 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.722828+0000 front 2024-04-25T14:16:49.722628+0000 (oldest deadline 2024-04-25T14:17:12.022695+0000) 2024-04-25T14:17:28.938 INFO:journalctl@ceph.osd.5.smithi151.stdout:Apr 25 14:17:28 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-5[44759]: 2024-04-25T14:17:28.646+0000 7fb233d33700 -1 osd.5 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.923838+0000 front 2024-04-25T14:16:57.923855+0000 (oldest deadline 2024-04-25T14:17:20.823710+0000) 2024-04-25T14:17:28.938 INFO:journalctl@ceph.osd.5.smithi151.stdout:Apr 25 14:17:28 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-5[44759]: 2024-04-25T14:17:28.646+0000 7fb233d33700 -1 osd.5 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:00.824220+0000 front 2024-04-25T14:17:00.824106+0000 (oldest deadline 2024-04-25T14:17:26.724317+0000) 2024-04-25T14:17:28.938 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:28 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:28.875+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:28.938 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:28 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:28.876+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:17:28.938 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:28 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:28.876+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:17:29.359 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:28 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:28.861+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.140614+0000 front 2024-04-25T14:16:47.140643+0000 (oldest deadline 2024-04-25T14:17:13.040930+0000) 2024-04-25T14:17:29.359 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:28 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:28.861+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.242964+0000 front 2024-04-25T14:16:58.242994+0000 (oldest deadline 2024-04-25T14:17:19.343170+0000) 2024-04-25T14:17:29.423 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:29 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:29.104+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.237196+0000 front 2024-04-25T14:16:55.237235+0000 (oldest deadline 2024-04-25T14:17:18.737337+0000) 2024-04-25T14:17:29.423 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:29 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:29.263+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:17:29.423 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:29 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:29.263+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:17:29.423 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:29 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:29.263+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.789654+0000 front 2024-04-25T14:17:02.789638+0000 (oldest deadline 2024-04-25T14:17:28.089414+0000) 2024-04-25T14:17:29.437 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:28 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:28.996+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:53.756445+0000 front 2024-04-25T14:16:53.756264+0000 (oldest deadline 2024-04-25T14:17:21.956954+0000) 2024-04-25T14:17:29.851 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:29 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:29.591+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:29.851 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:29 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:29.591+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.795847+0000 front 2024-04-25T14:16:57.795573+0000 (oldest deadline 2024-04-25T14:17:23.095608+0000) 2024-04-25T14:17:29.852 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:29 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:29.591+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:03.096238+0000 front 2024-04-25T14:17:03.096401+0000 (oldest deadline 2024-04-25T14:17:27.796160+0000) 2024-04-25T14:17:29.938 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:29 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:29.838+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:29.938 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:29 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:29.838+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:17:29.938 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:29 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:29.838+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:17:29.938 INFO:journalctl@ceph.osd.5.smithi151.stdout:Apr 25 14:17:29 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-5[44759]: 2024-04-25T14:17:29.684+0000 7fb233d33700 -1 osd.5 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.722828+0000 front 2024-04-25T14:16:49.722628+0000 (oldest deadline 2024-04-25T14:17:12.022695+0000) 2024-04-25T14:17:29.938 INFO:journalctl@ceph.osd.5.smithi151.stdout:Apr 25 14:17:29 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-5[44759]: 2024-04-25T14:17:29.684+0000 7fb233d33700 -1 osd.5 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.923838+0000 front 2024-04-25T14:16:57.923855+0000 (oldest deadline 2024-04-25T14:17:20.823710+0000) 2024-04-25T14:17:29.939 INFO:journalctl@ceph.osd.5.smithi151.stdout:Apr 25 14:17:29 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-5[44759]: 2024-04-25T14:17:29.684+0000 7fb233d33700 -1 osd.5 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:00.824220+0000 front 2024-04-25T14:17:00.824106+0000 (oldest deadline 2024-04-25T14:17:26.724317+0000) 2024-04-25T14:17:30.109 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:29 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:29.852+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.140614+0000 front 2024-04-25T14:16:47.140643+0000 (oldest deadline 2024-04-25T14:17:13.040930+0000) 2024-04-25T14:17:30.109 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:29 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:29.852+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.242964+0000 front 2024-04-25T14:16:58.242994+0000 (oldest deadline 2024-04-25T14:17:19.343170+0000) 2024-04-25T14:17:30.423 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:30 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:30.101+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.237196+0000 front 2024-04-25T14:16:55.237235+0000 (oldest deadline 2024-04-25T14:17:18.737337+0000) 2024-04-25T14:17:30.423 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:30 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:30.253+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:17:30.423 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:30 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:30.253+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:17:30.423 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:30 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:30.253+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.789654+0000 front 2024-04-25T14:17:02.789638+0000 (oldest deadline 2024-04-25T14:17:28.089414+0000) 2024-04-25T14:17:30.438 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:30 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:30.032+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:53.756445+0000 front 2024-04-25T14:16:53.756264+0000 (oldest deadline 2024-04-25T14:17:21.956954+0000) 2024-04-25T14:17:30.438 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:30 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:30.032+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:04.759056+0000 front 2024-04-25T14:17:04.759106+0000 (oldest deadline 2024-04-25T14:17:29.459176+0000) 2024-04-25T14:17:30.859 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:30 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:30.550+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:30.859 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:30 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:30.550+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.795847+0000 front 2024-04-25T14:16:57.795573+0000 (oldest deadline 2024-04-25T14:17:23.095608+0000) 2024-04-25T14:17:30.859 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:30 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:30.550+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:03.096238+0000 front 2024-04-25T14:17:03.096401+0000 (oldest deadline 2024-04-25T14:17:27.796160+0000) 2024-04-25T14:17:31.011 INFO:journalctl@ceph.osd.5.smithi151.stdout:Apr 25 14:17:30 smithi151 podman[67336]: 2024-04-25 14:17:30.715321824 +0000 UTC m=+5.485478528 container died 91b32bc024098d992ef05a5c52c823a6bd595dee7a4efdd1481dda99ed5f2719 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-25T14:17:31.011 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:30 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:30.841+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:31.011 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:30 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:30.841+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:17:31.011 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:30 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:30.841+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:17:31.359 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:30 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:30.870+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.140614+0000 front 2024-04-25T14:16:47.140643+0000 (oldest deadline 2024-04-25T14:17:13.040930+0000) 2024-04-25T14:17:31.359 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:30 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:30.870+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.242964+0000 front 2024-04-25T14:16:58.242994+0000 (oldest deadline 2024-04-25T14:17:19.343170+0000) 2024-04-25T14:17:31.359 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:30 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:30.870+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:06.046710+0000 front 2024-04-25T14:17:06.046659+0000 (oldest deadline 2024-04-25T14:17:30.146430+0000) 2024-04-25T14:17:31.370 INFO:journalctl@ceph.osd.5.smithi151.stdout:Apr 25 14:17:31 smithi151 podman[67336]: 2024-04-25 14:17:31.102893489 +0000 UTC m=+5.873050201 container cleanup 91b32bc024098d992ef05a5c52c823a6bd595dee7a4efdd1481dda99ed5f2719 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-25T14:17:31.370 INFO:journalctl@ceph.osd.5.smithi151.stdout:Apr 25 14:17:31 smithi151 bash[67336]: ceph-59334182-030c-11ef-bc93-c7b262605968-osd-5 2024-04-25T14:17:31.370 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:31 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:31.053+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:53.756445+0000 front 2024-04-25T14:16:53.756264+0000 (oldest deadline 2024-04-25T14:17:21.956954+0000) 2024-04-25T14:17:31.370 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:31 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:31.053+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:04.759056+0000 front 2024-04-25T14:17:04.759106+0000 (oldest deadline 2024-04-25T14:17:29.459176+0000) 2024-04-25T14:17:31.423 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:31 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:31.125+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.237196+0000 front 2024-04-25T14:16:55.237235+0000 (oldest deadline 2024-04-25T14:17:18.737337+0000) 2024-04-25T14:17:31.423 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:31 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:31.283+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:17:31.423 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:31 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:31.283+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:17:31.423 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:31 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:31.283+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.789654+0000 front 2024-04-25T14:17:02.789638+0000 (oldest deadline 2024-04-25T14:17:28.089414+0000) 2024-04-25T14:17:31.688 INFO:journalctl@ceph.osd.5.smithi151.stdout:Apr 25 14:17:31 smithi151 podman[67353]: 2024-04-25 14:17:31.371279693 +0000 UTC m=+0.652653029 container remove 91b32bc024098d992ef05a5c52c823a6bd595dee7a4efdd1481dda99ed5f2719 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-25T14:17:31.859 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:31 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:31.585+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:31.859 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:31 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:31.585+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.795847+0000 front 2024-04-25T14:16:57.795573+0000 (oldest deadline 2024-04-25T14:17:23.095608+0000) 2024-04-25T14:17:31.859 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:31 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:31.585+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:03.096238+0000 front 2024-04-25T14:17:03.096401+0000 (oldest deadline 2024-04-25T14:17:27.796160+0000) 2024-04-25T14:17:31.966 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:31 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:31.876+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:31.966 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:31 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:31.876+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:17:31.966 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:31 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:31.876+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:17:31.966 INFO:journalctl@ceph.osd.5.smithi151.stdout:Apr 25 14:17:31 smithi151 podman[67446]: 2024-04-25 14:17:31.597704694 +0000 UTC m=+0.017959798 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:17:31.967 INFO:journalctl@ceph.osd.5.smithi151.stdout:Apr 25 14:17:31 smithi151 podman[67446]: 2024-04-25 14:17:31.697348885 +0000 UTC m=+0.117603971 container create a74b9e6c85c912276b84b342bfe3c557ee72d2218b42893baea5d95dba4a1eca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T14:17:32.260 INFO:journalctl@ceph.osd.5.smithi151.stdout:Apr 25 14:17:31 smithi151 podman[67446]: 2024-04-25 14:17:31.967475901 +0000 UTC m=+0.387730987 container init a74b9e6c85c912276b84b342bfe3c557ee72d2218b42893baea5d95dba4a1eca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424) 2024-04-25T14:17:32.260 INFO:journalctl@ceph.osd.5.smithi151.stdout:Apr 25 14:17:31 smithi151 podman[67446]: 2024-04-25 14:17:31.970730232 +0000 UTC m=+0.390985328 container start a74b9e6c85c912276b84b342bfe3c557ee72d2218b42893baea5d95dba4a1eca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-25T14:17:32.260 INFO:journalctl@ceph.osd.5.smithi151.stdout:Apr 25 14:17:32 smithi151 podman[67446]: 2024-04-25 14:17:32.067670603 +0000 UTC m=+0.487925715 container attach a74b9e6c85c912276b84b342bfe3c557ee72d2218b42893baea5d95dba4a1eca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T14:17:32.260 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:32 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:32.020+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:53.756445+0000 front 2024-04-25T14:16:53.756264+0000 (oldest deadline 2024-04-25T14:17:21.956954+0000) 2024-04-25T14:17:32.260 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:32 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:32.020+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:04.759056+0000 front 2024-04-25T14:17:04.759106+0000 (oldest deadline 2024-04-25T14:17:29.459176+0000) 2024-04-25T14:17:32.359 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:31 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:31.861+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.140614+0000 front 2024-04-25T14:16:47.140643+0000 (oldest deadline 2024-04-25T14:17:13.040930+0000) 2024-04-25T14:17:32.359 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:31 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:31.861+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.242964+0000 front 2024-04-25T14:16:58.242994+0000 (oldest deadline 2024-04-25T14:17:19.343170+0000) 2024-04-25T14:17:32.359 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:31 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:31.861+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:06.046710+0000 front 2024-04-25T14:17:06.046659+0000 (oldest deadline 2024-04-25T14:17:30.146430+0000) 2024-04-25T14:17:32.422 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:32 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:32.109+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.237196+0000 front 2024-04-25T14:16:55.237235+0000 (oldest deadline 2024-04-25T14:17:18.737337+0000) 2024-04-25T14:17:32.423 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:32 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:32.320+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:17:32.423 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:32 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:32.320+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:17:32.423 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:32 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:32.320+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.789654+0000 front 2024-04-25T14:17:02.789638+0000 (oldest deadline 2024-04-25T14:17:28.089414+0000) 2024-04-25T14:17:32.555 INFO:journalctl@ceph.osd.5.smithi151.stdout:Apr 25 14:17:32 smithi151 podman[67446]: 2024-04-25 14:17:32.261058897 +0000 UTC m=+0.681313992 container died a74b9e6c85c912276b84b342bfe3c557ee72d2218b42893baea5d95dba4a1eca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-25T14:17:32.857 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@osd.5.service' 2024-04-25T14:17:32.937 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:17:32.938 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-25T14:17:32.938 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-25T14:17:32.938 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-59334182-030c-11ef-bc93-c7b262605968@osd.6 2024-04-25T14:17:32.963 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:32 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:32.812+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.140614+0000 front 2024-04-25T14:16:47.140643+0000 (oldest deadline 2024-04-25T14:17:13.040930+0000) 2024-04-25T14:17:32.963 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:32 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:32.812+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.242964+0000 front 2024-04-25T14:16:58.242994+0000 (oldest deadline 2024-04-25T14:17:19.343170+0000) 2024-04-25T14:17:32.964 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:32 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:32.812+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:06.046710+0000 front 2024-04-25T14:17:06.046659+0000 (oldest deadline 2024-04-25T14:17:30.146430+0000) 2024-04-25T14:17:32.964 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:32 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:32.630+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:32.964 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:32 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:32.630+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.795847+0000 front 2024-04-25T14:16:57.795573+0000 (oldest deadline 2024-04-25T14:17:23.095608+0000) 2024-04-25T14:17:32.964 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:32 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:32.631+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:03.096238+0000 front 2024-04-25T14:17:03.096401+0000 (oldest deadline 2024-04-25T14:17:27.796160+0000) 2024-04-25T14:17:33.188 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:33 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:33.030+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:53.756445+0000 front 2024-04-25T14:16:53.756264+0000 (oldest deadline 2024-04-25T14:17:21.956954+0000) 2024-04-25T14:17:33.188 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:33 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:33.030+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:04.759056+0000 front 2024-04-25T14:17:04.759106+0000 (oldest deadline 2024-04-25T14:17:29.459176+0000) 2024-04-25T14:17:33.188 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:32 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:32.850+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:33.188 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:32 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:32.850+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:17:33.188 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:32 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:32.850+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:17:33.345 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:33 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:33.084+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.237196+0000 front 2024-04-25T14:16:55.237235+0000 (oldest deadline 2024-04-25T14:17:18.737337+0000) 2024-04-25T14:17:33.346 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:33 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:33.084+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:05.739228+0000 front 2024-04-25T14:17:05.739275+0000 (oldest deadline 2024-04-25T14:17:32.139696+0000) 2024-04-25T14:17:33.359 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:32 smithi096 systemd[1]: Stopping Ceph osd.6 for 59334182-030c-11ef-bc93-c7b262605968... 2024-04-25T14:17:33.359 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:33 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:33.214+0000 7f7be5a4e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:17:33.359 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:33 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:33.214+0000 7f7be5a4e700 -1 osd.6 81 *** Got signal Terminated *** 2024-04-25T14:17:33.359 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:33 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:33.214+0000 7f7be5a4e700 -1 osd.6 81 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:17:33.672 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:33 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:33.345+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:17:33.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:33 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:33.345+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:17:33.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:33 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:33.345+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.789654+0000 front 2024-04-25T14:17:02.789638+0000 (oldest deadline 2024-04-25T14:17:28.089414+0000) 2024-04-25T14:17:34.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:33 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:33.664+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:34.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:33 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:33.664+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.795847+0000 front 2024-04-25T14:16:57.795573+0000 (oldest deadline 2024-04-25T14:17:23.095608+0000) 2024-04-25T14:17:34.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:33 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:33.664+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:03.096238+0000 front 2024-04-25T14:17:03.096401+0000 (oldest deadline 2024-04-25T14:17:27.796160+0000) 2024-04-25T14:17:34.110 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:33 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:33.812+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.140614+0000 front 2024-04-25T14:16:47.140643+0000 (oldest deadline 2024-04-25T14:17:13.040930+0000) 2024-04-25T14:17:34.110 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:33 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:33.812+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.242964+0000 front 2024-04-25T14:16:58.242994+0000 (oldest deadline 2024-04-25T14:17:19.343170+0000) 2024-04-25T14:17:34.110 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:33 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:33.812+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:06.046710+0000 front 2024-04-25T14:17:06.046659+0000 (oldest deadline 2024-04-25T14:17:30.146430+0000) 2024-04-25T14:17:34.189 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:34 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:34.025+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:53.756445+0000 front 2024-04-25T14:16:53.756264+0000 (oldest deadline 2024-04-25T14:17:21.956954+0000) 2024-04-25T14:17:34.189 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:34 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:34.025+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:04.759056+0000 front 2024-04-25T14:17:04.759106+0000 (oldest deadline 2024-04-25T14:17:29.459176+0000) 2024-04-25T14:17:34.189 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:33 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:33.879+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:34.190 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:33 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:33.879+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:17:34.190 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:33 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:33.879+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:17:34.385 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:34 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:34.062+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.237196+0000 front 2024-04-25T14:16:55.237235+0000 (oldest deadline 2024-04-25T14:17:18.737337+0000) 2024-04-25T14:17:34.385 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:34 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:34.062+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:05.739228+0000 front 2024-04-25T14:17:05.739275+0000 (oldest deadline 2024-04-25T14:17:32.139696+0000) 2024-04-25T14:17:34.672 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:34 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:34.385+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:17:34.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:34 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:34.385+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:17:34.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:34 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:34.385+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.789654+0000 front 2024-04-25T14:17:02.789638+0000 (oldest deadline 2024-04-25T14:17:28.089414+0000) 2024-04-25T14:17:35.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:34 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:34.710+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:35.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:34 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:34.710+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.795847+0000 front 2024-04-25T14:16:57.795573+0000 (oldest deadline 2024-04-25T14:17:23.095608+0000) 2024-04-25T14:17:35.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:34 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:34.710+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:03.096238+0000 front 2024-04-25T14:17:03.096401+0000 (oldest deadline 2024-04-25T14:17:27.796160+0000) 2024-04-25T14:17:35.109 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:34 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:34.766+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.140614+0000 front 2024-04-25T14:16:47.140643+0000 (oldest deadline 2024-04-25T14:17:13.040930+0000) 2024-04-25T14:17:35.109 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:34 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:34.766+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.242964+0000 front 2024-04-25T14:16:58.242994+0000 (oldest deadline 2024-04-25T14:17:19.343170+0000) 2024-04-25T14:17:35.110 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:34 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:34.766+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:06.046710+0000 front 2024-04-25T14:17:06.046659+0000 (oldest deadline 2024-04-25T14:17:30.146430+0000) 2024-04-25T14:17:35.188 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:35 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:35.001+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:53.756445+0000 front 2024-04-25T14:16:53.756264+0000 (oldest deadline 2024-04-25T14:17:21.956954+0000) 2024-04-25T14:17:35.188 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:35 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:35.001+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:04.759056+0000 front 2024-04-25T14:17:04.759106+0000 (oldest deadline 2024-04-25T14:17:29.459176+0000) 2024-04-25T14:17:35.188 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:34 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:34.840+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:35.188 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:34 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:34.840+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:17:35.188 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:34 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:34.840+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:17:35.422 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:35 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:35.022+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.237196+0000 front 2024-04-25T14:16:55.237235+0000 (oldest deadline 2024-04-25T14:17:18.737337+0000) 2024-04-25T14:17:35.423 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:35 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:35.022+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:05.739228+0000 front 2024-04-25T14:17:05.739275+0000 (oldest deadline 2024-04-25T14:17:32.139696+0000) 2024-04-25T14:17:35.922 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:35 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:35.428+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:17:35.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:35 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:35.428+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:17:35.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:35 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:35.428+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.789654+0000 front 2024-04-25T14:17:02.789638+0000 (oldest deadline 2024-04-25T14:17:28.089414+0000) 2024-04-25T14:17:35.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:35 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:35.428+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:10.990439+0000 front 2024-04-25T14:17:10.990451+0000 (oldest deadline 2024-04-25T14:17:35.090422+0000) 2024-04-25T14:17:36.089 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:36 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:36.018+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:53.756445+0000 front 2024-04-25T14:16:53.756264+0000 (oldest deadline 2024-04-25T14:17:21.956954+0000) 2024-04-25T14:17:36.089 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:36 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:36.018+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:04.759056+0000 front 2024-04-25T14:17:04.759106+0000 (oldest deadline 2024-04-25T14:17:29.459176+0000) 2024-04-25T14:17:36.089 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:36 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:36.018+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:11.760463+0000 front 2024-04-25T14:17:11.760420+0000 (oldest deadline 2024-04-25T14:17:35.860288+0000) 2024-04-25T14:17:36.090 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:35 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:35.838+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:36.090 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:35 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:35.838+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:17:36.090 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:35 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:35.838+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:17:36.109 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:35 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:35.804+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.140614+0000 front 2024-04-25T14:16:47.140643+0000 (oldest deadline 2024-04-25T14:17:13.040930+0000) 2024-04-25T14:17:36.109 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:35 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:35.804+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.242964+0000 front 2024-04-25T14:16:58.242994+0000 (oldest deadline 2024-04-25T14:17:19.343170+0000) 2024-04-25T14:17:36.109 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:35 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:35.804+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:06.046710+0000 front 2024-04-25T14:17:06.046659+0000 (oldest deadline 2024-04-25T14:17:30.146430+0000) 2024-04-25T14:17:36.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:35 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:35.757+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:36.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:35 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:35.757+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.795847+0000 front 2024-04-25T14:16:57.795573+0000 (oldest deadline 2024-04-25T14:17:23.095608+0000) 2024-04-25T14:17:36.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:35 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:35.757+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:03.096238+0000 front 2024-04-25T14:17:03.096401+0000 (oldest deadline 2024-04-25T14:17:27.796160+0000) 2024-04-25T14:17:36.386 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:35 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:35.984+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.237196+0000 front 2024-04-25T14:16:55.237235+0000 (oldest deadline 2024-04-25T14:17:18.737337+0000) 2024-04-25T14:17:36.386 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:35 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:35.984+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:05.739228+0000 front 2024-04-25T14:17:05.739275+0000 (oldest deadline 2024-04-25T14:17:32.139696+0000) 2024-04-25T14:17:36.672 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:36 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:36.386+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:17:36.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:36 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:36.386+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:17:36.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:36 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:36.386+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.789654+0000 front 2024-04-25T14:17:02.789638+0000 (oldest deadline 2024-04-25T14:17:28.089414+0000) 2024-04-25T14:17:36.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:36 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:36.386+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:10.990439+0000 front 2024-04-25T14:17:10.990451+0000 (oldest deadline 2024-04-25T14:17:35.090422+0000) 2024-04-25T14:17:37.109 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:36 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:36.841+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.140614+0000 front 2024-04-25T14:16:47.140643+0000 (oldest deadline 2024-04-25T14:17:13.040930+0000) 2024-04-25T14:17:37.109 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:36 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:36.841+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.242964+0000 front 2024-04-25T14:16:58.242994+0000 (oldest deadline 2024-04-25T14:17:19.343170+0000) 2024-04-25T14:17:37.109 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:36 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:36.841+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:06.046710+0000 front 2024-04-25T14:17:06.046659+0000 (oldest deadline 2024-04-25T14:17:30.146430+0000) 2024-04-25T14:17:37.109 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:36 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:36.841+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:10.146737+0000 front 2024-04-25T14:17:10.146796+0000 (oldest deadline 2024-04-25T14:17:36.046916+0000) 2024-04-25T14:17:37.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:36 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:36.737+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:37.110 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:36 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:36.737+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.795847+0000 front 2024-04-25T14:16:57.795573+0000 (oldest deadline 2024-04-25T14:17:23.095608+0000) 2024-04-25T14:17:37.110 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:36 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:36.737+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:03.096238+0000 front 2024-04-25T14:17:03.096401+0000 (oldest deadline 2024-04-25T14:17:27.796160+0000) 2024-04-25T14:17:37.110 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:36 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:36.737+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:12.997607+0000 front 2024-04-25T14:17:12.997700+0000 (oldest deadline 2024-04-25T14:17:36.497730+0000) 2024-04-25T14:17:37.188 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:36 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:36.820+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:37.188 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:36 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:36.820+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:17:37.188 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:36 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:36.820+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:17:37.188 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:36 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:36.991+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:53.756445+0000 front 2024-04-25T14:16:53.756264+0000 (oldest deadline 2024-04-25T14:17:21.956954+0000) 2024-04-25T14:17:37.188 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:36 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:36.991+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:04.759056+0000 front 2024-04-25T14:17:04.759106+0000 (oldest deadline 2024-04-25T14:17:29.459176+0000) 2024-04-25T14:17:37.188 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:36 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:36.991+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:11.760463+0000 front 2024-04-25T14:17:11.760420+0000 (oldest deadline 2024-04-25T14:17:35.860288+0000) 2024-04-25T14:17:37.392 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:36 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:36.977+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.237196+0000 front 2024-04-25T14:16:55.237235+0000 (oldest deadline 2024-04-25T14:17:18.737337+0000) 2024-04-25T14:17:37.392 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:36 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:36.977+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:05.739228+0000 front 2024-04-25T14:17:05.739275+0000 (oldest deadline 2024-04-25T14:17:32.139696+0000) 2024-04-25T14:17:37.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:37 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:37.392+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:17:37.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:37 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:37.392+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:17:37.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:37 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:37.392+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.789654+0000 front 2024-04-25T14:17:02.789638+0000 (oldest deadline 2024-04-25T14:17:28.089414+0000) 2024-04-25T14:17:37.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:37 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:37.392+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:10.990439+0000 front 2024-04-25T14:17:10.990451+0000 (oldest deadline 2024-04-25T14:17:35.090422+0000) 2024-04-25T14:17:38.110 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:37 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:37.719+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:38.110 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:37 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:37.719+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.795847+0000 front 2024-04-25T14:16:57.795573+0000 (oldest deadline 2024-04-25T14:17:23.095608+0000) 2024-04-25T14:17:38.110 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:37 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:37.719+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:03.096238+0000 front 2024-04-25T14:17:03.096401+0000 (oldest deadline 2024-04-25T14:17:27.796160+0000) 2024-04-25T14:17:38.110 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:37 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:37.719+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:12.997607+0000 front 2024-04-25T14:17:12.997700+0000 (oldest deadline 2024-04-25T14:17:36.497730+0000) 2024-04-25T14:17:38.111 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:37 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:37.878+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.140614+0000 front 2024-04-25T14:16:47.140643+0000 (oldest deadline 2024-04-25T14:17:13.040930+0000) 2024-04-25T14:17:38.111 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:37 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:37.878+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.242964+0000 front 2024-04-25T14:16:58.242994+0000 (oldest deadline 2024-04-25T14:17:19.343170+0000) 2024-04-25T14:17:38.111 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:37 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:37.878+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:06.046710+0000 front 2024-04-25T14:17:06.046659+0000 (oldest deadline 2024-04-25T14:17:30.146430+0000) 2024-04-25T14:17:38.111 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:37 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6[57199]: 2024-04-25T14:17:37.878+0000 7f7bddffc700 -1 osd.6 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:10.146737+0000 front 2024-04-25T14:17:10.146796+0000 (oldest deadline 2024-04-25T14:17:36.046916+0000) 2024-04-25T14:17:38.188 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:38 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:38.014+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:53.756445+0000 front 2024-04-25T14:16:53.756264+0000 (oldest deadline 2024-04-25T14:17:21.956954+0000) 2024-04-25T14:17:38.188 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:38 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:38.014+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:04.759056+0000 front 2024-04-25T14:17:04.759106+0000 (oldest deadline 2024-04-25T14:17:29.459176+0000) 2024-04-25T14:17:38.188 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:38 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:38.014+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:11.760463+0000 front 2024-04-25T14:17:11.760420+0000 (oldest deadline 2024-04-25T14:17:35.860288+0000) 2024-04-25T14:17:38.189 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:37 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:37.781+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:38.189 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:37 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:37.781+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:17:38.189 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:37 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:37.781+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:17:38.189 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:37 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:37.781+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.372555+0000 front 2024-04-25T14:17:13.372596+0000 (oldest deadline 2024-04-25T14:17:37.472628+0000) 2024-04-25T14:17:38.353 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:37 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:37.977+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.237196+0000 front 2024-04-25T14:16:55.237235+0000 (oldest deadline 2024-04-25T14:17:18.737337+0000) 2024-04-25T14:17:38.353 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:37 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:37.977+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:05.739228+0000 front 2024-04-25T14:17:05.739275+0000 (oldest deadline 2024-04-25T14:17:32.139696+0000) 2024-04-25T14:17:38.672 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:38 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:38.353+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:17:38.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:38 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:38.353+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:17:38.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:38 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:38.353+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.789654+0000 front 2024-04-25T14:17:02.789638+0000 (oldest deadline 2024-04-25T14:17:28.089414+0000) 2024-04-25T14:17:38.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:38 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:38.353+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:10.990439+0000 front 2024-04-25T14:17:10.990451+0000 (oldest deadline 2024-04-25T14:17:35.090422+0000) 2024-04-25T14:17:38.830 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:38 smithi096 podman[88758]: 2024-04-25 14:17:38.544421288 +0000 UTC m=+5.417979982 container died dfe424cc46343f12488c908086ea4049f300b1e740b7dabb66e94be5d38d0e35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS) 2024-04-25T14:17:38.830 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:38 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:38.669+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:38.830 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:38 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:38.669+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.795847+0000 front 2024-04-25T14:16:57.795573+0000 (oldest deadline 2024-04-25T14:17:23.095608+0000) 2024-04-25T14:17:38.830 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:38 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:38.669+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:03.096238+0000 front 2024-04-25T14:17:03.096401+0000 (oldest deadline 2024-04-25T14:17:27.796160+0000) 2024-04-25T14:17:38.831 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:38 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:38.669+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:12.997607+0000 front 2024-04-25T14:17:12.997700+0000 (oldest deadline 2024-04-25T14:17:36.497730+0000) 2024-04-25T14:17:39.109 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:39 smithi096 podman[88758]: 2024-04-25 14:17:39.046978663 +0000 UTC m=+5.920537358 container cleanup dfe424cc46343f12488c908086ea4049f300b1e740b7dabb66e94be5d38d0e35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-25T14:17:39.109 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:39 smithi096 bash[88758]: ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6 2024-04-25T14:17:39.188 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:39 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:39.060+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:53.756445+0000 front 2024-04-25T14:16:53.756264+0000 (oldest deadline 2024-04-25T14:17:21.956954+0000) 2024-04-25T14:17:39.188 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:39 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:39.060+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:04.759056+0000 front 2024-04-25T14:17:04.759106+0000 (oldest deadline 2024-04-25T14:17:29.459176+0000) 2024-04-25T14:17:39.188 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:39 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:39.060+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:11.760463+0000 front 2024-04-25T14:17:11.760420+0000 (oldest deadline 2024-04-25T14:17:35.860288+0000) 2024-04-25T14:17:39.189 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:38 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:38.823+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:39.189 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:38 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:38.823+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:17:39.189 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:38 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:38.823+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:17:39.189 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:38 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:38.823+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.372555+0000 front 2024-04-25T14:17:13.372596+0000 (oldest deadline 2024-04-25T14:17:37.472628+0000) 2024-04-25T14:17:39.372 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:39 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:39.002+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.237196+0000 front 2024-04-25T14:16:55.237235+0000 (oldest deadline 2024-04-25T14:17:18.737337+0000) 2024-04-25T14:17:39.373 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:39 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:39.002+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:05.739228+0000 front 2024-04-25T14:17:05.739275+0000 (oldest deadline 2024-04-25T14:17:32.139696+0000) 2024-04-25T14:17:39.665 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:39 smithi096 podman[88775]: 2024-04-25 14:17:39.407082114 +0000 UTC m=+0.863341242 container remove dfe424cc46343f12488c908086ea4049f300b1e740b7dabb66e94be5d38d0e35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:17:39.672 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:39 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:39.373+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:17:39.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:39 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:39.373+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:17:39.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:39 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:39.373+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.789654+0000 front 2024-04-25T14:17:02.789638+0000 (oldest deadline 2024-04-25T14:17:28.089414+0000) 2024-04-25T14:17:39.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:39 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:39.373+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:10.990439+0000 front 2024-04-25T14:17:10.990451+0000 (oldest deadline 2024-04-25T14:17:35.090422+0000) 2024-04-25T14:17:39.972 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:39 smithi096 podman[88867]: 2024-04-25 14:17:39.732236646 +0000 UTC m=+0.018214729 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:17:39.972 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:39 smithi096 podman[88867]: 2024-04-25 14:17:39.858209446 +0000 UTC m=+0.144187508 container create cbad08a24b0d472dacfc224f39587e26aa0024c20e4814f3b13d21d97b87f81d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-25T14:17:39.973 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:39 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:39.665+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:39.973 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:39 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:39.665+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.795847+0000 front 2024-04-25T14:16:57.795573+0000 (oldest deadline 2024-04-25T14:17:23.095608+0000) 2024-04-25T14:17:39.973 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:39 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:39.665+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:03.096238+0000 front 2024-04-25T14:17:03.096401+0000 (oldest deadline 2024-04-25T14:17:27.796160+0000) 2024-04-25T14:17:39.973 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:39 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:39.665+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:12.997607+0000 front 2024-04-25T14:17:12.997700+0000 (oldest deadline 2024-04-25T14:17:36.497730+0000) 2024-04-25T14:17:40.061 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:39 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:39.777+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:40.062 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:39 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:39.777+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:17:40.062 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:39 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:39.777+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:17:40.062 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:39 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:39.777+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.372555+0000 front 2024-04-25T14:17:13.372596+0000 (oldest deadline 2024-04-25T14:17:37.472628+0000) 2024-04-25T14:17:40.359 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:40 smithi096 podman[88867]: 2024-04-25 14:17:40.110940638 +0000 UTC m=+0.396918710 container init cbad08a24b0d472dacfc224f39587e26aa0024c20e4814f3b13d21d97b87f81d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2) 2024-04-25T14:17:40.359 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:40 smithi096 podman[88867]: 2024-04-25 14:17:40.114352754 +0000 UTC m=+0.400330821 container start cbad08a24b0d472dacfc224f39587e26aa0024c20e4814f3b13d21d97b87f81d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-25T14:17:40.359 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:40 smithi096 podman[88867]: 2024-04-25 14:17:40.193856943 +0000 UTC m=+0.479835019 container attach cbad08a24b0d472dacfc224f39587e26aa0024c20e4814f3b13d21d97b87f81d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True) 2024-04-25T14:17:40.410 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:39 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:39.963+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.237196+0000 front 2024-04-25T14:16:55.237235+0000 (oldest deadline 2024-04-25T14:17:18.737337+0000) 2024-04-25T14:17:40.410 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:39 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:39.963+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:05.739228+0000 front 2024-04-25T14:17:05.739275+0000 (oldest deadline 2024-04-25T14:17:32.139696+0000) 2024-04-25T14:17:40.410 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:39 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:39.963+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.840833+0000 front 2024-04-25T14:17:13.840776+0000 (oldest deadline 2024-04-25T14:17:39.740701+0000) 2024-04-25T14:17:40.438 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:40 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:40.062+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:53.756445+0000 front 2024-04-25T14:16:53.756264+0000 (oldest deadline 2024-04-25T14:17:21.956954+0000) 2024-04-25T14:17:40.438 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:40 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:40.062+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:04.759056+0000 front 2024-04-25T14:17:04.759106+0000 (oldest deadline 2024-04-25T14:17:29.459176+0000) 2024-04-25T14:17:40.438 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:40 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:40.062+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:11.760463+0000 front 2024-04-25T14:17:11.760420+0000 (oldest deadline 2024-04-25T14:17:35.860288+0000) 2024-04-25T14:17:40.659 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 14:17:40 smithi096 podman[88867]: 2024-04-25 14:17:40.408654421 +0000 UTC m=+0.694632487 container died cbad08a24b0d472dacfc224f39587e26aa0024c20e4814f3b13d21d97b87f81d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240424) 2024-04-25T14:17:40.672 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:40 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:40.410+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:17:40.672 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:40 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:40.410+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:17:40.672 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:40 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:40.410+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.789654+0000 front 2024-04-25T14:17:02.789638+0000 (oldest deadline 2024-04-25T14:17:28.089414+0000) 2024-04-25T14:17:40.672 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:40 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:40.410+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:10.990439+0000 front 2024-04-25T14:17:10.990451+0000 (oldest deadline 2024-04-25T14:17:35.090422+0000) 2024-04-25T14:17:40.997 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:40 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:40.659+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:40.997 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:40 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:40.659+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.795847+0000 front 2024-04-25T14:16:57.795573+0000 (oldest deadline 2024-04-25T14:17:23.095608+0000) 2024-04-25T14:17:40.997 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:40 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:40.659+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:03.096238+0000 front 2024-04-25T14:17:03.096401+0000 (oldest deadline 2024-04-25T14:17:27.796160+0000) 2024-04-25T14:17:40.997 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:40 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:40.659+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:12.997607+0000 front 2024-04-25T14:17:12.997700+0000 (oldest deadline 2024-04-25T14:17:36.497730+0000) 2024-04-25T14:17:41.012 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@osd.6.service' 2024-04-25T14:17:41.076 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:40 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:40.778+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:41.076 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:40 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:40.778+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:17:41.077 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:40 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:40.778+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:17:41.077 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:40 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:40.778+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.372555+0000 front 2024-04-25T14:17:13.372596+0000 (oldest deadline 2024-04-25T14:17:37.472628+0000) 2024-04-25T14:17:41.090 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:17:41.091 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-25T14:17:41.091 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-25T14:17:41.091 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-59334182-030c-11ef-bc93-c7b262605968@osd.7 2024-04-25T14:17:41.174 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:40 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:40.915+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.237196+0000 front 2024-04-25T14:16:55.237235+0000 (oldest deadline 2024-04-25T14:17:18.737337+0000) 2024-04-25T14:17:41.272 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:40 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:40.915+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:05.739228+0000 front 2024-04-25T14:17:05.739275+0000 (oldest deadline 2024-04-25T14:17:32.139696+0000) 2024-04-25T14:17:41.274 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:40 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:40.915+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.840833+0000 front 2024-04-25T14:17:13.840776+0000 (oldest deadline 2024-04-25T14:17:39.740701+0000) 2024-04-25T14:17:41.274 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:41 smithi097 systemd[1]: Stopping Ceph osd.7 for 59334182-030c-11ef-bc93-c7b262605968... 2024-04-25T14:17:41.438 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:41 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:41.077+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:53.756445+0000 front 2024-04-25T14:16:53.756264+0000 (oldest deadline 2024-04-25T14:17:21.956954+0000) 2024-04-25T14:17:41.438 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:41 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:41.077+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:04.759056+0000 front 2024-04-25T14:17:04.759106+0000 (oldest deadline 2024-04-25T14:17:29.459176+0000) 2024-04-25T14:17:41.438 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:41 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:41.077+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:11.760463+0000 front 2024-04-25T14:17:11.760420+0000 (oldest deadline 2024-04-25T14:17:35.860288+0000) 2024-04-25T14:17:41.518 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:41 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:41.380+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:17:41.518 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:41 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:41.380+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:17:41.518 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:41 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:41.380+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.789654+0000 front 2024-04-25T14:17:02.789638+0000 (oldest deadline 2024-04-25T14:17:28.089414+0000) 2024-04-25T14:17:41.518 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:41 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:41.380+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:10.990439+0000 front 2024-04-25T14:17:10.990451+0000 (oldest deadline 2024-04-25T14:17:35.090422+0000) 2024-04-25T14:17:41.916 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:41 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:41.518+0000 7fe1abe67700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:17:41.916 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:41 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:41.518+0000 7fe1abe67700 -1 osd.7 81 *** Got signal Terminated *** 2024-04-25T14:17:41.917 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:41 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:41.518+0000 7fe1abe67700 -1 osd.7 81 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:17:42.102 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:41 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:41.767+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:42.102 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:41 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:41.767+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:17:42.102 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:41 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:41.767+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:17:42.103 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:41 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:41.767+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.372555+0000 front 2024-04-25T14:17:13.372596+0000 (oldest deadline 2024-04-25T14:17:37.472628+0000) 2024-04-25T14:17:42.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:41 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:41.645+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:42.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:41 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:41.645+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.795847+0000 front 2024-04-25T14:16:57.795573+0000 (oldest deadline 2024-04-25T14:17:23.095608+0000) 2024-04-25T14:17:42.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:41 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:41.645+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:03.096238+0000 front 2024-04-25T14:17:03.096401+0000 (oldest deadline 2024-04-25T14:17:27.796160+0000) 2024-04-25T14:17:42.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:41 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:41.645+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:12.997607+0000 front 2024-04-25T14:17:12.997700+0000 (oldest deadline 2024-04-25T14:17:36.497730+0000) 2024-04-25T14:17:42.172 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:41 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:41.916+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.237196+0000 front 2024-04-25T14:16:55.237235+0000 (oldest deadline 2024-04-25T14:17:18.737337+0000) 2024-04-25T14:17:42.173 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:41 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:41.916+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:05.739228+0000 front 2024-04-25T14:17:05.739275+0000 (oldest deadline 2024-04-25T14:17:32.139696+0000) 2024-04-25T14:17:42.173 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:41 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:41.916+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.840833+0000 front 2024-04-25T14:17:13.840776+0000 (oldest deadline 2024-04-25T14:17:39.740701+0000) 2024-04-25T14:17:42.438 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:42 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:42.103+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:53.756445+0000 front 2024-04-25T14:16:53.756264+0000 (oldest deadline 2024-04-25T14:17:21.956954+0000) 2024-04-25T14:17:42.438 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:42 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:42.103+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:04.759056+0000 front 2024-04-25T14:17:04.759106+0000 (oldest deadline 2024-04-25T14:17:29.459176+0000) 2024-04-25T14:17:42.438 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:42 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:42.103+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:11.760463+0000 front 2024-04-25T14:17:11.760420+0000 (oldest deadline 2024-04-25T14:17:35.860288+0000) 2024-04-25T14:17:42.447 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:42 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:42.335+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:17:42.447 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:42 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:42.335+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:17:42.447 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:42 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:42.335+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.789654+0000 front 2024-04-25T14:17:02.789638+0000 (oldest deadline 2024-04-25T14:17:28.089414+0000) 2024-04-25T14:17:42.447 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:42 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:42.335+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:10.990439+0000 front 2024-04-25T14:17:10.990451+0000 (oldest deadline 2024-04-25T14:17:35.090422+0000) 2024-04-25T14:17:43.104 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:42 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:42.783+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:43.104 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:42 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:42.783+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:17:43.104 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:42 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:42.783+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:17:43.105 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:42 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:42.783+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.372555+0000 front 2024-04-25T14:17:13.372596+0000 (oldest deadline 2024-04-25T14:17:37.472628+0000) 2024-04-25T14:17:43.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:42 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:42.643+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:43.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:42 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:42.643+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.795847+0000 front 2024-04-25T14:16:57.795573+0000 (oldest deadline 2024-04-25T14:17:23.095608+0000) 2024-04-25T14:17:43.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:42 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:42.643+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:03.096238+0000 front 2024-04-25T14:17:03.096401+0000 (oldest deadline 2024-04-25T14:17:27.796160+0000) 2024-04-25T14:17:43.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:42 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:42.643+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:12.997607+0000 front 2024-04-25T14:17:12.997700+0000 (oldest deadline 2024-04-25T14:17:36.497730+0000) 2024-04-25T14:17:43.172 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:42 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:42.897+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.237196+0000 front 2024-04-25T14:16:55.237235+0000 (oldest deadline 2024-04-25T14:17:18.737337+0000) 2024-04-25T14:17:43.172 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:42 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:42.897+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:05.739228+0000 front 2024-04-25T14:17:05.739275+0000 (oldest deadline 2024-04-25T14:17:32.139696+0000) 2024-04-25T14:17:43.173 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:42 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:42.897+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.840833+0000 front 2024-04-25T14:17:13.840776+0000 (oldest deadline 2024-04-25T14:17:39.740701+0000) 2024-04-25T14:17:43.438 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:43 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:43.104+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:53.756445+0000 front 2024-04-25T14:16:53.756264+0000 (oldest deadline 2024-04-25T14:17:21.956954+0000) 2024-04-25T14:17:43.438 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:43 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:43.104+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:04.759056+0000 front 2024-04-25T14:17:04.759106+0000 (oldest deadline 2024-04-25T14:17:29.459176+0000) 2024-04-25T14:17:43.438 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:43 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:43.104+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:11.760463+0000 front 2024-04-25T14:17:11.760420+0000 (oldest deadline 2024-04-25T14:17:35.860288+0000) 2024-04-25T14:17:43.438 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:43 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:43.104+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:19.961379+0000 front 2024-04-25T14:17:19.961555+0000 (oldest deadline 2024-04-25T14:17:42.861401+0000) 2024-04-25T14:17:43.672 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:43 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:43.318+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:17:43.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:43 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:43.318+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:17:43.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:43 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:43.318+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.789654+0000 front 2024-04-25T14:17:02.789638+0000 (oldest deadline 2024-04-25T14:17:28.089414+0000) 2024-04-25T14:17:43.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:43 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:43.318+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:10.990439+0000 front 2024-04-25T14:17:10.990451+0000 (oldest deadline 2024-04-25T14:17:35.090422+0000) 2024-04-25T14:17:44.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:43 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:43.662+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:44.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:43 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:43.662+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.795847+0000 front 2024-04-25T14:16:57.795573+0000 (oldest deadline 2024-04-25T14:17:23.095608+0000) 2024-04-25T14:17:44.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:43 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:43.662+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:03.096238+0000 front 2024-04-25T14:17:03.096401+0000 (oldest deadline 2024-04-25T14:17:27.796160+0000) 2024-04-25T14:17:44.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:43 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:43.662+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:12.997607+0000 front 2024-04-25T14:17:12.997700+0000 (oldest deadline 2024-04-25T14:17:36.497730+0000) 2024-04-25T14:17:44.114 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:43 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:43.755+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:44.114 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:43 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:43.755+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:17:44.114 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:43 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:43.755+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:17:44.114 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:43 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:43.755+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.372555+0000 front 2024-04-25T14:17:13.372596+0000 (oldest deadline 2024-04-25T14:17:37.472628+0000) 2024-04-25T14:17:44.172 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:43 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:43.886+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.237196+0000 front 2024-04-25T14:16:55.237235+0000 (oldest deadline 2024-04-25T14:17:18.737337+0000) 2024-04-25T14:17:44.173 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:43 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:43.886+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:05.739228+0000 front 2024-04-25T14:17:05.739275+0000 (oldest deadline 2024-04-25T14:17:32.139696+0000) 2024-04-25T14:17:44.173 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:43 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:43.886+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.840833+0000 front 2024-04-25T14:17:13.840776+0000 (oldest deadline 2024-04-25T14:17:39.740701+0000) 2024-04-25T14:17:44.438 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:44 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:44.113+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:53.756445+0000 front 2024-04-25T14:16:53.756264+0000 (oldest deadline 2024-04-25T14:17:21.956954+0000) 2024-04-25T14:17:44.438 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:44 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:44.113+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:04.759056+0000 front 2024-04-25T14:17:04.759106+0000 (oldest deadline 2024-04-25T14:17:29.459176+0000) 2024-04-25T14:17:44.438 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:44 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:44.113+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:11.760463+0000 front 2024-04-25T14:17:11.760420+0000 (oldest deadline 2024-04-25T14:17:35.860288+0000) 2024-04-25T14:17:44.438 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:44 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:44.113+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:19.961379+0000 front 2024-04-25T14:17:19.961555+0000 (oldest deadline 2024-04-25T14:17:42.861401+0000) 2024-04-25T14:17:44.672 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:44 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:44.358+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:17:44.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:44 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:44.358+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:17:44.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:44 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:44.358+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.789654+0000 front 2024-04-25T14:17:02.789638+0000 (oldest deadline 2024-04-25T14:17:28.089414+0000) 2024-04-25T14:17:44.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:44 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:44.358+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:10.990439+0000 front 2024-04-25T14:17:10.990451+0000 (oldest deadline 2024-04-25T14:17:35.090422+0000) 2024-04-25T14:17:45.101 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:44 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:44.800+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:45.102 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:44 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:44.800+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:17:45.102 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:44 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:44.800+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:17:45.102 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:44 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:44.800+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.372555+0000 front 2024-04-25T14:17:13.372596+0000 (oldest deadline 2024-04-25T14:17:37.472628+0000) 2024-04-25T14:17:45.102 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:44 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:44.800+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:20.373797+0000 front 2024-04-25T14:17:20.373887+0000 (oldest deadline 2024-04-25T14:17:44.473980+0000) 2024-04-25T14:17:45.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:44 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:44.669+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:45.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:44 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:44.669+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.795847+0000 front 2024-04-25T14:16:57.795573+0000 (oldest deadline 2024-04-25T14:17:23.095608+0000) 2024-04-25T14:17:45.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:44 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:44.669+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:03.096238+0000 front 2024-04-25T14:17:03.096401+0000 (oldest deadline 2024-04-25T14:17:27.796160+0000) 2024-04-25T14:17:45.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:44 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:44.669+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:12.997607+0000 front 2024-04-25T14:17:12.997700+0000 (oldest deadline 2024-04-25T14:17:36.497730+0000) 2024-04-25T14:17:45.173 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:44 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:44.890+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.237196+0000 front 2024-04-25T14:16:55.237235+0000 (oldest deadline 2024-04-25T14:17:18.737337+0000) 2024-04-25T14:17:45.173 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:44 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:44.890+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:05.739228+0000 front 2024-04-25T14:17:05.739275+0000 (oldest deadline 2024-04-25T14:17:32.139696+0000) 2024-04-25T14:17:45.173 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:44 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:44.890+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.840833+0000 front 2024-04-25T14:17:13.840776+0000 (oldest deadline 2024-04-25T14:17:39.740701+0000) 2024-04-25T14:17:45.438 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:45 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:45.101+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:53.756445+0000 front 2024-04-25T14:16:53.756264+0000 (oldest deadline 2024-04-25T14:17:21.956954+0000) 2024-04-25T14:17:45.438 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:45 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:45.101+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:04.759056+0000 front 2024-04-25T14:17:04.759106+0000 (oldest deadline 2024-04-25T14:17:29.459176+0000) 2024-04-25T14:17:45.438 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:45 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:45.101+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:11.760463+0000 front 2024-04-25T14:17:11.760420+0000 (oldest deadline 2024-04-25T14:17:35.860288+0000) 2024-04-25T14:17:45.438 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:45 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:45.101+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:19.961379+0000 front 2024-04-25T14:17:19.961555+0000 (oldest deadline 2024-04-25T14:17:42.861401+0000) 2024-04-25T14:17:45.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:45 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:45.393+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:17:45.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:45 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:45.393+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:17:45.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:45 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:45.393+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.789654+0000 front 2024-04-25T14:17:02.789638+0000 (oldest deadline 2024-04-25T14:17:28.089414+0000) 2024-04-25T14:17:45.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:45 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:45.393+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:10.990439+0000 front 2024-04-25T14:17:10.990451+0000 (oldest deadline 2024-04-25T14:17:35.090422+0000) 2024-04-25T14:17:45.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:45 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:45.393+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:19.191383+0000 front 2024-04-25T14:17:19.191428+0000 (oldest deadline 2024-04-25T14:17:45.091518+0000) 2024-04-25T14:17:46.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:45 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:45.714+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:46.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:45 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:45.714+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.795847+0000 front 2024-04-25T14:16:57.795573+0000 (oldest deadline 2024-04-25T14:17:23.095608+0000) 2024-04-25T14:17:46.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:45 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:45.714+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:03.096238+0000 front 2024-04-25T14:17:03.096401+0000 (oldest deadline 2024-04-25T14:17:27.796160+0000) 2024-04-25T14:17:46.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:45 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:45.714+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:12.997607+0000 front 2024-04-25T14:17:12.997700+0000 (oldest deadline 2024-04-25T14:17:36.497730+0000) 2024-04-25T14:17:46.172 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:45 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:45.892+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.237196+0000 front 2024-04-25T14:16:55.237235+0000 (oldest deadline 2024-04-25T14:17:18.737337+0000) 2024-04-25T14:17:46.173 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:45 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:45.892+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:05.739228+0000 front 2024-04-25T14:17:05.739275+0000 (oldest deadline 2024-04-25T14:17:32.139696+0000) 2024-04-25T14:17:46.173 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:45 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:45.892+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.840833+0000 front 2024-04-25T14:17:13.840776+0000 (oldest deadline 2024-04-25T14:17:39.740701+0000) 2024-04-25T14:17:46.173 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:45 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7[51222]: 2024-04-25T14:17:45.892+0000 7fe1a4415700 -1 osd.7 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:22.041698+0000 front 2024-04-25T14:17:22.041716+0000 (oldest deadline 2024-04-25T14:17:45.541753+0000) 2024-04-25T14:17:46.188 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:46 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:46.063+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:53.756445+0000 front 2024-04-25T14:16:53.756264+0000 (oldest deadline 2024-04-25T14:17:21.956954+0000) 2024-04-25T14:17:46.188 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:46 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:46.063+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:04.759056+0000 front 2024-04-25T14:17:04.759106+0000 (oldest deadline 2024-04-25T14:17:29.459176+0000) 2024-04-25T14:17:46.188 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:46 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:46.063+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:11.760463+0000 front 2024-04-25T14:17:11.760420+0000 (oldest deadline 2024-04-25T14:17:35.860288+0000) 2024-04-25T14:17:46.188 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:46 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:46.063+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:19.961379+0000 front 2024-04-25T14:17:19.961555+0000 (oldest deadline 2024-04-25T14:17:42.861401+0000) 2024-04-25T14:17:46.188 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:45 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:45.845+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:46.188 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:45 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:45.845+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:17:46.188 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:45 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:45.845+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:17:46.188 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:45 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:45.845+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.372555+0000 front 2024-04-25T14:17:13.372596+0000 (oldest deadline 2024-04-25T14:17:37.472628+0000) 2024-04-25T14:17:46.188 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:45 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:45.845+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:20.373797+0000 front 2024-04-25T14:17:20.373887+0000 (oldest deadline 2024-04-25T14:17:44.473980+0000) 2024-04-25T14:17:46.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:46 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:46.373+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:17:46.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:46 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:46.373+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:17:46.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:46 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:46.373+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.789654+0000 front 2024-04-25T14:17:02.789638+0000 (oldest deadline 2024-04-25T14:17:28.089414+0000) 2024-04-25T14:17:46.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:46 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:46.373+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:10.990439+0000 front 2024-04-25T14:17:10.990451+0000 (oldest deadline 2024-04-25T14:17:35.090422+0000) 2024-04-25T14:17:46.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:46 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:46.373+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:19.191383+0000 front 2024-04-25T14:17:19.191428+0000 (oldest deadline 2024-04-25T14:17:45.091518+0000) 2024-04-25T14:17:47.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:46 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:46.757+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:47.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:46 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:46.757+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.795847+0000 front 2024-04-25T14:16:57.795573+0000 (oldest deadline 2024-04-25T14:17:23.095608+0000) 2024-04-25T14:17:47.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:46 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:46.757+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:03.096238+0000 front 2024-04-25T14:17:03.096401+0000 (oldest deadline 2024-04-25T14:17:27.796160+0000) 2024-04-25T14:17:47.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:46 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:46.757+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:12.997607+0000 front 2024-04-25T14:17:12.997700+0000 (oldest deadline 2024-04-25T14:17:36.497730+0000) 2024-04-25T14:17:47.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:46 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:46.757+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:21.699312+0000 front 2024-04-25T14:17:21.699430+0000 (oldest deadline 2024-04-25T14:17:46.399249+0000) 2024-04-25T14:17:47.188 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:47 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:47.027+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:53.756445+0000 front 2024-04-25T14:16:53.756264+0000 (oldest deadline 2024-04-25T14:17:21.956954+0000) 2024-04-25T14:17:47.188 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:47 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:47.027+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:04.759056+0000 front 2024-04-25T14:17:04.759106+0000 (oldest deadline 2024-04-25T14:17:29.459176+0000) 2024-04-25T14:17:47.188 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:47 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:47.027+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:11.760463+0000 front 2024-04-25T14:17:11.760420+0000 (oldest deadline 2024-04-25T14:17:35.860288+0000) 2024-04-25T14:17:47.188 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:47 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:47.027+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:19.961379+0000 front 2024-04-25T14:17:19.961555+0000 (oldest deadline 2024-04-25T14:17:42.861401+0000) 2024-04-25T14:17:47.188 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:46 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:46.817+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:47.189 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:46 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:46.817+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:17:47.189 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:46 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:46.817+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:17:47.189 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:46 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:46.817+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.372555+0000 front 2024-04-25T14:17:13.372596+0000 (oldest deadline 2024-04-25T14:17:37.472628+0000) 2024-04-25T14:17:47.189 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:46 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:46.817+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:20.373797+0000 front 2024-04-25T14:17:20.373887+0000 (oldest deadline 2024-04-25T14:17:44.473980+0000) 2024-04-25T14:17:47.335 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:47 smithi097 podman[66878]: 2024-04-25 14:17:47.049386194 +0000 UTC m=+5.761147564 container died bcf004867959fd2f0c07a5af6fe39219ddca549aa7bd55b19ed309b49b701c23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-25T14:17:47.610 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:47 smithi097 podman[66878]: 2024-04-25 14:17:47.427102342 +0000 UTC m=+6.138863707 container cleanup bcf004867959fd2f0c07a5af6fe39219ddca549aa7bd55b19ed309b49b701c23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-25T14:17:47.611 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:47 smithi097 bash[66878]: ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7 2024-04-25T14:17:47.611 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:47 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:47.408+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:17:47.611 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:47 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:47.408+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:17:47.611 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:47 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:47.408+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.789654+0000 front 2024-04-25T14:17:02.789638+0000 (oldest deadline 2024-04-25T14:17:28.089414+0000) 2024-04-25T14:17:47.611 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:47 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:47.408+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:10.990439+0000 front 2024-04-25T14:17:10.990451+0000 (oldest deadline 2024-04-25T14:17:35.090422+0000) 2024-04-25T14:17:47.611 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:47 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:47.408+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:19.191383+0000 front 2024-04-25T14:17:19.191428+0000 (oldest deadline 2024-04-25T14:17:45.091518+0000) 2024-04-25T14:17:47.922 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:47 smithi097 podman[66907]: 2024-04-25 14:17:47.611705878 +0000 UTC m=+0.561592365 container remove bcf004867959fd2f0c07a5af6fe39219ddca549aa7bd55b19ed309b49b701c23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:17:48.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:47 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:47.723+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:48.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:47 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:47.723+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.795847+0000 front 2024-04-25T14:16:57.795573+0000 (oldest deadline 2024-04-25T14:17:23.095608+0000) 2024-04-25T14:17:48.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:47 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:47.723+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:03.096238+0000 front 2024-04-25T14:17:03.096401+0000 (oldest deadline 2024-04-25T14:17:27.796160+0000) 2024-04-25T14:17:48.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:47 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:47.723+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:12.997607+0000 front 2024-04-25T14:17:12.997700+0000 (oldest deadline 2024-04-25T14:17:36.497730+0000) 2024-04-25T14:17:48.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:47 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:47.723+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:21.699312+0000 front 2024-04-25T14:17:21.699430+0000 (oldest deadline 2024-04-25T14:17:46.399249+0000) 2024-04-25T14:17:48.188 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:47 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:47.989+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:53.756445+0000 front 2024-04-25T14:16:53.756264+0000 (oldest deadline 2024-04-25T14:17:21.956954+0000) 2024-04-25T14:17:48.188 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:47 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:47.989+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:04.759056+0000 front 2024-04-25T14:17:04.759106+0000 (oldest deadline 2024-04-25T14:17:29.459176+0000) 2024-04-25T14:17:48.188 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:47 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:47.989+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:11.760463+0000 front 2024-04-25T14:17:11.760420+0000 (oldest deadline 2024-04-25T14:17:35.860288+0000) 2024-04-25T14:17:48.188 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:47 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:47.989+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:19.961379+0000 front 2024-04-25T14:17:19.961555+0000 (oldest deadline 2024-04-25T14:17:42.861401+0000) 2024-04-25T14:17:48.188 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:47 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:47.849+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:48.188 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:47 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:47.849+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:17:48.189 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:47 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:47.849+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:17:48.189 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:47 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:47.849+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.372555+0000 front 2024-04-25T14:17:13.372596+0000 (oldest deadline 2024-04-25T14:17:37.472628+0000) 2024-04-25T14:17:48.189 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:47 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:47.849+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:20.373797+0000 front 2024-04-25T14:17:20.373887+0000 (oldest deadline 2024-04-25T14:17:44.473980+0000) 2024-04-25T14:17:48.339 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:48 smithi097 podman[66996]: 2024-04-25 14:17:47.946237972 +0000 UTC m=+0.018152287 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:17:48.339 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:48 smithi097 podman[66996]: 2024-04-25 14:17:48.062604931 +0000 UTC m=+0.134519226 container create f09892f0f6c4933ae186e9368bc96a7cdabe2527b6c18e7c1062387392704e3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T14:17:48.339 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:48 smithi097 podman[66996]: 2024-04-25 14:17:48.281526697 +0000 UTC m=+0.353440999 container init f09892f0f6c4933ae186e9368bc96a7cdabe2527b6c18e7c1062387392704e3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T14:17:48.339 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:48 smithi097 podman[66996]: 2024-04-25 14:17:48.284601864 +0000 UTC m=+0.356516170 container start f09892f0f6c4933ae186e9368bc96a7cdabe2527b6c18e7c1062387392704e3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T14:17:48.673 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:48 smithi097 podman[66996]: 2024-04-25 14:17:48.339854259 +0000 UTC m=+0.411768563 container attach f09892f0f6c4933ae186e9368bc96a7cdabe2527b6c18e7c1062387392704e3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T14:17:48.673 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 14:17:48 smithi097 podman[66996]: 2024-04-25 14:17:48.580741815 +0000 UTC m=+0.652656114 container died f09892f0f6c4933ae186e9368bc96a7cdabe2527b6c18e7c1062387392704e3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-25T14:17:48.674 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:48 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:48.393+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:17:48.674 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:48 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:48.393+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:17:48.674 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:48 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:48.393+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.789654+0000 front 2024-04-25T14:17:02.789638+0000 (oldest deadline 2024-04-25T14:17:28.089414+0000) 2024-04-25T14:17:48.674 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:48 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:48.393+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:10.990439+0000 front 2024-04-25T14:17:10.990451+0000 (oldest deadline 2024-04-25T14:17:35.090422+0000) 2024-04-25T14:17:48.674 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:48 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:48.393+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:19.191383+0000 front 2024-04-25T14:17:19.191428+0000 (oldest deadline 2024-04-25T14:17:45.091518+0000) 2024-04-25T14:17:49.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:48 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:48.746+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:49.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:48 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:48.746+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.795847+0000 front 2024-04-25T14:16:57.795573+0000 (oldest deadline 2024-04-25T14:17:23.095608+0000) 2024-04-25T14:17:49.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:48 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:48.746+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:03.096238+0000 front 2024-04-25T14:17:03.096401+0000 (oldest deadline 2024-04-25T14:17:27.796160+0000) 2024-04-25T14:17:49.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:48 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:48.746+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:12.997607+0000 front 2024-04-25T14:17:12.997700+0000 (oldest deadline 2024-04-25T14:17:36.497730+0000) 2024-04-25T14:17:49.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:48 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:48.746+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:21.699312+0000 front 2024-04-25T14:17:21.699430+0000 (oldest deadline 2024-04-25T14:17:46.399249+0000) 2024-04-25T14:17:49.188 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:48 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:48.948+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:53.756445+0000 front 2024-04-25T14:16:53.756264+0000 (oldest deadline 2024-04-25T14:17:21.956954+0000) 2024-04-25T14:17:49.188 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:48 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:48.948+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:04.759056+0000 front 2024-04-25T14:17:04.759106+0000 (oldest deadline 2024-04-25T14:17:29.459176+0000) 2024-04-25T14:17:49.188 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:48 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:48.948+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:11.760463+0000 front 2024-04-25T14:17:11.760420+0000 (oldest deadline 2024-04-25T14:17:35.860288+0000) 2024-04-25T14:17:49.188 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:48 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:48.948+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:19.961379+0000 front 2024-04-25T14:17:19.961555+0000 (oldest deadline 2024-04-25T14:17:42.861401+0000) 2024-04-25T14:17:49.189 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:48 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:48.874+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:49.189 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:48 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:48.874+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:17:49.189 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:48 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:48.874+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:17:49.189 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:48 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:48.874+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.372555+0000 front 2024-04-25T14:17:13.372596+0000 (oldest deadline 2024-04-25T14:17:37.472628+0000) 2024-04-25T14:17:49.189 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:48 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:48.874+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:20.373797+0000 front 2024-04-25T14:17:20.373887+0000 (oldest deadline 2024-04-25T14:17:44.473980+0000) 2024-04-25T14:17:49.218 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@osd.7.service' 2024-04-25T14:17:49.255 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:17:49.255 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-25T14:17:49.256 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-25T14:17:49.256 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-59334182-030c-11ef-bc93-c7b262605968@osd.8 2024-04-25T14:17:49.569 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:49 smithi151 systemd[1]: Stopping Ceph osd.8 for 59334182-030c-11ef-bc93-c7b262605968... 2024-04-25T14:17:49.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:49 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:49.438+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:17:49.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:49 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:49.438+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:17:49.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:49 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:49.438+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.789654+0000 front 2024-04-25T14:17:02.789638+0000 (oldest deadline 2024-04-25T14:17:28.089414+0000) 2024-04-25T14:17:49.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:49 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:49.438+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:10.990439+0000 front 2024-04-25T14:17:10.990451+0000 (oldest deadline 2024-04-25T14:17:35.090422+0000) 2024-04-25T14:17:49.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:49 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:49.438+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:19.191383+0000 front 2024-04-25T14:17:19.191428+0000 (oldest deadline 2024-04-25T14:17:45.091518+0000) 2024-04-25T14:17:49.828 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:49 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:49.568+0000 7f81ad15f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:17:49.828 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:49 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:49.568+0000 7f81ad15f700 -1 osd.8 81 *** Got signal Terminated *** 2024-04-25T14:17:49.828 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:49 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:49.568+0000 7f81ad15f700 -1 osd.8 81 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:17:50.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:49 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:49.757+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:50.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:49 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:49.757+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.795847+0000 front 2024-04-25T14:16:57.795573+0000 (oldest deadline 2024-04-25T14:17:23.095608+0000) 2024-04-25T14:17:50.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:49 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:49.757+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:03.096238+0000 front 2024-04-25T14:17:03.096401+0000 (oldest deadline 2024-04-25T14:17:27.796160+0000) 2024-04-25T14:17:50.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:49 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:49.757+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:12.997607+0000 front 2024-04-25T14:17:12.997700+0000 (oldest deadline 2024-04-25T14:17:36.497730+0000) 2024-04-25T14:17:50.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:49 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:49.757+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:21.699312+0000 front 2024-04-25T14:17:21.699430+0000 (oldest deadline 2024-04-25T14:17:46.399249+0000) 2024-04-25T14:17:50.188 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:49 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:49.908+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:53.756445+0000 front 2024-04-25T14:16:53.756264+0000 (oldest deadline 2024-04-25T14:17:21.956954+0000) 2024-04-25T14:17:50.188 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:49 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:49.908+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:04.759056+0000 front 2024-04-25T14:17:04.759106+0000 (oldest deadline 2024-04-25T14:17:29.459176+0000) 2024-04-25T14:17:50.188 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:49 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:49.908+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:11.760463+0000 front 2024-04-25T14:17:11.760420+0000 (oldest deadline 2024-04-25T14:17:35.860288+0000) 2024-04-25T14:17:50.188 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:49 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:49.908+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:19.961379+0000 front 2024-04-25T14:17:19.961555+0000 (oldest deadline 2024-04-25T14:17:42.861401+0000) 2024-04-25T14:17:50.188 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:49 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:49.827+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:50.189 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:49 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:49.827+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:17:50.189 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:49 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:49.827+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:17:50.189 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:49 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:49.827+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.372555+0000 front 2024-04-25T14:17:13.372596+0000 (oldest deadline 2024-04-25T14:17:37.472628+0000) 2024-04-25T14:17:50.189 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:49 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:49.827+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:20.373797+0000 front 2024-04-25T14:17:20.373887+0000 (oldest deadline 2024-04-25T14:17:44.473980+0000) 2024-04-25T14:17:50.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:50 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:50.454+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:17:50.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:50 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:50.454+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:17:50.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:50 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:50.454+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.789654+0000 front 2024-04-25T14:17:02.789638+0000 (oldest deadline 2024-04-25T14:17:28.089414+0000) 2024-04-25T14:17:50.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:50 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:50.454+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:10.990439+0000 front 2024-04-25T14:17:10.990451+0000 (oldest deadline 2024-04-25T14:17:35.090422+0000) 2024-04-25T14:17:50.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:50 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:50.454+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:19.191383+0000 front 2024-04-25T14:17:19.191428+0000 (oldest deadline 2024-04-25T14:17:45.091518+0000) 2024-04-25T14:17:51.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:50 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:50.747+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:51.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:50 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:50.747+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.795847+0000 front 2024-04-25T14:16:57.795573+0000 (oldest deadline 2024-04-25T14:17:23.095608+0000) 2024-04-25T14:17:51.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:50 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:50.747+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:03.096238+0000 front 2024-04-25T14:17:03.096401+0000 (oldest deadline 2024-04-25T14:17:27.796160+0000) 2024-04-25T14:17:51.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:50 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:50.747+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:12.997607+0000 front 2024-04-25T14:17:12.997700+0000 (oldest deadline 2024-04-25T14:17:36.497730+0000) 2024-04-25T14:17:51.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:50 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:50.747+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:21.699312+0000 front 2024-04-25T14:17:21.699430+0000 (oldest deadline 2024-04-25T14:17:46.399249+0000) 2024-04-25T14:17:51.188 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:50 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:50.866+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:53.756445+0000 front 2024-04-25T14:16:53.756264+0000 (oldest deadline 2024-04-25T14:17:21.956954+0000) 2024-04-25T14:17:51.188 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:50 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:50.866+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:04.759056+0000 front 2024-04-25T14:17:04.759106+0000 (oldest deadline 2024-04-25T14:17:29.459176+0000) 2024-04-25T14:17:51.188 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:50 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:50.866+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:11.760463+0000 front 2024-04-25T14:17:11.760420+0000 (oldest deadline 2024-04-25T14:17:35.860288+0000) 2024-04-25T14:17:51.188 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:50 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:50.866+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:19.961379+0000 front 2024-04-25T14:17:19.961555+0000 (oldest deadline 2024-04-25T14:17:42.861401+0000) 2024-04-25T14:17:51.189 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:50 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:50.863+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:51.189 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:50 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:50.863+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:17:51.189 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:50 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:50.863+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:17:51.189 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:50 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:50.863+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.372555+0000 front 2024-04-25T14:17:13.372596+0000 (oldest deadline 2024-04-25T14:17:37.472628+0000) 2024-04-25T14:17:51.189 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:50 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:50.863+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:20.373797+0000 front 2024-04-25T14:17:20.373887+0000 (oldest deadline 2024-04-25T14:17:44.473980+0000) 2024-04-25T14:17:51.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:51 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:51.458+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:17:51.924 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:51 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:51.458+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:17:51.924 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:51 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:51.458+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.789654+0000 front 2024-04-25T14:17:02.789638+0000 (oldest deadline 2024-04-25T14:17:28.089414+0000) 2024-04-25T14:17:51.924 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:51 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:51.458+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:10.990439+0000 front 2024-04-25T14:17:10.990451+0000 (oldest deadline 2024-04-25T14:17:35.090422+0000) 2024-04-25T14:17:51.924 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:51 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:51.458+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:19.191383+0000 front 2024-04-25T14:17:19.191428+0000 (oldest deadline 2024-04-25T14:17:45.091518+0000) 2024-04-25T14:17:52.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:51 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:51.761+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:52.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:51 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:51.761+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.795847+0000 front 2024-04-25T14:16:57.795573+0000 (oldest deadline 2024-04-25T14:17:23.095608+0000) 2024-04-25T14:17:52.110 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:51 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:51.761+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:03.096238+0000 front 2024-04-25T14:17:03.096401+0000 (oldest deadline 2024-04-25T14:17:27.796160+0000) 2024-04-25T14:17:52.110 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:51 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:51.761+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:12.997607+0000 front 2024-04-25T14:17:12.997700+0000 (oldest deadline 2024-04-25T14:17:36.497730+0000) 2024-04-25T14:17:52.110 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:51 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:51.761+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:21.699312+0000 front 2024-04-25T14:17:21.699430+0000 (oldest deadline 2024-04-25T14:17:46.399249+0000) 2024-04-25T14:17:52.110 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:51 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:51.761+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:26.399830+0000 front 2024-04-25T14:17:26.399933+0000 (oldest deadline 2024-04-25T14:17:51.099730+0000) 2024-04-25T14:17:52.188 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:51 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:51.860+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:53.756445+0000 front 2024-04-25T14:16:53.756264+0000 (oldest deadline 2024-04-25T14:17:21.956954+0000) 2024-04-25T14:17:52.188 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:51 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:51.860+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:04.759056+0000 front 2024-04-25T14:17:04.759106+0000 (oldest deadline 2024-04-25T14:17:29.459176+0000) 2024-04-25T14:17:52.189 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:51 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:51.860+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:11.760463+0000 front 2024-04-25T14:17:11.760420+0000 (oldest deadline 2024-04-25T14:17:35.860288+0000) 2024-04-25T14:17:52.189 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:51 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:51.860+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:19.961379+0000 front 2024-04-25T14:17:19.961555+0000 (oldest deadline 2024-04-25T14:17:42.861401+0000) 2024-04-25T14:17:52.189 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:51 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:51.895+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:52.189 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:51 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:51.895+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:17:52.189 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:51 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:51.896+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:17:52.189 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:51 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:51.896+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.372555+0000 front 2024-04-25T14:17:13.372596+0000 (oldest deadline 2024-04-25T14:17:37.472628+0000) 2024-04-25T14:17:52.189 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:51 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:51.896+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:20.373797+0000 front 2024-04-25T14:17:20.373887+0000 (oldest deadline 2024-04-25T14:17:44.473980+0000) 2024-04-25T14:17:52.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:52 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:52.428+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:17:52.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:52 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:52.428+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:17:52.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:52 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:52.428+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.789654+0000 front 2024-04-25T14:17:02.789638+0000 (oldest deadline 2024-04-25T14:17:28.089414+0000) 2024-04-25T14:17:52.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:52 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:52.428+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:10.990439+0000 front 2024-04-25T14:17:10.990451+0000 (oldest deadline 2024-04-25T14:17:35.090422+0000) 2024-04-25T14:17:52.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:52 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:52.428+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:19.191383+0000 front 2024-04-25T14:17:19.191428+0000 (oldest deadline 2024-04-25T14:17:45.091518+0000) 2024-04-25T14:17:52.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:52 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:52.428+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:25.092157+0000 front 2024-04-25T14:17:25.092215+0000 (oldest deadline 2024-04-25T14:17:51.492610+0000) 2024-04-25T14:17:53.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:52 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:52.772+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:53.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:52 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:52.772+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.795847+0000 front 2024-04-25T14:16:57.795573+0000 (oldest deadline 2024-04-25T14:17:23.095608+0000) 2024-04-25T14:17:53.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:52 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:52.772+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:03.096238+0000 front 2024-04-25T14:17:03.096401+0000 (oldest deadline 2024-04-25T14:17:27.796160+0000) 2024-04-25T14:17:53.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:52 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:52.772+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:12.997607+0000 front 2024-04-25T14:17:12.997700+0000 (oldest deadline 2024-04-25T14:17:36.497730+0000) 2024-04-25T14:17:53.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:52 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:52.772+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:21.699312+0000 front 2024-04-25T14:17:21.699430+0000 (oldest deadline 2024-04-25T14:17:46.399249+0000) 2024-04-25T14:17:53.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:52 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:52.772+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:26.399830+0000 front 2024-04-25T14:17:26.399933+0000 (oldest deadline 2024-04-25T14:17:51.099730+0000) 2024-04-25T14:17:53.188 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:52 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:52.902+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:53.756445+0000 front 2024-04-25T14:16:53.756264+0000 (oldest deadline 2024-04-25T14:17:21.956954+0000) 2024-04-25T14:17:53.188 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:52 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:52.902+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:04.759056+0000 front 2024-04-25T14:17:04.759106+0000 (oldest deadline 2024-04-25T14:17:29.459176+0000) 2024-04-25T14:17:53.188 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:52 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:52.902+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:11.760463+0000 front 2024-04-25T14:17:11.760420+0000 (oldest deadline 2024-04-25T14:17:35.860288+0000) 2024-04-25T14:17:53.188 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:52 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:52.902+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:19.961379+0000 front 2024-04-25T14:17:19.961555+0000 (oldest deadline 2024-04-25T14:17:42.861401+0000) 2024-04-25T14:17:53.188 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:52 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:52.902+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:29.262838+0000 front 2024-04-25T14:17:29.262894+0000 (oldest deadline 2024-04-25T14:17:52.162940+0000) 2024-04-25T14:17:53.189 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:52 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:52.901+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:53.189 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:52 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:52.901+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:17:53.189 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:52 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:52.901+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:17:53.189 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:52 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:52.901+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.372555+0000 front 2024-04-25T14:17:13.372596+0000 (oldest deadline 2024-04-25T14:17:37.472628+0000) 2024-04-25T14:17:53.189 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:52 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:52.901+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:20.373797+0000 front 2024-04-25T14:17:20.373887+0000 (oldest deadline 2024-04-25T14:17:44.473980+0000) 2024-04-25T14:17:53.409 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:53 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:53.408+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:17:53.409 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:53 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:53.408+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:17:53.410 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:53 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:53.408+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.789654+0000 front 2024-04-25T14:17:02.789638+0000 (oldest deadline 2024-04-25T14:17:28.089414+0000) 2024-04-25T14:17:53.410 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:53 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:53.408+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:10.990439+0000 front 2024-04-25T14:17:10.990451+0000 (oldest deadline 2024-04-25T14:17:35.090422+0000) 2024-04-25T14:17:53.410 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:53 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:53.408+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:19.191383+0000 front 2024-04-25T14:17:19.191428+0000 (oldest deadline 2024-04-25T14:17:45.091518+0000) 2024-04-25T14:17:53.410 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:53 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:53.408+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:25.092157+0000 front 2024-04-25T14:17:25.092215+0000 (oldest deadline 2024-04-25T14:17:51.492610+0000) 2024-04-25T14:17:54.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:53 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:53.738+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:54.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:53 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:53.738+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.795847+0000 front 2024-04-25T14:16:57.795573+0000 (oldest deadline 2024-04-25T14:17:23.095608+0000) 2024-04-25T14:17:54.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:53 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:53.738+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:03.096238+0000 front 2024-04-25T14:17:03.096401+0000 (oldest deadline 2024-04-25T14:17:27.796160+0000) 2024-04-25T14:17:54.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:53 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:53.738+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:12.997607+0000 front 2024-04-25T14:17:12.997700+0000 (oldest deadline 2024-04-25T14:17:36.497730+0000) 2024-04-25T14:17:54.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:53 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:53.738+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:21.699312+0000 front 2024-04-25T14:17:21.699430+0000 (oldest deadline 2024-04-25T14:17:46.399249+0000) 2024-04-25T14:17:54.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:53 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:53.738+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:26.399830+0000 front 2024-04-25T14:17:26.399933+0000 (oldest deadline 2024-04-25T14:17:51.099730+0000) 2024-04-25T14:17:54.438 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:53 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:53.952+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:53.756445+0000 front 2024-04-25T14:16:53.756264+0000 (oldest deadline 2024-04-25T14:17:21.956954+0000) 2024-04-25T14:17:54.438 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:53 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:53.952+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:04.759056+0000 front 2024-04-25T14:17:04.759106+0000 (oldest deadline 2024-04-25T14:17:29.459176+0000) 2024-04-25T14:17:54.438 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:53 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:53.952+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:11.760463+0000 front 2024-04-25T14:17:11.760420+0000 (oldest deadline 2024-04-25T14:17:35.860288+0000) 2024-04-25T14:17:54.438 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:53 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:53.952+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:19.961379+0000 front 2024-04-25T14:17:19.961555+0000 (oldest deadline 2024-04-25T14:17:42.861401+0000) 2024-04-25T14:17:54.438 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:53 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8[51738]: 2024-04-25T14:17:53.952+0000 7f81a570d700 -1 osd.8 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:29.262838+0000 front 2024-04-25T14:17:29.262894+0000 (oldest deadline 2024-04-25T14:17:52.162940+0000) 2024-04-25T14:17:54.439 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:53 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:53.951+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:54.439 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:53 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:53.951+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:17:54.439 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:53 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:53.951+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:17:54.439 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:53 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:53.951+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.372555+0000 front 2024-04-25T14:17:13.372596+0000 (oldest deadline 2024-04-25T14:17:37.472628+0000) 2024-04-25T14:17:54.439 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:53 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:53.951+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:20.373797+0000 front 2024-04-25T14:17:20.373887+0000 (oldest deadline 2024-04-25T14:17:44.473980+0000) 2024-04-25T14:17:54.439 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:53 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:53.951+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:24.474506+0000 front 2024-04-25T14:17:24.474476+0000 (oldest deadline 2024-04-25T14:17:53.875124+0000) 2024-04-25T14:17:54.922 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:54 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:54.434+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:17:54.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:54 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:54.434+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:17:54.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:54 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:54.434+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.789654+0000 front 2024-04-25T14:17:02.789638+0000 (oldest deadline 2024-04-25T14:17:28.089414+0000) 2024-04-25T14:17:54.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:54 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:54.434+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:10.990439+0000 front 2024-04-25T14:17:10.990451+0000 (oldest deadline 2024-04-25T14:17:35.090422+0000) 2024-04-25T14:17:54.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:54 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:54.434+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:19.191383+0000 front 2024-04-25T14:17:19.191428+0000 (oldest deadline 2024-04-25T14:17:45.091518+0000) 2024-04-25T14:17:54.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:54 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:54.434+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:25.092157+0000 front 2024-04-25T14:17:25.092215+0000 (oldest deadline 2024-04-25T14:17:51.492610+0000) 2024-04-25T14:17:55.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:54 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:54.781+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:55.110 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:54 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:54.781+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.795847+0000 front 2024-04-25T14:16:57.795573+0000 (oldest deadline 2024-04-25T14:17:23.095608+0000) 2024-04-25T14:17:55.110 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:54 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:54.781+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:03.096238+0000 front 2024-04-25T14:17:03.096401+0000 (oldest deadline 2024-04-25T14:17:27.796160+0000) 2024-04-25T14:17:55.110 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:54 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:54.781+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:12.997607+0000 front 2024-04-25T14:17:12.997700+0000 (oldest deadline 2024-04-25T14:17:36.497730+0000) 2024-04-25T14:17:55.110 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:54 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:54.781+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:21.699312+0000 front 2024-04-25T14:17:21.699430+0000 (oldest deadline 2024-04-25T14:17:46.399249+0000) 2024-04-25T14:17:55.110 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:54 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:54.781+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:26.399830+0000 front 2024-04-25T14:17:26.399933+0000 (oldest deadline 2024-04-25T14:17:51.099730+0000) 2024-04-25T14:17:55.188 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:55 smithi151 podman[67568]: 2024-04-25 14:17:55.092504748 +0000 UTC m=+5.647473326 container died 9b0962026659037ffad9580f10b38f381c84cf5ef979671beaf37417fbf5d5a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T14:17:55.188 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:54 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:54.937+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:55.188 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:54 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:54.937+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:17:55.188 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:54 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:54.937+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:17:55.188 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:54 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:54.937+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.372555+0000 front 2024-04-25T14:17:13.372596+0000 (oldest deadline 2024-04-25T14:17:37.472628+0000) 2024-04-25T14:17:55.188 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:54 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:54.937+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:20.373797+0000 front 2024-04-25T14:17:20.373887+0000 (oldest deadline 2024-04-25T14:17:44.473980+0000) 2024-04-25T14:17:55.188 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:54 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:54.937+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:24.474506+0000 front 2024-04-25T14:17:24.474476+0000 (oldest deadline 2024-04-25T14:17:53.875124+0000) 2024-04-25T14:17:55.688 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:55 smithi151 podman[67568]: 2024-04-25 14:17:55.471056178 +0000 UTC m=+6.026024762 container cleanup 9b0962026659037ffad9580f10b38f381c84cf5ef979671beaf37417fbf5d5a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-25T14:17:55.688 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:55 smithi151 bash[67568]: ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8 2024-04-25T14:17:55.922 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:55 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:55.461+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:17:55.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:55 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:55.461+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:17:55.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:55 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:55.461+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.789654+0000 front 2024-04-25T14:17:02.789638+0000 (oldest deadline 2024-04-25T14:17:28.089414+0000) 2024-04-25T14:17:55.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:55 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:55.461+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:10.990439+0000 front 2024-04-25T14:17:10.990451+0000 (oldest deadline 2024-04-25T14:17:35.090422+0000) 2024-04-25T14:17:55.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:55 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:55.461+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:19.191383+0000 front 2024-04-25T14:17:19.191428+0000 (oldest deadline 2024-04-25T14:17:45.091518+0000) 2024-04-25T14:17:55.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:55 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:55.461+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:25.092157+0000 front 2024-04-25T14:17:25.092215+0000 (oldest deadline 2024-04-25T14:17:51.492610+0000) 2024-04-25T14:17:56.093 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:55 smithi151 podman[67586]: 2024-04-25 14:17:55.7313535 +0000 UTC m=+0.629626578 container remove 9b0962026659037ffad9580f10b38f381c84cf5ef979671beaf37417fbf5d5a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T14:17:56.093 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:55 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:55.919+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:56.093 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:55 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:55.919+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:17:56.093 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:55 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:55.919+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:17:56.093 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:55 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:55.919+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.372555+0000 front 2024-04-25T14:17:13.372596+0000 (oldest deadline 2024-04-25T14:17:37.472628+0000) 2024-04-25T14:17:56.093 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:55 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:55.919+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:20.373797+0000 front 2024-04-25T14:17:20.373887+0000 (oldest deadline 2024-04-25T14:17:44.473980+0000) 2024-04-25T14:17:56.093 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:55 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:55.919+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:24.474506+0000 front 2024-04-25T14:17:24.474476+0000 (oldest deadline 2024-04-25T14:17:53.875124+0000) 2024-04-25T14:17:56.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:55 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:55.734+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:56.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:55 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:55.734+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.795847+0000 front 2024-04-25T14:16:57.795573+0000 (oldest deadline 2024-04-25T14:17:23.095608+0000) 2024-04-25T14:17:56.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:55 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:55.734+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:03.096238+0000 front 2024-04-25T14:17:03.096401+0000 (oldest deadline 2024-04-25T14:17:27.796160+0000) 2024-04-25T14:17:56.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:55 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:55.734+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:12.997607+0000 front 2024-04-25T14:17:12.997700+0000 (oldest deadline 2024-04-25T14:17:36.497730+0000) 2024-04-25T14:17:56.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:55 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:55.734+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:21.699312+0000 front 2024-04-25T14:17:21.699430+0000 (oldest deadline 2024-04-25T14:17:46.399249+0000) 2024-04-25T14:17:56.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:55 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:55.734+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:26.399830+0000 front 2024-04-25T14:17:26.399933+0000 (oldest deadline 2024-04-25T14:17:51.099730+0000) 2024-04-25T14:17:56.376 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:56 smithi151 podman[67676]: 2024-04-25 14:17:55.996589031 +0000 UTC m=+0.018552139 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:17:56.376 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:56 smithi151 podman[67676]: 2024-04-25 14:17:56.099117403 +0000 UTC m=+0.121080530 container create 9bbfb197bd5f151bc9cd78d3106ad79103b7023f65c3b5edb942cd5bede92da1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T14:17:56.377 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:56 smithi151 podman[67676]: 2024-04-25 14:17:56.377316769 +0000 UTC m=+0.399279864 container init 9bbfb197bd5f151bc9cd78d3106ad79103b7023f65c3b5edb942cd5bede92da1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-25T14:17:56.670 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:56 smithi151 podman[67676]: 2024-04-25 14:17:56.380743858 +0000 UTC m=+0.402706943 container start 9bbfb197bd5f151bc9cd78d3106ad79103b7023f65c3b5edb942cd5bede92da1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0) 2024-04-25T14:17:56.670 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:56 smithi151 podman[67676]: 2024-04-25 14:17:56.469488919 +0000 UTC m=+0.491452006 container attach 9bbfb197bd5f151bc9cd78d3106ad79103b7023f65c3b5edb942cd5bede92da1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:17:56.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:56 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:56.438+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:17:56.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:56 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:56.438+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:17:56.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:56 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:56.438+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.789654+0000 front 2024-04-25T14:17:02.789638+0000 (oldest deadline 2024-04-25T14:17:28.089414+0000) 2024-04-25T14:17:56.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:56 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:56.438+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:10.990439+0000 front 2024-04-25T14:17:10.990451+0000 (oldest deadline 2024-04-25T14:17:35.090422+0000) 2024-04-25T14:17:56.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:56 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:56.438+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:19.191383+0000 front 2024-04-25T14:17:19.191428+0000 (oldest deadline 2024-04-25T14:17:45.091518+0000) 2024-04-25T14:17:56.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:56 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:56.438+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:25.092157+0000 front 2024-04-25T14:17:25.092215+0000 (oldest deadline 2024-04-25T14:17:51.492610+0000) 2024-04-25T14:17:56.938 INFO:journalctl@ceph.osd.8.smithi151.stdout:Apr 25 14:17:56 smithi151 podman[67676]: 2024-04-25 14:17:56.671028982 +0000 UTC m=+0.692992066 container died 9bbfb197bd5f151bc9cd78d3106ad79103b7023f65c3b5edb942cd5bede92da1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:17:56.938 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:56 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:56.905+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:56.938 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:56 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:56.905+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:17:56.938 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:56 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:56.905+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:17:56.938 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:56 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:56.905+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.372555+0000 front 2024-04-25T14:17:13.372596+0000 (oldest deadline 2024-04-25T14:17:37.472628+0000) 2024-04-25T14:17:56.938 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:56 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:56.905+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:20.373797+0000 front 2024-04-25T14:17:20.373887+0000 (oldest deadline 2024-04-25T14:17:44.473980+0000) 2024-04-25T14:17:56.938 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:56 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:56.905+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:24.474506+0000 front 2024-04-25T14:17:24.474476+0000 (oldest deadline 2024-04-25T14:17:53.875124+0000) 2024-04-25T14:17:57.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:56 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:56.770+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:57.110 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:56 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:56.770+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.795847+0000 front 2024-04-25T14:16:57.795573+0000 (oldest deadline 2024-04-25T14:17:23.095608+0000) 2024-04-25T14:17:57.110 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:56 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:56.770+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:03.096238+0000 front 2024-04-25T14:17:03.096401+0000 (oldest deadline 2024-04-25T14:17:27.796160+0000) 2024-04-25T14:17:57.110 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:56 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:56.770+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:12.997607+0000 front 2024-04-25T14:17:12.997700+0000 (oldest deadline 2024-04-25T14:17:36.497730+0000) 2024-04-25T14:17:57.110 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:56 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:56.770+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:21.699312+0000 front 2024-04-25T14:17:21.699430+0000 (oldest deadline 2024-04-25T14:17:46.399249+0000) 2024-04-25T14:17:57.110 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:56 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:56.770+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:26.399830+0000 front 2024-04-25T14:17:26.399933+0000 (oldest deadline 2024-04-25T14:17:51.099730+0000) 2024-04-25T14:17:57.401 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@osd.8.service' 2024-04-25T14:17:57.446 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:17:57.446 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-25T14:17:57.447 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-25T14:17:57.447 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-59334182-030c-11ef-bc93-c7b262605968@osd.9 2024-04-25T14:17:57.729 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:57 smithi096 systemd[1]: Stopping Ceph osd.9 for 59334182-030c-11ef-bc93-c7b262605968... 2024-04-25T14:17:57.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:57 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:57.488+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:17:57.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:57 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:57.488+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:17:57.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:57 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:57.488+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.789654+0000 front 2024-04-25T14:17:02.789638+0000 (oldest deadline 2024-04-25T14:17:28.089414+0000) 2024-04-25T14:17:57.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:57 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:57.488+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:10.990439+0000 front 2024-04-25T14:17:10.990451+0000 (oldest deadline 2024-04-25T14:17:35.090422+0000) 2024-04-25T14:17:57.924 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:57 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:57.488+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:19.191383+0000 front 2024-04-25T14:17:19.191428+0000 (oldest deadline 2024-04-25T14:17:45.091518+0000) 2024-04-25T14:17:57.924 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:57 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:57.488+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:25.092157+0000 front 2024-04-25T14:17:25.092215+0000 (oldest deadline 2024-04-25T14:17:51.492610+0000) 2024-04-25T14:17:58.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:57 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:57.729+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:58.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:57 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:57.729+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.795847+0000 front 2024-04-25T14:16:57.795573+0000 (oldest deadline 2024-04-25T14:17:23.095608+0000) 2024-04-25T14:17:58.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:57 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:57.729+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:03.096238+0000 front 2024-04-25T14:17:03.096401+0000 (oldest deadline 2024-04-25T14:17:27.796160+0000) 2024-04-25T14:17:58.110 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:57 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:57.729+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:12.997607+0000 front 2024-04-25T14:17:12.997700+0000 (oldest deadline 2024-04-25T14:17:36.497730+0000) 2024-04-25T14:17:58.110 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:57 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:57.729+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:21.699312+0000 front 2024-04-25T14:17:21.699430+0000 (oldest deadline 2024-04-25T14:17:46.399249+0000) 2024-04-25T14:17:58.110 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:57 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:57.729+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:26.399830+0000 front 2024-04-25T14:17:26.399933+0000 (oldest deadline 2024-04-25T14:17:51.099730+0000) 2024-04-25T14:17:58.110 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:57 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:57.756+0000 7f316e800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:17:58.110 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:57 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:57.756+0000 7f316e800700 -1 osd.9 81 *** Got signal Terminated *** 2024-04-25T14:17:58.110 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:57 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:57.756+0000 7f316e800700 -1 osd.9 81 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:17:58.188 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:57 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:57.916+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:58.188 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:57 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:57.916+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:17:58.189 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:57 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:57.916+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:17:58.189 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:57 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:57.916+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.372555+0000 front 2024-04-25T14:17:13.372596+0000 (oldest deadline 2024-04-25T14:17:37.472628+0000) 2024-04-25T14:17:58.189 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:57 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:57.916+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:20.373797+0000 front 2024-04-25T14:17:20.373887+0000 (oldest deadline 2024-04-25T14:17:44.473980+0000) 2024-04-25T14:17:58.189 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:57 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:57.916+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:24.474506+0000 front 2024-04-25T14:17:24.474476+0000 (oldest deadline 2024-04-25T14:17:53.875124+0000) 2024-04-25T14:17:58.922 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:58 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:58.504+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:17:58.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:58 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:58.504+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:17:58.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:58 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:58.504+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.789654+0000 front 2024-04-25T14:17:02.789638+0000 (oldest deadline 2024-04-25T14:17:28.089414+0000) 2024-04-25T14:17:58.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:58 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:58.504+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:10.990439+0000 front 2024-04-25T14:17:10.990451+0000 (oldest deadline 2024-04-25T14:17:35.090422+0000) 2024-04-25T14:17:58.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:58 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:58.504+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:19.191383+0000 front 2024-04-25T14:17:19.191428+0000 (oldest deadline 2024-04-25T14:17:45.091518+0000) 2024-04-25T14:17:58.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:58 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:58.504+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:25.092157+0000 front 2024-04-25T14:17:25.092215+0000 (oldest deadline 2024-04-25T14:17:51.492610+0000) 2024-04-25T14:17:59.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:58 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:58.769+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:17:59.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:58 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:58.769+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.795847+0000 front 2024-04-25T14:16:57.795573+0000 (oldest deadline 2024-04-25T14:17:23.095608+0000) 2024-04-25T14:17:59.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:58 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:58.769+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:03.096238+0000 front 2024-04-25T14:17:03.096401+0000 (oldest deadline 2024-04-25T14:17:27.796160+0000) 2024-04-25T14:17:59.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:58 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:58.769+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:12.997607+0000 front 2024-04-25T14:17:12.997700+0000 (oldest deadline 2024-04-25T14:17:36.497730+0000) 2024-04-25T14:17:59.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:58 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:58.769+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:21.699312+0000 front 2024-04-25T14:17:21.699430+0000 (oldest deadline 2024-04-25T14:17:46.399249+0000) 2024-04-25T14:17:59.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:58 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:58.769+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:26.399830+0000 front 2024-04-25T14:17:26.399933+0000 (oldest deadline 2024-04-25T14:17:51.099730+0000) 2024-04-25T14:17:59.110 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:58 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:58.769+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-04-25T14:17:32.800927+0000 front 2024-04-25T14:17:32.800893+0000 (oldest deadline 2024-04-25T14:17:58.700953+0000) 2024-04-25T14:17:59.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:58 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:58.954+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:17:59.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:58 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:58.954+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:17:59.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:58 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:58.954+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:17:59.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:58 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:58.954+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.372555+0000 front 2024-04-25T14:17:13.372596+0000 (oldest deadline 2024-04-25T14:17:37.472628+0000) 2024-04-25T14:17:59.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:58 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:58.954+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:20.373797+0000 front 2024-04-25T14:17:20.373887+0000 (oldest deadline 2024-04-25T14:17:44.473980+0000) 2024-04-25T14:17:59.439 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:58 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:58.954+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:24.474506+0000 front 2024-04-25T14:17:24.474476+0000 (oldest deadline 2024-04-25T14:17:53.875124+0000) 2024-04-25T14:17:59.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:59 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:59.548+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:17:59.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:59 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:59.548+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:17:59.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:59 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:59.548+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.789654+0000 front 2024-04-25T14:17:02.789638+0000 (oldest deadline 2024-04-25T14:17:28.089414+0000) 2024-04-25T14:17:59.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:59 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:59.548+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:10.990439+0000 front 2024-04-25T14:17:10.990451+0000 (oldest deadline 2024-04-25T14:17:35.090422+0000) 2024-04-25T14:17:59.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:59 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:59.548+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:19.191383+0000 front 2024-04-25T14:17:19.191428+0000 (oldest deadline 2024-04-25T14:17:45.091518+0000) 2024-04-25T14:17:59.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:59 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:59.548+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:25.092157+0000 front 2024-04-25T14:17:25.092215+0000 (oldest deadline 2024-04-25T14:17:51.492610+0000) 2024-04-25T14:17:59.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:17:59 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:17:59.548+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-04-25T14:17:35.895142+0000 front 2024-04-25T14:17:35.895160+0000 (oldest deadline 2024-04-25T14:17:58.794895+0000) 2024-04-25T14:18:00.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:59 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:59.722+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:18:00.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:59 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:59.722+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.795847+0000 front 2024-04-25T14:16:57.795573+0000 (oldest deadline 2024-04-25T14:17:23.095608+0000) 2024-04-25T14:18:00.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:59 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:59.722+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:03.096238+0000 front 2024-04-25T14:17:03.096401+0000 (oldest deadline 2024-04-25T14:17:27.796160+0000) 2024-04-25T14:18:00.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:59 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:59.722+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:12.997607+0000 front 2024-04-25T14:17:12.997700+0000 (oldest deadline 2024-04-25T14:17:36.497730+0000) 2024-04-25T14:18:00.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:59 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:59.722+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:21.699312+0000 front 2024-04-25T14:17:21.699430+0000 (oldest deadline 2024-04-25T14:17:46.399249+0000) 2024-04-25T14:18:00.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:59 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:59.722+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:26.399830+0000 front 2024-04-25T14:17:26.399933+0000 (oldest deadline 2024-04-25T14:17:51.099730+0000) 2024-04-25T14:18:00.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:17:59 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:17:59.722+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-04-25T14:17:32.800927+0000 front 2024-04-25T14:17:32.800893+0000 (oldest deadline 2024-04-25T14:17:58.700953+0000) 2024-04-25T14:18:00.188 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:59 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:59.935+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:18:00.188 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:59 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:59.935+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:18:00.188 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:59 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:59.935+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:18:00.188 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:59 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:59.935+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.372555+0000 front 2024-04-25T14:17:13.372596+0000 (oldest deadline 2024-04-25T14:17:37.472628+0000) 2024-04-25T14:18:00.188 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:59 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:59.935+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:20.373797+0000 front 2024-04-25T14:17:20.373887+0000 (oldest deadline 2024-04-25T14:17:44.473980+0000) 2024-04-25T14:18:00.189 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:17:59 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:17:59.935+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:24.474506+0000 front 2024-04-25T14:17:24.474476+0000 (oldest deadline 2024-04-25T14:17:53.875124+0000) 2024-04-25T14:18:00.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:00 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:00.579+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:18:00.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:00 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:00.579+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:18:00.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:00 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:00.579+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.789654+0000 front 2024-04-25T14:17:02.789638+0000 (oldest deadline 2024-04-25T14:17:28.089414+0000) 2024-04-25T14:18:00.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:00 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:00.579+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:10.990439+0000 front 2024-04-25T14:17:10.990451+0000 (oldest deadline 2024-04-25T14:17:35.090422+0000) 2024-04-25T14:18:00.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:00 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:00.579+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:19.191383+0000 front 2024-04-25T14:17:19.191428+0000 (oldest deadline 2024-04-25T14:17:45.091518+0000) 2024-04-25T14:18:00.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:00 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:00.579+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:25.092157+0000 front 2024-04-25T14:17:25.092215+0000 (oldest deadline 2024-04-25T14:17:51.492610+0000) 2024-04-25T14:18:00.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:00 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:00.579+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-04-25T14:17:35.895142+0000 front 2024-04-25T14:17:35.895160+0000 (oldest deadline 2024-04-25T14:17:58.794895+0000) 2024-04-25T14:18:01.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:18:00 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:18:00.735+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:18:01.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:18:00 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:18:00.735+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.795847+0000 front 2024-04-25T14:16:57.795573+0000 (oldest deadline 2024-04-25T14:17:23.095608+0000) 2024-04-25T14:18:01.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:18:00 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:18:00.735+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:03.096238+0000 front 2024-04-25T14:17:03.096401+0000 (oldest deadline 2024-04-25T14:17:27.796160+0000) 2024-04-25T14:18:01.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:18:00 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:18:00.735+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:12.997607+0000 front 2024-04-25T14:17:12.997700+0000 (oldest deadline 2024-04-25T14:17:36.497730+0000) 2024-04-25T14:18:01.110 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:18:00 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:18:00.735+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:21.699312+0000 front 2024-04-25T14:17:21.699430+0000 (oldest deadline 2024-04-25T14:17:46.399249+0000) 2024-04-25T14:18:01.110 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:18:00 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:18:00.735+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:26.399830+0000 front 2024-04-25T14:17:26.399933+0000 (oldest deadline 2024-04-25T14:17:51.099730+0000) 2024-04-25T14:18:01.110 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:18:00 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:18:00.735+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-04-25T14:17:32.800927+0000 front 2024-04-25T14:17:32.800893+0000 (oldest deadline 2024-04-25T14:17:58.700953+0000) 2024-04-25T14:18:01.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:00 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:00.941+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:18:01.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:00 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:00.941+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:18:01.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:00 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:00.941+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:18:01.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:00 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:00.941+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.372555+0000 front 2024-04-25T14:17:13.372596+0000 (oldest deadline 2024-04-25T14:17:37.472628+0000) 2024-04-25T14:18:01.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:00 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:00.941+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:20.373797+0000 front 2024-04-25T14:17:20.373887+0000 (oldest deadline 2024-04-25T14:17:44.473980+0000) 2024-04-25T14:18:01.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:00 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:00.941+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:24.474506+0000 front 2024-04-25T14:17:24.474476+0000 (oldest deadline 2024-04-25T14:17:53.875124+0000) 2024-04-25T14:18:01.922 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:01 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:01.590+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:18:01.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:01 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:01.590+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:18:01.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:01 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:01.590+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.789654+0000 front 2024-04-25T14:17:02.789638+0000 (oldest deadline 2024-04-25T14:17:28.089414+0000) 2024-04-25T14:18:01.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:01 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:01.590+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:10.990439+0000 front 2024-04-25T14:17:10.990451+0000 (oldest deadline 2024-04-25T14:17:35.090422+0000) 2024-04-25T14:18:01.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:01 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:01.590+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:19.191383+0000 front 2024-04-25T14:17:19.191428+0000 (oldest deadline 2024-04-25T14:17:45.091518+0000) 2024-04-25T14:18:01.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:01 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:01.590+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:25.092157+0000 front 2024-04-25T14:17:25.092215+0000 (oldest deadline 2024-04-25T14:17:51.492610+0000) 2024-04-25T14:18:01.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:01 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:01.590+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-04-25T14:17:35.895142+0000 front 2024-04-25T14:17:35.895160+0000 (oldest deadline 2024-04-25T14:17:58.794895+0000) 2024-04-25T14:18:02.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:18:01 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:18:01.738+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:18:02.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:18:01 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:18:01.738+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.795847+0000 front 2024-04-25T14:16:57.795573+0000 (oldest deadline 2024-04-25T14:17:23.095608+0000) 2024-04-25T14:18:02.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:18:01 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:18:01.738+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:03.096238+0000 front 2024-04-25T14:17:03.096401+0000 (oldest deadline 2024-04-25T14:17:27.796160+0000) 2024-04-25T14:18:02.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:18:01 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:18:01.738+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:12.997607+0000 front 2024-04-25T14:17:12.997700+0000 (oldest deadline 2024-04-25T14:17:36.497730+0000) 2024-04-25T14:18:02.110 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:18:01 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:18:01.738+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:21.699312+0000 front 2024-04-25T14:17:21.699430+0000 (oldest deadline 2024-04-25T14:17:46.399249+0000) 2024-04-25T14:18:02.110 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:18:01 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:18:01.738+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:26.399830+0000 front 2024-04-25T14:17:26.399933+0000 (oldest deadline 2024-04-25T14:17:51.099730+0000) 2024-04-25T14:18:02.110 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:18:01 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:18:01.738+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-04-25T14:17:32.800927+0000 front 2024-04-25T14:17:32.800893+0000 (oldest deadline 2024-04-25T14:17:58.700953+0000) 2024-04-25T14:18:02.439 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:01 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:01.973+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:18:02.439 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:01 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:01.973+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:18:02.439 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:01 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:01.973+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:18:02.439 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:01 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:01.973+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.372555+0000 front 2024-04-25T14:17:13.372596+0000 (oldest deadline 2024-04-25T14:17:37.472628+0000) 2024-04-25T14:18:02.440 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:01 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:01.973+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:20.373797+0000 front 2024-04-25T14:17:20.373887+0000 (oldest deadline 2024-04-25T14:17:44.473980+0000) 2024-04-25T14:18:02.440 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:01 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:01.973+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:24.474506+0000 front 2024-04-25T14:17:24.474476+0000 (oldest deadline 2024-04-25T14:17:53.875124+0000) 2024-04-25T14:18:02.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:02 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:02.547+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:18:02.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:02 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:02.547+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:18:02.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:02 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:02.547+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.789654+0000 front 2024-04-25T14:17:02.789638+0000 (oldest deadline 2024-04-25T14:17:28.089414+0000) 2024-04-25T14:18:02.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:02 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:02.547+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:10.990439+0000 front 2024-04-25T14:17:10.990451+0000 (oldest deadline 2024-04-25T14:17:35.090422+0000) 2024-04-25T14:18:02.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:02 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:02.547+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:19.191383+0000 front 2024-04-25T14:17:19.191428+0000 (oldest deadline 2024-04-25T14:17:45.091518+0000) 2024-04-25T14:18:02.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:02 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:02.547+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:25.092157+0000 front 2024-04-25T14:17:25.092215+0000 (oldest deadline 2024-04-25T14:17:51.492610+0000) 2024-04-25T14:18:02.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:02 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:02.547+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-04-25T14:17:35.895142+0000 front 2024-04-25T14:17:35.895160+0000 (oldest deadline 2024-04-25T14:17:58.794895+0000) 2024-04-25T14:18:03.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:18:02 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:18:02.700+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:49.193167+0000 front 2024-04-25T14:16:49.193135+0000 (oldest deadline 2024-04-25T14:17:10.893329+0000) 2024-04-25T14:18:03.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:18:02 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:18:02.700+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:57.795847+0000 front 2024-04-25T14:16:57.795573+0000 (oldest deadline 2024-04-25T14:17:23.095608+0000) 2024-04-25T14:18:03.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:18:02 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:18:02.700+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:03.096238+0000 front 2024-04-25T14:17:03.096401+0000 (oldest deadline 2024-04-25T14:17:27.796160+0000) 2024-04-25T14:18:03.109 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:18:02 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:18:02.700+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:12.997607+0000 front 2024-04-25T14:17:12.997700+0000 (oldest deadline 2024-04-25T14:17:36.497730+0000) 2024-04-25T14:18:03.110 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:18:02 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:18:02.700+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:21.699312+0000 front 2024-04-25T14:17:21.699430+0000 (oldest deadline 2024-04-25T14:17:46.399249+0000) 2024-04-25T14:18:03.110 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:18:02 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:18:02.700+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:26.399830+0000 front 2024-04-25T14:17:26.399933+0000 (oldest deadline 2024-04-25T14:17:51.099730+0000) 2024-04-25T14:18:03.110 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:18:02 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9[64246]: 2024-04-25T14:18:02.700+0000 7f316adb6700 -1 osd.9 81 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-04-25T14:17:32.800927+0000 front 2024-04-25T14:17:32.800893+0000 (oldest deadline 2024-04-25T14:17:58.700953+0000) 2024-04-25T14:18:03.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:03 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:03.006+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:18:03.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:03 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:03.006+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:18:03.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:03 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:03.006+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:18:03.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:03 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:03.006+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.372555+0000 front 2024-04-25T14:17:13.372596+0000 (oldest deadline 2024-04-25T14:17:37.472628+0000) 2024-04-25T14:18:03.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:03 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:03.006+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:20.373797+0000 front 2024-04-25T14:17:20.373887+0000 (oldest deadline 2024-04-25T14:17:44.473980+0000) 2024-04-25T14:18:03.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:03 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:03.006+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:24.474506+0000 front 2024-04-25T14:17:24.474476+0000 (oldest deadline 2024-04-25T14:17:53.875124+0000) 2024-04-25T14:18:03.530 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:18:03 smithi096 podman[88972]: 2024-04-25 14:18:03.281201786 +0000 UTC m=+5.642003312 container died f57826b9403ea9cab6193221f00df6d6c820e54ec8551d5e90b5273385e0e620 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-25T14:18:03.859 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:18:03 smithi096 podman[88972]: 2024-04-25 14:18:03.714042585 +0000 UTC m=+6.074844156 container cleanup f57826b9403ea9cab6193221f00df6d6c820e54ec8551d5e90b5273385e0e620 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T14:18:03.859 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:18:03 smithi096 bash[88972]: ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9 2024-04-25T14:18:03.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:03 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:03.531+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:18:03.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:03 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:03.531+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:18:03.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:03 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:03.531+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.789654+0000 front 2024-04-25T14:17:02.789638+0000 (oldest deadline 2024-04-25T14:17:28.089414+0000) 2024-04-25T14:18:03.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:03 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:03.531+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:10.990439+0000 front 2024-04-25T14:17:10.990451+0000 (oldest deadline 2024-04-25T14:17:35.090422+0000) 2024-04-25T14:18:03.924 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:03 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:03.531+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:19.191383+0000 front 2024-04-25T14:17:19.191428+0000 (oldest deadline 2024-04-25T14:17:45.091518+0000) 2024-04-25T14:18:03.924 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:03 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:03.531+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:25.092157+0000 front 2024-04-25T14:17:25.092215+0000 (oldest deadline 2024-04-25T14:17:51.492610+0000) 2024-04-25T14:18:03.924 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:03 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:03.531+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-04-25T14:17:35.895142+0000 front 2024-04-25T14:17:35.895160+0000 (oldest deadline 2024-04-25T14:17:58.794895+0000) 2024-04-25T14:18:04.326 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:18:03 smithi096 podman[88989]: 2024-04-25 14:18:03.915462134 +0000 UTC m=+0.633792220 container remove f57826b9403ea9cab6193221f00df6d6c820e54ec8551d5e90b5273385e0e620 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T14:18:04.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:04 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:04.010+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:18:04.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:04 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:04.010+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:18:04.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:04 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:04.010+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:18:04.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:04 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:04.010+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.372555+0000 front 2024-04-25T14:17:13.372596+0000 (oldest deadline 2024-04-25T14:17:37.472628+0000) 2024-04-25T14:18:04.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:04 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:04.010+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:20.373797+0000 front 2024-04-25T14:17:20.373887+0000 (oldest deadline 2024-04-25T14:17:44.473980+0000) 2024-04-25T14:18:04.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:04 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:04.010+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:24.474506+0000 front 2024-04-25T14:17:24.474476+0000 (oldest deadline 2024-04-25T14:17:53.875124+0000) 2024-04-25T14:18:04.578 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:18:04 smithi096 podman[89078]: 2024-04-25 14:18:04.22944433 +0000 UTC m=+0.018136882 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:18:04.579 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:18:04 smithi096 podman[89078]: 2024-04-25 14:18:04.349344707 +0000 UTC m=+0.138037245 container create 6f37864e5320886362168566de3072a6dc5b60120b20ea2143a21693e82a0c8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T14:18:04.579 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:18:04 smithi096 podman[89078]: 2024-04-25 14:18:04.576650436 +0000 UTC m=+0.365342975 container init 6f37864e5320886362168566de3072a6dc5b60120b20ea2143a21693e82a0c8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T14:18:04.579 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:18:04 smithi096 podman[89078]: 2024-04-25 14:18:04.579844766 +0000 UTC m=+0.368537308 container start 6f37864e5320886362168566de3072a6dc5b60120b20ea2143a21693e82a0c8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-25T14:18:04.859 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:18:04 smithi096 podman[89078]: 2024-04-25 14:18:04.685205311 +0000 UTC m=+0.473897873 container attach 6f37864e5320886362168566de3072a6dc5b60120b20ea2143a21693e82a0c8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-25T14:18:04.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:04 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:04.557+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:18:04.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:04 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:04.557+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:18:04.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:04 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:04.557+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.789654+0000 front 2024-04-25T14:17:02.789638+0000 (oldest deadline 2024-04-25T14:17:28.089414+0000) 2024-04-25T14:18:04.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:04 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:04.557+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:10.990439+0000 front 2024-04-25T14:17:10.990451+0000 (oldest deadline 2024-04-25T14:17:35.090422+0000) 2024-04-25T14:18:04.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:04 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:04.557+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:19.191383+0000 front 2024-04-25T14:17:19.191428+0000 (oldest deadline 2024-04-25T14:17:45.091518+0000) 2024-04-25T14:18:04.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:04 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:04.557+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:25.092157+0000 front 2024-04-25T14:17:25.092215+0000 (oldest deadline 2024-04-25T14:17:51.492610+0000) 2024-04-25T14:18:04.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:04 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:04.557+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-04-25T14:17:35.895142+0000 front 2024-04-25T14:17:35.895160+0000 (oldest deadline 2024-04-25T14:17:58.794895+0000) 2024-04-25T14:18:05.164 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 14:18:04 smithi096 podman[89078]: 2024-04-25 14:18:04.870688898 +0000 UTC m=+0.659381435 container died 6f37864e5320886362168566de3072a6dc5b60120b20ea2143a21693e82a0c8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-25T14:18:05.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:05 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:05.003+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:18:05.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:05 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:05.003+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:18:05.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:05 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:05.003+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:18:05.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:05 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:05.003+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.372555+0000 front 2024-04-25T14:17:13.372596+0000 (oldest deadline 2024-04-25T14:17:37.472628+0000) 2024-04-25T14:18:05.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:05 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:05.003+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:20.373797+0000 front 2024-04-25T14:17:20.373887+0000 (oldest deadline 2024-04-25T14:17:44.473980+0000) 2024-04-25T14:18:05.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:05 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:05.003+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:24.474506+0000 front 2024-04-25T14:17:24.474476+0000 (oldest deadline 2024-04-25T14:17:53.875124+0000) 2024-04-25T14:18:05.593 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@osd.9.service' 2024-04-25T14:18:05.632 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:18:05.632 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-25T14:18:05.632 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-25T14:18:05.633 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-59334182-030c-11ef-bc93-c7b262605968@osd.10 2024-04-25T14:18:05.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:05 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:05.603+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:18:05.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:05 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:05.603+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:18:05.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:05 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:05.603+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.789654+0000 front 2024-04-25T14:17:02.789638+0000 (oldest deadline 2024-04-25T14:17:28.089414+0000) 2024-04-25T14:18:05.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:05 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:05.603+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:10.990439+0000 front 2024-04-25T14:17:10.990451+0000 (oldest deadline 2024-04-25T14:17:35.090422+0000) 2024-04-25T14:18:05.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:05 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:05.603+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:19.191383+0000 front 2024-04-25T14:17:19.191428+0000 (oldest deadline 2024-04-25T14:17:45.091518+0000) 2024-04-25T14:18:05.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:05 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:05.603+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:25.092157+0000 front 2024-04-25T14:17:25.092215+0000 (oldest deadline 2024-04-25T14:17:51.492610+0000) 2024-04-25T14:18:05.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:05 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:05.603+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-04-25T14:17:35.895142+0000 front 2024-04-25T14:17:35.895160+0000 (oldest deadline 2024-04-25T14:17:58.794895+0000) 2024-04-25T14:18:05.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:05 smithi097 systemd[1]: Stopping Ceph osd.10 for 59334182-030c-11ef-bc93-c7b262605968... 2024-04-25T14:18:06.339 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:06 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:06.009+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:18:06.339 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:06 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:06.009+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:18:06.340 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:06 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:06.009+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:18:06.340 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:06 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:06.009+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.372555+0000 front 2024-04-25T14:17:13.372596+0000 (oldest deadline 2024-04-25T14:17:37.472628+0000) 2024-04-25T14:18:06.340 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:06 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:06.009+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:20.373797+0000 front 2024-04-25T14:17:20.373887+0000 (oldest deadline 2024-04-25T14:17:44.473980+0000) 2024-04-25T14:18:06.340 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:06 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:06.009+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:24.474506+0000 front 2024-04-25T14:17:24.474476+0000 (oldest deadline 2024-04-25T14:17:53.875124+0000) 2024-04-25T14:18:06.422 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:05 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:05.937+0000 7fe9b66a5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:18:06.423 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:05 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:05.937+0000 7fe9b66a5700 -1 osd.10 81 *** Got signal Terminated *** 2024-04-25T14:18:06.423 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:05 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:05.937+0000 7fe9b66a5700 -1 osd.10 81 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:18:06.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:06 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:06.612+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:18:06.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:06 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:06.612+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:18:06.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:06 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:06.612+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.789654+0000 front 2024-04-25T14:17:02.789638+0000 (oldest deadline 2024-04-25T14:17:28.089414+0000) 2024-04-25T14:18:06.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:06 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:06.612+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:10.990439+0000 front 2024-04-25T14:17:10.990451+0000 (oldest deadline 2024-04-25T14:17:35.090422+0000) 2024-04-25T14:18:06.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:06 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:06.612+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:19.191383+0000 front 2024-04-25T14:17:19.191428+0000 (oldest deadline 2024-04-25T14:17:45.091518+0000) 2024-04-25T14:18:06.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:06 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:06.612+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:25.092157+0000 front 2024-04-25T14:17:25.092215+0000 (oldest deadline 2024-04-25T14:17:51.492610+0000) 2024-04-25T14:18:06.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:06 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:06.612+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-04-25T14:17:35.895142+0000 front 2024-04-25T14:17:35.895160+0000 (oldest deadline 2024-04-25T14:17:58.794895+0000) 2024-04-25T14:18:07.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:06 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:06.982+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:18:07.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:06 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:06.982+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:18:07.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:06 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:06.982+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:18:07.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:06 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:06.982+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.372555+0000 front 2024-04-25T14:17:13.372596+0000 (oldest deadline 2024-04-25T14:17:37.472628+0000) 2024-04-25T14:18:07.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:06 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:06.982+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:20.373797+0000 front 2024-04-25T14:17:20.373887+0000 (oldest deadline 2024-04-25T14:17:44.473980+0000) 2024-04-25T14:18:07.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:06 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:06.982+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:24.474506+0000 front 2024-04-25T14:17:24.474476+0000 (oldest deadline 2024-04-25T14:17:53.875124+0000) 2024-04-25T14:18:07.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:07 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:07.588+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:18:07.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:07 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:07.588+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:18:07.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:07 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:07.588+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.789654+0000 front 2024-04-25T14:17:02.789638+0000 (oldest deadline 2024-04-25T14:17:28.089414+0000) 2024-04-25T14:18:07.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:07 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:07.588+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:10.990439+0000 front 2024-04-25T14:17:10.990451+0000 (oldest deadline 2024-04-25T14:17:35.090422+0000) 2024-04-25T14:18:07.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:07 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:07.588+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:19.191383+0000 front 2024-04-25T14:17:19.191428+0000 (oldest deadline 2024-04-25T14:17:45.091518+0000) 2024-04-25T14:18:07.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:07 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:07.588+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:25.092157+0000 front 2024-04-25T14:17:25.092215+0000 (oldest deadline 2024-04-25T14:17:51.492610+0000) 2024-04-25T14:18:07.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:07 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:07.588+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-04-25T14:17:35.895142+0000 front 2024-04-25T14:17:35.895160+0000 (oldest deadline 2024-04-25T14:17:58.794895+0000) 2024-04-25T14:18:08.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:08 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:08.015+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:18:08.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:08 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:08.015+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:18:08.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:08 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:08.015+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:18:08.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:08 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:08.015+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.372555+0000 front 2024-04-25T14:17:13.372596+0000 (oldest deadline 2024-04-25T14:17:37.472628+0000) 2024-04-25T14:18:08.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:08 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:08.015+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:20.373797+0000 front 2024-04-25T14:17:20.373887+0000 (oldest deadline 2024-04-25T14:17:44.473980+0000) 2024-04-25T14:18:08.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:08 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:08.015+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:24.474506+0000 front 2024-04-25T14:17:24.474476+0000 (oldest deadline 2024-04-25T14:17:53.875124+0000) 2024-04-25T14:18:08.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:08 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:08.635+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:18:08.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:08 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:08.635+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:18:08.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:08 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:08.635+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.789654+0000 front 2024-04-25T14:17:02.789638+0000 (oldest deadline 2024-04-25T14:17:28.089414+0000) 2024-04-25T14:18:08.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:08 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:08.635+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:10.990439+0000 front 2024-04-25T14:17:10.990451+0000 (oldest deadline 2024-04-25T14:17:35.090422+0000) 2024-04-25T14:18:08.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:08 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:08.635+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:19.191383+0000 front 2024-04-25T14:17:19.191428+0000 (oldest deadline 2024-04-25T14:17:45.091518+0000) 2024-04-25T14:18:08.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:08 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:08.635+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:25.092157+0000 front 2024-04-25T14:17:25.092215+0000 (oldest deadline 2024-04-25T14:17:51.492610+0000) 2024-04-25T14:18:08.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:08 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:08.635+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-04-25T14:17:35.895142+0000 front 2024-04-25T14:17:35.895160+0000 (oldest deadline 2024-04-25T14:17:58.794895+0000) 2024-04-25T14:18:09.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:09 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:09.049+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:18:09.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:09 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:09.049+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:18:09.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:09 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:09.049+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:18:09.439 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:09 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:09.049+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.372555+0000 front 2024-04-25T14:17:13.372596+0000 (oldest deadline 2024-04-25T14:17:37.472628+0000) 2024-04-25T14:18:09.439 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:09 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:09.049+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:20.373797+0000 front 2024-04-25T14:17:20.373887+0000 (oldest deadline 2024-04-25T14:17:44.473980+0000) 2024-04-25T14:18:09.439 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:09 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:09.049+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:24.474506+0000 front 2024-04-25T14:17:24.474476+0000 (oldest deadline 2024-04-25T14:17:53.875124+0000) 2024-04-25T14:18:09.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:09 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:09.647+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:18:09.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:09 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:09.647+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:18:09.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:09 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:09.647+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.789654+0000 front 2024-04-25T14:17:02.789638+0000 (oldest deadline 2024-04-25T14:17:28.089414+0000) 2024-04-25T14:18:09.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:09 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:09.647+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:10.990439+0000 front 2024-04-25T14:17:10.990451+0000 (oldest deadline 2024-04-25T14:17:35.090422+0000) 2024-04-25T14:18:09.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:09 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:09.647+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:19.191383+0000 front 2024-04-25T14:17:19.191428+0000 (oldest deadline 2024-04-25T14:17:45.091518+0000) 2024-04-25T14:18:09.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:09 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:09.647+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:25.092157+0000 front 2024-04-25T14:17:25.092215+0000 (oldest deadline 2024-04-25T14:17:51.492610+0000) 2024-04-25T14:18:09.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:09 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:09.647+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-04-25T14:17:35.895142+0000 front 2024-04-25T14:17:35.895160+0000 (oldest deadline 2024-04-25T14:17:58.794895+0000) 2024-04-25T14:18:10.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:10 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:10.080+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:18:10.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:10 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:10.081+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:18:10.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:10 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:10.081+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:18:10.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:10 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:10.081+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.372555+0000 front 2024-04-25T14:17:13.372596+0000 (oldest deadline 2024-04-25T14:17:37.472628+0000) 2024-04-25T14:18:10.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:10 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:10.081+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:20.373797+0000 front 2024-04-25T14:17:20.373887+0000 (oldest deadline 2024-04-25T14:17:44.473980+0000) 2024-04-25T14:18:10.439 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:10 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:10.081+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:24.474506+0000 front 2024-04-25T14:17:24.474476+0000 (oldest deadline 2024-04-25T14:17:53.875124+0000) 2024-04-25T14:18:10.439 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:10 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:10.081+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2024-04-25T14:17:46.077817+0000 front 2024-04-25T14:17:46.077723+0000 (oldest deadline 2024-04-25T14:18:09.577491+0000) 2024-04-25T14:18:10.922 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:10 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:10.628+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:48.087867+0000 front 2024-04-25T14:16:48.087723+0000 (oldest deadline 2024-04-25T14:17:13.387810+0000) 2024-04-25T14:18:10.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:10 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:10.628+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:58.088757+0000 front 2024-04-25T14:16:58.088829+0000 (oldest deadline 2024-04-25T14:17:22.788951+0000) 2024-04-25T14:18:10.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:10 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:10.628+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.789654+0000 front 2024-04-25T14:17:02.789638+0000 (oldest deadline 2024-04-25T14:17:28.089414+0000) 2024-04-25T14:18:10.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:10 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:10.628+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:10.990439+0000 front 2024-04-25T14:17:10.990451+0000 (oldest deadline 2024-04-25T14:17:35.090422+0000) 2024-04-25T14:18:10.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:10 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:10.628+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:19.191383+0000 front 2024-04-25T14:17:19.191428+0000 (oldest deadline 2024-04-25T14:17:45.091518+0000) 2024-04-25T14:18:10.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:10 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:10.628+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:25.092157+0000 front 2024-04-25T14:17:25.092215+0000 (oldest deadline 2024-04-25T14:17:51.492610+0000) 2024-04-25T14:18:10.923 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:10 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:18:10.628+0000 7fe9aec53700 -1 osd.10 81 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-04-25T14:17:35.895142+0000 front 2024-04-25T14:17:35.895160+0000 (oldest deadline 2024-04-25T14:17:58.794895+0000) 2024-04-25T14:18:11.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:11 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:11.047+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:18:11.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:11 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:11.047+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:18:11.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:11 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:11.047+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:18:11.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:11 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:11.047+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.372555+0000 front 2024-04-25T14:17:13.372596+0000 (oldest deadline 2024-04-25T14:17:37.472628+0000) 2024-04-25T14:18:11.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:11 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:11.047+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:20.373797+0000 front 2024-04-25T14:17:20.373887+0000 (oldest deadline 2024-04-25T14:17:44.473980+0000) 2024-04-25T14:18:11.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:11 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:11.047+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:24.474506+0000 front 2024-04-25T14:17:24.474476+0000 (oldest deadline 2024-04-25T14:17:53.875124+0000) 2024-04-25T14:18:11.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:11 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:11.047+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2024-04-25T14:17:46.077817+0000 front 2024-04-25T14:17:46.077723+0000 (oldest deadline 2024-04-25T14:18:09.577491+0000) 2024-04-25T14:18:11.637 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:11 smithi097 podman[67101]: 2024-04-25 14:18:11.380086897 +0000 UTC m=+5.558468730 container died d292f35c49847adf389bb4c473019b8e4b4c3abca5244c5a904a899eb4fb77f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-25T14:18:11.904 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:11 smithi097 podman[67101]: 2024-04-25 14:18:11.720611649 +0000 UTC m=+5.898993539 container cleanup d292f35c49847adf389bb4c473019b8e4b4c3abca5244c5a904a899eb4fb77f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T14:18:11.904 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:11 smithi097 bash[67101]: ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10 2024-04-25T14:18:12.172 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:11 smithi097 podman[67118]: 2024-04-25 14:18:11.905231406 +0000 UTC m=+0.519777905 container remove d292f35c49847adf389bb4c473019b8e4b4c3abca5244c5a904a899eb4fb77f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True) 2024-04-25T14:18:12.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:12 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:12.021+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:18:12.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:12 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:12.021+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:18:12.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:12 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:12.021+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:18:12.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:12 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:12.021+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.372555+0000 front 2024-04-25T14:17:13.372596+0000 (oldest deadline 2024-04-25T14:17:37.472628+0000) 2024-04-25T14:18:12.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:12 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:12.021+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:20.373797+0000 front 2024-04-25T14:17:20.373887+0000 (oldest deadline 2024-04-25T14:17:44.473980+0000) 2024-04-25T14:18:12.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:12 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:12.021+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:24.474506+0000 front 2024-04-25T14:17:24.474476+0000 (oldest deadline 2024-04-25T14:17:53.875124+0000) 2024-04-25T14:18:12.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:12 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:12.021+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2024-04-25T14:17:46.077817+0000 front 2024-04-25T14:17:46.077723+0000 (oldest deadline 2024-04-25T14:18:09.577491+0000) 2024-04-25T14:18:12.624 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:12 smithi097 podman[67212]: 2024-04-25 14:18:12.339309172 +0000 UTC m=+0.114038778 container create baf6aa7822e4abe2766a9c900123a82b84b2929f9ea29cd1133153331e0dc317 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T14:18:12.624 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:12 smithi097 podman[67212]: 2024-04-25 14:18:12.243307962 +0000 UTC m=+0.018037553 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:18:12.624 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:12 smithi097 podman[67212]: 2024-04-25 14:18:12.541368586 +0000 UTC m=+0.316098150 container init baf6aa7822e4abe2766a9c900123a82b84b2929f9ea29cd1133153331e0dc317 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T14:18:12.624 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:12 smithi097 podman[67212]: 2024-04-25 14:18:12.544438133 +0000 UTC m=+0.319167701 container start baf6aa7822e4abe2766a9c900123a82b84b2929f9ea29cd1133153331e0dc317 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-25T14:18:12.624 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:12 smithi097 podman[67212]: 2024-04-25 14:18:12.624674429 +0000 UTC m=+0.399403996 container attach baf6aa7822e4abe2766a9c900123a82b84b2929f9ea29cd1133153331e0dc317 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux ) 2024-04-25T14:18:12.922 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:12 smithi097 podman[67212]: 2024-04-25 14:18:12.841023154 +0000 UTC m=+0.615752719 container died baf6aa7822e4abe2766a9c900123a82b84b2929f9ea29cd1133153331e0dc317 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-25T14:18:13.366 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@osd.10.service' 2024-04-25T14:18:13.389 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:13 smithi097 podman[67238]: 2024-04-25 14:18:13.336115758 +0000 UTC m=+0.489848149 container remove baf6aa7822e4abe2766a9c900123a82b84b2929f9ea29cd1133153331e0dc317 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux ) 2024-04-25T14:18:13.389 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:13 smithi097 systemd[1]: ceph-59334182-030c-11ef-bc93-c7b262605968@osd.10.service: Deactivated successfully. 2024-04-25T14:18:13.389 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:13 smithi097 systemd[1]: Stopped Ceph osd.10 for 59334182-030c-11ef-bc93-c7b262605968. 2024-04-25T14:18:13.389 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 14:18:13 smithi097 systemd[1]: ceph-59334182-030c-11ef-bc93-c7b262605968@osd.10.service: Consumed 10.206s CPU time. 2024-04-25T14:18:13.404 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:18:13.404 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-25T14:18:13.405 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-25T14:18:13.405 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-59334182-030c-11ef-bc93-c7b262605968@osd.11 2024-04-25T14:18:13.432 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:13 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:13.003+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:18:13.432 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:13 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:13.003+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:18:13.432 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:13 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:13.003+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:18:13.432 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:13 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:13.003+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.372555+0000 front 2024-04-25T14:17:13.372596+0000 (oldest deadline 2024-04-25T14:17:37.472628+0000) 2024-04-25T14:18:13.432 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:13 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:13.003+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:20.373797+0000 front 2024-04-25T14:17:20.373887+0000 (oldest deadline 2024-04-25T14:17:44.473980+0000) 2024-04-25T14:18:13.433 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:13 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:13.003+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:24.474506+0000 front 2024-04-25T14:17:24.474476+0000 (oldest deadline 2024-04-25T14:17:53.875124+0000) 2024-04-25T14:18:13.433 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:13 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:13.003+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2024-04-25T14:17:46.077817+0000 front 2024-04-25T14:17:46.077723+0000 (oldest deadline 2024-04-25T14:18:09.577491+0000) 2024-04-25T14:18:13.688 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:13 smithi151 systemd[1]: Stopping Ceph osd.11 for 59334182-030c-11ef-bc93-c7b262605968... 2024-04-25T14:18:14.037 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:13 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:13.720+0000 7f0cac05a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:18:14.038 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:13 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:13.720+0000 7f0cac05a700 -1 osd.11 81 *** Got signal Terminated *** 2024-04-25T14:18:14.038 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:13 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:13.720+0000 7f0cac05a700 -1 osd.11 81 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:18:14.038 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:14 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:14.037+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:18:14.038 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:14 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:14.037+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:18:14.038 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:14 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:14.037+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:18:14.038 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:14 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:14.037+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.372555+0000 front 2024-04-25T14:17:13.372596+0000 (oldest deadline 2024-04-25T14:17:37.472628+0000) 2024-04-25T14:18:14.038 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:14 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:14.037+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:20.373797+0000 front 2024-04-25T14:17:20.373887+0000 (oldest deadline 2024-04-25T14:17:44.473980+0000) 2024-04-25T14:18:14.038 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:14 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:14.037+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:24.474506+0000 front 2024-04-25T14:17:24.474476+0000 (oldest deadline 2024-04-25T14:17:53.875124+0000) 2024-04-25T14:18:14.038 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:14 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:14.037+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2024-04-25T14:17:46.077817+0000 front 2024-04-25T14:17:46.077723+0000 (oldest deadline 2024-04-25T14:18:09.577491+0000) 2024-04-25T14:18:15.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:15 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:15.025+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:18:15.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:15 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:15.025+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:18:15.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:15 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:15.025+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:18:15.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:15 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:15.025+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.372555+0000 front 2024-04-25T14:17:13.372596+0000 (oldest deadline 2024-04-25T14:17:37.472628+0000) 2024-04-25T14:18:15.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:15 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:15.025+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:20.373797+0000 front 2024-04-25T14:17:20.373887+0000 (oldest deadline 2024-04-25T14:17:44.473980+0000) 2024-04-25T14:18:15.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:15 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:15.025+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:24.474506+0000 front 2024-04-25T14:17:24.474476+0000 (oldest deadline 2024-04-25T14:17:53.875124+0000) 2024-04-25T14:18:15.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:15 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:15.025+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2024-04-25T14:17:46.077817+0000 front 2024-04-25T14:17:46.077723+0000 (oldest deadline 2024-04-25T14:18:09.577491+0000) 2024-04-25T14:18:16.391 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:16 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:16.028+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:18:16.391 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:16 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:16.028+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:18:16.391 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:16 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:16.028+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:18:16.391 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:16 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:16.028+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.372555+0000 front 2024-04-25T14:17:13.372596+0000 (oldest deadline 2024-04-25T14:17:37.472628+0000) 2024-04-25T14:18:16.392 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:16 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:16.028+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:20.373797+0000 front 2024-04-25T14:17:20.373887+0000 (oldest deadline 2024-04-25T14:17:44.473980+0000) 2024-04-25T14:18:16.392 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:16 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:16.028+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:24.474506+0000 front 2024-04-25T14:17:24.474476+0000 (oldest deadline 2024-04-25T14:17:53.875124+0000) 2024-04-25T14:18:16.392 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:16 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:16.028+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2024-04-25T14:17:46.077817+0000 front 2024-04-25T14:17:46.077723+0000 (oldest deadline 2024-04-25T14:18:09.577491+0000) 2024-04-25T14:18:16.392 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:16 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:16.028+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6820 osd.8 since back 2024-04-25T14:17:54.079228+0000 front 2024-04-25T14:17:54.079188+0000 (oldest deadline 2024-04-25T14:18:15.179354+0000) 2024-04-25T14:18:17.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:16 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:16.984+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:18:17.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:16 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:16.984+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:18:17.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:16 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:16.984+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:18:17.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:16 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:16.984+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.372555+0000 front 2024-04-25T14:17:13.372596+0000 (oldest deadline 2024-04-25T14:17:37.472628+0000) 2024-04-25T14:18:17.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:16 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:16.984+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:20.373797+0000 front 2024-04-25T14:17:20.373887+0000 (oldest deadline 2024-04-25T14:17:44.473980+0000) 2024-04-25T14:18:17.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:16 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:16.984+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:24.474506+0000 front 2024-04-25T14:17:24.474476+0000 (oldest deadline 2024-04-25T14:17:53.875124+0000) 2024-04-25T14:18:17.439 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:16 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:16.984+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2024-04-25T14:17:46.077817+0000 front 2024-04-25T14:17:46.077723+0000 (oldest deadline 2024-04-25T14:18:09.577491+0000) 2024-04-25T14:18:17.439 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:16 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:16.984+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6820 osd.8 since back 2024-04-25T14:17:54.079228+0000 front 2024-04-25T14:17:54.079188+0000 (oldest deadline 2024-04-25T14:18:15.179354+0000) 2024-04-25T14:18:18.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:17 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:17.953+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T14:16:47.767370+0000 front 2024-04-25T14:16:47.767425+0000 (oldest deadline 2024-04-25T14:17:10.667485+0000) 2024-04-25T14:18:18.438 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:17 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:17.953+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T14:16:55.968615+0000 front 2024-04-25T14:16:55.968792+0000 (oldest deadline 2024-04-25T14:17:18.868805+0000) 2024-04-25T14:18:18.439 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:17 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:17.953+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-04-25T14:17:02.970033+0000 front 2024-04-25T14:17:02.970355+0000 (oldest deadline 2024-04-25T14:17:28.870187+0000) 2024-04-25T14:18:18.439 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:17 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:17.953+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T14:17:13.372555+0000 front 2024-04-25T14:17:13.372596+0000 (oldest deadline 2024-04-25T14:17:37.472628+0000) 2024-04-25T14:18:18.439 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:17 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:17.953+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T14:17:20.373797+0000 front 2024-04-25T14:17:20.373887+0000 (oldest deadline 2024-04-25T14:17:44.473980+0000) 2024-04-25T14:18:18.439 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:17 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:17.953+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-04-25T14:17:24.474506+0000 front 2024-04-25T14:17:24.474476+0000 (oldest deadline 2024-04-25T14:17:53.875124+0000) 2024-04-25T14:18:18.439 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:17 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:17.953+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2024-04-25T14:17:46.077817+0000 front 2024-04-25T14:17:46.077723+0000 (oldest deadline 2024-04-25T14:18:09.577491+0000) 2024-04-25T14:18:18.439 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:17 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11[58333]: 2024-04-25T14:18:17.953+0000 7f0ca4608700 -1 osd.11 81 heartbeat_check: no reply from 172.21.15.151:6820 osd.8 since back 2024-04-25T14:17:54.079228+0000 front 2024-04-25T14:17:54.079188+0000 (oldest deadline 2024-04-25T14:18:15.179354+0000) 2024-04-25T14:18:19.313 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:19 smithi151 podman[67793]: 2024-04-25 14:18:19.042624329 +0000 UTC m=+5.440286856 container died 27e6574c58ea061c73cac65aedfed08f58700c4b93d3d6ae123ca33c4eb92a41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T14:18:19.680 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:19 smithi151 podman[67793]: 2024-04-25 14:18:19.404779531 +0000 UTC m=+5.802442056 container cleanup 27e6574c58ea061c73cac65aedfed08f58700c4b93d3d6ae123ca33c4eb92a41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-25T14:18:19.680 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:19 smithi151 bash[67793]: ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11 2024-04-25T14:18:19.938 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:19 smithi151 podman[67810]: 2024-04-25 14:18:19.681138207 +0000 UTC m=+0.629060991 container remove 27e6574c58ea061c73cac65aedfed08f58700c4b93d3d6ae123ca33c4eb92a41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-25T14:18:20.342 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:20 smithi151 podman[67900]: 2024-04-25 14:18:19.93068367 +0000 UTC m=+0.018555990 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:18:20.342 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:20 smithi151 podman[67900]: 2024-04-25 14:18:20.032513887 +0000 UTC m=+0.120386195 container create d983a9e956f4b6bf5a791f85eea98b55a1e0bbb7191c10a498b86ebb4c3a803a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T14:18:20.343 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:20 smithi151 podman[67900]: 2024-04-25 14:18:20.268608194 +0000 UTC m=+0.356480501 container init d983a9e956f4b6bf5a791f85eea98b55a1e0bbb7191c10a498b86ebb4c3a803a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11-deactivate, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T14:18:20.343 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:20 smithi151 podman[67900]: 2024-04-25 14:18:20.271891693 +0000 UTC m=+0.359764001 container start d983a9e956f4b6bf5a791f85eea98b55a1e0bbb7191c10a498b86ebb4c3a803a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-25T14:18:20.343 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:20 smithi151 podman[67900]: 2024-04-25 14:18:20.343657117 +0000 UTC m=+0.431529440 container attach d983a9e956f4b6bf5a791f85eea98b55a1e0bbb7191c10a498b86ebb4c3a803a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=) 2024-04-25T14:18:20.688 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:20 smithi151 podman[67900]: 2024-04-25 14:18:20.559339129 +0000 UTC m=+0.647211439 container died d983a9e956f4b6bf5a791f85eea98b55a1e0bbb7191c10a498b86ebb4c3a803a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=) 2024-04-25T14:18:21.162 INFO:journalctl@ceph.osd.11.smithi151.stdout:Apr 25 14:18:21 smithi151 podman[67927]: 2024-04-25 14:18:21.13967811 +0000 UTC m=+0.575065485 container remove d983a9e956f4b6bf5a791f85eea98b55a1e0bbb7191c10a498b86ebb4c3a803a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T14:18:21.167 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@osd.11.service' 2024-04-25T14:18:21.248 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:18:21.248 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-25T14:18:21.248 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-25T14:18:21.248 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-59334182-030c-11ef-bc93-c7b262605968@mds.a 2024-04-25T14:18:21.609 INFO:journalctl@ceph.mds.a.smithi096.stdout:Apr 25 14:18:21 smithi096 systemd[1]: Stopping Ceph mds.a for 59334182-030c-11ef-bc93-c7b262605968... 2024-04-25T14:18:22.109 INFO:journalctl@ceph.mds.a.smithi096.stdout:Apr 25 14:18:21 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-a[70807]: 2024-04-25T14:18:21.692+0000 7fd786164700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:18:22.109 INFO:journalctl@ceph.mds.a.smithi096.stdout:Apr 25 14:18:21 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-a[70807]: 2024-04-25T14:18:21.692+0000 7fd786164700 -1 mds.a *** got signal Terminated *** 2024-04-25T14:18:25.859 INFO:journalctl@ceph.mds.a.smithi096.stdout:Apr 25 14:18:25 smithi096 podman[89207]: 2024-04-25 14:18:25.555602217 +0000 UTC m=+4.106921586 container died e1547329698111f23a474a41d29b59bdda6d25df54de770f388d06fa0170009a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-25T14:18:26.359 INFO:journalctl@ceph.mds.a.smithi096.stdout:Apr 25 14:18:26 smithi096 podman[89207]: 2024-04-25 14:18:26.107785387 +0000 UTC m=+4.659104764 container cleanup e1547329698111f23a474a41d29b59bdda6d25df54de770f388d06fa0170009a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T14:18:26.359 INFO:journalctl@ceph.mds.a.smithi096.stdout:Apr 25 14:18:26 smithi096 bash[89207]: ceph-59334182-030c-11ef-bc93-c7b262605968-mds-a 2024-04-25T14:18:26.588 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@mds.a.service' 2024-04-25T14:18:26.622 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:18:26.623 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-25T14:18:26.623 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-25T14:18:26.623 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-59334182-030c-11ef-bc93-c7b262605968@mds.d 2024-04-25T14:18:26.936 INFO:journalctl@ceph.mds.d.smithi096.stdout:Apr 25 14:18:26 smithi096 systemd[1]: Stopping Ceph mds.d for 59334182-030c-11ef-bc93-c7b262605968... 2024-04-25T14:18:27.359 INFO:journalctl@ceph.mds.d.smithi096.stdout:Apr 25 14:18:26 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-d[69836]: 2024-04-25T14:18:26.936+0000 7f251b77e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:18:27.359 INFO:journalctl@ceph.mds.d.smithi096.stdout:Apr 25 14:18:26 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-d[69836]: 2024-04-25T14:18:26.936+0000 7f251b77e700 -1 mds.d *** got signal Terminated *** 2024-04-25T14:18:28.770 INFO:journalctl@ceph.mds.d.smithi096.stdout:Apr 25 14:18:28 smithi096 podman[89329]: 2024-04-25 14:18:28.491897751 +0000 UTC m=+1.658211784 container died d0538fbe5a3c8c5550cf44748fa7934415bd0a17bc6d59b6570d0a2455ecd7ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-d, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2) 2024-04-25T14:18:29.109 INFO:journalctl@ceph.mds.d.smithi096.stdout:Apr 25 14:18:28 smithi096 podman[89329]: 2024-04-25 14:18:28.953358208 +0000 UTC m=+2.119672282 container cleanup d0538fbe5a3c8c5550cf44748fa7934415bd0a17bc6d59b6570d0a2455ecd7ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2) 2024-04-25T14:18:29.109 INFO:journalctl@ceph.mds.d.smithi096.stdout:Apr 25 14:18:28 smithi096 bash[89329]: ceph-59334182-030c-11ef-bc93-c7b262605968-mds-d 2024-04-25T14:18:29.534 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@mds.d.service' 2024-04-25T14:18:29.567 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:18:29.567 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-25T14:18:29.567 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-25T14:18:29.567 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-59334182-030c-11ef-bc93-c7b262605968@mds.g 2024-04-25T14:18:29.859 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 14:18:29 smithi096 systemd[1]: Stopping Ceph mds.g for 59334182-030c-11ef-bc93-c7b262605968... 2024-04-25T14:18:30.359 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 14:18:29 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-g[70446]: 2024-04-25T14:18:29.882+0000 7f694c823700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:18:30.359 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 14:18:29 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-g[70446]: 2024-04-25T14:18:29.882+0000 7f694c823700 -1 mds.g *** got signal Terminated *** 2024-04-25T14:18:33.862 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 14:18:33 smithi096 podman[89449]: 2024-04-25 14:18:33.552384707 +0000 UTC m=+3.772677398 container died 64e700cd8c53a8d3a47eac8b74b8023b98714c98fe68eb031bc95d022a043d45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-g, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-25T14:18:34.190 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 14:18:33 smithi096 podman[89449]: 2024-04-25 14:18:33.948624287 +0000 UTC m=+4.168916976 container cleanup 64e700cd8c53a8d3a47eac8b74b8023b98714c98fe68eb031bc95d022a043d45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-g, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T14:18:34.190 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 14:18:33 smithi096 bash[89449]: ceph-59334182-030c-11ef-bc93-c7b262605968-mds-g 2024-04-25T14:18:34.299 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@mds.g.service' 2024-04-25T14:18:34.329 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:18:34.329 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-25T14:18:34.330 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-25T14:18:34.330 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-59334182-030c-11ef-bc93-c7b262605968@mds.j 2024-04-25T14:18:34.609 INFO:journalctl@ceph.mds.j.smithi096.stdout:Apr 25 14:18:34 smithi096 systemd[1]: Stopping Ceph mds.j for 59334182-030c-11ef-bc93-c7b262605968... 2024-04-25T14:18:35.109 INFO:journalctl@ceph.mds.j.smithi096.stdout:Apr 25 14:18:34 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-j[69465]: 2024-04-25T14:18:34.646+0000 7efcc22f5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:18:35.109 INFO:journalctl@ceph.mds.j.smithi096.stdout:Apr 25 14:18:34 smithi096 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-j[69465]: 2024-04-25T14:18:34.646+0000 7efcc22f5700 -1 mds.j *** got signal Terminated *** 2024-04-25T14:18:37.859 INFO:journalctl@ceph.mds.j.smithi096.stdout:Apr 25 14:18:37 smithi096 podman[89570]: 2024-04-25 14:18:37.578047181 +0000 UTC m=+3.034089160 container died dc649cef1a74dc258a315697e69864118ef09dcb85512ad037bfee8104d683ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-25T14:18:38.184 INFO:journalctl@ceph.mds.j.smithi096.stdout:Apr 25 14:18:37 smithi096 podman[89570]: 2024-04-25 14:18:37.951445696 +0000 UTC m=+3.407487672 container cleanup dc649cef1a74dc258a315697e69864118ef09dcb85512ad037bfee8104d683ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-j, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T14:18:38.184 INFO:journalctl@ceph.mds.j.smithi096.stdout:Apr 25 14:18:37 smithi096 bash[89570]: ceph-59334182-030c-11ef-bc93-c7b262605968-mds-j 2024-04-25T14:18:38.323 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@mds.j.service' 2024-04-25T14:18:38.354 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:18:38.354 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-25T14:18:38.354 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-25T14:18:38.355 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-59334182-030c-11ef-bc93-c7b262605968@mds.b 2024-04-25T14:18:38.662 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 14:18:38 smithi097 systemd[1]: Stopping Ceph mds.b for 59334182-030c-11ef-bc93-c7b262605968... 2024-04-25T14:18:38.922 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 14:18:38 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-b[63225]: 2024-04-25T14:18:38.662+0000 7faf2fdff700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:18:38.923 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 14:18:38 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-b[63225]: 2024-04-25T14:18:38.662+0000 7faf2fdff700 -1 mds.b *** got signal Terminated *** 2024-04-25T14:18:41.860 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 14:18:41 smithi097 podman[67329]: 2024-04-25 14:18:41.54978101 +0000 UTC m=+2.995685361 container died 7bfcbedc3a91b98f1c10f9f05c54b7ef27a35b358f7e486349d1d95a14157689 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-b, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T14:18:42.173 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 14:18:41 smithi097 podman[67329]: 2024-04-25 14:18:41.861643628 +0000 UTC m=+3.307547974 container cleanup 7bfcbedc3a91b98f1c10f9f05c54b7ef27a35b358f7e486349d1d95a14157689 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-25T14:18:42.173 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 14:18:41 smithi097 bash[67329]: ceph-59334182-030c-11ef-bc93-c7b262605968-mds-b 2024-04-25T14:18:42.173 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 14:18:42 smithi097 podman[67341]: 2024-04-25 14:18:42.062949875 +0000 UTC m=+0.509853149 container remove 7bfcbedc3a91b98f1c10f9f05c54b7ef27a35b358f7e486349d1d95a14157689 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T14:18:42.291 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@mds.b.service' 2024-04-25T14:18:42.324 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:18:42.325 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-25T14:18:42.325 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-25T14:18:42.325 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-59334182-030c-11ef-bc93-c7b262605968@mds.e 2024-04-25T14:18:42.648 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 14:18:42 smithi097 systemd[1]: Stopping Ceph mds.e for 59334182-030c-11ef-bc93-c7b262605968... 2024-04-25T14:18:42.922 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 14:18:42 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-e[62606]: 2024-04-25T14:18:42.648+0000 7fcc6d6f9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:18:42.922 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 14:18:42 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-e[62606]: 2024-04-25T14:18:42.648+0000 7fcc6d6f9700 -1 mds.e *** got signal Terminated *** 2024-04-25T14:18:45.832 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 14:18:45 smithi097 podman[67459]: 2024-04-25 14:18:45.564991285 +0000 UTC m=+3.019091301 container died 5fa8c1d3bf3a8a10c5088878b15e0e6543a21767e8966daf30ae437899525240 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5) 2024-04-25T14:18:46.172 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 14:18:45 smithi097 podman[67459]: 2024-04-25 14:18:45.923835099 +0000 UTC m=+3.377935185 container cleanup 5fa8c1d3bf3a8a10c5088878b15e0e6543a21767e8966daf30ae437899525240 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux ) 2024-04-25T14:18:46.173 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 14:18:45 smithi097 bash[67459]: ceph-59334182-030c-11ef-bc93-c7b262605968-mds-e 2024-04-25T14:18:46.306 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@mds.e.service' 2024-04-25T14:18:46.338 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:18:46.339 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-25T14:18:46.339 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-25T14:18:46.339 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-59334182-030c-11ef-bc93-c7b262605968@mds.h 2024-04-25T14:18:46.638 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 14:18:46 smithi097 systemd[1]: Stopping Ceph mds.h for 59334182-030c-11ef-bc93-c7b262605968... 2024-04-25T14:18:46.922 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 14:18:46 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-h[62915]: 2024-04-25T14:18:46.638+0000 7f97026f4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:18:46.923 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 14:18:46 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-h[62915]: 2024-04-25T14:18:46.638+0000 7f97026f4700 -1 mds.h *** got signal Terminated *** 2024-04-25T14:18:49.810 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 14:18:49 smithi097 podman[67580]: 2024-04-25 14:18:49.552741818 +0000 UTC m=+2.999983929 container died 84a82aacbfa9e08877fda56721a7dded8656c9062ebeaa51822061a3f6669d84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-h, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-25T14:18:50.069 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 14:18:49 smithi097 podman[67580]: 2024-04-25 14:18:49.885521217 +0000 UTC m=+3.332763322 container cleanup 84a82aacbfa9e08877fda56721a7dded8656c9062ebeaa51822061a3f6669d84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-h, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-25T14:18:50.069 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 14:18:49 smithi097 bash[67580]: ceph-59334182-030c-11ef-bc93-c7b262605968-mds-h 2024-04-25T14:18:50.286 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@mds.h.service' 2024-04-25T14:18:50.319 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:18:50.320 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-25T14:18:50.320 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-25T14:18:50.320 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-59334182-030c-11ef-bc93-c7b262605968@mds.k 2024-04-25T14:18:50.672 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 14:18:50 smithi097 systemd[1]: Stopping Ceph mds.k for 59334182-030c-11ef-bc93-c7b262605968... 2024-04-25T14:18:50.673 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 14:18:50 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-k[62290]: 2024-04-25T14:18:50.613+0000 7f2512558700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:18:50.673 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 14:18:50 smithi097 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-k[62290]: 2024-04-25T14:18:50.613+0000 7f2512558700 -1 mds.k *** got signal Terminated *** 2024-04-25T14:18:53.838 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 14:18:53 smithi097 podman[67700]: 2024-04-25 14:18:53.543520215 +0000 UTC m=+3.005503099 container died e7a56b736f0f6a390c5347dd97f87272ae8bc80e0e6e9ce608f5f7f2bcd35411 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-k, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-25T14:18:54.138 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 14:18:53 smithi097 podman[67700]: 2024-04-25 14:18:53.922630906 +0000 UTC m=+3.384613747 container cleanup e7a56b736f0f6a390c5347dd97f87272ae8bc80e0e6e9ce608f5f7f2bcd35411 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-k, org.label-schema.build-date=20240424, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-25T14:18:54.139 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 14:18:53 smithi097 bash[67700]: ceph-59334182-030c-11ef-bc93-c7b262605968-mds-k 2024-04-25T14:18:54.274 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@mds.k.service' 2024-04-25T14:18:54.304 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:18:54.305 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-25T14:18:54.305 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-25T14:18:54.305 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-59334182-030c-11ef-bc93-c7b262605968@mds.c 2024-04-25T14:18:54.606 INFO:journalctl@ceph.mds.c.smithi151.stdout:Apr 25 14:18:54 smithi151 systemd[1]: Stopping Ceph mds.c for 59334182-030c-11ef-bc93-c7b262605968... 2024-04-25T14:18:54.938 INFO:journalctl@ceph.mds.c.smithi151.stdout:Apr 25 14:18:54 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-c[63552]: 2024-04-25T14:18:54.605+0000 7f7242644700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:18:54.938 INFO:journalctl@ceph.mds.c.smithi151.stdout:Apr 25 14:18:54 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-c[63552]: 2024-04-25T14:18:54.605+0000 7f7242644700 -1 mds.c *** got signal Terminated *** 2024-04-25T14:18:57.815 INFO:journalctl@ceph.mds.c.smithi151.stdout:Apr 25 14:18:57 smithi151 podman[68006]: 2024-04-25 14:18:57.544392966 +0000 UTC m=+3.048595463 container died 6b065404a694b5de3b0f20e10a779cfc8368a480cc2e715bc6f842fa27d5ddf4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T14:18:58.188 INFO:journalctl@ceph.mds.c.smithi151.stdout:Apr 25 14:18:57 smithi151 podman[68006]: 2024-04-25 14:18:57.907783805 +0000 UTC m=+3.411986307 container cleanup 6b065404a694b5de3b0f20e10a779cfc8368a480cc2e715bc6f842fa27d5ddf4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-c, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424) 2024-04-25T14:18:58.188 INFO:journalctl@ceph.mds.c.smithi151.stdout:Apr 25 14:18:57 smithi151 bash[68006]: ceph-59334182-030c-11ef-bc93-c7b262605968-mds-c 2024-04-25T14:18:58.388 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@mds.c.service' 2024-04-25T14:18:58.423 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:18:58.423 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-25T14:18:58.423 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-25T14:18:58.423 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-59334182-030c-11ef-bc93-c7b262605968@mds.f 2024-04-25T14:18:58.736 INFO:journalctl@ceph.mds.f.smithi151.stdout:Apr 25 14:18:58 smithi151 systemd[1]: Stopping Ceph mds.f for 59334182-030c-11ef-bc93-c7b262605968... 2024-04-25T14:18:59.188 INFO:journalctl@ceph.mds.f.smithi151.stdout:Apr 25 14:18:58 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-f[63245]: 2024-04-25T14:18:58.735+0000 7f82b5e03700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:18:59.188 INFO:journalctl@ceph.mds.f.smithi151.stdout:Apr 25 14:18:58 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-f[63245]: 2024-04-25T14:18:58.735+0000 7f82b5e03700 -1 mds.f *** got signal Terminated *** 2024-04-25T14:19:01.795 INFO:journalctl@ceph.mds.f.smithi151.stdout:Apr 25 14:19:01 smithi151 podman[68140]: 2024-04-25 14:19:01.539149798 +0000 UTC m=+2.897864845 container died 629f47d1bed267bb8c43de08c81796abfd1570312c2e8172cbc0a6f1bc4cfea7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-f, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T14:19:02.122 INFO:journalctl@ceph.mds.f.smithi151.stdout:Apr 25 14:19:01 smithi151 podman[68140]: 2024-04-25 14:19:01.895873932 +0000 UTC m=+3.254588971 container cleanup 629f47d1bed267bb8c43de08c81796abfd1570312c2e8172cbc0a6f1bc4cfea7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-f, ceph=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-25T14:19:02.122 INFO:journalctl@ceph.mds.f.smithi151.stdout:Apr 25 14:19:01 smithi151 bash[68140]: ceph-59334182-030c-11ef-bc93-c7b262605968-mds-f 2024-04-25T14:19:02.262 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@mds.f.service' 2024-04-25T14:19:02.294 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:19:02.294 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-25T14:19:02.294 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-25T14:19:02.295 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-59334182-030c-11ef-bc93-c7b262605968@mds.i 2024-04-25T14:19:02.688 INFO:journalctl@ceph.mds.i.smithi151.stdout:Apr 25 14:19:02 smithi151 systemd[1]: Stopping Ceph mds.i for 59334182-030c-11ef-bc93-c7b262605968... 2024-04-25T14:19:02.688 INFO:journalctl@ceph.mds.i.smithi151.stdout:Apr 25 14:19:02 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-i[63862]: 2024-04-25T14:19:02.609+0000 7fbc0f7f9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:19:02.688 INFO:journalctl@ceph.mds.i.smithi151.stdout:Apr 25 14:19:02 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-i[63862]: 2024-04-25T14:19:02.609+0000 7fbc0f7f9700 -1 mds.i *** got signal Terminated *** 2024-04-25T14:19:05.792 INFO:journalctl@ceph.mds.i.smithi151.stdout:Apr 25 14:19:05 smithi151 podman[68264]: 2024-04-25 14:19:05.5393664 +0000 UTC m=+3.025231635 container died e8f5258ca3f52470872c1addca1198027f1c47ec8e543a26193f9ebc7352414e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-i, ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True) 2024-04-25T14:19:06.068 INFO:journalctl@ceph.mds.i.smithi151.stdout:Apr 25 14:19:05 smithi151 podman[68264]: 2024-04-25 14:19:05.884681482 +0000 UTC m=+3.370546717 container cleanup e8f5258ca3f52470872c1addca1198027f1c47ec8e543a26193f9ebc7352414e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-i, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:19:06.068 INFO:journalctl@ceph.mds.i.smithi151.stdout:Apr 25 14:19:05 smithi151 bash[68264]: ceph-59334182-030c-11ef-bc93-c7b262605968-mds-i 2024-04-25T14:19:06.300 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@mds.i.service' 2024-04-25T14:19:06.320 INFO:journalctl@ceph.mds.i.smithi151.stdout:Apr 25 14:19:06 smithi151 podman[68279]: 2024-04-25 14:19:06.069595873 +0000 UTC m=+0.521973746 container remove e8f5258ca3f52470872c1addca1198027f1c47ec8e543a26193f9ebc7352414e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-i, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True) 2024-04-25T14:19:06.320 INFO:journalctl@ceph.mds.i.smithi151.stdout:Apr 25 14:19:06 smithi151 systemd[1]: ceph-59334182-030c-11ef-bc93-c7b262605968@mds.i.service: Deactivated successfully. 2024-04-25T14:19:06.320 INFO:journalctl@ceph.mds.i.smithi151.stdout:Apr 25 14:19:06 smithi151 systemd[1]: Stopped Ceph mds.i for 59334182-030c-11ef-bc93-c7b262605968. 2024-04-25T14:19:06.321 INFO:journalctl@ceph.mds.i.smithi151.stdout:Apr 25 14:19:06 smithi151 systemd[1]: ceph-59334182-030c-11ef-bc93-c7b262605968@mds.i.service: Consumed 1.532s CPU time. 2024-04-25T14:19:06.331 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:19:06.332 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-25T14:19:06.332 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-25T14:19:06.332 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-59334182-030c-11ef-bc93-c7b262605968@mds.l 2024-04-25T14:19:06.646 INFO:journalctl@ceph.mds.l.smithi151.stdout:Apr 25 14:19:06 smithi151 systemd[1]: Stopping Ceph mds.l for 59334182-030c-11ef-bc93-c7b262605968... 2024-04-25T14:19:06.938 INFO:journalctl@ceph.mds.l.smithi151.stdout:Apr 25 14:19:06 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-l[62926]: 2024-04-25T14:19:06.646+0000 7f65a1bcc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:19:06.938 INFO:journalctl@ceph.mds.l.smithi151.stdout:Apr 25 14:19:06 smithi151 ceph-59334182-030c-11ef-bc93-c7b262605968-mds-l[62926]: 2024-04-25T14:19:06.646+0000 7f65a1bcc700 -1 mds.l *** got signal Terminated *** 2024-04-25T14:19:09.914 INFO:journalctl@ceph.mds.l.smithi151.stdout:Apr 25 14:19:09 smithi151 podman[68386]: 2024-04-25 14:19:09.538371162 +0000 UTC m=+2.989831663 container died 545a1bff6ffd796639bf164e7d7f792db3e24dd31c743ba8197b825dd27df79d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-l, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424) 2024-04-25T14:19:10.188 INFO:journalctl@ceph.mds.l.smithi151.stdout:Apr 25 14:19:09 smithi151 podman[68386]: 2024-04-25 14:19:09.981072866 +0000 UTC m=+3.432533365 container cleanup 545a1bff6ffd796639bf164e7d7f792db3e24dd31c743ba8197b825dd27df79d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-l, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-25T14:19:10.188 INFO:journalctl@ceph.mds.l.smithi151.stdout:Apr 25 14:19:09 smithi151 bash[68386]: ceph-59334182-030c-11ef-bc93-c7b262605968-mds-l 2024-04-25T14:19:10.188 INFO:journalctl@ceph.mds.l.smithi151.stdout:Apr 25 14:19:10 smithi151 podman[68400]: 2024-04-25 14:19:10.148851153 +0000 UTC m=+0.605344137 container remove 545a1bff6ffd796639bf164e7d7f792db3e24dd31c743ba8197b825dd27df79d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-59334182-030c-11ef-bc93-c7b262605968-mds-l, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-25T14:19:10.352 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-59334182-030c-11ef-bc93-c7b262605968@mds.l.service' 2024-04-25T14:19:10.383 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:19:10.383 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-25T14:19:10.383 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 59334182-030c-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T14:19:10.571 INFO:teuthology.orchestra.run.smithi096.stdout:Deleting cluster with fsid: 59334182-030c-11ef-bc93-c7b262605968 2024-04-25T14:19:12.936 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 59334182-030c-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T14:19:13.125 INFO:teuthology.orchestra.run.smithi097.stdout:Deleting cluster with fsid: 59334182-030c-11ef-bc93-c7b262605968 2024-04-25T14:19:15.522 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 59334182-030c-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T14:19:15.712 INFO:teuthology.orchestra.run.smithi151.stdout:Deleting cluster with fsid: 59334182-030c-11ef-bc93-c7b262605968 2024-04-25T14:19:18.017 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T14:19:18.048 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T14:19:18.080 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T14:19:18.105 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-25T14:19:18.107 DEBUG:teuthology.misc:Transferring archived files from smithi096:/var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673194/remote/smithi096/crash 2024-04-25T14:19:18.108 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/crash -- . 2024-04-25T14:19:18.147 INFO:teuthology.orchestra.run.smithi096.stderr:tar: /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T14:19:18.147 INFO:teuthology.orchestra.run.smithi096.stderr:tar: Error is not recoverable: exiting now 2024-04-25T14:19:18.149 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673194/remote/smithi097/crash 2024-04-25T14:19:18.150 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/crash -- . 2024-04-25T14:19:18.250 INFO:teuthology.orchestra.run.smithi097.stderr:tar: /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T14:19:18.250 INFO:teuthology.orchestra.run.smithi097.stderr:tar: Error is not recoverable: exiting now 2024-04-25T14:19:18.252 DEBUG:teuthology.misc:Transferring archived files from smithi151:/var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673194/remote/smithi151/crash 2024-04-25T14:19:18.253 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar c -f - -C /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/crash -- . 2024-04-25T14:19:18.289 INFO:teuthology.orchestra.run.smithi151.stderr:tar: /var/lib/ceph/59334182-030c-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T14:19:18.290 INFO:teuthology.orchestra.run.smithi151.stderr:tar: Error is not recoverable: exiting now 2024-04-25T14:19:18.291 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-25T14:19:18.291 DEBUG:teuthology.orchestra.run.smithi096:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-25T14:19:18.333 INFO:tasks.cephadm:Compressing logs... 2024-04-25T14:19:18.333 DEBUG:teuthology.orchestra.run.smithi096:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T14:19:18.375 DEBUG:teuthology.orchestra.run.smithi097:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T14:19:18.378 DEBUG:teuthology.orchestra.run.smithi151:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T14:19:18.398 INFO:teuthology.orchestra.run.smithi096.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph.log 2024-04-25T14:19:18.398 INFO:teuthology.orchestra.run.smithi096.stderr:: No such file or directory 2024-04-25T14:19:18.398 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T14:19:18.407 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-25T14:19:18.407 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-25T14:19:18.407 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-25T14:19:18.407 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-25T14:19:18.407 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mds.j.log: /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-25T14:19:18.407 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-25T14:19:18.407 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mgr.x.log: 86.5% -- replaced with /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph.log.gzgzip -5 --verbose -- /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-25T14:19:18.407 INFO:teuthology.orchestra.run.smithi096.stderr: 2024-04-25T14:19:18.407 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mon.a.log: 90.2% -- replaced with /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T14:19:18.407 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T14:19:18.407 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-osd.6.log: gzip 94.6% -- replaced with /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-25T14:19:18.408 INFO:teuthology.orchestra.run.smithi096.stderr: -5 --verbose -- /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-25T14:19:18.408 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T14:19:18.408 INFO:teuthology.orchestra.run.smithi096.stderr: 83.2% -- replaced with /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T14:19:18.408 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-25T14:19:18.414 INFO:teuthology.orchestra.run.smithi151.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph.log: No such file or directory 2024-04-25T14:19:18.414 INFO:teuthology.orchestra.run.smithi151.stderr: 2024-04-25T14:19:18.414 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-25T14:19:18.414 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-25T14:19:18.415 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T14:19:18.415 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-25T14:19:18.415 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-25T14:19:18.415 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-25T14:19:18.415 INFO:teuthology.orchestra.run.smithi151.stderr: 86.4% -- replaced with /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T14:19:18.415 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-25T14:19:18.415 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mds.c.log: /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-25T14:19:18.415 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T14:19:18.415 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mds.i.log: 90.4% -- replaced with /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T14:19:18.415 INFO:teuthology.orchestra.run.smithi151.stderr: 92.3% -- replaced with /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-25T14:19:18.418 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-25T14:19:18.418 INFO:teuthology.orchestra.run.smithi097.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph.log 2024-04-25T14:19:18.418 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-04-25T14:19:18.418 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T14:19:18.419 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-25T14:19:18.419 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-25T14:19:18.419 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-25T14:19:18.419 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-25T14:19:18.419 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-25T14:19:18.419 INFO:teuthology.orchestra.run.smithi097.stderr: 86.4% -- replaced with /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T14:19:18.419 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-25T14:19:18.419 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-25T14:19:18.419 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mds.h.log: 90.3% -- replaced with /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T14:19:18.419 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T14:19:18.419 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-25T14:19:18.419 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph.cephadm.log: 82.9% -- replaced with /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T14:19:18.419 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T14:19:18.420 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mds.e.log: 92.2% -- replaced with /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-25T14:19:18.422 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-25T14:19:18.422 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-25T14:19:18.422 INFO:teuthology.orchestra.run.smithi151.stderr: 82.9% -- replaced with /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T14:19:18.422 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T14:19:18.424 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-25T14:19:18.425 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mds.d.log: 91.1% -- replaced with /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T14:19:18.425 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T14:19:18.425 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-osd.9.log: 94.7% -- replaced with /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-25T14:19:18.430 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-25T14:19:18.434 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-volume.log: 94.0% -- replaced with /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-25T14:19:18.435 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-25T14:19:18.435 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/cephadm.log: 93.9% -- replaced with /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-25T14:19:18.436 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T14:19:18.437 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T14:19:18.441 INFO:teuthology.orchestra.run.smithi096.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T14:19:18.442 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T14:19:18.443 INFO:teuthology.orchestra.run.smithi096.stderr: 93.9% -- replaced with /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-25T14:19:18.444 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mds.l.log: 91.0% -- replaced with /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T14:19:18.447 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/cephadm.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T14:19:18.461 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T14:19:18.462 INFO:teuthology.orchestra.run.smithi151.stderr: 93.4% -- replaced with /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-25T14:19:18.469 INFO:teuthology.orchestra.run.smithi151.stderr: 93.5% -- replaced with /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-25T14:19:18.483 INFO:teuthology.orchestra.run.smithi097.stderr: 93.9% -- replaced with /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-25T14:19:18.493 INFO:teuthology.orchestra.run.smithi097.stderr: 93.6% -- replaced with /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-25T14:19:18.495 INFO:teuthology.orchestra.run.smithi151.stderr: 93.6% -- replaced with /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-25T14:19:18.603 INFO:teuthology.orchestra.run.smithi096.stderr: 89.9% -- replaced with /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-25T14:19:18.648 INFO:teuthology.orchestra.run.smithi151.stderr: 91.9% -- replaced with /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-25T14:19:18.706 INFO:teuthology.orchestra.run.smithi097.stderr: 92.0% -- replaced with /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-25T14:19:18.833 INFO:teuthology.orchestra.run.smithi097.stderr: 95.8% -- replaced with /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-25T14:19:19.035 INFO:teuthology.orchestra.run.smithi096.stderr: 91.6% -- replaced with /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-25T14:19:19.111 INFO:teuthology.orchestra.run.smithi097.stderr: 92.1% -- replaced with /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-25T14:19:19.157 INFO:teuthology.orchestra.run.smithi096.stderr: 90.6% -- replaced with /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-25T14:19:19.201 INFO:teuthology.orchestra.run.smithi096.stderr: 91.6% -- replaced with /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-25T14:19:19.533 INFO:teuthology.orchestra.run.smithi097.stderr: 91.7% -- replaced with /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-25T14:19:19.599 INFO:teuthology.orchestra.run.smithi151.stderr: 92.9% -- replaced with /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-25T14:19:19.664 INFO:teuthology.orchestra.run.smithi096.stderr: 92.8% -- replaced with /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-25T14:19:19.673 INFO:teuthology.orchestra.run.smithi151.stderr: 92.2% -- replaced with /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-25T14:19:19.721 INFO:teuthology.orchestra.run.smithi096.stderr: 92.6% -- replaced with /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-25T14:19:19.723 INFO:teuthology.orchestra.run.smithi096.stderr: 2024-04-25T14:19:19.723 INFO:teuthology.orchestra.run.smithi096.stderr:real 0m1.334s 2024-04-25T14:19:19.723 INFO:teuthology.orchestra.run.smithi096.stderr:user 0m4.890s 2024-04-25T14:19:19.723 INFO:teuthology.orchestra.run.smithi096.stderr:sys 0m0.185s 2024-04-25T14:19:19.888 INFO:teuthology.orchestra.run.smithi151.stderr: 92.0% -- replaced with /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-25T14:19:20.004 INFO:teuthology.orchestra.run.smithi097.stderr: 92.1% -- replaced with /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-25T14:19:20.199 INFO:teuthology.orchestra.run.smithi097.stderr: 92.5% -- replaced with /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-25T14:19:21.511 INFO:teuthology.orchestra.run.smithi151.stderr: 95.3% -- replaced with /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-25T14:19:21.513 INFO:teuthology.orchestra.run.smithi151.stderr: 2024-04-25T14:19:21.513 INFO:teuthology.orchestra.run.smithi151.stderr:real 0m3.121s 2024-04-25T14:19:21.513 INFO:teuthology.orchestra.run.smithi151.stderr:user 0m7.223s 2024-04-25T14:19:21.513 INFO:teuthology.orchestra.run.smithi151.stderr:sys 0m0.296s 2024-04-25T14:19:22.056 INFO:teuthology.orchestra.run.smithi097.stderr: 93.5% -- replaced with /var/log/ceph/59334182-030c-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-25T14:19:22.058 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-04-25T14:19:22.058 INFO:teuthology.orchestra.run.smithi097.stderr:real 0m3.666s 2024-04-25T14:19:22.058 INFO:teuthology.orchestra.run.smithi097.stderr:user 0m9.385s 2024-04-25T14:19:22.058 INFO:teuthology.orchestra.run.smithi097.stderr:sys 0m0.309s 2024-04-25T14:19:22.058 INFO:tasks.cephadm:Archiving logs... 2024-04-25T14:19:22.058 DEBUG:teuthology.misc:Transferring archived files from smithi096:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673194/remote/smithi096/log 2024-04-25T14:19:22.059 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T14:19:22.800 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673194/remote/smithi097/log 2024-04-25T14:19:22.801 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T14:19:24.082 DEBUG:teuthology.misc:Transferring archived files from smithi151:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673194/remote/smithi151/log 2024-04-25T14:19:24.083 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T14:19:25.127 INFO:tasks.cephadm:Removing cluster... 2024-04-25T14:19:25.127 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 59334182-030c-11ef-bc93-c7b262605968 --force 2024-04-25T14:19:25.318 INFO:teuthology.orchestra.run.smithi096.stdout:Deleting cluster with fsid: 59334182-030c-11ef-bc93-c7b262605968 2024-04-25T14:19:25.620 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 59334182-030c-11ef-bc93-c7b262605968 --force 2024-04-25T14:19:25.809 INFO:teuthology.orchestra.run.smithi097.stdout:Deleting cluster with fsid: 59334182-030c-11ef-bc93-c7b262605968 2024-04-25T14:19:26.071 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 59334182-030c-11ef-bc93-c7b262605968 --force 2024-04-25T14:19:26.253 INFO:teuthology.orchestra.run.smithi151.stdout:Deleting cluster with fsid: 59334182-030c-11ef-bc93-c7b262605968 2024-04-25T14:19:26.520 INFO:tasks.cephadm:Removing cephadm ... 2024-04-25T14:19:26.521 DEBUG:teuthology.orchestra.run.smithi096:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T14:19:26.538 DEBUG:teuthology.orchestra.run.smithi097:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T14:19:26.554 DEBUG:teuthology.orchestra.run.smithi151:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T14:19:26.569 INFO:tasks.cephadm:Teardown complete 2024-04-25T14:19:26.569 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T14:19:26.580 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T14:19:26.580 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T14:19:26.583 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T14:19:26.596 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T14:19:26.705 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T14:19:26.715 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T14:19:26.716 DEBUG:teuthology.orchestra.run.smithi096:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T14:19:26.718 DEBUG:teuthology.orchestra.run.smithi097:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T14:19:26.721 DEBUG:teuthology.orchestra.run.smithi151:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T14:19:26.732 INFO:teuthology.orchestra.run.smithi097.stderr:bash: line 1: ntpq: command not found 2024-04-25T14:19:26.733 INFO:teuthology.orchestra.run.smithi096.stderr:bash: line 1: ntpq: command not found 2024-04-25T14:19:26.734 INFO:teuthology.orchestra.run.smithi151.stderr:bash: line 1: ntpq: command not found 2024-04-25T14:19:26.736 INFO:teuthology.orchestra.run.smithi097.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T14:19:26.736 INFO:teuthology.orchestra.run.smithi097.stdout:=============================================================================== 2024-04-25T14:19:26.737 INFO:teuthology.orchestra.run.smithi097.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 57 +962us[ +974us] +/- 44ms 2024-04-25T14:19:26.737 INFO:teuthology.orchestra.run.smithi097.stdout:^* hv02.front.sepia.ceph.com 3 6 377 56 -1673us[-1661us] +/- 64ms 2024-04-25T14:19:26.737 INFO:teuthology.orchestra.run.smithi097.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 58 +461us[ +473us] +/- 89ms 2024-04-25T14:19:26.737 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:19:26.738 INFO:teuthology.orchestra.run.smithi096.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T14:19:26.738 INFO:teuthology.orchestra.run.smithi096.stdout:=============================================================================== 2024-04-25T14:19:26.738 INFO:teuthology.orchestra.run.smithi096.stdout:^* hv01.front.sepia.ceph.com 3 7 377 56 +1025us[+1028us] +/- 44ms 2024-04-25T14:19:26.738 INFO:teuthology.orchestra.run.smithi096.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 58 -1593us[-1590us] +/- 64ms 2024-04-25T14:19:26.739 INFO:teuthology.orchestra.run.smithi096.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 59 +523us[ +526us] +/- 89ms 2024-04-25T14:19:26.739 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:19:26.739 INFO:teuthology.orchestra.run.smithi151.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T14:19:26.740 INFO:teuthology.orchestra.run.smithi151.stdout:=============================================================================== 2024-04-25T14:19:26.740 INFO:teuthology.orchestra.run.smithi151.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 60 +975us[ +979us] +/- 44ms 2024-04-25T14:19:26.740 INFO:teuthology.orchestra.run.smithi151.stdout:^* hv02.front.sepia.ceph.com 3 6 377 56 -1674us[-1670us] +/- 64ms 2024-04-25T14:19:26.740 INFO:teuthology.orchestra.run.smithi151.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 58 +467us[ +470us] +/- 89ms 2024-04-25T14:19:26.740 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:19:26.740 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T14:19:26.751 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T14:19:26.751 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T14:19:26.798 DEBUG:teuthology.orchestra.run.smithi096:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T14:19:26.874 DEBUG:teuthology.orchestra.run.smithi097:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T14:19:26.937 DEBUG:teuthology.orchestra.run.smithi151:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T14:19:27.000 DEBUG:teuthology.orchestra.run.smithi096:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T14:19:27.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:19:27.030 DEBUG:teuthology.orchestra.run.smithi097:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T14:19:27.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:19:27.056 DEBUG:teuthology.orchestra.run.smithi151:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T14:19:27.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:19:27.083 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T14:19:27.093 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T14:19:27.141 INFO:teuthology.task.internal:Duration was 1392.174382 seconds 2024-04-25T14:19:27.142 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T14:19:27.189 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T14:19:27.189 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T14:19:27.192 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T14:19:27.194 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T14:19:27.228 INFO:teuthology.orchestra.run.smithi097.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T14:19:27.229 INFO:teuthology.orchestra.run.smithi096.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T14:19:27.230 INFO:teuthology.orchestra.run.smithi151.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T14:19:27.727 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T14:19:27.727 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi096.front.sepia.ceph.com 2024-04-25T14:19:27.727 DEBUG:teuthology.orchestra.run.smithi096:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T14:19:27.749 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi097.front.sepia.ceph.com 2024-04-25T14:19:27.750 DEBUG:teuthology.orchestra.run.smithi097:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T14:19:27.779 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi151.front.sepia.ceph.com 2024-04-25T14:19:27.779 DEBUG:teuthology.orchestra.run.smithi151:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T14:19:27.808 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T14:19:27.808 DEBUG:teuthology.orchestra.run.smithi096:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T14:19:27.811 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T14:19:27.821 DEBUG:teuthology.orchestra.run.smithi151:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T14:19:27.874 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T14:19:27.875 DEBUG:teuthology.orchestra.run.smithi096:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T14:19:27.878 DEBUG:teuthology.orchestra.run.smithi097:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T14:19:27.888 DEBUG:teuthology.orchestra.run.smithi151:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T14:19:28.266 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T14:19:28.278 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T14:19:28.279 DEBUG:teuthology.orchestra.run.smithi096:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T14:19:28.302 DEBUG:teuthology.orchestra.run.smithi097:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T14:19:28.326 DEBUG:teuthology.orchestra.run.smithi151:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T14:19:28.350 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T14:19:28.363 DEBUG:teuthology.orchestra.run.smithi096:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T14:19:28.366 DEBUG:teuthology.orchestra.run.smithi097:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T14:19:28.368 DEBUG:teuthology.orchestra.run.smithi151:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T14:19:28.385 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern = core 2024-04-25T14:19:28.388 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = core 2024-04-25T14:19:28.412 INFO:teuthology.orchestra.run.smithi151.stdout:kernel.core_pattern = core 2024-04-25T14:19:28.432 DEBUG:teuthology.orchestra.run.smithi096:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T14:19:28.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:19:28.458 DEBUG:teuthology.orchestra.run.smithi097:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T14:19:28.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:19:28.472 DEBUG:teuthology.orchestra.run.smithi151:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T14:19:28.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:19:28.486 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T14:19:28.500 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T14:19:28.501 DEBUG:teuthology.misc:Transferring archived files from smithi096:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673194/remote/smithi096 2024-04-25T14:19:28.501 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T14:19:28.541 DEBUG:teuthology.misc:Transferring archived files from smithi097:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673194/remote/smithi097 2024-04-25T14:19:28.541 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T14:19:28.576 DEBUG:teuthology.misc:Transferring archived files from smithi151:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673194/remote/smithi151 2024-04-25T14:19:28.576 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T14:19:28.611 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T14:19:28.611 DEBUG:teuthology.orchestra.run.smithi096:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T14:19:28.614 DEBUG:teuthology.orchestra.run.smithi097:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T14:19:28.616 DEBUG:teuthology.orchestra.run.smithi151:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T14:19:28.659 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T14:19:28.673 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T14:19:28.673 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T14:19:28.700 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T14:19:28.700 DEBUG:teuthology.orchestra.run.smithi096:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T14:19:28.703 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T14:19:28.705 DEBUG:teuthology.orchestra.run.smithi151:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T14:19:28.715 INFO:teuthology.orchestra.run.smithi096.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 14:19 /home/ubuntu/cephtest 2024-04-25T14:19:28.717 INFO:teuthology.orchestra.run.smithi097.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 14:19 /home/ubuntu/cephtest 2024-04-25T14:19:28.719 INFO:teuthology.orchestra.run.smithi151.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 14:19 /home/ubuntu/cephtest 2024-04-25T14:19:28.720 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T14:19:28.801 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T14:19:28.903 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/direct_io}} duration: 1392.1743817329407 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-04-25T14:19:28.903 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T14:19:28.987 INFO:teuthology.run:pass